Senior Data Engineer (Databricks and Pyspark)-Mandarin Speaker
Senior Data Engineer with strong expertise in Databricks, PySpark, Delta Lake, and cloud-based data pipelines, responsible for implementing Lakehouse/Medallion architectures and integrating data from multiple internal and external systems.
- Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia +1 more
- Full-time
- Posted Nov 5, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareData & Analytics
Locations (2)
- Kuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia
- Singapore, Singapore
Responsibilities
- Design, build, and optimize data ingestion and transformation pipelines using Databricks, PySpark, and Python
- Implement Delta Lake and Medallion architecture for scalable enterprise data platforms
- Develop ingestion frameworks for data from SFTP, REST APIs, SharePoint/Graph API, AWS, and Azure sources
- Automate workflows using Databricks Workflows, ADF, Azure Functions, and CI/CD pipelines
- Optimize Spark jobs for performance, reliability, and cost efficiency
- Implement data validation, quality checks, and monitoring with automated alerts and retries
- Design secure and governed datasets using Unity Catalog and cloud security best practices
- Collaborate with analysts, business users, and cross-functional teams to deliver curated datasets for reporting and analytics
- Provide technical leadership and guidance to junior team members
Requirements
- Experience in a Oil & Gas
- Experience in a trading analytics
Nice to have: Oil & Gas, trading analytics, SPARTA, KPLER, IIR, OPEC, Power BI, Terraform, Scala, PostgreSQL, SharePoint development, .NET
Skills
Benefits
- one month Bonus
Full description
Company details are for members
Sign up for a free job alert to see Unison Group's profile, website and LinkedIn, and the same for every employer on Referso Jobs.