Back to search

Data Engineer

The role focuses on designing and building data models, curated datasets and pipelines to support business intelligence, analytics and data science use cases.

  • Singapore, Singapore
  • Posted Jan 1, 2024
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareData & Analytics

Responsibilities

  • Engage directly with business stakeholders to understand requirements and translate them into data models, datasets and pipelines
  • Design and develop data models and transformation pipelines to integrate, clean, harmonise and curate data from multiple source systems
  • Build and maintain scalable, reliable and reusable data pipelines using Databricks, Spark and SQL
  • Design datasets and data structures optimized for downstream reporting, analytics and data science applications
  • Implement data quality checks, validation and reconciliation to ensure data accuracy and consistency
  • Monitor and troubleshoot pipeline performance and operational issues
  • Document data models, datasets, transformations and pipeline dependencies
  • Work with data engineers, analysts and data scientists to deliver data solutions aligned with business needs

Requirements

  • Degree in Computer Science, Information Technology, Data Engineering or a related discipline
  • Experience in a government

Nice to have: Databricks Workflows, Unity Catalog, Data governance, Access control, Data quality practices

Skills

SQLPythonData modellingDimensional modellingETL/ELT pipelinesAWS data servicesDatabricksSparkSQLDelta LakeGitCI/CD

Full description

We are seeking a Data Engineer to join our data team within a government agency. The role focuses on designing and building data models, curated datasets and pipelines to support business intelligence, analytics and data science use cases. You will work directly with business stakeholders to understand data requirements, translate business needs into technical solutions, and deliver reliable, well-structured and reusable data products. Key Responsibilities Engage directly with business stakeholders to understand requirements and translate them into data models, datasets and pipelines. Design and develop data models and transformation pipelines to integrate, clean, harmonise and curate data from multiple source systems. Build and maintain scalable, reliable and reusable data pipelines using Databricks, Spark and SQL. Design datasets and data structures optimized for downstream reporting, analytics and data science applications. Implement data quality checks, validation and reconciliation to ensure data accuracy and consistency. Monitor and troubleshoot pipeline performance and operational issues. Document data models, datasets, transformations and pipeline dependencies. Work with data engineers, analysts and data scientists to deliver data solutions aligned with business needs. Requirements Approximately 3–8 years of relevant experience in Data Engineering or a related role. Strong proficiency in SQL and experience with relational databases and data warehousing. Strong experience in data modelling, including dimensional modelling and designing analytical datasets. Proficiency in Python for data processing and pipeline development. Hands-on experience with Databricks, Apache Spark and Delta Lake. Degree in Computer Science, Information Technology, Data Engineering or a related discipline, or equivalent relevant professional experience. Experience designing and developing ETL/ELT pipelines and data transformation workflows. Experience working with AWS data services and cloud-based data platforms. Ability to engage business stakeholders, understand business processes and translate requirements into technical data solutions. Familiarity with Git, CI/CD and data pipeline orchestration. Preferred Qualifications Experience with Databricks Workflows and Unity Catalog. Experience designing data products for business intelligence and self-service analytics. Familiarity with data governance, access control and data quality practices. Experience working in a government or other highly regulated environment.

Data Engineer at User Experience Researchers Pte. Ltd. | Referso Jobs