Responsible for data warehouse modeling, offline and near‑real‑time data pipelines, data quality, and optimization of SQL and storage to support business reporting and analytics.
Responsibilities
Design and implement ODS, DWD, DWS, ADS layers
Develop offline and near‑real‑time data pipelines using Hive, Spark, Flink
Translate business requirements into reusable models and data services
Establish data quality checks and resolve failures, delays, and inconsistencies
Optimize SQL, jobs, and storage to reduce batch run time and compute cost
Requirements
Bachelor's degree in Computer Science or related field from a 985/211 university
Responsible for data warehouse layered modeling, developing offline and near-real time data pipelines, optimizing SQL queries, and ensuring production data availability.
Responsibilities
Responsible for data warehouse layered modeling, completing ODS, DWD, DWS, ADS design and metric definition.
Develop offline and near-realtime data pipelines using Hive, Spark, Flink to ensure timely data entry.
Connect with business requirements to implement reports and metrics into reusable models and data services.
Establish data quality checks, handle task failures, delays, and metric inconsistencies.
Optimize SQL, jobs, and storage to reduce batch processing time and computing costs.
Requirements
Bachelor's degree in Computer Science or related fields from 985 or 211 universities
Data Engineer & Insights Analyst for the Commonwealth of Pennsylvania’s Longitudinal Data System (PALDS).
Responsibilities
Create and maintain optimal data pipeline architecture
Design and implement cloud-based data solutions using AWS services such as S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda
Build and maintain data orchestration workflows using tools such as Apache Airflow, AWS Step Functions, or similar services
Assemble large, complex data sets that meet functional / non-functional business area requirements
Engineer analytical data environments that support advanced statistical analysis, predictive modeling, machine learning, and evidence-building activities
The role involves designing and building data pipelines, integrating and transforming data from multiple sources, and establishing controls for responsible data use to support the Commonwealth's longitudinal data architecture.
Responsibilities
Create and maintain optimal data pipeline architecture.
Design and implement cloud-based data solutions using AWS services such as S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda.
Build and maintain data orchestration workflows using tools such as Apache Airflow, AWS Step Functions, or similar services.
Assemble large, complex data sets that meet functional / non-functional business area requirements.
Engineer analytical data environments that support advanced statistical analysis, predictive modeling, machine learning, and evidence-building activities.
Data Engineer & Insights Analyst responsible for designing, building, and maintaining data pipelines and analytics solutions for the Commonwealth of Pennsylvania.
Responsibilities
Create and maintain optimal data pipeline architecture
Design and implement cloud-based data solutions using AWS services such as S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda
Build and maintain data orchestration workflows using tools such as Apache Airflow, AWS Step Functions, or similar services
Assemble large, complex data sets that meet functional / non-functional business area requirements
Engineer analytical data environments that support advanced statistical analysis, predictive modeling, machine learning, and evidence-building activities
Support the Pennsylvania Longitudinal Data System (PALDS) by transforming complex data into trusted, usable information to support program delivery, operational improvements, and executive decision-making across state government agencies.
Responsibilities
Create and maintain optimal data pipeline architecture.
Design and implement cloud-based data solutions using AWS services such as S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda.
Build and maintain data orchestration workflows using Apache Airflow, AWS Step Functions, or similar services.
Assemble large, complex data sets that meet functional and non-functional business requirements.
Engineer analytical data environments that support advanced statistical analysis, predictive modeling, machine learning, and evidence-building activities.
This role combines cloud data engineering with analytical judgment to support complex and confidential datasets, designing and optimizing data pipelines, integrating and transforming data across systems, implementing quality and governance controls, and supporting data conversion during system modernization.
Data EngineerTD SYNNEX Corporation · Chengdu, China