Design, build, and maintain the data pipelines and models powering Manufacturing Intelligence data lakehouse.
Compensation
$68 - $78/hr.
Responsibilities
Build batch and streaming ingestion into Delta Lake using Auto Loader, Lakeflow Declarative Pipelines, Lakeflow Connect, or Structured Streaming, including CDC feeds from AWS DMS and SAP SLT
Design and maintain bronze, silver, and gold (medallion) layers that follow MOPS naming conventions and data dictionary standards
Build incremental dbt models on Databricks (dbt-databricks), using MERGE and upsert patterns, SCD Type 2, and handling for late or out-of-order records
Model manufacturing data for analytics, such as a single source of truth per unit (SFC), work orders, nonconformances, station results, and first-pass yield, and feed the semantic layer used by analysts
Process large and semi-structured data, such as JSON test payloads, using PySpark and Databricks SQL
Requirements
Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field
Nice to have: Databricks Certified Data Engineer Associate or Professional
Nice to have: AWS certification (Solutions Architect Associate)
Senior technical contributor within the Data Management and Analytics team responsible for the design, technical leadership, delivery, and operation of complex enterprise data solutions.
Provide technical ownership for complex data engineering initiatives, from detailed solution design through implementation, deployment, operational readiness, and production support.
Translate target-state architecture and approved reference patterns into detailed engineering designs, ensuring consistent implementation across projects, domains, and modernization efforts.
Lead the technical delivery of legacy modernization and migration initiatives, including assessment, dependency analysis, data reconciliation, cutover, performance validation, and decommissioning.
Design and deliver scalable enterprise data solutions across ingestion, transformation, orchestration, storage, modeling, and serving layers using SQL, Python, PySpark, Spark SQL, APIs, CDC, streaming, and file-based integration patterns.
Develop reusable engineering components and implementation patterns that improve consistency, maintainability, and operational support.
Requirements
Bachelor's degree in computer science, information systems, data engineering, or a related discipline
Build, enhance, and support scalable data solutions on Azure, including data pipelines, transformations, and trusted datasets for reporting and analytics.
Responsibilities
Develop and maintain data ingestion pipelines using Azure Data Factory
Integrate data from SAP, databases, APIs, flat files, and cloud applications
Implement incremental and CDC-based data loading patterns
Troubleshoot data integration and pipeline issues
Develop PySpark and SQL-based transformations in Azure Databricks
Requirements
Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field
Senior Data EngineerNewSkanska USA · Charlotte, US