Manufacturing Data Engineer
Design, build, and maintain the data pipelines and models powering Manufacturing Intelligence data lakehouse.
- Foster City, California, United States
- Contract
- On-site
- $68–$78 per hour
- Posted Nov 12, 2026
INDUSTRYManufacturing & ProductionIndustrial Engineering
Responsibilities
- Build batch and streaming ingestion into Delta Lake using Auto Loader, Lakeflow Declarative Pipelines, Lakeflow Connect, or Structured Streaming, including CDC feeds from AWS DMS and SAP SLT
- Design and maintain bronze, silver, and gold (medallion) layers that follow MOPS naming conventions and data dictionary standards
- Build incremental dbt models on Databricks (dbt-databricks), using MERGE and upsert patterns, SCD Type 2, and handling for late or out-of-order records
- Model manufacturing data for analytics, such as a single source of truth per unit (SFC), work orders, nonconformances, station results, and first-pass yield, and feed the semantic layer used by analysts
- Process large and semi-structured data, such as JSON test payloads, using PySpark and Databricks SQL
- Tune pipeline performance and cost through liquid clustering or partitioning, OPTIMIZE, VACUUM, file sizing, and right-sized job compute
- Own data quality with dbt tests, Lakeflow expectations, and Lakehouse Monitoring, including freshness SLAs and alerting
- Orchestrate workloads with Databricks Workflows or Airflow
- Migrate existing Redshift models and pipelines to Databricks: translate SQL, run in parallel, reconcile results, and cut over
- Partner with analysts and manufacturing stakeholders on requirements, and document lineage, data contracts, and table definitions in Unity Catalog
- Build and maintain CI/CD for MOPS jobs, pipelines and dbt models with Databricks Asset Bundles and GitHub Actions, across dev, staging and prod
- Manage MOPS-owned catalogs, schemas, volumes and grants in Unity Catalog, following the central platform team's governance model
- Configure job compute and SQL warehouses within central cluster policies; manage MOPS service principals, secret scopes, and OAuth machine-to-machine auth for Airflow and dbt
- Configure and troubleshoot the AWS ingestion layer that feeds the lakehouse (DMS, Kinesis, S3 landing buckets, IAM roles), partnering with infrastructure owners on provisioning through Terraform
- Monitor job health and DBU spend using system tables and tagging, and set up alerting for failures and SLA misses
- Document pipelines, runbooks, and operational procedures, and take part in incident response and root cause analysis for data pipeline issues
Requirements
- Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field
- Experience in a Manufacturing
- Experience in a Industrial data
- Experience in a Quality
- Experience in a Yield
- Experience in a Traceability
- Experience in a MES
Nice to have: Databricks Certified Data Engineer Associate or Professional, AWS certification (Solutions Architect Associate), Manufacturing data background, Data governance and compliance
Skills
SQLPythonPySparkDatabricksDelta LakedbtDimensional ModelingCDCIncremental LoadingGitGitHub ActionsAWS S3AWS IAMAWS DMSTerraformSAP MESSAP HANARedshiftDatabricks WorkflowsAirflowUnity CatalogAuto LoaderLakeflow Declarative PipelinesLakeflow ConnectStructured StreamingAWS DMSSAP SLTdbt-databricksGitHub ActionsDatabricks Asset BundlesKinesis
Benefits
- Medical
- Dental
- Vision
- Commuter
- 401K with company matching
More jobs nearby
Full description
Ursus, Inc.
Company details are for members
Sign up for a free job alert to see Ursus, Inc.'s profile, website and LinkedIn, and the same for every employer on Referso Jobs.