Back to search

Lead Data Engineer

Seeking a strong Databricks-focused Data Engineer with expertise in ETL design, data pipeline development, and analytics.

  • Bethlehem, New Jersey, United States
  • Full-time
  • Posted Nov 16, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareData & Analytics

Responsibilities

  • Design and develop scalable data pipelines using Databricks (PySpark, Delta Lake)
  • Build and optimize ETL/ELT workflows in cloud data platforms
  • Perform data analysis, transformation, and integration across systems
  • Collaborate with business and architecture teams to translate requirements into solutions
  • Lead ETL design, optimization, and migration (Informatica → Databricks)
  • Perform data quality checks, reconciliation, and performance tuning
  • Support deployment, monitoring, and production support activities
  • Support production issues, incident resolution, and performance tuning

Requirements

  • Experience in a SaaS

Nice to have: Informatica IDMC, Powercenter, Migration from legacy ETL tools to Databricks

Skills

DatabricksPySparkSQLETL/ELT designData warehousing conceptsData modelingAWSUnix shell scriptingDatabricksPySparkDelta LakeSQLInformatica IDMCPowercenterAWS

More jobs nearby

Full description

Seeking a strong Databricks-focused Data Engineer with expertise in ETL design, data pipeline development, and analytics, along with strong SQL skills. Informatica experience is preferred to support legacy integration and migration initiatives. Key Responsibilities • Design and develop scalable data pipelines using Databricks (PySpark, Delta Lake) • Build and optimize ETL/ELT workflows in cloud data platforms • Perform data analysis, transformation, and integration across systems • Collaborate with business and architecture teams to translate requirements into solutions • Lead ETL design, optimization, and migration (Informatica → Databricks) • Perform data quality checks, reconciliation, and performance tuning • Support deployment, monitoring, and production support activities • Support production issues, incident resolution, and performance tuning Required Skills 10+ years of experience in Data Engineering & ETL 8+ Years of experience in coordinating with onsite and offshore teams and business stakeholders on requirements and delivery Strong expertise in: Databricks (PySpark, Notebooks, Delta tables) SQL (advanced querying, optimization, analytics) Strong understanding of: ETL/ELT design and data warehousing concepts Data modeling and pipeline design Good exposure to cloud platforms (AWS) Unix shell scripting Experience working in hybrid/onshore-offshore delivery model Good-to-Have • Experience in: Informatica IDMC & Powercenter Migration from legacy ETL tools to Databricks Core Competencies • Strong analytical thinking and problem-solving ability • Ability to work with large datasets and complex transformations • Strong communication and stakeholder management skills • Experience working in Agile teams

Lead Data Engineer at TechDigital Corporation | Referso Jobs