Back to search

Databricks Data Engineer

Experienced Databricks Data Engineer with strong expertise in Apache Spark, PySpark, Azure data engineering, and large-scale data pipeline development.

  • Atlanta, Georgia, United States
  • Full-time
  • Posted Nov 16, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareData & Analytics

Responsibilities

  • Design, develop, and support batch-based, distributed, and real-time data pipelines.
  • Build highly available, fault-tolerant, self-healing, and adaptive data processing solutions.
  • Develop scalable data engineering solutions using Apache Spark and PySpark.
  • Implement data ingestion patterns and pipelines within Azure environments.
  • Design and manage workflow orchestration using Azure Data Factory (ADF) or Apache Airflow.
  • Optimize queries processing billions of rows across distributed query engines.
  • Perform root cause analysis and implement permanent solutions for technical and business process issues.
  • Provide technical consultation and lead the implementation of complex data engineering projects.
  • Develop and maintain technical documentation for assigned systems and projects.
  • Ensure data accuracy, reliability, scalability, and performance across data platforms.

Requirements

  • Experience in a Big Data Systems

Nice to have: Jupyter Notebooks, Google Colab, Databricks

Skills

Apache SparkPySparkAzure Data EngineeringAzure Data FactoryApache AirflowSQLData ModelingDatabricksApache SparkPySparkAzure Data FactoryApache AirflowJupyter NotebooksGoogle Colab

More jobs nearby

Full description

Job Title: Databricks Data Engineer Location: USA Experience: 8+ Years (Preferred) Job Description We are looking for an experienced Databricks Data Engineer with strong expertise in Apache Spark, PySpark, Azure data engineering, and large-scale data pipeline development. The ideal candidate will have hands-on experience building and supporting batch and real-time data pipelines, working with distributed computing frameworks, and optimizing data processing across large-scale datasets. Top Required Skills Apache Spark / PySpark – Strong hands-on expertise using Jupyter Notebooks, Google Colab, or Databricks (preferred). Azure Data Engineering – Hands-on experience in data pipeline development and data ingestion patterns. Data Orchestration & SQL – Experience with Azure Data Factory (ADF) or Apache Airflow, along with strong SQL skills. Data Modeling – Experience with denormalized data modeling for large-scale big data systems. Key Responsibilities Design, develop, and support batch-based, distributed, and real-time data pipelines. Build highly available, fault-tolerant, self-healing, and adaptive data processing solutions. Develop scalable data engineering solutions using Apache Spark and PySpark. Implement data ingestion patterns and pipelines within Azure environments. Design and manage workflow orchestration using Azure Data Factory (ADF) or Apache Airflow. Optimize queries processing billions of rows across distributed query engines. Perform root cause analysis and implement permanent solutions for technical and business process issues. Provide technical consultation and lead the implementation of complex data engineering projects. Develop and maintain technical documentation for assigned systems and projects. Ensure data accuracy, reliability, scalability, and performance across data platforms. Required Qualifications Strong hands-on experience with Apache Spark and PySpark. Experience working with Jupyter Notebooks, Google Colab, or Databricks (Databricks preferred). Hands-on experience developing data pipelines and implementing data ingestion patterns in Azure. Experience with Azure Data Factory (ADF) or Apache Airflow. Strong SQL development and query optimization skills. Experience with denormalized data modeling for big data systems. Understanding of distributed computing, batch processing, and real-time data processing. Experience handling large-scale datasets and distributed query engines. Strong problem-solving, analytical, and communication skills.

Databricks Data Engineer at Apolis | Referso Jobs