Back to search

Databricks Data Engineer

  • Charlotte, North Carolina, United States
  • Posted Nov 5, 2026
INDUSTRYSoftware & InternetData & Analytics

Responsibilities

  • Design and build Extract, Transform, Load (ETL) and ELT data pipelines using Azure Data Factory (ADF) and Azure Databricks
  • Write optimized data processing code using Apache Spark, PySpark, Python, Scala, and advanced SQL
  • Implement and manage data lakes using Azure Data Lake Storage (ADLS Gen2) and Delta Lake / lakehouse patterns
  • Configure data governance, lineage, and access controls using Databricks Unity Catalog and Azure IAM
  • Monitor, troubleshoot, and tune Spark jobs and data pipelines for performance, reliability, and cost-efficiency
  • Support deployment processes utilizing Azure DevOps, Git version control, and infrastructure-as-code tools like Terraform

Requirements

  • Bachelor's degree in Computer Science, Engineering, or a related quantitative field
  • Experience in a Data Engineering
  • Experience in a Cloud-based Data Platform Development

Skills

Azure DatabricksApache SparkPySparkPythonScalaSQLAzure Data FactoryAzure Data Lake StorageDelta LakeAzure IAMDatabricks Unity CatalogTerraformGitCI/CDAzure DevOpsAzure DatabricksApache SparkPySparkPythonScalaSQLAzure Data FactoryAzure Data Lake StorageAzure IAMDatabricks Unity CatalogTerraformGitAzure DevOps

More jobs nearby

Full description

Databricks Engineer to design, build, and maintain scalable data pipelines, lakehouse architectures, and analytics solutions using Microsoft Azure and the Databricks platform. Key Responsibilities Pipeline Development: Design and build Extract, Transform, Load (ETL) and ELT data pipelines using Azure Data Factory (ADF) and Azure Databricks Big Data Processing: Write optimized data processing code using Apache Spark, PySpark, Python, Scala, and advanced SQL. Storage & Architecture: Implement and manage data lakes using Azure Data Lake Storage (ADLS Gen2) and Delta Lake / lakehouse patterns. Governance & Security: Configure data governance, lineage, and access controls using Databricks Unity Catalog and Azure IAM. Optimization & Operations: Monitor, troubleshoot, and tune Spark jobs and data pipelines for performance, reliability, and cost-efficiency. CI/CD & DevOps: Support deployment processes utilizing Azure DevOps, Git version control, and infrastructure-as-code tools like Terraform. Required Skills & Qualifications Experience: 3 to 7+ years of total experience in data engineering or cloud-based data platform development. Core Technologies: Hands-on proficiency in Azure Databricks, Apache Spark, PySpark, and Azure Data Factory (ADF). Languages: Advanced SQL and solid programming skills in Python or Scala. Data Concepts: Strong grasp of Delta Lake, relational/NoSQL data storage, data modeling, and data quality standards. Education: Bachelor's degree in Computer Science, Engineering, or a related quantitative field (often preferred)

Databricks Data Engineer at Expert In Recruitment Solutions | Referso Jobs