Back to search

AZURE DATA ENGINEER

We are seeking an experienced Databricks Data Engineer to design, develop, and optimize enterprise-scale data platforms on Azure.

  • Toronto, Ontario, Canada
  • Full-time
  • Hybrid
  • Posted Nov 16, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareData & Analytics

Responsibilities

  • Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark
  • Build and optimize ETL/ELT workflows for processing large volumes of structured and unstructured data
  • Develop and manage Delta Lake-based Lakehouse solutions following Medallion Architecture (Bronze, Silver, Gold)
  • Integrate data from multiple sources including databases, APIs, cloud storage, and enterprise applications
  • Create and maintain data ingestion frameworks using Azure Data Factory (ADF)
  • Optimize Spark jobs, cluster configurations, and storage structures to improve performance and reduce costs
  • Implement data quality, validation, monitoring, logging, and alerting mechanisms
  • Collaborate with Data Architects and Business Analysts to gather and translate business requirements into technical solutions
  • Implement security, governance, and access controls using Unity Catalog and Azure security services
  • Support CI/CD deployment processes using Azure DevOps, Git, and automated release pipelines
  • Participate in code reviews, design discussions, and knowledge-sharing initiatives
  • Troubleshoot production issues and provide operational support for critical data pipelines

Requirements

  • Experience in a Lakehouse architectures

Nice to have: Snowflake, Microsoft Fabric, Kafka/Event Streaming, Power BI, Terraform/IaC, Azure Functions, REST API Integration, Data Governance and Metadata Management

Skills

Azure DatabricksPySparkDelta LakeAzure Data FactoryApache SparkSpark SQLDatabricks WorkflowsUnity CatalogPythonSQL / T-SQLData ModelingData Warehousing ConceptsAzure DevOpsGit/GitHubCI/CD PipelinesAzure DatabricksPySparkDelta LakeAzure Data FactoryADLS Gen2Azure Key VaultAzure Synapse AnalyticsPythonSQL / T-SQLAzure DevOpsGit/GitHub

Full description

ROLE: AZURE DATA ENGINEER TORONTO - HYBRID - FULLTIME Job Description – Databricks Data Engineer Experience: 6-10 Years Job Summary We are seeking an experienced Databricks Data Engineer to design, develop, and optimize enterprise-scale data platforms on Azure. The ideal candidate should have strong hands-on expertise in Azure Databricks, PySpark, Delta Lake, Azure Data Factory, and modern Lakehouse architectures. The role requires working closely with business stakeholders, data architects, and analytics teams to deliver scalable, high-performance data solutions. Key Responsibilities Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark. Build and optimize ETL/ELT workflows for processing large volumes of structured and unstructured data. Develop and manage Delta Lake-based Lakehouse solutions following Medallion Architecture (Bronze, Silver, Gold). Integrate data from multiple sources including databases, APIs, cloud storage, and enterprise applications. Create and maintain data ingestion frameworks using Azure Data Factory (ADF). Optimize Spark jobs, cluster configurations, and storage structures to improve performance and reduce costs. Implement data quality, validation, monitoring, logging, and alerting mechanisms. Collaborate with Data Architects and Business Analysts to gather and translate business requirements into technical solutions. Implement security, governance, and access controls using Unity Catalog and Azure security services. Support CI/CD deployment processes using Azure DevOps, Git, and automated release pipelines. Participate in code reviews, design discussions, and knowledge-sharing initiatives. Troubleshoot production issues and provide operational support for critical data pipelines. Mandatory Skills Databricks & Big Data Azure Databricks Apache Spark PySpark Spark SQL Delta Lake Databricks Workflows Unity Catalog Azure Data Services Azure Data Factory (ADF) Azure Data Lake Storage (ADLS Gen2) Azure Key Vault Azure Synapse Analytics Programming & Database Python SQL / T-SQL Data Modeling Data Warehousing Concepts DevOps & Version Control Azure DevOps Git/GitHub CI/CD Pipelines Preferred Skills Snowflake Microsoft Fabric Kafka/Event Streaming Power BI Terraform/IaC Azure Functions REST API Integration Data Governance and Metadata Management Desired Experience 6+ years of overall Data Engineering experience. Minimum 3+ years of hands-on experience with Azure Databricks and PySpark. Experience building enterprise-grade Lakehouse architectures. Strong understanding of performance tuning and optimization techniques for Spark workloads.

AZURE DATA ENGINEER at Epsilon Solutions LTD | Referso Jobs