Design, build, and optimize data pipelines using Databricks, PySpark, Python, and SQL; develop scalable ETL/ELT solutions and Lakehouse architectures; implement solutions using Delta Lake, Unity Catalog, Databricks Workflows, and Structured Streaming; develop and support Azure-based data platforms leveraging ADF and ADLS Gen2; optimize Spark workloads for performance, scalability, and cost efficiency; implement CI/CD, DataOps, and engineering best practices; collaborate with business and technical stakeholders to deliver high-quality data solutions; support data governance, quality, and security initiatives; mentor junior engineers and contribute to capability development.
Responsibilities
Design, build, and optimize data pipelines using Databricks, PySpark, Python, and SQL.
Develop scalable ETL/ELT solutions and Lakehouse architectures.
Implement solutions using Delta Lake, Unity Catalog, Databricks Workflows, and Structured Streaming.
Develop and support Azure-based data platforms leveraging ADF and ADLS Gen2.
Optimize Spark workloads for performance, scalability, and cost efficiency.
Requirements
Nice to have: Databricks Certified Data Engineer Associate
Nice to have: Microsoft Azure Data Engineer Associate (DP-203)
Build the data foundation powering Coveo's next generation of product innovation.
Responsibilities
Work with product engineering to turn questions about how users interact with Coveo into well-defined data models.
Build and maintain pipelines that process hundreds of millions of events a day, with tests and monitoring to keep them reliable.
Design batch and streaming transformations, from incremental dbt models in Snowflake to PySpark Structured Streaming jobs in Databricks, and choose the right one for each use case based on latency, cost and data volume.
Take ownership of complex data engineering initiatives, proactively improving data reliability, performance and maintainability while sharing your expertise with teammates.
Build AI agents that query our data models directly, and improve their answers by refining the models and definitions they rely on.
Design, develop, and maintain scalable data pipelines, data products, and backend services supporting real-time decisioning, reporting, analytics, and AI use cases.
Responsibilities
Design, develop, and maintain scalable data pipelines, data products, and backend services supporting real-time decisioning, reporting, analytics, and AI use cases.
Design and implement data models for analytical and operational workloads, ensuring performance, scalability, and maintainability.
Produce high-quality, secure, testable, and maintainable code following engineering best practices.
Develop and maintain data governance, data quality, and monitoring solutions across the platform.
Collaborate with cloud, security, and architecture teams to ensure compliance with enterprise standards and cloud best practices.
Requirements
Bachelor’s or Master’s degree in Computer Science, Data Analytics, Information Systems, or a related technical field; or equivalent training and professional experience.
EY GDS is seeking a Senior Data Engineer with strong expertise in Databricks, Azure, Python, and SQL to deliver enterprise-scale data modernization and cloud transformation programs, building modern Lakehouse solutions.
Responsibilities
Design, build, and optimize data pipelines using Databricks, PySpark, Python, and SQL.
Develop scalable ETL/ELT solutions and Lakehouse architectures.
Implement solutions using Delta Lake, Unity Catalog, Databricks Workflows, and Structured Streaming.
Develop and support Azure-based data platforms leveraging ADF and ADLS Gen2.
Optimize Spark workloads for performance, scalability, and cost efficiency.
Requirements
Nice to have: Databricks Certified Data Engineer Associate or Professional
Nice to have: Microsoft Azure Data Engineer Associate (DP-203)
Senior Data Analytics DeveloperDevelopment Coveo EN · Montreal, Canada
AI engineer-Databricks Data PlatformEcolab Inc. · Bangalore, India
GDS Consulting_Senior Data EngineerEY · Taguig, Philippines
Lead Databricks Data EngineerEXL · New York, US