Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, and SQL.
We are proud to offer a competitive salary alongside a strong insurance package. We pride ourselves on the growth of our employees, offering extensive learning and development resources.
- Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, and SQL.
- Integrate data from multiple sources, including databases, Amazon S3, files, and REST APIs.
- Build data pipelines with Databricks Unity Catalog.
- Implement business logic, data transformations, and dimensional data models.
- Create, schedule, monitor, and optimize Databricks Jobs and Workflows.
- Nice to have: Auto Loader
- Nice to have: Spark Declarative pipelines
- Nice to have: Kafka
- Nice to have: Airflow
- Nice to have: dbt
Senior Data EngineerNewShyftLabs · Coimbatore, India