Senior Data Engineer
Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory and Azure Databricks, focusing on large-scale distributed data processing.
- Coimbatore, Tamil Nadu, India
- Full-time
- Posted Nov 13, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareData & Analytics
Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory and Azure Databricks
- Develop data ingestion and transformation workflows using Python, SQL, and PySpark
- Implement batch processing, incremental loading, full loading, and Change Data Capture (CDC) mechanisms
- Integrate data from multiple sources, including relational databases, APIs, and cloud storage
- Develop and optimize data models, fact tables, dimension tables, and data warehouse structures
- Perform data cleansing, validation, deduplication, and aggregation
- Implement and manage workflows using Azure Data Factory and Apache Airflow
- Optimize SQL queries, Spark jobs, and data pipelines for better performance
- Manage version control and CI/CD pipelines using Git, GitHub, and Azure DevOps
- Troubleshoot production issues, perform root cause analysis, and provide production support
Requirements
- Experience in a Cloud-based data engineering
- Experience in a Data warehousing
Nice to have: Pandas, NumPy, OOP concepts, API integration
Skills
PythonSQLPySparkAzure Data FactoryAzure DatabricksApache SparkETL/ELT pipeline developmentChange Data Capture (CDC)Dimensional and relational data modelingGitGitHubAzure DevOpsAzure Data Lake Storage Gen2Azure Blob StorageAzure Synapse AnalyticsAzure Key VaultAzure MonitorApache AirflowMicrosoft SQL ServerPostgreSQLMySQLOracleCSVJSONXMLParquetDelta Lake
Full description
infynd
Company details are for members
Sign up for a free job alert to see infynd's profile, website and LinkedIn, and the same for every employer on Referso Jobs.