- Senior
- 3+ years
Design, develop, and maintain scalable data pipelines and Big Data solutions on Microsoft Azure and related platforms.
Responsibilities
- Design, develop, and maintain scalable data pipelines using PySpark, Spark SQL, Python, SQL, and Azure Synapse Analytics.
- Build and optimize ETL/ELT workflows for large-volume structured, semi-structured, and unstructured data processing.
- Develop and support Azure Synapse pipelines, notebooks, SQL scripts, stored procedures, views, and data transformation logic.
- Work with Azure Data Lake Storage Gen2, Azure Data Factory, Azure Databricks, Azure SQL Database, and related Azure data services.
- Develop and support Big Data solutions using technologies such as Apache Spark, Hadoop, HDFS, Hive, Sqoop, Kafka, HBase, and related distributed data processing components.