- Pune, Maharashtra, India
- Senior
- 6+ years
Senior Data Engineer responsible for designing, developing, and supporting scalable batch and near real‑time data pipelines, data products, and lakehouse‑ready data assets using Spark, Python, SQL, Hive, Databricks, and cloud‑native technologies.
Responsibilities
- Design, develop, and support scalable batch and near real‑time data pipelines, data products, and lakehouse‑ready data assets using Spark, Python, SQL, Hive, Databricks, and cloud‑native technologies.
- Build and optimize capabilities supporting the ONE Data Lake and Mastercard's unified data platform strategy.
- Develop ingestion, transformation, and orchestration solutions for batch and streaming/near real‑time data patterns using technologies such as Cribl, SSIS, Apache Ozone, Cloudera Data Services, Hive, Spark, Azure, Microsoft Fabric, Databricks, and related ecosystem tools.
- Enable trusted, governed, and accessible enterprise data through the implementation of data quality, lineage, and governance standards.
- Partner closely with Product, TPM, Data Governance, Architect, and Engineering teams to deliver strategic business outcomes.
Requirements
- Nice to have: Cribl or similar observability/log streaming platforms
- Nice to have: Microsoft Fabric
- Nice to have: OneLake
- Nice to have: cloud‑native data platforms
- Nice to have: enterprise data governance