Senior PySpark Data Engineer (AWS)
We are seeking a highly skilled PySpark Data Engineer with strong experience in building scalable, high-performance data platforms and ETL pipelines. The ideal candidate should possess deep expertise in PySpark, SQL, Python, and distributed data processing, along with exposure to AWS-based data engineering ecosystems including EMR, Redshift, S3, Glue, and Apache Iceberg.
- Bangalore, India +1 more
- Full-time
- ₹70K–₹130K per month
- Posted Nov 11, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareData & Analytics
Locations (2)
- Bangalore, India
- Hyderabad, Telangana, India
Responsibilities
- Design, develop, and optimize large-scale batch and data processing pipelines using PySpark and EMR.
- Build and maintain scalable ETL/ELT frameworks for ingestion, transformation, and consumption layers.
- Develop data pipelines that process high-volume datasets with a focus on performance, reliability, and scalability.
- Implement data quality validations, reconciliation frameworks, and monitoring capabilities.
- Work with structured, semi-structured, and unstructured data sources.
- Develop and optimize data models for analytics and reporting use cases.
- Collaborate with cross-functional teams to understand business requirements and transform them into technical solutions.
- Troubleshoot production issues, perform root cause analysis, and implement preventive measures.
- Mentor junior engineers and drive engineering best practices across the team.
Requirements
- Experience in a Cloud modernization
- Experience in a Data platform transformation
- Experience in a Medallion Architecture
Nice to have: Amazon Redshift, AWS Glue, Amazon S3, AWS Athena, AWS Lambda, AWS Step Functions, Apache Iceberg, Delta Lake, Lakehouse Architecture, Metadata-driven data pipelines, Kafka, Spark Streaming, Kinesis
Skills
Full description
ClifyX, INC
Company details are for members
Sign up for a free job alert to see ClifyX, INC's profile, website and LinkedIn, and the same for every employer on Referso Jobs.