- Singapore, Singapore, Singapore
Develop and deliver scalable transformation frameworks across Hadoop platforms by building robust data pipelines, ingestion frameworks, and automation capabilities.
Responsibilities
- Design, develop, and maintain scalable data pipelines within the Enterprise Data Lake platform using Cloudera CDP, Hadoop, Spark, Hive, Java, Python, and SQL.
- Develop high-performance batch and near real-time data ingestion frameworks for onboarding enterprise source systems into the Data Lake.
- Develop complex transformation logic using Spark, HiveQL, SQL, and Java-based processing frameworks.
- Design and implement data ingestion patterns supporting structured, semi-structured, and unstructured datasets.
- Build and maintain batch and near real-time data integration solutions.