As a Data Engineer, you will play a crucial role in developing, maintaining, and optimizing our data platform. Your expertise will drive the evolution of our platform, enabling seamless end-to-end transformation from raw data to insightful data products shared across the organization.
Responsibilities
Develop, maintain, and optimize the data platform
Collaborate with data product teams to understand needs and challenges
Contribute to the architecture and design decisions of the platform aligning with data mesh principles
Participate in continuous learning and training on data engineering and AI automation
Work on pipeline optimization for cost, performance, and reliability
Requirements
Graduate/post graduate degree in Computer science or Software engineering
Nice to have: Data security tools/RBAC
Nice to have: IAM
Nice to have: Data governance tools like Collibra/ Purview
As a Data Engineer, you will play a crucial role in developing, maintaining, and optimizing our data platform. Your expertise will drive the evolution of our platform, enabling seamless end-to-end transformation from raw data to insightful data products shared across the organization.
Responsibilities
Develop, maintain, and optimize the data platform
Collaborate with data product teams to understand needs and challenges
Contribute to the architecture and design decisions of the platform aligning with data mesh principles
Participate in continuous learning and training on data engineering and AI automation
Work on pipeline optimization for cost, performance, and reliability
Requirements
Graduate/post graduate degree in Computer science or Software engineering
Entwicklung und Betrieb von End-to-end Machine Learning und Agentic AI Lösungen, Architekturentwurf und Weiterentwicklung in einer innovativen Cloud-Umgebung.
Compensation
Faire Vergütung & volle Flexibilität: Marktgerechtes Gehalt, flexible Arbeitszeiten mit Arbeitszeitkonto und Teilzeitmodelle.
Responsibilities
Entwicklung und Betrieb von End-to-end Machine Learning Lösungen vom Training bis zum produktiven Betrieb (MLOps)
Entwicklung und Betrieb von Agentic AI Lösungen
Monitoring und kontinuierliche Verbesserung bestehender System-Architekturen für ML- und Agentic AI Anwendungen
Enge Zusammenarbeit mit Data Scientists, Data Engineers und Software Engineers
Beratung bei Architekturentscheidungen unter Berücksichtigung verschiedener Technologien wie Databricks, Cloud Umgebungen, Kubernetes und Apache Kafka
Requirements
Studium in Informatik, Data Science oder vergleichbare Qualifikation
Definir y diseñar la arquitectura tecnológica de datos que soporte los procesos de big data y la explotación estratégica de la información en la organización asegurando la integración, coherencia y escalabilidad de los sitemas de gestión de datos permitiendo la interoperabilidad entre diferentes plataformas y aplicaciones.
Responsibilities
Establecer estándares, normas y marcos de arquitectura de datos que aseguren la calidad, consistencia y accesibilidad de la información.
Optimizar el rendimiento y la eficiencia de los sistemas de almacenamiento y procesamiento de datos, facilitando la toma de decisiones basada en datos.
Documentar y mantener actualizada la arquitectura de datos, asegurando la trazabilidad y el conocimiento corporativo sobre el manejo y flujo de la información.
Analizar y evaluar la arquitectura de datos actual para identificar áreas de mejora y optimización.
Diseñar y crear modelos de datos y estructuras de bases de datos que soporten las necesidades del negocio y garanticen la integridad, consistencia y escalabilidad.
Requirements
estudios de ingeniería informática
Nice to have: Prácticas de integración y despliegue continuo
As a Senior Engineer, you will play a key role in driving our streaming and lakehouse strategies while also building highly scalable services and tools to help our customers acquire large volumes of high quality data at scale and making customers’ interactions with the ingested data pleasant and productive.
Locations (3)
Atlanta, Georgia, United States
San Francisco, California, United States
New York City, New York, United States
Compensation
The typical base salary range for this position is $172,500 - $260,100 annually. In select cities within the San Francisco and New York City metropolitan area, the base salary range for this role is $207,800 - $285,800 annually.
Responsibilities
Design and develop highly scalable and resilient streaming and batch pipelines for ingesting and processing structured and unstructured datasets
Design and implement Slack’s Lakehouse solutions and enable production use cases for customers
Design and develop highly scalable and resilient streaming and batch pipelines for ingesting and processing structured and unstructured datasets, build and operate the platform that ingests data into the Lakehouse, and lead technical initiatives within the Data Engineering organization.
Compensation
The typical base salary range for this position is $172,500 - $260,100 annually. In select cities within the San Francisco and New York City metropolitan area, the base salary range for this role is $207,800 - $285,800 annually.
Responsibilities
Design and develop highly scalable and resilient streaming and batch pipelines for ingesting and processing structured and unstructured datasets
Design and implement Slack’s Lakehouse solutions and enable production use cases for customers
Improve the reliability and performance of the services/pipelines through AI assisted and/or Agentic solutions and tools
Automate and handle the lifecycle of datasets (schema evolution, metadata management, change and backfill management, deprecation and migration)
Collaborate with cross functional partners and lead technical initiatives independently end to end
We are seeking a highly skilled PySpark Data Engineer with strong experience in building scalable, high-performance data platforms and ETL pipelines. The ideal candidate should possess deep expertise in PySpark, SQL, Python, and distributed data processing, along with exposure to AWS-based data engineering ecosystems including EMR, Redshift, S3, Glue, and Apache Iceberg.
Locations (2)
Bangalore, India
Hyderabad, Telangana, India
Compensation
Rs70,000.00 - Rs130,000.00 Monthly NOOT
Responsibilities
Design, develop, and optimize large-scale batch and data processing pipelines using PySpark and EMR.
Build and maintain scalable ETL/ELT frameworks for ingestion, transformation, and consumption layers.
Develop data pipelines that process high-volume datasets with a focus on performance, reliability, and scalability.
Implement data quality validations, reconciliation frameworks, and monitoring capabilities.
Work with structured, semi-structured, and unstructured data sources.