Distributed data processing jobs

Data Engineer

Weekday AI · Posted Nov 12, 2026

Open

Bengaluru, Karnataka, India
Full-time
₹1M–₹2M per year
7+ years

Apply with Referso

Data Engineer

Weekday AI · Posted Nov 12, 2026

Open

Bengaluru, Karnataka, India
Full-time
₹1M–₹2M per year
7+ years

Apply with Referso

Senior Azure Data Engineer

Cognizant · Posted Nov 13, 2026

Open

Senior
3+ years

Apply with RefersoAI Apply on careers page

Design, develop, and maintain scalable data pipelines and Big Data solutions on Microsoft Azure and related platforms.

Responsibilities

Design, develop, and maintain scalable data pipelines using PySpark, Spark SQL, Python, SQL, and Azure Synapse Analytics.
Build and optimize ETL/ELT workflows for large-volume structured, semi-structured, and unstructured data processing.
Develop and support Azure Synapse pipelines, notebooks, SQL scripts, stored procedures, views, and data transformation logic.
Work with Azure Data Lake Storage Gen2, Azure Data Factory, Azure Databricks, Azure SQL Database, and related Azure data services.
Develop and support Big Data solutions using technologies such as Apache Spark, Hadoop, HDFS, Hive, Sqoop, Kafka, HBase, and related distributed data processing components.

Skills

PySpark Spark SQL Python SQL Azure Synapse Analytics Azure Data Lake Storage Gen2 Azure Data Factory

Research Engineering Manager, STEM

Turing · Posted Nov 10, 2026

Open

United States
Manager
$260K–$350K per year
8+ years

Apply with RefersoAI Apply on careers page

Lead a team of research engineers to build synthetic data pipelines and benchmarks for frontier AI labs and enterprises.

Compensation

$260,000 to $350,000 + Equity

Responsibilities

Build and lead the research engineering team
Run hiring and performance management
Define operating rhythm and project selection
Coach engineers on experimental design and code quality
Partner with research scientists on hypothesis testing

Requirements

PhD or Master’s degree in AI, machine learning, computer science, statistics, or a closely related field
Nice to have: Frontier AI lab experience
Nice to have: public benchmark or dataset contribution
Nice to have: first-author papers at top conferences
Nice to have: RL environment design

Skills

Python PyTorch JAX ML engineering research engineering synthetic data pipelines

MS Fabric Specialist

Bosch Group · Posted Nov 10, 2026

Open

Bengaluru, Karnataka, India
Senior
10+ years

Apply with RefersoAI Apply on careers page

Lead enterprise-scale cloud data platform architecture using Microsoft Fabric, overseeing data engineering, integration, warehousing, real-time analytics, AI, security, governance, and operations.

Responsibilities

Define and govern enterprise-scale cloud data platform architecture
Translate business strategy into practical architecture
Design end-to-end cloud data and Microsoft Fabric architectures
Define platform roadmaps and modernization approaches
Provide technical leadership and architectural guidance

Requirements

Skills

Microsoft Fabric One Lake Lakehouse Data Warehouse Data Factory Real-Time Intelligence Power BI SQL T-SQL

Principal Data and Platform Engineer, DGX Cloud

NVIDIA · Posted Nov 9, 2026

Open

Santa Clara, California, United States
Principal
$248K–$391K per year
16+ years

Apply with RefersoAI Apply on careers page

Lead expert in data and platform engineering for NVIDIA’s DGX Cloud organization

Compensation

Your base salary will be determined based on your location, experience, and other objective job-related factors. The expected base salary range for this position is 248,000 USD – 391,000 USD. You will also be eligible for equity and benefits.

Responsibilities

Define the architecture and multiyear technical roadmap, setting system boundaries, shared interfaces, and engineering standards across ingestion, processing, storage, and data serving.
Lead delivery across platform engineers, data producers, and consumers, translating ambiguous requirements into actionable designs and milestones, resolving dependencies, and guiding implementation through production adoption.
Build and operate reusable batch and streaming ingestion capabilities for service provider and internal data, establishing supported patterns for APIs, change data capture, streams, database connectors, and files, including orchestration, retries, backfills, and recovery.
Establish shared identifiers, data contracts, consistent modeling, and automated quality and certification capabilities, partnering with source owners to define dataset meaning, keys, supported joins, historical coverage, compatibility, and ownership.
Guide cataloging, lineage, and governed access capabilities that help people and AI agents discover datasets, understand their definitions and relationships, and follow approved access procedures.

Senior Sales Engineer (m/f/d)

Ververica GmbH · Posted Oct 6, 2026

Open

Remote, United States
Remote
Full-time · Senior

Apply with RefersoAI Apply on careers page

We are seeking a Sales Engineer who combines strong technical acumen with business awareness, connecting customer problems to platform capabilities and translating technical value into business impact.

Responsibilities

Run technical discovery to identify customer pain points, current-state architecture, and desired outcomes.
Craft tailored streaming architectures and solutions that map Ververica's platform to the customer's data landscape.
Deliver product demos and lead proofs of concept that prove real, measurable value.
Provide the technical validation that supports the deal and earns customer trust.
Own the technical narrative in RFPs, security and architecture reviews, and executive briefings.

Requirements

Nice to have: Apache Flink
Nice to have: Apache Kafka
Nice to have: Docker
Nice to have: Kubernetes
Nice to have: open-source software

Skills

Sales Engineering Solutions Architecture streaming architectures distributed data processing data pipelines

Lead - Data Engineering

Syngene International Limited · Posted Nov 2, 2026

Open

Remote, India
Remote
3–5 years

Apply with RefersoAI Apply on careers page

Lead the data engineering team to support, maintain, and enhance PDS data products and ETL pipelines.

Responsibilities

Support, maintain, and enhance PDS data products, ETL pipelines built using DBT, AWS Glue, Databricks, and other cloud‑native technologies.
Analyze, troubleshoot, and resolve production incidents, ensuring high availability, reliability, and adherence to SLAs.
Develop pipeline enhancements, bug fixes, and operational automation to improve platform performance, scalability, and operational efficiency.
Ensure data quality, integrity, and operational stability through validation, monitoring, root cause analysis, and proactive issue resolution.
Collaborate with Analytics Engineering, Cloud Engineering, and Business teams to support feature releases, production deployments, and continuous improvements.

Requirements

Nice to have: data governance
Nice to have: access management
Nice to have: IAM
Nice to have: data security
Nice to have: compliance

Skills

AWS Glue Databricks DBT SQL

Data Architect - Enterprise Analytics

Lam Research · Posted Nov 6, 2026

Open

Fremont, California, United States
Hybrid
Senior
$137K–$287K per year
10+ years

Apply with RefersoAI Apply on careers page

Design and deliver scalable, governed, AI‑ready enterprise data solutions supporting analytics and digital transformation.

Compensation

CA San Francisco Bay Area Salary Range for this position: $137,000.00 - $287,000.00.

Responsibilities

Define and evolve enterprise data architecture, reference architectures, standards, patterns, and technology roadmaps.
Design conceptual, logical, physical, dimensional, canonical, and semantic data models.
Architect scalable data integration and ingestion patterns using batch, near‑real‑time, streaming, APIs, and event‑driven approaches.
Design modern lakehouse, data warehouse, medallion, semantic‑layer, and data‑product architectures using platforms such as Microsoft Fabric, Databricks, Azure, and related technologies.
Create AI‑ready data foundations ensuring enterprise data is well modeled, contextualized, governed, discoverable, secure, and optimized for downstream consumption.

Requirements

Bachelor’s degree in Computer Science, Information Technology, Data Engineering, Data Science, Engineering, or related discipline. Master’s degree preferred.
Nice to have: manufacturing
Nice to have: semiconductor
Nice to have: engineering
Nice to have: equipment

Skills

SQL

Data Architect - Enterprise Analytics

Lam Research · Posted Nov 6, 2026

Open

Fremont, California, United States
Hybrid
Full-time · Senior
$137K–$287K per year
10+ years

Apply with RefersoAI Apply on careers page

Design and deliver scalable, governed, and AI-ready enterprise data solutions supporting analytics, digital transformation, and AI use cases while providing technical leadership across architecture and modeling.

Compensation

CA San Francisco Bay Area Salary Range for this position: $137,000.00 - $287,000.00.

Responsibilities

Define and evolve enterprise data architecture, reference architectures, standards, patterns, and technology roadmaps.
Design conceptual, logical, physical, dimensional, canonical, and semantic data models representing complex business entities, relationships, and processes.
Architect scalable data integration and ingestion patterns using batch, near-real-time, streaming, APIs, and event-driven approaches.
Design modern lakehouse, data warehouse, medallion, semantic-layer, and data-product architectures using platforms such as Microsoft Fabric, Databricks, Azure, and related technologies.
Create AI-ready data foundations by ensuring enterprise data is well modeled, contextualized, governed, discoverable, secure, and optimized for downstream consumption.

Requirements

Bachelor’s degree in Computer Science
Information Technology
Data Engineering
Data Science
Engineering

Skills

SQL Python/PySpark

Lead Cloud Engineer

The Mitre Corporation · Posted Sep 28, 2026

Open

McLean, Virginia, United States
On-site
Full-time · Lead
$145K–$217K per year
8+ years

Apply with RefersoAI Apply on careers page

The Lead Cloud Engineer provides architectural and hands-on support to critical customer problems in cloud computing and DevOps as well as software development and architecture.

Compensation

$144,800 - $181,000 - $217,200 Annual

Responsibilities

Lead the identification, analysis, and resolution of infrastructure vulnerabilities and application deployment issues
Review existing systems and provide actionable recommendations to sponsors and internal teams for improvement
Collaborate with development teams to evaluate and identify optimal cloud solutions
Lead implementation of cloud-based solutions and DevOps practices across projects
Provide technical guidance and mentorship to team members and contribute to project planning and execution

Requirements

Bachelor’s degree
Master’s degree
PhD
Computer Science
Computer Engineering

Skills

AWS Azure Google Cloud DevOps GitLab

Distributed data processing jobs

Compensation

Requirements

Skills

Responsibilities

Skills

Compensation

Responsibilities

Requirements

Skills

Responsibilities

Requirements

Skills

Compensation

Responsibilities

Responsibilities

Requirements

Skills

Responsibilities

Requirements

Skills

Compensation

Responsibilities

Requirements

Skills

Compensation

Responsibilities

Requirements

Skills

Compensation

Responsibilities

Requirements

Skills

Requirements

Skills

Benefits

Benefits