We are looking for an experienced Data Engineer with strong healthcare clinical data knowledge and hands-on expertise in Databricks and Snowflake.
Responsibilities
Work directly with client stakeholders to understand clinical data requirements, clarify source-to-target mappings and translate requirements into scalable data engineering solutions.
Develop and maintain data pipelines using Databricks, Snowflake, Spark/PySpark and SQL.
Build and manage data layers within a Lakehouse/Medallion architecture, including Bronze, Silver and Gold datasets using Delta Lake and related data platforms.
Design and implement data ingestion, transformation and integration between Databricks, Snowflake and upstream/downstream healthcare systems.
Ingest, transform and standardize clinical data from EHR/EMR and other healthcare sources, including encounters, diagnoses, procedures, labs and medications.
Design, implement and document data architecture and enterprise data models for a specialty insurer in the London Market using Azure Databricks.
Compensation
A competitive salary with a generous bonus, private healthcare, critical illness life assurance at 4 x your annual salary, income protection insurance, and a rewarding pension.
Responsibilities
Design lakehouse architecture on Azure Databricks using Delta Lake and the Medallion pattern
Design and apply dimensional models such as star and snowflake schemas
Create and maintain conceptual, logical and physical data models
Optimize platform performance and cost
Manage model lifecycle and recommend reuse
Requirements
Bachelor’s degree or equivalent experience
Nice to have: Databricks certification
Nice to have: Gold-layer semantic models for Power BI
Credence has an immediate need for an AWS Databricks Engineer to join our growing AI and Automation practice. You will serve as a key technical contributor responsible for designing, building, and optimizing scalable cloud-based data solutions using AWS and Databricks.
Compensation
Salary Range: $155,000 - $185,000 annually. Actual compensation will be determined based on the selected candidate's experience, education, certifications, skills, and overall qualifications.
Responsibilities
Design, build, and optimize scalable data pipelines and processing workflows using Databricks, Apache Spark, PySpark, and Delta Lake.
Design and implement modern lakehouse solutions using Databricks and AWS services.
Build and maintain ETL/ELT pipelines integrating structured, semi-structured, and unstructured data across AWS services such as S3, Glue, RDS, Redshift, Lambda, and related cloud data platforms.
Develop efficient data transformation workflows using SQL, Python, and PySpark.
Optimize Spark workloads, queries, clusters, and storage configurations for performance, reliability, and cost efficiency.
Requirements
Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, Engineering, or a related technical field.
The Staff Data Engineer is a senior, hands-on technical contributor within an Enterprise Data domain, owning technical design and the most complex implementation work for assigned data products, capabilities, and integrations.
Compensation
The base pay for this position is $99,300.00 – $198,700.00. In specific locations, the pay range may vary from the range.
Responsibilities
Own technical design and hands-on delivery for the most complex or highest-risk work across assigned data products, including data models, pipeline architecture, integration patterns, and platform usage decisions.
Produce design documentation that allows others to understand, review, and build on technical decisions, and drive design review with domain and cross-domain peers.
Build and evolve assigned domain data products so they are documented, discoverable, governed, observable, and supported throughout their lifecycle.
Define and uphold data contracts for the products the domain publishes, including schemas, freshness and availability expectations.
Design data products for AI and machine learning consumption with reproducibility, lineage, timeliness, and defined data quality controls.
Requirements
Bachelor's degree in Data Science, Computer Science, Information Systems, Mathematics, or Engineering
A senior, hands-on technical contributor within an Enterprise Data domain, owning technical design and the most complex implementation work for assigned data products, capabilities, and integrations.
Compensation
The base pay for this position is $99,300.00 – $198,700.00. In specific locations, the pay range may vary from the range posted.
Responsibilities
Technical design and solutioning for the most complex or highest-risk work across assigned data products, capabilities, and initiatives, including data models, pipeline architecture, integration patterns, and platform usage decisions.
Produce design documentation that allows others to understand, review, and build on technical decisions.
Build and evolve assigned domain data products so they are documented, discoverable, governed, observable, and supported throughout their lifecycle.
Define and uphold data contracts for the products the domain publishes, including schemas, freshness and availability expectations, breaking-change policy, and producer and consumer responsibilities.
Design data products for AI and machine learning consumption with reproducibility, lineage, timeliness, and defined data quality controls.
Lead and scale a world‑class engineering organization of 100+ data engineers, ML engineers, and platform developers, including managing managers and Staff/Principal‑level engineers while driving hiring, retention, and talent development strategies.
Compensation
(This section is only applicable to US candidates) S&P Global states that the anticipated base salary range for this position is $185,000 to $265,000.
Responsibilities
Lead and scale a world‑class engineering organization of 100+ data engineers, ML engineers, and platform developers, including managing managers and Staff/Principal‑level engineers while driving hiring, retention, and talent development strategies
Provide strategic technical direction for enterprise‑scale lakehouse architecture using tools such as Databricks Data Intelligence Platform and AWS cloud services
Drive the organization’s transition to production‑grade autonomous AI systems leveraging Databricks AI stack
Establish rigorous engineering standards and operational excellence practices including SLOs, error budgets, incident management, and infrastructure‑as‑code
Oversee enterprise semantic modeling and RAG architecture strategies
Lead Databricks Data EngineerEXL · New York, US
Head of Engineering - Inbound DataS&P Global · New York, US