Data Platform Engineer
We are looking for DataOps engineer with hands-on experience to automate our Databricks Jobs/Workflows, DLT, and SQL tasks, implement and operate Unity Catalog at scale.
- Remote
- Full-time
- Remote
- Posted Nov 4, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareCloud & DevOpsAI & Machine Learning
Responsibilities
- Design, build, and maintain Databricks Workflows/Jobs (multi‑task graphs, retries, triggers, alerts) for batch, and SQL workloads.
- Implement CI/CD pipelines (GitHub Actions/Azure DevOps/GitLab): plan/apply for IaC, validate job JSON, environment promotion, approvals, and rollback.
- Build templates, parameterized job definitions (ingestion/batch/streaming/SQL) with standardized logging, notifications, and governance.
- Enforce run‑as service principals, job versioning, change auditability, and artifact immutability.
- Automate provisioning via Terraform (Databricks provider), Databricks SDK (Python/Go), and REST API 2.1 for Jobs, DLT, Repos, SQL Warehouses, and Unity Catalog.
- Idempotent creation of catalogs, schemas, default locations, owners, grants, volumes, storage credentials, and external locations using Terraform/SDK/REST.
- Standardize RBAC with templates (OWNERSHIP, USE CATALOG/SCHEMA, SELECT, MODIFY, CREATE, READ/WRITE FILES); integrate SCIM groups and service principals.
- Dynamic views for row/column-level security, data classification tags, audit logging, token policies, periodic permissions drift detection & remediation.
- Automate providers/recipients, share objects, and scheduled access reviews/expiry.
- Unity Catalog Migration (HMS → UC or multi-workspace to UC): discovery, wave planning, external/managed table registration, permission model migration, dual-run validation (row counts, schema parity, query replay), cutover & rollback.
- Define SLOs for pipeline success, latency, and data freshness; drive ≥99% T1 success and low MTTR.
- Implement data quality checks (DLT expectations, dbt-expectations/GX), idempotency, schema evolution strategy, checkpointing & replay.
- Build incident response playbooks; lead RCAs and preventative actions.
Requirements
- Experience in a data engineering
- Experience in a platform engineering
- Experience in a cloud infrastructure
Nice to have: Agentic AI, Genie spaces
Skills
Full description
Company details are for members
Sign up for a free job alert to see Bridgenext, Inc's profile, website and LinkedIn, and the same for every employer on Referso Jobs.