Senior Data Engineer (Databricks)
Seeking a Senior Data Engineer with deep expertise in Databricks to design, build, and evolve scalable, high-performance data architectures, leading the development of reliable data pipelines and complex processing solutions across batch and streaming environments.
- Remote, Brazil
- Full-time
- Remote
- Posted Nov 9, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareData & Analytics
Responsibilities
- Define and evolve data architectures using Databricks, ensuring pipeline optimization, processing efficiency, and effective resolution of complex technical challenges.
- Build, maintain, and optimize scalable, reliable data pipelines for data ingestion, transformation, and delivery.
- Design and implement Lakehouse architectures and Medallion structures (Bronze, Silver, and Gold).
- Develop batch and streaming data processing solutions, including Change Data Capture (CDC) and Structured Streaming.
- Build and orchestrate data ingestion and integration processes using Azure Data Factory (ADF), Auto Loader, and Lakeflow.
- Work with Apache Spark, Airflow, Kafka, dbt, SQL and NoSQL databases, and cloud data platforms.
- Implement data governance, security, lineage, and access control practices using Unity Catalog.
- Develop REST APIs, integrations, microservices, and automation solutions using Python.
- Establish CI/CD processes, version control, and automated deployments using Git and Databricks Workflows.
- Optimize Spark performance, compute resource allocation, and processing costs.
- Monitor and maintain production data pipelines, ensuring observability, reliability, and operational continuity.
- Act as a technical reference for the team, mentoring professionals and promoting engineering standards and best practices.
Requirements
- Databricks Certified Data Engineer Associate
- Databricks Certified Data Engineer Professional
- Experience in a Lakehouse
- Experience in a Medallion principles
Nice to have: Apache Spark, Airflow, Kafka, dbt, SQL, NoSQL databases, Photon, partitioning, shuffle optimization, execution plan analysis
Skills
DatabricksPySparkSpark SQLDelta LakeDelta Live Tables (DLT)Lakeflow Spark Declarative PipelinesChange Data Capture (CDC)Structured StreamingUnity CatalogAzure Data Factory (ADF)PythonREST APIsGitDatabricks WorkflowsDatabricksPySparkDelta LakeAzure Data FactoryAuto LoaderLakeflowApache SparkAirflowKafkadbtSQLNoSQLUnity CatalogPythonREST APIsGitDatabricks Workflows
Benefits
- Meal and/or food allowance
- Health insurance
- Dental insurance
- Pet health insurance
- Home-office allowance for remote positions
- Welcome kit during onboarding
- Life insurance
- Zenklub access
- TotalPass membership
- Certification and continuous learning program
- Partnerships and discounts supporting professional development
- Access to UOL content
- Maternity kit
- Extended maternity and paternity leave
- Employee referral program
Full description
JobgetherCompany details are for members
Sign up for a free job alert to see Jobgether's profile, website and LinkedIn, and the same for every employer on Referso Jobs.