Data Engineer Lead
Lead Databricks Engineer to drive the design, development, and optimization of data pipelines and analytics solutions on the Databricks Lakehouse platform.
- Remote, Brazil +5 more
- Full-time
- Remote
- Posted Jan 1, 2024
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareData & Analytics
Locations (6)
- Brazil
- Argentina
- Colombia
- Peru
- Costa Rica
- Mexico
Responsibilities
- Lead end-to-end development of data pipelines, ETL/ELT processes, and batch/streaming solutions using Databricks and Apache Spark
- Design and implement Lakehouse architectures that align with business and technical requirements
- Collaborate with data scientists, analysts, and engineers to deliver high-performance data products and ML features
- Define and enforce coding standards, best practices, and performance tuning strategies across Databricks notebooks and jobs
- Optimize data models in Delta Lake and implement data governance standards using Unity Catalog
- Manage integration of data sources across cloud platforms (e.g., AWS, Azure, GCP) using native and third-party connectors
- Contribute to and lead technical reviews, architecture sessions, and mentoring of less experienced engineers
- Automate infrastructure deployment with tools like Terraform, Databricks CLI, or others
- Ensure data platform solutions are secure, compliant, and scalable across global business units
Requirements
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field
- Experience in a Big Data
Nice to have: Databricks (Pro, Architect) certification, Machine learning pipelines, MLOps
Skills
Full description
CustomertimesCompany details are for members
Sign up for a free job alert to see Customertimes's profile, website and LinkedIn, and the same for every employer on Referso Jobs.