- Washington, District of Columbia, United States
- Full-time · Senior
- 8+ years
Design, develop, and deploy scalable data infrastructure on Databricks, leading data engineering initiatives for the Department of State’s data lake.
Responsibilities
- Design and implement scalable data pipelines using Databricks and the Medallion Architecture
- Develop and maintain PySpark and SQL code for ETL/ELT processes
- Integrate data ingestion pipelines via RESTful APIs
- Automate deployments with Azure DevOps and Databricks Asset Bundles
- Ensure data quality governance and monitoring
Requirements
- Nice to have: Tableau
- Nice to have: PowerBI
- Nice to have: Jupyter Notebooks
- Nice to have: d3js
- Nice to have: Elasticsearch
