- Hybrid
Architect role in a global organization focused on designing and optimizing scalable data and analytics solutions using Azure DevOps, Python, Databricks, SQL, Databricks Workflows and PySpark.
Responsibilities
- Design robust end to end data architectures that leverage Databricks SQL Databricks Workflows and PySpark to deliver secure and scalable analytics platforms
- Develop reusable Python and PySpark components that standardize data ingestion transformation and validation patterns
- Implement Azure DevOps based continuous integration and continuous delivery pipelines
- Collaborate with data engineers, data analysts and application teams to translate complex functional requirements into pragmatic technical designs
- Optimize Databricks SQL queries and cluster configurations to improve performance and manage costs
Requirements
- Nice to have: integrating Databricks with broader Azure data services
- Nice to have: data lake storage
- Nice to have: event based ingestion
- Nice to have: downstream visualization platforms
- Nice to have: hybrid work models