- Arlington, Virginia, United States
- 7+ years
Design, develop, and maintain scalable cloud-based data pipelines and engineering solutions for the Department of Veterans Affairs Summit Advanced Tooling, Lakehouse, and Artificial Intelligence Services program.
Responsibilities
- Design, develop, and maintain scalable data ingestion and processing pipelines using Azure Data Factory, Azure Databricks, Apache Kafka, and related cloud technologies.
- Engineer and optimize data lakehouse solutions, including Delta Lake architectures, semantic models, conformed dimensions, and partitioning strategies that support enterprise analytics and AI/ML workloads.
- Integrate data from enterprise systems, legacy platforms, and cloud-based sources using batch, streaming, and event-driven ingestion patterns.
- Implement automated data validation, reconciliation, and monitoring to ensure data accuracy, completeness, consistency, and freshness.
- Support migration of data pipelines and analytical workloads to Summit, validating parity with source systems and ensuring production readiness.
Requirements
- Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related technical discipline
- Nice to have: supporting VA or other federal agency data modernization initiatives
- Nice to have: Microsoft Azure Government cloud environments
- Nice to have: Azure Synapse Analytics
- Nice to have: dedicated SQL pools