- Remote, United States
- Remote
- Full-time · Senior
- $100K–$130K per year
- 5+ years
Architect, develop, and maintain ETL/ELT and C#/.NET pipeline services that move data reliably from source systems into Candid's warehouse and lakehouse.
Compensation
$100,000 - $130,000 (this range is for the NYC area and will be adjusted for other localities; additionally, factors like skills and experience will be considered).
Responsibilities
- Architect, develop, and maintain ETL/ELT and C#/.NET pipeline services (SSIS, Airflow, AWS Lambda, enhancer and message-processing services) that move data reliably from source systems into Candid's warehouse and lakehouse.
- Support Candid's evolving cloud data platform, including Iceberg ingestion and transformation pipelines, DynamoDB, and query performance tuning via Trino/Starburst.
- Monitor data quality end-to-end and investigate production pipeline issues at the root cause, documenting findings and driving fixes through to reprocessing.
- Scope and onboard new external and regulatory data sources end-to-end, and contribute to schema and architecture decisions, coordinating changes across ingestion and transformation layers.
- Maintain SOP and technical documentation, mentor and review the work of Associate-level engineers, and participate in the team's release and deployment process.
Requirements
- Nice to have: AWS
- Nice to have: open table formats