- Remote, United States
- Remote
- Full-time · Senior
- $140K–$187K per year
- 8+ years
The Data Platform Team at Scout is dedicated to unlocking the full potential of data by building a secure, scalable, and distributed platform that enables real-time insights, drives informed decision-making, and fosters innovation across the organization. This role is focused on building foundation for deploying AI-enabled use cases across company operations.
Compensation
Initial base salary range = $140,000.00 - $170,000.00 Initial California base salary range= $154,000 - $187,000
Responsibilities
- Contribute to the design, implementation, automation, maintenance, and operational support of enterprise-scale cloud data platforms leveraging Databricks, AWS cloud services, and modern Infrastructure as Code (IaC) practices.
- Lead the design, development, optimization, and operational management of enterprise-scale ETL/ELT pipelines within Databricks.
- Build and maintain scalable batch and streaming data pipelines using PySpark, Spark SQL, Delta Lake, and Databricks Workflows.
- Configure, optimize, and manage Databricks clusters for performance, scalability, reliability, and cost efficiency.
- Work closely with software development and systems teams to build Data Integration solutions.
Requirements
- Bachelor's degree in computer science, information technology, or related field or equivalent work experience
- Nice to have: Streaming and event-based architecture
- Nice to have: Data lakehouse solutions
- Nice to have: Structured databases
- Nice to have: Unstructured databases
