AWS Databricks Engineer
Credence has an immediate need for an AWS Databricks Engineer to join our growing AI and Automation practice. You will serve as a key technical contributor responsible for designing, building, and optimizing scalable cloud-based data solutions using AWS and Databricks.
- McLean, Virginia, United States
- $155K–$185K per year
- Posted Nov 5, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise Software
Responsibilities
- Design, build, and optimize scalable data pipelines and processing workflows using Databricks, Apache Spark, PySpark, and Delta Lake.
- Design and implement modern lakehouse solutions using Databricks and AWS services.
- Build and maintain ETL/ELT pipelines integrating structured, semi-structured, and unstructured data across AWS services such as S3, Glue, RDS, Redshift, Lambda, and related cloud data platforms.
- Develop efficient data transformation workflows using SQL, Python, and PySpark.
- Optimize Spark workloads, queries, clusters, and storage configurations for performance, reliability, and cost efficiency.
- Implement data quality, cataloging, lineage, governance, and access-control practices across Databricks and AWS environments.
- Automate data platform deployments and workflows using Infrastructure as Code, CI/CD pipelines, Git-based development practices, and AWS-native deployment tools.
- Work closely with data engineers, software engineers, data scientists, cloud engineers, and mission stakeholders to translate business and technical requirements into production-ready data solutions.
- Develop clean, maintainable, and well-documented code while following engineering, security, and federal development standards.
- Support code reviews, testing, troubleshooting, and technical design documentation.
Requirements
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, Engineering, or a related technical field.
- Experience in a federal government
Nice to have: Databricks Lakehouse Architecture, Databricks Unity Catalog, Delta Live Tables, Databricks Workflows, Pandas, Pydantic, Terraform, OpenTofu, AWS CDK, AWS Certified Data Engineer - Associate, AWS Certified Solutions Architect, Databricks Certified Data Engineer, Federal Cybersecurity, RMF, FedRAMP
Skills
DatabricksApache SparkPySparkDelta LakeAWS S3AWS GlueAWS RDSAWS RedshiftAWS LambdaSQLPythonGitCI/CDInfrastructure as CodeTerraformOpenTofuAWS CDKCloudFormationUnity CatalogDelta Live TablesDatabricks WorkflowsData Lakehouse ArchitectureMedallion ArchitectureETL/ELTData GovernanceData QualityData ModelingData TransformationData PipelineData LakeData LakehouseData EngineeringCloud EngineeringSecurityIdentity and Access ManagementEncryptionFederal CybersecurityRMFFedRAMPDatabricksApache SparkPySparkDelta LakeAWS S3AWS GlueAWS RDSAWS RedshiftAWS LambdaSQLPythonGitGitLabGitHub ActionsJenkinsAWS CodePipelineTerraformOpenTofuAWS CDKCloudFormation
Benefits
- Health Care Plan (Medical, Dental & Vision)
- Retirement Plan (401k, IRA)
- Life Insurance (Basic, Voluntary & AD&D)
- Paid Time Off (Vacation, Sick & Public Holidays)
- Family Leave (Maternity, Paternity)
- Short Term & Long Term Disability
- Training & Development
- Wellness Resources
More jobs nearby
Full description
Company details are for members
Sign up for a free job alert to see Credence Management Solutions's profile, website and LinkedIn, and the same for every employer on Referso Jobs.