We are seeking a highly motivated Platform Software Engineer to join our Enterprise Data Platform team. In this role you will work closely with engineering, operations, and architecture teams to understand the ingestion process, identify inefficiencies, and develop/enhance the tools.
Compensation
This position is hybrid with a requirement to be onsite four or more days per week. #LI-Hybrid #LI-MG1
Responsibilities
Design and Build Data Pipelines: Architect, develop, and maintain scalable data pipelines and microservices that support real-time and batch processing on GCP.
Service-Oriented Architecture (SOA) and Microservices: Design and implement SOA and microservices-based architectures to ensure modular, flexible, and maintainable data solutions.
Full-Stack Integration: Leverage your full-stack expertise to contribute to the seamless integration of front-end and back-end components, ensuring robust data access and UI-driven data exploration.
Data Ingestion and Integration: Lead the ingestion and integration of data from various sources into the data platform, ensuring data is standardized and optimized for analytics.
GCP Data Solutions: Utilize GCP services (BigQuery, Dataflow, Pub/Sub, Cloud Functions, etc.) to build and manage data platforms that meet business needs.
Design and build scalable data pipelines and microservices on GCP, integrate front‑end and back‑end components, and ensure data governance and security for the Enterprise Data Platform.
Compensation
This position is a salary grade 5 – salary grade 8 and ranges from $68,300-$192,900.
Responsibilities
Design and Build Data Pipelines
Service‑Oriented Architecture and Microservices
Full‑Stack Integration
Data Ingestion and Integration
GCP Data Solutions
Requirements
BS in Computer Science
Bachelor’s degree in computer science, Data Engineering or related technical field
Build the connectivity, profiling, and graph/search backbone that the ontology runs on — turning raw source systems into clean, lineage-tracked, query able knowledge.
Compensation
The annual salary for this position is between $80,000 - $140,000 depending on experience and other qualifications of the successful candidate.
Responsibilities
Own data source connectivity: build and maintain adapters to connect master data and work-ticket APIs via BigQuery, Cloud SQL, REST/Apigee, GCS, and Pub/Sub.
Own data profiling and inference: profile source data for quality, statistical patterns, dependencies, and lineage ahead of ontology mapping.
Stand up and operate the Knowledge Core — the graph and search backbone — using Spanner Graph (or an equivalent property-graph/RDF platform), Dataplex, BigQuery, and Vector Search.
Build batch and streaming ingestion pipelines that keep the graph and catalog current as source systems change.
Implement data quality checks, schema discovery, and the metadata catalog, surfacing issues to the Architect and Data Domain SME.
Define the data requirements and structure for the application. Model and design the application data structure, storage and integration using GenAI and Vertex AI.
Responsibilities
Own end-to-end agentic architecture for enterprise GenAI platforms using Google ADK and Vertex AI
Define multi-agent and agent-of-agents architectures leveraging A2A (Agent-to-Agent) communication patterns
Design and operationalize MCP-based orchestration layers for agent lifecycle management, coordination, and governance
Establish reference architectures, design patterns, and reusable frameworks for scalable agent platforms
Architect advanced RAG pipelines, including optimal chunking strategies, embedding selection, retrieval tuning, and re-ranking
Lead the architecture of our modern data stack on BigQuery and dbt, setting the patterns, standards and trade-offs that new and migrated workloads follow.
Responsibilities
Lead the architecture of our modern data stack on BigQuery and dbt
Design and evolve the architecture of our GCP data platform
Own the data ingestion layer
Define and maintain the cross-cutting foundations of the platform
Act as the technical reference for data engineers and analysts
Lead the technical direction of the search engine behind Progress’s Agentic RAG product, designing and implementing distributed Rust-based search and indexing systems.
Compensation
Generous remuneration package
Employee Stock Purchase Plan Enrollment
Vacation, Family, and Health
23 vacation days annually
Birthday day off
Community service time off
International Women's Day - March 8 is an official holiday for all employees
Life and Medical Insurance
Responsibilities
Own the architecture and roadmap of our distributed search and indexing engine, written in Rust.
Design and implement vector, full-text and graph indexes (HNSW, quantization, multi-vector, Tantivy) and the query and filtering layer on top of them.
Evolve how indexes are built, merged, sharded and replicated on object storage, PostgreSQL and NATS.
Lead large migrations and performance work, based on profiling and benchmarks, without customer downtime.
Contribute to the Python services, SDKs and PyO3 bindings built on the engine, keeping public APIs stable.
Seeking an experienced and adaptable Senior Data Engineer to lead the ownership, design, and delivery of data pipelines and warehousing infrastructure that feeds analytics and reporting across The Beatport Group.
Locations (2)
Pacific Northwest, Canada
Pacific Northwest, United States
Responsibilities
Ownership of production data pipelines across our comprehensive GCP based warehousing, from orchestration to reporting, focusing on reliability, performance, cost efficiency, and integrity.
Writing scalable dbt and BigQuery SQL across multiple business functions to build and maintain robust data models for all elements of The Beatport Group’s stack.
Planning and designing RDBMS and data warehouses, data models, repositories and pipelines with best practices for performance and scalability
Maintaining and developing integrations with partners of the group with relevant APIs and file based exchange services (SFTP, GCS etc.)
Create and maintain documentation for APIs, data pipelines, data models and data dictionaries
Platform Software EngineerFord · Dearborn, US
Software EngineerBLP Digital AG · Cologne
Sr. GCP Data Engineer (Remote)Cognizant · Remote
Senior Data EngineerBeatport · Remote, Canada+1 more