AI Data Engineer 3
- Redmond, Washington, United States
- $85–$90 per hour
- Posted Nov 9, 2026
INDUSTRYSoftware & InternetAI & Machine Learning
Responsibilities
- Develop, maintain, and optimize Python pipelines for data ingestion, OCR processing, text extraction, cleaning, normalization, and deduplication.
- Run and monitor large-scale batch processing jobs, investigate failures, and implement reliable retry and recovery mechanisms.
- Build automated quality checks to identify missing or corrupted files, incomplete OCR outputs, duplicate content, formatting issues, and metadata inconsistencies.
- Prepare training-ready datasets through filtering, text segmentation, tokenization, and conversion into agreed data formats.
- Manage bulk uploads, downloads, and synchronization between local processing environments and Azure storage; verify transfer completeness and data integrity.
Requirements
Nice to have: OCR pipelines, Azure Databricks, Azure Machine Learning, Azure Data Factory, Hugging Face Datasets, tokenization, near-duplicate detection, training/evaluation data separation
Skills
data engineeringdata pipeline developmentdata sciencePythonprocessing large structured and unstructured datasetsAzure Blob StorageAzure Data Lake StorageOCR pipelinesAzure DatabricksAzure Machine LearningAzure Data FactoryHugging Face Datasetstokenizationnear-duplicate detectiontraining/evaluation data separationPythonAzure Blob StorageAzure Data Lake StorageAzure DatabricksAzure Machine LearningAzure Data FactoryHugging Face Datasets
Benefits
- End-to-end AI data management and dataset curation.
- Collaborative work environment.
- Opportunities for professional growth and development.
More jobs nearby
Full description
Artech LLC
Company details are for members
Sign up for a free job alert to see Artech LLC's profile, website and LinkedIn, and the same for every employer on Referso Jobs.