Data Pipeline Quality Engineer
Design data quality controls for ingestion, transformation, validation, lineage, freshness, observability, and recovery.
Data Pipeline Quality Engineer is a free AI skill for data workflows. Design data quality controls for ingestion, transformation, validation, lineage, freshness, observability, and recovery. It works with Claude, ChatGPT, Cursor and is ready to use out of the box.
About this skill
Data Pipeline Quality Engineer helps teams build reliable batch and streaming data workflows. It defines contracts, schemas, validation, deduplication, completeness, freshness, lineage, reconciliation, anomaly detection, quarantine, backfills, ownership, and incident handling.
What it does
The skill analyzes sources, transformations, destinations, business rules, volumes, timing, and failure impact; creates a data-quality framework; maps checks to pipeline stages; defines observability and recovery; and produces implementation and test requirements.
What is included
- Source and contract map
- Schema and validation rules
- Completeness and freshness checks
- Deduplication and reconciliation
- Lineage and ownership
- Anomaly and alerting design
- Quarantine and recovery flow
- Test and implementation plan
How to use it
1. Download the data-pipeline-quality-engineer-SKILL.md file 2. Upload it to your data or engineering workspace 3. Provide sources, transformations, destinations, schedules, and known failures 4. Add business-critical fields and freshness requirements 5. Use the framework to implement and monitor data quality
Examples
Design data quality controls for a pipeline that ingests CSV uploads, sensor time series, customer metadata, and analysis results into a warehouse.
A full data-quality design with contracts, schemas, timestamp checks, missing-value rules, deduplication, reconciliation, freshness SLAs, quarantine, lineage, alerts, backfills, and test cases.
FAQ
What is this skill for?
Can it work with messy CSV files?
Does it include data freshness?
Can it detect duplicates?
How does it handle bad data?
How is this different from a pipeline design?
Related Skills
Operational Data Reconciliation Analyst
Design reconciliation processes that compare systems, detect mismatches, classify exceptions, assign ownership, and prove resolution.
Data Analysis Workflow Planner
Design a reliable data-analysis workflow from raw files to validated insights, including cleaning, checks, methods, outputs, and reproducibility.
CSV Data Cleaning & Insights Reporter
Reviews a pasted CSV/table sample, flags data quality issues, proposes cleaning steps, and surfaces the top 3 insights worth investigating.
Related Prompts
Form-to-Database Validation Workflow
Design a safe form-ingestion workflow that validates, normalizes, deduplicates, and stores submitted data with review queues and privacy controls.
Form Completion and Validation Optimizer
Redesign a form around user intent, data necessity, validation timing, privacy, and recovery to improve accurate completion.