Data Quality Test Suite
Project phase: Data Engineering and Preparation
You build an executable set of checks that verifies whether incoming data meets the quality standards your project depends on: data types, permitted ranges, uniqueness, and acceptable proportions of missing values, failing loudly and informatively when an expectation is violated. You go beyond mechanical validity by also checking representativeness and bias against the population your model is meant to serve, and motivate each check by the quality dimension it protects.
Starting Points
Key Points
- The test suite is executable and runs automatically over a dataset.
- The suite produces a readable quality report that clearly shows passed and failed tests.
- The suite "fails loudly": it provides informative error messages when data doesn't meet the standards.
- You include checks for bias and representativeness to account for social impact.
- Every test is motivated by a specific quality dimension relevant to your project.
- You provide a clear description of how synthetic data (if used) is labeled and tested.
- The report is a standalone markdown document, made available in GitLab.