Open-source ML tooling · Case study
pandas-eda-check
An installable Python package for repeatable DataFrame profiling and reference-versus-current comparisons across schema, completeness, distributions, dates, and categories.
Open-source ML toolingProject overview
pandas-eda-check
An installable Python package for repeatable DataFrame profiling and reference-versus-current comparisons across schema, completeness, distributions, dates, and categories.
- Release
- Version 0.3.0
- Role
- Package author and maintainer
- Input
- One DataFrame or a reference/current pair
- Support
- Python 3.9+ and pandas 1.5+
- Status
- PyPI package with tests and CI
The problem
Small data changes can quietly become large model and reporting failures
Schema shifts, missing values, duplicate rows, altered distributions, and new categories can pass unnoticed between exploratory analysis and production use. Teams need the same checks to be repeatable in notebooks, tests, and automated workflows.
The system
Profile once, compare consistently, inspect the result
The package converts common exploratory and comparison checks into structured, reusable operations that work with familiar pandas DataFrames.
What I built
Data-quality checks packaged for reuse
Created a concise Python API for profiling a DataFrame and comparing current data with a reference dataset.
Covered schema, missingness, duplicates, numeric distributions, date ranges, and categorical values.
Returned structured results that can be inspected by people or incorporated into automated checks.
Added test coverage for empty frames, nullable dtypes, mixed objects, unhashable values, infinities, all-null columns, and comparison thresholds.
Added GitHub Actions testing across Python 3.9, 3.11, and 3.13 plus verified build and Trusted Publishing workflows.
Published documentation and examples alongside the source and PyPI distribution.
Quality and evaluation
The package itself is tested like production tooling
Automated tests exercise the public API, comparison behavior, and important edge cases. Packaging checks verify that the same project can be installed and used outside its source directory.
Thresholds and statistical outputs are evidence for review, not universal pass/fail rules. Users remain responsible for selecting checks appropriate to their dataset and application.
Technology
Evidence