Joining and splitting text corrupted addresses.
One source put state and ZIP into the city field; another put a suite number there. The parser was corrected, regression tests added and affected records re-imported. Fixing code alone was not enough.
Automation
Public-data scraper
From public rosters to a checked dataset: source collection, address corrections, duplicate checks and human review.
Good data starts at the source.
Public lists can be downloaded. The harder problem is turning them into a dataset without repeated facilities, inconsistent address fields or suspicious records whose origin cannot be traced.
The Python workflow discovered and fetched public rosters, retained source snapshots and read CSV, XLSX and JSON formats. Different column names were mapped to a common structure, records validated and duplicates checked before projection into a usable table. A review queue and CSV/Markdown output generation were also implemented. Generating an output does not mean automatically sending emails.
What did the workflow do?
Discover and retain the source
The program discovers public sources and fetches data from portals, APIs and files. Retained snapshots preserve provenance so the result can later be checked against the source.
Read the format
Structured extractors separate fields. Different source column names map into a shared structure.
Validate and normalise
Address components are separated, values checked and duplicates identified. A matching column name does not guarantee matching content.
Review exceptions
Suspicious records enter review. AI can assist, but corrections and quality checks must precede promotion of the data.
Produce a usable output
The checked structure can feed a directory or an export of selected fields. A public source does not automatically make every subsequent use of personal data appropriate.
What failed, and what changed?
A repeated import must not silently add duplicate rows.
A replayed import introduced duplicates. The dataset was rebuilt. A procedural limitation remained: the duplicate guard depended on the projected table, which therefore had to be rebuilt before re-import.
Privacy rules also need exception review.
Street addresses for residential provider types were suppressed using type markers. A new or unexpected type description could evade that check; this is not a guarantee that every sensitive field is always detected.
My role and the AI contribution
I set the data’s purpose, target definitions and budget constraints. AI agents I directed helped research sources, write extractors, author tests and implement corrections. Review findings had to become fixes before data was promoted.
Where did the data go?
The database and files were local. Public sources were fetched externally, and AI-assisted review used an external model service. Public data and local storage do not automatically mean unrestricted use or entirely local processing.
This case is based on a review of project source and local checks on 11 September 2026, not an independent production audit. Examples are illustrative, not customer data.
Next step
Get in touch Want to build something similar?
Let us talk about how to build with AI while you keep control of the result.