In plain terms
This is a small library of text-cleaning tools that any developer can install in one command - and, more importantly, the automated process that publishes it.
The library itself is unglamorous. What makes it worth showing is the release pipeline: one command produces the tested build, the correct version number, the release notes and the publication to three places. Nothing depends on a person remembering a sequence of steps.
Why it matters commercially: manual release steps are where outages come from. Every step a human performs under time pressure is a step that eventually gets skipped, and the skipped step is discovered by customers. The habits proven here are the ones I now apply to deploying business software.
The problem
Every NLP project I touched started the same way: copy the tokenising, cleaning and splitting helpers out of the last one, tweak two functions, forget which version was the good one. Multiply that across a team and you get five slightly different versions of the same logic, each with its own bugs, none of them fixed once.
Data Preprocessors was the decision to stop doing that. Build it once, in the open, and depend on it everywhere.
The part that actually mattered
Publishing changes your relationship with your own code. Once strangers can install it, a breaking change is no longer a refactor - it is an event with consequences for people you have never met, who will find out at the worst possible moment.
That forced three habits I now apply to internal business systems too:
Version numbers that mean something, enforced by automation. The version comes from the release itself, not from a file someone edits by hand and forgets. A breaking change means a major version bump, and the pipeline is what says so. Users can tell at a glance whether an update is routine or risky - which is exactly what a customer needs to know before accepting an update.
Release notes generated from the work itself. Commit messages compile into the changelog automatically. Release notes become a build output rather than a chore, which means they are never stale and never skipped at 11pm on a Friday.
Reproducible builds. Dependencies are pinned exactly, so the version built in the pipeline is the version that was tested. It removes the entire class of “it worked before release” investigations.
The pipeline
Pushing a tag is the entire release ritual:
git tag v1.2.3 && git push --tagsFrom there, automation runs the tests, derives the version from the tag, builds the package, generates the changelog from the commit history, publishes to two package registries, and pushes a container image so the tooling is usable without setting up Python at all.
No step in that list is one I can forget, do out of order, or do differently at midnight than at noon. That is the entire point. Written up in full: A Release Should Be One Git Tag.
The result
- Shared logic lives in one place, so a fix benefits every project instead of one.
- Releasing costs one command, which means it happens often instead of being deferred.
- Users can judge an update’s risk from its version number alone.
- What was tested is what ships, removing a whole family of release-day surprises.
What it demonstrates
Small library, disproportionate lesson. Release engineering is the difference between code that exists and code that other people can rely on - and the same automated, reproducible, one-command approach is exactly what I now build for multi-customer SaaS deployments, where the cost of a skipped step is measured in downtime rather than embarrassment.