The problem

Every NLP project I touched started the same way: copy the tokenising, cleaning and splitting helpers out of the last one, tweak two functions, forget which version was the good one.

Data Preprocessors was the decision to stop doing that. The library itself is unglamorous - reusable text preprocessing utilities. What made it worth building is everything around it.

The part that mattered

Publishing a package changes your relationship with your own code. Once someone else can pip install it, a breaking change is no longer a refactor - it is an event with consequences for people you have never met.

That forced three habits I now apply to internal services too:

Semantic versioning that CI enforces. The version is derived from the release tag, not edited by hand in a file someone forgets. A breaking change means a major bump, and the pipeline is what says so.

Changelogs generated from commits. Conventional commit messages compile into the changelog automatically. The release notes are a build artefact, which means they are never stale and never skipped at 11pm.

Reproducible builds. Poetry pins the resolution in a lockfile, so the wheel built in CI is the wheel built on my laptop.

The pipeline

Pushing a tag is the entire release ritual:

git tag v1.2.3 && git push --tags

From there, GitHub Actions runs the test matrix, derives the version from the tag, builds the wheel and sdist with Poetry, generates the changelog from commit history, publishes to PyPI and GitHub Packages, and pushes a Docker image so the toolchain is usable without a Python environment at all.

No step in that list is one I can forget, do out of order, or do differently at midnight than at noon. That is the whole point.

What it demonstrates

Small library, disproportionate lesson. Release engineering is the difference between code that exists and code that other people can rely on - and the same tag-driven, automated, reproducible approach is what I now build for multi-tenant SaaS deployments.