The migration failed five minutes before the release window closed. A missing new column in production broke the deploy, and the clock did not care.
Adding a new column sounds simple. It isn’t. Done wrong, it can lock tables, stall queries, and burn uptime. A clean database schema change requires planning, zero-downtime techniques, and a rollback path.
First, define the new column with correct data type and constraints. Avoid adding default values that force a full table rewrite unless absolutely necessary. Use NULL by default, populate in backfill jobs, then apply NOT NULL constraints afterward.
For high-traffic systems, perform a phased deployment. Step one: introduce the new column with a migration that runs fast. Step two: deploy application code that writes to both old and new columns. Step three: backfill data in small batches. Only after verifying completeness should you switch reads to the new column and remove the old one if needed.