The migration script stopped cold. A missing column in production broke the deploy. You saw the error, scanned the schema, and knew the fix was simple: add a new column. But in production systems, nothing is ever just simple.
Adding a new column in a database table changes the shape of your data model. Done wrong, it can cause downtime, block writes, or lock tables during traffic spikes. With large datasets, a careless ALTER TABLE can lock for minutes or hours, burning through your error budgets.
The safest way to add a new column is to plan for zero downtime. In most relational databases like PostgreSQL and MySQL, you can add a nullable column without locking reads or writes. Avoid defaults that require backfilling all rows at once—use a two-step approach: first add the column as nullable; then backfill in batches; then, if needed, enforce null constraints.
For schema migrations in distributed environments, avoid assumptions about code and database being deployed simultaneously. Deploy code that can handle both old and new schemas. Then run the migration. This pattern—forward-compatible changes first, backward-incompatible changes last—prevents race conditions and schema drift.