The migration failed. The logs pointed to one line: ALTER TABLE users ADD COLUMN status VARCHAR(20);. A simple new column, but it halted the release and cost two hours of uptime.
Adding a new column in production is not a neutral operation. It can lock the table, block writes, and trigger full table rewrites depending on the database engine and settings. In high-traffic systems, these locks can cascade into timeouts and partial outages. Knowing how to introduce a column safely is critical.
First, evaluate the database. PostgreSQL handles ADD COLUMN operations with no lock when adding a nullable column without a default. MySQL, depending on the version and storage engine, may require a copy of the table. Use pt-online-schema-change or native online DDL methods for safer execution.
Second, control defaults. Adding a column with a non-null default can rewrite every row, creating long-running locks. If you need a default, add the column as nullable, backfill values with a batched process, then alter it to be non-null.