The migration was green, but the query failed. The reason sat buried in the schema: the new column was missing.
Adding a new column sounds simple. In production, it can break deployments, block writes, and lock tables. Whether you use PostgreSQL, MySQL, or a distributed database, the strategy for adding columns must balance speed, safety, and minimal downtime.
First, define the column’s data type and constraints. Never default to text when a numeric or boolean makes the schema self-documenting. Always be explicit about nullability. Adding a NOT NULL column without a default will fail on existing rows. Adding it with a default can trigger a table rewrite and lock the table.
For high-traffic databases, add the column as nullable first. Backfill in batches to avoid transaction bloat and I/O spikes. Then alter the column to set NOT NULL once the backfill completes. This two-step process minimizes locks and keeps the system responsive.