The migration froze halfway. The database waited, hanging on a missing new column.
Adding a new column sounds simple. It rarely is in production. Schema changes can lock tables, block queries, or crash deployments if not done with care. The larger the dataset, the higher the stakes. The key is to add a new column without downtime, without blocking writes, and without corrupting data.
Start by defining the new column with a nullable default. This avoids backfilling every row during the schema change, which can lock the table. In PostgreSQL, use ALTER TABLE ... ADD COLUMN ... with NULL allowed, or in MySQL, avoid NOT NULL with default values during the first step. Once the column exists, backfill data in small batches, using an id-range or timestamp cursor to prevent heavy locks.
If the new column requires an index, create it in a separate step. Most modern databases support concurrent index creation (CREATE INDEX CONCURRENTLY in PostgreSQL, ALGORITHM=INPLACE in MySQL). This prevents full table reads and reduces blocking. Always monitor replication lag in any high-traffic system.