The migration failed halfway through. The logs showed a syntax error. The culprit was a missing new column in the production schema.
Adding a new column sounds simple. It can also destroy uptime if done without care. Databases under load react badly to schema changes. Long locks, blocking writes, and inconsistent reads are common. A safe path requires planning and precise execution.
Start by defining the new column in a migration script. Use an ALTER TABLE statement that specifies the column name, data type, and default value if needed. For large tables, avoid adding non-nullable columns with defaults in one step; it can rewrite the entire table. Instead, add the column as nullable, backfill data in controlled batches, and then set constraints.
In PostgreSQL, use ALTER TABLE ... ADD COLUMN ... with caution on tables in active use. MySQL users can rely on online DDL in recent versions, but always test with production-sized data. Non-blocking migrations are essential.