The migration finished at 02:14 UTC. One table was heavier than expected. The fix was a single new column.
Adding a new column sounds simple. It can be complex under load. Schema changes touch data, queries, indexes, and code paths. Done wrong, they lock tables, stall transactions, and slow the system. Done right, they roll out live with zero downtime.
The steps are direct. First, plan the ALTER TABLE with the exact column definition: type, nullability, default values. Avoid defaults that trigger full table rewrites. For large datasets, use an online migration tool like pt-online-schema-change or gh-ost. These tools copy rows in batches and keep replicas in sync.
Second, deploy code that can handle both the old schema and the new column. This dual-read/write phase prevents errors when different nodes see different schemas. Monitor error rates and query plans during this phase.