The migration froze halfway through. A new column had been added to the table, but the service was already live. Traffic kept flowing. Queries queued. Locks spread. The error logs filled with rows about null constraints and mismatched types.
Adding a new column is the smallest schema change with the highest chance of surprise. It looks simple: ALTER TABLE my_table ADD COLUMN new_column_name data_type; But the impact ripples through every part of the stack. The larger the dataset, the more important it is to plan for zero-downtime execution.
The first step is choosing the right data type and default value. If the column is non-nullable, backfilling must happen before enforcing constraints. Bulk updates can block reads and writes. Instead, batch updates in small transactions keep the service responsive.
Schema changes should be deployed with versioned migrations. First, add the column as nullable. Second, backfill with parallel workers, ensuring indexes are ready before application code starts writing. Finally, make the column required if needed. This split migration pattern avoids long table locks on large datasets.