The build froze one minute after deployment. The logs showed nothing. The only change was a new column.
Adding a new column should be simple. In real systems, it can become a fault line. Schema migrations touch production data at scale. On small tables, ALTER TABLE ... ADD COLUMN is instant. On large datasets, it can lock writes, block queries, and cascade failures.
The safest approach is explicit. Decide if the new column allows NULL. If not, set a default value and backfill data in a controlled batch. Use asynchronous jobs to populate rows without blocking the main workload. Monitor replication lag in multi-node databases before and after the change.
In distributed systems, a new column must be forward-compatible. Deploy schema changes first. Then release application code that reads and writes to it. Avoid breaking old code paths until all nodes run the updated version. This two-step deploy prevents downtime and data corruption.