The query hit production at noon, and everything slowed to half speed. Logs flooded with waits, deadlocks, and timeouts. The culprit was simple: a new column.
Adding a new column in a live database can be fast in dev but dangerous in prod. Schema changes block writes, lock tables, or trigger full table rewrites depending on the database engine and storage format. In systems with millions of rows, even a single ALTER TABLE ADD COLUMN can stall critical services.
The safest way to add a new column is to match the change to your database’s locking and migration behavior. For PostgreSQL, adding a nullable column without a default is instant. Setting a default for existing rows rewrites data; avoid this on large tables in one step. Instead, add the column first, then populate it in small batches. MySQL and MariaDB vary based on storage engine—InnoDB supports fast metadata changes for some types, but not all. Test the exact statement on a copy of production data before running it live.