The logs showed nothing unusual. The schema was fine—until someone spotted the missing new column.
Adding a new column sounds simple. In production, it can break your system if you do it wrong. Schema changes lock tables, cause replication lag, and trigger unexpected side effects in code. A clean deployment plan for a new column is critical.
Start with the data model. Verify that the new column has a clear purpose and correct data type. Avoid adding it directly to critical hot tables during peak load. Use an ALTER TABLE only when you understand how your database engine handles online schema changes. For large datasets, consider tools like pt-online-schema-change or native online DDL features to avoid downtime.
Decide on defaults carefully. Null values might be safer than a non-null default that forces full table rewrites. Backfill in stages so you don’t saturate I/O. Track write amplification during the migration.