The migration ran at midnight and the error logs lit up red. The culprit: a missing new column that every downstream service expected.
Adding a new column to a database table is simple in theory, but in production it can decide whether your API stays up or falls apart. Plan it wrong and indexes lock, queries stall, and latency spikes. Plan it right and your deployment is invisible.
When introducing a new column, first confirm the schema change strategy. For large datasets, use ALTER TABLE with ADD COLUMN in a non-blocking way if your database supports it—PostgreSQL 11+ adds most columns instantly if they have no default. In MySQL, schema changes on massive tables can still cause downtime unless you use tools like gh-ost or pt-online-schema-change.
Always define the column type and constraints up front. Avoid NULL defaults unless they're required; use sensible, non-breaking defaults to support backward compatibility. If the column will be indexed, create the index in a separate migration to reduce locking risk.