Rows were missing. The schema was wrong. The culprit was simple: a new column.
Adding a new column should be trivial. In production, it can be lethal. Schema changes touch live traffic, complicate deployments, and carry risk of downtime or data loss. The right process turns that risk into a controlled, repeatable step.
A new column changes the shape of your table. Whether in Postgres, MySQL, or any relational database, the core steps are the same:
- Assess the impact on queries, indexes, and application code.
- Create the column as nullable to avoid locking writes on large tables.
- Backfill data in small, batched updates to reduce load on the database.
- Deploy code that reads and writes to the new column.
- Once stable, enforce constraints or make the column non-nullable.
Never combine altering the schema and backfilling data in the same migration for large datasets. This avoids prolonged locks and improves rollback safety. Feature flags can coordinate the moment your application begins reading from the new column.