The migration was running and the table was locked. You had seconds to fix it before the queue backed up. The fix was simple: add a new column. The challenge was doing it safely, without downtime, and without breaking production.
A new column in a relational database is not just a field; it’s a schema change that can impact indexes, queries, and applications. On smaller datasets, adding columns is trivial. On large production tables, it’s risky. The operation can trigger table rewrites, lock writes, and block reads, depending on the database engine and the exact command.
To add a new column without disruption, you start by understanding your database’s alter table semantics. PostgreSQL can add a nullable new column without a rewrite. MySQL, depending on the storage engine and version, may still block writes briefly. Always check your version-specific documentation.
If the application depends on the new column immediately after deployment, design your release in phases. First, deploy code that can handle nulls for that column. Then, run the migration to add the column. Only afterward, backfill data in controlled batches to avoid spikes in load. Finally, deploy the code that relies on it for required functionality.