The deployment had stalled because the database migration failed. The error log was blunt: column does not exist. The fix was clear—add a new column. The challenge was doing it without downtime, data loss, or risk to production.
A new column in a relational database seems simple. In reality, schema changes in production demand precision. You must assess how the new column interacts with indexes, constraints, triggers, and application code. Adding it is only the first step—backfilling data must be safe, predictable, and fast.
For PostgreSQL, ALTER TABLE ADD COLUMN is straightforward, but default values on large tables can lock writes. The optimal path is adding the column as nullable, backfilling in batches, then applying the default in a separate migration. MySQL and MariaDB have similar considerations, especially around lock behavior, storage engines, and replication lag.
A new column also means new assumptions in your application. The code that writes to it must be feature-flagged until the schema exists in all environments. Reads must handle the column being null. Deployment orchestration should ensure backward compatibility, making rollback possible if a migration fails after deployment.