The migration failed on the last record. The logs pointed to one issue: a missing new column.
Adding a new column in a production database can be simple or dangerous, depending on how you approach it. Schema updates, especially with large tables, can lock writes and impact uptime. The goal is to introduce the column without blocking queries, breaking API contracts, or corrupting data.
First, define the new column in a way that matches current data patterns. Always choose explicit data types. Avoid NULL defaults unless the application logic is ready for them. For critical systems, add the column as nullable before enforcing constraints. This gives you a deployment window for backfilling.
Next, backfill in small batches to avoid write amplification and replication lag. Use indexed lookups where possible. Monitor CPU, memory, and I/O during the process. If your database supports it, run the migration in an online schema change tool to reduce lock contention.