The migration broke at 02:14. A missing new column derailed the deployment, freezing dashboards and locking writes. Everyone was watching the logs.
Adding a new column should be routine. In practice, it can trigger downtime, schema drift, or lost data if done carelessly. The process starts with defining the column’s structure: name, datatype, default values, constraints, and whether it allows nulls. Precision here prevents cascading errors later.
Migrations must be explicit. Modify the schema in version control. Use tools like Liquibase, Flyway, or native ORM migrations to track changes. A new column must be introduced in a way that older application versions can still operate. This often means deploying schema changes first, then updating code to consume them.
Be mindful of large tables. Adding a new column with a default value can lock rows for hours. Slice migrations into smaller batches or create the column without defaults, backfilling incrementally via background jobs. This avoids blocking writes and keeps latency low.