The build had passed, but the database schema was wrong. A missing field. A table out of sync. The fix was simple: add a new column. The problem was doing it without breaking production or blocking deploys.
Creating a new column in a live environment is never just about ALTER TABLE. It’s about schema migration strategy, performance impact, and the order of execution. On large datasets, adding a column can lock the table and stall queries. In a distributed system, it can trigger replication lag. The solution is planning the change to be safe, fast, and observable.
Before adding a new column, define its type, nullability, and default value with intent. Changing these later is harder and riskier than getting them right from the start. Avoid defaults that rewrite the entire table unless necessary. For high-traffic services, use migrations that apply incrementally, and test them on production-like data.