The build had passed. The deploy was clean. But the database needed one more thing: a new column.
Adding a new column seems simple. In production, it can be dangerous. Schema changes can lock tables, slow queries, and cause outages if done without planning. The solution is to approach every new column with a process that balances speed, safety, and forward compatibility.
First, define the new column with clear purpose and precise data type. A VARCHAR(255) might seem safe, but using the smallest possible type prevents bloat. Decide if it should allow NULL values up front. Adding NOT NULL with no default on a large table can block writes for minutes or hours.
Next, run the migration in a way that avoids downtime. For Postgres, use tools like pg-online-schema-change or background migrations to create the column without locking the table. In MySQL, consider pt-online-schema-change or native instant DDL if available. Test these changes in a production-like environment with realistic data volumes.