The logs showed the culprit: a missing new column in the production database.
Adding a new column should be simple, but it’s where code meets data, and mistakes here can cascade fast. In relational databases, creating a new column changes the schema. That means updates to migrations, application code, tests, and deployment processes. Done wrong, it can cause downtime, data loss, or silent failures that surface weeks later.
The first step is clarity. Define the exact column name, type, default value, and constraints. Avoid vague names. Check how the new column will be used by queries, joins, and indexes. Adding an index at creation may improve performance, but for large tables, it can lock writes during the operation. Always test these effects with representative data.
Use migrations that can run online without blocking requests. In Postgres, ADD COLUMN without a default is fast, but adding one with a default can rewrite the table. In MySQL, older versions require a full table lock for schema changes, while newer ones can handle many changes online. Research your database’s version-specific behavior before running the migration in production.