Adding a new column to a production database is never just adding a column. It can lock tables, cause downtime, and trigger cascading schema issues. The larger the dataset, the higher the stakes. In distributed systems, a new column touches replication, indexing, application code, and every query that relies on the old schema.
The safest path is a zero-downtime migration. First, create the new column in a non-blocking way. In PostgreSQL, that might mean adding the column with a default of NULL, then backfilling data in controlled batches. Avoid default value expressions that rewrite the whole table. In MySQL, use tools like pt-online-schema-change or native ALGORITHM=INPLACE to reduce impact.
After creation, update application code to write to both old and new structures if needed. Use feature flags to control rollout. Monitor query plans — an unexpected sequential scan on a new column can kill performance. In high-traffic workloads, even metadata changes can compete for locks. Test the migration in a staging environment with production-like load before touching live databases.