How to Configure Opensearch Schema Migration

Published
Length
3 min read

Luis Felipe Dias

All posts

You run an opensearch schema migration by building a new index with updated field mappings. Then you copy existing documents over and switch your index alias to the new target. Teams often run these migration jobs using broad cluster admin roles to avoid permission errors. Broad roles expose production indices to accidental deletes and unrestricted reads of raw user records. How an opensearch schema migration works in practice Search engines lock field types after you create an index and writ

Your coding agents run with your access.

Claude Code reaches prod with a developer's credentials, and approval prompts get clicked through. hoop.dev checks every query, masks sensitive data, and holds risky changes for a human.

You run an opensearch schema migration by building a new index with updated field mappings. Then you copy existing documents over and switch your index alias to the new target. Teams often run these migration jobs using broad cluster admin roles to avoid permission errors. Broad roles expose production indices to accidental deletes and unrestricted reads of raw user records.

How an opensearch schema migration works in practice

Search engines lock field types after you create an index and write documents to it. You can add new fields to a live index without breaking existing search queries. Still, you cannot change the data type of an existing field in place. Changing a text field to a keyword or date requires a full index rebuild.

Teams use index aliases to make this transition invisible to the search app. Your app reads and writes through a stable alias name instead of the physical index. When you need a change, an engineer creates a second physical index with new mappings. Next, a background reindex task copies every document from the old index to the new one.

During the copy phase, the system transforms fields to match the new schema rules. Once the document counts match, an admin updates the alias in one atomic step. The alias drops the old index and points to the new index right away. After the switch, the team checks app traffic and deletes the old index to save space.

Managing permissions and roles during index updates

A schema update needs specific cluster and index permissions to run from start to finish. The runner must create indices, update mappings, run reindex tasks, and swap index aliases. The runner also needs read permissions on the source index to copy the old data. Many teams bundle these permissions into one admin role and share it across the team.

Static role assignments create risk when engineers run migration tasks from local laptops. If an engineer holds index delete rights, a small typo can drop an active index. Also, read permissions for reindexing allow that user to query raw documents at any time. Those documents often hold sensitive fields like emails, phone numbers, card numbers, or national IDs.

End-to-End Encryption + API Schema Validation: architecture patterns and best practices

Continue reading? Get the full guide.

Free. No spam. Unsubscribe anytime.

What native cluster roles solve and what they leave open

Native security roles inside the search cluster handle basic login checks and index permissions well. You can scope a role so it only touches indices that match a naming pattern. You can also split roles between automated build pipelines and human engineers on your team. This setup stops unauthorized users from logging into the cluster or viewing unrelated indices.

Still, native cluster roles only check who logs in at the start of a session. They do not inspect or approve individual commands while the session is active. Once a user has permission to swap aliases or delete indices, every command runs right away. Native roles also struggle to rewrite sensitive fields in query results without blocking reindex jobs.

To close this gap, teams separate routine read queries from high-risk schema changes. Engineers need a way to inspect mappings and sample data where masking rewrites matched fields in results. At the same time, destructive actions like deleting old indices should require peer approval before execution. Adding command-level controls keeps your search data safe while schema updates move forward.

Check database and API queries on the wire

Identity tools, roles, and network rules decide who opens a session. Setup alone cannot check what each query does or mask sensitive values in a response once a session starts. When workflows touch Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes APIs, SSH, or internal HTTP APIs, hoop.dev sits inline in the wire protocol beside those services as an open-source proxy, a sidecar for your Postgres or API. On every query an AI agent or engineer runs against production, hoop.dev decides to pass, mask, block, or hold it for human review.

Your login details and client tools stay as they are while hoop.dev guardrails refuse bad deletes or drops before they run, and hoop.dev data masking rewrites matched sensitive values in responses in memory. Furthermore, hoop.dev holds SOC 2 Type II, is MIT licensed, and writes every query and decision to an audit trail that generates evidence for HIPAA, GDPR, and PCI DSS. See the source on GitHub.

Frequently asked questions

Can you change an existing field type in OpenSearch without reindexing?

No, you cannot alter the data type of an existing field in place. Changing a field type changes how the engine stores and indexes data on disk. You must create a new index with the updated mapping and copy your documents over.

How do index aliases prevent downtime during a schema update?

An index alias acts as a pointer that sits between your app and the physical index. Your app sends all search and write requests to the alias name. When the new index is ready, you repoint the alias in one atomic operation.

What permissions are required to run a reindex task?

The user or service account needs read permissions on the source index to pull documents. It also needs write and index creation permissions on the destination index. Finally, it requires cluster permission to manage aliases and start the reindex operation.

The query runs. The emails stay hidden.