Curation Federation
You set up curation federation by linking local data catalogs to a central rule engine. Each domain team keeps ownership of its tables while sharing tags and access rules. Most teams start this process by mapping central identity groups to local database roles. Broad roles often let users read or change raw tables they do not need. When permissions are too wide, one mistake can expose private records across every domain. How curation federation works in practice A federated model splits data
Identity Federation: the complete guide
Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.
You set up curation federation by linking local data catalogs to a central rule engine. Each domain team keeps ownership of its tables while sharing tags and access rules. Most teams start this process by mapping central identity groups to local database roles. Broad roles often let users read or change raw tables they do not need. When permissions are too wide, one mistake can expose private records across every domain.
How curation federation works in practice
A federated model splits data duties between a central platform group and domain teams. Domain owners define table schemas, write descriptions, and label private columns in their systems. The central system reads this metadata and builds a unified directory for the company. Users search the shared directory to find datasets and request role assignments.
Identity federation handles the login step across these linked databases and storage buckets. A central identity provider checks who the user is and sends a signed token. The target database reads the token and maps the user to a local role. That local role controls which schemas, tables, and views the user can query.
Data stewards also use federated workflows to review and approve changes to shared tags. When a domain team marks a column as sensitive, the central catalog updates its index. Downstream tools read these shared tags to enforce group rules during user sessions. This keeps data definitions consistent without forcing one team to manage every database.
What native federation solves and what it misses
Native federation solves the problem of scattered accounts and duplicate login keys. Admins do not need to create separate user profiles in twenty different data stores. When an engineer leaves the team, revoking one central account blocks future logins everywhere. It also helps data stewards track where sensitive columns live across the whole company.
Still, standard setup stops working the moment a user logs into the database. Role checks decide who can start a session, not what individual queries do in flight. Once a session begins, a user with read rights can pull millions of rows. Native catalog tags rarely stop a bad update query or a bulk data dump.
Static role mappings also create friction when engineers need quick access to fix bugs. Admins often grant wide database roles so they do not block daily repair work. Those wide roles stay active for months and expose emails, phone numbers, and card numbers. Auditing what each person actually ran across ten different database engines remains very hard.
Identity Federation: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
Managing query risk across federated domains
To reduce risk, teams must look beyond catalog tags and static login roles. Security controls need to inspect individual commands and queries right when users run them. Instead of giving permanent table rights, teams should check every request against active rules. Masking rules can rewrite emails, phone numbers, or national IDs in query results.
Centralizing command logs is just as important as centralizing table definitions in a catalog. Every domain database formats its native audit logs in a different way. Bringing query logs into one standard format helps security teams spot bad patterns fast. Good controls link every single SQL statement back to a real user identity.
Check database and API queries on the wire
Identity tools, roles, and network rules decide who opens a session. Setup alone cannot check what each query does or mask sensitive values in a response once a session starts. When workflows touch Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes APIs, SSH, or internal HTTP APIs, hoop.dev runs beside those services as an open-source proxy, a sidecar for your Postgres or API. It inspects requests coming in and results going back on the wire.
Your login details and client tools stay as they are while hoop.dev guardrails refuse bad deletes or drops before they run. On responses, hoop.dev data masking rewrites matched sensitive values in memory. Furthermore, hoop.dev holds SOC 2 Type II, is MIT licensed, and writes every query and decision to an audit trail that generates evidence for HIPAA, GDPR, and PCI DSS. See the source on GitHub.
Frequently asked questions
What is the difference between centralized curation and curation federation?
Centralized curation makes one team clean, tag, and manage every dataset in the company. Curation federation lets domain teams manage their own data while following shared central rules.
How do you keep sensitive fields safe across federated databases?
Domain stewards tag columns that hold card numbers, phone numbers, or emails. Runtime rules then use those tags to rewrite matched fields in query results.
Why do federated roles still leave security gaps during incidents?
Federated roles only check identity at login and often grant wide table rights. They do not inspect live queries or block risky commands after a session starts.
Short role checks help teams work fast. Simple rules make each review clear. Good logs keep every team safe. Scoped keys stop bad changes early.