Event Source
An event source is any app, database, or queue that sends out change alerts. You set it up by linking an identity role to a downstream worker. Teams often build these links so apps react to new data right away. Still, teams often grant broad read roles just to make the pipeline work. When a role is too wide, a single bug can expose whole tables. How an event source works in your system At its core, an event source tracks state changes in a host system. When a user updates a record, the host w
Security Information & Event Management (SIEM) + Snyk Open Source: the complete guide
Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.
An event source is any app, database, or queue that sends out change alerts. You set it up by linking an identity role to a downstream worker. Teams often build these links so apps react to new data right away. Still, teams often grant broad read roles just to make the pipeline work. When a role is too wide, a single bug can expose whole tables.
How an event source works in your system
At its core, an event source tracks state changes in a host system. When a user updates a record, the host writes a new log entry. A reader process polls this log or listens for a push message. Then, the reader sends that message to a function or worker app. This design keeps your main app fast while background tasks run on their own.
To make this work, you must set up permissions between both sides. The worker needs a role that lets it read from the upstream stream. The stream also needs a rule that lets it trigger the worker app. In most cloud setups, an admin binds a role to the worker identity. That role lists which streams the worker can read and which queues it can update.
Teams also set up retry rules and dead-letter queues for failed messages. If a worker fails to read a record, the system stores that message elsewhere. Engineers then log in to inspect the bad record and fix the bug. This flow keeps data moving, even when one part of the system breaks.
Why native event source roles leave a security gap
Native roles do a good job of controlling which app can start a session. They verify the worker identity before any data leaves the host stream. They also block outside apps from reading the stream without a valid key. For basic app-to-app links, these native checks are the first line of defense.
Still, native setup rules only check who logs in at the start. They do not inspect what each request or payload holds in flight. Once a worker or engineer has a valid role, they get full stream visibility. If the stream carries card numbers or emails, those fields flow in plain text. An engineer who logs in to check a dead-letter queue sees that raw data. Data masking solves this by rewriting matched fields like phone numbers in the results.
Security Information & Event Management (SIEM) + Snyk Open Source: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
Shared admin keys make this problem much worse for growing teams. When an incident happens, several engineers might use the same high-level role to debug. Native stream logs show the shared role name, not the person at the keyboard. You cannot tell who ran a read command or who changed a stream rule.
Best practices for managing stream permissions
Start by splitting app roles from human roles across every single stream. Give your automated workers a role that can only read specific message types. Never let a background worker use an admin role meant for human debugging. This limits the blast radius if a worker key leaks in a log file.
Next, review how your team checks failed messages during an outage. Engineers often query raw stream tables to see why a payload broke a worker. Those raw payloads often hold national IDs, emails, or card numbers from users. Put rules in place that rewrite those fields before the results reach the screen. Finally, tie every manual check to an individual user identity for clear audit logs.
Check database and API queries on the wire
Identity tools, roles, and network rules decide who opens a session. Setup alone cannot check what each query does or mask sensitive values in a response once a session starts. When workflows touch Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes APIs, SSH, or internal HTTP APIs, hoop.dev runs beside those services as an open-source proxy, a sidecar for your Postgres or API. It inspects requests coming in and results going back on the wire.
Your login details and client tools stay as they are while hoop.dev guardrails refuse bad deletes or drops before they run. On responses, hoop.dev data masking rewrites matched sensitive values in memory. Furthermore, hoop.dev holds SOC 2 Type II, is MIT licensed, and writes every query and decision to an audit trail that generates evidence for HIPAA, GDPR, and PCI DSS. See the source on GitHub.
Frequently asked questions
What is the difference between an event source and an event bus?
An event source is the origin system where a data change first happens. An event bus is the middle router that receives messages and sends them to workers.
Why do stream payloads often expose sensitive user data?
Streams often copy an entire database row whenever a single field changes. This sends emails, phone numbers, and card numbers to every downstream reader at once.
How should engineers debug failed stream messages safely?
Engineers should query failed records using individual identities instead of shared admin keys. Masking rules should also rewrite sensitive fields like card numbers in the query results.