Temporal Search
You set up temporal search by defining custom attributes in your workflow visibility store. Then your team filters active or closed workflow runs using simple query rules. Engineering teams often build this setup to find stuck jobs across large clusters. Still, teams often grant broad read and write roles to the whole cluster. Broad roles let any user view sensitive workflow inputs or stop running jobs. How temporal search works under the hood Every workflow engine tracks the state of running
Session Search & Indexing: the complete guide
Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.
You set up temporal search by defining custom attributes in your workflow visibility store. Then your team filters active or closed workflow runs using simple query rules. Engineering teams often build this setup to find stuck jobs across large clusters. Still, teams often grant broad read and write roles to the whole cluster. Broad roles let any user view sensitive workflow inputs or stop running jobs.
How temporal search works under the hood
Every workflow engine tracks the state of running and past jobs in a database. Standard queries only let you find jobs by their main ID or status. When your app runs thousands of jobs, finding one customer order is hard. You need a better way to tag and filter each workflow run.
To fix this, you set up temporal search in your cluster visibility store. First, an admin registers custom attributes with specific data types like text or numbers. Next, the app code attaches key and value pairs to each workflow run. For example, a payment job might tag an account ID or a region code.
The engine sends these tags to an index store in the background. Once indexed, engineers can run list queries from the command line or web UI. They filter workflows by matching the custom tags to specific values or time ranges. This flow keeps the main workflow database fast while making status checks simple.
Where native visibility roles fall short
Native cluster roles help you control who can log in to a namespace. You can give an engineer a read role for one project space. You can also give an admin a write role to register new search tags. These basic rules stop outside users from viewing your job list.
Still, namespace roles only check who logs in at the start of a session. They do not inspect what individual queries or commands do in flight. Once an engineer logs in, they can read every payload in that namespace. Workflow inputs and search tags often hold customer emails, phone numbers, or card numbers.
Even a read role returns full job payloads to the engineer screen. Native roles cannot rewrite matched fields in results to hide private customer data. Also, a user with write rights can cancel a whole batch of workflows. Native roles do not require peer approval before a risky command runs.
Session Search & Indexing: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
How to keep search attributes fast and safe
Good search setup starts with strict rules on what data goes into tags. Never put raw card numbers, national IDs, or phone numbers into search attributes. Search tags are meant for routing keys, status flags, and tenant IDs. Keep private customer details inside encrypted payloads or external databases instead.
You should also limit the number of custom attributes in each namespace. Too many indexed fields will slow down the visibility store during peak load. Review your attribute list with your team before you push new workflow code.
Finally, separate your human debug access from your automated worker roles. App workers need broad rights to update job state and write tags. Human engineers only need scoped queries to check failed jobs and fix bugs. Keeping these paths separate reduces the risk of accidental data leaks or dropped jobs.
Check database and API queries on the wire
Identity tools, roles, and network rules decide who opens a session. Setup alone cannot check what each query does or mask sensitive values in a response once a session starts. When workflows touch Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes APIs, SSH, or internal HTTP APIs, hoop.dev runs beside those services as an open-source proxy, a sidecar for your Postgres or API. It inspects requests coming in and results going back on the wire.
Your login details and client tools stay as they are while hoop.dev guardrails refuse bad deletes or drops before they run. On responses, hoop.dev data masking rewrites matched sensitive values in memory. Furthermore, hoop.dev holds SOC 2 Type II, is MIT licensed, and writes every query and decision to an audit trail that generates evidence for HIPAA, GDPR, and PCI DSS. See the source on GitHub.
Frequently asked questions
What is the difference between default and custom search attributes?
Default attributes track basic system data like start time, close time, and run status. Custom attributes are extra fields you define to track app data like order IDs.
Can I store sensitive customer data inside search attributes?
You should avoid putting private data like emails or card numbers in search tags. Anyone with read access to the namespace can query and view those indexed fields.
Why do my new workflow tags not show up right away?
The workflow engine updates the visibility index in the background after state changes. This design keeps job execution fast, so search results may lag by a second.