How to Configure Elasticsearch Slow Query Log
You set up the elasticsearch slow query log by setting index time limits. The cluster then writes slow search and fetch jobs to disk. Teams often grant broad index permissions so engineers can tune these time limits. Broad roles let users change cluster rules and read raw data in logs. A tight setup limits who can change log rules and read log files. How the elasticsearch slow query log works The search engine runs each query across one or more index shards. So, the slow log tracks time at t
Your coding agents run with your access.
Claude Code reaches prod with a developer's credentials, and approval prompts get clicked through. hoop.dev checks every query, masks sensitive data, and holds risky changes for a human.
You set up the elasticsearch slow query log by setting index time limits. The cluster then writes slow search and fetch jobs to disk.
Teams often grant broad index permissions so engineers can tune these time limits. Broad roles let users change cluster rules and read raw data in logs. A tight setup limits who can change log rules and read log files.
How the elasticsearch slow query log works
The search engine runs each query across one or more index shards. So, the slow log tracks time at the shard level, not the node level. This helps you spot a single slow shard when others run fast.
Search work splits into two main steps inside the engine. First, the query phase finds matching documents and scores them inside the shard. Second, the fetch phase pulls the actual document fields to build the result. You set separate time thresholds for both the query phase and the fetch phase.
You can define four log levels: warn, info, debug, and trace. Each level gets its own time threshold for query and fetch tasks. When a shard task takes longer than your threshold, the node logs it. You can update these index rules live through the REST API. You do not need to restart any nodes when you change a threshold.
Teams often lower thresholds during an incident to catch bad search requests. When the issue ends, they raise the thresholds or turn the log off. This stops the disk from filling up with huge text files during peak traffic.
What native slow logs solve and what they miss
Native slow logs help engineers find heavy search patterns and broken index mappings. You can see which shard took too long and which query caused the delay. This makes it much easier to fix slow dashboards or heavy app searches.
Elasticsearch Security + End-to-End Encryption: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
Still, native logs only record events after a slow query hits the cluster. They do not stop a user from running an expensive wildcard search in production. Native role controls only decide who can log in and query an index. They do not inspect what each statement does while it runs in flight.
Slow logs also create a big privacy risk for compliance and security teams. When a search crosses the time limit, the engine logs the full query body. That query body often holds sensitive search terms typed by users or engineers. For example, a slow search for a customer email or phone number writes that raw value to disk. Card numbers and national IDs in search filters also end up in log shipping tools.
Also, shared API keys hide who ran the slow query in the first place. The log shows the index and the query text, but not the real person. When an engineer runs an ad-hoc query, the log only shows a generic app role. Teams need a way to tie each search request to a real user identity. They also need a way to rewrite sensitive fields in results before data leaves the system.
Govern statements and responses across databases and APIs
Good config and scoped roles are needed setup. They do not check single queries in flight or mask sensitive fields in returned rows. Across Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes APIs, SSH, and internal HTTP endpoints, hoop.dev sits inline in the protocol beside each supported service as an open-source proxy, a sidecar for your database or API. For every query or command an AI agent or engineer runs against production, hoop.dev decides to pass, mask, block, or hold it for human approval.
First, hoop.dev guardrails block banned queries before they run or hold risky ones for review, and hoop.dev data masking rewrites matched sensitive values in responses in memory. Furthermore, hoop.dev holds SOC 2 Type II and is open source on GitHub.
Frequently asked questions
Does the slow query log measure total request time or shard time?
It measures the time spent on each individual shard during a search. A fast shard stays quiet while a slow shard writes an entry to the log.
Can I change slow log thresholds while the cluster is running?
Yes, you can update slow log thresholds at any time using the index settings API. The cluster applies the new rules right away without a node restart.
Why do slow query logs create data privacy risks?
The log records the raw search body when a query crosses your time limit. If a user searches for an email or card number, that value goes into your log files.
Related guides
In this post