Elasticsearch Promql
You run an elasticsearch promql setup by pairing a Prometheus scraper with your search cluster. You can also use a dashboard app that translates PromQL into native search queries. Teams often set this up to view metric series alongside log indices in one place. To move fast, admins often grant wide read roles to the query service account. This broad role lets any user query sensitive index data through the dashboard. How an elasticsearch promql architecture works Prometheus and Elasticsearch
Elasticsearch Security: the complete guide
Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.
You run an elasticsearch promql setup by pairing a Prometheus scraper with your search cluster. You can also use a dashboard app that translates PromQL into native search queries. Teams often set this up to view metric series alongside log indices in one place. To move fast, admins often grant wide read roles to the query service account. This broad role lets any user query sensitive index data through the dashboard.
How an elasticsearch promql architecture works
Prometheus and Elasticsearch store and read data in very different ways. Prometheus stores numeric time series with labels and uses PromQL for math on metrics. Elasticsearch stores JSON documents inside indices and uses its own query language for search. To bridge them, teams usually deploy a metric exporter or a unified dashboard tool. The exporter reads cluster health stats and turns them into standard Prometheus metric formats. The Prometheus server then scrapes those metrics on a set timer. Users can then write PromQL queries to check node health, memory use, and disk space.
Another path uses a visualization app or a storage adapter between the two systems. In this design, Prometheus sends remote read or write requests to the search cluster. An adapter translates the PromQL query into a native search aggregation request on the fly. The search cluster runs the aggregation over its time series indices and returns the numbers. This lets engineers use one query language for both system metrics and app logs. Each query still relies on the base role assigned to the translation service.
How native roles work for metric queries
The search cluster controls access through internal roles and index permissions. You create a role that grants read rights to specific metric or log indices. Next, you bind that role to a user account or an API key. The dashboard tool or storage adapter uses this key to log in and run queries. Native role checks happen when the client first sends the request to the cluster.
For higher security, teams can enable field and document level security rules. These rules limit which specific fields or documents a given role can read. Still, setting up fine rules for every metric label takes a lot of manual work. Because of this, most teams assign one shared read role to their visualization tool. Every engineer who logs into the dashboard shares that exact same cluster permission level.
Elasticsearch Security: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
What native setup solves and what it misses
Native roles do a good job of blocking outside users from your data cluster. They ensure that only your chosen dashboard or scraper can connect and read indices. They also let you separate write rights for ingestion from read rights for charts. This stops a person viewing a chart from deleting an index by mistake.
The gap appears once the shared service account logs into the cluster. Native setup checks who logs in, not what each individual query does in flight. If a log index holds emails, phone numbers, or card numbers, queries return them raw. A broad query can also pull millions of documents and slow down the whole cluster. Native roles cannot rewrite matched fields in results per user or pause risky queries for review.
Check database and API queries on the wire
Identity tools, roles, and network rules decide who opens a session. Setup alone cannot check what each query does or mask sensitive values in a response once a session starts. When workflows touch Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes APIs, SSH, or internal HTTP APIs, hoop.dev runs beside those services as an open-source proxy, a sidecar for your Postgres or API. It inspects requests coming in and results going back on the wire.
Your login details and client tools stay as they are while hoop.dev guardrails refuse bad deletes or drops before they run. On responses, hoop.dev data masking rewrites matched sensitive values in memory. Furthermore, hoop.dev holds SOC 2 Type II, is MIT licensed, and writes every query and decision to an audit trail that generates evidence for HIPAA, GDPR, and PCI DSS. See the source on GitHub.
Frequently asked questions
Can Elasticsearch run PromQL queries without extra tools?
No, the search engine uses its own JSON query language and piped query language. You need an adapter, a metric exporter, or a dashboard tool to translate PromQL requests.
Why do shared dashboard keys create data risks?
A shared key gives every dashboard user the same broad read rights on the cluster. Anyone with dashboard access can query indices that hold emails, card numbers, or national IDs.
How can teams protect sensitive fields during metric and log queries?
Teams should inspect queries in flight and apply data masking to the returned results. Masking rewrites matched fields like phone numbers and emails before the user sees the output.