Datadog Observability Platform
The datadog observability platform collects metrics, logs, and traces from your apps in one place. You set it up by installing a host agent and assigning user roles. Many teams start by giving every developer broad admin rights to read all logs. This wide role lets anyone view raw event streams across every server. When permissions are too broad, people can see sensitive user data in error logs. How the datadog observability platform architecture works The system relies on a local agent that
AI Observability + Platform Engineering Security: the complete guide
Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.
The datadog observability platform collects metrics, logs, and traces from your apps in one place. You set it up by installing a host agent and assigning user roles. Many teams start by giving every developer broad admin rights to read all logs. This wide role lets anyone view raw event streams across every server. When permissions are too broad, people can see sensitive user data in error logs.
How the datadog observability platform architecture works
The system relies on a local agent that runs on your servers and containers. This agent gathers system metrics, app traces, and log files as they happen. It then sends this data over encrypted network links to the central cloud service. Once the data arrives, the service indexes it so teams can search and alert. Admins manage who can view this data by setting up role-based controls.
You can tie these roles to your single sign-on tool to manage team logins. Built-in roles split users into admins, standard users, and read-only viewers. Admins can also build custom roles that limit which log indexes a team reads. The platform also uses ingestion pipelines to parse and scrub logs before storage. You can define patterns to find and hide card numbers or emails in logs.
Where native log and role controls fall short
Native roles and log pipelines solve basic dashboard security and help with storage rules. They decide who can log in and which stored log views a team opens. Still, monitoring tools only show what happened after a query or command already ran. When an alert fires, an engineer still has to log into the live server. They must connect to the database or host to fix the broken app.
The monitoring tool does not control what happens during that live fix session. Native roles in your dashboard cannot block a bad query on a production database. They also cannot mask phone numbers or national IDs returned in a live shell. Once an engineer logs into a host, dashboard permissions no longer limit their actions. Teams need a way to check every live command and rewrite sensitive query results.
AI Observability + Platform Engineering Security: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
Managing live debug sessions after an alert fires
During an outage, engineers move fast from reading charts to running live server commands. They often share admin keys or use broad database roles to fix the issue. This jump from the dashboard to the terminal creates a major blind spot. Audit logs in the chart tool will not show what queries ran directly. You must govern the live session itself to keep production data safe during fixes.
A strong setup checks every statement and command before it hits the target system. It also rewrites matched fields like emails or card numbers in the live results. This approach keeps sensitive data safe while engineers fix the root cause of alerts.
Add runtime guardrails and masking beside your databases and APIs
Login rules, role grants, and CI checks govern who connects to your stack. Yet any allowed script, engineer, or AI agent still runs commands with the full scope of its role. For workloads on Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes, SSH, or internal HTTP services, hoop.dev sits inline in the protocol beside the service as an open-source proxy, a sidecar for your database or API. On every production query an agent or human runs, hoop.dev decides whether to pass, mask, block, or escalate it for review.
Before a query runs, hoop.dev guardrails check the statement against your rules, an AI Analyzer you configure can score intent or hold risky calls for human approval, and hoop.dev data masking rewrites sensitive fields in responses in memory. Explore the MIT-licensed project on GitHub.
Frequently asked questions
How does the platform collect data from servers?
You install a lightweight agent on your hosts or container clusters to gather data. The agent collects metrics, traces, and logs and sends them to the cloud backend.
Can native roles restrict which logs a team sees?
Yes, admins can set up custom roles and scope them to specific log indexes. This limits which stored log streams a user can search inside the web dashboard.
Why do teams need extra controls during an incident?
Dashboards only show past metrics and logs, so engineers must still access live servers. Extra controls govern the live commands and queries engineers run to fix the outage.