How to Configure Haystack ClickHouse Credentials

Published
Length
3 min read

Luis Felipe Dias

All posts

To set up haystack clickhouse credentials, create a dedicated database user with SQL access control. Pass that user login into your Haystack store or node settings. Teams often start by sharing a broad admin login across every pipeline. This fast path creates a big risk in production. If a pipeline has full rights, one bad query can drop tables. It can also read private customer columns that the model does not need. How to set up haystack clickhouse credentials in ClickHouse ClickHouse treats

Your coding agents run with your access.

Claude Code reaches prod with a developer's credentials, and approval prompts get clicked through. hoop.dev checks every query, masks sensitive data, and holds risky changes for a human.

To set up haystack clickhouse credentials, create a dedicated database user with SQL access control. Pass that user login into your Haystack store or node settings. Teams often start by sharing a broad admin login across every pipeline. This fast path creates a big risk in production. If a pipeline has full rights, one bad query can drop tables. It can also read private customer columns that the model does not need.

How to set up haystack clickhouse credentials in ClickHouse

ClickHouse treats users, roles, row policies, settings profiles, and quotas as access entities. You can create and manage all of these entities with SQL. First, you must turn on SQL-driven access control on the server. This step needs a storage directory set in the server settings. It also needs at least one user with access management turned on. That user also needs named collection control set to one.

Once you turn this on, an admin can create a new user for Haystack. Creating a user requires the create user privilege in the database. Note that a user can only grant privileges that they already hold. When you create the user, you define how that account logs in. You can also restrict which hosts the Haystack app can log in from. Finally, you can assign default roles and specific user settings to that account.

After you make the account, grant it only the rights your pipeline needs. Many retrieval pipelines only need to read vector data and text chunks. You can grant read rights on specific columns instead of the whole table. The Haystack user can then query only those exact columns. If the pipeline runs a query for other columns, the database blocks it. A wildcard select query also fails unless the table holds only the allowed columns. Since version 24.10, you can also grant read rights using a table name prefix. A trailing asterisk lets you match a group of tables for Haystack.

What native database roles solve and what they miss

Native ClickHouse users and column grants solve the first layer of pipeline safety. They stop an AI app from reading banned tables or writing bad data. Host limits also make sure logins only come from your known app servers. Giving Haystack its own role stops you from sharing admin keys across teams.

Still, static database logins still leave a gap during live pipeline runs. A database login only checks who logs in at the start. It does not inspect the contents of allowed columns while queries run. An allowed text column in a document store might still hold raw user data. It could contain emails, phone numbers, card numbers, or national IDs. Column grants let the whole text column through to the Haystack pipeline.

Ephemeral Credentials + ClickHouse Access Management: architecture patterns and best practices

Continue reading? Get the full guide.

Free. No spam. Unsubscribe anytime.

Static keys stored in app settings also create key rotation work for engineers. Every time you rotate a password, you must update the Haystack app settings. Also, shared app logins make auditing hard for security teams. When five pipelines share one database user, all queries look the same in logs. You cannot tie a strange query back to a specific run or person.

How to tighten pipeline security

Start by auditing every table that your Haystack pipelines read or write. Remove wildcard grants and switch to explicit column grants where possible. Group these grants into a clear role for each pipeline type. Keep read-only retrieval pipelines separate from ingestion jobs that write new rows. Review your host rules so local test apps cannot log into production stores. Finally, plan how to mask sensitive fields like emails or card numbers in results. Masking rewrites matched fields in results before they reach the model prompt.

Add runtime guardrails and masking beside ClickHouse

Database grants and network rules control who can reach ClickHouse. Even so, an allowed client or agent can send an alter mutation that deletes rows or read raw personal data. In practice, hoop.dev runs beside the server as a sidecar for your ClickHouse and checks native TCP queries with no change to your drivers.

With hoop.dev ClickHouse support, guardrails refuse banned queries before they run, and data masking rewrites matched string columns in results. The audit trail logs which entity types were masked and how many without storing raw values. Furthermore, hoop.dev holds SOC 2 Type II and is open source on GitHub.

Frequently asked questions

Why did my wildcard query fail after setting column grants?

ClickHouse blocks wildcard select queries when a user only has column-level grants. The query only works if the table has solely the granted columns. Update your Haystack store query to list the exact allowed column names.

How do I turn on SQL access control for ClickHouse users?

You must configure a storage directory path in the server settings file. You also need at least one user with access management set to one. That user must also have named collection control set to one.

Can I grant Haystack read access to multiple tables at once?

Yes, since version 24.10, you can use a trailing asterisk on table names. This lets you grant read access across all tables sharing that name prefix.

The query runs. The emails stay hidden.