Service Accounts

Connect a Sidecar issues one token per Sidecar before it is deployed. That works for a few Sidecars. It does not work for a fleet that automation deploys: every workspace would need a call to the Control Plane, a secret to store and a secret to inject, before the pod even exists. A Sidecar can authenticate with the identity its platform already gives it instead:

A Kubernetes service account. A projected token, signed by the cluster, rotated by the kubelet. GKE, EKS, AKS and self-managed clusters.
A Google service account. An ID token from the metadata server, on GKE with Workload Identity, GCE or Cloud Run.

An admin adds one sidecar service account entry per cluster or project. Every workspace whose service account matches it enrolls on its first handshake. The manifest of a workspace is static: no token, no pre-step.

This page says workspace for whatever unit you already use to separate workloads: a namespace per team, application, environment or tenant on Kubernetes, or a VM or Cloud Run service on Google Cloud. Each workspace runs its own Sidecar, in front of the databases and services its workload uses, and gets its own Sidecar in the Control Plane. Identity Mapping shows the common layouts.

The token flow keeps working unchanged. A Control Plane can serve Sidecars of both kinds at the same time.

How it works

Two halves: the Sidecar gets a token from its platform, and the Control Plane checks that token against keys the issuer publishes.

1. The Sidecar gets a token

Nothing calls the Sidecar. It reads a token its platform already maintains, on every request to the Control Plane. The kubelet never talks to the Sidecar: it only keeps the file current.

2. The Control Plane checks the token

The Control Plane only fetches public keys. It never calls the Kubernetes API or a Google API with credentials, and it never connects to the Sidecar.

Connectivity the Control Plane needs

Issuer	Outbound HTTPS from the Control Plane to
GKE	`container.googleapis.com`, the cluster’s issuer and its `jwks` path
EKS	`oidc.eks.<region>.amazonaws.com`, the cluster’s issuer and its keys
Google service account	`accounts.google.com` (discovery) and `www.googleapis.com` (keys)
Self-managed cluster, AKS without a public OIDC issuer, any issuer on a private network	Nothing: paste the keys into the entry’s `jwks`. See Clusters the Control Plane cannot reach.

Discovery connects only to public addresses and does not go through HTTPS_PROXY. A Control Plane without direct egress to the issuer uses a static jwks too. Keys are cached per issuer for an hour, fetched again when a token names a key the cache does not hold (at most once a minute), and refused once a refresh has been failing for 24 hours. The Sidecar needs outbound HTTPS to the Control Plane only, plus the metadata server for a Google service account.

What the Control Plane decides

The name is the key. The entry turns the token’s subject into a Sidecar name. The Control Plane looks the Sidecar up by that name in your organization and creates it only when none exists. Restarts, new pods, rollouts and replicas all produce the same name, so they all reach the same Sidecar.
A Sidecar is bound to the first service account that reaches it. From then on only that service account reaches it. Another service account whose entry renders the same name is refused until an admin clears the binding, so a namespace that happens to share a name cannot take a Sidecar over.
The identity only authorizes. A token proves which service account is calling; the entry decides whether that service account may reach a Sidecar, and which one.
First boot is the normal import. A new Sidecar holds no configuration, so the handshake answers 412 and the Sidecar seeds it from its local config file, exactly as in Connect a Sidecar. From then on the Control Plane owns the configuration.
The token is re-read on every request. A projected token lives an hour by default and the kubelet replaces the file before it expires. A Google ID token is cached until five minutes before it expires.

Steps

Give each workspace its own service account

The service account is the identity the Control Plane matches. Create one per workspace, before the Sidecar is deployed.

Kubernetes
Google

One Kubernetes service account per workspace namespace. The same name in every namespace is enough: hoop-sidecar in ws-acme and hoop-sidecar in ws-beta are two identities, because the namespace is part of the subject.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: hoop-sidecar
  namespace: ws-acme

Or kubectl -n ws-acme create serviceaccount hoop-sidecar. With the Helm chart, serviceAccount.create: true and serviceAccount.name: hoop-sidecar create it for you. Its tokens carry this subject:

system:serviceaccount:ws-acme:hoop-sidecar

The service account needs no RBAC role: the Sidecar does not call the Kubernetes API.

One Google service account per workspace. It needs no IAM role on the project to obtain an ID token from the metadata server.

gcloud iam service-accounts create sidecar-acme --project my-project
# → sidecar-acme@my-project.iam.gserviceaccount.com

Then run the Sidecar as that service account:

GKE with Workload Identity: the cluster needs Workload Identity, and the project needs the IAM Credentials API, which the metadata server uses to mint the ID token. Autopilot clusters have Workload Identity on by default. For a Standard cluster:
gcloud services enable iamcredentials.googleapis.com gcloud container clusters create eu \ --workload-pool my-project.svc.id.goog \ --workload-metadata GKE_METADATA
Allow the workspace’s Kubernetes service account to act as the Google service account. This binding is the one IAM grant the setup needs:
gcloud iam service-accounts add-iam-policy-binding sidecar-acme@my-project.iam.gserviceaccount.com \ --role roles/iam.workloadIdentityUser \ --member "serviceAccount:my-project.svc.id.goog[ws-acme/hoop-sidecar]"
Then annotate the Kubernetes service account with iam.gke.io/gcp-service-account: sidecar-acme@my-project.iam.gserviceaccount.com. With the Helm chart, set it in serviceAccount.annotations (Step 3): the chart creates the service account, so a kubectl annotate before the install finds nothing to annotate. Without the chart, kubectl -n ws-acme annotate serviceaccount hoop-sidecar iam.gke.io/gcp-service-account=….
A new binding takes a minute or two to apply. Until it does, the Sidecar exits with 403 Forbidden … iam.serviceAccounts.getOpenIdToken and Kubernetes restarts it. It enrolls once the binding is live, with nothing to redo.
GCE: gcloud compute instances create … --service-account sidecar-acme@my-project.iam.gserviceaccount.com
Cloud Run: gcloud run deploy … --service-account sidecar-acme@my-project.iam.gserviceaccount.com

Its tokens carry email: sidecar-acme@my-project.iam.gserviceaccount.com.

Add a sidecar service account entry

One entry per cluster (Kubernetes) or per project (Google). Entries are managed through the API by an admin; the organization needs an Enterprise license.Read the cluster’s issuer:

kubectl get --raw /.well-known/openid-configuration | jq -r .issuer
# GKE: https://container.googleapis.com/v1/projects/my-project/locations/europe-west1/clusters/eu
# EKS: https://oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE

Create the entry:

curl -X POST https://hoop.your-company.com/api/sidecar-service-accounts \
  -H "Authorization: Bearer $HOOP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "gke-eu",
    "issuer": "https://container.googleapis.com/v1/projects/my-project/locations/europe-west1/clusters/eu",
    "audience": "https://hoop.your-company.com",
    "claim": "sub",
    "subject_pattern": "system:serviceaccount:ws-*:hoop-sidecar",
    "name_template": "gke-eu-{1}"
  }'

With this entry, the service account hoop-sidecar in namespace ws-acme reaches the Sidecar gke-eu-acme. See The entry for every field.

Deploy the workspace

Helm chart
Kubernetes manifest
Google service account

values.yaml

serviceAccount:
  create: true
  name: hoop-sidecar
controlPlane:
  url: https://hoop.your-company.com
  identity: kubernetes
config:
  listeners:
    - name: appdb
      protocol: postgres
      listen: 0.0.0.0:15432
      upstream: appdb:5432   # a short name resolves in the pod's own namespace
  admin:
    listen: 0.0.0.0:19000

The chart mounts a projected token with the Control Plane URL as its audience and sets HOOP_SIDECAR_IDENTITY_TYPE=kubernetes. It refuses to render controlPlane.token and controlPlane.identity together.

spec:
  serviceAccountName: hoop-sidecar
  containers:
  - name: sidecar
    image: hoophq/hoopsidecar:latest
    env:
    - {name: HOOP_CONTROL_PLANE_URL, value: https://hoop.your-company.com}
    - {name: HOOP_SIDECAR_IDENTITY_TYPE, value: kubernetes}
    volumeMounts:
    - {name: config, mountPath: /etc/hoop-inspect, readOnly: true}
    - {name: hoop-sidecar-identity, mountPath: /var/run/hoop-sidecar, readOnly: true}
  volumes:
  - name: config
    configMap: {name: hoop-sidecar}   # key config.yaml, imported on the first boot
  - name: hoop-sidecar-identity
    projected:
      sources:
      - serviceAccountToken:
          audience: https://hoop.your-company.com
          expirationSeconds: 3600
          path: token

The image reads its config file from /etc/hoop-inspect/config.yaml; the first boot imports it. With HOOP_SIDECAR_IDENTITY_TYPE=kubernetes the Sidecar reads its token from /var/run/hoop-sidecar/token; mount it elsewhere and set HOOP_SIDECAR_IDENTITY_TOKEN_FILE to the path. Use a projected token with the Control Plane URL as its audience, not the default service account token: that one’s audience is the Kubernetes API server, and the Control Plane refuses it.

On GKE, with the Helm chart:

values.yaml

serviceAccount:
  create: true
  name: hoop-sidecar
  annotations:
    iam.gke.io/gcp-service-account: sidecar-acme@my-project.iam.gserviceaccount.com
controlPlane:
  url: https://hoop.your-company.com
  identity: gcp
config:
  listeners:
    - name: appdb
      protocol: postgres
      listen: 0.0.0.0:15432
      upstream: appdb:5432
  admin:
    listen: 0.0.0.0:19000

To keep one values file for every workspace, pass the annotation per release. The dots in the key need escaping:

helm upgrade --install sidecar oci://ghcr.io/hoophq/helm-charts/hoopsidecar-chart \
  -n ws-acme -f values.yaml \
  --set-string "serviceAccount.annotations.iam\.gke\.io/gcp-service-account=sidecar-acme@my-project.iam.gserviceaccount.com"

On GCE, Cloud Run or anywhere else the process runs as the Google service account:

export HOOP_CONTROL_PLANE_URL=https://hoop.your-company.com
export HOOP_SIDECAR_IDENTITY_TYPE=gcp
hoop start sidecar --config config.yaml

A pod whose Kubernetes service account has no annotation does not fall back to the node’s service account. It exits with code 1 and the metadata server’s 404, before it reaches the Control Plane.

Check it enrolled

The Sidecar log says:

control plane connected  url=https://hoop.your-company.com
configuration imported into the control plane  listeners=1

GET /api/sidecars lists it under the rendered name, with the service account that reached it in identity_issuer and identity_subject, and a recent last_seen_at.

Credentials on the Sidecar

A Sidecar connected to a Control Plane uses exactly one credential: the token, or the identity type. Both set is a startup error that names both.

Variable	Credential
`HOOP_SIDECAR_TOKEN`, or `--token`	The token issued when the Sidecar is registered. Unchanged.
`HOOP_SIDECAR_IDENTITY_TYPE`	`kubernetes` reads a projected Kubernetes service account token from a file. `gcp` fetches a Google ID token from the metadata server. Empty means the token. Any other value stops startup.
`HOOP_SIDECAR_IDENTITY_TOKEN_FILE`	`kubernetes` only. The token’s path; default `/var/run/hoop-sidecar/token`. Read on every request; an empty file or one over 16 KiB is refused.
`HOOP_SIDECAR_IDENTITY_AUDIENCE`	`gcp` only. The Google ID token’s audience; default the Control Plane URL. For a projected token the audience is set on the volume. See One organization per issuer and audience. `GCE_METADATA_HOST` overrides the metadata host.

HOOP_SIDECAR_IDENTITY_TOKEN_FILE or HOOP_SIDECAR_IDENTITY_AUDIENCE set for another type stops startup, because it would do nothing. The identity travels in the hoop-sidecar-identity request header. A credential set without HOOP_CONTROL_PLANE_URL, or a URL without a credential, stops startup with a message that names the fix.

The entry

Field	Meaning
`name`	A label for the entry, unique in the organization.
`issuer`	The exact `iss` of the tokens. An `https` URL; the Control Plane fetches its keys through OIDC discovery at `{issuer}/.well-known/openid-configuration`, unless `jwks` is set. Discovery only reaches public addresses.
`audience`	The `aud` the tokens must carry. The Control Plane URL the Sidecars dial, unless the Control Plane is shared by several organizations.
`claim`	`sub` for a Kubernetes service account, `email` for a Google service account. With `email`, the token must carry `email_verified: true`.
`subject_pattern`	An exact value, or one with a single `*` that matches one or more characters.
`name_template`	The Sidecar name a matching token reaches. `{1}` is the text the `*` matched. The rendered name must be a valid resource name.
`jwks`	Optional. A static key set, for a cluster whose issuer the Control Plane cannot reach.
`allow_any_subject`	Allows the bare `*` pattern. See Any valid identity.
`adopt_existing_sidecars`	Lets a matching service account take over a Sidecar that was registered with a token and no service account has reached yet. Default `false`. See Moving token Sidecars to service accounts.

Clusters the Control Plane cannot reach

GKE and EKS publish their issuer’s keys on the internet, so discovery needs nothing more. A self-managed cluster, or AKS without its OIDC issuer enabled, usually does not. Discovery also refuses private, loopback and link-local addresses, so an issuer on a private network needs a static key set. Paste the cluster’s key set into the entry:

kubectl get --raw /openid/v1/jwks

{ "name": "onprem", "issuer": "https://kubernetes.default.svc.cluster.local", "jwks": { "keys": [ ... ] }, ... }

A static key set does not follow a rotation of the cluster’s signing key. Update the entry when the cluster rotates it.

Several clusters

Give each cluster’s entry its own prefix. Two clusters that both have a ws-acme namespace then reach two Sidecars, not one:

Entry	Issuer	`name_template`	`ws-acme` becomes
`gke-eu`	the EU cluster	`gke-eu-{1}`	`gke-eu-acme`
`gke-us`	the US cluster	`gke-us-{1}`	`gke-us-acme`

Without the prefix, both clusters render acme and share one Sidecar and its configuration.

Which entry wins

An exact subject_pattern beats a wildcard one.
Among wildcards, the one with the longest literal text wins.
A token that matches entries in more than one organization is refused.

One organization per issuer and audience

An issuer and audience pair belongs to one organization. Another organization that writes the same pair gets 409:

this issuer and audience are already used by another organization; give this organization's sidecars their own audience (HOOP_SIDECAR_IDENTITY_AUDIENCE, or controlPlane.identityAudience in the helm chart) and use the same value here

On a Control Plane that serves one organization, the Control Plane URL is the audience and nothing else is needed. On one shared by several organizations, each picks its own audience, any string the issuer will mint, for example https://hoop.your-company.com/org/acme:

Kubernetes: set it on the projected token volume, or controlPlane.identityAudience in the Helm chart.
Google: HOOP_SIDECAR_IDENTITY_AUDIENCE, or controlPlane.identityAudience in the Helm chart.

Several entries in the same organization may share a pair, for example one per namespace pattern in one cluster.

Any valid identity

A bare * pattern admits every service account of the issuer. It needs "allow_any_subject": true, set on purpose:

{ "issuer": "<your cluster's issuer>", "subject_pattern": "*", "allow_any_subject": true, "name_template": "gke-eu-{1}", ... }

Use it only on an issuer your organization owns, such as your own cluster. It is refused for https://accounts.google.com: every Google account in the world holds a valid token from that issuer. A Google entry must use claim: email and end in a literal @<project>.iam.gserviceaccount.com:

{ "issuer": "https://accounts.google.com", "claim": "email", "subject_pattern": "sidecar-*@my-project.iam.gserviceaccount.com", "name_template": "gcp-{1}", ... }

What happens when…

Event	Result
The pod restarts, is rescheduled, or rolls out with new pod names	Same name, same Sidecar, same configuration. Nothing is created.
The Deployment has several replicas, or scales	All replicas reach one Sidecar. Its check-in shows whichever pod handshook last.
The projected token rotates	The Sidecar reads the new file on its next request. Nothing restarts.
The local `config.yaml` changes after the first boot	Ignored. The Control Plane owns the configuration, and the Sidecar logs `the config file's listeners are ignored: the control plane owns the running config`. Edit the configuration in the Control Plane.
A Sidecar with the rendered name already exists, created with a token	Refused unless the entry sets `adopt_existing_sidecars: true`. With it, the service account reaches the Sidecar and binds to it; its configuration stays, and its token keeps working.
Another service account renders the name of a Sidecar already bound to a service account	`401`. The Sidecar stays with the service account that reached it first.
The service account or namespace is renamed	The Sidecar stays bound to the old service account. A renamed namespace renders a new name, so a new Sidecar is created. To keep the old Sidecar, add an exact entry for the new subject whose `name_template` is the old name, then clear the binding.
An admin deletes the Sidecar	It stays deleted. The next handshake gets `401` and nothing is created. Running pods keep serving their last configuration and log `control plane handshake failed; serving the last good config`. Clear the name to allow it again; the next boot then creates a new Sidecar and imports the local file.
A service account matches no entry	`401`, nothing created. The message names the subject.
Both a token and an identity are set	The Sidecar refuses to start. A request that carries both headers gets `400`.
The Control Plane is unreachable at boot	The Sidecar does not start, as with a token. Running Sidecars keep serving their last configuration.

Deleted names

Deleting a Sidecar that a service account reached records its name, so the next heartbeat does not bring it back. Deleting a Sidecar created with a token records nothing; its token simply stops working.

# names that are blocked
curl https://hoop.your-company.com/api/sidecar-deleted-names -H "Authorization: Bearer $HOOP_API_KEY"

# allow one again
curl -X DELETE https://hoop.your-company.com/api/sidecar-deleted-names/gke-eu-acme -H "Authorization: Bearer $HOOP_API_KEY"

Deleting, clearing the name and restarting the pod is also how a Sidecar starts over from its local config file.

Bindings

GET /api/sidecars shows the service account a Sidecar is bound to in identity_issuer and identity_subject. Clearing it lets the next service account that reaches the Sidecar bind to it:

curl -X DELETE https://hoop.your-company.com/api/sidecars/gke-eu-acme/identity -H "Authorization: Bearer $HOOP_API_KEY"

Moving token Sidecars to service accounts

Sidecars registered with a token keep their names and configuration when they move to service accounts. Create the entry with "adopt_existing_sidecars": true and a name_template that renders each Sidecar’s existing name, then redeploy each workspace with controlPlane.identity. Each Sidecar binds to the first service account that reaches it, and its token keeps working. Turn the flag off once the move is done.

Revoking access

One workspace: delete or disable its service account, or delete its Sidecar.
A whole cluster: narrow or delete its entry.

Revoking stops configuration updates and keeps new pods from starting. It does not stop traffic: running pods keep serving their last configuration while the Control Plane refuses them. To stop traffic now, also scale down or delete the workspace’s Sidecar pods.

Fleets

One entry per cluster, not per workspace. 2,000 workspaces in one cluster are one entry and 2,000 Sidecars.
One service account per workspace. A service account shared by several workspaces is one identity: all of them reach the same Sidecar and serve the same configuration, and any of them can read it.
Rules are copied per Sidecar. Each Sidecar imports the rules in its own config file as its own rule items. A rule shared by the whole fleet is changed in each copy, or in each file followed by a fresh import.
Rows outlive workspaces. A torn-down workspace leaves its Sidecar in the list. Delete it when the workspace goes away.

Identity Mapping shows every layout of credentials, clusters, projects and workspaces, with tokens, Kubernetes and Google service accounts side by side.

Troubleshooting

The Sidecar prints the Control Plane’s reason after the subject it presented:

the control plane at https://hoop.your-company.com rejected the service account identity (issuer "…", subject "system:serviceaccount:ws-acme:hoop-sidecar"): <reason>

Before a token is verified, the Control Plane says nothing more than the service account token failed verification, so an unauthenticated caller learns nothing about its configuration. The precise cause is in the Control Plane’s log, with the issuer.

Reason	Fix
`the service account token failed verification`	Read the Control Plane log for the cause: no entry names this exact `issuer`; the projected token’s audience differs from the entry’s `audience`; the token expired; or the keys do not match (a static `jwks` that is out of date, or an issuer on a private address with no `jwks`).
`no sidecar service account of this issuer allows subject "…"; check the mapping's audience and pattern`	The subject does not match any `subject_pattern`. Kubernetes: check the namespace and the service account name. Google: check the email.
`the token's email is not verified`	A Google token without `email_verified: true`.
`… renders the sidecar name "…", which is refused`	The rendered name is not a valid resource name. Change `name_template` or the pattern.
`the token matches sidecar service accounts in more than one organization`	Two organizations admit this subject. Narrow the patterns.
`sidecar "…" is bound to another service account`	Another service account reached this Sidecar first. Clear the binding if this one should own it.
`sidecar "…" was registered with a token`	Set `adopt_existing_sidecars` on the entry. See Moving token Sidecars to service accounts.
`this sidecar was deleted in the control plane`	Clear the name in Deleted names.
`access denied`, with no other reason	The Control Plane predates service account support. Upgrade it before the Sidecars.
`more than one control plane credential is set`	Set either `HOOP_SIDECAR_TOKEN` (or `--token`) or `HOOP_SIDECAR_IDENTITY_TYPE`, not both.
`HOOP_SIDECAR_IDENTITY_TOKEN_FILE is set, but only HOOP_SIDECAR_IDENTITY_TYPE=kubernetes reads it`	Add `HOOP_SIDECAR_IDENTITY_TYPE=kubernetes`, or drop the path.

A Google service account can also fail before the Control Plane is reached. The Sidecar exits with code 1 and prints the metadata server’s answer:

Reason	Fix
`the GCP metadata server … answered 404 Not Found …: Your Kubernetes service account (…) is not annotated with a target Google service account`	Annotate the Kubernetes service account with `iam.gke.io/gcp-service-account`, or set `serviceAccount.annotations` in the Helm chart.
`the GCP metadata server … answered 403 Forbidden …: Permission 'iam.serviceAccounts.getOpenIdToken' denied`	The `roles/iam.workloadIdentityUser` binding is missing, names another namespace or service account, or has not applied yet. A new binding takes a minute or two; the pod enrolls on a later restart.

Connect a Sidecar

The handshake, the license and staying in sync.

Kubernetes

The Helm chart’s Control Plane attributes.

Identity Mapping

Every layout of credentials, clusters and workspaces.

Getting Started

Sidecar

Features

Control Plane Setup

How it works

1. The Sidecar gets a token

2. The Control Plane checks the token

Connectivity the Control Plane needs

What the Control Plane decides

Steps

Credentials on the Sidecar

The entry

Clusters the Control Plane cannot reach

Several clusters

Which entry wins

One organization per issuer and audience

Any valid identity

What happens when…

Deleted names

Bindings

Moving token Sidecars to service accounts

Revoking access

Fleets

Troubleshooting

Next

Connect a Sidecar

Kubernetes

Identity Mapping

​How it works

​1. The Sidecar gets a token

​2. The Control Plane checks the token

​Connectivity the Control Plane needs

​What the Control Plane decides

​Steps

​Credentials on the Sidecar

​The entry

​Clusters the Control Plane cannot reach

​Several clusters

​Which entry wins

​One organization per issuer and audience

​Any valid identity

​What happens when…

​Deleted names

​Bindings

​Moving token Sidecars to service accounts

​Revoking access

​Fleets

​Troubleshooting

​Next

Connect a Sidecar

Kubernetes

Identity Mapping

How it works

1. The Sidecar gets a token

2. The Control Plane checks the token

Connectivity the Control Plane needs

What the Control Plane decides

Steps

Credentials on the Sidecar

The entry

Clusters the Control Plane cannot reach

Several clusters

Which entry wins

One organization per issuer and audience

Any valid identity

What happens when…

Deleted names

Bindings

Moving token Sidecars to service accounts

Revoking access

Fleets

Troubleshooting

Next