Vertex AI Extensions
Vertex ai extensions link foundation models to external APIs and private data sources. You set them up by defining an API schema and assigning cloud roles. Teams often start by giving the model a single service account role. That role lets the model call internal databases and third-party tools. When permissions are too broad, the model can run risky commands or pull extra data. How vertex ai extensions work in practice A foundation model on its own only knows its training data. It cannot rea
AI Agent Security: the complete guide
Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.
Vertex ai extensions link foundation models to external APIs and private data sources. You set them up by defining an API schema and assigning cloud roles. Teams often start by giving the model a single service account role. That role lets the model call internal databases and third-party tools. When permissions are too broad, the model can run risky commands or pull extra data.
How vertex ai extensions work in practice
A foundation model on its own only knows its training data. It cannot read live records or trigger actions in other apps. Vertex ai extensions bridge this gap during a live user chat. You give the platform a manifest that describes your target API endpoints. This document tells the model what each endpoint does and what inputs it needs.
When a user asks a question, the model reads the prompt first. It checks if any registered tool can help answer the request. If a match exists, the model builds a structured API call. The cloud platform then sends that request to the external service. Your service runs the action and sends the response back to the platform. Finally, the model reads the raw output and writes a plain answer for the user.
To make this work, you must set up cloud permissions. You assign a service account or token to the tool definition. The target API checks this identity before it accepts any incoming call. You also set up network rules so the cloud platform can reach private endpoints. This keeps traffic off the public web when calling internal apps.
What native setup solves and what it misses
Native cloud roles handle the basic login step very well. They prove that the request came from your cloud project. Network settings also block outside attackers from reaching your private endpoints. These controls decide which service account can talk to a given API. For simple public data lookups, this native setup is often enough.
Still, login checks only happen at the door. Once the service account logs in, the target app trusts every request it sends. The native role does not inspect what individual statements or commands do in flight. If a model generates a bad query, the database will still run it. A broad role might let a model delete rows instead of just reading them.
AI Agent Security: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
Data exposure is another major gap during live execution. An API often returns full records when the model only needs one field. Those records might hold emails, phone numbers, card numbers, or national IDs. Without field-level filtering, this data flows straight into the model prompt context. Data masking rewrites matched fields in results before they reach the model. Teams also need clear logs of every tool call and user identity. Shared service accounts hide which human user triggered a specific model action.
Managing permissions for model tool calls
To reduce risk, teams break large APIs into smaller, focused endpoints. You can create separate read and write tools with distinct service accounts. Giving a read-only role to a lookup tool stops accidental state changes. You should also strip unneeded fields from your API responses at the source.
Even with split roles, managing dozens of service accounts takes heavy manual work. Every new tool adds another key or role to rotate and audit. Security teams still need a way to check requests and log actions in real time.
Add runtime guardrails and masking beside your databases and APIs
Login rules, role grants, and CI checks govern who connects to your stack. Yet any allowed script, engineer, or AI agent still runs commands with the full scope of its role. For workloads on Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes, SSH, or internal HTTP services, hoop.dev sits beside the service as an open-source proxy, a sidecar for your database or API.
Before a query runs, hoop.dev guardrails check the statement against your rules. An AI Analyzer you configure can score intent or hold risky calls for human review. Next, hoop.dev data masking rewrites sensitive fields in responses, and each verdict is logged per listener. Explore the MIT-licensed project on GitHub.
Frequently asked questions
What is the difference between function calling and vertex ai extensions?
Function calling stops after the model outputs a structured JSON request for your app. Your own code must execute the call and return the result. Extensions let the cloud platform run the API call directly on your behalf.
How do extensions handle login credentials for external APIs?
You attach a cloud service account or stored secret to the extension settings. The platform uses this credential to log in when it calls your API endpoint. Because the credential is shared across calls, you must scope its role carefully.
Can extensions reach databases inside a private network?
Yes, you can route extension traffic through private cloud network peering. You must still expose the database logic through a standard API endpoint first. Make sure to restrict which queries that endpoint can run on your data.