Don T Adapt Small Language Models for Tools Adapt Tool Schemas to the Models
To get reliable tool calls, strip nested fields and flatten your input definitions. You must follow the rule: don t adapt small language models for tools adapt tool schemas to the models. Teams often fine-tune small models to match complex backend interfaces. This approach wastes compute and still fails when backend roles hold broad permissions. Why you don t adapt small language models for tools adapt tool schemas to the models Small language models have tight context limits and less reasoni
Rego Policy Language + AI Tool Use Governance: the complete guide
Architecture patterns, implementation strategies, and security best practices. Delivered to your inbox.
To get reliable tool calls, strip nested fields and flatten your input definitions. You must follow the rule: don t adapt small language models for tools adapt tool schemas to the models. Teams often fine-tune small models to match complex backend interfaces. This approach wastes compute and still fails when backend roles hold broad permissions.
Why you don t adapt small language models for tools adapt tool schemas to the models
Small language models have tight context limits and less reasoning depth than huge models. When you feed them deep object trees, they guess field names and fail. Fine-tuning a model to learn complex tool inputs takes weeks of work. Every time an API changes, you must train the model all over again.
Shaping the schema to fit the model solves this problem right away. You replace deep nested inputs with flat text strings and clear number fields. You break one big multi-purpose tool into three small, single-action tools. Each tool gets a short name and a plain description of what it does.
You also limit the choices for each field to a short list of valid words. The model reads the simple schema and picks the right values on the first try. Your app code then maps those simple inputs back to the full backend format. The model stays fast and stateless while your code handles the structural work.
How native tool permissions work and what they miss
Once the schema works, the app needs permissions to run the tool against your backend. Teams set up a service role in the database or cloud app for the agent. That role holds the rights to read tables, write rows, or call internal APIs. The native system checks the role at log in and lets the session start.
This native setup proves who the agent is when the session starts. It blocks outside users who do not have a valid key or role. Still, native roles decide who logs in, not what each request does in flight. A small model with a clean schema can still send a bad query or command.
Rego Policy Language + AI Tool Use Governance: architecture patterns and best practices
Continue reading? Get the full guide.
Free. No spam. Unsubscribe anytime.
If the service role can read a whole table, any query from the model works. The native database role does not inspect individual statements before they run. It also returns raw results, so sensitive fields like emails or card numbers flow back. Without checks on each request and result, a simple prompt trick can expose private data.
How to design safe and simple schemas for agents
Start by auditing every tool your small model can call during a task. Remove optional fields that the model does not need to finish the main job. Set hard defaults in your app code instead of asking the model to pass them. Never let the model pass raw query strings or open filter objects in a tool.
Next, split read actions and write actions into separate tool definitions with separate backend roles. Give the read role access to specific views rather than full base tables. Even with flat schemas, you must watch what happens when the tool runs in production. Masking rules that rewrite emails, phone numbers, and national IDs in results keep data safe.
Govern statements and responses across databases and APIs
Good config and scoped roles are needed setup. They do not check single queries in flight or mask sensitive fields in returned rows. Across Postgres, MySQL, SQL Server, MongoDB, ClickHouse, Kubernetes APIs, SSH, and internal HTTP endpoints, hoop.dev runs beside each supported service as an open-source proxy, a sidecar for your database or API.
First, hoop.dev guardrails block banned queries before they run. Then hoop.dev data masking rewrites matched sensitive values in responses in memory. The audit trail records every statement and verdict per resource. Furthermore, hoop.dev holds SOC 2 Type II and is open source on GitHub.
Frequently asked questions
Why do small models fail on complex tool schemas?
Small models lose track of deep brackets, optional objects, and long rules inside prompts. Flat fields and short value lists match how these models predict tokens best.
Should we ever fine-tune a small model for tool use?
Fine-tuning makes sense only when the tool format never changes and speed is critical. For most teams, simplifying the schema gives better results in far less time.
Does a simple tool schema stop unsafe database queries?
A simple schema reduces syntax errors, but it does not stop bad data requests. You still need runtime checks on every statement and data masking on results.