Guardrails
An organization can attach a guardrail policy to a model alias in the Portal.
The policy is a list of checks — regex, presidio entity detection, or an
LLM judge — that run on the way to the provider (input) and on the way back
(output). Each check blocks the call, redacts the offending text, or
masks it and puts it back.
It can also attach one to an LLM connection: an alias with no policy of its own is then checked with the policies of the connections it can be served by (which policy applies). To your agent the two are the same thing — a policy on the alias it calls — and everything below applies to either.
The policy binds every agent granted that alias, with no opt-out. There is no
SDK parameter to select, skip, or soften it: your agent cannot see which policy is
attached, and passing anything guardrail-shaped to core.llm.call does nothing.
Your only job as an agent author is to handle the two refusals below — though you can
also ask a policy for a verdict before spending a call.
The two refusals
from im_sdk import ImGuardrailBlockedError, ImGuardrailUnavailableError
try:
result = await core.llm.call(messages=[...], alias="gpt-mini")
except ImGuardrailBlockedError as exc:
# A check refused the content. Do not retry the same text — it will refuse again.
check = exc.context.get("check") # which check fired, e.g. "pii-email"
phase = exc.context.get("phase") # "pre" (your prompt) or "post" (the reply)
except ImGuardrailUnavailableError:
# No verdict could be reached, so the call was refused rather than sent unchecked.
...| Exception | HTTP | What happened |
|---|---|---|
ImGuardrailBlockedError | 403 | A check judged the content and refused it. context carries check, phase, and policy. |
ImGuardrailUnavailableError | 503 | A check could not reach a verdict — its backend is down, or the gateway holds a policy the plugin does not. |
ImGuardrailBlockedError subclasses ImForbiddenError, so an existing
except ImForbiddenError: already catches it; catch the precise class when you
want to react to content rejection specifically — e.g. rephrasing a prompt, or
telling the user which check objected.
A rewrite is not an error
A check whose action is redact or mask rewrites the text and lets the call
through. Your agent gets an ordinary successful ResourceResult. The names of the
guardrails that rewrote it are on the envelope:
result = await core.llm.call(messages=[...], alias="gpt-mini")
for check, phase, action in result.metadata.get("guardrails_redacted", []):
... # e.g. ("pii-email", "pre", "mask")Each entry is a (check, phase, action) triple: which check rewrote, on which side of
the call (pre is your prompt, post is the reply), and what it did (redact or
mask). The same fact is recorded on the execution trace, so you don't need to log it
yourself — read it only when the agent's own logic should know its text was rewritten.
The triple is the field's settled shape — it was a (check, phase) pair before guardrails
reached agents — so unpack all three.
redact loses the value; mask gives it back
A redact is one-way: the match is replaced with a placeholder, and neither the
provider nor your agent ever sees the original again.
A mask is a round trip. On the way out the match becomes a numbered token —
[[EMAIL_ADDRESS_1]], [[PERSON_2]] — so the model can still reason about it
("email the invoice to [[EMAIL_ADDRESS_1]]") without receiving the value. On the
way back the originals are put in again, so the text your agent reads is its own.
The same value always gets the same token, so a name mentioned three times is one
person to the model.
You do not have to do anything to un-mask a reply, and you should not try: the restore happens in the gateway, after the provider and after the provider-side call log, which is the point. If a model quotes a token back at you verbatim, that is the one case where a masked value can surface — the restore resolves it like any other.
Asking before you call
core.guardrail() runs an alias's policy over text and returns the verdict, without
calling the model:
verdict = await core.guardrail("Draft a reply to this: …", alias="gpt-4o-safe")
if not verdict.allow:
print(verdict.reason) # 'guardrail "tone": unsafe'
for check in verdict.checks:
print(check.guardrail, check.allow, check.usage)Pass a list of strings to check several at once, or check="tone" to run one named
check instead of the whole policy. verdict.rewritten gives the texts as the policy's
checks would leave them — redactions applied, masks shown as tokens — which is how you
see what the provider would actually receive.
Three things worth knowing:
- It needs no extra grant. Asking about an alias requires exactly what calling it
requires, so a policy verdict is available wherever
core.llm.callis. - It changes nothing. This is a question, not a configuration: the policy still runs in full on every real call, whatever a verdict said beforehand.
- A judge check costs money. No model is called for you here, but a
judge_llmcheck is itself an LLM call —check.usagereports the tokens it spent. Because of that, a session may ask a bounded number of times; past the ceiling the call raisesImQuotaErrorand the enforced policy carries on unaffected. The payload is bounded too — at most 20 texts of 20,000 characters each per ask; the policy enforced on the call itself has no such limit.
An alias with no policy raises rather than answering allow: True: "nothing is
attached" and "nothing objected" are different facts. A policy that cannot be evaluated
raises ImGuardrailUnavailableError, exactly as an enforced call would.
What is never checked
Guardrails run on chat completions only. core.embedding.embed cannot select a
policy — the bypass is structural (the transport's embeddings call takes no policy
argument at all), not a setting someone forgot. The same holds for every non-LLM
capability: document_parse, web_search, web_crawler, data, and
http_capability are outside the guardrail seam entirely.
Don't route text through embedding to dodge a policy — the block is enforced
where the content reaches a provider, and an org that guards its chat alias will
see the gap in the execution trace.
Which policy applies
An alias is checked with one policy, worked out by the Portal before any call:
- The alias's own policy wins. An alias with a policy of its own is checked with exactly that one, and the policies on its connections are ignored for it. Clearing the alias's policy brings its connections' back.
- Otherwise, its connections' policies. An alias with no policy of its own inherits the policy of the connection that serves it.
- Several connections: all of them. An alias whose targets can be served by several connections (a fallback chain across two providers, or a model two connections price) is checked with the union of their policies, because the check runs before the platform knows which connection will answer. It is never weaker than the policy of whichever connection ends up serving the call. A check two connections declare identically runs once.
Some unions cannot be one policy: two checks with the same name but different rules, or a
mask and a redact on the same side. The Portal refuses to save a policy that would cause
one. If one arises anyway, every call on that alias raises ImGuardrailUnavailableError until
an admin resolves it: the alias fails closed rather than running part of a policy.
None of this changes your code. Handle the two refusals on every alias, guarded or not today: an admin can attach a policy to a connection your alias uses at any time, and it applies as soon as it is published, with no change to your agent.
Who answers an LLM judge check
A judge_llm check is answered by the organization's own guardrail judge: one of its own
LLM connections, which an admin marks for it. There is no platform judge, and an organization
is never judged by another organization's connection. A self-hosted connection (a LiteLLM in
front of the organization's own models) can be the judge, so an installation with no vendor
key anywhere can still run judge checks.
For the organization admin, the setup is four steps:
- Set up a judge. In Settings → Organization → Services, open the LLM connection to judge with and, on its Guardrails tab, turn on Use for guardrail checks. One connection per organization holds it; the models listed on its Models tab are the models it can judge with, and it keeps serving its own aliases. A judge carries no guardrail checks of its own.
- Use it in a policy. On a model alias, or on a connection, add an LLM judge check under Guardrails, say in plain words what to reject, and pick the judge model.
- See what it costs. Every judgement is priced at the judge connection's rates, and shows as Guardrail checks on Dashboard → Cost: beside the agents' spend, never added to it or to a budget. The organization can also show it on each run's execution page (per judged call, and as a run total), labelled as not counted.
- Read its verdicts. Blocks show on the run's timeline and at the top of its execution page; Dashboard → Enforcement counts every judgement across the organization.
An Azure OpenAI resource can be the judge: connect it as an OpenAI connection whose endpoint
is https://<resource>.openai.azure.com/openai, and list the resource's deployment names as its
models — Azure answers DeploymentNotFound for a model id that is not one of its deployments.
The Azure OpenAI connection type itself cannot be the judge.
For your agent: a judge's DENY is an ImGuardrailBlockedError like any other block, and an
organization with no judge (or a judge that is down) makes its judge checks raise
ImGuardrailUnavailableError. A judge's cost is never part of the cost your run reports.
Streamed calls
A policy runs on core.llm.stream exactly as on call. A check on your prompt refuses the
stream before any text arrives; a check on the reply can refuse it part-way, after some
deltas have already reached you, and then ImGuardrailBlockedError is raised from the
iteration (Streaming shows the pattern). Treat the text
received before the error as discarded.
For policy authors: a pattern check on a streamed reply looks at one chunk at a time, so a match that straddles two chunks is not caught. Where a pattern must never reach the agent, call without streaming on that alias.
Ordering, for policy authors
Within a phase, check order is load-bearing: a redact ahead of a block means
the block judges already-redacted text. A judge_llm check can only block — it
returns a verdict, not rewritten text.
One combination is refused outright: a mask and a redact on the same side.
The redaction would overwrite the tokens the mask has to restore, and the agent
would receive placeholders where its own data belongs. Masking the input while
redacting the output is fine — those are different sides.
Policies are authored in the Portal, not in the agent project.