Guardrails

An organization can attach a guardrail policy to a model alias in the Portal. The policy is a list of checks — regex, presidio entity detection, or an LLM judge — that run on the way to the provider (input) and on the way back (output). Each check blocks the call, redacts the offending text, or masks it and puts it back.

It can also attach one to an LLM connection: an alias with no policy of its own is then checked with the policies of the connections it can be served by (which policy applies). To your agent the two are the same thing — a policy on the alias it calls — and everything below applies to either.

The policy binds every agent granted that alias, with no opt-out. There is no SDK parameter to select, skip, or soften it: your agent cannot see which policy is attached, and passing anything guardrail-shaped to core.llm.call does nothing. Your only job as an agent author is to handle the two refusals below — though you can also ask a policy for a verdict before spending a call.

The two refusals

from im_sdk import ImGuardrailBlockedError, ImGuardrailUnavailableError
 
try:
    result = await core.llm.call(messages=[...], alias="gpt-mini")
except ImGuardrailBlockedError as exc:
    # A check refused the content. Do not retry the same text — it will refuse again.
    check = exc.context.get("check")    # which check fired, e.g. "pii-email"
    phase = exc.context.get("phase")    # "pre" (your prompt) or "post" (the reply)
except ImGuardrailUnavailableError:
    # No verdict could be reached, so the call was refused rather than sent unchecked.
    ...
ExceptionHTTPWhat happened
ImGuardrailBlockedError403A check judged the content and refused it. context carries check, phase, and policy.
ImGuardrailUnavailableError503A check could not reach a verdict — its backend is down, or the gateway holds a policy the plugin does not.

ImGuardrailBlockedError subclasses ImForbiddenError, so an existing except ImForbiddenError: already catches it; catch the precise class when you want to react to content rejection specifically — e.g. rephrasing a prompt, or telling the user which check objected.

A rewrite is not an error

A check whose action is redact or mask rewrites the text and lets the call through. Your agent gets an ordinary successful ResourceResult. The names of the guardrails that rewrote it are on the envelope:

result = await core.llm.call(messages=[...], alias="gpt-mini")
for check, phase, action in result.metadata.get("guardrails_redacted", []):
    ...  # e.g. ("pii-email", "pre", "mask")

Each entry is a (check, phase, action) triple: which check rewrote, on which side of the call (pre is your prompt, post is the reply), and what it did (redact or mask). The same fact is recorded on the execution trace, so you don't need to log it yourself — read it only when the agent's own logic should know its text was rewritten.

The triple is the field's settled shape — it was a (check, phase) pair before guardrails reached agents — so unpack all three.

redact loses the value; mask gives it back

A redact is one-way: the match is replaced with a placeholder, and neither the provider nor your agent ever sees the original again.

A mask is a round trip. On the way out the match becomes a numbered token — [[EMAIL_ADDRESS_1]], [[PERSON_2]] — so the model can still reason about it ("email the invoice to [[EMAIL_ADDRESS_1]]") without receiving the value. On the way back the originals are put in again, so the text your agent reads is its own. The same value always gets the same token, so a name mentioned three times is one person to the model.

You do not have to do anything to un-mask a reply, and you should not try: the restore happens in the gateway, after the provider and after the provider-side call log, which is the point. If a model quotes a token back at you verbatim, that is the one case where a masked value can surface — the restore resolves it like any other.

Asking before you call

core.guardrail() runs an alias's policy over text and returns the verdict, without calling the model:

verdict = await core.guardrail("Draft a reply to this: …", alias="gpt-4o-safe")
if not verdict.allow:
    print(verdict.reason)          # 'guardrail "tone": unsafe'
for check in verdict.checks:
    print(check.guardrail, check.allow, check.usage)

Pass a list of strings to check several at once, or check="tone" to run one named check instead of the whole policy. verdict.rewritten gives the texts as the policy's checks would leave them — redactions applied, masks shown as tokens — which is how you see what the provider would actually receive.

Three things worth knowing:

  • It needs no extra grant. Asking about an alias requires exactly what calling it requires, so a policy verdict is available wherever core.llm.call is.
  • It changes nothing. This is a question, not a configuration: the policy still runs in full on every real call, whatever a verdict said beforehand.
  • A judge check costs money. No model is called for you here, but a judge_llm check is itself an LLM call — check.usage reports the tokens it spent. Because of that, a session may ask a bounded number of times; past the ceiling the call raises ImQuotaError and the enforced policy carries on unaffected. The payload is bounded too — at most 20 texts of 20,000 characters each per ask; the policy enforced on the call itself has no such limit.

An alias with no policy raises rather than answering allow: True: "nothing is attached" and "nothing objected" are different facts. A policy that cannot be evaluated raises ImGuardrailUnavailableError, exactly as an enforced call would.

What is never checked

Guardrails run on chat completions only. core.embedding.embed cannot select a policy — the bypass is structural (the transport's embeddings call takes no policy argument at all), not a setting someone forgot. The same holds for every non-LLM capability: document_parse, web_search, web_crawler, data, and http_capability are outside the guardrail seam entirely.

Don't route text through embedding to dodge a policy — the block is enforced where the content reaches a provider, and an org that guards its chat alias will see the gap in the execution trace.

Which policy applies

An alias is checked with one policy, worked out by the Portal before any call:

  • The alias's own policy wins. An alias with a policy of its own is checked with exactly that one, and the policies on its connections are ignored for it. Clearing the alias's policy brings its connections' back.
  • Otherwise, its connections' policies. An alias with no policy of its own inherits the policy of the connection that serves it.
  • Several connections: all of them. An alias whose targets can be served by several connections (a fallback chain across two providers, or a model two connections price) is checked with the union of their policies, because the check runs before the platform knows which connection will answer. It is never weaker than the policy of whichever connection ends up serving the call. A check two connections declare identically runs once.

Some unions cannot be one policy: two checks with the same name but different rules, or a mask and a redact on the same side. The Portal refuses to save a policy that would cause one. If one arises anyway, every call on that alias raises ImGuardrailUnavailableError until an admin resolves it: the alias fails closed rather than running part of a policy.

None of this changes your code. Handle the two refusals on every alias, guarded or not today: an admin can attach a policy to a connection your alias uses at any time, and it applies as soon as it is published, with no change to your agent.

Who answers an LLM judge check

A judge_llm check is answered by the organization's own guardrail judge: one of its own LLM connections, which an admin marks for it. There is no platform judge, and an organization is never judged by another organization's connection. A self-hosted connection (a LiteLLM in front of the organization's own models) can be the judge, so an installation with no vendor key anywhere can still run judge checks.

For the organization admin, the setup is four steps:

  1. Set up a judge. In Settings → Organization → Services, open the LLM connection to judge with and, on its Guardrails tab, turn on Use for guardrail checks. One connection per organization holds it; the models listed on its Models tab are the models it can judge with, and it keeps serving its own aliases. A judge carries no guardrail checks of its own.
  2. Use it in a policy. On a model alias, or on a connection, add an LLM judge check under Guardrails, say in plain words what to reject, and pick the judge model.
  3. See what it costs. Every judgement is priced at the judge connection's rates, and shows as Guardrail checks on Dashboard → Cost: beside the agents' spend, never added to it or to a budget. The organization can also show it on each run's execution page (per judged call, and as a run total), labelled as not counted.
  4. Read its verdicts. Blocks show on the run's timeline and at the top of its execution page; Dashboard → Enforcement counts every judgement across the organization.

An Azure OpenAI resource can be the judge: connect it as an OpenAI connection whose endpoint is https://<resource>.openai.azure.com/openai, and list the resource's deployment names as its models — Azure answers DeploymentNotFound for a model id that is not one of its deployments. The Azure OpenAI connection type itself cannot be the judge.

For your agent: a judge's DENY is an ImGuardrailBlockedError like any other block, and an organization with no judge (or a judge that is down) makes its judge checks raise ImGuardrailUnavailableError. A judge's cost is never part of the cost your run reports.

Streamed calls

A policy runs on core.llm.stream exactly as on call. A check on your prompt refuses the stream before any text arrives; a check on the reply can refuse it part-way, after some deltas have already reached you, and then ImGuardrailBlockedError is raised from the iteration (Streaming shows the pattern). Treat the text received before the error as discarded.

For policy authors: a pattern check on a streamed reply looks at one chunk at a time, so a match that straddles two chunks is not caught. Where a pattern must never reach the agent, call without streaming on that alias.

Ordering, for policy authors

Within a phase, check order is load-bearing: a redact ahead of a block means the block judges already-redacted text. A judge_llm check can only block — it returns a verdict, not rewritten text.

One combination is refused outright: a mask and a redact on the same side. The redaction would overwrite the tokens the mask has to restore, and the agent would receive placeholders where its own data belongs. Masking the input while redacting the output is fine — those are different sides.

Policies are authored in the Portal, not in the agent project.