Capabilities
Every capability hangs off the ImCore client as a lazy namespace, and every
call resolves against a grant the agent was given in the Portal. Read what a
given agent may use with im-cli agent models (aliases) and im-cli agent services
(connected services) — don't guess names.
Two addressing styles:
- Model-addressed — pass
alias=. Resources:llm,embedding. - Service-addressed — pass
service=(the connected-service name). Resources:document_parse,web_search,web_crawler,web_fetch,data,object_store,sharepoint,vector,mcp,http_capability.
Every method returns a ResourceResult except
llm.stream (an async iterator of stream events) and llm.call_structured
(a parsed instance of your pydantic model). core.vars is not a resource
call at all — it reads the organization variables the agent was granted.
Most methods take an optional timeout (seconds). It bounds the call
end-to-end: the gateway enforces it and answers with a typed 504
(ImTimeoutError), and the SDK caps its own wait at the same value plus a
small grace so that typed error wins over a generic client-side timeout.
Without it, the SDK waits up to 925 s. For llm, embedding and web_fetch a
per-call timeout can only tighten the deadline — the gateway (or, for
web_fetch, the connected service) caps those calls at its own ceiling (900 s
by default for llm), so a larger value is silently capped, never extended.
http_capability, data, sharepoint, vector and mcp are the exceptions —
they have no timeout param; their budget comes from the connected service's
configuration.
core.llm
result = await core.llm.call(
messages=[{"role": "user", "content": "hello"}],
alias="gpt-mini",
max_tokens=512, # optional; see below
)
text = result.textcall(messages, *, alias, ...) — the optional keyword-only params are
max_tokens, system, temperature, top_p, top_k, stop_sequences,
tools, tool_choice, thinking, seed, reasoning_effort, timeout, and
provider_params. top_k is best-effort: the Anthropic family honors it;
providers without a top_k knob (OpenAI, Azure, Gemini) ignore it.
provider_params reaches provider-native sampling and format knobs the
normalized parameters do not name — OpenAI logprobs / top_logprobs /
frequency_penalty / presence_penalty / logit_bias / max_completion_tokens
/ parallel_tool_calls / verbosity, Gemini generationConfig (its sampling
and thinkingConfig keys), Mistral random_seed / safe_prompt, Ollama behind
LiteLLM num_ctx / num_predict / repeat_penalty and the like. It is an
allow-list: any other key refuses the call with ImValidationError
(VALIDATION_REQUEST) before anything is sent, naming the refused keys in
context["refused_provider_params"]. That includes every key that could carry
prompt or tool content (system, messages, contents, systemInstruction,
tools, …): a key like that would reach the model without passing the
organization's guardrails. It also includes the name of a typed parameter
(temperature, max_tokens, …): set that through the typed parameter instead.
top_k is the one key that is both; there the typed parameter wins.
result.output is an LlmOutput whose blocks are typed
content blocks (TextBlock / ToolUseBlock / ThinkingBlock); result.text
concatenates the text blocks only.
If the organization attached a guardrail policy to the alias, this call may be
refused with ImGuardrailBlockedError or ImGuardrailUnavailableError, or
succeed with rewritten content — see Guardrails. The policy
binds every agent granted the alias and there is no parameter to opt out.
Also on core.llm:
stream(...)— same params;awaitit, then iterate the stream events as the text arrives (see Streaming).call_structured(messages, response_model, *, alias, ...)— pass a pydanticBaseModeltype and get a parsed instance of it back (not aResourceResult).
core.embedding
result = await core.embedding.embed(["first text", "second text"], alias="text-embed")
vectors = result.output.vectors # list[list[float]]embed(input, *, alias, dimensions=None, timeout=None, provider_params=None) —
input is a string or list of strings (positional). Returns a ResourceResult
with an EmbeddingOutput (.vectors) and token usage. result.text is empty
for embeddings — read result.output.vectors. provider_params takes
output-shape knobs only (encoding_format, dimensions, input_type,
task_type, output_dimensionality, truncate); any other key refuses the call
as it does on core.llm.
Embeddings are never guardrail-checked: the call cannot select a policy at all (see Guardrails).
core.document_parse
Service-addressed. The document arrives as exactly one of file (the
content itself, Base64-encoded) or file_ref (the id of a file uploaded to
the Portal — workspace files). With file_ref the gateway fetches the bytes
from platform storage server-side, scoped to the calling execution's
workspace, so the document never enters the agent process; an unknown,
expired, or foreign ref fails the call with the typed file_not_found error.
import base64
# Inline content:
payload = base64.b64encode(open("/tmp/report.pdf", "rb").read()).decode()
result = await core.document_parse.partition(service="unstructured", file=payload)
# Portal upload (id from the run input, e.g. data["files"][0]):
result = await core.document_parse.partition(service="unstructured", file_ref=file_id)
elements = result.output.elements # DocumentOutput.elements
text = result.text # newline-joined element textpartition(service, file=None, *, file_ref=None, filename=None, strategy=None, chunking_strategy=None, combine_under_n_chars=None, max_characters=None, extract_image_block_types=None, timeout=None)extract(service, file=None, *, file_ref=None, ...)— richer element metadata (defaults tohi_res).ocr(service, file=None, *, file_ref=None, filename=None, languages=None, timeout=None)— forces OCR.
core.web_search
Service-addressed.
result = await core.web_search.search(service="tavily", query="in·manas platform")
answer = result.text # SearchOutput.answer
hits = result.citations # the search resultssearch(service, query, *, search_depth=None, max_results=None, include_domains=None, exclude_domains=None, timeout=None, provider_params=None)extract(service, urls, *, extract_depth=None, timeout=None, provider_params=None)— returns anExtractOutputwith.pages.
core.web_crawler
Service-addressed.
run_actor(service, actor_id, *, input=None, timeout=None)→CrawlOutput(.items,.run).fetch_dataset(service, dataset_id, *, limit=None, offset=None).
core.data
Service-addressed table CRUD against a database your organization connected,
through one of two providers: DreamFactory or
Microsoft Dataverse. One connected service is one database (for Dataverse, one
environment): which database — and, optionally, which tables — an agent may
reach is fixed in the service config by an admin, so service names the
connection, never a provider. filter, fields and order are written in
the connected service's own syntax and passed through — OData for Dataverse.
await core.data.create("customer-db", "agent_memory", [{"key": "run:1", "value": '{"step": "a"}'}])
result = await core.data.read("customer-db", "agent_memory", filter="key like 'run:%'", limit=50)
rows = result.records # == result.output.recordscreate(service, table, records, *, fields=None, atomic=True)—recordsis a list of row dicts. Returns the inserted rows: ids only unlessfieldsasks for more (fields="*").read(service, table, *, filter=None, ids=None, fields=None, related=None, order=None, limit=None, offset=None, include_count=None)—relatedandoffsetare DreamFactory only.update(service, table, records, *, ids=None, filter=None, atomic=True)— address rows byids, or, on DreamFactory only, byfilteror by a per-recordidinside each element ofrecords. One record plusids/filterapplies the same patch to every matched row.delete(service, table, *, ids=None, filter=None, atomic=True)—idsorfilteris required (filteron DreamFactory only); an unscoped delete would mean "every row", so it raisesImValidationErrorbefore leaving your process.
On Dataverse, update and delete address rows by ids alone, at most 100 per
call (the same cap applies to read by ids); related, offset and a
filter on a write are refused before any request.
Four things to know before you write against it:
- Serialize JSON column values yourself (DreamFactory). A nested
dictis rejected even by ajsonbcolumn; passjson.dumps(...). Reads come back the same way — a JSON column is a JSON-encoded string, not a nested object. - Timestamps are not ISO-
T(DreamFactory). They arrive as"2026-08-03 10:08:55.926853+00"(a space), so they are notdatetime.fromisoformat-safe on older Pythons. Treat row values as opaque and parse deliberately. - Paging is explicit. Pass
limit(andoffseton DreamFactory); nothing auto-paginates, so one call can't pull a whole table.result.output.countis the total match count but only when you passinclude_count=True— otherwise it staysNone(absent, not zero). On DreamFactory the next-page offset is atresult.raw["meta"]["next"]; Dataverse has no offset, so order by a column and filter past the last value read. - Multi-row writes are all-or-nothing. That's the default and it's what you
want: without it a 10-row create failing on row 7 leaves 6 rows committed and
a retry duplicates them.
atomic=Falseopts into that native behaviour. On Dataverse a multi-row write is one$batchchangeset.
Writes return the rows they touched, so len(result.records) is a usable
affected-row count on update and delete. Setup, permissions and each
provider's limits: docs/dreamfactory.md and docs/dataverse.md in the
platform repository.
core.file_store
Reads a file a user uploaded in the Portal, by id — the same opaque id
document_parse accepts as file_ref.
file_id = inputs["files"][0] # conventional run-input key
result = await core.file_store.get(file_id)
rows = result.output.text # decoded when the object is UTF-8 text
raw = result.output.content_b64 # Base64 bytes for anything elseget(file_ref)→FileOutput(.content_b64,.text,.filename,.mime_type,.size_bytes).
No service, and no grant. Uploads live on the platform's own buffer disk, so
there is no third party to name and nothing to authorize against: the gateway
resolves the ref against your execution's workspace, which is the whole
tenancy story. An unknown, expired or foreign ref all fail the same way
(ImValidationError, file_not_found) — deliberately indistinguishable, so an
error cannot tell you a file exists somewhere you cannot reach.
The bytes arrive Base64 in the result envelope, bounded by the object-size ceiling — nothing is written into the agent's working directory.
An upload expires (3 days). To keep one, write it to a data bucket with
core.object_store.put_from_upload— one call, and the bytes never pass through your process. The whole story, from theio.jsonfile input to the bucket, is on Files & storage.
core.object_store
Service-addressed. Objects in a bucket the organization connected — the durable
counterpart to file_store, addressed by key instead of by id.
# Keep an upload past its TTL: the gateway copies the bytes buffer → bucket,
# server-side. You send two strings.
stored = await core.object_store.put_from_upload("archiv", file_ref=file_id)
# Write something you produced yourself.
await core.object_store.put("archiv", key="berichte/q3.csv", content_b64=csv_b64)
# Read it back, in this run or any later one.
text = (await core.object_store.get("archiv", "berichte/q3.csv")).output.text
# Enumerate; next_cursor is None on the last page.
page = await core.object_store.list("archiv", prefix="berichte/")
keys = [o.key for o in page.output.objects]| Method | Output variant | Notes |
|---|---|---|
get(service, key) | ObjectOutput | .content_b64, .text, .content_type, .size_bytes |
stat(service, key) | ObjectStatOutput | metadata only — the cheap existence/size check |
list(service, prefix=…, limit=…, cursor=…) | ObjectListOutput | .objects, .next_cursor |
put(service, key=…, content_b64=…) | ObjectWriteOutput | create-only — an existing key raises object_exists |
replace(service, key=…, content_b64=…) | ObjectWriteOutput | writes whether or not the key exists |
put_from_upload(service, file_ref=…, key=…) | ObjectWriteOutput | buffer → bucket, server-side; create-only |
delete(service, key) | ObjectDeleteOutput | one key; .deleted is False when there was nothing there |
Four things worth knowing before you write against it:
- Every operation is granted separately.
putandreplaceare different permissions on purpose — an agent granted onlyputcan add objects and can never destroy one. Expect to be told which ones you have; a missing grant is a policy refusal, not a bug in your code. - Keys are relative to the service. Each connected service owns a prefix in
its bucket and the gateway adds and strips it, so the key you send is the key
you get back.
.., a leading/, empty segments, control characters, unpaired surrogates, a key too long once that prefix is added, and the reserved.im-namespace are refused before the store is reached. deletetakes exactly one key. There is no prefix or bulk delete.- Another service's object reads as missing (
object_not_found), same as one that never existed.
Written up end to end — key rules, what list does and does not return, the
per-operation grants and the size cap — on
Files & storage. What an admin configures and how the
bucket is backed up: docs/agent-data-bucket.md in the platform repository.
core.sharepoint
Service-addressed. Files in one SharePoint Online document library and the items
of SharePoint lists, in a site the organization already owns. AI OS reaches it
through Microsoft Graph with the organization's own Entra app; it never acts as
a person. One connected service is one site, one library and an optional folder
inside it. core.sharepoint needs an im-sdk later than 1.2.0.
import base64
# Read a file; .text decodes UTF-8, .content_b64 holds the raw bytes.
offer = await core.sharepoint.get("vertrieb", "Angebote/2026/offer.pdf")
# Create a file. put refuses to overwrite; replace is the one that does.
csv_b64 = base64.b64encode(report.encode()).decode()
await core.sharepoint.put("vertrieb", path="Berichte/q3.csv", content_b64=csv_b64)
# Enumerate a folder; next_cursor is None on the last page.
page = await core.sharepoint.list("vertrieb", path="Berichte")
paths = [o.key for o in page.output.objects]
# Lists: fields are keyed by each column's internal name.
rows = await core.sharepoint.list_items("vertrieb", "Kontakte")
created = await core.sharepoint.create_item(
"vertrieb", "Kontakte", {"Title": "Muster GmbH", "Unit_x0020_Price": 120}
)
item_id = created.output.item.id| Method | Output variant | Notes |
|---|---|---|
get(service, path) | ObjectOutput | .content_b64, .text, .content_type, .size_bytes; at most 26 MiB |
stat(service, path) | ObjectStatOutput | metadata only; a folder is refused |
list(service, path=…, limit=…, cursor=…) | ObjectListOutput | .objects, .next_cursor; folders end in /; limit defaults to 100, capped at 200 |
put(service, path=…, content_b64=…, content_type=…) | ObjectWriteOutput | create-only — an existing file raises object_exists |
replace(service, path=…, content_b64=…, content_type=…) | ObjectWriteOutput | writes whether or not the file exists |
delete(service, path) | ObjectDeleteOutput | one file, to the site's recycle bin; .deleted is False when there was nothing there |
put_from_upload(service, file_ref=…, path=…) | ObjectWriteOutput | Portal upload → library; create-only; without path it lands at uploads/<file_ref>-<filename> |
list_items(service, list, limit=…, cursor=…) | ListItemsOutput | .items (each .id, .fields, .etag, .last_modified), .next_cursor |
get_item(service, list, item_id) | ListItemOutput | .item; a missing item raises item_not_found |
create_item(service, list, fields) | ListItemOutput | fields must not be empty |
update_item(service, list, item_id, fields) | ListItemOutput | patches the columns you send; the rest keep their values |
delete_item(service, list, item_id) | ListItemDeleteOutput | .deleted is False when the item was not there |
The file methods answer with the same output variants as
core.object_store, with .key holding the path, so code
written against a bucket reads the same against a library.
Paths are relative to the service folder and cannot leave it. The connected
service fixes the site, the library and the folder; the path you send is the
path you get back. A path is refused before any request reaches Microsoft when
it has an empty, . or .. segment, any of " * : < > ? \ | or a control
character, a segment ending in . or a space, or is longer than 400 characters
together with the folder. A leading or trailing / is ignored.
put and replace are separate grants. Every one of the twelve methods is
granted on its own, and an agent granted only put can add files but never
overwrite one. A method the agent was not granted is a policy refusal
(POLICY_ACTION_NOT_GRANTED) before any Microsoft request. The site's own grant
on the Microsoft side is the ceiling: an agent granted put on a site where the
app holds only read gets a 403 from Graph.
Only the lists in allowed_lists are reachable, and an empty list means
none. A list outside it is refused with list_not_allowed before any
Microsoft request. Lists are named exactly as SharePoint displays them; a list
that is allowed but missing on the site raises list_not_found.
Columns are addressed by their internal name, in the fields you send and
the fields you read. The internal name is fixed when a column is created and
does not follow a later rename; spaces and umlauts are encoded (Unit Price is
Unit_x0020_Price), and the title column is Title. Read one item with
get_item to see the names a list uses.
Pass next_cursor back unchanged as cursor, to the same folder or list it
came from: a cursor that does not continue the listing you name, or that holds a
. or .. segment, is refused with invalid_cursor. What an admin sets up on the
Microsoft tenant, the connection form and troubleshooting:
docs/sharepoint.md in the platform repository.
core.vector
Service-addressed. Embeddings in a vector database the organization connected
(today via Qdrant) — the end of the retrieval pipeline
document_parse and embedding start. One connected service is one
collection, created when an admin saves the service; service names the
connection, and nothing you pass can widen it.
# Store — re-using an id overwrites that point, so a re-ingest is idempotent.
await core.vector.upsert("handbuch", points=[
{"id": i, "vector": v, "payload": {"source": "handbuch.pdf", "chunk": i}}
for i, v in enumerate(vectors)
])
# Search — best-first hits, with the payload you stored.
result = await core.vector.search("handbuch", vector=query_vector, top_k=5)
for hit in result.output.hits:
hit.id, hit.score, hit.payload
# Delete — by id, or by filter (how a re-ingest replaces a document).
await core.vector.delete("handbuch", filter={
"must": [{"key": "source", "match": "handbuch.pdf"}],
})| Method | Output variant | Notes |
|---|---|---|
upsert(service, *, points) | VectorUpsertOutput | .upserted; max 1000 points per call, an oversized batch is refused rather than partly written |
search(service, *, vector, top_k=None, filter=None, with_payload=None) | VectorSearchOutput | .hits (.id, .score, .payload), .count; top_k defaults to 10, ceiling 250 |
delete(service, *, ids=None, filter=None) | VectorDeleteOutput | .deleted — the ids you passed, or None for a filter-delete; max 1000 ids |
Four things to know before you write against it:
- The vector dimension is the admin's setting. It is fixed when the service is connected, because a collection cannot be resized. A vector of another width is refused with both numbers in the message, which almost always means the embedding model is not the one this service was created for. And embed the query with the same model that embedded the documents — nothing can check that, and a mismatch returns confident nonsense rather than an error.
- Each operation is granted separately.
searchalone is a retrieval assistant that cannot alter the corpus;upsertalone is an ingest job. A missing grant isImForbiddenError— policy, not a bug in your code. filteris a small vendor-neutral dialect:{"must": [{"key": …, "match": …}]}— exact matches on payload keys, AND-ed. Qdrant'sshould/must_not/ ranges are not exposed, and anything outside the subset is refused, not ignored.deleteneedsidsorfilter, exactly one. Neither is refused rather than read as "everything", both is refused rather than guessed at — locally, before anything is sent, likedata.delete.
result.text is empty for every vector call — read result.output. Query
vectors, payloads, filter values and point ids never reach trace exports, and no
setting opts into that.
Written up end to end — the pipeline, the filter dialect, re-ingest, the typed
errors — on Vector store & RAG. What an admin configures:
docs/vector-store.md in the platform repository.
core.web_fetch
Service-addressed. The bytes at a URL you choose — the one resource where
the agent names the destination rather than an administrator. service names a
fetcher connection whose configuration is the policy: which content types and
ports are permissible, whether internal hosts are reachable at all, whether
robots.txt is honoured, and the byte and time ceilings a per-call value may
only tighten.
# A feed, a page, a JSON document — decoded for you.
feed = await core.web_fetch.fetch("web", "https://example.org/feed.xml")
entries = parse(feed.text)
# Anything binary: Base64, which is what document_parse takes.
pdf = await core.web_fetch.fetch("web", url, as_text=False)
elements = (await core.document_parse.partition(
"unstructured", file=pdf.output.content_base64,
)).output.elements
# "Is it still there, and how big?" — a HEAD, not a download.
check = await core.web_fetch.probe("web", url)
if check.output.reachable and (check.output.content_length or 0) < 5_000_000:
...| Method | Output variant | Notes |
|---|---|---|
fetch(service, url, *, as_text=None, max_bytes=None, timeout=None, accept=None, headers=None) | FetchOutput | .content_text or .content_base64 (never both), .status, .final_url, .redirect_chain, .content_type, .charset, .size_bytes, .truncated, .not_modified, .etag, .last_modified. result.text reads either |
probe(service, url, *, follow_redirects=None, timeout=None) | ProbeOutput | .reachable, .status, .final_url, .redirect_chain, .content_type, .content_length, .elapsed_ms; result.text is empty |
Five things to know before you write against it:
- Internal addresses are unreachable, and no argument changes that. Loopback, private, CGNAT and link-local ranges — the cloud metadata endpoint included — are refused after resolution and checked on the resolved address, so a hostname that points inward does not get through either. An admin can re-enable named hosts per service; an agent cannot.
as_textdecides text or bytes, and is refused for the wrong media type. It defaults to true and Base64-ing a PDF into a text field is exactly what that refusal prevents. Passas_text=Falsefor anything binary and read.output.content_base64.- Refusals are typed and distinguishable. A disallowed content type, a
blocked port or address, and a
robots.txtthat says no areImForbiddenErrorand naming what was received and what is allowed — none of them is worth a retry, and the remedy is a different URL or a conversation about the service's configuration.ImTimeoutErrorandImResourceErrorare the retryable ones. probeanswers, it does not raise. A 404 isreachable=Falsewithstatus=404; nothing answering at all leavesstatusasNone. The address and scheme rules still apply, so an internal target is still a refusal — "not permitted" must not read the same as "not there".- Conditional requests are supported and cheap. Keep
.output.etagand send it back asheaders={"if-none-match": etag}; an unchanged resource answersnot_modified=Truewith no body at all.headerstakes onlyif-none-match/if-modified-since/accept/accept-language, and anything else —AuthorizationandCookieabove all — is refused locally. The fetcher forwards no credentials, ever.
Fetched bodies and full URLs never reach trace exports, and no setting opts into
that: what the trace keeps is the host and the structural fields (status,
content_type, size_bytes, truncated, reachable, elapsed_ms).
Written up end to end — the two ceilings, conditional requests, the probe's
verdicts, the typed refusals — on Fetching a URL. What an admin
configures, and the thirteen security rules behind it: docs/web-fetch.md in the
platform repository.
core.mcp
Service-addressed. The tools on a Model Context Protocol
server an organization connected — service is the connection's name, and the
grant unit is the tool: an agent may list and call only the tools it was
granted on that connection.
tools = await core.mcp.list_tools("acme-tools") # only what this agent was granted
print(tools.output.names) # ["search_docs", "files.read"]
result = await core.mcp.call_tool("acme-tools", "search_docs", {"q": "refund policy"})
if result.output.is_error: # the tool ran and reported failure: an answer, not an exception
...
print(result.output.text) # the tool's text; its images are in result.output.images| Method | Output variant | Notes |
|---|---|---|
list_tools(service) | McpToolsOutput | .tools — a list of McpTool (name, description, input_schema), in the order the platform lists them; .names is just the names. result.text is the names, one per line |
call_tool(service, tool, arguments=None) | McpToolResultOutput | .tool, .text (the tool's text, merged into one string), .images (a list of McpImage: data is base64, mime_type), .is_error. result.text reads the same as .text |
Things to know before you write against it:
- Nothing is granted unless it is named.
list_toolsreturns the server's tools narrowed to the ones this agent was granted — a connection ticked with no tools lists none. An administrator grants tools per agent in the Portal, one tool at a time; granting "everything" means naming everything. - Tool names are never normalized. They are case-sensitive and may contain
-and.(Delete_Fileis notdelete_file, andfiles.readis a name). Whatlist_toolsreturned is whatcall_toolsends. Only the helpers below rename anything, and only on the way to a model. - A tool that failed is an answer. The server ran it and said it failed: you get a
normal result with
output.is_errorset, and the Portal bills it like any other call. Checkis_errorbefore you trustoutput.text— the text is then the tool's own error message. - A result is text, images and
is_error, and nothing else. The platform runs the call through a tool router that hands back one flattened answer, so the SDK does not pretend to more than that: see Known limits below. An empty result istext == ""andimages == []. argumentsis adictorNone.call_toolraisesImValidationErrorlocally, with no request, for an empty or non-stringtoolor for arguments that are not adict.- There is no
timeoutargument. One call's budget is the connection'stimeout(120 s by default, 270 s at most); on expiry the call raisesImTimeoutErrorand is not retried, because the tool may have run. - MCP tools are never injected into
core.llmcalls. You decide what the model sees and you run what it asks for — the helpers below make that short. - The execution trace never shows what you sent or got back. Each MCP call is one row on the
run's Timeline in the Portal — the tool's name, the connection, the status, what ended it, the
latency and the cost — and the arguments and the result are in none of it. The SDK's trace event
for an MCP call carries no error text either, because a failed call's message can quote the value
it rejected; the typed
exc.context["kind"]below is how the Portal tells what happened.
Errors
Every failure is one of the SDK's existing exceptions, with the MCP condition in
exc.context["kind"]. A call_tool is billed only when the tool answered — a result with
is_error set included, which is why it is returned, not raised — so none of the kinds below is
billed. A list_tools is never billed: it reads the platform's copy of the list and never
reaches the server.
exc.context["kind"] | Raised as | exc.code | Billed? | What it means |
|---|---|---|---|---|
tool_not_granted | ImForbiddenError | POLICY_ACTION_NOT_GRANTED | No | The tool is not callable under this agent's grant. Nothing was sent to the server. exc.context["reason"] says why: absent — the grant does not name the tool (add it in the Portal; the message lists what is granted); grant_lag — the grant names it and was changed moments ago, the platform's tool router has not caught up (try again in a minute); grant_mismatch — this session's grant names it, but the platform no longer admits it for the organization (revoked or changed after the session began, or not this organization's connection): start a new session and check the grants; access_blocked — the organization is suspended |
tool_name_collision | ImForbiddenError | POLICY_FORBIDDEN | No | The server lists another tool whose name folds onto this one (case, or - against _), so the platform cannot tell which is meant and refuses the call. Nothing ran. The server has to rename one of them |
virtual_key_missing | ImResourceError | RESOURCE_MISCONFIGURED | No | The organization's key to the platform's tool router is unknown, so the call was not sent. Not something the agent can fix: an operator converges the key (bifrost:converge --now) |
tool_unavailable | ImResourceError | RESOURCE_ERROR | No | The grant is fine, but Bifrost does not hold that tool for this key: the server no longer lists a tool by that exact name. Arguments the server refuses are a protocol_error |
auth_failed | ImResourceError | RESOURCE_ERROR | No | The server answered 401/403, or the connection needs to be authorized again: the connection's credential was refused |
protocol_error | ImResourceError | RESOURCE_ERROR | No | The server refused the arguments, or the call failed in a way the platform cannot classify further; the tool may have run |
result_unreadable | ImResourceError | RESOURCE_ERROR | No | The tool ran, but its result has a content type the platform cannot read (see Known limits). Do not retry blindly: the tool's effect happened |
response_too_large | ImResourceError | RESOURCE_ERROR | No | The answer was over the platform's size cap; the tool ran |
unreachable | ImResourceError | RESOURCE_UPSTREAM_FAILED | No | The platform could not connect to the server; nothing ran |
aborted | ImResourceError | RESOURCE_UPSTREAM_FAILED | No | The connection broke mid-call; the tool may have run |
timeout | ImTimeoutError | RESOURCE_TIMEOUT | No | No answer in time — the tool may have run, and the call is not retried |
connect_timeout | ImTimeoutError (no connection in time) or ImResourceError (could not connect) | RESOURCE_TIMEOUT or RESOURCE_UPSTREAM_FAILED | No | The call never left the gateway: the platform's tool router did not accept the connection. Nothing ran |
tool_not_granted and tool_unavailable are the two to tell apart: the first is the agent's
grant (fix it in the Portal), the second is the server (the tool was renamed or removed, or
the arguments were wrong). ImForbiddenError and ImResourceError are what you already catch
for every other resource; branch on exc.context["kind"] when you need more.
Known limits of a result
The platform runs MCP calls through a tool router (Bifrost) that returns one flattened string per result. What it cannot give back, the SDK does not claim:
- Text blocks arrive merged, as one string with no separator, so where one block ends and the
next begins is lost — and so is the order of text relative to images.
.textand.imagesare two separate lists, not a sequence. - Only text and images are carried. Audio is dropped (a result of audio only is an empty
result, not an error). A resource link or an embedded resource arrives as the text the router
wrote for it — the link as its JSON, an embedded resource as the literal
[Embedded Resource Response: resource], so its content is lost: a tool that returns a file as an embedded resource (GitHub'sget_file_contents, for one) gives you only the placeholder. This lifts with the platform's next router upgrade that includes the upstream fix. A content type the router does not read at all, or an image without data or a media type, fails the call asresult_unreadable. - Structured content and
_metaare not returned. A tool that also writes its structured result as text gives you that text; one that returns only structured content gives you an empty result. - A tool's text that looks exactly like the router's image marker is read as an image. The router
writes an image as
[Image Response: <base64>, MIME: <type>], and a tool that writes that very string as text cannot be told apart. The effect is a wrong entry in.images(and the text missing from.text), never more than the tool could already put in front of your model. - A listed tool is
name,descriptionandinput_schema. Titles, output schemas and annotations are not carried. - Strict mode is off. The router drops
additionalProperties: falsefrom a tool's schema, so nostrictflag is set on a definition, andcore.llm.callsends none; the server validates the arguments.
Giving the tools to a model
list_tools returns McpTool entries. One free function takes them from there, as plain dicts
(the SDK imports no provider library): mcp_tools(tools, *, prefix=None) returns
(definitions, name_map).
Pass the definitions to core.llm.call, whatever the model behind the alias. Its tools=
takes {name, description, input_schema}, which is their shape, and the gateway translates it
for the provider (the loop below). The name map takes
the names the model calls back to the server's, for call_tool.
| Returned | What it is |
|---|---|
definitions | [{"name", "description", "input_schema"}] — what core.llm.call(tools=…) takes, whatever the model |
name_map | {provider name: the server's exact tool name}, keyed by the definitions' names in the same order — the way back from the name a model called. Indexing a name the model invented is a KeyError |
tools is what list_tools returned: the result itself, or its .output. Anything else, a list
of McpTool or a raw tools/list dict included, is a TypeError. mcp_tools is importable
from im_sdk.
input_schemapasses through unchanged. Providers insist on an object schema, so two cases are adjusted. A schema withpropertiesbut notypeis kept and gets"type": "object"added. A schema that is empty, is declared with anothertype(or has neithertypenorproperties) is replaced by{"type": "object", "properties": {}}, and that tool can still be called with no arguments.- No strict schema. A definition carries no
strictflag andcore.llm.callsends none. The platform's tool router dropsadditionalProperties: falsefrom the schemas it lists, so a schema that reaches you cannot be shown to meet a provider's strict mode. The server validates the arguments the model sends. - The description is the tool's
description(""when the server gave none). - Names are converted, because the two sides disagree. A provider's tool name is
[a-zA-Z0-9_-]and at most 64 characters; an MCP name may contain.and be longer. A name that is already valid is never touched (with aprefix, it is the prefixed spelling that has to be valid). Every other character becomes_; a name that collides with another, or is over 64 characters, gets a short stable hash of the original appended. The mapping depends only on the tool list, never on its order, but a different list can name a colliding tool differently. That is why one call returns both: hold the map next to the definitions it came with, and never rebuild it from a later listing. prefix(for example the connection name) goes in front, joined by__, before anything is sanitized:prefix="acme-tools"turnssearch_docsintoacme-tools__search_docs. Use it when one agent offers tools from more than one connection, so equal tool names cannot collide.
A complete tool loop, on the platform
A deployed agent has no outbound network and no provider keys, so the loop that works everywhere
runs the model through core.llm: the call is resolved against a model alias, billed, and
guardrailed like every other. The definitions mcp_tools returns have exactly the {name, description, input_schema} shape tools= validates as a ToolDefinition (the gateway translates it for
whichever provider the alias routes to), and the model's calls come back as tool_use blocks.
The conversation travels in the Chat Completions shape: the assistant turn keeps its
tool_calls, and each result is a role: "tool" message carrying the call's tool_call_id.
import json
from im_sdk import ImCore, ImError, mcp_tools
SERVICE = "acme-tools"
ALIAS = "gpt-mini" # a model alias this agent was granted
async def run_tool(core: ImCore, names: dict[str, str], use) -> str:
"""What the model reads back for one tool call. It never raises: a failure is an answer."""
name = names.get(use.name)
if name is None: # a model can invent a tool; that is not a call to forward
return f"There is no tool called {use.name}."
try:
result = await core.mcp.call_tool(SERVICE, name, use.input)
except ImError as exc: # not granted, unreachable, timed out, ...
return f"The tool could not be run: {exc}"
text = result.output.text
return f"The tool reported an error: {text}" if result.output.is_error else text
async def main(core: ImCore, data: dict) -> str:
listing = await core.mcp.list_tools(SERVICE)
defs, names = mcp_tools(listing, prefix=SERVICE) # names: provider name -> the server's exact name
messages = [{"role": "user", "content": data["prompt"]}]
for _ in range(8): # bound the loop
reply = await core.llm.call(messages, alias=ALIAS, max_tokens=1024, tools=defs)
uses = [b for b in reply.output.blocks if b.type == "tool_use"]
if not uses:
return reply.text
messages.append({
"role": "assistant",
"content": reply.text,
"tool_calls": [
{"id": u.id, "type": "function",
"function": {"name": u.name, "arguments": json.dumps(u.input)}}
for u in uses
],
})
for use in uses:
messages.append({
"role": "tool", "tool_call_id": use.id,
"content": await run_tool(core, names, use),
})
return "Stopped: too many tool rounds."The model only ever sees acme-tools__files_read; names is how that becomes files.read again.
run_tool gives the model output.text only. A tool that also returns images has them in
output.images (base64 data and a mime_type); pass them on only if your model accepts images.
With your own client
The definitions are also the Anthropic Messages tools= shape, so with Anthropic's own client
they go in as they are, and the calls arrive as tool_use content blocks whose input is
already a dict:
from anthropic import AsyncAnthropic
from im_sdk import ImCore, ImError, mcp_tools
SERVICE = "acme-tools"
async def run_tool(core: ImCore, names: dict[str, str], use) -> dict:
"""The `tool_result` block for one `tool_use`."""
block = {"type": "tool_result", "tool_use_id": use.id, "is_error": True}
name = names.get(use.name)
if name is None:
return {**block, "content": f"There is no tool called {use.name}."}
try:
result = await core.mcp.call_tool(SERVICE, name, use.input)
except ImError as exc: # not granted, unreachable, timed out, ...
return {**block, "content": f"The tool could not be run: {exc}"}
return {**block, "content": result.output.text, "is_error": result.output.is_error}
async def main(core: ImCore, data: dict) -> str:
listing = await core.mcp.list_tools(SERVICE)
defs, names = mcp_tools(listing, prefix=SERVICE)
client = AsyncAnthropic()
messages = [{"role": "user", "content": data["prompt"]}]
for _ in range(8):
reply = await client.messages.create(
model="claude-sonnet-5-5", max_tokens=1024, messages=messages, tools=defs,
)
uses = [b for b in reply.content if b.type == "tool_use"]
if not uses:
return "".join(b.text for b in reply.content if b.type == "text")
messages.append({"role": "assistant", "content": reply.content})
messages.append({
"role": "user",
"content": [await run_tool(core, names, use) for use in uses],
})
return "Stopped: too many tool rounds."What an administrator configures — connecting a server, discovery, per-tool grants, the audit and
billing rules: docs/mcp.md in the platform repository.
core.http_capability
Service-addressed escape hatch for a connected HTTP service the Portal exposes.
result = await core.http_capability.request(service="my-api", path="/v1/things", verb="get")
body = result.output.body # RawOutput.bodyrequest(service, path, verb="post", body=None, headers=None, query_params=None)
— verb is one of get / post / put / patch / delete.
body is sent as JSON. For an upstream that expects multipart/form-data, set that
Content-Type header: the gateway then sends one part per top-level key of body — a string as a
plain form field, anything else as an application/json part — and writes the boundary itself.
result = await core.http_capability.request(
service="eu-funding-search",
path="/search-api/prod/rest/search",
verb="post",
headers={"Content-Type": "multipart/form-data"},
body={"query": {"bool": {"must": [...]}}, "languages": ["en"]},
query_params={"apiKey": "SEDIA", "text": "***"},
)core.vars
The organization variables granted to this agent, fixed when the client authorized — not a
resource call, no ResourceResult, no gateway round trip.
base_url = core.vars["SHOP_URL"] # KeyError: not granted or unknown
region = core.vars.get("REGION", "eu-west-1")
everything = core.vars.all() # dict copy
pages = core.vars.get_int("MAX_PAGES", 10) # int — 10 when not granted
rate = core.vars.get_float("SAMPLE_RATE") # float
dry_run = core.vars.get_bool("DRY_RUN", False) # bool — true/false, 1/0, yes/no, on/off
config = core.vars.get_json("SCRAPER_CONFIG") # whatever the JSON holdsValues are stored as strings, exactly as entered, and core.vars["KEY"] returns the string. The
typed reads parse it: a variable that is not granted returns default when you pass one, else the
same KeyError; a value that does not parse raises ValueError naming the key and the expected
type, never the value. core.vars needs im-sdk 1.2.0 or newer.
Secrets are never readable here. Use one by writing core.secret("KEY") where the value belongs
in a call to a connected service: it returns a SecretRef placeholder carrying this session's
nonce, which the gateway substitutes on the way out — only where the connection pins the
destination — and masks out of that call's answer (im-sdk 1.2.0 or newer). Setup, grants, the rules and the error codes:
Variables & secrets.
The ResourceResult envelope
Every call returns one shape:
| Attribute | What it is |
|---|---|
result.text | The text answer (delegates to the active output variant). |
result.output | The typed answer — LlmOutput, EmbeddingOutput, SearchOutput, ExtractOutput, DocumentOutput, CrawlOutput, RecordsOutput, RawOutput, one of the five Object*Outputs, one of the three Vector*Outputs, FetchOutput, ProbeOutput, McpToolsOutput or McpToolResultOutput. Narrow with isinstance, or switch on output.kind. |
result.citations | Search hits, or None. |
result.records | DB rows from a data call, or None. |
result.usage / result.usage_unit | Metering (tokens / pages / requests). |
result.cost / result.cost_currency | The provider-reported cost, for self-pricing drivers only (e.g. Apify). None for LLM and metered non-LLM calls — see below. |
result.model | The resolved backend model (model-addressed resources). |
result.provider / result.connected_service_id | Which provider / connected service served it. |
result.metadata / result.raw | Extra metadata and the full provider response. |
The output variant types are importable from the package root, e.g.
from im_sdk import LlmOutput, SearchOutput, DocumentOutput.
Advanced LLM usage
The tools, tool_choice, and thinking shapes are importable from
im_sdk.models.
Typed (structured) output
call_structured derives a JSON schema from a pydantic model, asks the model to
fill it, and returns a parsed instance of that model (not a ResourceResult):
from pydantic import BaseModel
class Sentiment(BaseModel):
label: str
score: float
sentiment = await core.llm.call_structured(
messages=[{"role": "user", "content": "I love this."}],
response_model=Sentiment,
alias="gpt-mini",
)
sentiment.label # -> "positive"Tool / function calling
Pass tools (each ToolDefinition is name + optional description +
input_schema) and, optionally, a ToolChoice (type is auto / any /
tool / none, with name when forcing a specific tool). Tool calls come back
as ToolUseBlocks in result.output.blocks:
from im_sdk.models import ToolDefinition
result = await core.llm.call(
messages=[{"role": "user", "content": "Weather in Berlin?"}],
alias="gpt-mini",
tools=[ToolDefinition(
name="get_weather",
description="Look up the current weather for a city.",
input_schema={"type": "object", "properties": {"city": {"type": "string"}}},
)],
)
for block in result.output.blocks:
if block.type == "tool_use":
block.name, block.input # dispatch it, then send the result back in a follow-up callExtended thinking
Give reasoning models a thinking budget with a ThinkingConfig
(budget_tokens); the reasoning surfaces as ThinkingBlocks and via
result.output.thinking (kept out of .text). This is distinct from the
reasoning_effort parameter — don't set both for the same provider.
Streaming
core.llm.stream(...) takes the same parameters as call and returns an async
iterator of StreamEvents — await it, then iterate:
stream = await core.llm.stream(messages=[...], alias="gpt-mini")
async for event in stream:
if event.type == "content_block_delta" and event.delta.type == "text_delta":
print(event.delta.text, end="", flush=True)
elif event.type == "message_stop":
breakText streams as the model produces it: the first content_block_delta arrives
while the model is still writing, not after it has finished. The events, in
order:
content_block_delta— one per piece of text, to append to content blockindex. Joined, a block's deltas are the textcallwould have returned.message_delta— the stop reason and the call's token usage, once.message_stop— the end.
A streamed call is authorized, routed, guarded, audited and billed exactly like
the same call: same alias resolution and fallback chain, same guardrail
policy, same usage and cost. Tool calls are not streamed — a tool-calling
answer yields no deltas for them; use call to read them.
Errors arrive where they happen:
- Before any text — an ungranted alias, a guardrail that blocks the
request, an unreachable provider — the
awaitraises, exactly ascallwould. - Part-way — a guardrail that blocks the model's output raises
ImGuardrailBlockedErrorfrom the iteration, after the deltas that preceded the blocked text; the call'stimeout(or the gateway's 900 s ceiling, which bounds the whole stream) raisesImTimeoutError; a connection that breaks raisesImTransportError, and so does a gateway fault mid-stream — the same class that fault raises oncall, so code that retries onImTransportErrorretries a stream and a call alike.
A stream that raised is never a complete answer, so don't treat the text you already received as one:
from im_sdk import ImGuardrailBlockedError
text = []
try:
async for event in await core.llm.stream(messages=[...], alias="gpt-mini"):
if event.type == "content_block_delta":
text.append(event.delta.text)
except ImGuardrailBlockedError as exc:
... # the policy refused the output part-way; `text` is only what came beforeLeaving the loop early (break, or the agent stopping) closes the connection,
and the platform stops the model call upstream. It is recorded as cancelled by
the agent; tokens the provider reports for it are billed, and none are
estimated for it when it reports none.
Against a platform that does not stream yet, the same sequence is produced from
the complete answer — one delta per text block, all at the end — so code
written against stream works either way.
Concurrency
Concurrent resource calls are supported. Tracing spans are task-local (a
ContextVar), so calls fanned out with asyncio.gather (or create_task) each
nest under the span active when the task forked, and spans may close in any
order. Fan out core.* calls when it speeds up your agent.