Capabilities

Every capability hangs off the ImCore client as a lazy namespace, and every call resolves against a grant the agent was given in the Portal. Read what a given agent may use with im-cli agent models (aliases) and im-cli agent services (connected services) — don't guess names.

Two addressing styles:

  • Model-addressed — pass alias=. Resources: llm, embedding.
  • Service-addressed — pass service= (the connected-service name). Resources: document_parse, web_search, web_crawler, web_fetch, data, object_store, sharepoint, vector, mcp, http_capability.

Every method returns a ResourceResult except llm.stream (an async iterator of stream events) and llm.call_structured (a parsed instance of your pydantic model). core.vars is not a resource call at all — it reads the organization variables the agent was granted.

Most methods take an optional timeout (seconds). It bounds the call end-to-end: the gateway enforces it and answers with a typed 504 (ImTimeoutError), and the SDK caps its own wait at the same value plus a small grace so that typed error wins over a generic client-side timeout. Without it, the SDK waits up to 925 s. For llm, embedding and web_fetch a per-call timeout can only tighten the deadline — the gateway (or, for web_fetch, the connected service) caps those calls at its own ceiling (900 s by default for llm), so a larger value is silently capped, never extended. http_capability, data, sharepoint, vector and mcp are the exceptions — they have no timeout param; their budget comes from the connected service's configuration.

core.llm

result = await core.llm.call(
    messages=[{"role": "user", "content": "hello"}],
    alias="gpt-mini",
    max_tokens=512,          # optional; see below
)
text = result.text

call(messages, *, alias, ...) — the optional keyword-only params are max_tokens, system, temperature, top_p, top_k, stop_sequences, tools, tool_choice, thinking, seed, reasoning_effort, timeout, and provider_params. top_k is best-effort: the Anthropic family honors it; providers without a top_k knob (OpenAI, Azure, Gemini) ignore it.

provider_params reaches provider-native sampling and format knobs the normalized parameters do not name — OpenAI logprobs / top_logprobs / frequency_penalty / presence_penalty / logit_bias / max_completion_tokens / parallel_tool_calls / verbosity, Gemini generationConfig (its sampling and thinkingConfig keys), Mistral random_seed / safe_prompt, Ollama behind LiteLLM num_ctx / num_predict / repeat_penalty and the like. It is an allow-list: any other key refuses the call with ImValidationError (VALIDATION_REQUEST) before anything is sent, naming the refused keys in context["refused_provider_params"]. That includes every key that could carry prompt or tool content (system, messages, contents, systemInstruction, tools, …): a key like that would reach the model without passing the organization's guardrails. It also includes the name of a typed parameter (temperature, max_tokens, …): set that through the typed parameter instead. top_k is the one key that is both; there the typed parameter wins. result.output is an LlmOutput whose blocks are typed content blocks (TextBlock / ToolUseBlock / ThinkingBlock); result.text concatenates the text blocks only.

If the organization attached a guardrail policy to the alias, this call may be refused with ImGuardrailBlockedError or ImGuardrailUnavailableError, or succeed with rewritten content — see Guardrails. The policy binds every agent granted the alias and there is no parameter to opt out.

Also on core.llm:

  • stream(...) — same params; await it, then iterate the stream events as the text arrives (see Streaming).
  • call_structured(messages, response_model, *, alias, ...) — pass a pydantic BaseModel type and get a parsed instance of it back (not a ResourceResult).

core.embedding

result = await core.embedding.embed(["first text", "second text"], alias="text-embed")
vectors = result.output.vectors            # list[list[float]]

embed(input, *, alias, dimensions=None, timeout=None, provider_params=None) — input is a string or list of strings (positional). Returns a ResourceResult with an EmbeddingOutput (.vectors) and token usage. result.text is empty for embeddings — read result.output.vectors. provider_params takes output-shape knobs only (encoding_format, dimensions, input_type, task_type, output_dimensionality, truncate); any other key refuses the call as it does on core.llm.

Embeddings are never guardrail-checked: the call cannot select a policy at all (see Guardrails).

core.document_parse

Service-addressed. The document arrives as exactly one of file (the content itself, Base64-encoded) or file_ref (the id of a file uploaded to the Portal — workspace files). With file_ref the gateway fetches the bytes from platform storage server-side, scoped to the calling execution's workspace, so the document never enters the agent process; an unknown, expired, or foreign ref fails the call with the typed file_not_found error.

import base64
 
# Inline content:
payload = base64.b64encode(open("/tmp/report.pdf", "rb").read()).decode()
result = await core.document_parse.partition(service="unstructured", file=payload)
 
# Portal upload (id from the run input, e.g. data["files"][0]):
result = await core.document_parse.partition(service="unstructured", file_ref=file_id)
 
elements = result.output.elements          # DocumentOutput.elements
text = result.text                         # newline-joined element text
  • partition(service, file=None, *, file_ref=None, filename=None, strategy=None, chunking_strategy=None, combine_under_n_chars=None, max_characters=None, extract_image_block_types=None, timeout=None)
  • extract(service, file=None, *, file_ref=None, ...) — richer element metadata (defaults to hi_res).
  • ocr(service, file=None, *, file_ref=None, filename=None, languages=None, timeout=None) — forces OCR.

Service-addressed.

result = await core.web_search.search(service="tavily", query="in·manas platform")
answer = result.text                       # SearchOutput.answer
hits = result.citations                    # the search results
  • search(service, query, *, search_depth=None, max_results=None, include_domains=None, exclude_domains=None, timeout=None, provider_params=None)
  • extract(service, urls, *, extract_depth=None, timeout=None, provider_params=None) — returns an ExtractOutput with .pages.

core.web_crawler

Service-addressed.

  • run_actor(service, actor_id, *, input=None, timeout=None) → CrawlOutput (.items, .run).
  • fetch_dataset(service, dataset_id, *, limit=None, offset=None).

core.data

Service-addressed table CRUD against a database your organization connected, through one of two providers: DreamFactory or Microsoft Dataverse. One connected service is one database (for Dataverse, one environment): which database — and, optionally, which tables — an agent may reach is fixed in the service config by an admin, so service names the connection, never a provider. filter, fields and order are written in the connected service's own syntax and passed through — OData for Dataverse.

await core.data.create("customer-db", "agent_memory", [{"key": "run:1", "value": '{"step": "a"}'}])
 
result = await core.data.read("customer-db", "agent_memory", filter="key like 'run:%'", limit=50)
rows = result.records                       # == result.output.records
  • create(service, table, records, *, fields=None, atomic=True) — records is a list of row dicts. Returns the inserted rows: ids only unless fields asks for more (fields="*").
  • read(service, table, *, filter=None, ids=None, fields=None, related=None, order=None, limit=None, offset=None, include_count=None) — related and offset are DreamFactory only.
  • update(service, table, records, *, ids=None, filter=None, atomic=True) — address rows by ids, or, on DreamFactory only, by filter or by a per-record id inside each element of records. One record plus ids/filter applies the same patch to every matched row.
  • delete(service, table, *, ids=None, filter=None, atomic=True) — ids or filter is required (filter on DreamFactory only); an unscoped delete would mean "every row", so it raises ImValidationError before leaving your process.

On Dataverse, update and delete address rows by ids alone, at most 100 per call (the same cap applies to read by ids); related, offset and a filter on a write are refused before any request.

Four things to know before you write against it:

  • Serialize JSON column values yourself (DreamFactory). A nested dict is rejected even by a jsonb column; pass json.dumps(...). Reads come back the same way — a JSON column is a JSON-encoded string, not a nested object.
  • Timestamps are not ISO-T (DreamFactory). They arrive as "2026-08-03 10:08:55.926853+00" (a space), so they are not datetime.fromisoformat-safe on older Pythons. Treat row values as opaque and parse deliberately.
  • Paging is explicit. Pass limit (and offset on DreamFactory); nothing auto-paginates, so one call can't pull a whole table. result.output.count is the total match count but only when you pass include_count=True — otherwise it stays None (absent, not zero). On DreamFactory the next-page offset is at result.raw["meta"]["next"]; Dataverse has no offset, so order by a column and filter past the last value read.
  • Multi-row writes are all-or-nothing. That's the default and it's what you want: without it a 10-row create failing on row 7 leaves 6 rows committed and a retry duplicates them. atomic=False opts into that native behaviour. On Dataverse a multi-row write is one $batch changeset.

Writes return the rows they touched, so len(result.records) is a usable affected-row count on update and delete. Setup, permissions and each provider's limits: docs/dreamfactory.md and docs/dataverse.md in the platform repository.

core.file_store

Reads a file a user uploaded in the Portal, by id — the same opaque id document_parse accepts as file_ref.

file_id = inputs["files"][0]                     # conventional run-input key
result = await core.file_store.get(file_id)
rows = result.output.text                        # decoded when the object is UTF-8 text
raw = result.output.content_b64                  # Base64 bytes for anything else
  • get(file_ref) → FileOutput (.content_b64, .text, .filename, .mime_type, .size_bytes).

No service, and no grant. Uploads live on the platform's own buffer disk, so there is no third party to name and nothing to authorize against: the gateway resolves the ref against your execution's workspace, which is the whole tenancy story. An unknown, expired or foreign ref all fail the same way (ImValidationError, file_not_found) — deliberately indistinguishable, so an error cannot tell you a file exists somewhere you cannot reach.

The bytes arrive Base64 in the result envelope, bounded by the object-size ceiling — nothing is written into the agent's working directory.

An upload expires (3 days). To keep one, write it to a data bucket with core.object_store.put_from_upload — one call, and the bytes never pass through your process. The whole story, from the io.json file input to the bucket, is on Files & storage.

core.object_store

Service-addressed. Objects in a bucket the organization connected — the durable counterpart to file_store, addressed by key instead of by id.

# Keep an upload past its TTL: the gateway copies the bytes buffer → bucket,
# server-side. You send two strings.
stored = await core.object_store.put_from_upload("archiv", file_ref=file_id)
 
# Write something you produced yourself.
await core.object_store.put("archiv", key="berichte/q3.csv", content_b64=csv_b64)
 
# Read it back, in this run or any later one.
text = (await core.object_store.get("archiv", "berichte/q3.csv")).output.text
 
# Enumerate; next_cursor is None on the last page.
page = await core.object_store.list("archiv", prefix="berichte/")
keys = [o.key for o in page.output.objects]
MethodOutput variantNotes
get(service, key)ObjectOutput.content_b64, .text, .content_type, .size_bytes
stat(service, key)ObjectStatOutputmetadata only — the cheap existence/size check
list(service, prefix=…, limit=…, cursor=…)ObjectListOutput.objects, .next_cursor
put(service, key=…, content_b64=…)ObjectWriteOutputcreate-only — an existing key raises object_exists
replace(service, key=…, content_b64=…)ObjectWriteOutputwrites whether or not the key exists
put_from_upload(service, file_ref=…, key=…)ObjectWriteOutputbuffer → bucket, server-side; create-only
delete(service, key)ObjectDeleteOutputone key; .deleted is False when there was nothing there

Four things worth knowing before you write against it:

  • Every operation is granted separately. put and replace are different permissions on purpose — an agent granted only put can add objects and can never destroy one. Expect to be told which ones you have; a missing grant is a policy refusal, not a bug in your code.
  • Keys are relative to the service. Each connected service owns a prefix in its bucket and the gateway adds and strips it, so the key you send is the key you get back. .., a leading /, empty segments, control characters, unpaired surrogates, a key too long once that prefix is added, and the reserved .im- namespace are refused before the store is reached.
  • delete takes exactly one key. There is no prefix or bulk delete.
  • Another service's object reads as missing (object_not_found), same as one that never existed.

Written up end to end — key rules, what list does and does not return, the per-operation grants and the size cap — on Files & storage. What an admin configures and how the bucket is backed up: docs/agent-data-bucket.md in the platform repository.

core.sharepoint

Service-addressed. Files in one SharePoint Online document library and the items of SharePoint lists, in a site the organization already owns. AI OS reaches it through Microsoft Graph with the organization's own Entra app; it never acts as a person. One connected service is one site, one library and an optional folder inside it. core.sharepoint needs an im-sdk later than 1.2.0.

import base64
 
# Read a file; .text decodes UTF-8, .content_b64 holds the raw bytes.
offer = await core.sharepoint.get("vertrieb", "Angebote/2026/offer.pdf")
 
# Create a file. put refuses to overwrite; replace is the one that does.
csv_b64 = base64.b64encode(report.encode()).decode()
await core.sharepoint.put("vertrieb", path="Berichte/q3.csv", content_b64=csv_b64)
 
# Enumerate a folder; next_cursor is None on the last page.
page = await core.sharepoint.list("vertrieb", path="Berichte")
paths = [o.key for o in page.output.objects]
 
# Lists: fields are keyed by each column's internal name.
rows = await core.sharepoint.list_items("vertrieb", "Kontakte")
created = await core.sharepoint.create_item(
    "vertrieb", "Kontakte", {"Title": "Muster GmbH", "Unit_x0020_Price": 120}
)
item_id = created.output.item.id
MethodOutput variantNotes
get(service, path)ObjectOutput.content_b64, .text, .content_type, .size_bytes; at most 26 MiB
stat(service, path)ObjectStatOutputmetadata only; a folder is refused
list(service, path=…, limit=…, cursor=…)ObjectListOutput.objects, .next_cursor; folders end in /; limit defaults to 100, capped at 200
put(service, path=…, content_b64=…, content_type=…)ObjectWriteOutputcreate-only — an existing file raises object_exists
replace(service, path=…, content_b64=…, content_type=…)ObjectWriteOutputwrites whether or not the file exists
delete(service, path)ObjectDeleteOutputone file, to the site's recycle bin; .deleted is False when there was nothing there
put_from_upload(service, file_ref=…, path=…)ObjectWriteOutputPortal upload → library; create-only; without path it lands at uploads/<file_ref>-<filename>
list_items(service, list, limit=…, cursor=…)ListItemsOutput.items (each .id, .fields, .etag, .last_modified), .next_cursor
get_item(service, list, item_id)ListItemOutput.item; a missing item raises item_not_found
create_item(service, list, fields)ListItemOutputfields must not be empty
update_item(service, list, item_id, fields)ListItemOutputpatches the columns you send; the rest keep their values
delete_item(service, list, item_id)ListItemDeleteOutput.deleted is False when the item was not there

The file methods answer with the same output variants as core.object_store, with .key holding the path, so code written against a bucket reads the same against a library.

Paths are relative to the service folder and cannot leave it. The connected service fixes the site, the library and the folder; the path you send is the path you get back. A path is refused before any request reaches Microsoft when it has an empty, . or .. segment, any of " * : < > ? \ | or a control character, a segment ending in . or a space, or is longer than 400 characters together with the folder. A leading or trailing / is ignored.

put and replace are separate grants. Every one of the twelve methods is granted on its own, and an agent granted only put can add files but never overwrite one. A method the agent was not granted is a policy refusal (POLICY_ACTION_NOT_GRANTED) before any Microsoft request. The site's own grant on the Microsoft side is the ceiling: an agent granted put on a site where the app holds only read gets a 403 from Graph.

Only the lists in allowed_lists are reachable, and an empty list means none. A list outside it is refused with list_not_allowed before any Microsoft request. Lists are named exactly as SharePoint displays them; a list that is allowed but missing on the site raises list_not_found.

Columns are addressed by their internal name, in the fields you send and the fields you read. The internal name is fixed when a column is created and does not follow a later rename; spaces and umlauts are encoded (Unit Price is Unit_x0020_Price), and the title column is Title. Read one item with get_item to see the names a list uses.

Pass next_cursor back unchanged as cursor, to the same folder or list it came from: a cursor that does not continue the listing you name, or that holds a . or .. segment, is refused with invalid_cursor. What an admin sets up on the Microsoft tenant, the connection form and troubleshooting: docs/sharepoint.md in the platform repository.

core.vector

Service-addressed. Embeddings in a vector database the organization connected (today via Qdrant) — the end of the retrieval pipeline document_parse and embedding start. One connected service is one collection, created when an admin saves the service; service names the connection, and nothing you pass can widen it.

# Store — re-using an id overwrites that point, so a re-ingest is idempotent.
await core.vector.upsert("handbuch", points=[
    {"id": i, "vector": v, "payload": {"source": "handbuch.pdf", "chunk": i}}
    for i, v in enumerate(vectors)
])
 
# Search — best-first hits, with the payload you stored.
result = await core.vector.search("handbuch", vector=query_vector, top_k=5)
for hit in result.output.hits:
    hit.id, hit.score, hit.payload
 
# Delete — by id, or by filter (how a re-ingest replaces a document).
await core.vector.delete("handbuch", filter={
    "must": [{"key": "source", "match": "handbuch.pdf"}],
})
MethodOutput variantNotes
upsert(service, *, points)VectorUpsertOutput.upserted; max 1000 points per call, an oversized batch is refused rather than partly written
search(service, *, vector, top_k=None, filter=None, with_payload=None)VectorSearchOutput.hits (.id, .score, .payload), .count; top_k defaults to 10, ceiling 250
delete(service, *, ids=None, filter=None)VectorDeleteOutput.deleted — the ids you passed, or None for a filter-delete; max 1000 ids

Four things to know before you write against it:

  • The vector dimension is the admin's setting. It is fixed when the service is connected, because a collection cannot be resized. A vector of another width is refused with both numbers in the message, which almost always means the embedding model is not the one this service was created for. And embed the query with the same model that embedded the documents — nothing can check that, and a mismatch returns confident nonsense rather than an error.
  • Each operation is granted separately. search alone is a retrieval assistant that cannot alter the corpus; upsert alone is an ingest job. A missing grant is ImForbiddenError — policy, not a bug in your code.
  • filter is a small vendor-neutral dialect: {"must": [{"key": …, "match": …}]} — exact matches on payload keys, AND-ed. Qdrant's should / must_not / ranges are not exposed, and anything outside the subset is refused, not ignored.
  • delete needs ids or filter, exactly one. Neither is refused rather than read as "everything", both is refused rather than guessed at — locally, before anything is sent, like data.delete.

result.text is empty for every vector call — read result.output. Query vectors, payloads, filter values and point ids never reach trace exports, and no setting opts into that.

Written up end to end — the pipeline, the filter dialect, re-ingest, the typed errors — on Vector store & RAG. What an admin configures: docs/vector-store.md in the platform repository.

core.web_fetch

Service-addressed. The bytes at a URL you choose — the one resource where the agent names the destination rather than an administrator. service names a fetcher connection whose configuration is the policy: which content types and ports are permissible, whether internal hosts are reachable at all, whether robots.txt is honoured, and the byte and time ceilings a per-call value may only tighten.

# A feed, a page, a JSON document — decoded for you.
feed = await core.web_fetch.fetch("web", "https://example.org/feed.xml")
entries = parse(feed.text)
 
# Anything binary: Base64, which is what document_parse takes.
pdf = await core.web_fetch.fetch("web", url, as_text=False)
elements = (await core.document_parse.partition(
    "unstructured", file=pdf.output.content_base64,
)).output.elements
 
# "Is it still there, and how big?" — a HEAD, not a download.
check = await core.web_fetch.probe("web", url)
if check.output.reachable and (check.output.content_length or 0) < 5_000_000:
    ...
MethodOutput variantNotes
fetch(service, url, *, as_text=None, max_bytes=None, timeout=None, accept=None, headers=None)FetchOutput.content_text or .content_base64 (never both), .status, .final_url, .redirect_chain, .content_type, .charset, .size_bytes, .truncated, .not_modified, .etag, .last_modified. result.text reads either
probe(service, url, *, follow_redirects=None, timeout=None)ProbeOutput.reachable, .status, .final_url, .redirect_chain, .content_type, .content_length, .elapsed_ms; result.text is empty

Five things to know before you write against it:

  • Internal addresses are unreachable, and no argument changes that. Loopback, private, CGNAT and link-local ranges — the cloud metadata endpoint included — are refused after resolution and checked on the resolved address, so a hostname that points inward does not get through either. An admin can re-enable named hosts per service; an agent cannot.
  • as_text decides text or bytes, and is refused for the wrong media type. It defaults to true and Base64-ing a PDF into a text field is exactly what that refusal prevents. Pass as_text=False for anything binary and read .output.content_base64.
  • Refusals are typed and distinguishable. A disallowed content type, a blocked port or address, and a robots.txt that says no are ImForbiddenError and naming what was received and what is allowed — none of them is worth a retry, and the remedy is a different URL or a conversation about the service's configuration. ImTimeoutError and ImResourceError are the retryable ones.
  • probe answers, it does not raise. A 404 is reachable=False with status=404; nothing answering at all leaves status as None. The address and scheme rules still apply, so an internal target is still a refusal — "not permitted" must not read the same as "not there".
  • Conditional requests are supported and cheap. Keep .output.etag and send it back as headers={"if-none-match": etag}; an unchanged resource answers not_modified=True with no body at all. headers takes only if-none-match / if-modified-since / accept / accept-language, and anything else — Authorization and Cookie above all — is refused locally. The fetcher forwards no credentials, ever.

Fetched bodies and full URLs never reach trace exports, and no setting opts into that: what the trace keeps is the host and the structural fields (status, content_type, size_bytes, truncated, reachable, elapsed_ms).

Written up end to end — the two ceilings, conditional requests, the probe's verdicts, the typed refusals — on Fetching a URL. What an admin configures, and the thirteen security rules behind it: docs/web-fetch.md in the platform repository.

core.mcp

Service-addressed. The tools on a Model Context Protocol server an organization connected — service is the connection's name, and the grant unit is the tool: an agent may list and call only the tools it was granted on that connection.

tools = await core.mcp.list_tools("acme-tools")          # only what this agent was granted
print(tools.output.names)                                # ["search_docs", "files.read"]
 
result = await core.mcp.call_tool("acme-tools", "search_docs", {"q": "refund policy"})
if result.output.is_error:     # the tool ran and reported failure: an answer, not an exception
    ...
print(result.output.text)      # the tool's text; its images are in result.output.images
MethodOutput variantNotes
list_tools(service)McpToolsOutput.tools — a list of McpTool (name, description, input_schema), in the order the platform lists them; .names is just the names. result.text is the names, one per line
call_tool(service, tool, arguments=None)McpToolResultOutput.tool, .text (the tool's text, merged into one string), .images (a list of McpImage: data is base64, mime_type), .is_error. result.text reads the same as .text

Things to know before you write against it:

  • Nothing is granted unless it is named. list_tools returns the server's tools narrowed to the ones this agent was granted — a connection ticked with no tools lists none. An administrator grants tools per agent in the Portal, one tool at a time; granting "everything" means naming everything.
  • Tool names are never normalized. They are case-sensitive and may contain - and . (Delete_File is not delete_file, and files.read is a name). What list_tools returned is what call_tool sends. Only the helpers below rename anything, and only on the way to a model.
  • A tool that failed is an answer. The server ran it and said it failed: you get a normal result with output.is_error set, and the Portal bills it like any other call. Check is_error before you trust output.text — the text is then the tool's own error message.
  • A result is text, images and is_error, and nothing else. The platform runs the call through a tool router that hands back one flattened answer, so the SDK does not pretend to more than that: see Known limits below. An empty result is text == "" and images == [].
  • arguments is a dict or None. call_tool raises ImValidationError locally, with no request, for an empty or non-string tool or for arguments that are not a dict.
  • There is no timeout argument. One call's budget is the connection's timeout (120 s by default, 270 s at most); on expiry the call raises ImTimeoutError and is not retried, because the tool may have run.
  • MCP tools are never injected into core.llm calls. You decide what the model sees and you run what it asks for — the helpers below make that short.
  • The execution trace never shows what you sent or got back. Each MCP call is one row on the run's Timeline in the Portal — the tool's name, the connection, the status, what ended it, the latency and the cost — and the arguments and the result are in none of it. The SDK's trace event for an MCP call carries no error text either, because a failed call's message can quote the value it rejected; the typed exc.context["kind"] below is how the Portal tells what happened.

Errors

Every failure is one of the SDK's existing exceptions, with the MCP condition in exc.context["kind"]. A call_tool is billed only when the tool answered — a result with is_error set included, which is why it is returned, not raised — so none of the kinds below is billed. A list_tools is never billed: it reads the platform's copy of the list and never reaches the server.

exc.context["kind"]Raised asexc.codeBilled?What it means
tool_not_grantedImForbiddenErrorPOLICY_ACTION_NOT_GRANTEDNoThe tool is not callable under this agent's grant. Nothing was sent to the server. exc.context["reason"] says why: absent — the grant does not name the tool (add it in the Portal; the message lists what is granted); grant_lag — the grant names it and was changed moments ago, the platform's tool router has not caught up (try again in a minute); grant_mismatch — this session's grant names it, but the platform no longer admits it for the organization (revoked or changed after the session began, or not this organization's connection): start a new session and check the grants; access_blocked — the organization is suspended
tool_name_collisionImForbiddenErrorPOLICY_FORBIDDENNoThe server lists another tool whose name folds onto this one (case, or - against _), so the platform cannot tell which is meant and refuses the call. Nothing ran. The server has to rename one of them
virtual_key_missingImResourceErrorRESOURCE_MISCONFIGUREDNoThe organization's key to the platform's tool router is unknown, so the call was not sent. Not something the agent can fix: an operator converges the key (bifrost:converge --now)
tool_unavailableImResourceErrorRESOURCE_ERRORNoThe grant is fine, but Bifrost does not hold that tool for this key: the server no longer lists a tool by that exact name. Arguments the server refuses are a protocol_error
auth_failedImResourceErrorRESOURCE_ERRORNoThe server answered 401/403, or the connection needs to be authorized again: the connection's credential was refused
protocol_errorImResourceErrorRESOURCE_ERRORNoThe server refused the arguments, or the call failed in a way the platform cannot classify further; the tool may have run
result_unreadableImResourceErrorRESOURCE_ERRORNoThe tool ran, but its result has a content type the platform cannot read (see Known limits). Do not retry blindly: the tool's effect happened
response_too_largeImResourceErrorRESOURCE_ERRORNoThe answer was over the platform's size cap; the tool ran
unreachableImResourceErrorRESOURCE_UPSTREAM_FAILEDNoThe platform could not connect to the server; nothing ran
abortedImResourceErrorRESOURCE_UPSTREAM_FAILEDNoThe connection broke mid-call; the tool may have run
timeoutImTimeoutErrorRESOURCE_TIMEOUTNoNo answer in time — the tool may have run, and the call is not retried
connect_timeoutImTimeoutError (no connection in time) or ImResourceError (could not connect)RESOURCE_TIMEOUT or RESOURCE_UPSTREAM_FAILEDNoThe call never left the gateway: the platform's tool router did not accept the connection. Nothing ran

tool_not_granted and tool_unavailable are the two to tell apart: the first is the agent's grant (fix it in the Portal), the second is the server (the tool was renamed or removed, or the arguments were wrong). ImForbiddenError and ImResourceError are what you already catch for every other resource; branch on exc.context["kind"] when you need more.

Known limits of a result

The platform runs MCP calls through a tool router (Bifrost) that returns one flattened string per result. What it cannot give back, the SDK does not claim:

  • Text blocks arrive merged, as one string with no separator, so where one block ends and the next begins is lost — and so is the order of text relative to images. .text and .images are two separate lists, not a sequence.
  • Only text and images are carried. Audio is dropped (a result of audio only is an empty result, not an error). A resource link or an embedded resource arrives as the text the router wrote for it — the link as its JSON, an embedded resource as the literal [Embedded Resource Response: resource], so its content is lost: a tool that returns a file as an embedded resource (GitHub's get_file_contents, for one) gives you only the placeholder. This lifts with the platform's next router upgrade that includes the upstream fix. A content type the router does not read at all, or an image without data or a media type, fails the call as result_unreadable.
  • Structured content and _meta are not returned. A tool that also writes its structured result as text gives you that text; one that returns only structured content gives you an empty result.
  • A tool's text that looks exactly like the router's image marker is read as an image. The router writes an image as [Image Response: <base64>, MIME: <type>], and a tool that writes that very string as text cannot be told apart. The effect is a wrong entry in .images (and the text missing from .text), never more than the tool could already put in front of your model.
  • A listed tool is name, description and input_schema. Titles, output schemas and annotations are not carried.
  • Strict mode is off. The router drops additionalProperties: false from a tool's schema, so no strict flag is set on a definition, and core.llm.call sends none; the server validates the arguments.

Giving the tools to a model

list_tools returns McpTool entries. One free function takes them from there, as plain dicts (the SDK imports no provider library): mcp_tools(tools, *, prefix=None) returns (definitions, name_map).

Pass the definitions to core.llm.call, whatever the model behind the alias. Its tools= takes {name, description, input_schema}, which is their shape, and the gateway translates it for the provider (the loop below). The name map takes the names the model calls back to the server's, for call_tool.

ReturnedWhat it is
definitions[{"name", "description", "input_schema"}] — what core.llm.call(tools=…) takes, whatever the model
name_map{provider name: the server's exact tool name}, keyed by the definitions' names in the same order — the way back from the name a model called. Indexing a name the model invented is a KeyError

tools is what list_tools returned: the result itself, or its .output. Anything else, a list of McpTool or a raw tools/list dict included, is a TypeError. mcp_tools is importable from im_sdk.

  • input_schema passes through unchanged. Providers insist on an object schema, so two cases are adjusted. A schema with properties but no type is kept and gets "type": "object" added. A schema that is empty, is declared with another type (or has neither type nor properties) is replaced by {"type": "object", "properties": {}}, and that tool can still be called with no arguments.
  • No strict schema. A definition carries no strict flag and core.llm.call sends none. The platform's tool router drops additionalProperties: false from the schemas it lists, so a schema that reaches you cannot be shown to meet a provider's strict mode. The server validates the arguments the model sends.
  • The description is the tool's description ("" when the server gave none).
  • Names are converted, because the two sides disagree. A provider's tool name is [a-zA-Z0-9_-] and at most 64 characters; an MCP name may contain . and be longer. A name that is already valid is never touched (with a prefix, it is the prefixed spelling that has to be valid). Every other character becomes _; a name that collides with another, or is over 64 characters, gets a short stable hash of the original appended. The mapping depends only on the tool list, never on its order, but a different list can name a colliding tool differently. That is why one call returns both: hold the map next to the definitions it came with, and never rebuild it from a later listing.
  • prefix (for example the connection name) goes in front, joined by __, before anything is sanitized: prefix="acme-tools" turns search_docs into acme-tools__search_docs. Use it when one agent offers tools from more than one connection, so equal tool names cannot collide.

A complete tool loop, on the platform

A deployed agent has no outbound network and no provider keys, so the loop that works everywhere runs the model through core.llm: the call is resolved against a model alias, billed, and guardrailed like every other. The definitions mcp_tools returns have exactly the {name, description, input_schema} shape tools= validates as a ToolDefinition (the gateway translates it for whichever provider the alias routes to), and the model's calls come back as tool_use blocks. The conversation travels in the Chat Completions shape: the assistant turn keeps its tool_calls, and each result is a role: "tool" message carrying the call's tool_call_id.

import json
 
from im_sdk import ImCore, ImError, mcp_tools
 
SERVICE = "acme-tools"
ALIAS = "gpt-mini"                      # a model alias this agent was granted
 
 
async def run_tool(core: ImCore, names: dict[str, str], use) -> str:
    """What the model reads back for one tool call. It never raises: a failure is an answer."""
    name = names.get(use.name)
    if name is None:                    # a model can invent a tool; that is not a call to forward
        return f"There is no tool called {use.name}."
    try:
        result = await core.mcp.call_tool(SERVICE, name, use.input)
    except ImError as exc:              # not granted, unreachable, timed out, ...
        return f"The tool could not be run: {exc}"
    text = result.output.text
    return f"The tool reported an error: {text}" if result.output.is_error else text
 
 
async def main(core: ImCore, data: dict) -> str:
    listing = await core.mcp.list_tools(SERVICE)
    defs, names = mcp_tools(listing, prefix=SERVICE)    # names: provider name -> the server's exact name
 
    messages = [{"role": "user", "content": data["prompt"]}]
    for _ in range(8):                                  # bound the loop
        reply = await core.llm.call(messages, alias=ALIAS, max_tokens=1024, tools=defs)
        uses = [b for b in reply.output.blocks if b.type == "tool_use"]
        if not uses:
            return reply.text
        messages.append({
            "role": "assistant",
            "content": reply.text,
            "tool_calls": [
                {"id": u.id, "type": "function",
                 "function": {"name": u.name, "arguments": json.dumps(u.input)}}
                for u in uses
            ],
        })
        for use in uses:
            messages.append({
                "role": "tool", "tool_call_id": use.id,
                "content": await run_tool(core, names, use),
            })
    return "Stopped: too many tool rounds."

The model only ever sees acme-tools__files_read; names is how that becomes files.read again.

run_tool gives the model output.text only. A tool that also returns images has them in output.images (base64 data and a mime_type); pass them on only if your model accepts images.

With your own client

The definitions are also the Anthropic Messages tools= shape, so with Anthropic's own client they go in as they are, and the calls arrive as tool_use content blocks whose input is already a dict:

from anthropic import AsyncAnthropic
from im_sdk import ImCore, ImError, mcp_tools
 
SERVICE = "acme-tools"
 
 
async def run_tool(core: ImCore, names: dict[str, str], use) -> dict:
    """The `tool_result` block for one `tool_use`."""
    block = {"type": "tool_result", "tool_use_id": use.id, "is_error": True}
    name = names.get(use.name)
    if name is None:
        return {**block, "content": f"There is no tool called {use.name}."}
    try:
        result = await core.mcp.call_tool(SERVICE, name, use.input)
    except ImError as exc:              # not granted, unreachable, timed out, ...
        return {**block, "content": f"The tool could not be run: {exc}"}
    return {**block, "content": result.output.text, "is_error": result.output.is_error}
 
 
async def main(core: ImCore, data: dict) -> str:
    listing = await core.mcp.list_tools(SERVICE)
    defs, names = mcp_tools(listing, prefix=SERVICE)
 
    client = AsyncAnthropic()
    messages = [{"role": "user", "content": data["prompt"]}]
    for _ in range(8):
        reply = await client.messages.create(
            model="claude-sonnet-5-5", max_tokens=1024, messages=messages, tools=defs,
        )
        uses = [b for b in reply.content if b.type == "tool_use"]
        if not uses:
            return "".join(b.text for b in reply.content if b.type == "text")
        messages.append({"role": "assistant", "content": reply.content})
        messages.append({
            "role": "user",
            "content": [await run_tool(core, names, use) for use in uses],
        })
    return "Stopped: too many tool rounds."

What an administrator configures — connecting a server, discovery, per-tool grants, the audit and billing rules: docs/mcp.md in the platform repository.

core.http_capability

Service-addressed escape hatch for a connected HTTP service the Portal exposes.

result = await core.http_capability.request(service="my-api", path="/v1/things", verb="get")
body = result.output.body                  # RawOutput.body

request(service, path, verb="post", body=None, headers=None, query_params=None) — verb is one of get / post / put / patch / delete.

body is sent as JSON. For an upstream that expects multipart/form-data, set that Content-Type header: the gateway then sends one part per top-level key of body — a string as a plain form field, anything else as an application/json part — and writes the boundary itself.

result = await core.http_capability.request(
    service="eu-funding-search",
    path="/search-api/prod/rest/search",
    verb="post",
    headers={"Content-Type": "multipart/form-data"},
    body={"query": {"bool": {"must": [...]}}, "languages": ["en"]},
    query_params={"apiKey": "SEDIA", "text": "***"},
)

core.vars

The organization variables granted to this agent, fixed when the client authorized — not a resource call, no ResourceResult, no gateway round trip.

base_url = core.vars["SHOP_URL"]               # KeyError: not granted or unknown
region = core.vars.get("REGION", "eu-west-1")
everything = core.vars.all()                   # dict copy
 
pages = core.vars.get_int("MAX_PAGES", 10)     # int — 10 when not granted
rate = core.vars.get_float("SAMPLE_RATE")      # float
dry_run = core.vars.get_bool("DRY_RUN", False) # bool — true/false, 1/0, yes/no, on/off
config = core.vars.get_json("SCRAPER_CONFIG")  # whatever the JSON holds

Values are stored as strings, exactly as entered, and core.vars["KEY"] returns the string. The typed reads parse it: a variable that is not granted returns default when you pass one, else the same KeyError; a value that does not parse raises ValueError naming the key and the expected type, never the value. core.vars needs im-sdk 1.2.0 or newer.

Secrets are never readable here. Use one by writing core.secret("KEY") where the value belongs in a call to a connected service: it returns a SecretRef placeholder carrying this session's nonce, which the gateway substitutes on the way out — only where the connection pins the destination — and masks out of that call's answer (im-sdk 1.2.0 or newer). Setup, grants, the rules and the error codes: Variables & secrets.

The ResourceResult envelope

Every call returns one shape:

AttributeWhat it is
result.textThe text answer (delegates to the active output variant).
result.outputThe typed answer — LlmOutput, EmbeddingOutput, SearchOutput, ExtractOutput, DocumentOutput, CrawlOutput, RecordsOutput, RawOutput, one of the five Object*Outputs, one of the three Vector*Outputs, FetchOutput, ProbeOutput, McpToolsOutput or McpToolResultOutput. Narrow with isinstance, or switch on output.kind.
result.citationsSearch hits, or None.
result.recordsDB rows from a data call, or None.
result.usage / result.usage_unitMetering (tokens / pages / requests).
result.cost / result.cost_currencyThe provider-reported cost, for self-pricing drivers only (e.g. Apify). None for LLM and metered non-LLM calls — see below.
result.modelThe resolved backend model (model-addressed resources).
result.provider / result.connected_service_idWhich provider / connected service served it.
result.metadata / result.rawExtra metadata and the full provider response.

The output variant types are importable from the package root, e.g. from im_sdk import LlmOutput, SearchOutput, DocumentOutput.

Advanced LLM usage

The tools, tool_choice, and thinking shapes are importable from im_sdk.models.

Typed (structured) output

call_structured derives a JSON schema from a pydantic model, asks the model to fill it, and returns a parsed instance of that model (not a ResourceResult):

from pydantic import BaseModel
 
class Sentiment(BaseModel):
    label: str
    score: float
 
sentiment = await core.llm.call_structured(
    messages=[{"role": "user", "content": "I love this."}],
    response_model=Sentiment,
    alias="gpt-mini",
)
sentiment.label   # -> "positive"

Tool / function calling

Pass tools (each ToolDefinition is name + optional description + input_schema) and, optionally, a ToolChoice (type is auto / any / tool / none, with name when forcing a specific tool). Tool calls come back as ToolUseBlocks in result.output.blocks:

from im_sdk.models import ToolDefinition
 
result = await core.llm.call(
    messages=[{"role": "user", "content": "Weather in Berlin?"}],
    alias="gpt-mini",
    tools=[ToolDefinition(
        name="get_weather",
        description="Look up the current weather for a city.",
        input_schema={"type": "object", "properties": {"city": {"type": "string"}}},
    )],
)
for block in result.output.blocks:
    if block.type == "tool_use":
        block.name, block.input   # dispatch it, then send the result back in a follow-up call

Extended thinking

Give reasoning models a thinking budget with a ThinkingConfig (budget_tokens); the reasoning surfaces as ThinkingBlocks and via result.output.thinking (kept out of .text). This is distinct from the reasoning_effort parameter — don't set both for the same provider.

Streaming

core.llm.stream(...) takes the same parameters as call and returns an async iterator of StreamEvents — await it, then iterate:

stream = await core.llm.stream(messages=[...], alias="gpt-mini")
async for event in stream:
    if event.type == "content_block_delta" and event.delta.type == "text_delta":
        print(event.delta.text, end="", flush=True)
    elif event.type == "message_stop":
        break

Text streams as the model produces it: the first content_block_delta arrives while the model is still writing, not after it has finished. The events, in order:

  • content_block_delta — one per piece of text, to append to content block index. Joined, a block's deltas are the text call would have returned.
  • message_delta — the stop reason and the call's token usage, once.
  • message_stop — the end.

A streamed call is authorized, routed, guarded, audited and billed exactly like the same call: same alias resolution and fallback chain, same guardrail policy, same usage and cost. Tool calls are not streamed — a tool-calling answer yields no deltas for them; use call to read them.

Errors arrive where they happen:

  • Before any text — an ungranted alias, a guardrail that blocks the request, an unreachable provider — the await raises, exactly as call would.
  • Part-way — a guardrail that blocks the model's output raises ImGuardrailBlockedError from the iteration, after the deltas that preceded the blocked text; the call's timeout (or the gateway's 900 s ceiling, which bounds the whole stream) raises ImTimeoutError; a connection that breaks raises ImTransportError, and so does a gateway fault mid-stream — the same class that fault raises on call, so code that retries on ImTransportError retries a stream and a call alike.

A stream that raised is never a complete answer, so don't treat the text you already received as one:

from im_sdk import ImGuardrailBlockedError
 
text = []
try:
    async for event in await core.llm.stream(messages=[...], alias="gpt-mini"):
        if event.type == "content_block_delta":
            text.append(event.delta.text)
except ImGuardrailBlockedError as exc:
    ...  # the policy refused the output part-way; `text` is only what came before

Leaving the loop early (break, or the agent stopping) closes the connection, and the platform stops the model call upstream. It is recorded as cancelled by the agent; tokens the provider reports for it are billed, and none are estimated for it when it reports none.

Against a platform that does not stream yet, the same sequence is produced from the complete answer — one delta per text block, all at the end — so code written against stream works either way.

Concurrency

Concurrent resource calls are supported. Tracing spans are task-local (a ContextVar), so calls fanned out with asyncio.gather (or create_task) each nest under the span active when the task forked, and spans may close in any order. Fan out core.* calls when it speeds up your agent.