Get started with Unstructured Transform for Pydantic AI

Learn how to connect the Unstructured Transform MCP server to Pydantic AI agents. Your agents can then point to your files and have Transform start producing partitioned, enriched, chunked, and embedded data based on your files in minutes.

Unlike end-user AI tools that add the Transform MCP server through a settings screen, Pydantic AI connects to it from code. An MCPToolset loads the Transform MCP server’s tools as native Pydantic AI tools, which you then pass to an agent through the agent’s toolsets argument. Because Transform is a hosted remote MCP server, there is nothing to install or run locally: you point the toolset at the server URL and authenticate with your Unstructured API key.

Requirements

You will need:

• Python 3.10 or later on your local development machine. To check, in your terminal, run python --version or python3 --version. Install Python.
• An Unstructured API key, which the Transform MCP server uses as a bearer token. Get an API key.
• A chat model provider and its API key for the agent’s underlying model. For example, an Anthropic or OpenAI key.

Install pydantic-ai

In your terminal, install pydantic-ai:

pip install -U pydantic-ai

The example below also uses httpx, which pydantic-ai installs as a dependency.

Connect to the Transform MCP server

The Transform MCP server speaks the streamable HTTP transport and authenticates with your Unstructured API key sent as a bearer token. Store your key in an environment variable rather than hard-coding it:

export UNSTRUCTURED_API_KEY="<your-unstructured-api-key>"

Create an MCPToolset that points at the server URL and passes your key through the auth argument, which sends it as an Authorization: Bearer header:

import os

from pydantic_ai.mcp import MCPToolset

transform = MCPToolset(
    "https://mcp.transform.unstructured.io",
    auth=os.environ["UNSTRUCTURED_API_KEY"],
)

MCPToolset gives you direct control of the connection, which you need in order to pass the bearer token. Pydantic AI also offers a higher-level MCP capability that takes a server URL as a one-line shortcut. Use MCPToolset when you need to configure authentication or the transport.

Use the tools in a Pydantic AI agent

Register the toolset with an agent through its toolsets argument, then open the connection with async with agent around your agent runs. Transform generates a signed upload URL for each local file, and expects the client to upload the bytes, so this example gives the agent two small local tools: one to read a file’s metadata and one to upload its bytes. The agent decides when to call each Transform tool and each local tool.

import asyncio
import mimetypes
import os
from pathlib import Path

import httpx
from pydantic_ai import Agent
from pydantic_ai.mcp import MCPToolset

transform = MCPToolset(
    "https://mcp.transform.unstructured.io",
    auth=os.environ["UNSTRUCTURED_API_KEY"],
)

agent = Agent(
    "anthropic:claude-sonnet-4-6",
    toolsets=[transform],
    system_prompt=(
        "You partition documents with the Unstructured Transform tools. "
        "To partition a local file: call file_info to get its name, MIME type, "
        "and size; pass those to request_file_upload_url; call put_file to "
        "upload the bytes to the returned signed URL; then call "
        "start_transform_job with the returned file_ref. Poll "
        "check_job_status until the job is COMPLETED, then call "
        "get_job_results and report a short summary."
    ),
)


@agent.tool_plain
def file_info(local_path: str) -> dict:
    """Return the filename, MIME type, and byte size of a local file."""
    path = Path(local_path)
    content_type = mimetypes.guess_type(path.name)[0] or "application/octet-stream"
    return {
        "filename": path.name,
        "content_type": content_type,
        "size_bytes": path.stat().st_size,
    }


@agent.tool_plain
def put_file(local_path: str, upload_url: str, content_type: str) -> str:
    """Upload the bytes of a local file to a signed upload URL."""
    data = Path(local_path).read_bytes()
    response = httpx.put(
        upload_url,
        content=data,
        headers={"Content-Type": content_type},
        timeout=60,
    )
    response.raise_for_status()
    return f"Uploaded {len(data)} bytes."


async def main() -> None:
    async with agent:
        result = await agent.run(
            "Partition the file at ./report.pdf and give me a short summary of what it contains."
        )
        print(result.output)


if __name__ == "__main__":
    asyncio.run(main())

When you run this, the agent reads the file’s metadata, requests an upload URL, uploads the file, starts the transform, polls for status until the job completes, and then fetches the results.

If your file is already reachable at a public HTTPS URL, you do not need the upload tools at all; the agent can pass the URL directly to the Transform start_transform_job tool. The upload tools shown above are only needed for files on your local machine.

Partition your source files

Transform MCP partitioning requests have the following limits:

• Each file must be of a supported file type.
• Each file must be 50 MB or less in size.
• Each request must have 10 files or fewer.
• Only 5 requests can be running at a time.

The Transform MCP server is designed to report these limits back to the agent through its tool responses. Because of this, your agent should notify you whenever it encounters a file that exceeds 50 MB in size, and it should formulate strategies to send requests that are 10 files or fewer and not cause more than 5 requests to be running at a time. You can reinforce this behavior in your agent’s system prompt.

Authenticate each user with their own API key

The bearer-token approach shown above is the simplest and works well for scripts, backends, and shared-service setups. In a multi-user application where each user has their own Unstructured credentials, build the toolset per run so that each run authenticates as the right user. Use the @agent.toolset decorator to read the user’s token from the run’s dependencies.

from dataclasses import dataclass

from pydantic_ai import Agent, RunContext
from pydantic_ai.mcp import MCPToolset


@dataclass
class UserDeps:
    unstructured_api_key: str


agent = Agent("anthropic:claude-sonnet-4-6", deps_type=UserDeps)


@agent.toolset
def transform_toolset(ctx: RunContext[UserDeps]) -> MCPToolset:
    return MCPToolset(
        "https://mcp.transform.unstructured.io",
        auth=ctx.deps.unstructured_api_key,
    )

The auth argument also accepts the literal string 'oauth' to enable an interactive OAuth flow instead of a static token. See the MCPToolset documentation for details.

Troubleshooting

• 401 Unauthorized on tool calls. Confirm the UNSTRUCTURED_API_KEY environment variable is set and that you are passing it to MCPToolset through the auth argument.
• McpError: Session terminated or 404 Not Found when connecting. Verify the server URL is exactly https://mcp.transform.unstructured.io, with no /mcp path suffix, and check that your network allows outbound HTTPS to mcp.transform.unstructured.io.
• ModuleNotFoundError: pydantic_ai. Re-run pip install -U pydantic-ai in the same Python environment that runs your agent.
• ImportError for MCPToolset. Upgrade to a recent version of pydantic-ai with pip install -U pydantic-ai.
• RuntimeError about the event loop. agent.run(...) and async with agent are async. Call them from an async function (for example, via asyncio.run(main())) rather than at the top level of a script.

Next steps

• Control Transform file partitioning output: Control how the Unstructured Transform MCP server instructs Transform to partition, enrich, chunk, and embed the data based on your files.
• Control Transform structured data extraction: Control how the Unstructured Transform MCP server extracts and formats structured data from your files.
• Control Transform generated sample code: Control how the Unstructured Transform MCP server generates sample curl or Python code that demonstrates how to use Transform to partition, enrich, chunk, and embed the data based on your files.

Questions? Need help?

• For technical support, request support.