Get started with Unstructured Transform for CrewAI

Learn how to connect your CrewAI agents to the Unstructured Transform MCP server so they can partition and extract data from your files.

Unlike end-user AI tools that add the Transform MCP server through a settings screen, CrewAI connects to it from code. The MCPServerAdapter class from crewai-tools loads the Transform MCP server’s tools as native CrewAI tools. Pass them to any CrewAI agent. Because Transform is a hosted remote MCP server, there is nothing to install or run locally. Point the adapter at the server URL and authenticate with your Unstructured API key.

Requirements

You will need:

• Python 3.12 to 3.13 on your local development machine. Install Python.
  • CrewAI may support older Python versions, but mcpadapt, the MCP adapter layer used internally by crewai-tools, requires Python 3.12 or later.
  • The supported Python version range may have changed since this page was last published. For the current supported range, see Getting started with the installation in the CrewAI documentation.
• An Unstructured API key, which the Transform MCP server uses as a bearer token. To get one, see Get your API key and API URL.
• An LLM provider and its API key (for example, an Anthropic or OpenAI key) for the agent’s underlying model. See LLMs in the CrewAI documentation for supported providers.
• A file to partition. You will provide the path to your file at the top of the example script.

Install the packages

In your terminal, install crewai, crewai-tools with MCP support, and a compatible version of the MCP SDK:

pip install -U "crewai[<LLM-provider>]>=0.108.0" "crewai-tools[mcp]>=0.76.0" "mcp>=1.10.1,<2.0.0" httpx

The crewai-tools[mcp] package is required for this procedure. It provides MCPServerAdapter, the class that loads the Transform MCP server’s tools as native CrewAI tools. The [mcp] extras flag installs the additional dependencies needed for MCP transport support.

Replace <LLM-provider> with the name for your LLM provider. For example:

• Anthropic:
  pip install -U "crewai[anthropic]>=0.108.0" "crewai-tools[mcp]>=0.76.0" "mcp>=1.10.1,<2.0.0" httpx
  
• OpenAI:
  pip install -U "crewai[openai]>=0.108.0" "crewai-tools[mcp]>=0.76.0" "mcp>=1.10.1,<2.0.0" httpx
  

For a full list of supported providers, see LLMs in the CrewAI documentation. For additional built-in CrewAI tools beyond MCP, see crewai-tools on PyPI.

The mcp package version 2.0.0 and later introduced a change that causes MCPServerAdapter to fail with an ImportError when connecting to a remote MCP server. The install command above pins mcp below version 2.0.0 to avoid this issue. If you see an ImportError after installing, check that your mcp version is below 2.0.0 by running pip show mcp.

To verify the installation, run:

crewai --version

Connect to the Transform MCP server

The Transform MCP server speaks the streamable HTTP transport and authenticates with your Unstructured API key sent as a bearer token. Store your key in an environment variable rather than hard-coding it. To get your key, see Get your API key and API URL.

export UNSTRUCTURED_API_KEY="<your-unstructured-api-key>"

Pass the server configuration to MCPServerAdapter inside a with block. The adapter is synchronous, so there is no async with:

import os

from crewai_tools import MCPServerAdapter

with MCPServerAdapter(
    {
        "url": "https://mcp.transform.unstructured.io",
        "transport": "streamable-http",
        "headers": {"Authorization": f"Bearer {os.environ['UNSTRUCTURED_API_KEY']}"},
    },
    connect_timeout=60,
) as mcp_tools:
    # mcp_tools is a list of native CrewAI tools loaded from the Transform server.
    print([t.name for t in mcp_tools])

The transport string is "streamable-http" with a hyphen, not an underscore. Using "streamable_http" will produce a transport-not-found error at connection time.

Use the tools in a CrewAI agent

This example partitions a local file with the Transform MCP tools, walking through the full sequence from file upload to results:

• Defines two local tools the agent can call: one to upload a local file to a pre-signed URL, and one to download partitioned results from a pre-signed URL.
• Creates a CrewAI agent with access to both local tools and all the Transform MCP tools. The agent uses an LLM to decide when to call each tool.
• Gives the agent a task: partition a file and return a short summary. The task description walks the agent through the full sequence: request an upload URL, upload the file, start the job, poll for status, download the results.
• Runs the crew, which kicks off the agent and prints the result.

Set your API key environment variables

Before you run this example, set your API keys as environment variables:

• Your Unstructured API key as UNSTRUCTURED_API_KEY. If you followed the Connect to the Transform MCP server section, this is already set. To confirm:

  echo $UNSTRUCTURED_API_KEY
  

• Your LLM provider API key as the appropriate environment variable.

  • Anthropic example:
    export ANTHROPIC_API_KEY="<your-anthropic-api-key>"
    
  • OpenAI example:
    export OPENAI_API_KEY="<your-openai-api-key>"
    

  For the correct environment variable name for your provider, see LLMs in the CrewAI documentation.

To confirm the variable is set:

echo $ANTHROPIC_API_KEY

Replace ANTHROPIC_API_KEY with the variable name for your provider.

CrewAI uses LiteLLM to connect to LLM providers. LiteLLM reads your provider API key automatically from the environment based on the provider prefix in the script variable LLM_MODEL (set in the next section). You do not need to pass your API key explicitly in the script.

Update the script variables

Then open the script and update the two variables at the top:

1. Set FILE_PATH to the path of the file you want the Transform MCP server to partition.
2. Set LLM_MODEL to the model string for your LLM provider. The example defaults to "anthropic/claude-sonnet-4-6".

Run the script

import mimetypes
import os
from pathlib import Path

FILE_PATH = "./report.pdf"  # Replace with the path to your file.
LLM_MODEL = "anthropic/claude-sonnet-4-6"  # Format: provider/model-name. See https://docs.crewai.com/concepts/llms

import httpx
from crewai import LLM, Agent, Crew, Task
from crewai.tools import tool
from crewai_tools import MCPServerAdapter


@tool("Upload file to pre-signed URL")
def upload_file(local_path: str, upload_url: str) -> str:
    """Upload a local file's bytes to a pre-signed upload URL."""
    path = Path(local_path)
    content_type = mimetypes.guess_type(path.name)[0] or "application/octet-stream"
    data = path.read_bytes()
    response = httpx.put(
        upload_url,
        content=data,
        headers={"Content-Type": content_type},
        timeout=60,
    )
    response.raise_for_status()
    return f"Uploaded {len(data)} bytes."


@tool("Download results from pre-signed URL")
def download_results(download_url: str) -> str:
    """Fetch partitioned results from a pre-signed download URL."""
    response = httpx.get(download_url, timeout=60)
    response.raise_for_status()
    return response.text


def main() -> None:
    with MCPServerAdapter(
        {
            "url": "https://mcp.transform.unstructured.io",
            "transport": "streamable-http",
            "headers": {"Authorization": f"Bearer {os.environ['UNSTRUCTURED_API_KEY']}"},
        },
        connect_timeout=60,
    ) as mcp_tools:
        llm = LLM(model=LLM_MODEL)

        agent = Agent(
            role="Document Processor",
            goal="Partition documents with the Unstructured Transform tools and return a summary.",
            backstory=(
                "You are an expert at using the Unstructured Transform MCP tools to partition "
                "and extract information from documents."
            ),
            tools=[*mcp_tools, upload_file, download_results],
            llm=llm,
            verbose=True,
        )

        task = Task(
            description=(
                f"Partition the file at {FILE_PATH} using the Unstructured Transform tools. "
                "To do this: call request_file_upload_url to get a signed upload URL and a "
                "file_ref; call upload_file with the local path and that upload URL to send "
                "the bytes (do not include an Authorization header; the URL is pre-signed); "
                "call start_transform_job with the file_ref; poll check_job_status until the "
                "job status is COMPLETED; call get_job_results to obtain the download URL; "
                "call download_results with that URL to retrieve the partitioned content. "
                "Return a short summary of the document."
            ),
            agent=agent,
            expected_output="A short summary of the document's contents.",
        )

        crew = Crew(agents=[agent], tasks=[task], verbose=True)
        result = crew.kickoff()
        print(result)


if __name__ == "__main__":
    main()

When you run this, the agent requests an upload URL, uploads the file, and starts the transform job. It then polls for status, fetches the results, and returns a summary.

If your file is already reachable at a public HTTPS URL, pass the URL directly to start_transform_job. No upload tools are needed. If you do upload a local file, do not include an Authorization header in the httpx.put() call. Pre-signed upload URLs encode their own credentials; adding an Authorization header causes the upload to fail with a 403 error.

Partition your source files

Transform MCP partitioning requests have the following limits:

• Each file must be of a supported file type.
• Each file must be 50 MB or less in size.
• Each request must have 10 files or fewer.
• Only 5 requests can be running at a time.

The Transform MCP server is designed to report these limits back to the agent through its tool responses. Because of this, your agent should notify you when a file exceeds 50 MB. It should also batch requests to 10 files or fewer and keep no more than 5 requests running at a time. You can reinforce this behavior in your agent’s task description or backstory.

Troubleshooting

• crewai: command not found or ModuleNotFoundError: No module named 'crewai'. The install instructions on this page do not use a virtual environment, but if you chose to set one up yourself, it may not be activated. Run source <venv>/bin/activate on Mac or Linux, or <venv>\Scripts\activate on Windows, then retry.
• ImportError: Unable to initialize LLM with model '...'. The LLM_MODEL string has an unrecognized provider prefix or model name. The error message suggests installing LiteLLM, but the fix is to correct the provider/model-name string. For supported providers and their model name formats, see LLMs in the CrewAI documentation.
• ImportError: cannot import name 'streamablehttp_client'. The mcp package version 2.0.0 and later introduced a change that causes this error. Confirm your mcp version is below 2.0.0 by running pip show mcp, then downgrade with pip install "mcp>=1.10.1,<2.0.0" if needed.
• 401 Unauthorized on tool calls. Confirm the UNSTRUCTURED_API_KEY environment variable is set and that the Authorization header is formatted as Bearer <your-unstructured-api-key>.
• 403 Forbidden when uploading a file. You included an Authorization header on the httpx.put() call for the pre-signed upload URL. Remove it. The URL encodes its own credentials.
• Connection timeout when opening the adapter. Increase connect_timeout in the MCPServerAdapter call (the default is low). A value of 60 is usually sufficient for the hosted Transform server.
• ValueError: Unknown transport 'streamable_http'. The transport string requires a hyphen: "streamable-http", not an underscore.
• mcp_tools is empty after connecting. Verify the URL is exactly https://mcp.transform.unstructured.io and that your network allows outbound HTTPS to mcp.transform.unstructured.io.
• RuntimeError: Python 3.12 or later is required. The mcpadapt library that backs streamable HTTP support requires Python 3.12 or later, even though CrewAI itself supports Python 3.10. Upgrade your Python version.
• 502 Bad Gateway during polling. This is a transient server error. The transform job continues running server-side and is not lost. Wait a moment, then re-run the script to submit a new job.
• Task was destroyed but it is pending! or RuntimeWarning messages after the script finishes. These are expected cleanup messages from the interaction between mcpadapt and the MCP SDK when the adapter closes. They do not affect your results and can be safely ignored.

Next steps

• Control Transform file partitioning output: Control how the Unstructured Transform MCP server instructs Transform to partition, enrich, chunk, and embed the data based on your files.
• Control Transform structured data extraction: Control how the Unstructured Transform MCP server extracts and formats structured data from your files.
• Control Transform generated sample code: Control how the Unstructured Transform MCP server generates sample curl or Python code that demonstrates how to use Transform to partition, enrich, chunk, and embed the data based on your files.

Questions? Need help?

• For technical support, request support.