Get started with Unstructured Transform for Vercel AI SDK

Learn how to connect the Unstructured Transform MCP server to the Vercel AI SDK. Your agents can then point to your files and have Transform start producing partitioned, enriched, chunked, and embedded data based on your files in minutes.

The Vercel AI SDK is an open-source TypeScript toolkit for building AI applications and agents. Unlike end-user AI tools that add the Transform MCP server through a settings screen, the AI SDK connects to it from code: you load the Transform MCP server’s tools through the SDK’s built-in MCP client and hand them to a model call such as generateText or streamText. Because Transform is a hosted remote MCP server, there is nothing to install or run locally: you point the client at the server URL and authenticate with your Unstructured API key.

If you want a running app rather than a script, deploy the ready-made Transform MCP × Vercel AI SDK template. It is a Next.js chat app wired to the Transform MCP server, with a one-click Deploy to Vercel button that prompts for the two API keys below.

Requirements

You will need:

• Node.js 18 or later on your local development machine. To check, in your terminal, run node --version. Install Node.js.
• An Unstructured API key, which the Transform MCP server uses as a bearer token. Get an API key.
• An Anthropic API key, or another model provider that the AI SDK supports, for the agent’s underlying chat model.

Install the packages

In your terminal, install the AI SDK, the Anthropic provider, the AI SDK MCP client, and zod for the local tool schema. tsx runs the TypeScript example directly:

npm install ai @ai-sdk/anthropic @ai-sdk/mcp zod
npm install --save-dev tsx

This guide was verified against ai@7, @ai-sdk/anthropic@4, and @ai-sdk/mcp@2. Newer versions typically work, but check the AI SDK release notes if an API has changed.

Connect to the Transform MCP server

The Transform MCP server speaks the streamable HTTP transport and authenticates with your Unstructured API key sent as a bearer token. Store your key in an environment variable rather than hard-coding it:

export UNSTRUCTURED_API_KEY="<your-unstructured-api-key>"

Create an MCP client with the http transport, then load the server’s tools with mcpClient.tools():

import { createMCPClient } from '@ai-sdk/mcp';

const mcpClient = await createMCPClient({
  transport: {
    type: 'http',
    url: 'https://mcp.transform.unstructured.io',
    headers: { Authorization: `Bearer ${process.env.UNSTRUCTURED_API_KEY}` },
  },
});

// Loads the Transform tools: request_file_upload_url, start_transform_job,
// check_job_status, get_job_results, suggest_extraction_schema_for_file,
// start_extraction_job, and get_instructions.
const tools = await mcpClient.tools();

Always close the client with await mcpClient.close() when the request finishes (use a finally block, or streamText’s onFinish callback) so the connection is released.

Add local helper tools for waiting and downloading

Transform’s job lifecycle needs two things the MCP tools do not cover on their own. First, transforms run as async jobs, so the agent must pace its status polling; without a pause it will call check_job_status many times in a row and exhaust its step budget before the job finishes. Second, get_job_results returns a pre-signed download_url rather than the partitioned text inline, so retrieving the content is a plain HTTP GET. The example below registers two small local tools alongside the MCP tools:

• wait: Pauses between status checks so a multi-second transform has time to complete.
• downloadText: Fetches the finished Markdown from the pre-signed download_url with an HTTP GET.

Pre-signed URLs carry their own credentials in the URL itself. The downloadText tool must not send the Authorization header, or the storage service rejects the request. It also restricts downloads to the Transform host, so a crafted prompt cannot turn the tool into a server-side request forgery (SSRF) vector.

Use the tools in an agent

The following example connects to the Transform MCP server, registers the two helper tools, and lets the model drive the full job lifecycle: start the transform, poll for status with paced waits, fetch the results, and download the Markdown output. Save it as transform-agent.ts:

import { createMCPClient } from '@ai-sdk/mcp';
import { anthropic } from '@ai-sdk/anthropic';
import { generateText, stepCountIs, tool } from 'ai';
import { z } from 'zod';

const TRANSFORM_MCP_URL = 'https://mcp.transform.unstructured.io';

const SYSTEM_PROMPT = `You partition documents with Unstructured's Transform MCP tools.
Transforms are ASYNC jobs. Follow this exact sequence:
1. Call start_transform_job with the document URL, output format markdown.
2. Call the "wait" tool (a few seconds), THEN check_job_status. Repeat
   wait -> check until the status is COMPLETED. Never poll without waiting first.
3. Call get_job_results, then downloadText on the first file's download_url
   to read the Markdown. Summarize the content for the user.`;

const wait = tool({
  description:
    'Pause for a few seconds before polling an async job again. Use between check_job_status calls.',
  inputSchema: z.object({ seconds: z.number().min(1).max(10) }),
  execute: async ({ seconds }) => {
    await new Promise((resolve) => setTimeout(resolve, seconds * 1000));
    return { waited: seconds };
  },
});

// Restrict downloads to the Transform host (where the pre-signed download_url is
// served) so a crafted prompt cannot turn this tool into an SSRF vector.
const ALLOWED_DOWNLOAD_HOST = new URL(TRANSFORM_MCP_URL).host;

const downloadText = tool({
  description:
    'Download transformed output from a pre-signed download_url with an HTTP GET.',
  inputSchema: z.object({ url: z.string().url() }),
  execute: async ({ url }) => {
    const parsed = new URL(url);
    if (parsed.protocol !== 'https:' || parsed.host !== ALLOWED_DOWNLOAD_HOST) {
      throw new Error(`Refusing to download from "${parsed.host}".`);
    }
    // The URL is pre-signed: do not send an Authorization header here.
    const response = await fetch(parsed);
    if (!response.ok) throw new Error(`Download failed: ${response.status}`);
    return await response.text();
  },
});

async function main() {
  const mcpClient = await createMCPClient({
    transport: {
      type: 'http',
      url: TRANSFORM_MCP_URL,
      headers: { Authorization: `Bearer ${process.env.UNSTRUCTURED_API_KEY}` },
    },
  });

  try {
    const tools = { ...(await mcpClient.tools()), wait, downloadText };

    const { text } = await generateText({
      model: anthropic('claude-opus-4-8'),
      system: SYSTEM_PROMPT,
      tools,
      // Async jobs need several wait -> poll cycles; give the loop room.
      stopWhen: stepCountIs(25),
      prompt:
        'Partition this document into clean markdown and summarize its sections. ' +
        'URL: https://raw.githubusercontent.com/Unstructured-IO/unstructured/main/example-docs/pdf/layout-parser-paper.pdf',
    });

    console.log(text);
  } finally {
    await mcpClient.close();
  }
}

main().catch((error) => {
  console.error(error);
  process.exit(1);
});

To use a different model provider, swap anthropic('claude-opus-4-8') for any provider the AI SDK supports, and set that provider’s API key.

Partition your source files

Transform MCP partitioning requests have the following limits:

• Each file must be of a supported file type.
• Each file must be 50 MB or less in size.
• Each request must have 10 files or fewer.
• Only 5 requests can be running at a time.

The Transform MCP server is designed to report these limits back to the agent through its tool responses. Because of this, your agent should notify you whenever it encounters a file that exceeds 50 MB in size, and it should formulate strategies to send requests that are 10 files or fewer and not cause more than 5 requests to be running at a time. You can reinforce this behavior in the system prompt, as the example above does for the polling sequence.

Run the agent

Set your model provider’s key, then run the example. Transforms take from about 10 seconds to several minutes, depending on page count:

export ANTHROPIC_API_KEY="<your-anthropic-api-key>"

npx tsx transform-agent.ts

The agent works through the tool sequence: start_transform_job, paced wait and check_job_status cycles, get_job_results, and downloadText. It then prints a summary of the partitioned Markdown.

Extract structured data

Transform can also pull specific fields out of a document and return them as JSON matching a schema, rather than converting the whole document. Two MCP tools cover this:

• suggest_extraction_schema_for_file: Drafts a JSON Schema from one partitioned document, for when you do not have a schema yet.
• start_extraction_job: Runs the extraction against a schema and returns a job_id.

Both tools read the element JSON that a partition produces, not the raw file, so an extraction always follows a transform job. Every file entry that get_job_results returns carries a durable output_ref (a u10d://output/... value) alongside its download_url. That output_ref is the input to the extraction tools, and it is present whichever output format you rendered, so the agent does not need to download the partitioned output before extracting.

Extraction reuses check_job_status and get_job_results, so it needs no new helper tools. The downloadText helper is not used here: extraction results are returned inline rather than behind a pre-signed URL.

Point the system prompt at the sequence:

const EXTRACTION_SYSTEM_PROMPT = `You extract structured data with Unstructured's Transform MCP tools.
Follow this exact sequence:
1. Call start_transform_job with the document URL, and stages
   {"partition": {"strategy": "vlm"}} for an image, PowerPoint file, or PDF.
2. Call the "wait" tool, THEN check_job_status. Repeat until COMPLETED.
3. Call get_job_results and keep each file's output_ref. Do NOT downloadText it.
4. If the user gave you no schema, call suggest_extraction_schema_for_file with
   that output_ref and show the draft schema before extracting.
5. Call start_extraction_job with element_json_refs set to the output_refs, and
   schema_to_extract set to the JSON Schema passed as a JSON string.
6. Poll with wait -> check_job_status, then call get_job_results. Results come
   back inline, wrapped with filename, filetype, processed_date_utc, and
   source_file_uri alongside extracted_data. Report the filename with each
   object and do not strip that wrapper.`;

One schema applies to every reference in a single start_extraction_job call, so batch only documents of the same kind. The transform limits carry over: up to 10 references per call, and 5 active jobs at a time.

Extraction can only surface what the partition captured, so partition quality sets the ceiling. For images, PowerPoint, and PDFs, start with strategy: "vlm". If a result comes back sparse, re-partition with strategy: "hi_res" plus the image_description, generative_ocr, and table_to_html enrichments, then extract from the new output_ref. For prompt patterns, see Structured data extraction.

Troubleshooting

• 401 Unauthorized or an invalid_token error on connect. Confirm the UNSTRUCTURED_API_KEY environment variable is set and that the Authorization header is formatted as Bearer <your-unstructured-api-key>.
• 404 or a not-found error when connecting. Verify the server URL is exactly https://mcp.transform.unstructured.io, with no path such as /mcp appended.
• The agent stops before the job finishes. Each model step counts toward stopWhen: stepCountIs(...), and each poll cycle uses two steps (wait plus check_job_status). Raise the step count, or increase the seconds the wait tool pauses, for longer transforms. An extraction runs a second job after the partition, so a partition-then-extract prompt needs roughly twice the step budget of a partition alone.
• An extraction is rejected as invalid input. The extraction tools accept the output_ref from a completed transform job, not a file URL and not a file_ref. Partition the document first, then pass its output_ref in element_json_refs. Element JSON that carries embeddings is also rejected, so extract from partition or chunk output rather than the output of an embed stage.
• An extraction returns empty or mostly null fields. This is usually the partition rather than the schema. Re-partition the file at higher fidelity, as described in Extract structured data, and extract from the new output_ref.
• The download request is rejected. The download_url value is pre-signed. Send that request without the Authorization header.
• A hanging or leaked connection. Call await mcpClient.close() when the request finishes, in a finally block or streamText’s onFinish callback.

Next steps

• Control Transform file partitioning output: Control how the Unstructured Transform MCP server instructs Transform to partition, enrich, chunk, and embed the data based on your files.
• Control Transform structured data extraction: Control how the Unstructured Transform MCP server extracts and formats structured data from your files.
• Control Transform generated sample code: Control how the Unstructured Transform MCP server generates sample curl or Python code that demonstrates how to use Transform to partition, enrich, chunk, and embed the data based on your files.

Questions? Need help?

• For technical support, request support.