Skip to main content

AI SDK by Vercel integration

View Markdown

Temporal's integration with Vercel's AI SDK lets you use the AI SDK's API directly in Workflow code while Temporal handles Durable Execution.

Like all API calls, LLM API calls are non-deterministic. In a Temporal Application, that means you cannot make LLM calls directly from a Workflow; they must run as Activities. The AI SDK plugin handles this automatically: when you call methods in the AI SDK such as generateText(), streamText(), or streamObject(), the plugin wraps those calls in Activities behind the scenes. This preserves the Vercel AI SDK's developer experience that you are already familiar with while Temporal handles Durable Execution for you.

info

Import the plugin (AiSdkPlugin) from @temporalio/ai-sdk in your Worker setup, but import Workflow-side helpers such as temporalProvider and TemporalMCPClient from the @temporalio/ai-sdk/workflow subpath. The /workflow subpath is safe to use in the deterministic Workflow sandbox.

All code snippets in this guide are taken from the TypeScript SDK ai-sdk samples. Refer to the samples for the complete code and run them locally.

Prerequisites​

  • This guide assumes you are already familiar with the Vercel AI SDK. If you aren't, refer to the Vercel AI SDK documentation for more details.
  • If you are new to Temporal, we also recommend you read the Understanding Temporal document or take the Temporal 101 course to understand the basics of Temporal.
  • Ensure you have set up your local development environment by following the Set up your local with the TypeScript SDK guide. When you are done, leave the Temporal Development Server running if you want to test your code locally.

Configure Workers to use the AI SDK​

Workers are the compute layer of a Temporal Application. They are responsible for executing the code that defines your Workflows and Activities. Before you can execute a Workflow or Activity with the Vercel AI SDK, you need to create a Worker and configure it to use the AI SDK plugin.

Follow the steps below to configure your Worker.

  1. Install the @temporalio/ai-sdk package.

    npm install @temporalio/ai-sdk

  2. Create a worker.ts file and configure the Worker to use the AI SDK plugin.

    import { openai } from '@ai-sdk/openai';
    import { AiSdkPlugin } from '@temporalio/ai-sdk';

    //... other import statements, initializing a connection
    //  to the Temporal Service to be used by the Worker

    const worker = await Worker.create({
      plugins: [
        new AiSdkPlugin({
          modelProvider: openai,
        }),
      ],
      connection,
      namespace: 'default',
      taskQueue: 'ai-sdk',
      workflowsPath: require.resolve('./workflows'),
      activities,
    });

    // ... code that runs the worker

    The modelProvider specifies which AI provider to use when creating models. Choose the provider that best suits your needs. In the Worker options, you are also specifying that the Worker polls the ai-sdk Task Queue for work in the default Namespace. Make sure that you configure your Client application to use the same Task Queue and Namespace.

  3. Run the Worker. This Worker will now poll the Temporal Service for work on the ai-sdk Task Queue in the default Namespace until you stop it.

    nodemon worker.ts

    You must ensure the Worker process has access to your API credentials. Most provider SDKs read credentials from environment variables. Refer to the Vercel AI SDK documentation for instructions on how to set up your environment variables for the provider you chose.

    tip

    You only need to give provider credentials to the Worker process. The client application, meaning the application that sends requests to the Temporal Service to start Workflow Executions, doesn't need to know about the credentials.

See the full example at ai-sdk samples.

Develop a Simple Haiku Agent​

To help you get started, you can develop a simple Haiku Agent that generates haikus based on a prompt.

If you weren't using Temporal, you would write code like this to generate a haiku:

import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';

async function haikuAgent(prompt: string): Promise<string> {
  const result = await generateText({
    model: openai('gpt-4o-mini'),
    prompt,
    system: 'You only respond in haikus.',
  });
  return result.text;
}

To add Durable Execution to your agent, implement the agent as a Temporal Workflow. Use the AI SDK as you normally would, but pass temporalProvider.languageModel() as the model. The string you provide (like 'gpt-4o-mini') is passed to your configured modelProvider to create the model.

import { generateText } from 'ai';
import { temporalProvider } from '@temporalio/ai-sdk/workflow';

export async function haikuAgent(prompt: string): Promise<string> {
  const result = await generateText({
    model: temporalProvider.languageModel('gpt-4o-mini'),
    prompt,
    system: 'You only respond in haikus.',
  });
  return result.text;
}

With only two line changes, you have added Durable Execution to your agent. Your agent now gets automatic retries, timeouts, and the ability to run for extended periods without losing state if the process crashes.

Provide your durable agent with tools​

The Vercel AI SDK lets you provide tools to your agents, and when the model calls them, they execute in the Workflow. Since tool functions run in Workflow context, they must follow Workflow rules. That means they must call Activities or Child Workflows to perform non-deterministic operations like API calls.

For example, if you want to call an external API to get the weather, you would implement it as an Activity and call it from the tool function. The following is an example of an Activity that gets the weather for a given location:

ai-sdk/src/activities.ts

export async function getWeather(input: {
  location: string;
}): Promise<{ city: string; temperatureRange: string; conditions: string }> {
  return {
    city: input.location,
    temperatureRange: '14-20C',
    conditions: 'Sunny with wind.',
  };
}

Then in your agent implementation, provide the tool to the model using the tools option and instruct the model to use the tool when needed.

import { proxyActivities } from '@temporalio/workflow';
import { generateText, tool } from 'ai';
import { temporalProvider } from '@temporalio/ai-sdk/workflow';
import { z } from 'zod';

const { getWeather } = proxyActivities<typeof activities>({
  startToCloseTimeout: '1 minute',
});

export async function toolsAgent(question: string): Promise<string> {
  const result = await generateText({
    model: temporalProvider.languageModel('gpt-4o-mini'),
    prompt: question,
    system: 'You are a helpful agent.',
    tools: {
      getWeather: tool({
        description: 'Get the weather for a given city',
        inputSchema: z.object({
          location: z.string().describe('The location to get the weather for'),
        }),
        execute: getWeather,
      }),
    },
    stopWhen: stepCountIs(5),
  });
  return result.text;
}

Integrate with Model Context Protocol (MCP) servers​

Model Context Protocol (MCP) is an open standard that lets AI applications connect to external tools and data sources. Calls to MCP servers, being calls to external APIs, are non-deterministic and would usually need to be implemented as Activities. The Temporal AI SDK integration handles this for you and provides a built-in implementation of a stateless MCP client that you can use inside Workflows.

Follow the steps below to integrate your agent with an MCP server.

  1. Create a connection to the MCP servers using the experimental_createMCPClient function from the @ai-sdk/mcp package. You can register multiple MCP servers by providing multiple factory functions in mcpClientFactories.

    import { experimental_createMCPClient as createMCPClient } from '@ai-sdk/mcp';
    import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';

    const mcpClientFactories = {
      testServer: () =>
        createMCPClient({
          transport: new StdioClientTransport({
            command: 'node',
            args: ['lib/mcp-server.js'],
          }),
        }),
    };

    The example uses StdioClientTransport as the transport mechanisms for client-server communication. Each time the Worker processes a Task that requires communication with the MCP server, it will start the server process and connect to it as required by the Task.

  2. Configure the Worker to use the MCP client factories.

    const worker = await Worker.create({
      plugins: [
        new AiSdkPlugin({
          modelProvider: openai,
          mcpClientFactories
        }),
      ]},
      ...
    );

  3. In your agent Workflow, use TemporalMCPClient to get tools from the MCP server by referencing it by name:

    import { TemporalMCPClient, temporalProvider } from '@temporalio/ai-sdk/workflow';

    export async function mcpAgent(prompt: string): Promise<string> {
      const mcpClient = new TemporalMCPClient({ name: 'testServer' });
      const tools = await mcpClient.tools();
      const result = await generateText({
        model: temporalProvider.languageModel('gpt-4o-mini'),
        prompt,
        tools,
        system: 'You are a helpful agent, You always use your tools when needed.',
        stopWhen: stepCountIs(5),
      });
      return result.text;
    }

    Both listing tools and calling them run as Activities behind the scenes, giving you automatic retries, timeouts, and full observability.

Stream model output from a Workflow​

The AI SDK's streamText() and streamObject() functions stream a response incrementally instead of returning it all at once. The plugin runs the model call in an Activity and publishes each delta onto a Workflow Stream topic. External consumers can subscribe to that topic by Workflow ID to render tokens live as they arrive, while the Workflow durably reassembles the final result.

info

Inside the Workflow, the streamed result is reassembled after the Activity completes, so the Workflow's own textStream / partialObjectStream is not incremental. Live, token-by-token deltas are delivered to external consumers through a WorkflowStreamClient. This is inherent to Durable Execution: the Activity must run to completion before the Workflow can observe a deterministic, replayable result.

Streaming uses the @temporalio/workflow-streams package. Install it alongside the AI SDK plugin:

npm install @temporalio/workflow-streams

To stream text, configure a provider with a streamingTopic (this enables streaming and names the topic that deltas are published to), host a WorkflowStream in your Workflow, and consume the AI SDK stream as usual:

import { streamText } from 'ai';
import { TemporalProvider } from '@temporalio/ai-sdk/workflow';
import { WorkflowStream } from '@temporalio/workflow-streams/workflow';

export const STREAM_TOPIC = 'text-stream';

const streamingProvider = new TemporalProvider({
  languageModel: { streamingTopic: STREAM_TOPIC },
});

ai-sdk/src/workflows.ts

export async function streamingAgent(prompt: string): Promise<string> {
  // Host the WorkflowStream as the first statement of the workflow so its
  // publish-signal handler is registered before the streaming activity starts
  // publishing deltas to it.
  new WorkflowStream();

  const result = streamText({
    model: streamingProvider.languageModel('gpt-4o-mini'),
    prompt,
    system: 'You only respond in haikus.',
  });

  // The model call runs in an activity that publishes each delta to
  // STREAM_TOPIC for external subscribers. Inside the workflow the deltas are
  // replayed after the activity completes, so this loop durably reassembles
  // the full text.
  let text = '';
  for await (const delta of result.textStream) {
    text += delta;
  }

  // Briefly hold the run open so a subscriber's final poll can deliver the
  // last deltas before the workflow exits and its in-memory stream log is gone.
  await sleep('500 milliseconds');
  return text;
}

The consumer subscribes to the topic by Workflow ID and renders each delta as it arrives. Each item's payload is the JSON-encoded AI SDK stream part:

ai-sdk/src/client.ts

// Subscribe to a Workflow's stream topic and render each text delta live as it
// is published by the streaming activity. Each item's payload is the
// JSON-encoded AI SDK stream part; `resultType: true` decodes it to raw bytes.
async function renderStream(client: Client, workflowId: string, topic: string): Promise<void> {
  const streamClient = WorkflowStreamClient.create(client, workflowId);
  for await (const item of streamClient.subscribe<Uint8Array>(topic, 0, { resultType: true })) {
    const part = JSON.parse(new TextDecoder().decode(item.data));
    if (part.type === 'text-delta') process.stdout.write(part.delta);
    if (part.type === 'finish') break;
  }
  process.stdout.write('\n');
}

Stream structured output​

streamObject() streams a structured object as its JSON is generated. It flows through the same model path as streamText(), so no extra wiring is needed beyond choosing a distinct streamingTopic so concurrent streams stay separable. External subscribers see the object build up incrementally, and the Workflow durably resolves the final, validated object.

ai-sdk/src/workflows.ts

export async function streamObjectAgent(prompt: string): Promise<string> {
  new WorkflowStream();

  const result = streamObject({
    model: objectStreamingProvider.languageModel('gpt-4o-mini'),
    schema: z.object({
      recipe: z.object({
        name: z.string(),
        ingredients: z.array(z.object({ name: z.string(), amount: z.string() })),
        steps: z.array(z.string()),
      }),
    }),
    prompt,
  });

  // External subscribers see the object build up incrementally via the partial
  // JSON deltas published to OBJECT_STREAM_TOPIC. The workflow drains the
  // partial stream and durably resolves the final, validated object.
  for await (const _partial of result.partialObjectStream) {
    // Draining drives the stream; the consumer renders the live partials.
  }
  const object = await result.object;

  await sleep('500 milliseconds');
  return object.recipe.name;
}

See the full example at ai-sdk samples.