AI Vercel v5 SDK notes
Simple example
import {anthropic} from '@ai-sdk/anthropic';
import {generateText} from 'ai';
// 👇 Here we can choose the model
const model = anthropic('claude-3-5-haiku-latest');
export const answerMyQuestion = async (prompt: string) => {
const {text} = await generateText({
model,
prompt,
});
return text;
};
const answer = await answerMyQuestion('What is the capital of France?');
Stream text
We can stream text token by token
import { streamText } from "ai";
const { textStream } = await streamText({
model,
prompt,
});
// Stream UI Mesasge
for await (const chunk of stream.toUIMessageStream() {
console.log(chunk)
}
// Output
{ type: 'start' }
{ type: 'start-step' }
{ type: 'text-start', id: '0' }
{ type: 'text-delta', id: '0', delta: 'Upon' }
{
type: 'text-delta',
id: '0',
delta: ' a cushion, soft and warm he lies,\nOur Steven, sleek and ginger'
}
{
type: 'text-delta',
id: '0',
delta: ', king of cats.\nHe watches sunbeams dance with golden eyes,\nAnd'
}
{
type: 'text-delta',
id: '0',
delta: ' dreams of birds and chasing after rats.\n' +
'A rumbling purr, a gentle, rhythmic sound,\n' +
'Vibrates within his frame, a peaceful song'
}
{
type: 'text-delta',
id: '0',
delta: '.\n' +
'He stretches limbs upon the hallowed ground,\n' +
'Where dust motes gather, and the day grows long.\n' +
'A sudden flicker, then a playful'
}
{
type: 'text-delta',
id: '0',
delta: ' pounce,\n' +
'A feathered toy becomes his eager prey.\n' +
'He bats and claws, a captivating bounce,\n' +
'Then curls up close, at ending of the day.\n' +
'So sleeps our Steven, noble, proud, and free,\n' +
'A ginger'
}
{ type: 'text-delta', id: '0', delta: ' god in feline majesty.\n' }
{ type: 'text-end', id: '0' }
{ type: 'finish-step' }
{ type: 'finish' }
Steam to UI
// Server
export const POST = async (req: Request): Promise<Response> => {
const body = await req.json();
// receive UI Messages
const messages: UIMessage[] = body.messages;
// convert to model
const modelMessages: ModelMessage[] = convertToModelMessages(messages);
// stream ...
const streamTextResult = streamText({
model: google('gemini-2.0-flash'),
messages: modelMessages,
});
// convert back to UI
const stream = streamTextResult.toUIMessageStream();
return createUIMessageStreamResponse({
stream,
});
};
// UI
const { messages, sendMessage } = useChat();
messages.map(//render UI)
Sending a file
You can send a multiple-part message
// UI
const { messages, sendMessage } = useChat({});
sendMessage({
parts: [
{
type: "text",
text: input,
},
{
type: "file",
mediaType: file.type,
url: fileURL,
},
],
});
System prompts
Sometimes we need AI to act in a certain way
const {textStream} = streamText({
model,
prompt,
👉 system: 'Your are a translator. Translate English to Vietnamese.',
});
Message history
To know what the shape of a conversation history is, let’s explore the CoreMessage type.
import {CoreMessage} from 'ai'
// the converations are stored in an array
// having a role and content property
const message: CoreMessage[] = [
{
role: 'system',
content: 'Answer questions in a simple way',
},
{
role: 'user',
content: 'What is the capital of France?',
},
{
role: 'assistant',
content: 'The capital of France is Paris.',
},
];
Here we can simply feed the AI with this message array so it can keep the context of the previous conversation.
Structured Outputs
We can get back from the LLM structured outputs such as JSON. we can do that by using the generateObject function
const { object } = await generateObject({
model,
schema,
prompt,
schemaName: "Recipe",
system:
`You are helping a user create a recipe. ` +
`Use British English variants of ingredient names,` +
`like Coriander over Cilantro.`,
});
And here we need to pass in a zod schema - the shape of our desired JSON output.
// NOTE: `describe` is used to give the LLM semantic context about each field
const schema = z.object({
recipe: z.object({
name: z
.string()
.describe("The title of the recipe"),
ingredients: z
.array(
z.object({
name: z.string(),
amount: z.string(),
}),
)
.describe(
"The ingredients needed for the recipe",
),
steps: z
.array(z.string())
.describe("The steps to make the recipe"),
}),
});
Stream object
We can also stream the object by using streamObject
const result = await streamObject({...})
// process each chunk
for await (const obj of result.partialObjectStream) {
console.clear();
console.dir(obj, { depth: null });
}
// OR wait for all chunks
const finalObject = await result.object;
Generate enum
Another classic use case for LLMs is classification
Example - we can pass a user comment to LLM to classify it as positive or negative
const { object } = await generateObject({
model,
👉 output: "enum",
enum: ["positive", "negative", "neutral"],
prompt: text,
system:
`Classify the sentiment of the text as either ` +
`positive, negative, or neutral.`,
});
Generate array
Example - generate fake data
export const createFakeUsers = async (
input: string,
) => {
const { object } = await generateObject({
model,
prompt: input,
system: `You are generating fake user data.`,
👉 output: "array",
schema,
});
return object;
};
Image
Yes, LLMs can take a look at an image and do things with it
Example - generate alt text from an image
const systemPrompt =
`You will receive an image. ` +
`Please create an alt text for the image. ` +
`Be concise. ` +
`Use adjectives only when necessary. ` +
`Do not pass 160 characters. ` +
`Use simple language. `;
export const describeImage = async (
imageUrl: string,
) => {
const { text } = await generateText({
model,
system: systemPrompt,
👉 messages: [
{
role: "user",
content: [
{
👉 type: "image",
👉 image: new URL(imageUrl),
},
],
},
],
});
return text;
};
Same thing above with PDF
Example - extract data from an invoice
const { object } = await generateObject({
model,
system:
`You will receive an invoice. ` +
`Please extract the data from the invoice.`,
schema,
messages: [
{
role: "user",
content: [
{
👉 type: "file",
👉 data: readFileSync(invoicePath),
👉 mimeType: "application/pdf",
},
],
},
],
});
Tool calling
We can give a bunch of functions (tools) to the LLMs to use
import { tool, stepCountIs, type InferUITools } from "ai";
const logToConsoleTool = tool({
description: "Log a message to the console",
parameters: z.object({
// 👉 Describe a parameter
message: z
.string()
.describe("The message to log to the console"),
}),
execute: async ({ message }) => {
// 👉 Our function, in this case, simple console log
console.log(message);
},
});
const tools = {
// 👉 Pass tool to the LLMs
logToConsole: logToConsoleTool,
}
export type MyUIMessage = UIMessage<never, never, InferUITools<typeof tools>>;
const logToConsole = async (prompt: string) => {
const { steps } = await generateText({
model,
prompt,
system:
`Your only role in life is to log ` +
`messages to the console. ` +
`Use the tool provided to log the ` +
`prompt to the console.`,
tools: tools,
// 👉 stop. condition
stopWhen: [stepCountIs(10)],
});
// 👉 we can log step
console.dir(steps[0]?.toolCalls, { depth: null });
};
// Example output
[
{
type: 'tool-call',
toolCallId: 'toolu_016VgqkcqE9Bgw1wa8MkfmZw',
toolName: 'logToConsole',
args: { message: 'Hello world!' }
}
]
// FE can infer the correct types including the tools
const { messages, sendMessage } = useChat<MyUIMessage>({});
First Agent
In the above example, LLMs can use a tool. Furthermore, it can react to the information received from the tool. This creates a feedback loop where the LLM can learn in the real world. This feedback loop is called agents
Example - LLMs call the get weather tool and feed the result to itself. Finally, it gives us the answer
If we don’t set maxSteps Here, the default value is 1 - meaning that LLMs calls the tool but doesn’t use the result in a follow-up step
const { textStream } = await streamText({
model,
prompt,
tools: {
getWeather: getWeatherTool,
},
// 👉 very easy, we can simply set the steps here
maxSteps: 2,
});
for await (const text of textStream) {
process.stdout.write(text);
}
Persistance
onFinish
const result = streamText({
...
onFinish: ({ response }) => {
// 'response.messages' is an array of ToolModelMessage and AssistantModelMessage,
// which are the model messages that were generated during the stream.
// This is useful if you don't need UIMessages - for simpler applications.
console.log('streamText.onFinish');
console.log(' response.messages');
console.dir(response.messages, { depth: null });
},
});
return result.toUIMessageStreamResponse({
originalMessages: messages,
onFinish: ({ messages, responseMessage }) => {
// 👉 we need to persist the UI messages, richer data
// 'messages' is the full message history, including the original messages
// you pass in to originalMessages.
console.log('toUIMessageStreamResponse.onFinish');
console.log(' messages');
console.dir(messages, { depth: null });
// 'responseMessage' is the last message in the message history.
console.log('toUIMessageStreamResponse.onFinish');
console.log(' responseMessage');
console.dir(responseMessage, { depth: null });
},
});
chatId
Messages need to be grouped into a chat or a thread
The Frontend can generate the chatId and pass it to the server
// Frontend
const {messages, sendMessage} = useChat({
id: searchParam.get('chatId') ?? crypto.randomUUID()
})
Persisting Chat Messages
We will save the UI message data to the database, it would look like so
{
"chats": [
{
// 👉 Chat ID
"id": "e109a946-ec23-48ab-ba5c-d9afc2de8bc4",
// 👉 Message array
"messages": [
// user message 1
{
"parts": [
{
"type": "text",
"text": "Who's the best football player in the world?"
}
],
"id": "YjW6GNNU1Hzc8Jyo",
"role": "user"
},
// assistant response 1
{
"id": "",
"role": "assistant",
"parts": [
{
"type": "step-start"
},
{
"type": "text",
"text": "That's a question that sparks endless debate! ....",
"state": "done"
}
]
},
// user message 2
{
"parts": [
{
"type": "text",
"text": "who is messi"
}
],
"id": "mUL9oGn8mQBBLiMC",
"role": "user"
},
// assistant response 2
{
"id": "",
"role": "assistant",
"parts": [
{
"type": "step-start"
},
{
"type": "text",
"text": "Lionel Messi is an Argentinian professional footballer who plays as a forward for Inter Miami in Major League Soccer (MLS). He is widely regarded as one of the greatest football players of all time.\n\nHere's a quick rundown of his key achievements and attributes:\n\n* **Nationality:** Argentinian\n* **Current Club:** Inter Miami (MLS)\n* **Position:** Forward\n* **Key Skills:** Dribbling, Passing, Finishing, Free-kicks, Playmaking\n* **Major Achievements:**\n * **Ballon d'Or Awards:** 8 (record holder)\n * **Champions League Titles:** 4\n * **La Liga Titles:** 10\n * **Copa America Title:** 1\n * **FIFA World Cup Title:** 1\n* **Known For:**\n * Incredible dribbling skills and agility.\n * Prolific goal-scoring record.\n * Exceptional vision and passing ability.\n * Masterful free-kick taker.\n * A quiet demeanor off the field, but a fierce competitor on it.\n\nMessi spent the majority of his career playing for FC Barcelona, where he achieved legendary status. He is also the captain and all-time leading goalscorer for the Argentina national team.\n",
"state": "done"
}
]
}
],
"createdAt": "2025-12-01T19:57:46.109Z",
"updatedAt": "2025-12-01T19:57:52.900Z"
}
]
}
Frontend
// 1. Generate chatId if needed
const [backupChatId, setBackupChatId] = useState(
crypto.randomUUID(),
);
// 2. Check if existing chat by checking the ?chatId from the url
const chatIdFromSearchParams = searchParams.get('chatId');
// 3. API call - Get current chat content
const { data } = useSuspenseQuery({
queryKey: ['chat', chatIdFromSearchParams],
queryFn: () => {
// If no id, no need to fetch
if (!chatIdFromSearchParams) {
return null;
}
// fetch from DB
return fetch(
`/api/chat?chatId=${chatIdFromSearchParams}`,
).then((res): Promise<DB.Chat> => res.json());
},
});
// 4. AI Hook
const { messages, sendMessage } = useChat({
// chat id: get from the url OR create one
id: chatIdFromSearchParams ?? backupChatId,
// existing chat messages
messages: data?.messages ?? [],
});
// 5. On submit
onSubmit() {
sendMessage({ text: input });
setInput('');
if (chatIdFromSearchParams) {
return;
}
// 1. Push URL
setSearchParams({ chatId: backupChatId });
// 2. Refresh the backup chat id
setBackupChatId(crypto.randomUUID());
}
Backend
export const POST = async (req: Request): Promise<Response> => {
const body: {messages: UIMessage[]; id: string} = await req.json();
const {messages, id} = body;
const mostRecentMessage = messages[messages.length - 1];
// validation
if (!mostRecentMessage) {
return new Response('No messages provided', {status: 400});
}
if (mostRecentMessage.role !== 'user') {
return new Response('Last message must be from the user', {
status: 400,
});
}
let chat = await getChat(id);
// 1. Create a new chat OR append last user message to the exising chat
if (!chat) {
chat = await createChat(id, messages);
} else {
await appendToChatMessages(id, [mostRecentMessage]);
}
const result = streamText({
model: google('gemini-2.0-flash-001'),
messages: convertToModelMessages(messages),
});
// 2. wait for the stream to finish and append the last message (AI response) to the chat
return result.toUIMessageStreamResponse({
onFinish: async ({responseMessage}) => {
await appendToChatMessages(id, [responseMessage]);
},
});
};
Context Engineering
Prompt template
The prompt template takes advantage of how LLMs work. When we pass input to an LMM, it tends to be biased toward the content at the beginning and end of the prompt. The middle sections are not as influential.
High-level context at start
Background data in the middle
Most critical elemetns (ask, instructions, thinking, output formatting) at the end
📚 See more: https://youtu.be/ysPbXH0LpIE?si=TfwrITJQke1LuuzB
Example
<task-context>
You are a helpful assistant that can generate titles for conversations.
</task-context>
<conversation-history>
${INPUT}
</conversation-history>
<rules>
Find the most concise title that captures the essence of the conversation.
Titles should be at most 30 characters.
Titles should be formatted in sentence case, with capital letters at the start of each word. Do not provide a period at the end.
</rules>
<the-ask>
Generate a title for the conversation.
</the-ask>
<output-format>
Return only the title.
</output-format>
Give examples
We can remove some rules if we provide a concrete example
<examples>
<example>
<input>What's the difference between TypeScript and JavaScript? Should I learn TypeScript first or JavaScript?</input>
<expected>TypeScript vs JavaScript Comparison</expected>
</example>
<example>
<input>I want to start investing but I'm a complete beginner. What are the safest options for someone with $5000 to invest?</input>
<expected>Beginner Investment Options</expected>
</example>
</examples>
Retrieval - Web scraping
We can use Tavily to scrape the web
// Scraping the web
const tavilyClient = tavily({
apiKey: process.env.TAVILY_API_KEY,
});
const scrapeResult = await tavilyClient.extract([url]);
const rawContent = scrapeResult.results[0]?.rawContent;
// Prompt template to summarize the content
const result = await streamText({
model: google('gemini-2.0-flash-lite'),
prompt: `
<task-context>
You are a helpful assistant that summarizes the content of a URL.
</task-context>
<background-data>
Here is the content of the website:
<url>
${url}
</url>
<content>
${rawContent}
</content>
</background-data>
<rules>
- Use the content of the website to answer the question.
- If the question is not related to the content of the website, say "I'm sorry, I can only answer questions about the content of the website."
- Use quotes from the content of the website to answer the question.
- Use paragraphs in your output.
</rules>
<conversation-history>
${input}
</conversation-history>
<the-ask>
Summarize the content of the website based on the conversation history.
</the-ask>
<output-format>
Return only the summary.
</output-format>
`,
});
Chain of Thought
We ask the LLM to think first before giving the answer. Also, here, we can guide it how it should think about the problem
<the-ask>
Explain the code, using the article as a reference.
</the-ask>
<thinking-instructions>
Think about your answer first before you respond. Consider the optimal path for the user to understand the code. Consider all of the knowledge dependencies - the pieces of knowledge that rely on other pieces of knowledge. Assume the user knows very little about TypeScript. Create a list of the pieces of knowledge that the user needs to know, in order of dependency.
</thinking-instructions>
<output-format>
Return two sections - a <thinking> block and an answer.
- The <thinking> block should contain your thought process, and be wrapped in a <thinking> tag.
- The answer should be unwrapped.
- The answer should be in markdown format, using code blocks for the TypeScript code.
</output-format>