Vercel AI SDK

Beztack includes @beztack/ai, a lightweight wrapper package that provides pre-configured Vercel AI SDK integration with Amazon Bedrock support.

Why Vercel AI SDK?

We chose Vercel AI SDK because:

Framework Agnostic: Works with any JavaScript framework
Streaming First: Built-in support for streaming responses
Type-Safe: Fully typed with TypeScript
Provider Abstraction: Easy to switch between AI providers
React Hooks: Ready-to-use hooks for chat and completion UIs
Edge Ready: Works in serverless and edge environments

Package Overview

The @beztack/ai package provides:

AI SDK Re-exports: Complete Vercel AI SDK functionality
Amazon Bedrock Provider: Pre-configured Bedrock provider with AWS credential chain
Zero Configuration: Works out of the box with AWS credentials

Installation

The package is already included in the monorepo. To use it in your app:

pnpm add @beztack/ai

Configuration

Environment Variables

Set up your AWS credentials using one of the standard methods:

# Option 1: Environment variables
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key
AWS_REGION=us-east-1

# Option 2: AWS CLI profile (recommended for development)
# Configure with: aws configure

The package uses AWS Node.js credential provider chain, which automatically detects credentials from:

Environment variables
AWS CLI configuration files
IAM roles (when running on AWS)
Container credentials (ECS/EKS)

Basic Usage

Generate Text

import { generateText, bedrock } from '@beztack/ai';

const result = await generateText({
  model: bedrock('anthropic.claude-3-sonnet-20240229-v1:0'),
  prompt: 'Explain quantum computing in simple terms',
});

console.log(result.text);

Stream Text

import { streamText, bedrock } from '@beztack/ai';

const result = await streamText({
  model: bedrock('anthropic.claude-3-sonnet-20240229-v1:0'),
  prompt: 'Write a short story about a robot',
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

Generate Object (Structured Output)

import { generateObject, bedrock } from '@beztack/ai';
import { z } from 'zod';

const result = await generateObject({
  model: bedrock('anthropic.claude-3-sonnet-20240229-v1:0'),
  schema: z.object({
    recipe: z.object({
      name: z.string(),
      ingredients: z.array(z.object({
        name: z.string(),
        amount: z.string(),
      })),
      steps: z.array(z.string()),
    }),
  }),
  prompt: 'Generate a recipe for chocolate chip cookies',
});

console.log(result.object.recipe);

React Integration

Chat Component

import { useChat } from '@beztack/ai/react';

export function Chat() {
  const { messages, input, handleInputChange, handleSubmit, isLoading } = useChat({
    api: '/api/chat',
  });

  return (
    <div>
      {messages.map((message) => (
        <div key={message.id}>
          <strong>{message.role}:</strong> {message.content}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Say something..."
          disabled={isLoading}
        />
        <button type="submit" disabled={isLoading}>
          Send
        </button>
      </form>
    </div>
  );
}

Completion Component

import { useCompletion } from '@beztack/ai/react';

export function Completion() {
  const { completion, input, handleInputChange, handleSubmit, isLoading } = useCompletion({
    api: '/api/completion',
  });

  return (
    <div>
      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Enter a prompt..."
        />
        <button type="submit" disabled={isLoading}>
          Generate
        </button>
      </form>

      {completion && <div>{completion}</div>}
    </div>
  );
}

API Routes

Chat Endpoint (Nitro)

Create a chat endpoint in apps/api/server/routes/api/chat.post.ts:

import { streamText, bedrock } from '@beztack/ai';

export default defineEventHandler(async (event) => {
  const { messages } = await readBody(event);

  const result = await streamText({
    model: bedrock('anthropic.claude-3-sonnet-20240229-v1:0'),
    messages,
  });

  return result.toDataStreamResponse();
});

Completion Endpoint (Nitro)

Create a completion endpoint in apps/api/server/routes/api/completion.post.ts:

import { streamText, bedrock } from '@beztack/ai';

export default defineEventHandler(async (event) => {
  const { prompt } = await readBody(event);

  const result = await streamText({
    model: bedrock('anthropic.claude-3-sonnet-20240229-v1:0'),
    prompt,
  });

  return result.toDataStreamResponse();
});

Available Models

Amazon Bedrock Models

The package comes pre-configured for Amazon Bedrock. Available models include:

Model	ID
Claude 3.5 Sonnet	`anthropic.claude-3-5-sonnet-20241022-v2:0`
Claude 3 Sonnet	`anthropic.claude-3-sonnet-20240229-v1:0`
Claude 3 Haiku	`anthropic.claude-3-haiku-20240307-v1:0`
Claude 3 Opus	`anthropic.claude-3-opus-20240229-v1:0`
Llama 3.1 70B	`meta.llama3-1-70b-instruct-v1:0`
Llama 3.1 8B	`meta.llama3-1-8b-instruct-v1:0`

Using Other Providers

Since the package re-exports the full AI SDK, you can use any provider:

import { generateText } from '@beztack/ai';
import { openai } from '@ai-sdk/openai';
import { anthropic } from '@ai-sdk/anthropic';

// Using OpenAI
const result = await generateText({
  model: openai('gpt-4-turbo'),
  prompt: 'Hello!',
});

// Using Anthropic directly
const result2 = await generateText({
  model: anthropic('claude-3-sonnet-20240229'),
  prompt: 'Hello!',
});

Advanced Usage

Tool Calling

import { generateText, bedrock, tool } from '@beztack/ai';
import { z } from 'zod';

const result = await generateText({
  model: bedrock('anthropic.claude-3-sonnet-20240229-v1:0'),
  prompt: 'What is the weather in San Francisco?',
  tools: {
    getWeather: tool({
      description: 'Get the weather for a location',
      parameters: z.object({
        location: z.string().describe('The city name'),
      }),
      execute: async ({ location }) => {
        // Call your weather API here
        return { temperature: 72, condition: 'sunny' };
      },
    }),
  },
});

import { generateText, bedrock } from '@beztack/ai';

const result = await generateText({
  model: bedrock('anthropic.claude-3-sonnet-20240229-v1:0'),
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'What is in this image?' },
        { type: 'image', image: imageUrl },
      ],
    },
  ],
});

Embeddings

import { embed, bedrock } from '@beztack/ai';

const result = await embed({
  model: bedrock.embedding('amazon.titan-embed-text-v1'),
  value: 'The quick brown fox jumps over the lazy dog',
});

console.log(result.embedding); // Vector array

Best Practices

1. Use Streaming for Long Responses

Streaming provides better UX for long-running generations:

const result = await streamText({
  model: bedrock('anthropic.claude-3-sonnet-20240229-v1:0'),
  prompt: 'Write a detailed essay...',
});

// Stream to response
return result.toDataStreamResponse();

2. Handle Errors Gracefully

try {
  const result = await generateText({
    model: bedrock('anthropic.claude-3-sonnet-20240229-v1:0'),
    prompt: 'Hello!',
  });
} catch (error) {
  if (error.code === 'RATE_LIMIT_EXCEEDED') {
    // Implement retry logic
  }
  if (error.code === 'CONTEXT_LENGTH_EXCEEDED') {
    // Reduce input size
  }
}

3. Use Structured Output When Possible

// Prefer this
const result = await generateObject({
  model: bedrock('anthropic.claude-3-sonnet-20240229-v1:0'),
  schema: yourZodSchema,
  prompt: 'Extract data...',
});

// Over parsing text manually

4. Set Reasonable Token Limits

const result = await generateText({
  model: bedrock('anthropic.claude-3-sonnet-20240229-v1:0'),
  prompt: 'Summarize this article...',
  maxTokens: 500, // Prevent runaway costs
});