@mux/ai + Vercel Workflows Starter

@mux/ai + Vercel Workflows Starter

A Next.js starter template demonstrating how to build durable video AI pipelines with @mux/ai and the Vercel Workflow DevKit.

🚀 Deploy to Vercel

Three Integration Layers

Resumable workflows (try it)

This project showcases resumable, durable workflows out of the box:

• Start a workflow (captions, dubbing, or summary).
• Refresh the page, or navigate away and back.
• You should see the workflow still running asynchronously, with status rehydrated from browser localStorage.

Quick Start

1
npm install
2
npm run dev

Inspect workflow runs locally:

1
npx workflow web

Rate Limiting

This demo includes IP-based rate limiting to protect against excessive API costs. Limits are automatically bypassed in development mode.

See DOCS/RATE-LIMITS.md for implementation details and maintenance.

Remotion support

Remotion is used within this example app for composing @mux/ai with video rendering.

Local Development

1
# Open the Remotion Studio for live preview and iteration
2
npm run remotion:studio
3

4
# Render a video locally (for testing)
5
# Pass the composition name as an argument
6
npm run remotion:render:local default-composition
7

8
# Optionally specify an output path
9
npm run remotion:render:local default-composition out/foo.mp4

Production Deployment

1
# Deploy Remotion site to AWS Lambda for serverless rendering
2
npm run remotion:deploy

Note: remotion:deploy bundles and deploys your Remotion site to AWS Lambda for production video rendering. This is not for development — use remotion:studio and remotion:render:local for local dev and testing.

Automated Deployment

Remotion is automatically deployed to AWS Lambda when changes to remotion/ are merged into main. See DOCS/AUTOMATED-REMOTION-DEPLOYMENTS.md for details.

Environment Variables

See AGENTS.md for the full list. At minimum you'll need:

1
# Mux credentials
2
MUX_TOKEN_ID=
3
MUX_TOKEN_SECRET=
4

5
# OpenAI (required for embeddings)
6
OPENAI_API_KEY=
7

8
# Database (PostgreSQL with pgvector) — required to store/search the Mux catalog metadata
9
DATABASE_URL=

Database setup + importing your Mux catalog

This project stores your Mux catalog metadata in Postgres and generates pgvector embeddings for semantic search. The database schema and migrations are managed with Drizzle (see db/schema.ts and db/migrations/), and the db:* scripts use Drizzle Kit.

1) Configure your database connection

Create a .env.local file (this is what both Drizzle and the import script load):

1
# Database (PostgreSQL + pgvector)
2
DATABASE_URL="postgresql://USER:PASSWORD@HOST:5432/DB_NAME"
3

4
# Mux (used by the import script)
5
MUX_TOKEN_ID="..."
6
MUX_TOKEN_SECRET="..."
7

8
# Embeddings (used by the import script)
9
OPENAI_API_KEY="..."

Your Postgres must support pgvector. The first migration will run CREATE EXTENSION IF NOT EXISTS vector;.

2) Run database migrations

Apply the migrations in db/migrations/ (creates tables + indexes and enables pgvector):

1
npm run db:migrate

3) Import Mux assets (and generate embeddings)

This fetches all ready Mux assets with playback IDs, upserts rows into videos, and writes embedding rows into video_chunks.

1
npm run import-mux-assets

To embed subtitles from a specific captions track language, pass --language (defaults to en). This should match the language of an existing captions track on the source Mux asset — it does not translate captions:

1
npm run import-mux-assets -- --language en

4) Understand the database scripts

• npm run db:generate: Generates new migration files from db/schema.ts (use this after changing the schema).
• npm run db:migrate: Applies migrations to the database defined by DATABASE_URL.
• npm run db:studio: Opens Drizzle Studio to inspect tables/rows locally (also uses DATABASE_URL).

Media Detail Page Structure

The media detail page (/media/[slug]) is organized into co-located feature folders:

1
app/media/[slug]/
2
├── media-content.tsx
3
├── page.tsx
4
├── localization/
5
│   ├── actions.ts      (captions & audio translation)
6
│   ├── constants.ts
7
│   └── ui.tsx
8
├── player/
9
│   ├── context.ts
10
│   ├── provider.tsx
11
│   ├── ui.tsx
12
│   └── use-player.ts
13
├── social-clips/
14
│   ├── actions.ts      (clip creation & Remotion Lambda rendering)
15
│   ├── constants.ts
16
│   ├── preview.tsx     (client-side Remotion Player preview)
17
│   └── ui.tsx
18
├── summarize-and-tag/
19
│   ├── actions.ts      (start/poll summary generation workflow)
20
│   └── ui.tsx
21
├── transcript/
22
│   ├── actions.ts      (semantic search within video transcript)
23
│   ├── helpers.ts
24
│   └── ui.tsx
25
└── workflows-panel/
26
    ├── helpers.ts
27
    └── ui.tsx          (includes StatusBadge, StepProgress, etc.)

Learn More

• context/application-explained.md — what the app does and why
• context/design-explained.md — visual design and UX patterns
• context/implementation-explained.md — routes, data model, and code patterns
• AGENTS.md — guidance for AI coding assistants
• DOCS/RATE-LIMITS.md — rate limiting configuration and maintenance

See Also

• pgvector — vector embeddings and similarity search for Postgres