Semantic Image Search

An open-source AI semantic image search app template built with Next.js, the Vercel AI SDK, OpenAI, Vercel Postgres, Vercel Blob and Vercel KV.

Features · Model Providers · Deploy Your Own · Running locally · Authors

Features

• Next.js App Router
• React Server Components (RSCs), Suspense, and Server Actions
• Vercel AI SDK for multimodal prompting, generating & embedding image metadata, and streaming images from Server to Client
• Support for OpenAI (default), Gemini, Anthropic, Cohere, or custom AI chat models
• shadcn/ui
  • Styling with Tailwind CSS
  • Radix UI for headless component primitives
• Query caching with Vercel KV
• Embeddings powered by Vercel Postgres, pgvector, and Drizzle ORM
• File (image) storage with Vercel Blob

Model Providers

This template ships with OpenAI GPT-4o as the default. However, thanks to the Vercel AI SDK, you can switch LLM providers to Gemini, Anthropic, Cohere, and more with just a few lines of code.

Deploy Your Own

You can deploy your own version of the Semantic Image Search App to Vercel with one click:

Setup

Creating a KV Database Instance

Follow the steps outlined in the quick start guide provided by Vercel. This guide will assist you in creating and configuring your KV database instance on Vercel, enabling your application to interact with it.

Remember to update your environment variables (KV_URL, KV_REST_API_URL, KV_REST_API_TOKEN, KV_REST_API_READ_ONLY_TOKEN) in the .env file with the appropriate credentials provided during the KV database setup.

Creating a Postgres Database Instance

Follow the steps outlined in the quick start guide provided by Vercel. This guide will assist you in creating and configuring your Postgres database instance on Vercel, enabling your application to interact with it.

Once you have instantiated your Vercel Postgres instance, run the following code to enable pgvector:

1
CREATE EXTENSION vector;

Remember to update your environment variables (POSTGRES_URL, POSTGRES_PRISMA_URL, POSTGRES_URL_NON_POOLING, POSTGRES_USER, POSTGRES_HOST, POSTGRES_PASSWORD, POSTGRES_DATABASE) in the .env file with the appropriate credentials provided during the Postgres database setup.

Creating a Blob Instance

Follow the steps outlined in the quick start guide provided by Vercel. This guide will assist you in creating and configuring your Blob instance on Vercel, enabling your application to interact with it.

Remember to update your environment variable (BLOB_READ_WRITE_TOKEN) in the .env file with the appropriate credentials provided during the Blob setup.

Running locally

You will need to use the environment variables defined in .env.example [blocked] to run Next.js Semantic Image Search. It's recommended you use Vercel Environment Variables for this, but a .env file is all that is necessary.

Note: You should not commit your .env file or it will expose secrets that will allow others to control access to your various OpenAI and authentication provider accounts.

1. Install Vercel CLI: npm i -g vercel
2. Link local instance with Vercel and GitHub accounts (creates .vercel directory): vercel link
3. Download your environment variables: vercel env pull

1
pnpm install

Add OpenAI API Key

Be sure to add your OpenAI API Key to your .env.

Database Setup

To push your schema changes to your Vercel Postgres database, run the following command.

1
pnpm run db:generate
2
pnpm run db:push

Prepare your Images (Indexing Step)

To get your application ready for Semantic search, you will have to complete three steps.

1. Upload Images to storage
2. Send Images to a Large Language Model to generate metadata (title, description)
3. Iterate over each image, embed the metadata, and then save to the database

Upload Images

Put the images you want to upload in the images-to-index directory (.jpg format) at the root of your application. Run the following command.

1
pnpm run upload

This script will upload the images to your Vercel Blob store. Depending on how many photos you are uploading, this step could take a while.

Generate Metadata

Run the following command.

1
pnpm run generate-metadata

This script will generate metadata for each of the images you uploaded in the previous step. Depending on how many photos you are uploading, this step could take a while.

Embed Metadata and Save to Database

Run the following command.

1
pnpm run embed-and-save

Depending on how many photos you are uploading, this step could take a while. This script will embed the descriptions generated in the previous step and save them to your Vercel Postgres instance.

Starting the Server

Run the following command

1
pnpm run dev

Your app template should now be running on localhost:3000.

Authors

This library is created by Vercel and Next.js team members, with contributions from:

• Jared Palmer (@jaredpalmer) - Vercel
• Shu Ding (@shuding_) - Vercel
• shadcn (@shadcn) - Vercel
• Lars Grammel (@lgrammel) - Vercel
• Nico Albanese (@nicoalbanese10) - Vercel