Agora Voice Agents

Agora Realtime React package

agora-realtime-react is the AI SDK-compatible React package for building Agora WebRTC Voice Agents in Vercel and React apps.

The package handles the browser side of the stack: Agora RTC join/leave, session state, microphone capture, remote audio, voice meters, and the voice ring UI. Your Vercel route handles the server side: App Certificate storage, RTC token generation, and ConvoAI startup.

Install target

App developers can install it directly:

npm install agora-realtime-react

Requirements

  • React ^18.3.0 or ^19.0.0. The package should stay compatible with normal create-next-app React versions instead of pinning one React patch release.
  • npm or another package manager that installs peer dependencies. If peers are not installed automatically, add agora-rtc-sdk-ng, @ai-sdk/react@4.0.9, and ai@7.0.8 in the app. The AI SDK packages provide experimental realtime types and UI message contracts.
  • A server route that can mint Agora RTC tokens and start/end the ConvoAI or AI Studio agent. The browser must never receive the Agora App Certificate.
  • Optional website analytics use NEXT_PUBLIC_GA_MEASUREMENT_ID; leave it unset to disable analytics.
  • A secure browser context for microphone capture: HTTPS in production, or http://localhost for local development.
  • For the Vercel / Next.js example path, use a current Node.js runtime compatible with Next.js and shadcn CLI. This repo itself uses Node.js >=24.0.0.

The package exports:

import {
  VoiceRingButton,
  microphoneCompatibilityWarning,
  useAgoraRealtime,
  useVoiceMeter,
} from 'agora-realtime-react';

What it does

Use the package when a Vercel-hosted React app needs to join an Agora RTC channel and talk to an Agora AI agent without rebuilding the browser media layer.

It owns:

  1. Browser-side Agora RTC session lifecycle through useAgoraRealtime.
  2. React state for status, messages, events, capture state, playback state, and debug traces.
  3. Microphone and remote-agent voice level analysis through useVoiceMeter.
  4. A reusable dual-channel VoiceRingButton for human and AI voice activity.
  5. Typed setup contracts between the browser and your Vercel route.

The package does not own Agora credentials or ConvoAI startup. Those stay on your server. The template shows the Vercel / Next.js route boundary.

How it fits with Vercel

Vercel gives the agent app a clean deployment target:

  • The React app renders the voice UI.
  • A Next.js route mints Agora RTC tokens with server-side environment variables.
  • The same route starts the ConvoAI agent with the published AI Studio pipeline ID.
  • Preview and Production deployments can use separate Agora projects or pipeline IDs.

The browser receives a scoped RTC token, channel name, UID, and agent metadata. It never receives the Agora App Certificate.

Relationship to Vercel AI SDK

Vercel AI SDK is the broader TypeScript toolkit for AI applications. agora-realtime-react does not replace it and does not make AI SDK speak Agora RTC.

The package is AI SDK-compatible in the narrow, testable sense: it reuses AI SDK experimental realtime types, accepts AI SDK-shaped session config, returns a hook shape aligned with Experimental_UseRealtimeReturn, and keeps messages close to UIMessage. It is not yet a native provider for experimental_useRealtime({ model }).

The overlap is developer ergonomics. React developers get a hook-shaped API with familiar state and controls, while Agora remains the media and agent runtime:

  • React API: hook state for status, messages, events, capture, playback, and controls.
  • Realtime transport: Agora Web SDK RTC media session.
  • Audio path: microphone audio publishes directly into Agora RTC.
  • Agent runtime: Agora ConvoAI or a published AI Studio pipeline.
  • Server role: RTC token generation plus ConvoAI join/leave.

What you can build with it

  • A Vercel app for low-latency Agora Voice Agents.
  • Customer support, tutoring, interview, sales, or concierge agent UIs.
  • A public demo page that shows agent voice UI from sample audio without consuming live Agora quota.
  • A product-specific Next.js, Remix, or React app that keeps its own server routes but reuses the Agora client runtime.
  • A design-system path where teams consume only VoiceRingButton and useVoiceMeter through shadcn.

Package boundary

Keep these responsibilities in the package:

  • typed React hooks;
  • browser RTC session control;
  • browser media and voice metering;
  • reusable voice UI primitives;
  • shared types for setup responses and runtime events.

Keep these responsibilities outside the package:

  • Agora App Certificate storage;
  • ConvoAI pipeline creation and publishing;
  • Vercel environment variables and project deployment;
  • app-specific server routes;
  • product-specific prompts, copy, layout, and workflow decisions.

This boundary lets the same npm package power the hosted demo, the Vercel template, and future custom applications.

References

  • Agora: https://www.agora.io/en/
  • Agora Docs (English): https://docs.agora.io/en/
  • Agora realtime voice docs: https://docs.agora.io/en/realtime-media/voice
  • Agora AI and Conversational AI docs: https://docs.agora.io/en/ai
  • Use with Vercel AI SDK: /docs/use-with-vercel-ai-sdk
  • Vercel AI SDK docs: https://ai-sdk.dev/docs
  • Vercel environment variables: https://vercel.com/docs/environment-variables
  • Agora realtime hook: packages/agora-realtime-react/src/use-agora-realtime.ts
  • Package exports: packages/agora-realtime-react/src/index.ts