Documentation

Back to Home

Avatario Plugin for LiveKit Agents

Bringing lifelike AI avatars with synchronized speech and facial animation to your LiveKit-powered applications

Introduction

Avatario brings lifelike AI avatars with synchronized speech and facial animation to your LiveKit-powered applications. This plugin enables seamless integration of Avatario avatars as participants in your LiveKit rooms.

Installation

Install the plugin using PyPi:

bash
pip install livekit-plugins-avatarioai

Authentication & Setup

The Avatario plugin requires a Avatario API key.

Set AVATARIO_API_KEY in your environment or .env file where your LiveKit worker code is running:

bash
# In your environment or .env file
AVATARIO_API_KEY=your_api_key_here

Available Avatars

We provide a list of stock avatars that can be accessed using the following curl command:

bash
curl --request GET \
  --url https://app.onezot.work/api/avatars/stock \
  -H "Content-Type: application/json"

This returns a list of our stock avatars along with their attributes and IDs that you can use in your integration.

You can also find detailed information about our stock avatars in the Stock Avatars documentation.

Usage

Use the plugin in an AgentSession. For example, you can use this avatar in theVoice AI quickstart

python
from livekit import agents
from livekit.agents import AgentSession, RoomOutputOptions
from livekit.plugins import avatarioai

async def entrypoint(ctx: agents.JobContext):
   await ctx.connect()

   session = AgentSession(
      # ... stt, llm, tts, etc.
   )

   avatar = avatarioai.AvatarSession(
      avatar_id="...",  # ID of the avatario avatar to use
      video_info=avatarioai.VideoInfo(
         custom_background_url="https://xyz.com",
         video_frame_height=720,
         video_frame_width=1080,
      )
   )

   # Start the avatar and wait for it to join
   await avatar.start(session, room=ctx.room)

   # Start your agent session with the user
   await session.start(
      room=ctx.room,
      room_output_options=RoomOutputOptions(
         # Disable audio output to the room. The avatar plugin publishes audio separately.
         audio_enabled=False,
      ),
      # ... agent, room_input_options, etc....
   )

Parameters

This section describes some of the available parameters. See the plugin reference for a complete list of all available parameters.

avatar_id string Required

ID of the Avatario avatar to use. See Stock Avatars for reference.

avatar_participant_name string Optional Default: avatario-avatar-agent

The name of the participant to use for the avatar.

video_info VideoInfo Optional

Video configuration settings including custom background and frame dimensions.

  • custom_background_url (string): URL of custom background image. If URL is invalid or image cannot be downloaded, a plain white background will appear.
  • video_frame_width (integer): Width of video frame (default: 1080, maximum: 1920)
  • video_frame_height (integer): Height of video frame (default: 720, maximum: 1080)

video_frame_width integer Optional Default: 1280

The width of video frame to be streamed (maximum 1920)

video_frame_height integer Optional Default: 720

The height of the video frame to be streamed (maximum 1080)

Frontend Integration

There are two main ways to integrate Avatario avatars in your frontend applications using LiveKit:

Method 1: LiveKit Agents Playground

The easiest way to test and interact with Avatario avatars is through the LiveKit Agents Playground.

Steps:

  1. Visit the LiveKit Agents Playground
  2. Choose one of two connection methods:
    • LiveKit Cloud: Click "Continue with LiveKit Cloud" for the easiest setup
    • Manual: Click the "Manual" tab and enter your LiveKit token and WebSocket URL
  3. Click "Connect" to establish the connection
  4. Start interacting with your Avatario avatar in real-time

Note: This method is perfect for testing and prototyping. For production applications, use the HTML/JavaScript integration method below.

Method 2: HTML/JavaScript Integration

For custom frontend applications, you can integrate directly using the LiveKit client SDK.

Minimal HTML Example:

html
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Avatario Avatar Demo</title>
    <script src="https://cdn.jsdelivr.net/npm/livekit-client@2.13.3/dist/livekit-client.umd.min.js"></script>
    <style>
        body {
            font-family: Arial, sans-serif;
            max-width: 800px;
            margin: 0 auto;
            padding: 20px;
            background: #000;
            color: #fff;
        }
        .video-container {
            position: relative;
            width: 100%;
            max-width: 640px;
            aspect-ratio: 16/9;
            background: #222;
            border-radius: 8px;
            overflow: hidden;
            margin: 20px 0;
        }
        video {
            width: 100%;
            height: 100%;
            object-fit: cover;
        }
        .local-video {
            position: absolute;
            bottom: 10px;
            right: 10px;
            width: 150px;
            height: 100px;
            border: 2px solid #fff;
            border-radius: 8px;
        }
        .controls {
            display: flex;
            gap: 10px;
            margin: 20px 0;
        }
        button {
            padding: 10px 20px;
            border: none;
            border-radius: 5px;
            background: #007bff;
            color: white;
            cursor: pointer;
        }
        button:disabled {
            background: #666;
            cursor: not-allowed;
        }
        button.disconnect {
            background: #dc3545;
        }
    </style>
</head>
<body>
    <h1>Avatario Avatar Demo</h1>
    
    <div class="controls">
        <button id="connectBtn">Connect to Avatar</button>
        <button id="disconnectBtn" disabled>Disconnect</button>
        <button id="toggleAudio">Mute Audio</button>
        <button id="toggleVideo">Turn Off Video</button>
    </div>
    
    <div class="video-container">
        <video id="remoteVideo" autoplay playsinline></video>
        <video id="localVideo" class="local-video" autoplay playsinline muted></video>
        <audio id="remoteAudio" autoplay></audio>
    </div>
    
    <div id="status">Ready to connect...</div>

    <script>
        const { Room, RoomEvent, Track } = LiveKitClient;
        
        let room = null;
        let isAudioEnabled = true;
        let isVideoEnabled = true;
        
        // Get DOM elements
        const connectBtn = document.getElementById('connectBtn');
        const disconnectBtn = document.getElementById('disconnectBtn');
        const toggleAudioBtn = document.getElementById('toggleAudio');
        const toggleVideoBtn = document.getElementById('toggleVideo');
        const remoteVideo = document.getElementById('remoteVideo');
        const localVideo = document.getElementById('localVideo');
        const remoteAudio = document.getElementById('remoteAudio');
        const status = document.getElementById('status');
        
        // Replace these with your actual values
        const LIVEKIT_URL = 'wss://your-livekit-server.com';
        const ACCESS_TOKEN = 'your-access-token';
        
        async function connectToRoom() {
            try {
                status.textContent = 'Connecting...';
                connectBtn.disabled = true;
                
                // Create room instance
                room = new Room({
                    adaptiveStream: true,
                    dynacast: true
                });
                
                // Set up event listeners
                room.on(RoomEvent.Connected, () => {
                    status.textContent = 'Connected! Waiting for avatar...';
                    disconnectBtn.disabled = false;
                });
                
                room.on(RoomEvent.Disconnected, () => {
                    status.textContent = 'Disconnected';
                    connectBtn.disabled = false;
                    disconnectBtn.disabled = true;
                });
                
                room.on(RoomEvent.ParticipantConnected, (participant) => {
                    if (participant.identity.includes('avatar')) {
                        status.textContent = 'Avatar connected!';
                    }
                });
                
                room.on(RoomEvent.TrackSubscribed, (track, publication, participant) => {
                    if (track.kind === Track.Kind.Video) {
                        track.attach(remoteVideo);
                    } else if (track.kind === Track.Kind.Audio) {
                        track.attach(remoteAudio);
                    }
                });
                
                // Connect to room
                await room.connect(LIVEKIT_URL, ACCESS_TOKEN);
                
                // Enable camera and microphone
                await room.localParticipant.enableCameraAndMicrophone();
                
                // Attach local video
                const videoTrack = room.localParticipant.getTrackPublication(Track.Source.Camera);
                if (videoTrack?.track) {
                    videoTrack.track.attach(localVideo);
                }
                
            } catch (error) {
                console.error('Connection failed:', error);
                status.textContent = 'Connection failed: ' + error.message;
                connectBtn.disabled = false;
            }
        }
        
        async function disconnectFromRoom() {
            if (room) {
                room.disconnect();
                room = null;
            }
        }
        
        async function toggleAudio() {
            if (room) {
                isAudioEnabled = !isAudioEnabled;
                await room.localParticipant.setMicrophoneEnabled(isAudioEnabled);
                toggleAudioBtn.textContent = isAudioEnabled ? 'Mute Audio' : 'Unmute Audio';
            }
        }
        
        async function toggleVideo() {
            if (room) {
                isVideoEnabled = !isVideoEnabled;
                await room.localParticipant.setCameraEnabled(isVideoEnabled);
                toggleVideoBtn.textContent = isVideoEnabled ? 'Turn Off Video' : 'Turn On Video';
            }
        }
        
        // Event listeners
        connectBtn.addEventListener('click', connectToRoom);
        disconnectBtn.addEventListener('click', disconnectFromRoom);
        toggleAudioBtn.addEventListener('click', toggleAudio);
        toggleVideoBtn.addEventListener('click', toggleVideo);
    </script>
</body>
</html>

Setup Requirements:

  1. Replace LIVEKIT_URL with your LiveKit server WebSocket URL
  2. Replace ACCESS_TOKEN with a valid LiveKit access token
  3. Ensure your LiveKit server is running with the Avatario plugin configured
  4. Make sure your avatar agent is running and connected to the same room

Important: This is a minimal example for demonstration. For production use, implement proper error handling, token refresh, and additional LiveKit features as needed.

Additional Resources: