Voice Lab
Voice Lab is a testing and evaluation framework for LLM-powered agents, optimized for voice agents but applicable to any LLM agent. It helps developers compare models, test prompts, and define custom metrics using LLM-as-a-Judge, enabling data-driven decisions for cost and performance optimization.
✨ Key features
- Define custom metrics in JSON and score with LLM-as-a-Judge
- Test multiple prompt variations and personas systematically
- Compare performance across different language models
- Simulate diverse user types and interaction styles
- Generate comprehensive comparison tables across models
- Track performance metrics across different configurations
🎯 Use cases
- Evaluate and migrate between language models (e.g., Claude to GPT-4)
- Optimize costs by testing smaller, more efficient models
- Refine prompts with clear performance metrics
- Test edge cases and diverse user personas for voice agents
📦 Installation
🧰 Requirements: Python 3, OpenAI API key, and a .env file with OPENAI_API_KEY.
git clone https://github.com/saharmor/voice-lab.git
cd voice-lab
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
Create a .env file in the project root with:
OPENAI_API_KEY=your_openai_api_key
🚀 Usage
Run the pre-defined test:
python llm_testing/example_test.py
For advanced configuration, use the Voice Lab Configuration Editor to generate JSON config files.
⚠️ Good to know
Currently only supports testing the text part of a voice agent (the underlying language model and prompt), not audio analysis or voice-specific features.
❓ FAQ
Does Voice Lab support audio or voice analysis?
No, currently it only supports testing the text part of a voice agent, i.e., the underlying language model and prompt.
How do I add new test scenarios?
You can edit the test_details.json file in the llm_testing directory or use the Voice Lab Configuration Editor to generate JSON config files.
What language models are supported?
The README mentions examples like Claude Sonnet, GPT-4, and GPT-4 Mini, but it requires an OpenAI API key, suggesting OpenAI models are supported.
Can I use Voice Lab for non-voice LLM agents?
Yes, the README states it is valuable for any LLM-powered agent evaluation needs, though optimized for voice agents.
📊 Repository
🤖 Overview, features, install steps and FAQ were generated from the project's README on Sep 4, 2026. Always check the original source before running commands.