ToolBench
ToolBench is an open-source, large-scale, high-quality instruction tuning dataset for training LLMs to use real-world APIs. It solves the problem of enabling open-source LLMs to master thousands of diverse APIs by providing a dataset, training/evaluation scripts, and a fine-tuned model ToolLLaMA. It is for developers and researchers building tool-using language models.
✨ Key features
- 16464 real-world REST APIs from RapidAPI
- 126,486 instruction instances with solution paths
- Single-tool and multi-tool scenarios
- DFSDT-based answer annotation with reasoning traces
- API retriever for open-domain tool use
- Pre-trained ToolLLaMA models and retriever
🎯 Use cases
- Fine-tune LLaMA to use APIs for task completion
- Train a tool retriever to select relevant APIs
- Evaluate LLM tool-use performance with ToolEval
- Build open-domain tool-using assistants
📦 Installation
🧰 Requirements: Python >=3.9, PyTorch, and access to RapidAPI (via ToolBench key or local server).
git clone git@github.com:OpenBMB/ToolBench.git
cd ToolBench
pip install -r requirements.txt
For ToolEval only: pip install -r toolbench/tooleval/requirements.txt
Download data:
wget --no-check-certificate 'https://drive.google.com/uc?export=download&id=1XFjDxVZdUY7TXYF2yvzx3pJlS2fy78jk&confirm=yes' -O data.zip
unzip data.zip
🚀 Usage
export PYTHONPATH=./
python toolbench/inference/qa_pipeline.py \
--tool_root_dir data/toolenv/tools/ \
--backbone_model toolllama \
--model_path ToolBench/ToolLLaMA-7b \
--max_observation_length 1024 \
--observ_compress_method truncate \
--method DFS_woFilter_w2 \
--input_query_file data/test_instruction/G1_instruction.json \
--output_answer_file toolllama_dfs_inference_result \
--toolbench_key $TOOLBENCH_KEY
⚠️ Good to know
The project is for research and educational purposes; the dataset is automatically generated and filtered, and the model may not cover all real-world tools.
❓ FAQ
What is ToolBench?
ToolBench is a large-scale instruction tuning dataset for training LLMs to use real-world APIs, along with training and evaluation scripts and a fine-tuned model ToolLLaMA.
How do I get access to the RapidAPI server?
You need to fill out the form at https://forms.gle/S4hqVLtnqeXcNTCJA to request a ToolBench key, which will be sent after review.
What models are available?
ToolLLaMA-2-7b-v2, ToolLLaMA-7b-v1, ToolLLaMA-7b-LoRA-v1, and a tool retriever are released on Hugging Face.
Can I train my own model with ToolBench?
Yes, the repository provides fine-tuning scripts for both full fine-tuning and LoRA, using the preprocessed data.
📊 Repository
🤖 Overview, features, install steps and FAQ were generated from the project's README on Sep 4, 2026. Always check the original source before running commands.