agentlego
AgentLego is an open-source library of versatile tool APIs to extend and enhance large language model (LLM) based agents. It provides a rich set of tools for multimodal extensions, including visual perception, image generation and editing, speech processing, and visual-language reasoning. It is designed for developers building LLM agents who need easy integration with frameworks like LangChain, Transformers Agents, and Lagent.
✨ Key features
- Rich set of tools for multimodal extensions of LLM agents
- Flexible tool interface for custom tools with arbitrary arguments and outputs
- Easy integration with LangChain, Transformers Agents, and Lagent
- Support tool serving and remote accessing for heavy ML models
🎯 Use cases
- Enhance LLM agents with image description and OCR capabilities
- Add speech-to-text and text-to-speech functions to agents
- Integrate image generation and editing tools into agent workflows
- Use remote tools that require GPU or special environments
📦 Installation
🧰 Requirements: Python environment with pip; some tools require additional dependencies like openmim and mmpretrain; optional GPU for heavy models.
pip install agentlego
Some tools require extra packages. For example, to use ImageDescription, install:
pip install -U openmim
mim install -U mmpretrain
🚀 Usage
from agentlego import list_tools, load_tool
print(list_tools()) # list tools in AgentLego
image_caption_tool = load_tool('ImageDescription', device='cuda')
print(image_caption_tool.description)
image = './examples/demo.png'
caption = image_caption_tool(image)
⚠️ Good to know
Some tools require extra dependencies and may need GPU or special environments; users must comply with licenses of underlying models.
❓ FAQ
How do I install AgentLego?
You can install it via pip: pip install agentlego. Some tools may require additional dependencies, which are listed in each tool's readme.
Can I use AgentLego with LangChain?
Yes, AgentLego supports easy integration with LangChain, Transformers Agents, and Lagent, as shown in the examples directory.
What tools are available in AgentLego?
AgentLego includes tools for general abilities (Calculator, GoogleSearch), speech (TextToSpeech, SpeechToText), image processing (ImageDescription, OCR, VQA, pose estimation, etc.), and AIGC (TextToImage, image editing, ControlNet series, ImageBind series).
Do I need a GPU to use AgentLego?
Not necessarily, but some tools with heavy ML models may require GPU and CUDA. You can specify device='cuda' when loading a tool, or use remote serving for such tools.
📊 Repository
🤖 Overview, features, install steps and FAQ were generated from the project's README on Sep 4, 2026. Always check the original source before running commands.