NeMo Agent Toolkit
NVIDIA NeMo Agent Toolkit is a framework-agnostic toolkit that adds enterprise-grade instrumentation, observability, and optimization to AI agents built with any framework. It solves the problem of understanding and improving agent performance in production by providing profiling, evaluation, and fine-tuning tools. It is for developers building and deploying AI agents at scale.
✨ Key features
- Framework agnostic: works with LangChain, LlamaIndex, CrewAI, Semantic Kernel, Google ADK, and custom agents.
- Profiling from agent level down to individual tokens to identify bottlenecks.
- Observability with tracing and performance tracking in production.
- Evaluation system for offline validation of agentic workflows.
- Hyper-parameter and prompt optimizer to automatically find best configurations.
- Fine-tuning LLMs with reinforcement learning for agent-specific tasks.
🎯 Use cases
- Instrument and observe LangChain or CrewAI agents in production.
- Profile token usage and latency to optimize agent performance.
- Run offline evaluations to validate workflow accuracy before deployment.
- Automatically tune prompts and hyper-parameters for better agent outcomes.
- Fine-tune an LLM with RL to encode workflow-specific knowledge.
📦 Installation
🧰 Requirements: Python 3.11, 3.12, or 3.13. An NVIDIA API key is required for using NVIDIA NIMs in examples.
To install the latest stable version from PyPI:
pip install nvidia-nat
Optional dependencies are grouped by framework. For example, to install the LangChain/LangGraph plugin:
pip install "nvidia-nat[langchain]"
For running examples, clone the repository and install from source. See the Installation Guide for details.
🚀 Usage
- Set the NVIDIA API key:
export NVIDIA_API_KEY=<your_api_key>
- Create a
workflow.ymlfile:
functions:
wikipedia_search:
_type: wiki_search
max_results: 2
llms:
nim_llm:
_type: nim
model_name: nvidia/nemotron-3.5-lightning-30b-a3b
temperature: 0.0
chat_template_kwargs:
enable_thinking: false
workflow:
_type: react_agent
tool_names: [wikipedia_search]
llm_name: nim_llm
verbose: true
parse_agent_response_max_retries: 3
- Run the workflow:
nat run --config_file workflow.yml --input "List five subspecies of Aardvarks"
⚠️ Good to know
Some features are experimental (e.g., Dynamo integration) and the roadmap indicates ongoing improvements; third-party plugins are managed externally and may have independent release cycles.
❓ FAQ
Which Python versions are supported?
Python 3.11, 3.12, or 3.13 are supported.
Can I use NeMo Agent Toolkit with LangChain?
Yes, it is framework agnostic and includes a LangChain/LangGraph plugin that can be installed via pip install "nvidia-nat[langchain]".
How do I get an NVIDIA API key?
You can obtain an API key by visiting build.nvidia.com and creating an account.
Is telemetry enabled by default?
In interactive mode, you are prompted on first run and the default is yes. In non-interactive contexts, telemetry is off unless explicitly enabled via the NAT_TELEMETRY_ENABLED environment variable.
📊 Repository
🤖 Overview, features, install steps and FAQ were generated from the project's README on Sep 4, 2026. Always check the original source before running commands.