SimWorld
SimWorld is a simulation platform built on Unreal Engine 5 for developing and evaluating LLM/VLM AI agents in complex physical and social environments. It provides realistic, open-ended world simulation with accurate physics and language-based procedural generation, along with a rich interface for multimodal perception and natural language actions. It is intended for researchers and developers working on embodied AI and agent systems.
🎬 SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds · SimWorld
✨ Key features
- Realistic, open-ended world simulation with accurate physics.
- Language-based procedural generation of city layouts and scenes.
- Rich interface for LLM/VLM agents with multimodal perception.
- Supports natural language actions and local action planning.
- Gym-like API for agent-environment interaction.
- Modular architecture with support for custom environments and agents.
🎯 Use cases
- Training and evaluating LLM/VLM agents in navigation tasks.
- Simulating traffic and testing autonomous driving agents.
- Procedural generation of city layouts for agent testing.
- Research on embodied AI and social reasoning scenarios.
- Customizing simulations with user-provided environments and assets.
📦 Installation
🧰 Requirements: Requires Windows or Linux OS, a dedicated GPU with ≥6GB VRAM, 32GB RAM, and 50-200GB disk space. Python 3.10 or later is needed for the client.
git clone https://github.com/SimWorld-AI/SimWorld.git
cd SimWorld
conda create -n simworld python=3.10
conda activate simworld
pip install -e .
Then download and extract the Base UE server package from the provided links (Windows or Linux). Optionally download Additional Environments (100+ maps) and copy .pak files into SimWorld/Content/Paks/.
🚀 Usage
Start the UE server from the extracted package directory:
- Windows:
./SimWorld.exe <MAP_PATH> - Linux:
./SimWorld.sh <MAP_PATH>
If no MAP_PATH is given, the default map /Game/Maps/demo_1 opens. Then run Python examples from the examples/ directory.
❓ FAQ
What are the system requirements for SimWorld?
SimWorld requires Windows or Linux, a dedicated GPU with at least 6GB VRAM, 32GB RAM, and 50-200GB disk space depending on the package.
How do I install the Python client?
Clone the repository, create a conda environment with Python 3.10, and run pip install -e . from the SimWorld directory.
Can I use my own environments or agents?
Yes, SimWorld supports importing customized environments and agents via .pak files, as described in the 'Make Your SimWorld' section.
What maps are included in the Base package?
The Base package includes two lightweight city scenes and one empty map for quick testing.
📊 Repository
🤖 Overview, features, install steps and FAQ were generated from the project's README on Sep 4, 2026. Always check the original source before running commands.