Yi-Lightning
Yi is a series of open-source bilingual (English/Chinese) large language models trained from scratch by 01.AI. It aims to provide strong performance in language understanding, reasoning, and reading comprehension for developers and researchers. The models come in various sizes (6B, 9B, 34B) and include base and chat variants.
✨ Key features
- Bilingual (English/Chinese) LLMs trained on 3T multilingual corpus
- Multiple model sizes: 6B, 9B, 34B
- Chat models with SFT training for diverse responses
- Quantized versions (4-bit, 8-bit) for consumer GPUs
- Long context variants up to 200K tokens
- Open-source with community license
🎯 Use cases
- Language understanding and generation tasks
- Commonsense reasoning and reading comprehension
- Coding and math tasks (especially Yi-9B)
- Creative writing and diverse response generation
- Fine-tuning for specific downstream applications
📦 Installation
🧰 Requirements: Python, PyTorch, and Hugging Face Transformers; for quantized models, consumer-grade GPUs like 3090 or 4090 are sufficient.
pip install transformers
For quantized models, install additional dependencies:
pip install bitsandbytes accelerate
🚀 Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("01-ai/Yi-6B-Chat")
tokenizer = AutoTokenizer.from_pretrained("01-ai/Yi-6B-Chat")
inputs = tokenizer("Hello, who are you?", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=128)
print(tokenizer.decode(outputs[0]))
⚠️ Good to know
Chat models may exhibit hallucination, non-determinism in re-generation, and cumulative errors; adjust generation parameters like temperature to mitigate.
❓ FAQ
What is the difference between base and chat models?
Base models are pre-trained on raw text, while chat models are fine-tuned with Supervised Fine-Tuning (SFT) to produce more diverse and instruction-following responses.
Can I use Yi models commercially?
Yes, the Yi series models are open-source with a community license that permits commercial use, subject to the terms of the license agreement.
How do I handle the long context window?
Models with 200K context are available (e.g., Yi-34B-200K). For other models, the context can be extended up to 32K during inference.
What are the hardware requirements for running quantized models?
Quantized models (4-bit and 8-bit) can be deployed on consumer-grade GPUs such as NVIDIA 3090 or 4090.
📊 Repository
🤖 Overview, features, install steps and FAQ were generated from the project's README on Sep 4, 2026. Always check the original source before running commands.