DeepSeek V3/R1
DeepSeek-V3 is a large Mixture-of-Experts (MoE) language model with 671B total parameters, activating 37B per token. It achieves state-of-the-art performance on benchmarks, comparable to leading closed-source models, while being cost-efficient to train. It is designed for developers and researchers needing a high-performance open-source LLM for various NLP tasks.
✨ Key features
- 671B total params, 37B activated per token
- Auxiliary-loss-free load balancing strategy
- Multi-Token Prediction (MTP) objective for performance and speculative decoding
- FP8 mixed precision training for efficiency
- Supports 128K context length
- Distilled reasoning from DeepSeek-R1
🎯 Use cases
- Chatbot and conversational AI
- Code generation and completion
- Mathematical reasoning and problem solving
- Long-context document understanding
- Open-ended text generation
📦 Installation
🧰 Requirements: Linux with Python 3.10 for local inference; GPU with sufficient memory (e.g., H800) recommended; API access via DeepSeek platform for hosted use.
git clone https://github.com/deepseek-ai/DeepSeek-V3.git
cd DeepSeek-V3/inference
pip install -r requirements.txt
Download model weights from Hugging Face and place them in /path/to/DeepSeek-V3.
🚀 Usage
cd inference
python fp8_cast_bf16.py --input-fp8-hf-path /path/to/fp8_weights --output-bf16-hf-path /path/to/bf16_weightspython convert.py --hf-ckpt-path /path/to/DeepSeek-V3 --save-path /path/to/DeepSeek-V3-Demo --n-experts 256 --model-parallel 16
torchrun --nnodes 2 --nproc-per-node 8 --node-rank $RANK --master-addr $ADDR generate.py --ckpt-path /path/to/DeepSeek-V3-Demo --config configs/config_671B.json --interactive --temperature 0.7 --max-new-tokens 200⚠️ Good to know
Hugging Face Transformers not directly supported; MTP support under active development; FP8 weights provided, BF16 conversion required for some use cases.
❓ FAQ
What is the model size and activation?
DeepSeek-V3 has 671B total parameters with 37B activated per token.
How can I run DeepSeek-V3 locally?
You can use the DeepSeek-Infer demo, SGLang, LMDeploy, TensorRT-LLM, vLLM, or LightLLM. The README provides steps for the demo.
Does it support FP8 inference?
Yes, FP8 weights are provided and supported by several inference frameworks like SGLang and vLLM.
Is there an API available?
Yes, DeepSeek provides an OpenAI-compatible API at platform.deepseek.com.
📊 Repository
🤖 Overview, features, install steps and FAQ were generated from the project's README on Sep 4, 2026. Always check the original source before running commands.