StarCoder 2
StarCoder2 is a family of open-source code generation models (3B, 7B, and 15B) trained on 600+ programming languages from The Stack v2 and natural language text. It solves code completion tasks for developers, researchers, and tool builders. The models are not instruction-tuned, so they are best used for autocomplete-style generation.
✨ Key features
- Trained on 600+ programming languages from The Stack v2
- Available in 3B, 7B, and 15B parameter sizes
- Context window of 16,384 tokens with sliding window attention
- Supports full precision, bfloat16, and 8-bit/4-bit quantization
- Fine-tuning scripts with LoRA and 4-bit quantization provided
- Integration with Hugging Face Transformers and text-generation-inference
🎯 Use cases
- Code completion in IDEs or editors
- Generating boilerplate code from function signatures
- Fine-tuning on domain-specific code datasets
- Building code generation tools and services
- Evaluating code LLMs with BigCode evaluation harness
📦 Installation
🧰 Requirements: Python environment with PyTorch, Hugging Face Transformers (from source), and an HF token for gated models. GPU recommended for full model usage.
pip install -r requirements.txtexport HF_TOKEN=xxxFor full precision usage, install transformers from source:
pip install git+https://github.com/huggingface/transformers.git
🚀 Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
checkpoint = "bigcode/starcoder2-15b"
device = "cuda" # or "cpu"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)
inputs = tokenizer.encode("def print_hello_world():", return_tensors="pt").to(device)
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))
⚠️ Good to know
StarCoder2 models are not instruction models; commands like 'Write a function that computes the square root.' do not work well. They are intended for code completion.
❓ FAQ
What are the model sizes available?
StarCoder2 comes in 3B, 7B, and 15B parameter versions.
How do I run the model on multiple GPUs?
Use device_map="auto" when loading the model with AutoModelForCausalLM.from_pretrained.
Can I fine-tune StarCoder2?
Yes, the repository provides fine-tuning scripts using LoRA and 4-bit quantization with the SFTTrainer from TRL.
What is the context window size?
The context window is 16,384 tokens with sliding window attention of 4,096 tokens.
📊 Repository
🤖 Overview, features, install steps and FAQ were generated from the project's README on Sep 4, 2026. Always check the original source before running commands.