StarCoder 2

Models & AI Labs 💻 Python ⚖️ Apache-2.0 🔴 No recent commits
2.1k stars

StarCoder2 is a family of open-source code generation models (3B, 7B, and 15B) trained on 600+ programming languages from The Stack v2 and natural language text. It solves code completion tasks for developers, researchers, and tool builders. The models are not instruction-tuned, so they are best used for autocomplete-style generation.

✨ Key features

  • Trained on 600+ programming languages from The Stack v2
  • Available in 3B, 7B, and 15B parameter sizes
  • Context window of 16,384 tokens with sliding window attention
  • Supports full precision, bfloat16, and 8-bit/4-bit quantization
  • Fine-tuning scripts with LoRA and 4-bit quantization provided
  • Integration with Hugging Face Transformers and text-generation-inference

🎯 Use cases

  • Code completion in IDEs or editors
  • Generating boilerplate code from function signatures
  • Fine-tuning on domain-specific code datasets
  • Building code generation tools and services
  • Evaluating code LLMs with BigCode evaluation harness

📦 Installation

🧰 Requirements: Python environment with PyTorch, Hugging Face Transformers (from source), and an HF token for gated models. GPU recommended for full model usage.

pip install -r requirements.txt
export your HF token, found here: https://huggingface.co/settings/account
export HF_TOKEN=xxx

For full precision usage, install transformers from source:

pip install git+https://github.com/huggingface/transformers.git

🚀 Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

checkpoint = "bigcode/starcoder2-15b"
device = "cuda" # or "cpu"

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)

inputs = tokenizer.encode("def print_hello_world():", return_tensors="pt").to(device)
outputs = model.generate(inputs)
print(tokenizer.decode(outputs[0]))

⚠️ Good to know

StarCoder2 models are not instruction models; commands like 'Write a function that computes the square root.' do not work well. They are intended for code completion.

❓ FAQ

What are the model sizes available?

StarCoder2 comes in 3B, 7B, and 15B parameter versions.

How do I run the model on multiple GPUs?

Use device_map="auto" when loading the model with AutoModelForCausalLM.from_pretrained.

Can I fine-tune StarCoder2?

Yes, the repository provides fine-tuning scripts using LoRA and 4-bit quantization with the SFTTrainer from TRL.

What is the context window size?

The context window is 16,384 tokens with sliding window attention of 4,096 tokens.

📊 Repository

Stars★ 2,087
Forks🍴 198
Open issues🐛 19
Last commit🕒 Mar 21, 2024
Created📅 Dec 2023
Language💻 Python
License⚖️ Apache-2.0

🤖 Overview, features, install steps and FAQ were generated from the project's README on Sep 4, 2026. Always check the original source before running commands.