Senior Prompt Engineer — Development

Q: What should I do if RAG relevance is low?

Treat relevance < 0.80 as a retrieval problem (chunking, embeddings, filters) and fix retrieval before rewriting the generation prompt.

Q: How many examples should I use in few-shot prompting?

Start with zero examples and add only for failure clusters, using 3–5 max ordered simple → edge → negative.

This skill provides eval-driven prompt optimization, RAG quality measurement, and agent workflow validation through three command-line tools. It helps users improve prompts, evaluate retrieval and grounding, and validate agent configurations, all in a model-agnostic way.

engineering claude-code

📂 View skill on GitHub ↗ 🚀 How to use

✨ What it does

Static prompt analysis with token estimation and clarity/structure scores.
Baseline comparison to track prompt changes over time.
RAG evaluation metrics including relevance, precision@k, coverage, and groundedness.
Agent config validation for tool wiring, loop risks, and token estimates.
Cost estimation with user-supplied pricing.
Support for extracting few-shot examples and comparing revisions.

🎯 When to use it

When you need to optimize a prompt for clarity, token efficiency, or output consistency.
When you want to measure RAG retrieval quality or answer faithfulness against an eval set.
When you need to validate an agent configuration for tool wiring, loop risks, or cost estimates.
When designing structured output contracts or few-shot examples for LLM calls.

🚀 How to use

Trigger by asking to optimize a prompt, evaluate RAG quality, or validate an agent config. Provide the prompt file, JSON files for RAG evaluation, or a YAML/JSON agent config. Use the provided CLI tools with appropriate flags. Example prompts:

Optimize this prompt file prompt.txt and compare against baseline.
Evaluate RAG retrieval using contexts.json and questions.json.
Validate agent.yaml and estimate cost for 100 runs.

📄 Output: Reports, JSON baselines, optimized prompt files, and validation results.

📦 Add this skill to Claude Code

# 1. Get the skills repo
git clone --depth 1 https://github.com/alirezarezvani/claude-skills /tmp/claude-skills

# 2. Copy this skill into your project (or ~/.claude/skills for all projects)
mkdir -p .claude/skills
cp -r /tmp/claude-skills/engineering-team/skills/senior-prompt-engineer .claude/skills/senior-prompt-engineer

Skill source: engineering-team/skills/senior-prompt-engineer/SKILL.md

⚠️ Good to know

Requires Python 3 and the user to supply current pricing for cost estimates; does not handle model-specific features beyond tokenizer inference.

❓ FAQ

How do I get a cost estimate?

Pass your provider's current per-Mtok rates via --price-per-mtok (or --input-price-per-mtok and --output-price-per-mtok for agent cost estimation).

What should I do if RAG relevance is low?