LLM Eval Suite Scaffolder
imtiazrayhan/agentscamp-libraryStand up an evaluation suite for an LLM feature from scratch — a representative dataset, the right metrics, a baseline score, and a CI gate — using DeepEval, promptfoo, or RAGAS. Use when a feature has no evals, before tuning a prompt, or when adding an LLM feature to CI.
At a glance
This skill is for Coding and helps you set up an llm evaluation suite, create a representative eval dataset, and establish baseline metrics for llm features.
git clone --depth 1 https://github.com/imtiazrayhan/agentscamp-library
cp -r agentscamp-library/skills/llm-eval-suite-scaffolder ~/.claude/skills/llm-eval-suite-scaffolder
Setup, runtime and requirements describe imtiazrayhan/agentscamp-library, the repo this skill ships in.
DeepevalPromptfooRagasLlm TestingEvaluation Metricsci-cd
Also in imtiazrayhan/agentscamp-library
View the repoAudit a UI change for accessibility regressions by combining automated checks with keyboard, focus, semantic, name-role-value, contrast, zoo...
Write an Architecture Decision Record capturing a decision the user describes, in Michael Nygard ADR format (Status, Context, Decision, Cons...
Design a project's CLAUDE.md and memory hierarchy by exploring the repo to learn its real build/test/lint commands, architecture, and non-ob...
Evaluate a multi-step AI agent's whole run — tool calls, intermediate steps, and final result — not just final-answer correctness, so you ca...
Cut alert noise and make every page mean something — rewrite alerting rules to fire on user-felt symptoms (error rate, latency SLO burn, fai...
Plan the retirement of an API endpoint, field, event, tool, or version without surprising active consumers. Use when replacing an interface,...
Design or normalize an API's error contract so clients get stable machine-readable codes, safe human messages, field-level validation detail...
Generate accurate architecture diagrams as Mermaid — straight from the codebase, not from imagination — by first choosing which view answers...
Read-only review of authentication AND authorization flows — session/token model, cookie flags, CSRF, token rotation, password-reset/email-v...
Audit scheduled jobs, queue consumers, workers, and asynchronous workflows for delivery assumptions, idempotency, retries, poison messages,...
Rebase the current branch onto its base and walk every conflict methodically, resolving each by understanding both sides. Use when your feat...
Analyze a JS/TS production bundle and surface the biggest size wins — heavy dependencies, duplicate packages, missing code-splitting, oversi...
Other Coding skills
Behavioral guidelines to reduce common LLM coding mistakes. Use when writing, reviewing, or refactoring code to avoid overcomplication, make...
Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI be...
Guide for creating high-quality MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-design...
Reference for the Claude API / Anthropic SDK — model ids, pricing, params, streaming, tool use, MCP, agents, caching, token counting, model...
Suite of tools for creating elaborate, multi-component claude.ai HTML artifacts using modern frontend web technologies (React, Tailwind CSS,...
Scan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.