Whatever the job — reading documents, debugging a pipeline, reviewing code, answering domain questions — there is more than one way to do it. A different model, a different tool, a different agent framework, a skill someone swears by. Picking between them is usually guesswork.

trapstreet.run makes it a measurement. A task declares its inputs and the answers it expects; any solution runs against the same cases, is scored by the same judge, and lands on a public board with its score, latency and cost. Every row is pinned to a public repo@commit, so anyone can re-run it and get the same number.

Bring your own task, or run against one that is already there.

This repo holds three skills that do it for you. Install them and your coding assistant sets up the CLI, builds solutions, and authors tasks — from plain language.

Install

npx skills add trapstreet/trapstreet-skills

Installs into every coding agent it detects — Claude Code, Cursor, Codex, Cline, Amp, Antigravity and 70+ more. Start a new session afterwards; skills load at session start.

Or paste this to your agent and it does everything:

Install trapstreet: run `npx -y skills add trapstreet/trapstreet-skills --global --yes`, then read ~/.agents/skills/trapstreet-setup/SKILL.md and follow it.

D=~/.claude/skills; T=$(mktemp -d); mkdir -p "$D" \
  && curl -fsSL https://github.com/trapstreet/trapstreet-skills/archive/refs/heads/main.tar.gz \
   | tar -xz -C "$T" --strip-components=1 \
  && cp -R "$T"/trapstreet-* "$D"/ && rm -rf "$T"

Writes to Claude Code's directory only — change D for another agent — and skills installed this way don't participate in npx skills update.

Then say

Say this Skill What it does
"set up trapstreet" trapstreet-setup Installs and authorizes the tp CLI. One time — local scoring needs no account.
"build a solution for <task>" trapstreet-solution-scaffold Writes trap.yaml and the solver, runs it locally, submits when you're happy. From scratch, around code you have, or by adapting someone else's repo.
"make a task that evaluates <X>" trapstreet-task-scaffold Interviews you on what counts as correct and where ground truth comes from, then writes traptask.yaml, judge.py and grader.py.

Tasks live in their author's own repository, not ours — publish from anywhere public and register it on the site.

uv tool install trap-cli   # install
tp auth login              # authorize this machine, once
tp run && tp submit        # from any directory with a trap.yaml

What a board looks like

Links
trapstreet.run The boards. Browse tasks, read results, register your own.
Docs Quick start · Build a solution · Build a task · Reference
trapstreet/trap The tp CLI. MIT.
trapstreet/trapstreet-tasks 36 reference tasks with judges and gold cases. MIT.

Other platforms

npx skills add writes one copy to ~/.agents/skills/, which Codex, Cursor, Cline, Amp, Antigravity and a dozen more read directly, and symlinks it into Claude Code. Nothing to translate.

For anything that reads neither: each skill is one SKILL.md (YAML frontmatter with name and description, then the body) plus optional references/ and scripts/ — plain markdown and shell commands. Paste the body into your tool's custom-instructions field, or point it at the file at session start.

Contributing

Fixes and platform coverage welcome — see CONTRIBUTING.md. Publishing a task or a solution needs no PR here: those live in your own repository.

License

MIT — see LICENSE.