Claude Skills · Code Review & Testing
Experiment Forensics
wanshuiyin/Anti-AutoresearchAudit experiment integrity against the evidence ledger. At L2 (repo + result files present) a fresh cross-model reviewer reads the eval code line-by-line for fake/derived ground truth, score self-normalization, phantom results (a paper number with no backing file/key), dead/uncalled metric code, verified-scope inflation, method-described ≠ method-evaluated drift, synthesized-looking results, placeholder/fake data still wired into a released result, code-output ≠ reported-number mismatch, and missing reproducibility artifacts (an empirical/agent/LLM paper shipping neither code nor the prompts/c...
At a glance
This skill is for Code Review & Testing and helps you audit experiment code for integrity, detect fake or derived ground truth, and identify phantom results without backing files.
git clone --depth 1 https://github.com/wanshuiyin/Anti-Autoresearch
cp -r Anti-Autoresearch/skills/experiment-forensics ~/.claude/skills/experiment-forensics
Setup, runtime and requirements describe wanshuiyin/Anti-Autoresearch, the repo this skill ships in.
Empirical ValidationExperiment IntegrityCode AuditResearch ReproducibilityMetrics VerificationFraud Detection
Also in wanshuiyin/Anti-Autoresearch
View the repoSynthesize the single strongest EVIDENCE-BOUND reviewer case to reject a paper, built ONLY from the evidence ledger (claims.json) + the othe...
Transparent, itemized impressions of AI-generated WRITING STYLE — the repo's ONLY non-integrity track. Two passes: a deterministic defensive...
End-to-end substantive-integrity forensic sweep of a research paper (especially autoresearch / AI-Scientist-style output). Orchestrates the...
Audit whether a paper's baseline comparisons are COMPLETE, FAIR, and SIGNIFICANT: a required recent SOTA baseline is missing while 'best/SOT...
Citation-integrity forensics: is every reference real, correctly attributed, and used in a context the cited work actually supports? Catches...
Flagship intra-paper self-consistency forensics: does the paper contradict ITSELF across abstract/intro/tables/body/appendix, and does the m...
Audit whether a paper's EVALUATION DESIGN actually measures what it claims and whether its reporting is complete — the validity layer family...
Build the deterministic evidence ledger (artifact_manifest.json + claims.json) that every other Anti-Autoresearch auditor reads. One pass in...
MEMO-ONLY prior-work overlap advisory: surfaces the two ADVISORY taxonomy signals neither a tool nor a model can decide from the paper alone...
Checkable-ish surface presentation signals a reviewer notices first — duplicate/near-identical tables, leftover pipeline/template strings, t...
Family-G proof & derivation integrity forensics: does a THIRD PARTY's written proof/derivation actually establish its theorem, or does it sk...
Other Code Review & Testing skills
A relentless interview to sharpen a plan or design, which also creates docs (ADR's and glossary) as we go.
Use when you need to resolve an in-progress git merge/rebase conflict.
Create exercise directory structures with sections, problems, solutions, and explainers that pass linting. Use when user wants to scaffold e...
Migrate test files from `as` type assertions to @total-typescript/shoehorn. Use when user mentions shoehorn, wants to replace `as` in tests,...
Move issues and external PRs through a state machine of triage roles — categorise, verify, grill if needed, and write agent-ready briefs.
A relentless interview that asks every frontier question at once, round by round.