Decision-Grade Data Science

Open-source AI agent skills for reliable data analysis, machine learning, and AI modeling — from ambiguous questions and messy data to validated, reproducible decisions.

한국어 · Methodology · Evaluation · Contributing

Decision-Grade Data Science — From messy data to defensible decisions.

From messy data to defensible decisions — see the evidence-first methodology.

Seven skills — a router, a lifecycle orchestrator, and five specialists — that teach coding agents to preserve the chain from a real decision to the evidence that supports it. Built for work where a polished notebook or a high validation score is not enough.

Why this exists

AI agents can write analysis code quickly. They can also silently choose the wrong row grain, treat a convenient field as ground truth, leak future information, move an evaluation target after seeing results, or declare completion after one successful run.

This suite adds a human-controlled operating discipline:

decision contract
  → data and ground-truth audit
  → leakage-safe baseline
  → controlled experiments
  → independent validation
  → failure diagnosis
  → clean-room reproduction and handoff

The goal is not to slow work down. It is to make speed come from clear decisions, reusable evidence, and falsifiable checks instead of skipped validation.

Skills

One router picks the right entry, one orchestrator runs the full lifecycle, and five specialists own its stages:

using-data-analysis  (router — start here when the right skill is unclear)
  └─ running-decision-grade-data-science  (orchestrates the lifecycle)
       1. auditing-data-and-ground-truth      — audit
       2. designing-leakage-safe-experiments  — experiment design
       3. validating-models-and-claims        — validation
       4. diagnosing-ml-failures              — diagnosis
       5. shipping-reproducible-results       — reproducible handoff
Skill Use it for
Using Data Analysis Skills — entry point Route any data analysis or ML request to the narrowest matching skill
Run Decision-Grade Data Science Orchestrate an ambiguous or end-to-end data science project
Audit Data and Ground Truth Verify grain, joins, time semantics, missingness, labels, and source-of-truth reliability
Design Leakage-Safe Experiments Lock prediction timing, splits, baselines, metrics, and fair comparisons
Validate Models and Claims Determine exactly which model or analytical claims the evidence supports
Diagnose ML Failures Isolate regressions across data, labels, pipelines, artifacts, metrics, and runtime
Ship Reproducible Results Package provenance, reproduction status, limitations, and handoff ownership

Each skill includes focused instructions, deeper reference material, and a reusable output template. Use the orchestrator for a full project and the narrower skills for isolated goals; the router selects for you when the match is not obvious.

Install

Inspect any third-party skill before installation.

GitHub CLI

With a GitHub CLI release that includes the gh skill public preview:

gh skill preview aiopshwang/data-analysis-ml-agent-skills
gh skill install aiopshwang/data-analysis-ml-agent-skills running-decision-grade-data-science --agent codex

Replace the skill name or agent host as needed. Supported hosts in the GitHub CLI include Codex, Claude Code, Cursor, GitHub Copilot, and Gemini CLI.

Agent Skills CLI

Install all skills globally for Codex:

npx skills add aiopshwang/data-analysis-ml-agent-skills --skill '*' --global --agent codex --copy --yes

For Claude Code, replace codex with claude-code.

Manual installation

Clone the repository, review the selected directory, and copy it into your agent's user or project skill directory:

git clone https://github.com/aiopshwang/data-analysis-ml-agent-skills.git

The repository also ships plugin manifests for Claude Code at .claude-plugin/plugin.json and for Codex at .codex-plugin/plugin.json.

Try it

Examples:

Use $using-data-analysis to pick the right skill in this suite for my analysis task.

Use $running-decision-grade-data-science to turn this vague churn-model request into a decision-ready project.

Use $auditing-data-and-ground-truth to inspect these tables and labels before modeling.

Use $designing-leakage-safe-experiments to create a fair grouped and temporal evaluation.

Use $validating-models-and-claims to check whether this report supports the launch claim.

Use $diagnosing-ml-failures to isolate the first broken layer behind this regression.

Use $shipping-reproducible-results to prepare an independent clean-room handoff.

Skills allow implicit discovery by default, so ordinary requests that clearly match a goal can also select them.

Operating principles

  • Fix the decision, unit of analysis, target, and time boundary before choosing a model.
  • Preserve raw inputs and make derived data traceable.
  • Treat ground truth as evidence to audit, not a column name to trust.
  • Establish a transparent baseline before adding complexity.
  • Change one material factor at a time and retain failed experiments.
  • Match every claim to evidence of the same scope.
  • Separate model judgment, deterministic computation, and human authority.
  • Do not call partial execution, a green unit test, or one successful example complete.

Read the methodology for the complete rationale and boundaries.

What this is not

  • an AutoML framework;
  • a replacement for domain expertise or statistical judgment;
  • a promise that every analysis needs every gate;
  • permission for an agent to access private data, incur material cost, deploy, or publish;
  • a guarantee of model performance, fairness, safety, or regulatory compliance.

The skills adapt depth to the decision risk and ask for human input only when a material choice cannot be safely inferred.

Validation

The repository checks manifest and frontmatter integrity, local links, UI metadata, unfinished scaffold text, trigger-evaluation coverage, and public-content secret patterns.

python -m pip install -r requirements-dev.txt
python scripts/validate_repo.py
python scripts/scan_public.py
pytest

The trigger prompt set contains direct, indirect, and negative cases for all seven skills. Static checks do not prove behavior; see the v0.1.0 independent forward-test record for one realistic readiness scenario run by an independent evaluator on a synthetic fixture.

Privacy and security

The public suite contains generalized procedures and synthetic templates, not customer data or copied project artifacts. Do not paste sensitive data into an agent unless the surrounding product, workspace, permissions, and retention policy are appropriate.

See SECURITY.md for vulnerability reporting and trust boundaries.

aiopshwang skill family

Independent, evidence-first Agent Skills that work well together:

  • goal-to-proof — the general completion gate: finish authorized work and prove the requested outcome.
  • verify-regression-tests — prove that a regression test actually detects its intended defect.
  • ship-mobile-app — production mobile work across domain, state, lifecycle, platform, and release boundaries.
  • fresh-eyes-check — a context-free second model checks whether an earlier instruction still fits before you act on it.

License

MIT