The self-owned AI control plane.

The only AI infrastructure you actually own. Most teams rent this layer: someone else's dashboard holds their prompts, their keys, and their audit trail. This is the version you compile and keep. One Rust binary, one PostgreSQL, four commands from git clone to governed inference. 43 scripted demos prove every claim on your own laptop.

        all install paths →

systemprompt.io · Documentation · Guides · Enterprise factsheet (PDF) · Discord

Not a mockup. A live capture of the dashboard mid-demo: a third-party coding agent (Pi) is driving Claude, GPT, Gemini, and Cerebras models through the governed gateway as user [email protected]. An operator has just disabled Gemini for that user. The agent's next Gemini call was refused with a 403 before any provider was contacted. One audit row, one deny reason, zero restarts.

This project uses systemprompt.io, self-hosted AI governance infrastructure. It is the evaluation template for systemprompt-core, published on crates.io as systemprompt.

Quick start

git clone https://github.com/systempromptio/systemprompt-template
cd systemprompt-template
just setup-local            # prompts: pick a provider (Gemini/Anthropic/OpenAI), enter its key
just start                  # serves governance + agents + MCP + admin on :8080

setup-local prompts for a provider key, or takes keys non-interactively (just setup-local <anthropic_key> [openai_key] [gemini_key]; the first becomes the default provider). Discover the CLI with systemprompt --help. All other install paths, including running a second clone on different ports, are in docs/README.md.

Bring an agent we have never seen. Watch it get governed.

Every AI gateway vendor shows you their own client talking to their own dashboard. That proves nothing. The honest test is a third-party agent, unmodified, pointed at your infrastructure, and the screenshot above is that test running.

Pi is an open-source coding agent from a different company. We did not fork it, patch it, or wrap it. We gave it one provider entry in ~/.pi/agent/models.json pointing at the gateway's Anthropic-compatible /v1/messages endpoint. From that single config block, everything below works out of the box.

One endpoint, four upstream providers. The gateway routes by model id: claude-* to Anthropic, gpt-* to OpenAI, gemini-* to Google, gpt-oss-120b to Cerebras. Pi's /model picker hops between them mid-session. The agent speaks one protocol; the gateway owns the provider sprawl, the keys, and the bill.

Every request is somebody's. The demo registers a fresh non-admin user, issues them an API key from the admin API, and hands that identity to Pi. From then on every call the agent makes is attributed: user, session, model, provider, tokens in and out, cost in microdollars, latency, and the policy decisions that ran before dispatch. One 18-column Postgres table. One query answers "what did the agent do."

Models are permissions, not config. Open Model Selection in the dashboard, pick the user, click Disable on a model. That writes a user-scoped deny rule which the gateway evaluates live on the next request. The agent's next call to that model gets a 403 with a structured reason. No restart, no token rotation, no redeploy. Click Enable and it works again. This is the difference between a router and a control plane: a router forwards what it is given, a control plane decides.

The prompt and the tools are governed too. A Pi extension wires the same four-policy pipeline (scope check, secret scan, blocklist, rate limit) into the agent's input and tool_call events. Paste a live AWS key into a prompt and the turn is denied before any provider sees it. Ask the agent to write a credential to disk and the tool call is blocked before execution, with the reason handed back to the model.

The evidence is not a claim, it is a page. Everything above lands in the dashboard while it happens: Model Selection (/admin/models) shows the per-user toggles next to that user's usage; the request audit trail (/admin/entities/requests) holds the full chain of custody for every call, including the denied ones.

The same demo run from the audit side: 27 requests across four providers, $0.0341 total spend, and 5 errors, which are the deliberate governance denials. Captured from /admin/entities/requests seconds after the run.

Run it yourself, from a fresh clone, in about ten minutes:

examples/pi/setup.sh          # install Pi, wire the gateway provider, branded theme
examples/pi/routes.sh         # split demo models into individually governable routes
examples/pi/new-user.sh       # register a demo user, issue their key, hand Pi the identity

pi -p --provider systemprompt --model gpt-oss-120b "hello"    # Cerebras, governed
pi -p --provider systemprompt --model claude-sonnet-4-6 "hi"  # Anthropic, same endpoint

Then open /admin/models, disable a model for the user, and run the prompt again. The full scripted walkthrough, including the deny-and-recover loop and the tool-gate demos, is in examples/pi/WALKTHROUGH.md.

The point is not Pi. The point is that Pi needed nothing special. Any client that speaks the Anthropic Messages protocol (Claude Code included, see docker/claude-code-clean-room) inherits the same identity binding, the same per-user model permissions, and the same audit spine, because governance lives at the transport layer instead of inside any one tool.

Five properties, each one demonstrable on your laptop before any procurement call.

• A single query answers every AI audit. Every request, scope decision, tool call, model output, and cost lands in one 18-column Postgres table. Six correlation columns (UserId, SessionId, TaskId, TraceId, ContextId, ClientId) bind identity at construction time, so a row without a trace is a programming error.
• Credentials physically cannot enter the context window. The governance process is the parent of every MCP tool subprocess. Keys are decrypted from a ChaCha20-Poly1305 store and injected into the child's environment by Command::spawn(). The parent, which owns the LLM context, never writes the value. 35+ regex patterns deny any tool call that tries to pass a secret through arguments.
• Self-hosted, air-gap capable, single artifact. One Rust binary. One PostgreSQL. No Redis, no Kafka, no Kubernetes, no SaaS handoff. The same binary runs on a laptop, a VM, and an air-gapped appliance without modification. Zero outbound telemetry by default.
• Policy-as-code on PreToolUse hooks. Destructive operations, blocklists, department scoping, six-tier RBAC (Admin, User, Service, A2A, MCP, Anonymous). Rate limiting at 300 req/min per session with role multipliers. Every deny reason is structured and auditable.
• Certifications-ready, not certification-marketing. Tiered log retention from debug (1 day) through error (90 days). 10 identity lifecycle event variants. SIEM-ready JSON events for Splunk, ELK, Datadog, Sumo. Built for SOC 2 Type II, ISO 27001, HIPAA, and the OWASP Agentic Top 10.

Every claim in this README has a script that executes it against the live binary.

./demo/00-preflight.sh                    # acquire token, verify services, create admin
./demo/01-seed-data.sh                    # populate analytics + trace data

# Governance: the audit line
./demo/governance/01-happy-path.sh        # allowed tool call, full trace chain
./demo/governance/05-governance-denied.sh # scope check rejects out-of-role call
./demo/governance/06-secret-breach.sh     # secret-detection blocks exfiltration
./demo/governance/07-rate-limiting.sh     # 300 req/min per session enforced
./demo/governance/08-hooks.sh             # PreToolUse policy-as-code

# Observability: the audit table
./demo/analytics/01-overview.sh           # conversations, costs, anomalies
./demo/infrastructure/04-logs.sh          # structured JSON events, SIEM-ready

# Scale: the overhead budget
./demo/performance/02-load-test.sh        # 3,308 req/s burst, p99 22.7 ms

Full index: demo/README.md. 41 of 43 scripts are free; two cost ~$0.01 each (real model calls).

Every tool call passes a chain of in-process checks, synchronously, in under 5 ms. The chain is extensible; these ship built in. Every decision lands in an 18-column audit row.

  LLM Agent
      │
      ▼
  Governance pipeline  (in-process, synchronous, <5 ms p99)
      │
      ├─ 1. JWT validation       (HS256, verified locally, offline-capable)
      ├─ 2. RBAC scope check     (Admin · User · Service · A2A · MCP · Anonymous)
      ├─ 3. Secret detection     (35+ regex: API keys, PATs, PEM, AWS prefixes)
      ├─ 4. Blocklist            (destructive operation categories)
      └─ 5. Rate limiting        (300 req/min per session, role multipliers)
      │
      ▼
  ALLOW or DENY   →  18-column audit row, always
      │
      ▼ (ALLOW)
  spawn_server()
      │
      ├─ decrypt secrets from ChaCha20-Poly1305 store
      └─ inject into subprocess env vars only (never parent)
      │
      ▼
  MCP tool process     credentials live here, never in the LLM context path

Run it: ./demo/governance/05-governance-denied.sh · Feature detail

Not a policy that asks agents nicely. A process boundary: the parent that owns the LLM context never writes the credential value.

When a tool call passes the pipeline, spawn_server() decrypts credentials from the ChaCha20-Poly1305 store and injects them into the child process environment. Source: systemprompt-core/crates/domain/mcp/src/services/process/spawner.rs.

let secrets = SecretsBootstrap::get()?;

let mut child_command = Command::new(&binary_path);

// Child env only. The parent (LLM context path) never touches the value.
if let Some(key) = &secrets.anthropic {
    child_command.env("ANTHROPIC_API_KEY", key);
}
if let Some(key) = &secrets.github {
    child_command.env("GITHUB_TOKEN", key);
}

// Detach; parent forgets the child after spawn.
let child = child_command.spawn()?;
std::mem::forget(child);

Before spawn, secret detection scans tool arguments for 35+ credential patterns. A tool call that tries to pass a secret through the context window is blocked even if the agent has scope to run the tool. The hero recording above is the scripted proof: ./demo/governance/06-secret-breach.sh.

Governance that adds more than 1% latency gets bypassed. This one doesn't. Each request performs JWT validation, scope resolution, three rule evaluations, and an async audit write.

Reproduce: just benchmark. Numbers measured on the author's laptop.

• http://localhost:8080: admin UI, live audit table, session viewer.
• systemprompt analytics overview: conversations, tool calls, costs in microdollars, anomalies flagged above 2x/3x of rolling average.
• systemprompt infra logs audit <request-id> --full: the full trace for any request: identity, scope, rule evaluations, tool call, model output, cost. One query, one row, one answer.
• Point Claude Code, Claude Desktop, or any MCP client at it. Permissions follow the user, not the client. Try to exfiltrate a key through a tool argument and watch the secret-detection layer deny it before the tool process spawns.
• ./demo/governance/06-secret-breach.sh: the scripted version of that denial, recorded above.

Runtime configuration is flat YAML under services/, loaded through services/config/config.yaml. Unknown keys fail loudly (#[serde(deny_unknown_fields)]). No database-stored config, no admin UI required. Every change is a diff.

services/
  config/config.yaml        Root aggregator
  agents/<id>.yaml          Agent: scope, model, tool access
  mcp/<name>.yaml           MCP server: OAuth2 config, scopes
  skills/<id>.yaml          Skill: config + markdown instruction body
  plugins/<name>.yaml       Plugin bindings (references agents, skills, MCP)
  ai/config.yaml            AI provider config (Anthropic, OpenAI, Gemini)
  scheduler/config.yaml     Background job schedule
  web/config.yaml           Web frontend, navigation, theme
  content/config.yaml       Content sources and indexing

Eight CLI domains cover every operational surface. No dashboard required for any task.

Each recording is a live capture of the named script running against the binary.

Infrastructure: one binary, one process, one database. Same artifact runs laptop to air-gap.

All data on your infrastructure, zero outbound telemetry · ./demo/infrastructure/01-services.sh · Feature

Profile YAML promotes environments without rebuilding · ./demo/cloud/01-cloud-overview.sh · Feature

Every operational surface has a CLI verb · ./demo/infrastructure/03-jobs.sh · Feature

MCP, OAuth 2.0, PostgreSQL, Git · zero proprietary protocols · ./demo/mcp/01-mcp-servers.sh · Feature

MCP governance, analytics, closed-loop agents, compliance.

Each MCP server is an isolated OAuth2 resource server with per-server scope validation · ./demo/mcp/02-mcp-access-tracking.sh · Feature

Nine analytics subcommands, anomaly detection, SIEM-ready JSON · ./demo/analytics/01-overview.sh · Feature

Agents query their own error rate, cost, and latency via MCP tools and adjust · ./demo/agents/04-agent-tracing.sh · Feature

Tiered retention, 10 identity lifecycle events, SOC 2 / ISO 27001 / HIPAA / OWASP Agentic Top 10 · ./demo/users/03-session-management.sh · Feature

Integrations: any provider, web publisher, extensions.

Anthropic, OpenAI, Gemini swap at the profile level · cost attribution in integer microdollars · ./demo/agents/01-list-agents.sh · Feature

Same binary serves your website, blog, and docs · systemprompt.io runs on this binary · ./demo/web/01-web-config.sh · Feature

Your code compiles into your binary via the Extension trait · no runtime reflection · ./demo/skills/04-plugin-management.sh · Feature

3,308 req/s burst, p99 22.7 ms · just benchmark

Claude for Work ships with extension points for inference, identity, and audit. Point them at this binary and every prompt, tool call, and cost line lands in a Postgres row you own.

  Developer Machine              Enterprise Gateway              Upstream Inference
  (Pi, Claude Code, curl)        (this binary, your VPC)         (pluggable)
  ───────────────── ──────────▶  ─────────────────────  ──────▶  ─────────────────
  Access token                   /v1/messages                    Anthropic direct
  Managed MCP list               Governance pipeline             Bedrock / Vertex
  Signed plugins                 Audit to Postgres               OpenAI / Groq
                                                                 On-prem vLLM / Qwen
                                                                 Air-gap capable

The same governance pipeline described above enforces scope, secrets, policy, and quota before a byte leaves your network, in-process against a cached entitlement table: p99 22.7 ms, <1% of AI response time.

How it compares

Any client that speaks the Anthropic Messages API points at this gateway with an access token issued on /admin/access/tokens. Full walkthrough: examples/pi/.

POST /v1/messages at the Anthropic wire format. Every inference request flows through the same governance pipeline as every tool call. A route maps a requested model pattern to a provider you declared:

gateway:
  enabled: true
  default_provider: anthropic
  routes:
    - model_pattern: "claude-*"
      provider: anthropic
    - model_pattern: "MiniMax-*"
      provider: minimax

Routes evaluate in order; first match wins. Anthropic is a transparent byte proxy; OpenAI-compatible providers get full request/response/SSE conversion. Provider declarations, CLI route configuration, route access control, and the extensible provider registry: docs/gateway-routes.md.

License

This template is MIT. Fork it, modify it, use it however you like.

systemprompt-core is BSL-1.1: free for evaluation, testing, and non-production use. Production use requires a commercial license. Each version converts to Apache 2.0 four years after publication. Licensing enquiries: [email protected].

You can rent your AI control plane, or you can compile it. Clone, build, run the 43 demos. Then decide.