Openagent

A modern desktop AI agent client — built with Tauri, SvelteKit, and Rust.

Table of contents

• Table of contents
• What's new in 0.24
• Highlights
  • Agent runtime
  • Interactive output
  • Tools and integrations
  • Desktop experience
• Quick Start
  • Install a release build
  • Prerequisites
  • Clone & install
  • Run in dev mode
  • Build a distributable
• Configure your first provider
  • Enable web search (optional)
  • Choose a tool approval mode
• Example: write a Skill
• Reusable roles and progressive Skill discovery
  • Reusable delegated roles
  • Progressive Skill discovery
• Interactive prompts with ask_user
• AGUI — Inline Interactive Components
• HTML Preview via Tool Call (render_html)
• Memory file format
• Agent memory controls
• Architecture at a glance
• Project structure
• Repository activity
  • Star history
• Contributors
• Roadmap
• Contributing
• Observability (optional)
  • Tool-call structured output integration test
• Further reading
• License

What's new in 0.24

• One window per workspace — Open multiple projects at once while each window keeps its own active workspace; selecting an already-open workspace focuses its existing window.
• Reusable delegated roles — Save specialized agent workflows globally or per project, search them by responsibility, and dispatch them into live child conversations.
• Smarter Skill discovery — Group Skills with metadata.category; uncategorized Skills can be classified by a Flash task so large catalogs use progressive disclosure instead of filling every prompt.
• Durable multimodal branches — Image, PDF, and text attachments have rich previews, survive reloads and checkpoints, and remain available when editing a message to create a new branch.
• Faster long conversations — Conversation search and pagination, virtualized completed history, stable streaming rows, queued follow-up messages, and cancellable compaction keep large histories responsive.
• Richer tool output — Validated Mermaid rendering, image/video AGUI media, source-line previews, and managed background terminal sessions make long-running and visual work easier to follow.

For the complete release history and fixes, see CHANGELOG.md.

Highlights

Agent runtime

• Multi-Agent & Flash Agents Architecture — A primary streaming Chat Agent for main conversations, and a suite of dedicated async Flash Agents (including Memory Agent for long-term memory synthesis, Title Agent for dynamic conversation renaming, and Hook Agent for background scheduled tasks).
• Sub-Agent Delegation — The Chat Agent can call spawn_agent to delegate tasks to nested sub-agents; progress streams in real-time into a sub-conversation shown nested under the parent in the sidebar.
• Reusable Agent Roles — Create global or project-scoped role workflows, discover them with hybrid search, and dispatch them as specialized child agents. Roles can be created automatically on first use or managed from the Roles panel.
• Goal & Graph Loops (Autonomous Execution) — Type /goal in the chat to run a self-correcting loop directly aiming at the objective. Type /graph to first plan a structured DAG task graph (create_goal_graph_config) and execute nodes asynchronously with parallel processing, feeding live graph status updates to the frontend.
• Hybrid Long-Term Memory — SQLite + FTS5 + 384-dim embeddings (fastembed AllMiniLML6V2) blended with time decay for cross-session recall. Before retrieval, an optional Flash task rewrites the latest message into a focused semantic query, so stored memories are matched to intent rather than just wording.
• Interactive User Prompts (ask_user) — The agent can pause mid-task and surface a structured form to the user — text, select, checkbox_group, confirm, date, and more. The agent blocks until the user responds, then continues with the collected values. No more one-shot guessing on ambiguous instructions.

Interactive output

• AGUI — Inline Interactive Components — The agent can embed file and URL capsules, ECharts visualisations, source-line previews, and image/video media directly in its prose — all rendered live by the streamdown engine.
• HTML Preview via Tool Call (render_html) — The agent executes a dedicated render_html tool to display sandboxed native HTML previews (mockup pages, layouts, visual components) directly in the conversation stream, with fixed-height or expanded display modes and PNG export.
• Validated Mermaid Rendering — A dedicated render tool validates Mermaid source before presenting the diagram, while keeping the source available and supporting fullscreen inspection.

Tools and integrations

• MCP-Native — Connect external MCP servers over HTTP or stdio; tools are injected into the agent at call time.
• First-class Dev Tools — Built-in file, search, fetch, and terminal tools. Managed terminal sessions support interactive or long-running background processes; fetch retrieves pages locally with Spider, extracts readable text, and supports pagination.
• Tool Approval & Workspace Sandbox — Choose per-call review, model-assisted approval, no approval, or a workspace-oriented sandbox for file and terminal tools.
• Skills System — Drop a SKILL.md into ~/.agents/skills/ or <workspace>/.agents/skills/. Category-based progressive discovery keeps large global and project catalogs compact, with optional Flash classification for uncategorized Skills.
• Checkpoints & File-Change Rollback — Every turn is a checkpoint with reverse diffs; undo a single file or rewind the whole agent.
• Pluggable LLMs & Multimodal — Support multi-model selection and durable image, PDF, and text attachments with drag/paste, rich previews, checkpoint restoration, and branch editing across Anthropic, OpenAI, and compatible providers.

Desktop experience

• Context Compaction & Tree Conversations — Automatically or manually compact long conversations into tree structures to save tokens, preserving history lineage via search-based message recall.
• Responsive Conversation History — Search and paginate the sidebar, queue follow-up messages during a run, and navigate virtualized transcripts without loading the entire history into the DOM.
• Scheduled Chat Hooks — Define recurring or one-off tasks triggered in the background. Hooks are fully persistent, auto-restored on startup, and supported by system tray notifications.
• Project Drafts & Global/Local Scopes — Keep drafts, memory, and skills scoped globally (in ~/.config/openagent) or locally to your active workspace (in .agents/).
• DESIGN.md & MDX Editor — Dedicated edit panel for DESIGN.md in your workspace, plus a rich markdown editor (MdxMarkdownEditor) integrated into memory and skill management.
• Multi-Workspace Desktop Integration — Open each workspace in a dedicated window, focus existing workspace windows instead of duplicating them, launch on startup, minimize to the system tray, and reveal workspace locations in the native file manager.
• Observability — Optional Langfuse tracing via OpenTelemetry (gen_ai.* attributes).
• Polished UI — Apple-style design language, streaming markdown, Mermaid & ECharts rendering, light/dark themes, i18n (zh / en).

Quick Start

Install a release build

Download the latest installer or app bundle from GitHub Releases. OpenAgent publishes separate beta and stable update channels; you can also check for updates manually from Settings.

To build from source instead, continue below.

Prerequisites

On Windows the Tauri prerequisites also include WebView2 and the MSVC build tools. See the official Tauri prerequisites for platform-specific setup.

Clone & install

git clone https://github.com/BANG404/openagent.git
cd openagent
bun install

Run in dev mode

# Full Tauri desktop app (frontend + Rust backend)
bun tauri dev

# OR — frontend only (port 14221, no Rust)
bun run dev

Build a distributable

bun tauri build

The built installers / app bundle land in src-tauri/target/release/bundle/.

Configure your first provider

On first launch OpenAgent creates ~/.config/openagent/config.toml. Open Settings → Providers and add a provider, or edit the file directly. Until an available model is configured, the composer keeps sending disabled and provides a Configure models shortcut to Settings.

[[providers]]
id = "anthropic-main"
name = "Anthropic"
provider = "anthropic"
api_key = "sk-ant-..."
base_url = "https://api.anthropic.com"
enabled = true

[defaults]
chat_model   = { provider_id = "anthropic-main", model = "claude-sonnet-4-6" }
flash_model  = { provider_id = "anthropic-main", model = "claude-haiku-4-5" }

OpenAI-compatible endpoints (DeepSeek, OpenRouter, local Ollama, etc.) work the same — just point base_url at the right host and set provider = "openai". You may enter a host, a /v1 API root, or a full /chat/completions URL; OpenAgent normalizes it to the API root.

Enable web search (optional)

Open Settings → Web Search and configure the provider you want to use. The web_search tool is available only after the selected provider has the required value: a Brave or Tavily API key, or a SearXNG base URL. The independent fetch tool always retrieves a page directly with Spider, returns readable text, and paginates long results according to the configured fetch page size. Omit page for the first page and use subsequent 1-based page numbers for the rest. Saving Settings refreshes the tool list immediately.

Choose a tool approval mode

Open Settings → General → Approval Mode to control how agent tool calls run. When a call needs your approval, OpenAgent pauses the conversation, shows the exact tool name and arguments, and resumes after you approve or deny it.

Example: write a Skill

Create ~/.agents/skills/python-review/SKILL.md:

---
name: python-review
description: Review Python diffs for type-hint coverage, error handling, and PEP 8 compliance.
---

When asked to review Python code:
1. Check that public functions have type hints.
2. Flag bare `except:` clauses and silent failures.
3. Suggest more idiomatic stdlib alternatives where appropriate.

That's it — OpenAgent picks it up on the next message and lists it in the agent's system prompt. The agent reads the full body on demand via read_file. OpenAgent also installs the bundled find-skills Skill into this global directory when it is missing.

Reusable roles and progressive Skill discovery

Reusable delegated roles

Open Roles in the sidebar to create and manage specialized workflows such as a code reviewer, release manager, or research assistant. A role contains a stable name plus the responsibilities, boundaries, workflow, and delivery standards appended to the delegated agent's system prompt.

• Global roles are reusable in every workspace.
• Project roles are visible only in the current workspace.
• The main agent can create a role on first dispatch, find saved roles by name or responsibility, and reuse them in child conversations. The Roles panel shows usage count and last-used time.

Progressive Skill discovery

For a large Skill catalog, add a category under the frontmatter metadata map:

---
name: python-review
description: Review Python diffs for correctness and maintainability.
metadata:
  category: code-quality
---

OpenAgent initially exposes compact category summaries and loads the matching Skill descriptions on demand. When the optional Settings → Flash Tasks → Skill Category Task is enabled, the app categorizes ungrouped Skills in the background after startup or a workspace switch and persists the result to metadata.category in each SKILL.md.

Interactive prompts with ask_user

When the agent needs a decision before continuing — ambiguous instruction, technology choice, destructive operation, missing parameter — it calls ask_user to surface a structured form in the chat panel. The agent blocks until you respond; everything in the form is typed, so you can select from dropdowns, tick checkboxes, confirm a boolean, or pick a date without typing a sentence.

Supported field types:

The agent is guided to ask once, ask clearly, and prefer structured fields over open text boxes.

AGUI — Inline Interactive Components

Beyond markdown, the agent can embed interactive components directly in its responses. The frontend's streamdown renderer picks them up and renders them as rich, clickable elements — no copy-pasting paths or URLs needed.

Syntax: ComponentName(prop: value, prop2: "string")

Multi-series charts use series: [{name, data}, ...].

HTML Preview via Tool Call (render_html)

Instead of writing inline component tags, the agent can execute a dedicated render_html tool to display sandboxed HTML previews (such as mockup pages, layouts, or visual components) directly in the conversation stream. The rendered frame uses the HTML as provided without injecting app theme styles, supports a fixed-height window or expanded content height, and includes best-effort PNG download.

Memory file format

Memory files have two zones. The Memory Agent only writes below the marker comment:

## [User] Personal habits
<!-- You edit freely here; the agent never touches this section -->

## [Agent] Recent context summary
<!-- Memory Agent only operates below this comment -->

• Global memory → ~/.config/openagent/memory.md (every conversation)
• Local memory → <workspace>/.agents/memory.md (workspace-scoped)

Agent memory controls

Open Settings → Flash Tasks → Memory Task to configure the long-term-memory workflow. Both options are enabled by default:

• Agent memory retrieval uses the Flash model to turn the current message into a focused query, retrieves relevant structured memory, and adds the results to the chat agent's system prompt. Turn it off when you want retrieval to use the message directly.
• Personalized new-conversation greeting runs only when the Memory Agent adds or removes memory. It generates and persists one short, natural greeting; memory may shape its tone or lightly suggest one relevant topic, but is never displayed as a user profile.

Disabling the Memory Agent stops its post-conversation extraction task; the two switches above control their respective follow-up behaviors independently.

Architecture at a glance

┌──────────────────────────────┐        Tauri IPC         ┌──────────────────────────────┐
│   SvelteKit Webview (src/)   │  ◄──────────────────►   │  Rust Core (src-tauri/)       │
│ ── routes/+page.svelte       │  invoke / event          │ ── commands/                  │
│ ── lib/components/...        │  chat-chunk              │ ── tools.rs · terminal.rs     │
│ ── lib/chatStream.ts         │  chat-tool-call          │ ── conversation_memory.rs     │
│ ── lib/streamdown/...        │  chat-done               │ ── skills.rs · mcp.rs         │
└──────────────────────────────┘                          │ ── checkpoint.rs · state.rs   │
                                                          │ ── openagent-runtime crate    │
                                                          └──────────────────────────────┘
                                                                       │
                                          ┌────────────────────────────┼────────────────────────────┐
                                          ▼                            ▼                            ▼
                                ┌──────────────────┐         ┌──────────────────┐         ┌──────────────────┐
                                │  SQLite + FTS5   │         │   LLM Providers   │         │   MCP Servers     │
                                │  + fastembed     │         │  Claude · OpenAI  │         │  HTTP · stdio     │
                                └──────────────────┘         └──────────────────┘         └──────────────────┘

See AGENTS.md for the current contributor and architecture guide.

Project structure

.
├── src/                      # SvelteKit frontend (Svelte 5 · TypeScript)
│   ├── routes/               # Page components
│   └── lib/                  # Components, stores, streamdown, types
├── src-tauri/                # Tauri / Rust backend
│   ├── crates/
│   │   └── openagent-runtime/ # Tauri-independent input routing contract
│   ├── src/
│   │   ├── lib.rs            # App setup, plugin registration
│   │   ├── commands/         # Typed IPC commands grouped by domain
│   │   ├── config.rs         # Config schema parsing and management
│   │   ├── state.rs          # Global application state definition
│   │   ├── context.rs        # System prompt assembly
│   │   ├── context_compaction.rs # Context compaction logic for tree-based history
│   │   ├── conversation_memory.rs # SQLite + fastembed hybrid retrieval
│   │   ├── checkpoint.rs     # Per-turn checkpoints + reverse diffs
│   │   ├── skills.rs         # SKILL.md discovery
│   │   ├── mcp.rs            # MCP transports + tool injection
│   │   ├── tools.rs          # Built-in dev tools (read/write file, web search, fetch, etc.)
│   │   ├── terminal.rs       # Shell command execution and session management
│   │   ├── sub_agent.rs      # Reusable role and child-agent execution
│   │   └── tracing_setup.rs  # OpenTelemetry and Langfuse observability initialization
│   └── icons/                # App icons (all platforms)
└── docs/                     # Design specs and architecture notes

Repository activity

Star history

Contributors

Thanks goes to these wonderful people:

Roadmap

• First-class screenshots & demo GIFs in this README
• Pre-built installers via GitHub Releases (beta / stable channels)
• In-app skill marketplace
• Dedicated windows for multiple workspaces

See open issues for the full backlog and feel free to file new ones.

Contributing

Contributions are very welcome — feature ideas, bug fixes, and docs improvements alike.

1. Fork the repo and create a branch from master.
2. Follow the Conventional Commits style — see existing log for the scopes we use (feat(toast):, fix(mermaid):, refactor(ui):, etc.).
3. Run bun run check, bun run lint:actions, and cargo check --manifest-path src-tauri/Cargo.toml before opening a PR.
4. Open a PR — describe why, not just what.

Project conventions live in AGENTS.md; the UI/UX spec lives in docs/design.md.

Observability (optional)

Drop a .env in the project root to enable Langfuse tracing:

LANGFUSE_PUBLIC_KEY=pk-...
LANGFUSE_SECRET_KEY=sk-...
LANGFUSE_HOST=https://cloud.langfuse.com

When keys are present, Chat & Memory agent calls are instrumented with gen_ai.* OpenTelemetry attributes and exported via batch processor.

Tool-call structured output integration test

The Rust test suite includes an opt-in integration test that verifies the configured model can produce structured results for title, memory, hook, and context-compaction tasks by calling the task-specific capture tool. Put the test provider settings in the repository root .env:

OPENAGENT_STRUCTURED_OUTPUT_TESTS=1
OPENAGENT_STRUCTURED_OUTPUT_MODEL=your-model
OPENAGENT_STRUCTURED_OUTPUT_API_KEY=your-api-key
OPENAGENT_STRUCTURED_OUTPUT_BASE_URL=https://your-provider.example/v1

# Optional Langfuse tracing
LANGFUSE_PUBLIC_KEY=pk-...
LANGFUSE_SECRET_KEY=sk-...
LANGFUSE_HOST=https://cloud.langfuse.com

Then run:

cd src-tauri
cargo test env_configured_provider_supports_all_structured_output_tasks -- --nocapture

See .env.example for a copyable template. The test defaults to the OpenAI-compatible provider, so OPENAGENT_STRUCTURED_OUTPUT_MODEL, OPENAGENT_STRUCTURED_OUTPUT_API_KEY, and OPENAGENT_STRUCTURED_OUTPUT_BASE_URL are enough for third-party OpenAI-compatible chat endpoints.

For Anthropic, set OPENAGENT_STRUCTURED_OUTPUT_PROVIDER=anthropic and provide ANTHROPIC_API_KEY.

Further reading

• AGENTS.md — Contributor and architecture reference
• CHANGELOG.md — Full release history
• docs/release.md — Versioning, beta/stable channels, and publishing workflow
• docs/design.md — Apple-style design spec
• Tauri docs · SvelteKit docs · rig (Rust LLM)

License

OpenAgent is dual-licensed:

• Open-source option: GNU GPL v3.0 or later (GPL-3.0-or-later). If you distribute OpenAgent or a derivative work under this option, the GPL requires the corresponding source and GPL freedoms to be provided under its terms; it does not allow a derivative work to be distributed as proprietary.
• Commercial option: a separate commercial license is available for organizations that need to distribute a proprietary derivative work or otherwise need rights outside the GPL.

Commercial licensing is provided only through a separate written agreement. The OpenAgent name and branding remain subject to TRADEMARKS.md.