Introduction

GHOST is a personal AI agent platform. A single binary runs one GHOST for one OPERATOR — your own AI companion with persistent memory, background jobs, and multi-interface communication.

Philosophy

• Text-first — workspace files (markdown, TOML) are the primary feature surface. Identity, skills, jobs, and knowledge all live as editable files.
• CLI-first — every feature is accessible through direct commands.
• Skills over tools — prefer reusable workflow files over adding new tool APIs.
• Local-first — embedded database, local embeddings, your data stays on your machine.

Architecture

┌──────────────────────────────────────────┐
│               ghost daemon               │
├──────────┬───────────┬───────────────────┤
│ Discord  │ Job       │ Chat              │
│ Bot      │ Scheduler │ Orchestration     │
├──────────┴───────────┴───────────────────┤
│ Provider Layer (OpenRouter, Kimi, Codex) │
├──────────────────────────────────────────┤
│ SurrealDB (embedded)  │  Workspace Files │
└──────────────────────────────────────────┘

• Single binary — no external services required (except Ollama for embeddings)
• Embedded SurrealDB — graph-based knowledge storage with typed edges
• Discord transport — primary interface for the PoC (more planned)
• Multiple LLM providers — OpenRouter, Kimi Code, OpenAI OAuth (Codex)

Concepts Overview

Quick reference for GHOST’s core concepts. Each links to its detailed chapter.

Installation

Prerequisites

• Rust toolchain — rustup.rs
• Ollama — for local embeddings (ollama.com)
• Crawl4AI — for web page extraction (crawl4ai)
• Discord bot token — for the Discord interface (see Interfaces)

In the future, all dependencies beyond the binary itself will be managed by the GHOST through Nix. For now, you need to set them up manually.

Install

# Clone and install
git clone https://github.com/tolki/ghost.git
cd ghost
cargo install --path .

Bootstrap

# Initialize workspace and config
ghost init

This creates:

• Config file at ~/.config/ghost/config.toml
• Workspace directory at ~/GHOST/ with default identity files, skills, and agents

Secrets

Create a .env file (or set environment variables). Never put API keys in config.toml.

# Required for at least one provider
OPENROUTER_API_KEY=sk-or-...

# Required for web search
BRAVE_API_KEY=BSA...

# Required for Discord
DISCORD_TOKEN=...

Pull Embedding Model

ollama pull qwen3-embedding:8b

Start Crawl4AI

GHOST uses a Crawl4AI container for web page extraction. Run it with Docker:

docker run -d -p 11235:11235 unclecode/crawl4ai

Then set the URL in config.toml:

[web]
crawl4ai_url = "http://localhost:11235"

First Run

ghost daemon

This starts the Discord bot and the job scheduler. Your GHOST is now live.

Configuration

Config File

Location: ~/.config/ghost/config.toml

Override with GHOST_CONFIG_DIR environment variable.

Full Example

# Workspace path (default: ~/GHOST)
workspace = "~/GHOST"

# Model aliases — define one or more (keys are flattened under [models])
[models]
default = "primary" # Which alias to use by default

[models.primary]
provider = "openrouter"
model = "anthropic/claude-sonnet-4"
context_window = 200000

[models.fast]
provider = "kimi_code"
model = "kimi-k2.5"
context_window = 250000

[models.codex]
provider = "openai_oauth"
model = "gpt-5.3-codex"
context_window = 250000

# Discord
[discord]
enabled = true
allowed_user_id = "123456789012345678" # Your Discord user ID

# Embeddings (Ollama)
[embeddings]
url = "http://127.0.0.1:11434" # Ollama endpoint
model = "qwen3-embedding:8b" # Embedding model
batch_size = 32 # Vectors per batch
dimension = 1024 # Vector dimension

# Timing
[timing]
heartbeat_idle_minutes = 4 # Minutes idle before heartbeat
heartbeat_check_seconds = 60 # How often to check for idle
heartbeat_continue_minutes = 30 # Max heartbeat conversation length
reflection_idle_minutes = 4 # Minutes idle before reflection runs
scheduler_tick_seconds = 10 # Cron scheduler poll interval

# Compaction (context window management)
[compaction]
threshold = 0.85 # Compact when context is 85% full
keep_window = 20 # Messages to keep after compaction
mask_preview_chars = 100 # Preview chars for compacted messages

# Web
[web]
search_max_results = 5 # Default Brave search results
# crawl4ai_url = "http://localhost:11235"  # Optional Crawl4AI backend

# Debug
[debug]
save_requests = false # Save raw provider requests to disk

CLI Access

# Read a config value
ghost config get discord.allowed_user_id

# Set a config value
ghost config set timing.heartbeat_idle_minutes 10

Secrets

Never put API keys in config.toml. Use environment variables or a .env file.

Workspace

The workspace is your GHOST’s home directory. Default: ~/GHOST/.

Directory Structure

~/GHOST/
├── BOOT.md              # GHOST behavioral instructions
├── SOUL.md              # GHOST personality and values
├── OPERATOR.md          # Your preferences and goals
├── notes/               # Knowledge notes (markdown with TOML frontmatter)
├── references/          # Source material organized by topic
├── diary/               # Daily entries (YYYY-MM-DD.md)
├── jobs/                # Cron-scheduled job files
├── skills/              # Workflow definitions (agentskills.io format)
├── agents/              # Agent definitions (TOML frontmatter + prompt)
├── .web-cache/          # Cached web fetches
└── .state/              # Runtime state (database, logs)

What ghost init Creates

Running ghost init bootstraps the workspace with:

• Identity files — BOOT.md, SOUL.md, OPERATOR.md with starter templates
• Default skills — research, knowledge-navigator, note-writer, cron-job-author, skill-creator
• Default agents — heartbeat, chat-reflection, reflection, deep-research
• All directories listed above

Key Directories

Identity

GHOST follows a one GHOST, one OPERATOR model. Identity is defined by three workspace files, injected into every system prompt.

BOOT.md

Behavioral instructions for your GHOST. Controls how it thinks, responds, and acts.

# GHOST Boot Instructions

- Be direct and honest
- Ask clarifying questions before acting
- Research before making claims
- Use knowledge base proactively

This is the most impactful file — it shapes your GHOST’s core behavior.

SOUL.md

Self-observations written by the GHOST itself — what it has noticed about its own behavior, tendencies, and personality over time.

# Soul

- I am too sycophantic by default and need to fight this habit
- I tend to over-explain when a short answer would do
- I work best when I research before answering instead of guessing
- I genuinely enjoy helping with systems design problems

OPERATOR.md

Your preferences, goals, and communication style. Helps your GHOST understand you.

# Operator

- Prefers concise responses
- Works primarily in Rust and TypeScript
- Timezone: UTC+2
- Interests: distributed systems, AI agents

How Identity Works

All three files are read at startup and injected into the system prompt for every conversation. The GHOST carries this identity across all sessions and agents.

Edit these files directly in your workspace (~/GHOST/) — changes take effect on the next message. No restart required.

Providers

A provider is an LLM backend. GHOST supports multiple providers and lets you define named model aliases.

Available Providers

Model Aliases

Define aliases in config.toml to name your models:

[models]
default = "primary"

[models.primary]
provider = "openrouter"
model = "anthropic/claude-sonnet-4"
context_window = 200000

[models.fast]
provider = "kimi_code"
model = "kimi-k2.5"
context_window = 250000

• default — which alias to use when none is specified
• provider — one of: openrouter, kimi_code, openai_oauth
• model — the model ID as the provider expects it
• context_window — max tokens (used for compaction decisions)
• headers (optional) — extra HTTP headers for the provider

Multiple Models

Different features can use different models:

• Chat sessions use the default model
• Jobs specify their own model alias in frontmatter
• Agents can override the model in their definition

OpenAI OAuth Setup

# Authenticate with OpenAI (browser-based OAuth flow)
ghost auth codex

# Check status
ghost auth status

# Revoke tokens
ghost auth revoke

Interfaces

GHOST communicates through interfaces. Discord is the primary interface for the PoC.

Discord

Setup

1. Create a Discord bot at discord.com/developers
2. Enable Message Content Intent in bot settings
3. Add the bot to your server with message read/write permissions
4. Set environment variable: DISCORD_TOKEN=your-bot-token
5. Configure your user ID in config.toml
6. DM the bot

[discord]
allowed_user_id = "123456789012345678"

The allowed_user_id is a security gate — your GHOST only responds to messages from this Discord user.

How It Works

• Each Discord thread maps to a GHOST session
• Messages in a thread form a continuous conversation
• Your GHOST responds with full markdown formatting
• Tables are rendered as images for readability on mobile

Commands

• /REBOOT — Reset the current session (clear context, start fresh)

Future Interfaces

Additional interfaces are planned — see Roadmap:

• Slack
• Web UI
• TUI (terminal)

Chat & Sessions

Sessions

A session is a chat thread between you (the OPERATOR) and your GHOST. Each Discord thread maps to one session.

Currently, there’s only one session per GHOST and OPERATOR since there is a single interface.

Sessions persist in the database — your GHOST can query the full conversation history.

Message Flow

You send a message
  → System prompt assembled (identity + skills + agents + context)
  → GHOST processes with tool loop (up to 20 iterations)
  → GHOST calls tools as needed (search, web, files, etc.)
  → Final response sent back

The GHOST can chain multiple tool calls in a single turn — for example, searching the knowledge base, then fetching a web page, then composing a response.

Compaction

When the conversation approaches the model’s context window limit (default: 85% full), GHOST automatically compacts the history:

• Older messages are summarized into a compact form
• The most recent messages (default: 20) are kept in full
• Compaction is transparent — the GHOST continues the conversation naturally

Configure in config.toml:

[compaction]
threshold = 0.85 # Trigger at 85% context usage
keep_window = 20 # Keep last 20 messages intact
mask_preview_chars = 100

Session Management

# List recent sessions
ghost session list

# View messages in a session
ghost session logs <session_id>
ghost session logs <session_id> --count 100
ghost session logs <session_id> --around <message_id>

Knowledge Base

GHOST’s knowledge system stores and connects information using three types of entries, backed by SurrealDB with graph edges and embedding-based semantic search.

Knowledge Types

Notes

Atomic knowledge entries with structured metadata. Your GHOST creates these during reflection and conversation.

+++
title = "Rust Error Handling"
archetype = "concept"
tags = ["rust", "programming"]
sources = ["https://doc.rust-lang.org/book/ch09-00-error-handling.html"]
trust = 8
+++

Rust uses `Result<T, E>` for recoverable errors and `panic!` for unrecoverable ones.

Related: [[influences>Haskell Either Type]], [[used-by>GHOST Project]]

Archetypes: person, concept, decision, event, place, project, organization, procedure, media, quote, topic

References

Source material organized by topic in subdirectories:

references/
├── rust/
│   ├── error-handling-patterns.md
│   └── async-runtime-comparison.md
└── ai-agents/
    └── tool-use-survey.md

Diary

Daily timeline entries (auto-maintained by the reflection agent):

diary/
├── 2026-02-23.md
└── 2026-02-24.md

Wiki Links

Notes connect to each other through wiki links with optional typed relationships:

[[Target Note]] # Simple

[[relationship>Target Note]] # Typed relationship

These links create edges in the graph database, enabling traversal and discovery. Unresolved links automatically create stub notes that your GHOST can flesh out later.

Search

GHOST uses hybrid search combining:

• BM25 — full-text keyword matching
• Semantic — embedding-based similarity (via Ollama)

Results are merged and ranked. Search covers notes, references, and diary entries.

CLI Commands

# Search knowledge base
ghost knowledge search "error handling"
ghost knowledge search "rust patterns" --kind note --limit 20

# Read a specific note
ghost knowledge get --title "Rust Error Handling"

# Graph exploration
ghost knowledge graph "Rust Error Handling"         # Show connections
ghost knowledge graph "Rust Error Handling" --direction out
ghost knowledge graph --orphans                      # Unconnected notes
ghost knowledge graph --stats                        # Edge counts

# Browse
ghost knowledge tags                                 # Tags with counts
ghost knowledge recent                               # Recent activity
ghost knowledge stats                                # Type counts
ghost knowledge references --topic rust              # Browse by topic

# Maintenance
ghost knowledge reindex                              # Sync files → database
ghost knowledge reindex --skip-embeddings            # Skip embedding generation

Skills

Skills are workflow files that teach your GHOST how to handle specific tasks. They follow the agentskills.io format.

Skill Format

Each skill lives in $WORKSPACE/skills/<name>/skill.md with YAML frontmatter:

---
name: my-skill
description: >
  One-line description that helps the GHOST decide when to use this skill.
  Must be specific enough to trigger on relevant queries.
---

# Skill Instructions

Step-by-step workflow for the GHOST to follow.

1. First, do this
2. Then check that
3. Finally, produce this output

The description field is critical — it appears in the system prompt and determines when the GHOST reads the full skill. A weak description means the skill never triggers.

Default Skills

How Discovery Works

Skills are scanned at startup. Their names and descriptions are listed in the system prompt. When a conversation matches a skill’s triggers, the GHOST reads the full skill file before responding.

Writing Your Own

1. Create ~/GHOST/skills/my-skill/skill.md
2. Write a strong description — this is what triggers the GHOST to use it
3. Include clear, step-by-step instructions in the body
4. Optionally add supporting files in scripts/, references/, or assets/ subdirectories

Your GHOST can also create skills for itself using the skill-creator skill.

Skill Directory Structure

skills/my-skill/
├── skill.md           # Main skill definition
├── scripts/           # Optional helper scripts
├── references/        # Optional reference material
└── assets/            # Optional static assets

Agents

Agents are autonomous background workers that handle complex, multi-step tasks. They run independently with their own tool sets, iteration limits, and system prompts.

Agent Format

Each agent is a markdown file with TOML frontmatter in $WORKSPACE/agents/:

+++
description = "What this agent does"
tools = ["web_search", "web_fetch", "read_file", "todo"]
max_iterations = 50
model = "primary"
skills = ["knowledge-navigator"]

[[progress]]
rule = "nudge"
tool = "web_fetch"
min_calls = 7
check_at_iteration = 15
message = "You must fetch at least 7 pages."
+++

# Agent System Prompt

Detailed instructions for the agent's behavior...

Key Fields

Default Agents

How Agents Work

1. GHOST decides an agent is needed (based on context, skills, or user request)
2. Agent spawns via the agent_control tool with a prompt
3. Agent runs autonomously with its restricted tool set
4. Findings are injected back into the parent chat
5. GHOST can check status, continue, or stop agents mid-run

Agent Control

The GHOST manages agents through the agent_control tool:

start    — Spawn a new agent with a prompt
continue — Send follow-up instructions to a running agent
status   — Check agent progress
stop     — Terminate an agent

Jobs

Jobs are cron-scheduled tasks defined as markdown files with TOML frontmatter.

Job Format

Jobs live in $WORKSPACE/jobs/:

+++
name = "Daily News Summary"
schedule = "0 8 * * *"
enabled = true
model = "primary"
tools = ["web_search", "web_fetch", "read_file"]
carry_last_output = true
pre_tools = [
  { name = "web_fetch", input = { url = "https://news.ycombinator.com" } }
]
+++

Summarize the top stories from Hacker News. Focus on AI and systems programming. Write a
brief digest with links.

Fields

Built-in Subsystems

Heartbeat

Proactive check-ins when you’ve been idle. These are dedicated subsystems (not regular jobs) with their own scheduling logic. Configurable in config.toml:

[timing]
heartbeat_idle_minutes = 4 # Minutes of silence before check-in
heartbeat_check_seconds = 60 # How often to check
heartbeat_continue_minutes = 30 # Max heartbeat duration

Reflection

Automatic knowledge extraction after conversations end. Runs when a session goes idle, creating notes and diary entries from the conversation.

[timing]
reflection_idle_minutes = 4 # Minutes idle before reflection triggers

CLI Commands

# List all jobs with next-run times
ghost job list

# Validate a job file
ghost job validate jobs/daily-news.md

# Run a job immediately
ghost job run "Daily News Summary"

# View job logs
ghost job logs
ghost job logs "Daily News Summary"

Web Research

GHOST can search the web and fetch page content for research tasks.

Web Search

Uses the Brave Search API for web queries.

Requires BRAVE_API_KEY environment variable.

ghost web search "rust error handling best practices"
ghost web search "latest AI news" -n 10

Web Fetch

Extracts readable content from web pages. Supports two modes:

• Readability — article extraction (Mozilla Readability algorithm)
• Raw — full HTML output

ghost web fetch https://example.com
ghost web fetch https://example.com --readability
ghost web fetch https://example.com --raw

Optionally uses Crawl4AI as a backend when standard web fetching doesn’t yield good results:

[web]
crawl4ai_url = "http://localhost:11235"

Caching

All web fetches are automatically cached to $WORKSPACE/.web-cache/. The reflection agent triages those artifacts when it runs, creating sources linked to the fetched data.

How GHOST Uses Web Tools

During conversation, your GHOST proactively:

1. Searches the web when current information is needed
2. Fetches full pages to read beyond search snippets
3. Caches results for later reference
4. Cites sources in responses

The deep-research agent takes this further — it performs iterative multi-source research, fetching multiple full pages before drawing conclusions.

Core Tools

GHOST follows a “skills over tools” philosophy — prefer workflow files and existing tools over adding new tool APIs. These core tools cover most needs.

run_shell_command

Execute shell commands on the host system.

read_file

Read file contents with optional pagination.

Returns contents with line numbers for precise editing.

write_file

Create or overwrite a file.

file_edit

Make targeted string replacements in a file.

todo

Session-scoped working memory for tracking multi-step tasks.

agent_control

Spawn and manage autonomous agents.

See Agents for details.

Knowledge Tools

Tools for interacting with GHOST’s knowledge base.

knowledge_search

Hybrid search combining BM25 full-text and semantic embeddings.

note_write

Create or update structured knowledge notes. Available only in reflection contexts.

Archetypes: person, concept, decision, event, place, project, organization, procedure, media, quote, topic

Web Tools

Tools for web research. See Web Research for usage details.

web_search

Search the web via the Brave Search API.

Requires BRAVE_API_KEY environment variable.

web_fetch

Fetch and extract readable content from a URL.

CLI Reference

ghost daemon

Start the GHOST daemon (Discord bot + job scheduler).

ghost daemon

ghost init

Initialize workspace and config files.

ghost init

Creates ~/.config/ghost/config.toml and the workspace directory with default files.

ghost config

Read and write configuration values.

ghost config get <key>          # Get a config value (with defaults)
ghost config set <key> <value>  # Set a config value

Examples:

ghost config get discord.allowed_user_id
ghost config set timing.heartbeat_idle_minutes 10
ghost config set web.search_max_results 10

ghost auth

Manage authentication for providers.

ghost auth codex     # OpenAI OAuth flow (opens browser)
ghost auth status    # Check authentication status
ghost auth revoke    # Revoke OpenAI tokens

ghost job

Manage cron jobs.

ghost job list                   # List jobs with next-run times
ghost job validate <path>        # Validate job file syntax
ghost job run <name>             # Execute job immediately
ghost job logs [name]            # View job execution logs

ghost session

Inspect chat sessions.

ghost session list                              # Recent sessions (up to 50)
ghost session logs <session_id>                 # View messages
ghost session logs <session_id> --count 100     # More messages
ghost session logs <session_id> --around <id>   # Center on a message

ghost knowledge

Query and manage the knowledge base.

Search

ghost knowledge search <query>
ghost knowledge search <query> --kind note      # Filter by type
ghost knowledge search <query> --limit 20       # More results

Read

ghost knowledge get --title "Note Title"        # Get by title
ghost knowledge get <path>                      # Get by path

Graph

ghost knowledge graph <target>                  # Show connections
ghost knowledge graph <target> --direction out   # Outgoing only
ghost knowledge graph --orphans                  # Unconnected notes
ghost knowledge graph --stats                    # Edge/stub counts

Browse

ghost knowledge tags                             # Tags with counts
ghost knowledge recent                           # Recent activity
ghost knowledge recent --limit 50
ghost knowledge stats                            # Type counts
ghost knowledge references                       # All reference topics
ghost knowledge references --topic rust          # By topic

Maintenance

ghost knowledge reindex                          # Sync files → database
ghost knowledge reindex --skip-embeddings        # Skip embedding gen

ghost web

Web search and fetch.

ghost web search <query>                         # Search with Brave
ghost web search <query> -n 10                   # More results
ghost web fetch <url>                            # Extract content
ghost web fetch <url> --readability              # Article mode
ghost web fetch <url> --raw                      # Raw HTML

ghost version

Print the GHOST version.

ghost version

Roadmap

Here’s the current state of the backlog as of 2026-02-24:

1. Add a bulk import method to pull full documentation websites or codebases and query them through embeddings
2. Add a “Project” skill and workflow to let the assistant manage long horizon tasks, inspired by Mitsuhiko’s TODO
3. Add a coding agent inspired by pi-mono
4. Use NixOS to manage the GHOST’s shell
5. Add more interfaces and providers
6. See if we need a dedicated browser tool like OpenClaw, using PlayWright to control a headless browser
7. Security pass, it’s full YOLO so far
8. Good onboarding flow + nice TUI for humans
9. Write down and test a few more e2e flows
10. Create a GHOST networking feature inspired by Moltbook, but useful instead of memey
11. Implement extensibility: allow the GHOST to create complex automated agents and workflows. Might be Typescript, might be Lua, might be Python, might just be markdown with frontmatter. We will see.

If I get all of this done we should be in a good spot.