Skip to content

Tool of the Day|Agent Cockpit — makes every research claim traceable back to the code that produced it

Nov 2, 20261 min
TL;DRAgent Cockpit is an AGPL-3.0 local whiteboard and MCP server: any MCP-capable agent (Claude Code, Codex, Cursor, Gemini CLI) draws its research work onto the same board. Install: `git clone https://github.com/chutingyean/Agent-Cockpit.git && cd Agent-Cockpit && npm install && npm run demo`. It solves the problem of an agent quietly editing a helper function and nobody noticing that a previously 'verified' claim no longer holds — by hashing every experiment run, every file it depends on, and every function it actually reaches. When a hash changes, the claim is flagged. No LLM calls, zero tokens.

Look up & save

Loading dictionary…

🌏 中文版

Tool info

ItemValue
NameAgent Cockpit
TypeLocal whiteboard + MCP server (any MCP-capable agent: Claude Code, Codex, Cursor, Gemini CLI)
GitHubchutingyean/Agent-Cockpit
Stars132 (created 2026-10-26, climbed up in just over a week)
LanguageJavaScript (Node.js; the web board is framework-free, no build step)
LicenseAGPL-3.0-or-later
Installgit clone https://github.com/chutingyean/Agent-Cockpit.git && cd Agent-Cockpit && npm install && npm run demo

What problem it solves

Let an AI agent run research for a few days and the same thing keeps happening: it edits a grader, a data pipeline, or a shared statistics helper, and a claim that was "verified" quietly stops holding — nobody notices. The change itself goes unreported, or the agent reports only part of what it touched. Decisions meant for a human pile up unanswered, untested assumptions harden into foundations, and the agent keeps tweaking a hyperparameter "one more time." The codebase grows faster than anyone can actually read it.

Agent Cockpit turns these failure modes into rules a machine can check, all shown on one board. The core idea is an evidence chain: a claim has to cite a value that some recorded run actually wrote — it can't just assert something. Every run records the hash of its own script, the hash of every file it depends on, and the hash of every function it actually reaches. If any of those hashes change, the run is marked stale, and every claim built on it gets flagged for a recheck. Revert the change and the hashes line up again — the claim recovers automatically, no human has to track it by hand. Every write — from MCP, the CLI, or the board itself — passes through the same validation gate; a non-compliant write is rejected with a reason, so the agent can't claim it didn't know the rule, and rejections are logged too. All of this is hash comparisons and fixed rules — no LLM call, zero token cost.

How a claim goes from valid to stale
How a claim goes from valid to staleA claim must cite a value a run actually wrote; afterward, whether reported or not, if a file hash involved changes, the claim is flagged stale pending recheck. cockpit_run runs experiment hashes script/files/functions cockpit_claim registers claim evidence cites this run's output a file changes later whether reported or not hash still match? script/deps/functions claim stays valid claim flagged stale revert it and hashes match again yes no
  1. cockpit_run executes an experiment, hashing the script, its dependency files, and the functions it reaches
  2. the agent registers a claim with cockpit_claim, citing a field this run actually wrote
  3. a file changes later, whether reported or not
  4. compare the current hash against the one on record
  5. hashes match — the claim stays valid
  6. hashes don't match — the claim is flagged stale, queued for recheck

Simplified from Agent Cockpit's README, "content hashes, not timestamps." Reverting the change makes the hashes match again and the claim recovers automatically — that path is left out here for simplicity.

Good fit: a research project where several agents hand off work in turn (say, Claude Code lays the groundwork and Codex picks it up); a setting where certain decisions genuinely need to stop and wait for a human instead of letting the agent plow ahead on its own; and anywhere you want to be able to replay the whole event sequence after something breaks, instead of guessing from memory where it went wrong.

Quick start

Install

git clone https://github.com/chutingyean/Agent-Cockpit.git
cd Agent-Cockpit
npm install
npm run demo                                       # builds a demo research repo: 6 real runs, 6 claims
node bin/cockpit.mjs serve --repo demo/research    # open http://127.0.0.1:4180

Basic usage

# switch to your own research repo (must already be a git repo)
cd /path/to/your-research
node /path/to/Agent-Cockpit/bin/cockpit.mjs init     # writes MCP configs, AGENTS.md, Stop hook, pre-commit
node /path/to/Agent-Cockpit/bin/cockpit.mjs serve    # opens the board
node /path/to/Agent-Cockpit/bin/cockpit.mjs run -- python3 train.py --lr 1e-3   # records this experiment

init only writes inside your own repo. Any agent that opens the repo picks up AGENTS.md and the MCP config, and gets tools including:

cockpit_overview   # call this first: pending decisions, claims needing recheck, unreported changes
cockpit_claim      # register a claim; evidence must be a file and field a recorded run actually wrote
cockpit_run        # run an experiment and record it
cockpit_decide     # ask a human, and wait for the answer before continuing
cockpit_report     # report which files changed

Advanced usage

node bin/cockpit.mjs demo --break      # quietly loosens the grader's matching rule, unreported
node bin/cockpit.mjs demo --restore    # put it back: hashes line up again, the claim recovers

This pair demonstrates the whole point: an unreported change still gets caught, because what's being checked is a hash, not a promise that the agent will mention everything it touched.

Compared to existing approaches

Agent CockpitManually tracking git log/diffJust asking the agent "what did you change"
Detects unreported file changes✅ content-hash comparison catches it even when unreportedSomeone has to remember to diff it — easy to miss❌ if the agent says it didn't, that's taken at face value
Stale claims auto-flag and propagate downstream✅Have to manually track which claim depends on which file❌
Needs an LLM call to check❌ zero tokens, pure rule evaluation—The check itself is an LLM call — not reliable
Handing off between agents on the same project✅ identity comes from the MCP handshake, board is sharedRelies on verbal handoff or docs, easy to dropEasy to drop
Can replay the whole event sequence afterward✅ append-only event log, scrubbableGuess the order from git log❌

Caveats

  • AGPL-3.0-or-later: if you distribute a modified version to others over a network (say, running a shared board as a service for your team), the license requires you to offer the source for your version. Running it locally for yourself or your own team usually doesn't trigger that, but check your actual use case before relying on it in a commercial product.
  • The project is young (created 2026-10-26, version 0.1.0). The README, tutorial, and demo are thorough, but there's no track record yet at production scale. Run the demo and get a feel for how it decides "stale" before trusting it with a claim that actually matters.
  • The built-in code graph and code review are lightweight fallbacks for when nothing else is installed; install the real graphify or open-code-review and Agent Cockpit switches to them automatically — behavior may differ slightly.
  • The board binds to 127.0.0.1 by default; it's designed as a local tool, not something meant to be exposed as a service. Sharing it across people means figuring that out yourself (same machine, or your own reverse proxy — which brings back the AGPL network clause above).

Takeaway

Until now, the question of whether a research agent quietly changed something has mostly relied on manual review or on trusting the agent's own report. Agent Cockpit turns that trust problem into a plain engineering one: content hashing makes "does this claim still hold" a question a machine can answer directly, without anyone's honesty required. It's basically the security principle "don't trust input, verify it" — except this time the input being verified is the agent's own account of its work.

References