Ultimate Harness — Roadmap
Last updated: 2026-05-29. Source of truth for issue state is Linear; human-readable epic index for Ultimate Harness.
Last updated: 2026-05-29. Source of truth for issue state is Linear (the canonical UH team); GitHub mirrors via the native 2-way integration. This file is a human-readable index.
Now
v0.9.0 (current release, "Memory & adapter matrix" — Linear UH-131 / UH-136 / UH-137; GitHub PRs #204–#206 / #214 / #215 / #216): bundles everything merged to dev since v0.8.0. Ships a native Anthropic adapter (anthropic, status: experimental, #214) — the official, ToS-clean pay-per-token Messages-API path (ANTHROPIC_API_KEY env-only, blocked classification, graceful no-key checker, optional live-smoke CI job); Honcho memory operations (#215) — honcho_search / honcho_remember harness-side ops + per-mission runtime_config.honcho_memory opt-out on every Honcho-aware adapter; team-run dogfood fixes (#216) — UH-127 passed_partial verdict (no more false BLOCKED), UH-128 per-worker artifact-bleed fix, UH-129 integration_report_path default under .harness/missions/<id>/team/, UH-130 constraints-advisory warning; and Phase-0 DX hardening (#204–#206) — real uh --version from package.json, an opt-in PostHog telemetry primitive (unwired; UH-135 follow-up), adoption docs, a curated npm files: allowlist, and CI plugin gates. Will be published as @agenticengineeringagency/ultimate-harness@0.9.0 on npm latest (tagged v0.9.0 + plugin-v0.9.0 on main via the dev→main release PR). See CHANGELOG.md [0.9.0].
Epics 2–5 plus Epics 6–8 (live SSE tail + cancel, adapter capability routing + cost, SDD hardening) shipped and were released as v0.6.0 — published to npm as @agenticengineeringagency/ultimate-harness@0.6.0. v0.6.0 also corrected two v0.5.0 gaps: the auto-router uh mission run --auto (UH-101, claimed shipped but never built) and runtime.usage token capture (which the cost features assumed). The Epics 6–8 execution spec is Epics 6–8 execution spec; per-slice detail is in the CHANGELOG [0.6.0]. v0.7.0: OpenRouter adapter ([#134]), vanilla pi adapter ([#135], shipped active), sandbox backend abstraction + directory backend ([#136]) and a container stub ([#137]), verify-then-promote auto-trigger ([#139]), capability-match enforcement ([#138], already shipped), and a shared adapter-artifact-helper refactor ([#133]). v0.8.0: OpenSandbox-gated container execution backend ([#155]/[#157]) with ADR + runbook ([#154]), and oh-my-pi graduated to active ([#156]); published as @agenticengineeringagency/ultimate-harness@0.8.0. v0.9.0 (current release, see Now above): native anthropic adapter (experimental, [#214]), Honcho memory operations + opt-out ([#215]), team-run dogfood fixes ([#216]), and Phase-0 DX hardening ([#204]–[#206]). Issue tracking: Linear is the canonical source of truth for the UH team; GitHub issues mirror Linear via the native 2-way integration (per-issue auto-creation + state/title/body sync). Cite Linear IDs (UH-NNN) in commits, PRs, and docs — the GitHub mirror picks up the back-link automatically. The v0.5.0/v0.6.0 epic records (UH-92..UH-111) and the v0.7.0/v0.8.0 milestone slices (UH-112..UH-125) were backfilled to Linear post-workspace-upgrade so the history matches the CHANGELOG.
Epic 5 — Hermes Dashboard plugin v2: multi-run history + replay + observability (UH-84)
Surface the per-run artifact backend (UH-82, shipped in v0.4.0-prep) in the dashboard UI. Today's plugin shows one mission ↔ one "latest result"; v2 turns the dashboard into a triage surface for missions with dozens of runs.
| Phase | Issue | Slice | Size | Status |
|---|---|---|---|---|
| Core | UH-85 | Recent runs pane on Mission detail | S | shipped — #98 |
| Core | UH-86 | Per-run drilldown (?run_id= param) | M | shipped — #98 |
| Core | UH-88 | Filter + search runs in the pane | S | shipped — #98 |
| Later | UH-89 | Compare two runs side-by-side | M | shipped — #101 |
| Later | UH-87 | Replay a run with same/modified overrides | M | shipped — #101 |
| Optional | UH-90 | Run history retention policy | S | shipped — #100 |
Epic 5 shipped end-to-end across #98 + #100 + #101. Backend foundation shipped earlier in PR #96 (UH-82): runs/<run_id>/ per-run dirs, runs/index.json history, latest.json pointer, per-run artifact route honoring run_id, get_mission returns runs[]. TUI polish slices (UH-48 / UH-49 / UH-50 / UH-51) shipped in #99. Cut as v0.4.0 (#103 + #105 + #107); superseded by v0.5.0 and v0.6.0, all live on npm.
Epic 3 — Hermes Dashboard plugin for UH (UH-60)
Ship a Hermes dashboard plugin that exposes UH end-to-end from the Hermes web UI — adapter health, mission list, mission run trigger, artifact drilldown, and theme integration. The plugin is a drop-in at ~/.hermes/plugins/uh/dashboard/: manifest.json + IIFE JS bundle (using window.__HERMES_PLUGIN_SDK__) + Python FastAPI router that shells out to the uh CLI. No daemon, no FFI, no Hermes fork.
| Phase | Issue | Slice | Size | Status |
|---|---|---|---|---|
| Core | UH-61 | Plugin scaffold + dev workflow | S | shipped — #89 |
| Core | UH-62 | Python ⇄ uh CLI FastAPI backend bridge | M | shipped — #89 |
| Core | UH-65 | UH overview tab (adapters + missions + recent runs) | M | shipped — #89 |
| Core | UH-64 | Mission run trigger + live event-stream tail | M | shipped — #89 |
| Later | UH-63 | Mission drilldown viewers (prompt / final-message / diff / runtime-result) | M | shipped — #89 |
| Later | UH-66 | Workflow profile viewer + verification report viewer | M | shipped — #89 |
| Later | UH-69 | UH theme YAML + sessions:bottom slot cross-link | S | shipped — #89 |
| Optional | UH-67 | New-mission wizard + workflow editor | M | shipped — #89 |
| Optional | UH-68 | Plugin packaging, install runbook, and distribution repo | S | shipped — #89 |
All Core + Later + Optional slices shipped in #89 after 13 rounds of Codex adversarial review — 26+ documented correctness findings (race conditions, silent failure modes, security gaps, schema drift, response shape inconsistencies) all addressed before merge. See the Linear issues for per-slice detail; follow-ups UH-81 / UH-82 / UH-83 track the deferred items.
Epic 4 — Team mission shape (UH-70)
Distill the convergent learning from oh-my-claudecode and oh-my-codex into a UH-native mission shape: a single mission that fans out across multiple adapters in parallel as workers, each in its own worktree, with a leader/integrator role that merges their work and runs the existing verification pipeline over the integrated result. Extends UH-56 mission run-all from comparison to collaboration.
| Phase | Issue | Slice | Size | Status |
|---|---|---|---|---|
| Core | UH-71 | Mission schema for team shape with adapter-bound workers | S | shipped — #88 |
| Core | UH-72 | Per-worker worktree auto-isolation + leader merge / integration-report.md | M | shipped — #87 + #90 |
| Core | UH-73 | Staged workflow profile family (plan → prd → exec → verify → fix) | M | shipped — #87 |
Standalone discipline-layer slices inspired by the same research (not part of Epic 4 but sequenced after it):
| Phase | Issue | Slice | Size | Status |
|---|---|---|---|---|
| Later | UH-74 | Adversarial QA workflow profile (adversarial-qa) — OMX $ultraqa analog | M | shipped — #87 + #90 |
| Later | UH-75 | Companion design.md artifact alongside mission.yaml — OMX $design analog | S | shipped — #88 |
Research source: docs/research/inspiration-systems.md §"oh-my-claudecode (OMC)", §"oh-my-codex (OMX)", §"Combined OMC/OMX takeaways"; verdicts in docs/research/adopt-reject-defer.md; side-by-side row in docs/research/comparison-matrix.md.
Discipline-layer slices from GSD-2 research
Standalone slices distilled from the GSD-2 / gsd-pi deep-dive (see docs/research/inspiration-systems.md §"GSD 2 (gsd-pi)"). They are orthogonal to Epic 3 and Epic 4 — each lands independently.
| Phase | Issue | Slice | Size | Status |
|---|---|---|---|---|
| Core | UH-76 | Three-verdict runtime-result status (pass / needs-attention / needs-remediation) + uh mission verdict manual override — GSD-2 /gsd verdict analog | S | shipped — #88 |
| Core | UH-77 | Drift detection + idempotent repair registry for uh validate — GSD-2 ADR-017 analog | M | shipped — #88 |
| Core | UH-78 | uh status --json LLM-less query mode — GSD-2 gsd headless query analog; feeds Hermes Dashboard (UH-62) | S | shipped — #88 |
| Later | UH-79 | Canonical docs/VISION.md with explicit "what we won't accept" — GSD-2 VISION.md analog | S | shipped — #88 |
| Later | UH-80 | Pre-inlined dispatch context contract — formalize the existing buildMissionPrompt shape across all four adapters | M | shipped — #88 |
Research source: docs/research/inspiration-systems.md §"GSD 2 (gsd-pi)"; verdicts in docs/research/adopt-reject-defer.md; side-by-side row in docs/research/comparison-matrix.md.
Epic 2 — Interactive TUI for UH (UH-41)
Build uh tui on OpenTUI — the same engine OpenCode uses in production (native Zig core, TypeScript bindings, Bun-first, MIT-licensed). Replaces today's cat-driven mission review with a navigable live terminal app using OpenTUI's Diff, Code (tree-sitter), ScrollBox, Select, and Input components.
| Step | Issue | Slice | Size | Status |
|---|---|---|---|---|
| 1 | UH-45 | Spike: install opentui, framework choice (vanilla / React / Solid), hello-world bound to UH state | S | shipped — Solid won; see docs/research/tui-framework.md |
| 2 | UH-46 | Dashboard: live adapters + missions + sandboxes (three-pane) | M | shipped — Mission Control dashboard; see #57 + docs/architecture/tui.md |
| 3 | UH-47 | Mission browser: drilldown with Code + Diff viewers | M | shipped — read-only mission artifact drilldown; see #59 |
| 4 | UH-44 | Mission run flow: trigger from TUI, stream events live | M | shipped — R opens a run dialog, live tail of events.ndjson, S stops via SIGTERM; see #63 |
| 5 | UH-43 | Adapter + sandbox manager: live checks, create/discard from inside | M | shipped — c re-check, n create-sandbox dialog, d discard with F force toggle; see #64 |
| 6 | UH-42 | Polish: keymap overlay, theming, error states, exit handling, Agent Skill install | S | shipped — ? overlay, per-project state persistence, operator runbook; see #61 + #65. Theming / $EDITOR / Ctrl+Z spun out as UH-48 / UH-49 / UH-50. |
Shipped this cycle (2026-05-17 → 2026-05-19)
Epic 1 — Hermes proxy adapter (UH-32) — DONE
Adapter promoted to status: active on 2026-05-18 after live E2E smoke against hermes proxy start --provider nous. ToS-positioned subscription routing now first-class alongside hermes / codex / oh-my-pi.
| Step | Issue | Slice | PR |
|---|---|---|---|
| 1 | UH-36 | Wire-format spike + recommendation | #49 |
| 2 | UH-35 | Schema + manifest + template + dispatch stub | #50 |
| 3 | UH-39 | Adapter impl: planner, runner, SSE parser, sentinel, blocked classification | #51 |
| 4 | UH-37 | Live HTTP adapter check probe | #53 |
| 5 | UH-40 | Architecture doc + setup runbook | #54 |
| 6 | UH-38 | E2E smoke + promote to status: active | #55 |
Other slices this cycle
| Issue | Title | PR |
|---|---|---|
| UH-45 | TUI spike: opentui framework selection + hello-world | #52 |
| UH-46 | TUI dashboard: live adapters + missions + sandboxes | #57 |
| UH-47 | TUI mission browser: drilldown with Code + Diff viewers | #59 |
| UH-44 | TUI mission run flow with live events.ndjson tail | #63 |
| UH-43 | TUI adapter + sandbox manager (create/discard/recheck) | #64 |
| UH-42 | TUI polish: keymap overlay (?) + per-project persistence + runbook | #61 + #65 |
Runtime intelligence + operator polish + npm publish automation + Honcho memory (2026-05-19)
| Issue | Slice | PR |
|---|---|---|
| UH-57 | Runtime intelligence + operator polish: TUI screenshot capture, adapter-check age footer, runtime.cancelled event, mission capability enforcement | #77 |
| UH-58 | Package renamed @agenticengineeringagency/ultimate-harness, publishConfig.access=public, CI dry-run + release publish workflow | #79 |
| UH-59 | Honcho persistent-memory extension for the oh-my-pi adapter — env-driven enrich/record at the harness layer with fail-fast misconfig and graceful network degradation | #83 |
Discipline layers — Spec-Driven, Test-Driven, Cross-runtime QA (shipped)
| Issue | Slice | PR |
|---|---|---|
| UH-54 | SDD: structured acceptance_criteria with per-AC verify | #68 |
| UH-55 | TDD: tdd workflow profile + test-first verify gate | #69 |
| UH-56 | Cross-runtime QA: uh mission run-all + side-by-side comparison | #70 |
All three layers compose: see docs/architecture/sdd-tdd-qa.md.
Shipped previous cycle (2026-05-13 → 2026-05-17)
| Issue | Title |
|---|---|
| UH-23 | Codex adapter — CLI transport via codex exec |
| UH-24 | Cancelled (Anthropic ToS friction; superseded by UH-27 + UH-32) |
| UH-25 | oh-my-pi adapter + runtime_config bucket + dispatch table + sandbox routing parity |
| UH-26 | Per-runtime strict runtime_config validation (typo safety) |
| UH-27 | Per-mission runtime_config_overrides + Anthropic-via-OMP path |
| UH-28 | Runtime-final-message capture protocol (shared sentinel) |
| UH-29 | createSandbox seeds bound mission into the worktree |
| UH-30 | Codex CLI flag drift + child-stdin close + codex status: active |
| UH-31 | Minimum Hermes ≥ 0.14.0 version pin |
| UH-33 | runtime_config_overrides parity for hermes + codex |
| UH-34 | Diff capture includes untracked new files |
Adapter status as of 2026-05-29 (v0.9.0):
| Adapter | Status |
|---|---|
| hermes | active (pinned ≥ 0.14.0) |
| codex | active (verified against codex-cli 0.130.0 in UH-30 smoke) |
| hermes-proxy | active (verified against hermes proxy start --provider nous in UH-38 smoke; manifest default nousresearch/hermes-4-405b) |
| openrouter | active (OpenAI-compat HTTP to openrouter.ai; OPENROUTER_API_KEY; live-smoke verified) |
| pi | active (vanilla pi agent CLI pi --print --mode json --no-session; shipped v0.7.0 #135/#150; live-smoke verified) |
| oh-my-pi | active (graduated v0.8.0 #156; ToS posture per docs/runbooks/anthropic-via-omp.md §Auth & ToS posture; users opt in) |
| anthropic | experimental (native pay-per-token Anthropic Messages API; ANTHROPIC_API_KEY env-only; shipped v0.9.0 #214; graduation pending live-smoke) |
Medium-term proposals (not filed)
These are tracked in narrative form until they earn the priority to be filed:
Road to v1.0 is planned in docs/prds/uh-roadmap-to-1.0-v1.0-prd.md (milestones v0.8.0 → v1.0.0). Remaining proposals:
- Native ANTHROPIC_API_KEY adapter — shipped in v0.9.0 ([#214], UH-136) as
anthropic(status: experimental): the ToS-clean pay-per-token Messages-API path that complements UH-32.ANTHROPIC_API_KEYenv-only, blocked classification, graceful no-key checker, optional live-smoke CI job. Graduation toactivepending a live-smoke promotion record. - Container sandbox backend — shipped in v0.8.0 ([#155]/[#157]) behind the OpenSandbox seam, with #154 spike + ADR + runbook. Live docker-CLI fallback rejected as default by the ADR pivot rule.
- Honcho-memory follow-ups —
codex/hermeswired into enrichment +recordMissionExchange(v0.6.0); shipped in v0.9.0 ([#215], UH-137):honcho_search/honcho_rememberharness-side memory operations + per-missionruntime_config.honcho_memoryopt-out on every Honcho-aware adapter. - Telemetry instrumentation — an opt-in PostHog telemetry primitive shipped unwired in v0.9.0 ([#204]–[#206], UH-131); wiring the actual event emission is the UH-135 follow-up (planned v0.10.0). UH still ships no telemetry by default.
- Capability-declaration enforcement — make manifest/mission
capabilities:binding (warn +--strict); planned v0.10.0. (Distinct from the already-shippedruntime_requirementspreflight.)
Shipped (no longer proposals): the native anthropic pay-per-token adapter (v0.9.0 #214, experimental); Honcho memory operations + per-mission opt-out (v0.9.0 #215); cross-runtime QA uh mission run-all --runtimes with side-by-side diff/sentinel comparison; the Epic 2 TUI polish UH-48..53 (theme, $EDITOR, Ctrl+Z, tui screenshot, footer adapter-check age, runtime.cancelled on SIGTERM).
Strategic (decisions needed)
- Muta integration — UH-as-consumer, UH-as-component, or independent? Needs a co-founder conversation. Deferred entirely to post-1.0 (see the road-to-1.0 PRD).
Resolved since: the Pi adapter shipped active in v0.7.0 (#135/#150); oh-my-pi graduated to active in v0.8.0 (#156); the native anthropic adapter shipped experimental in v0.9.0 (#214); Honcho memory operations + opt-out shipped in v0.9.0 (#215); mission capability-declaration enforcement is planned for v0.10.0 (manifest capabilities: binding, warn + --strict) and runtime_requirements preflight already enforces compatibility today.
References
- Documentation home
- Architecture overview
- Runtime adapter contract (includes UH-28 sentinel protocol)
- Codex E2E smoke runbook
- Anthropic-via-OMP runbook (notes the ToS posture that UH-32's hermes-proxy supersedes)
- Honcho memory runbook (UH-59)
- Honcho memory design (UH-59)
- Hermes dashboard extension docs (Epic 3 reference)