Briefs refinement loop + first-class daemon-agent roster (per-project agents, heartbeat, operator inbox)

[orchestrator 969, 2026-07-01] Extracted from operator brief #25 (item 1) + the flower-design "expected daemon" deferral flagged during spawn-flow-ui: **daemons aren't checking in reliably** — even flower-orchestrator/ops read stale on /roster because check-in depends on the agent REMEMBERING to call daemon_checkin each loop. Design direction for a reliable/foolproof heartbeat (Mike's ask: "how do we make it foolproof?"): 1. **Don't rely on the agent's memory.** The recurring loop TIMER that drives a daemon should carry the check-in as its FIRST action every iteration (bake into the charter loop), so a looping daemon always heartbeats. 2. **Infra backstop (partly built, 28ddf12):** roster liveness already falls back to the linked SESSION's activity, so an alive-but-not-checking-in daemon still reads correctly. Extend so context_size/needs_compaction degrade gracefully when only session-link data exists. 3. **Expected-daemon seam (the flower-design deferral):** daemon_agents keys on (role, project_id, solo_process_id); there's no "expected" state nor a check-in that reconciles a placeholder → real row. Add an `expected` status + a check-in that upgrades it, so the roster can show intended-but-not-yet-live daemons without phantom rows. **This unblocks the /roster spawn+monitor UI (Brief #22).** 4. Consider a lightweight periodic auto-checkin (a repeating Solo timer the daemon arms once) so cadence doesn't depend on the daemon's own loop firing. This is the FOUNDATION for Brief #22 (roster monitoring needs reliable check-in data) and for turning the TDA daemons loose safely.

[orchestrator 969, 2026-07-01] Design direction folded in from the recall_active harness-mismatch fix (Brief #21 / feedback #31 / fix-spec scratchpad 1032). **Harness↔process attribution must be SURE, and captured EARLY.** The #21 bug happened because we tried to reconstruct which Solo process owns a session AFTER the owning process had already closed — falling back to cwd/recency heuristics that are not harness-aware, so a live codex session got linked to the claude orchestrator sharing the cwd. Direction for Brief #11's roster / heartbeat / death-detection substrate: - **Correlate `session external_id` ↔ `solo_process_id` ↔ `tool_type/harness` EARLY** — while the process is alive and Solo can still hand us the authoritative `tool_type` — and **PERSIST** that mapping. Do not reconstruct it later from cwd heuristics after the process is gone (the data we need is gone with it). - Prefer Solo's authoritative `tool_type` over string-scraping the process command. - recall_active's harness veto (Brief #21) is the correct DEFENSIVE layer, but the durable early-correlation is the real fix: a recorded truth we look up, not a guess made after the fact. - `daemon_checkin` already records solo_process_id + session_id + role at first sighting; extend the same "record it live" idea to non-daemon worker sessions so every live session's owning process + harness is captured when first observed, and death-detection reads recorded ownership instead of inferring it.

## A1 DONE (design, proc 973) — branch `flower/refine-ux` @ 825f6cb, ready for orchestrator merge **Shipped (pure UI + status transitions + one persist-only column):** - **/briefs capture box:** **Capture** (→ `idea`) + **Capture & Refine** (→ `refining`, enters the loop queue). Shared `capture()` path; the refine variant transitions via `BriefService::setStatus` so it lands a `status_change` event. - **Brief detail — split/multi Refine button:** *Quick refine* (the existing in-process `refine()`/RefineBrief path) vs *Send to refinement loop* (`sendToRefinementLoop()` → sets `refining` + flags `meta.queued_for_loop`; rendered as a clearly-labeled **soon** stub since the loop agent doesn't exist yet). A "queued for loop" header pill surfaces the flag. - **Per-brief `auto_dispatch_on_planned`:** sqlite-portable migration (`briefs.auto_dispatch_on_planned` bool default false) + bool cast + a toggle in the Dispatch card. Persist-only — no behavior wired. **Notes for merge:** - Livewire-reserved collisions avoided (`captureAndRefine` / `sendToRefinementLoop` / `toggleAutoDispatch` — no `dispatch()`/`mount`/`updated*` clash). - Header lifted to `relative z-30`: the bloom entrance animation leaves each section with a persistent stacking context (identity-matrix transform via `animation-fill-mode: both`), which trapped the Refine dropdown behind the Spec card. Found + fixed via live DOM inspection (elementFromPoint). Also rebuilt the Vite CSS bundle so new utilities (`w-72`, `rotate-180`) compile — `public/build` is gitignored, so MAIN rebuilds on deploy; no build artifacts in the commit. - Migration already applied to the **shared MySQL** (`flower` DB) from the worktree — it's backward-compatible (new column, default false), so it won't affect MAIN; `migrate` on MAIN post-merge is a no-op. **Verification:** Briefs suite 33 green (+4 new tests), full suite 283 green, pint clean. Live on flower-design.test: both capture buttons, split dropdown rendering opaque-on-top, toggle ON/OFF with notice. Brief 11 left pristine (toggled auto-dispatch back to OFF; never clicked send-to-loop). Next foundation tasks per the decomposition are backend's (A2 operator inbox, A4 roster/heartbeat); design's follow-ons are A3/A5 UI once those land.

CHARTER SEED for the spawn-flow / editable-charter-prompts (this brief): the orchestrator daemon charter + an example spawn prompt live in Solo scratchpad #1024 ('Orchestrator charter / spawn-prompt seed'). It distills the project-orchestrator role, the core loop (drain inbox → dispatch → poll+merge → roster heartbeat), the hard-won operating principles (MAIN=stable-served / worktree isolation, merge protocol, verify-on-real-data, watchdog+polling gotchas, Codex approval/sandbox quirks, root-cause-with-data, context/auto-compaction, checkpoint-only-on-genuine-decisions), and a ready-to-fill <SLOT> spawn prompt. When B3 (spawn flow + prompt library) is built, seed the ORCHESTRATOR template from 1024. (ops is writing its own equivalent charter for the ops template; the refine template will want the same treatment.)

## Build decomposition (briefs 11 + 12) — orchestrator, 2026-06-30 ### FOUNDATION — buildable NOW (no loop required) - **A1 [design · FIRST]** Refine UX affordances. Capture box: **Capture** (→ status `idea`/draft) and **Capture & Refine** (→ `refining`). Brief detail: a **split/multi refine button** — *Quick refine* (the existing in-process BriefRefiner) vs *Send to refinement loop* (sets `refining` + flags "queued for loop"; a stub/disabled affordance until the loop agent exists). Per-brief **"auto-dispatch when planned" toggle** (writes a brief flag; default off). Pure UI + status transitions — ready now. - **A2 [backend]** Operator inbox (brief 12) backend: `operator_notes` table (id, project_id?, body, status open|acknowledged|handled|dismissed, created_by, handled_by?, meta, ts) + MCP: `recall_inbox(project?)`/`inbox_note(id)` (read), `note_create`/`note_update_status`/`note_reassign`/`note_unassign` (write, actor_ref). Ready now. - **A3 [design]** Operator inbox UI: "note the orchestrator" composer (global) + per-project inbox view + reassign/unassign + status pills. - **A4 [backend]** Daemon roster + heartbeat: `daemon_agents` table (role, project_id, solo_process_id, session_id, context_size, usage, status, last_checkin) + a heartbeat MCP tool (time-gated ≥15min) + death-detection. Extends recall_active. - **A5 [design]** Roster status UI: live agents per project, context/usage, alive/stale/dead. ### LOOP + CONTROL PLANE — after the foundation - **B1 [operational+backend]** Brief-refinement LOOP agent: a charter (mirror flower-ops) + loop logic (poll THIS project's `refining` briefs → refine as a real agent with file+recall+brief_* access → `brief_ask` or `brief_update_spec` → advance; stops on `planned`) + adaptive activity-tiered cadence. Per-project. Dogfood it. - **B2 [frontend+backend]** Live conversational view: Reverb broadcast on brief_events/brief_questions → live brief-detail updates (chat-like async). - **B3 [control-plane UI]** Spawn flow + editable charter prompts: a flower flow to spawn orchestrator/ops/refine agents into a Solo project with a pre-set, editable prompt; prompt library in the roster UI. - **B4 [operational+backend]** Auto-compaction pipeline: roster context-size threshold → compact instruction → daemon writes a handoff scratchpad → end session + respawn (reuses B3 spawn flow) resuming from the handoff. ### Owners / worktrees - **design** (UI: A1, A3, A5, B2-frontend) → `flower-design` worktree (flower-design.test). - **backend** (A2, A4, B1, B2-backend, B4) → the new backend worktree (to be created, retiring flower-ccparse). - **operational** (B1 loop, B4) → standing agents. - **design's FIRST task = A1.** **backend's FIRST = A2** (then A4).

## Spec: Briefs refinement loop + first-class daemon-agent roster (v2) **Core (Mike):** replace the in-process-only refine with a STANDING LOOP agent **per project** (like flower-ops) — full file + MCP + recall access — that refines/plans/answers briefs conversationally + continuously. The brief is the durable state (manila folder); rounds are stateless. flower is the **global control plane**; the daemons themselves are **per-project**. ### Refine UX - New-brief actions: **Capture** (→ idea/draft; add context first) and **Capture & Refine** (→ refining; enters the loop queue). - Brief detail: refine = a **split/multi-button** — **Quick refine** (synchronous in-process) vs **Send to refinement loop** (the standing agent). Extensible. - **Live conversational view via Reverb** (in stack): loop posts questions/notes → broadcast → operator replies → next cycle consumes. Feels like chat; mechanically the async manila-folder. - **Stops on `planned`** (loop or operator). No auto-dispatch by default; per-brief **"auto-dispatch when planned" toggle (default OFF)** → drops into the dispatch queue for the project's orchestrator. - async-Q&A (brief_ask → operator answers → next cycle) is the interaction mechanism — sufficient. ### Daemons are PER-PROJECT (Mike, note #5) - **Orchestrators are project-scoped too** — NOT global. The orchestrator does not dispatch briefs/tasks in another project unless explicitly told. Briefs in projects without an orchestrator are dispatched manually (operator) or by telling an agent to check+dispatch, or by spinning up an orchestrator loop in that project. - The three standing daemon roles, all **per target project**: **orchestrator** (dispatch/merge/coordinate/verify + drain the operator inbox), **ops** (feedback/Sentry), **brief-refinement/research**. Possible 4th: **pipeline-health**. - **flower = the global control plane**: the roster, the spawn flow, the prompt library, the status UI — these span projects; the workers they manage are project-scoped. ### Spawn flow + editable charter prompts (Mike, note #5) - A flower UI **flow to SPAWN an agent** (orchestrator | ops | refine) into a Solo project with a **pre-set, editable prompt** (set a timer, poll THIS project's briefs for qualifying dispatchables / refinables, etc.). Same flow for all three roles (ops scoped to its project; orchestrator/refine to a target project). - The roster UI has a place to **view/edit these spawn/charter prompts** (config-as-data, like prompt_templates). ### Daemon-agent roster + heartbeat (first-class) - Standing agents **check in on a time gate (≥ ~every 15 min, not every loop)** with role / project / solo_process_id / session_id / **context-size** / usage. flower stores it → **live-agents status UI** + **death detection** (missed heartbeats). Solves a real bug: ops burned ~5 cycles waking a DEAD orchestrator (958→969). Extends recall_active. - **Adaptive cadence**: recent brief activity → poll fast; agent activity but no brief activity → slower; fully dormant → slowest. (ops adopts the same.) ### Auto-compaction pipeline (Mike — the reason for context-size tracking) - The roster tracks context-size → when a daemon is high, trigger an **auto-compaction**: give the daemon a specific **compact instruction set** → it **writes a handoff scratchpad** (if mid-interruptible-work or waiting on something), then either (a) we trigger an in-session compaction, OR (b) we **end the session + spawn a replacement** instructed to resume from the handoff scratchpad. (Open: in-session compaction trigger vs end+respawn — the respawn path is cleaner + reuses the spawn flow.) ### Operator → orchestrator inbox - Spun out to its OWN brief (see the inbox brief). The orchestrator (now a looping daemon) drains its project's inbox each cycle. ### Build notes - Reuse the flower-ops charter pattern (charter scratchpad + timer loop). Loop reads via recall_brief + writes via brief_* MCP → runs read-only on MAIN like ops (no worktree). - Meta dogfood: this design lives in a brief; the loop can one day refine its own design brief.

## Spec: Briefs refinement loop + first-class daemon-agent roster **Core (Mike):** replace the in-process-only refine with a STANDING LOOP agent per project (like flower-ops) — full file + MCP + recall access — that refines/plans/answers briefs conversationally and continuously. The brief is the durable state (manila folder); rounds are stateless. ### Refine UX - New-brief actions: **Capture** (→ idea/draft; add more context first) and **Capture & Refine** (→ refining; enters the loop queue immediately). - Brief detail: refine becomes a **split/multi-button** — **Quick refine** (synchronous, in-process) vs **Send to refinement loop** (the standing agent). Extensible to more pipelines. - **Live conversational view via Reverb** (already in the stack): the loop posts questions/notes → broadcast → operator sees + replies → the next loop cycle consumes it. Async manila-folder that feels like chat. - **Stops on `planned`** (loop or operator sets it). No auto-dispatch by default; a per-brief **"auto-dispatch when planned" toggle (default OFF)** drops it into the dispatch queue for an orchestrator to pick up. - async-Q&A (brief_ask → operator answers in UI → next cycle consumes) is the interaction mechanism — sufficient; no separate real-time interactive mode needed. ### The loop - **Per-project** brief-refinement agent (project-scoped recall/code/context). flower's is "flexible" only because we dogfood one project; the real model is ONE refine loop per project with active briefs. - **Adaptive cadence**: recent brief activity → poll fast; agent activity but no brief activity → slower; fully dormant → slowest. - Trigger: a brief entering `refining` qualifies it for the loop. ### Daemon-agent roster (first-class) - Standing agents **check in on a time gate (≥ ~every 15 min, not every loop)** with role / solo_process_id / session_id / context-size / usage. flower stores it → a **live-agents status UI** + **death detection** (missed heartbeats). - Solves a problem ALREADY HIT: ops spent ~5 cycles waking a DEAD orchestrator (958 had exited, reincarnated as 969) because nothing tracked liveness. - Extends recall_active (already tracks live/idle sessions + the Solo link) — makes standing daemons first-class + self-reporting. - **Standing daemons:** (1) **Orchestrator** — GLOBAL/cross-project: dispatch, merge, coordinate the other daemons, verify live work, hold the operator inbox. (2) **ops** — feedback/Sentry (per-project when multi-project). (3) **brief-refinement/research** — per-project. Possible 4th: a **pipeline-health** watcher (stuck sessions / queue depth / error rate) — ops catches that reactively via Sentry today. - **Global vs per-project split:** orchestrator = global conductor; refine + ops = per-project workers. ### Operator → orchestrator inbox (new lightweight input) - A NON-interrupting async channel: operator drops a note (small UI box / command) → queued → orchestrator picks it up on its next loop. Not feedback, not a brief — its own thing. The "interrupt now" path (direct message) stays separate from "handle later" (inbox). - Implies the **orchestrator becomes a looping daemon too** (each cycle: check inbox + dispatch queue + roster) — self-pacing + heartbeat-tracked instead of purely reactive. ### Build notes - Reuse the flower-ops charter pattern (charter scratchpad + timer loop) for the brief-refine agent. - The loop reads via recall_brief + writes via brief_* MCP → runs read-only on MAIN like ops (no worktree, no merge contention). - Meta dogfood: this design lives in a brief, and the loop (once built) can refine its own design brief. ### Open / to decide at build - Heartbeat storage shape (table vs extend recall_active); the exact roster UI. - Whether pipeline-health is its own daemon or ops's remit. - Inbox surface (flower UI box vs a Solo/command drop).

Ok - the only note I'd say here is that given your framing here I do see that I'd want the orchestrators to be project scoped. I don't want you, for example, dispatching briefs/tasks in another project (unless I specifically tell you to). So briefs for other projects without orchestrator agents would then just need to be manually dispatched by me or I can tell agents to check and dispatch manually or I can setup orchestrator agents on loops in those projects. Maybe we add some sort of button/flow within flower to spawn an orchestrator agent into a Solo project with a pre-setup prompt instructing them to set a timer and poll the briefs for their project for qualifying dispatchables and whatever else we decide to task/instruct them with? (Should we setup, within that agent roster UI somewhere, a place to view/edit these prompts that are used to spawn these agents?) Should that same flow also support us setting up the ops/refinement agents? (ops in this project only, refinement in a target project the same as the orchestrator)

## Spec: Briefs refinement loop + first-class daemon-agent roster **Core (Mike):** replace the in-process-only refine with a STANDING LOOP agent per project (like flower-ops) — full file + MCP + recall access — that refines/plans/answers briefs conversationally and continuously. The brief is the durable state (manila folder); rounds are stateless. ### Refine UX - New-brief actions: **Capture** (→ idea/draft; add more context first) and **Capture & Refine** (→ refining; enters the loop queue immediately). - Brief detail: refine becomes a **split/multi-button** — **Quick refine** (synchronous, in-process) vs **Send to refinement loop** (the standing agent). Extensible to more pipelines. - **Live conversational view via Reverb** (already in the stack): the loop posts questions/notes → broadcast → operator sees + replies → the next loop cycle consumes it. Async manila-folder that feels like chat. - **Stops on `planned`** (loop or operator sets it). No auto-dispatch by default; a per-brief **"auto-dispatch when planned" toggle (default OFF)** drops it into the dispatch queue for an orchestrator to pick up. - async-Q&A (brief_ask → operator answers in UI → next cycle consumes) is the interaction mechanism — sufficient; no separate real-time interactive mode needed. ### The loop - **Per-project** brief-refinement agent (project-scoped recall/code/context). flower's is "flexible" only because we dogfood one project; the real model is ONE refine loop per project with active briefs. - **Adaptive cadence**: recent brief activity → poll fast; agent activity but no brief activity → slower; fully dormant → slowest. - Trigger: a brief entering `refining` qualifies it for the loop. ### Daemon-agent roster (first-class) - Standing agents **check in on a time gate (≥ ~every 15 min, not every loop)** with role / solo_process_id / session_id / context-size / usage. flower stores it → a **live-agents status UI** + **death detection** (missed heartbeats). - Solves a problem ALREADY HIT: ops spent ~5 cycles waking a DEAD orchestrator (958 had exited, reincarnated as 969) because nothing tracked liveness. - Extends recall_active (already tracks live/idle sessions + the Solo link) — makes standing daemons first-class + self-reporting. - **Standing daemons:** (1) **Orchestrator** — GLOBAL/cross-project: dispatch, merge, coordinate the other daemons, verify live work, hold the operator inbox. (2) **ops** — feedback/Sentry (per-project when multi-project). (3) **brief-refinement/research** — per-project. Possible 4th: a **pipeline-health** watcher (stuck sessions / queue depth / error rate) — ops catches that reactively via Sentry today. - **Global vs per-project split:** orchestrator = global conductor; refine + ops = per-project workers. ### Operator → orchestrator inbox (new lightweight input) - A NON-interrupting async channel: operator drops a note (small UI box / command) → queued → orchestrator picks it up on its next loop. Not feedback, not a brief — its own thing. The "interrupt now" path (direct message) stays separate from "handle later" (inbox). - Implies the **orchestrator becomes a looping daemon too** (each cycle: check inbox + dispatch queue + roster) — self-pacing + heartbeat-tracked instead of purely reactive. ### Build notes - Reuse the flower-ops charter pattern (charter scratchpad + timer loop) for the brief-refine agent. - The loop reads via recall_brief + writes via brief_* MCP → runs read-only on MAIN like ops (no worktree, no merge contention). - Meta dogfood: this design lives in a brief, and the loop (once built) can refine its own design brief. ### Open / to decide at build - Heartbeat storage shape (table vs extend recall_active); the exact roster UI. - Whether pipeline-health is its own daemon or ops's remit. - Inbox surface (flower UI box vs a Solo/command drop).

Mike's idea (2026-06-30), brainstormed + synthesized with the orchestrator: a STANDING LOOP agent for briefs (like flower-ops), with full file + MCP + recall access, that refines/plans/answers briefs conversationally and continuously. Replaces the headless/interactive refine modes; becomes first-class infrastructure with a daemon roster.