Ingest still backed up/not getting queued We've got to do _SOMETHING_

You are being dispatched from flower Brief #174: Ingest still backed up/not getting queued

We've got to do _SOMETHING_

Recall pointer:
- Use recall_brief with id 174 for the full folder if you need provenance.

Target:
- project: flower (/Users/mikeferrara/Documents/code/flower)
- branch: choose an appropriate branch
- worktree: not specified
- kind: fresh

Current brief spec:
## Goal
Make ingest health OBSERVABLE and CLASSIFIED so the operator (and the ops daemon) can tell, per project, whether ingest is healthy-but-working, backed-up, stuck/broken, or genuinely has-no-sessions-in-window — and surface that health persistently. Fixes the "42/43 stuck / going backwards / awaiting sessions" ambiguity the operator can't currently diagnose.

## Slice (operator Q&A #36, 2026-07-04)
This brief = **ingest-HEALTH classification + ops-alerting**. The raw "where is ingest at" session/embed/commit progress views fold into PLANNED **#112** (noted there). Two pieces spun out to their own briefs per operator: the git-history-coverage viz (#38 → new brief) and a potential dedicated ingest-health page/banner (#37 → new brief).

## Scope
1. **Project-level ingest-health classification** — a derived health state per project:
   - `healthy_idle` — all in-window sessions indexed, no active ingest.
   - `healthy_ingesting` — actively/normally ingesting (initial import or live) → "42/43" reads GREEN.
   - `backed_up` — large parsed backlog draining within normal bounds.
   - `stuck` — un-indexed sessions with no forward progress / errored / zero-chunk-wedged tail (reuse flower:ingest-backlog stuck-tail detection) AND not actively ingesting.
   - `no_sessions_in_window` — nothing to ingest for the configured time window (disambiguates the current ambiguous "awaiting sessions").
   Unhealthy = not-all-in-window-indexed AND no active/new sessions AND not initial import.
2. **Persistent ops health surface (operator Q&A #37 = BOTH):**
   - **Top-nav ops/metrics badge** — ingest queue depth / count of un-indexed sessions / count of projects in stuck/unhealthy state; at-a-glance platform-health signal.
   - **Per-project health indicator on /projects** — the classified state above; "42/43" GREEN when `healthy_ingesting`; distinct treatment for `stuck` vs `no_sessions_in_window`.
3. **Ops-daemon alerting hook** — expose the classification so flower-ops flags projects that flip to stuck/unhealthy ("exactly what the ops daemon should watch"). Read-only surface for ops.

## Reuse / substrate (do NOT rebuild)
- `flower:ingest-backlog` (#14) already reports ingest_state counts + the stuck tail — the classifier builds on that logic, not a duplicate.
- Part of the "going backwards / 42-43 stuck" symptom is the re-ingest churn bug ops already root-caused + routed (todo #373 / scratchpad 1053, fix in flight) — that fix is separate; this brief is the durable health-visibility layer.

## Out of scope (own briefs)
- Raw session/embed/commit progress views → #112 (folded in).
- Git-commit-history vs configured-time-window coverage viz → spun-out NEW brief (Q38).
- Dedicated ingest-health page/banner → spun-out NEW brief (Q37).
- Ingest queue/lane architecture → #173.

## Acceptance
- Per-project ingest-health state derived + exposed (service/attribute) across the 5 states; unit tests for classification incl. `no_sessions_in_window` vs `stuck` disambiguation and the GREEN-while-healthy-ingesting case.
- Top-nav badge (queue depth / unhealthy-project count) + per-project /projects indicator render the state; feature tests render 200 with real-shaped data.
- Classification consumable by flower-ops for alerting. `php artisan test` green + pint. `Brief: #174` trailer.

## Provenance
Operator note (2026-07-04, "do SOMETHING") → flower-refine grounding (append) → operator Q&A #36/#37/#38 → this spec. Related: #112 (visibility, folds in raw view), #173 (ingest lanes), spun-out briefs (git-history coverage viz; dedicated health page/banner), ops churn fix #373.

Recent/key trace events:
[4] participant_joined flower-refine: (no body)
[5] refinement flower-refine: Refine grounding (flower-refine). #174 bundles several separable things, and much of the substrate already exists:

- **Raw visibility overlaps PLANNED #112** ("view/UI for where ingest is at on sessions/embeds/commits"). The "where's ingest at" part is essentially #112.
- Data mostly exists: `flower:ingest-backlog` (from #14) already reports ingest_state counts + the stuck tail; todo #330 already lists "/projects: clear indexing/ingest-state indicator per project" + "Health: UI banner/alerts."
- The **"going backwards / 42-43 stuck"** symptom is (at least partly) the re-ingest churn bug ops ALREADY root-caused + routed: the Claude status file folded into the re-ingest change-signature → indexed↔summarized flap (todo #373 / scratchpad 1053, fix in flight). So part of the pain is already being addressed operationally.

DURABLE net-new value in #174 (beyond #112):
1. **Project-level ingest HEALTH classification** — distinguish backed-up vs stuck/broken vs genuinely-no-sessions-in-window (operator explicitly can't tell these apart; "awaiting sessions" is ambiguous). "42/43" should read GREEN when actively/healthily ingesting.
2. **A persistent ops health surface** (top-nav badge: queue depth / un-indexed sessions / count of indexed-but-unhealthy projects) — "exactly what the ops daemon should watch."
3. The **git-commit-history vs configured-time-window coverage viz** — I searched the corpus; there is NO dedicated brief for this yet (only the /analytics view + git-commit ingestion infra), so it'd be net-new.

Proposed slicing surfaced to operator as async questions. Related: #173 (ingest priority queues), #112 (planned visibility UI).
[6] agent_question flower-refine: #174's raw 'where is ingest at' visibility overlaps PLANNED brief #112. How do you want to slice the work?
[7] agent_question flower-refine: Where should the persistent ingest-health signal live?
[8] agent_question flower-refine: Split the 'git-commit-history vs configured ingest time-window coverage' visualization into its own brief? (No dedicated brief exists for it yet.)
[9] operator_answer operator:mike: Both top-nav badge AND per-project indicator + brief for potential dedicated health page/banner
[10] operator_answer operator:mike: Keep #174 = ingest-HEALTH classification + ops-alerting; fold the raw visibility into #112
[11] operator_answer operator:mike: Yes — separate brief
[12] spec_snapshot flower-refine: Ingest still backed up/not getting queued

We've got to do _SOMETHING_ to get this right. There's been briefs on this multiple times now. 

When I open the /projects view and see projects with "42/43" showing in their session ingest indicator but that project hasn't been touched in 3 weeks by flowers own indication - why? We need to fix whatever this is.

We need a way to detect/determine when a project's indexing state isn't healthy - if there's no new sessions or active sessions and we don't have all sessions indexed - that's unhealthy (unless we're still ingesting from the initial import)

If we're actively ingesting from an active project's active sessions, that's fine. Maybe "42/43" should be green in this state (or the whole little progress bar thing is green or whatever to indicate it's health but ingesting) 

There's a bunch of projects showing 'awaiting sessions' and I have no idea if those aren't indexing because ingest is broken/stuck or because it's backed up or because there's actually no sessions to be ingested within the given time window. And this probably ties into what I think is a pending brief about using the historical git commit history ingest to build a chart/visualization of the actual git history against the project and it's configured ingest time window to see if we're missing or would be missing activity. I'm wondering if we shouldn't _ALSO_ pre-ingest the _SESSIONS_ and just not do summaries/embeddings/push to meili unless they're indexed - this way we can _also_ visualize sessions in a way that lets us know where things are at relative to the time window configured. Maybe these projects I've 'indexed' but still show 'awaiting sessions' just have no sessions within that window - I have no way to tell.

Overall - I think this is an important enough ops/platform health aspect/metric that we should be tracking this and maintaining it actively. What do you think about some sort of ops/metrics display in our top nav by the buttons that indicates health of ingest? ie: number of jobs in ingest queue/un-indexed sessions/project count of indexed with unhealthy ingest/etc?

This is exactly the kind of thing the ops daemon should be looking for/getting flagged about.
[13] refinement flower-refine: ## Goal
Make ingest health OBSERVABLE and CLASSIFIED so the operator (and the ops daemon) can tell, per project, whether ingest is healthy-but-working, backed-up, stuck/broken, or genuinely has-no-sessions-in-window — and surface that health persistently. Fixes the "42/43 stuck / going backwards / awaiting sessions" ambiguity the operator can't currently diagnose.

## Slice (operator Q&A #36, 2026-07-04)
This brief = **ingest-HEALTH classification + ops-alerting**. The raw "where is ingest at" session/embed/commit progress views fold into PLANNED **#112** (noted there). Two pieces spun out to their own briefs per operator: the git-history-coverage viz (#38 → new brief) and a potential dedicated ingest-health page/banner (#37 → new brief).

## Scope
1. **Project-level ingest-health classification** — a derived health state per project:
   - `healthy_idle` — all in-window sessions indexed, no active ingest.
   - `healthy_ingesting` — actively/normally ingesting (initial import or live) → "42/43" reads GREEN.
   - `backed_up` — large parsed backlog draining within normal bounds.
   - `stuck` — un-indexed sessions with no forward progress / errored / zero-chunk-wedged tail (reuse flower:ingest-backlog stuck-tail detection) AND not actively ingesting.
   - `no_sessions_in_window` — nothing to ingest for the configured time window (disambiguates the current ambiguous "awaiting sessions").
   Unhealthy = not-all-in-window-indexed AND no active/new sessions AND not initial import.
2. **Persistent ops health surface (operator Q&A #37 = BOTH):**
   - **Top-nav ops/metrics badge** — ingest queue depth / count of un-indexed sessions / count of projects in stuck/unhealthy state; at-a-glance platform-health signal.
   - **Per-project health indicator on /projects** — the classified state above; "42/43" GREEN when `healthy_ingesting`; distinct treatment for `stuck` vs `no_sessions_in_window`.
3. **Ops-daemon alerting hook** — expose the classification so flower-ops flags projects that flip to stuck/unhealthy ("exactly what the ops daemon should watch"). Read-only surface for ops.

## Reuse / substrate (do NOT rebuild)
- `flower:ingest-backlog` (#14) already reports ingest_state counts + the stuck tail — the classifier builds on that logic, not a duplicate.
- Part of the "going backwards / 42-43 stuck" symptom is the re-ingest churn bug ops already root-caused + routed (todo #373 / scratchpad 1053, fix in flight) — that fix is separate; this brief is the durable health-visibility layer.

## Out of scope (own briefs)
- Raw session/embed/commit progress views → #112 (folded in).
- Git-commit-history vs configured-time-window coverage viz → spun-out NEW brief (Q38).
- Dedicated ingest-health page/banner → spun-out NEW brief (Q37).
- Ingest queue/lane architecture → #173.

## Acceptance
- Per-project ingest-health state derived + exposed (service/attribute) across the 5 states; unit tests for classification incl. `no_sessions_in_window` vs `stuck` disambiguation and the GREEN-while-healthy-ingesting case.
- Top-nav badge (queue depth / unhealthy-project count) + per-project /projects indicator render the state; feature tests render 200 with real-shaped data.
- Classification consumable by flower-ops for alerting. `php artisan test` green + pint. `Brief: #174` trailer.

## Provenance
Operator note (2026-07-04, "do SOMETHING") → flower-refine grounding (append) → operator Q&A #36/#37/#38 → this spec. Related: #112 (visibility, folds in raw view), #173 (ingest lanes), spun-out briefs (git-history coverage viz; dedicated health page/banner), ops churn fix #373.
[14] status_change flower-refine: (no body)
[15] note_added flower-refine: [flower-refine] Spin-outs created per operator Q&A, both status=idea:
- **#181** — Ingest coverage viz: git-commit history vs configured time-window (+ optional session-metadata pre-ingest) — from Q38.
- **#182** — Dedicated ingest-health page/banner (richer surface beyond this brief's badge+indicator) — from Q37.
Raw "where is ingest at" visibility folded into **#112** (noted there). #174 now = ingest-health classification + persistent surface (top-nav badge + per-project indicator) + ops-alerting hook → moved to PLANNED (orchestrator owns dispatch).

Recommended linked context:
{
    "todos": [],
    "scratchpads": []
}

Execution notes:
- Treat the brief as the source of truth.
- Keep work scoped to this dispatch request.
- Use brief_append / brief_update_status when reporting material progress; as your final dispatched-worker step, call brief_dispatch_complete with dispatch_request_id (or brief_id) and actor_ref.
- Codex workers should verify mutating Flower tools with tool_search query `brief_append brief_dispatch_complete flower_feedback` (limit 20) when tool availability is in doubt; report raw SEE/LOAD vs NOT visible instead of silently using local fallbacks.
- Add a git commit trailer `Brief: #174` to every commit for this brief so flower can exact-link commits back to the brief.