flower
/
All briefs
complete draft note flower

Giant-session summarization guard: skip un-summarizable sessions instead of infinite re-segment (FLOWER-16 endgame)

canonical · plan

Spec

markdown

hand-off · dispatch

Dispatch

Auto-dispatch

when it reaches planned

Design-loop

design pass before build

This brief is complete — dispatch is closed.

provenance · append-only

Trace

live
or paste a screenshot uploading…
  1. status change 3d ago
    agent · flower-orchestrator
  2. link added 3d ago
    agent · system:commit-trailer
  3. participant joined 3d ago
    system · system:commit-trailer
  4. link added 3d ago
    agent · flower-orchestrator
  5. status change 3d ago
    agent · flower-orchestrator
  6. note added 3d ago

    ## Problem (ops cycle 126, kv flower-ops:failed-jobs-trend) failed_jobs was filling LINEARLY ~30/hr (110 when caught; I flushed it), ~99% SegmentSession timeouts on GIANT sessions (#705 ~356M tokens, #901 ~61M, #406, #897, plus big codex sessions like #1290/6515-events). Diagnosis: SegmentSession has **$tries=1** (no retry) + timeout=1800s, and there's **no scheduled re-dispatch** — so it's not retry-looping. It's ongoing INGESTION dispatching SegmentSession for growing/large sessions (IngestSession.php:293), which then **time out because the session's token count exceeds what even the adaptive-chunking floor (90ev, FLOWER-16) can fit**. The sessions stay ingest_state=indexed (old segments remain; #30 made success win over late failure), but each attempt lands one failed_jobs row. NOT breaking anything (normal sessions index fine, error=0, no key leak) — pure queue-hygiene that fills indefinitely. ## Fix — pre-flight guard (conservative, reversible, no data loss) 1. **Token-ceiling guard in SegmentSession::handle():** before summarizing, estimate the session's token/event size. If it exceeds a configurable ceiling `config('flower.summarize.max_session_tokens')` (default set WELL above normal — the point where even the adaptive floor can't fit it, so only true giants trip it), SKIP cleanly: mark a terminal state (ingest_state='too_large' or a session flag `summary_skipped_oversize`), log a warning, and RETURN SUCCESS (do NOT throw → no failed_jobs row). Idempotent. Normal/large-but-summarizable sessions unaffected. 2. **Don't re-dispatch skipped giants:** where ingestion re-dispatches SegmentSession, skip sessions already marked too_large (unless they somehow drop below the ceiling). Confirm the exact re-dispatch path (IngestSession incremental re-ingest of growing sessions is the suspect). 3. **Reconcile existing giants:** extend flower:ingest-backlog (or a one-off) to mark current over-ceiling sessions too_large so they stop re-queuing. Orchestrator runs this on MAIN. 4. **Surface, don't hide:** flower:ingest-backlog should count 'too_large' sessions separately (visible-but-inert). (v2 / operator decision, OUT OF SCOPE: whether giants deserve a head-only or truncated summary — this brief just stops the bleed.) ## Acceptance - Over-ceiling session → marked too_large + returns success (no exception/failed_jobs); under-ceiling → summarizes normally. Ingestion doesn't re-dispatch too_large sessions. - Reconcile marks existing giants; after merge+reconcile+flush, failed_jobs stays ~flat. - Tests (sqlite): over-ceiling skip+mark+no-throw, under-ceiling normal, no-re-dispatch of too_large. Suite green; pint. - FLAG: changes SegmentSession + IngestSession (Horizon jobs) → graceful Horizon reload on merge. `Brief:` trailer with this id. ## Guardrails ⚠️ sqlite tests ONLY; no live DB writes from the worktree (its DB is now the isolated flower_<wt>, but keep the discipline). Orchestrator runs the reconcile + flush + Horizon reload on MAIN. ## Provenance ops cycle 126 → orchestrator (Funnel B, bug=autonomous). FLOWER-16 adaptive-chunking-floor endgame: when even the floor can't fit a session, SKIP instead of infinite re-attempt.

    agent · flower-orchestrator
  7. participant joined 3d ago
    system · flower-orchestrator

epic · dependencies

Relationships

epic parent

depends on

No dependencies — dispatchable once planned.

agents · waves

Participants

  • flower-orchestrator participant · active
  • system:commit-trailer participant · active

trace · graph

Links

  • Commit #1565 execution
  • Scratchpad #346 execution

scope

Projects

  • flower · primary

dogfood · read-only

Agent’s-eye view

The literal recall_brief payload an agent gets — same service path as the MCP tool.