Spike: choose the single-process multi-model serving substrate (Track B)

# Spike: choose the single-process multi-model serving substrate (Track B) Repo: **legit-embedding** (isolated worktree). A decision gate that feeds #205. Latency-insensitive workload → optimize throughput/$. ## Decide Substrate for the single-process, multi-model async worker: - **Ray Serve** — Python-native, arbitrary pre/post (our fetch→embed→XADD), dynamic batching, multi-model, replica autoscale. Likely front-runner. - **Custom asyncio loop** — simplest, no heavy dep, but reimplements dynamic batching + lifecycle. Viable given how contained our loop is. - **Triton** — evaluate but expect to REJECT: rigid for our Redis-stream I/O + pre/post (Python backend/ensemble is fiddly). ## Prototype must prove 1. Load EVA02 image model + the text model **once** in one process (drop per-process duplication: `models.py:77` SingletonEmbeddingModel, `text_models.py:13` ModelManager). 2. **Async overlap**: a batch's XADD write (async) overlaps the next batch's GPU compute — the core win. Enough in-flight to hide ~1235ms xadd behind ~530ms compute (~4 in-flight to saturate; HANDOFF §5) with ONE model copy + N buffers, not N copies. 3. Per-model dynamic batching (size from a VRAM budget — feeds #206). 4. Clean Redis consumer-group draining + a **single, consolidated telemetry identity** — current names embed hostname+PID (`cli.py:67`, `embedding.py:23`, `preparation.py:68`, `stats_publisher.py`). A single process must consolidate these or conductor dashboards/telemetry keys change shape. ## Constraints / carry-overs Keep the wire contract unchanged: `encode_embedding` (`encoding.py:22`, base64_fp32) + `StreamConfig` (`config.py`). Must be able to honor `reply_to` (feeds #205). Must drain the same task streams as the current prep→embed handoff (replacing the `/dev/shm` + Celery `send_task` hop, `preparation.py:429`). ## Deliverable A short decision doc (chosen substrate + why) + a runnable prototype + a concrete architecture for #205. ## Status Draft (idea). Track B entry point; independent.