Worker-type VRAM cost table + greedy packer (Track B)

# Worker-type VRAM cost table + greedy packer (Track B) Repo: **legit-embedding**, on the #205 single-process runtime. **Depends on #205.** This is the operator's "dynamic memory / heterogeneous worker mix" idea in its simplest correct form. ## What A per-worker-type resource table + a **greedy** packer that fits a heterogeneous model mix onto whatever GPU we get, weighted by queue depth. ### Cost table (static config, measured; versioned) Per `worker_type ∈ {image_embed, text_embed, (future) audio, video}`: - `base_mb` — model weights + context per resident model (MEASURE via nvidia-smi on a real pod) - `per_item_mb` — activation per item at its batch (MEASURE ≥2 points to fit base vs per-item) - `min_batch` / `max_batch` - `in_flight` — async concurrency to hide the xadd write (the I/O-overlap knob, NOT a compute knob) - `target_depth_per_worker` / `priority` — the queue-depth signal ### Greedy packer (NOT a solver) At **pod startup** (latency-insensitive → no runtime re-packing in v1): detect free VRAM (nvidia-smi, reuse #202); for each model type with backlog, greedily allocate resident model + batch + in_flight, subtracting `base_mb + batch*per_item_mb` from the budget, ordered by backlog pressure/priority, until the budget is exhausted or all backlogged types are served. Industry pattern — **resource requests + bin-packing** (Borg/K8s/Nomad) + **queue-depth autoscaling** (KEDA: `desired ≈ ceil(depth/target)`). Greedy keeps it debuggable — the "no horribly complicated system" guarantee. ### v1 scope / explicitly deferred - v1: compute the mix ONCE at startup from current per-queue depths; a pod is short-lived → re-pack on next spawn. - Deferred (phase 2): runtime dynamic re-packing (add/evict models mid-life) — adds restart complexity, rarely worth it. - **Ruled out: MIG** (fleet A40/A4500/3090/4090 don't support it). **MPS** = escalation only if co-located models contend for SMs (GPU ~30% util → headroom exists; defer until measured). ## Payoff Real value lands with the 2nd+ concurrent workload — text now (#207), video/vodmanager later. Single-workload (image-only) still benefits: one model copy + more in_flight = a bigger batch than N duplicate processes could afford. ## Status Draft (idea). Depends #205.

Per-worker-type resource table (base_mb, per_item_mb, batch bounds, in_flight, target_depth) + a GREEDY VRAM packer that fits a heterogeneous model mix weighted by queue depth, computed at pod startup. Industry pattern: resource requests + bin-packing + KEDA-style queue-depth autoscaling. Depends on #3. Full spec to follow.