cv2(gpu-death): guest self-detect permanent renderer/GPU death → signal + recycle (the #154 residual fix) #41

Merged

triform-admin merged 1 commit from cv2/gpu-death-recycle into main

2026-07-10 07:47:42 +00:00

triform-admin commented

2026-07-10 07:46:38 +00:00

Owner

The #154 residual — the last thing between CV2 WebRTC and ≥90%

Root-caused live 2026-07-10 by correlating a failed portal session to physics logs: the full WebRTC handshake completes (offer + answer + ICE all delivered bidirectionally, ICE connected) yet the guest sends 0 RTP / 0 frames. That is the run11-class residual — a guest whose BeginFrame ack-loop is outwardly healthy (issued≈acked≈29fps post-GPU-reinit) while the renderer produces ZERO frames forever. No watchdog-timing fix sees it; the capture-rearm fix (#38) doesn't cover it (renderer never produces from the start, not a nav-swap). The per-worker poisoned-golden gate + recycler (tf-multiverse) don't catch it either — the worker's golden is healthy; this is a per-session boot death.

The fix (guest-side detect → signal → recycle)

Layer 1 (cb_begin_frame_driver): count consecutive EmitDiagnostic ticks with captured_delta==0 (renderer PRODUCTION, not ack-chain liveness — the only substrate that sees this failure). At 6 ticks / 30s (kZeroProductionTicksBeforeDeath) fire a one-shot SetPermanentDeathCallback + CV2-GPU-DEATH log. 30s = 2× the =15 watchdog's 15s worst-case rebind, so a slow-but-recovering guest's catch-up burst resets the streak first — never false-positives.
Layer 2 (cb_wire_envelope + cb_offerer_driver): new EnvelopeType::kSessionUnhealthy (distinct from bye so physics tells "user closed tab" from "guest self-detected death"); CloseUnhealthy() emits it before teardown. + round-trip test.
Layer 3 (main_parts): OnGpuPermanentDeath → offerer_driver_->CloseUnhealthy("gpu-permanent-death") → delayed quit (flush the envelope) so the dead microVM frees promptly and physics re-pins a fresh guest.

Physics-side consumer (kSessionUnhealthy → registry.release + recycle) already landed in tf-multiverse (#157). This is the guest emitter half — authored 2026-07-06, never merged.

Strings-gate marker: CV2-GPU-DEATH. Composes cleanly with the deployed capture-rearm fix (#38) — verified SetRecaptureOnRvhSwapCallback intact.

🤖 Generated with Claude Code

## The #154 residual — the last thing between CV2 WebRTC and ≥90% Root-caused live 2026-07-10 by correlating a failed portal session to physics logs: the full WebRTC handshake completes (offer + answer + ICE all delivered bidirectionally, ICE connected) yet the guest sends **0 RTP / 0 frames**. That is the run11-class residual — a guest whose BeginFrame ack-loop is outwardly healthy (issued≈acked≈29fps post-GPU-reinit) while the renderer produces ZERO frames forever. No watchdog-timing fix sees it; the capture-rearm fix (#38) doesn't cover it (renderer never produces from the start, not a nav-swap). The per-worker poisoned-golden gate + recycler (tf-multiverse) don't catch it either — the worker's golden is healthy; this is a *per-session* boot death. ## The fix (guest-side detect → signal → recycle) - **Layer 1** (`cb_begin_frame_driver`): count consecutive EmitDiagnostic ticks with `captured_delta==0` (renderer PRODUCTION, not ack-chain liveness — the only substrate that sees this failure). At 6 ticks / 30s (`kZeroProductionTicksBeforeDeath`) fire a one-shot `SetPermanentDeathCallback` + `CV2-GPU-DEATH` log. 30s = 2× the =15 watchdog's 15s worst-case rebind, so a slow-but-recovering guest's catch-up burst resets the streak first — never false-positives. - **Layer 2** (`cb_wire_envelope` + `cb_offerer_driver`): new `EnvelopeType::kSessionUnhealthy` (distinct from `bye` so physics tells "user closed tab" from "guest self-detected death"); `CloseUnhealthy()` emits it before teardown. + round-trip test. - **Layer 3** (`main_parts`): `OnGpuPermanentDeath` → `offerer_driver_->CloseUnhealthy("gpu-permanent-death")` → delayed quit (flush the envelope) so the dead microVM frees promptly and physics re-pins a fresh guest. Physics-side consumer (`kSessionUnhealthy` → `registry.release` + recycle) already landed in tf-multiverse (#157). This is the guest emitter half — authored 2026-07-06, never merged. Strings-gate marker: `CV2-GPU-DEATH`. Composes cleanly with the deployed capture-rearm fix (#38) — verified `SetRecaptureOnRvhSwapCallback` intact. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

triform-admin added 1 commit

2026-07-10 07:46:38 +00:00

cv2(gpu-death): guest self-detect permanent renderer/GPU death -> signal + recycle

CI / Docs link check (pull_request) Failing after 2s

Details

CI / Lint (pull_request) Failing after 30s

Details

CodeQL / Analyze javascript-typescript (pull_request) Failing after 30s

Details

CodeQL / Analyze go (pull_request) Failing after 31s

Details

native-peer-gate / Build chromeless:ci for gate (pull_request) Failing after 1m1s

Details

native-peer-gate / native-peer-gate-scaffold (permissive) (pull_request) Has been skipped

Details

native-peer-gate / native-peer-gate-strict (M7 gate) (pull_request) Has been skipped

Details

E2E / docker-compose + Playwright (pull_request) Failing after 1m8s

Details

CI / Build container image (pull_request) Failing after 1m11s

Details

CI / Container smoke test (pull_request) Has been skipped

Details

0a997b2142

Run11-class residual (~8% after the =15 GPU-crash fix): guest whose BeginFrame ack-loop is healthy but renderer produces ZERO frames forever. Guest detects (captured_delta==0 for 30s once armed) -> session_unhealthy wire envelope -> CloseUnhealthy -> clean exit so physics re-pins a fresh guest. Review-hardened: arm-gate prevents warm-boot latch-burnout; PrepareForTeardown before close; +tests. Strings-gate marker CV2-GPU-DEATH.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>