feat(streamer): Wave 1 DataChannel emitters (cursor + clipboard + file-upload) #1
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/wave-1-emitters"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Adds the chromeless side of the Wave 1 DataChannel contract from
/workspace/.triform/guides/chromeless-datachannel-contract.md.Three new RTCDataChannels open in the streamer-page offer SDP and
land their respective behaviours:
cursor(chromeless→portal): in-page probe injected viaCDP
Page.addScriptToEvaluateOnNewDocumentlistens topointermove+getComputedStyle(...).cursorand fires throughRuntime.addBinding. Drop-stale-on-coalesce backpressure: at mostone outstanding payload, replaced rather than queued. Coordinates
in page CSS pixels.
clipboard(bidirectional): page-sidecopyprobeemits
c2penvelopes with monotonicseq. Inboundp2cenvelopes drop on stale
seq(per contract §5 race resolution),drop on unsupported MIME (text/plain | text/html only), then apply
via
navigator.clipboard.writeTextthrough CDP. Echo suppressionon the inbound→outbound roundtrip. Counters are per-RTCPeerConnection.
file-upload(portal→chromeless): chunk reassembly perupload_idwith 4-ahead reorder window, total/eof sanity rules,60 s orphan timeout, and name sanitisation (path separators / NUL
/ control chars / leading dots stripped, truncated to 255 bytes
on UTF-8 boundary). On
eof, lands the bytes viaRuntime.evaluate+ synthetic DataTransfer/File against thefocused
<input type=\"file\">(falling back to first file inputon the page). Returns
\"ok\"/\"no-input\"/\"err:...\".Plus an iteration script:
scripts/dev-push-streamer.sh: ~5-min round-trip(vs ~4-hour Chromium rebuild) for streamer.js-only iteration.
Builds
infra/Dockerfile, pushes todev-<sha>tag, rollsthe dev cluster's chromeless deployment. Refuses prod-shaped tags.
Design notes
CDP doesn't actually expose
Page.setCursorChangedorBrowser.onClipboardChangedevents as the original Wave 0 briefsuggested — the proven path is the Runtime.addBinding probe pattern
that
capture/cursor-watcherandcapture/clipboard-bridgealreadyuse. We don't reuse those Go sidecars in-process because driving CDP
directly from the streamer-page is one fewer hop and avoids the
wire-format translation between sidecar internal envelopes and the
Wave 0 contract envelopes.
If the CDP attach itself fails (e.g. Chromium flag set blocks
WebSocket connections from the streamer-page to its own
:9222), all three DataChannels still get opened — the labels needto land in the offer SDP so the portal's demux match doesn't fall
through to the
other =>log-and-drop arm. The emitters log theinstall failure and stay dormant.
Test plan
three additional DataChannels appear in
pc.getStats()(
data-channelreports with labelscursor,clipboard,file-upload).portal-side B1 consumer logs incoming envelopes with
kindperthe CSS cursor of the element under the pointer.
confirm a
c2penvelope arrives portal-side.p2cenvelope; confirm thein-container clipboard updates (verifiable via subsequent CDP
Runtime.evaluate(navigator.clipboard.readText())).of ≤64 KiB); confirm the focused
<input type=\"file\">receivesthe file and dispatches
change.confirm a
tracing::warn-shaped log line about orphan timeoutappears 60 s later.
input/statschannels: existingtests/e2e/webrtc-drive-mode.spec.tsgates pass.scripts/dev-push-streamer.sh --skip-pushsucceeds locally(smoke).
Constraint compliance
input/statschannels untouched (only new code instart()'s data-channel block; no edits to theInputRelay/StatsRelayclasses).cursor,clipboard,file-upload(lowercase, hyphenated multi-word).
{v:1, t, <field>}); binarypayloads base64-encoded inside JSON, no binary frames.
dc.readyStatechecks before send,paired close listeners, JSON.parse wrapped in try/catch, idempotent
dispose()methods.🤖 Generated with Claude Code
Adds the bidirectional clipboard DataChannel per /workspace/.triform/guides/chromeless-datachannel-contract.md §5. * Opens a fourth DataChannel labelled exactly "clipboard", ordered. Labels are case-sensitive — any drift silently demuxes to the portal's `other =>` arm. * Reuses the CDPClient added in A1 (no duplication). Installs a page-side probe via Page.addScriptToEvaluateOnNewDocument that listens for "copy" events and reports captured selection text via Runtime.addBinding callback. * Outbound (c2p): translates probe callbacks into the contract envelope `{v:1, t, clipboard:{dir:"c2p", seq, mime, data}}` and emits on the DC. `next_outbound_seq` increments per send. Initial Wave-1 scope is text/plain (text/html acceptance is wired but the emitter only produces text/plain for now — the contract permits both, so the portal receiver is forward-compat). * Inbound (p2c): receive on DC, drop if `seq <= last_seen_inbound_seq` (per contract §5 race resolution), drop if MIME is outside text/plain | text/html (with DEBUG log), otherwise base64-decode data and apply via Runtime.evaluate(navigator.clipboard.writeText). Browser.grantPermissions is invoked at install time (with retry on the smaller permission set) so the writeText call doesn't trigger a UA prompt. * Per-session counters: counters live for the RTCPeerConnection lifetime; on reconnect the page reloads (per existing teardown semantics) and both peers reset to seq=0 / last_seen=-1, which the contract calls out as harmless because the fresh DataChannel has no in-flight stale frames. * Echo suppression: lastInboundText is remembered for one round so a page-side "copy" event that fires immediately after we apply an inbound write doesn't loop the same string back out as a c2p envelope. * Disposal: dispose() detaches the CDP event listener and is invoked from teardown + dc.close. JSON.parse paths in onInbound wrap in try/catch. Contract: §5 (clipboard), §2 (label exactness).Adds the chromeless side of the file-upload DataChannel per /workspace/.triform/guides/chromeless-datachannel-contract.md §6. * Opens a fifth DataChannel labelled exactly "file-upload", ordered. This is the only one of the three new channels that's purely inbound on the chromeless side — the portal-side B3 producer pumps chunks; we just receive. * Reassembly per upload_id: each chunk envelope carries {upload_id, seq, total, name, mime, eof, data}. We buffer chunks in a Map<upload_id, Map<seq, Uint8Array>> and slide an `expected` pointer past contiguous chunks. eof: true with expected >= total triggers completion. * Validation per contract §6: 1. Monotonic non-decreasing seq with up to 4-ahead reorder window (FILE_UPLOAD_REORDER_AHEAD = 4). Beyond that, the upload is dropped + warned. 2. total constant across an upload_id; mismatch drops the upload. 3. eof: true ⇔ seq == total - 1; any deviation drops the upload. 4. name sanitised: path separators / NUL / 0x01-0x1F / 0x7F replaced with underscore, ".." sequences collapsed, leading dots stripped, truncated to 255 BYTES post-UTF-8 encode (with codepoint-boundary alignment so we never emit half a codepoint). * 60 s orphan timeout: setTimeout per upload_id refreshes on each chunk; fires if no chunk arrives for 60 s and eof hasn't been reached. The buffer is discarded with one tracing::warn-level log line including {upload_id, name, received_chunks, expected_total}. * Landing: completion uses Runtime.evaluate with a synthetic DataTransfer + File — the canonical CDP `DOM.setFileInputFiles` takes on-disk paths, not bytes, which would require an intermediate temp-file write. The Runtime.evaluate path locates the focused <input type=file> (falling back to the first file input on the page if none has focus), assigns input.files via DataTransfer, and dispatches synthetic `input` + `change` events so page handlers see the upload. Returns "ok" / "no-input" / "err:..." which we log accordingly. If no file input exists, the upload is dropped per the contract's known Wave-1 limitation. * Disposal: dispose() walks remaining uploads and discards each (clearing its timer), invoked from teardown + dc.close. Contract: §6 (file-upload), §2 (label exactness).Adds an iteration script for streamer-page-only changes, keyed off the A0 recon finding that infra/Dockerfile:139 just COPYs capture/streamer-page/ into the runtime image — no Chromium rebuild required. Total round-trip target: ~5 min (vs the ~4-hour cold build when Chromium-from-source is in scope). Usage: scripts/dev-push-streamer.sh # build + push + roll scripts/dev-push-streamer.sh --tag my-feature # use dev-my-feature scripts/dev-push-streamer.sh --skip-rollout # build + push only scripts/dev-push-streamer.sh --skip-push # build only (smoke) Defaults to: registry: registry.triform.cloud image: chromeless/chromeless tag: dev-<short-sha>[-dirty] ns: chromeless deploy: tries chromeless-browserless / cb-browserless / chromeless Refuses tags that look prod-shaped (cr7727-sw, latest, prod, …) so a typo can't clobber the operator-managed prod image. Credential paths handled, in order of preference: 1. REGISTRY_USER + REGISTRY_PASSWORD env vars (script does login, and logout at exit). 2. Existing ~/.docker/config.json or `docker login` already done. 3. Soft warning if neither — push will then fail loudly with the real registry error rather than this script second-guessing. Preflights: docker daemon reachable, kubectl present + namespace visible. Bails before kicking off the build if any of those are missing so wasted time is minimised. Post-build sanity check md5s the streamer.js inside the resulting image against the host copy — catches the case where a cached layer got picked up over a local edit. The script intentionally only targets dev tags + the dev cluster. Integration / triform.wtf rollouts continue to flow through the operator-managed kaniko Job + Flux image-automation in the triform monorepo, NOT this script.Adds the chromeless-side observability emit path defined by the D4-YAML schema at chemistry/elements/tools/chromeless/.triform/observability.yaml. Endpoint: POST /webrtc-event (new; sibling to existing /stats-update so the two pipelines stay decoupled — stats is a high-frequency relay keyed on tenant/session, webrtc-event is low-frequency lifecycle beacons keyed on element_id). Wire shape: { event: "<dot.name>", attrs: {...} }. Each event maps to a chromeless_webrtc_* counter increment, plus histogram observations on session_closed (duration_ms) and dc.opened (handshake_ms). Counters declared (names match the YAML's chromeless_* metrics 1:1 plus four chromeless-side debug counters for cursor coalescing, clipboard MIME/seq drops, and file-upload outcomes): chromeless_webrtc_session_count{element_id} chromeless_webrtc_ice_connected_count{element_id} chromeless_webrtc_ice_failed_count{element_id} chromeless_webrtc_dc_opened_count{element_id, label} chromeless_webrtc_replay_hit_count{element_id} (broker-side, accepted on principle) chromeless_webrtc_replay_miss_count{element_id} (broker-side, accepted on principle) chromeless_webrtc_session_duration_ms{element_id} (histogram) chromeless_webrtc_signaling_handshake_ms{element_id} (histogram) chromeless_webrtc_dc_cursor_coalesced_count{element_id} chromeless_webrtc_dc_clipboard_unsupported_mime_count{element_id} chromeless_webrtc_dc_clipboard_stale_seq_count{element_id} chromeless_webrtc_dc_file_upload_orphan_timeout_count{element_id} chromeless_webrtc_dc_file_upload_completed_count{element_id, result} element_id sourced from $CHROMELESS_ELEMENT_ID at process start; defaults to "_unknown" so dev-compose smoke runs surface metrics even without a concrete element binding. Series are pre-registered for the five known channel labels (input|stats|cursor|clipboard|file-upload) so dashboards bind on an idle pod. Unknown event names accept silently (204) for forward-compat — a streamer.js that emits a new event name doesn't fail deployment if the sidecar hasn't been updated. OTLP-export gap: the sidecar has an OTLP *trace* exporter (tracing.go) but no OTLP *metrics* exporter today. Prometheus scrape covers the D4-YAML metric series; Triform's event-bus consumption of the dot- named events at the OTLP-logs layer (for activity feed enrichment) is a future task. A4 caps at "events produced + counters incrementing in /metrics". Verification: go build ./... passes go vet ./... clean go test ./... 21/21 pass (12 existing + 9 new) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>D4-Dash's "Active sessions = created − closed" panel had no counter to read for the "closed" half — session_closed events fed only the duration histogram, which double-counts across pod restarts and isn't a clean monotonic counter. Add chromeless_webrtc_session_closed_count, dimensioned by element_id (same shape as chromeless_webrtc_session_count). Bumps unconditionally on session_closed, independent of whether duration_ms is present — a closed session is closed regardless of whether the emitter knew its duration. Touch points: - webrtc_handler.go: declare counter, increment on session_closed - main.go: pre-register so the series exists on /metrics from startup, before the first session - webrtc_handler_test.go: extend SessionClosedDuration to assert the counter and tighten the histogram check via histSampleCount; add SessionClosedNoDurationStillCountsCounter for the no-duration path Pairs with the matching YAML declaration in chemistry/elements/tools/chromeless/.triform/observability.yaml (separate commit in the monorepo because chromeless and chemistry live in different repos). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>