selkies-ws-core

WebSocket streaming core: the page-side half of the WebSocket transport, started by selkies-core.js when the stored stream mode is websockets.

One socket at <route prefix>/api/websockets carries the whole session. Binary messages are typed by their first byte. From the server: 0x01 audio (Opus, with the RED redundancy layout documented on extractOpusFrames), 0x03 a JPEG stripe (u8 reserved, u16 frame id, u16 stripe Y, JPEG data), 0x04 an H.264 stripe or full frame (u8 keyframe, u16 frame id, u16 stripe Y, u16 width, u16 height, Annex-B data), and 0x05 a gzip-wrapped control text once the client advertised _gz,1. From the client: 0x02 microphone Opus, 0x06 webcam frames (startWebcamCapture), and 0x05 gzipped large text once the server echoed _gz,1. Text messages are control. The client sends SETTINGS,{json}, r,WxH,displayId, START_VIDEO, STOP_VIDEO, START_AUDIO, STOP_AUDIO, REQUEST_KEYFRAME, CLIENT_FRAME_ACK <id>, cr, REQUEST_CLIPBOARD, the chunked clipboard upload of lib/clipboard-worker-bridge.js, cmd,<command>, SET_NATIVE_CURSOR_RENDERING,<0|1> and the input verbs of lib/input.js. The server sends MODE websockets, AUTH_SUCCESS,{json}, ROLE_UPDATE,{json}, MK_ACCESS,<0|1>, VIDEO_STARTED, VIDEO_STOPPED, AUDIO_STARTED, AUDIO_STOPPED, AUDIO_DISABLED, MICROPHONE_DISABLED, WEBCAM_DISABLED, WEBCAM_KEYFRAME, PIPELINE_RESETTING <display>, DISPLAY_CONFIG_UPDATE,{json}, cursor,{json}, system,{json}, KILL <reason>, the clipboard family (clipboard,, clipboard_binary,, clipboard_start,, clipboard_data,, clipboard_finish, clipboard_reply,), and JSON objects typed server_settings, server_apps, pipeline_status, stream_resolution, system_stats, gpu_stats and network_stats.

Video is decoded with WebCodecs: a JPEG stripe through ImageDecoder, an H.264 stripe through a VideoDecoder per row offset, a controller's full frame in the video worker or through the row-0 stripe decoder, and a shared viewer's full frame through the main decoder. Decoded frames reach the screen through the first sink available: a track generator feeding a <video>, the worker's OffscreenCanvas, or the page canvas, with the striped modes composited on a back-buffer and blitted whole at frame boundaries. Audio is decoded in a worker and played through an AudioWorklet, the microphone is encoded to Opus in a worker, and the webcam is lib/webcam-capture.js.

Dashboards talk to the core over same-origin window messages. The core handles setVolume, setMute, setScaleLocally, setSynth, showVirtualKeyboard, setUseCssScaling, setAntiAliasing, setUseBrowserCursors, setManualResolution, resetResolutionToWindow, settings, getStats, clipboardUpdateFromUI, clipboardImageUpdate, pipelineStatusUpdate, pipelineControl, audioDeviceSelected, gamepadControl, requestFullscreen, command, touchinput:trackpad, touchinput:touch and sidebarVisibilityChanged, and posts pipelineStatusUpdate, sidebarButtonStatusUpdate, serverSettings, systemApps, stats (to the parent window), clientRoleUpdate, effectiveCursorState, trackpadModeUpdate, clipboardContentUpdate, the clipboard preview of lib/clipboard-sync.js, fileUpload, toggleDashboard and toggleTouchGamepad. The window globals it publishes for the dashboards and the tests are webrtcInput (the Input handler), fps, videoChunksReceived, system_stats, gpu_stats, network_stats, selkiesVideoStats, currentAudioBufferSize, currentAudioBufferDuration, currentAudioLevel, currentAudioUnderrunSamples, currentAudioWorkletDropped, currentAudioDropped, is_manual_resolution_mode, enable_resize, streamResolutionDiverged, isAudioInitializing, isFallingBack, isCleaningUp and applyTimestamp, plus one window[key] per server setting mirrored by sanitizeAndStoreSettings.

Settings are read from localStorage at init with fallbacks only and persist nothing, so a fresh profile keeps every key unset and server-pushed defaults stay re-pushable; only genuine user actions, and sanitizeAndStoreSettings for keys the user already overrode, write localStorage. Keys in PER_DISPLAY_SETTINGS carry a _display2 suffix on the secondary display.

Functions

audioTsNewer()

function audioTsNewer(a, b): boolean;

Defined in: selkies-ws-core.js:138

32-bit wrap-safe comparison of audio timestamps.

Parameters

Returns

boolean

True when a is strictly newer than b.

extractOpusFrames()

function extractOpusFrames(arrayBuffer): ArrayBuffer[];

Defined in: selkies-ws-core.js:158

Parses an audio message body into the ordered Opus frames to decode, using RED redundancy to recover frames the sender dropped under backpressure (pcmflux's delivery ring and the server's audio queue both drop-oldest, and a dropped frame rides along as redundancy in the next packet).

n_red == 0 is the plain path: [0x01, 0x00] + opus. n_red > 0 is [0x01, n_red, pts32] + n_red * (4-byte header) + 1-byte primary header + block data, redundant blocks oldest-first and then the primary; each block's timestamp is pts - tsOffset. Every frame is decoded at most once, in order: any block newer than the last one already played is taken, so a redundant copy fills the gap left by a dropped primary. The first RED packet anchors on its primary without replaying its redundancy.

Parameters

Returns

ArrayBuffer[]

Opus frames in decode order; empty for a malformed packet.

websockets()

function websockets(): void;

Defined in: selkies-ws-core.js:204

Starts the WebSocket streaming core in this page. Everything below is closure state of one session; the public surface is the window contract described in the module docblock.

Returns

void

applyWsMessageBudget()

function applyWsMessageBudget(bytes): void;

Defined in: selkies-ws-core.js:380

Adopts the server's advertised receive ceiling and resizes clipboard chunks to it.

Parameters

Returns

void

autoDeriveDpi()

function autoDeriveDpi(): number;

Defined in: selkies-ws-core.js:411

Derives the default scaling_dpi from the local display density so the remote desktop's UI matches the local one.

Returns

number

The nearest entry of DPI_STOPS, clamped at both ends.

applyEffectiveCursorSetting()

function applyEffectiveCursorSetting(): void;

Defined in: selkies-ws-core.js:432

Applies the cursor preference to the input handler, forced to browser cursors whenever a second display is involved, and posts the value in effect as effectiveCursorState so the dashboard toggle reflects the override rather than the preference alone.

Returns

void

setRealViewportHeight()

function setRealViewportHeight(): void;

Defined in: selkies-ws-core.js:445

Publishes the real viewport height as the --vh CSS unit (mobile browser chrome excluded).

Returns

void

safeSetItem()

function safeSetItem(key, value): void;

Defined in: selkies-ws-core.js:552

localStorage write that degrades a full or unavailable store to a warning instead of throwing QuotaExceededError into the caller.

Parameters

Returns

void

rememberSoftwareDecode()

function rememberSoftwareDecode(enabled): void;

Defined in: selkies-ws-core.js:592

Persists or clears the software-decode preference.

Parameters

Returns

void

decoderConfigFor()

function decoderConfigFor(config): VideoDecoderConfig;

Defined in: selkies-ws-core.js:611

Applies the acceleration preference to a VideoDecoder config; every decoder (main, stripe, SPS-driven, worker) goes through here so they agree. Unset, the UA default picks a hardware decoder when one works.

Parameters

Returns

VideoDecoderConfig

retireCrashCountWhenHealthy()

function retireCrashCountWhenHealthy(): void;

Defined in: selkies-ws-core.js:624

Clears the crash count once this session has proven healthy; runs on every metrics tick.

Returns

void

getIntParam()

function getIntParam(key, default_value): number;

Defined in: selkies-ws-core.js:708

Reads an integer setting from localStorage under the app prefix; keys in PER_DISPLAY_SETTINGS carry a _display2 suffix on the secondary display. The get/set helpers below share that key scheme.

Parameters

Returns

number

getFloatParam()

function getFloatParam(key, default_value): any;

Defined in: selkies-ws-core.js:718

Float variant of getIntParam, for range settings with fractional bounds.

Parameters

Returns

any

setIntParam()

function setIntParam(key, value): void;

Defined in: selkies-ws-core.js:729

Stores an integer setting; null removes the key.

Parameters

Returns

void

getBoolParam()

function getBoolParam(key, default_value): any;

Defined in: selkies-ws-core.js:742

Reads a boolean setting stored as 'true'/'false'.

Parameters

Returns

any

setBoolParam()

function setBoolParam(key, value): void;

Defined in: selkies-ws-core.js:755

Stores a boolean setting; null removes the key.

Parameters

Returns

void

getStringParam()

function getStringParam(key, default_value): any;

Defined in: selkies-ws-core.js:768

Reads a string setting.

Parameters

Returns

any

setStringParam()

function setStringParam(key, value): void;

Defined in: selkies-ws-core.js:778

Stores a string setting; null removes the key.

Parameters

Returns

void

sanitizeAndStoreSettings()

function sanitizeAndStoreSettings(serverSettings): object;

Defined in: selkies-ws-core.js:810

Reconciles the stored settings with the server's server_settings payload and mirrors the result onto window[key] for the runtime.

Only genuine user overrides are persisted: a server value with no stored override is applied to the runtime but never written to localStorage, so a later server-side change can still be re-pushed. A stored value outside the server's range or allowed list is dropped back to the server default, and a locked setting always wins at runtime without touching the user's key. The stored key can differ from the server's name (storageKeyForServerKey: HiDPI stores as useCssScaling), and range settings are read as floats so a fractional pick survives. An operator-overridden boolean with no stored pick is reported as a change so the runtime consumers apply it; a plain value (audio_channels) configures pipelines rather than preferences and is mirrored only.

Parameters

Returns

object

Settings whose effective value changed and must be applied by the caller.

enterFullscreen()

function enterFullscreen(gaming): void;

Defined in: selkies-ws-core.js:939

Enters fullscreen through the input handler, which owns both modes; before it exists only plain fullscreen is possible.

Parameters

Returns

void

playStream()

function playStream(): void;

Defined in: selkies-ws-core.js:952

Hides the start overlay and keeps the screen awake once the user starts the stream.

Returns

void

updateStatusDisplay()

function updateStatusDisplay(): void;

Defined in: selkies-ws-core.js:964

Shows loadingText, or else the sentence-cased status word (the internal value stays lower-case for comparisons).

Returns

void

alignResolution()

function alignResolution(num): number;

Defined in: selkies-ws-core.js:988

Floors a dimension to the encoder's alignment: 16 when force_aligned_resolution is set, 2 otherwise.

Parameters

Returns

number

createVideoTrackGenerator()

function createVideoTrackGenerator(): object;

Defined in: selkies-ws-core.js:1175

Creates the main-thread (Chromium) track generator; the worker-only VideoTrackGenerator is handled by the video worker instead.

Returns

object

track

track: MediaStreamTrack;

writable

writable: WritableStream;

ensureMstgWriter()

function ensureMstgWriter(): boolean;

Defined in: selkies-ws-core.js:1192

Lazily wires the <video> element to a fresh track generator; a writable that later errors or closes falls back to the canvas so the element never freezes.

Returns

boolean

True when the writer is ready.

teardownMstgWriter()

function teardownMstgWriter(): void;

Defined in: selkies-ws-core.js:1216

Closes the track generator writer and detaches its stream from the <video>.

Returns

void

presentFrameToVideo()

function presentFrameToVideo(frame): boolean;

Defined in: selkies-ws-core.js:1234

Presents a VideoFrame through the main-thread track generator, showing the <video> and hiding the canvas once it has rendered. Until then the frame is also painted on the canvas, since a fresh connection has nothing there yet and an empty <video> would show black. The resize handlers re-show the canvas with a fresh transform, so it is re-hidden every frame and its box re-mirrored whenever it changed; a backpressured sink drops the frame rather than building latency.

Parameters

Returns

boolean

True when consumed (the caller must not close it), false to fall back to the canvas.

ensureVideoWorker()

function ensureVideoWorker(): boolean;

Defined in: selkies-ws-core.js:1299

Lazily creates the video worker and completes its capability handshake. The worker self-probes VideoTrackGenerator on startup and reports vtg (it transferred a track back for <video>.srcObject) or canvas (it is handed an OffscreenCanvas to composite on). Its other messages: ack per consumed frame, error when the generator writable failed, presented once its canvas has real content, needKeyframe (no_key after a reconfigure, overload when the decode backlog forced a resync; throttled to one per 800 ms) and decoderError, after which chunks return to main-thread decode while the sink stays up for transferred frames.

Returns

boolean

True once a sink is wired; until then frames fall back to the main canvas.

deactivateVideoWorker()

function deactivateVideoWorker(): void;

Defined in: selkies-ws-core.js:1372

Terminates the video worker and returns presentation to the main canvas. The worker decoder config is forgotten so a recreated worker is configured afresh, and a transferred OffscreenCanvas, which can never be transferred again, is replaced by a fresh <canvas> element.

Returns

void

activateWorkerSinkDisplay()

function activateWorkerSinkDisplay(): boolean;

Defined in: selkies-ws-core.js:1406

Shows the active worker sink (<video> for VTG, the worker canvas otherwise), hides the main canvas once the sink has rendered (requestVideoFrameCallback for VTG, the worker's one-time presented message for canvas mode), and mirrors the canvas box onto the sink whenever it changed.

Returns

boolean

False while no sink target exists yet.

presentFrameToWorker()

function presentFrameToWorker(frame): boolean;

Defined in: selkies-ws-core.js:1448

Transfers a main-thread-decoded VideoFrame to the worker sink, the fallback while the worker decoder warms up. A frame past the in-flight cap is dropped rather than queued behind a stalled decoder, and a frame that postMessage detached or closed is reported consumed so the caller never reuses it.

Parameters

Returns

boolean

True when consumed (the caller must not close it).

logWorkerDecoderConfig()

function logWorkerDecoderConfig(
   codec, 
   w, 
   h
): void;

Defined in: selkies-ws-core.js:1474

Rate-limited log of worker decoder reconfigures. A healthy stream reconfigures about once per session (join, resolution change), so a storm with flipping codec strings is the diagnostic; at most one line per interval, with a suppressed count so repeats stay visible.

Parameters

Returns

void

feedWorkerDecoder()

function feedWorkerDecoder(
   isKey, 
   dataBuf, 
   w, 
   h, 
   codec
): boolean;

Defined in: selkies-ws-core.js:1497

Forwards an encoded full-frame H.264 chunk to the worker's own decoder, reconfiguring it when the codec or coded dimensions change and requesting the keyframe WebCodecs needs after a configure.

Parameters

Returns

boolean

True when handled there, false to fall back to main-thread decode.

deactivateMstg()

function deactivateMstg(): void;

Defined in: selkies-ws-core.js:1514

Returns presentation from the main-thread track generator to the canvas; idempotent.

Returns

void

getDynamicH264Codec()

function getDynamicH264Codec(
   width, 
   height, 
   is444, 
   fps
): string;

Defined in: selkies-ws-core.js:1535

Pre-stream guess of the H.264 codec string. Decoder creation re-derives the exact codec from the first keyframe's SPS (codecFromKeyframe), and outside Chromium only a conservative baseline is guessed because Safari rejects a stream whose real profile or level exceeds the configured one.

Parameters

Returns

string

parseAvcCodecFromAnnexB()

function parseAvcCodecFromAnnexB(bytes): string;

Defined in: selkies-ws-core.js:1565

Reads the codec string from a keyframe's SPS: scans the Annex-B payload for the first SPS NAL and builds avc1.PPCCLL from it.

Parameters

Returns

string

null when no SPS is found, so the caller falls back to the heuristic guess.

codecFromKeyframe()

function codecFromKeyframe(keyframeBytes, fallback): string;

Defined in: selkies-ws-core.js:1610

The H.264 codec string a keyframe's in-band SPS declares. Every engine uses this: Safari's VideoDecoder errors when the configured profile or level is lower than the stream's real one, and the parsed value always matches the bitstream.

Parameters

Returns

string

maybeReconfigureMainDecoderFromSps()

function maybeReconfigureMainDecoderFromSps(keyframeBytes): boolean;

Defined in: selkies-ws-core.js:1628

Chromium only: reconfigures the main decoder when a keyframe's SPS profile or level differs from the current config. The caller decodes that keyframe right after, as WebCodecs requires after a configure.

Parameters

Returns

boolean

True when reconfigured.

updateCanvasImageRendering()

function updateCanvasImageRendering(): void;

Defined in: selkies-ws-core.js:1658

Picks the canvas image-rendering: pixelated for a 1:1 display or when anti-aliasing is off, smoothed whenever the picture is scaled (manual resolution, high-DPR CSS scaling, shared mode). Part of cssText, so the box is re-mirrored to the active sink.

Returns

void

injectCSS()

function injectCSS(): void;

Defined in: selkies-ws-core.js:1685

Installs the page's base stylesheet: the video container, its sinks, the overlay input and the start button.

Returns

void

sendFullSettingsUpdateToServer()

function sendFullSettingsUpdateToServer(reason): void;

Defined in: selkies-ws-core.js:1789

Sends the full SETTINGS,{json} payload; never from a shared viewer.

Parameters

Returns

void

getCurrentSettingsPayload()

function getCurrentSettingsPayload(): object;

Defined in: selkies-ws-core.js:1813

Builds the SETTINGS payload. Only keys with a stored (user-set) value are included, so the fallbacks here never override server-configured defaults for an untouched setting; scaling_dpi is the exception, being client-authoritative (the derived default or the dashboard's pick, sent live so it reaches the running server; the desktop DPI is independent of the resolution). The payload also carries the keyboard layout, the client geometry or manual resolution, the display identity and the audio-RED capability that makes the server enable Opus redundancy.

Returns

object

updateToggleButtonAppearance()

function updateToggleButtonAppearance(buttonElement, isActive): void;

Defined in: selkies-ws-core.js:1881

Labels a pipeline toggle button with its name and ON/OFF state.

Parameters

Returns

void

sendResolutionToServer()

function sendResolutionToServer(width, height): void;

Defined in: selkies-ws-core.js:1905

Sends r,WxH,displayId with the aligned, DPR-scaled and 4080-capped stream resolution; blocked in shared mode, where the viewer follows the controller.

Parameters

Returns

void

syncSinkToCanvasStyle()

function syncSinkToCanvasStyle(): void;

Defined in: selkies-ws-core.js:1945

Mirrors the canvas box onto the active video sink right after a canvas-style writer rewrote it. The present paths do the same, but only when frames flow: on a static remote a resize would otherwise leave the stale canvas covering the live sink until the next decoded frame. A sink that has proven it renders gets the geometry and hides the canvas immediately; during warm-up nothing changes. Covers all three sinks (main-thread and worker generators drive the <video>, the OffscreenCanvas worker drives videoWorkerCanvas).

Returns

void

applyManualCanvasStyle()

function applyManualCanvasStyle(
   targetWidth, 
   targetHeight, 
   scaleToFit
): void;

Defined in: selkies-ws-core.js:1977

Sizes the canvas for a manual resolution: the backing buffer at the target size (DPR-scaled unless CSS scaling, shared mode or manual mode pin it to 1), the CSS box either scaled to fit the container or exact and centered. The overlay input follows the box and the input handler is told to resize. The per-row JPEG stripe ids, keyed by row offset, are reset because a geometry change invalidates them.

Parameters

Returns

void

resetCanvasStyle()

function resetCanvasStyle(streamWidth, streamHeight): void;

Defined in: selkies-ws-core.js:2071

Sizes the canvas for the stream's own resolution: the backing buffer at the DPR-scaled size and the CSS box at the stream size, centered in the container, with the overlay input following. The per-row JPEG stripe ids are reset as in applyManualCanvasStyle.

Parameters

Returns

void

enableAutoResize()

function enableAutoResize(): void;

Defined in: selkies-ws-core.js:2143

Switches the window resize listener to the automatic (stream follows the viewport) handler and applies it once.

Returns

void

directManualLocalScalingHandler()

function directManualLocalScalingHandler(): void;

Defined in: selkies-ws-core.js:2164

Resize listener for manual resolution: restyles the canvas box without touching the stream size.

Returns

void

disableAutoResize()

function disableAutoResize(): void;

Defined in: selkies-ws-core.js:2171

Switches the window resize listener to the manual-resolution handler and applies it once.

Returns

void

updateUIForSharedMode()

function updateUIForSharedMode(): void;

Defined in: selkies-ws-core.js:2186

Marks the container as a shared viewer (default cursor) and disables file upload.

Returns

void

initializeUI()

function initializeUI(): void;

Defined in: selkies-ws-core.js:2212

Builds the page: the video container with its status bar, overlay input, canvas, the sink elements the engine can use, and the start button, plus the hidden file input and keyboard-assist input on the body. Chooses the video sink (see supportsWindowMSTG), logging it once since a canvas fallback explains a session's CPU cost, and starts the worker handshake early so its decoder is ready before the first frame, then sizes the canvas for shared, manual or automatic resolution.

Returns

void

clearAllVncStripeDecoders()

function clearAllVncStripeDecoders(): void;

Defined in: selkies-ws-core.js:2367

Closes every stripe decoder and forgets their soft-error counts.

Returns

void

handleStripeDecodeError()

function handleStripeDecodeError(e, vncStripeYStart): void;

Defined in: selkies-ws-core.js:2399

Routes a stripe decoder error. Safari's main-thread VideoDecoder rejects streams its worker decoder plays fine, so while the worker path is elected for a full-frame encoder and still healthy the error is handoff noise: the stripe decoder is rebuilt on the next keyframe instead of escalating into the fallback ladder, which closes the socket and reloads. A burst that keeps repeating within the window still reaches the ladder.

Parameters

Returns

void

stripeDecodesDrained()

function stripeDecodesDrained(): boolean;

Defined in: selkies-ws-core.js:2423

Whether every stripe chunk handed to a stripe decoder has come back out.

Returns

boolean

processPendingChunksForStripe()

function processPendingChunksForStripe(stripe_y_start): void;

Defined in: selkies-ws-core.js:2437

Decodes the chunks a stripe queued while its decoder was still configuring.

Parameters

Returns

void

ensureStripeBackBuffer()

function ensureStripeBackBuffer(): CanvasRenderingContext2D;

Defined in: selkies-ws-core.js:2481

Creates the back-buffer, resized to the canvas.

Returns

CanvasRenderingContext2D

deactivateStripeWorker()

function deactivateStripeWorker(): void;

Defined in: selkies-ws-core.js:2530

Terminates the stripe compositor worker.

Returns

void

ensureStripeWorker()

function ensureStripeWorker(): boolean;

Defined in: selkies-ws-core.js:2540

Creates the stripe compositor worker; idempotent.

Returns

boolean

False when it cannot run, so the caller composites on the main-thread back-buffer instead.

stripeCompositeBegin()

function stripeCompositeBegin(): boolean;

Defined in: selkies-ws-core.js:2573

Starts a stripe compositing cycle on the worker (its back-buffer resized to the canvas) or the main-thread back-buffer.

Returns

boolean

False while the canvas has no size yet.

stripeCompositeDraw()

function stripeCompositeDraw(stripe, yPos): void;

Defined in: selkies-ws-core.js:2593

Composites one decoded stripe at its row offset; always consumes the stripe.

Parameters

Returns

void

stripeCompositePresent()

function stripeCompositePresent(): void;

Defined in: selkies-ws-core.js:2608

Presents the composited frame: the worker commits an ImageBitmap, the main thread blits its back-buffer. Counted as the striped modes' displayed frame, which is what window.fps reports for them.

Returns

void

clearStartVideoWatchdog()

function clearStartVideoWatchdog(): void;

Defined in: selkies-ws-core.js:2628

Disarms the START_VIDEO watchdog.

Returns

void

armVisibleFrameProbe()

function armVisibleFrameProbe(): void;

Defined in: selkies-ws-core.js:2643

Proves a stream the returning tab believes is running: a reconnect or reload while hidden can leave the server holding this display stopped, and a screen with no damage since sends nothing to repaint the cleared canvas either way. A keyframe request answers the second case; if nothing arrives within VISIBLE_FRAME_PROBE_MS, the stream really is stopped and is restarted.

Returns

void

onStartVideoWatchdogTimeout()

function onStartVideoWatchdogTimeout(): void;

Defined in: selkies-ws-core.js:2664

Resends START_VIDEO while no video arrives, up to the attempt limit, then forces a reconnect through the onclose path. Stands down when the tab is hidden again (the visibilitychange path owns that state, and a shared viewer's resume can be rate-limited by the server) or the socket is not open (the reconnect logic owns recovery).

Returns

void

armStartVideoWatchdog()

function armStartVideoWatchdog(): void;

Defined in: selkies-ws-core.js:2681

Arms the START_VIDEO watchdog with a fresh attempt count for this visibility cycle.

Returns

void

clearSharedStallWatchdog()

function clearSharedStallWatchdog(): void;

Defined in: selkies-ws-core.js:2688

Disarms the shared-mode stall watchdog.

Returns

void

armSharedStallWatchdog()

function armSharedStallWatchdog(): void;

Defined in: selkies-ws-core.js:2702

Arms the shared-mode stall watchdog (see sharedStallWatchdogId). While the viewer is hidden, paused or not yet ready it expects no chunks, so the clock is kept fresh and the watchdog cannot fire the instant those states end.

Returns

void

handleDecodedVncStripeFrame()

function handleDecodedVncStripeFrame(yPos, frame): void;

Defined in: selkies-ws-core.js:2738

Output callback of the stripe decoders. A full-frame h264enc frame (the single decoder at row 0) is presented the instant it decodes, for the lowest glass-to-glass latency, superseding anything still queued: through the main-thread track generator, else the worker sink, else the canvas. h264enc-striped composites partial-height stripes and drains through the rAF queue instead.

Parameters

Returns

void

requestWakeLock()

function requestWakeLock(): Promise<void>;

Defined in: selkies-ws-core.js:2776

Requests a screen wake lock so the device does not sleep mid-session.

Returns

Promise<void>

releaseWakeLock()

function releaseWakeLock(): Promise<void>;

Defined in: selkies-ws-core.js:2795

Releases the screen wake lock if one is held.

Returns

Promise<void>

debounce()

function debounce(func, delay): Function;

Defined in: selkies-ws-core.js:2808

Trailing-edge debounce.

Parameters

Returns

Function

startStream()

function startStream(): void;

Defined in: selkies-ws-core.js:2819

Marks the stream as started and hides the status bar and start button.

Returns

void

initializeInput()

function initializeInput(): void;

Defined in: selkies-ws-core.js:2836

Creates the Input handler on the overlay input once the server has assigned the client's role and slot, wires its dashboard chords to the dashboards (toggleDashboard, toggleTouchGamepad window messages; fullscreen, Ctrl+Shift+F, stays inside Input), publishes it as window.webrtcInput, installs the automatic or manual resize handling, and attaches file drop and mobile keyboard assistance. A viewer role keeps the gamepad but has its pointer and keyboard context detached.

Returns

void

handleResizeUI()

function handleResizeUI(): void;

Defined in: selkies-ws-core.js:2950

Automatic resize: sends the aligned, capped viewport size and restyles the canvas. Skipped in shared and manual mode, and on the primary when enable_resize=false pins its resolution server-side (a secondary's resize is its layout bring-up and stays allowed, matching the server). Stripe decoders are closed first, since rows that vanish on shrink would keep a live decoder nothing feeds, and the divergence flag is reset for stream_resolution to re-flag.

Returns

void

watchDevicePixelRatio()

function watchDevicePixelRatio(): void;

Defined in: selkies-ws-core.js:3008

Re-runs the automatic resize when devicePixelRatio changes. The stream resolution is logical size times DPR, but a DPR change alone (a window dragged to a monitor of another density, an OS scaling change) fires no resize event. matchMedia resolution queries are one-shot at a given dppx, so the query is re-armed after each change.

Returns

void

applyOutputDevice()

function applyOutputDevice(): Promise<void>;

Defined in: selkies-ws-core.js:3093

Routes playback to the preferred output device. Audio plays out of the AudioContext (no media element carries it), so this needs AudioContext.setSinkId; where it is missing, or the context is not running yet, playback stays on the default device.

Returns

Promise<void>

postSidebarButtonUpdate()

function postSidebarButtonUpdate(): void;

Defined in: selkies-ws-core.js:3122

Posts sidebarButtonStatusUpdate with the state of every pipeline toggle to the dashboards.

Returns

void

receiveMessage()

function receiveMessage(event): void;

Defined in: selkies-ws-core.js:3149

Handles the window messages the dashboards post to the core (same origin only): volume and mute, local scaling, the virtual keyboard, CSS scaling, anti-aliasing, cursor rendering, manual resolution and its reset, pipeline and gamepad control, audio device selection, stream commands, clipboard pushes, and the getStats and settings requests. See the module docblock for the full vocabulary. A setUseCssScaling with persist: false is server-authored and leaves the user's stored key untouched; the resolution paths honour enable_resize=false, which pins the primary's resolution server-side while a secondary stays resizable; and clipboardImageUpdate reports every skip so a dead click never reads as a bug.

Parameters

Returns

void

notifyClipboardImageSkip()

function notifyClipboardImageSkip(reason, code): void;

Defined in: selkies-ws-core.js:3625

Tells the dashboard why a clipboard-image upload was skipped, in the fileUpload warning channel transfer warnings already use.

Parameters

Returns

void

notifyClipboardImageWriteFailed()

function notifyClipboardImageWriteFailed(error): void;

Defined in: selkies-ws-core.js:3640

Tells the dashboard that a server image never reached the local clipboard.

The panel shows nothing of an inbound image but this notice, so a write the browser refuses would otherwise read as the feature not working at all.

Parameters

Returns

void

sendClipboardData()

function sendClipboardData(
   data, 
   mimeType?, 
   onSkip?
): Promise<void>;

Defined in: selkies-ws-core.js:3663

Sends local clipboard content to the server as a chunked transfer (lib/clipboard-worker-bridge.js, the same wire protocol and worker offload as the WebRTC core), gated on the clipboard-in setting and the change-only sync. A bufferedAmount backpressure gate keeps a burst from starving uploads and input on the same socket; only a completed transfer marks the content synced, so an aborted one stays re-sendable.

Parameters

Returns

Promise<void>

handleSettingsMessage()

function handleSettingsMessage(settings, fromServer?): void;

Defined in: selkies-ws-core.js:3729

Applies a settings payload to the runtime and pushes the result to the server. A dashboard-authored payload is persisted; a server-authored one (the locked and overridden values replayed on every connect) is applied but never written to the user's own keys, where it would outlive the lock and masquerade as their pick. An encoder switch tears the decoders down and asks for a keyframe once the server's restart settles, in case its restart IDR beat the reset over the wire.

Parameters

Returns

void

fetchLatestRCvalue()

function fetchLatestRCvalue(newMode): void;

Defined in: selkies-ws-core.js:3913

Re-reads the stored value the new rate-control mode governs (bitrate for cbr, CRF for crf).

Parameters

Returns

void

sendStatsMessage()

function sendStatsMessage(): void;

Defined in: selkies-ws-core.js:3922

Posts a stats snapshot (server, network, client fps, buffers, pipeline state) to the parent window.

Returns

void

initWebsockets()

function initWebsockets(): void;

Defined in: selkies-ws-core.js:3953

Runs the connection: pre-flight checks and the page build, the clipboard gesture wiring, the tab visibility handling, the paint loop, audio setup, and the socket with its message dispatch, reconnect and fallback paths. Called once the document has loaded.

Returns

void

initializeDecoder()

function initializeDecoder(): Promise<boolean>;

Defined in: selkies-ws-core.js:3960

Configures the main VideoDecoder (shared full-frame viewing) for the current target resolution, decoding a keyframe stashed while it was configuring and requesting a fresh IDR when deltas were lost meanwhile.

Returns

Promise<boolean>

False when configuration failed and the fallback ladder ran.

clearVideoCanvasVisually()

function clearVideoCanvasVisually(): void;

Defined in: selkies-ws-core.js:4088

Clears the canvas so a paused stream does not show a stale frame.

Returns

void

decodeAndQueueJpegStripe()

function decodeAndQueueJpegStripe(
   startY, 
   jpegData, 
   frameId
): Promise<void>;

Defined in: selkies-ws-core.js:4196

Decodes a JPEG stripe and queues it for the paint loop: ImageDecoder (WebCodecs) where the context is secure, createImageBitmap elsewhere; both yield an image the render and cleanup paths handle alike.

Parameters

Returns

Promise<void>

handleDecodedFrame()

function handleDecodedFrame(frame): void;

Defined in: selkies-ws-core.js:4225

Output callback of the main VideoDecoder, which only shared full-frame viewing feeds (controllers decode through the JPEG and per-stripe paths), so a frame decoded outside shared mode is closed. The frame is presented through the first sink that takes it, else queued for the paint loop.

Parameters

Returns

void

schedulePaintVideoFrame()

function schedulePaintVideoFrame(): void;

Defined in: selkies-ws-core.js:4272

Schedules the next paint tick on one rAF chain; starting the loop again (a reconnect) must never create a second permanent chain.

Returns

void

paintVideoFrame()

function paintVideoFrame(): void;

Defined in: selkies-ws-core.js:4295

The per-rAF paint tick. Full-frame h264enc presents only the newest queued frame; the striped modes composite their stripes and present the whole frame as soon as its last row lands (the server emits a frame's stripes in ascending order, so the last row proves it complete) or the socket and the decoders go quiet (the stripe clock), falling back to presenting at frame-id boundaries while stripes still flow; JPEG skips stripes that decoded out of order; the shared main decoder path keeps the adaptive jitter cushion, closing everything older than it in one tick because draining one per rAF would let a burst back up the decoder's bounded output pool. Leaving a full-frame mode tears both video sinks down symmetrically, or a worker canvas would stay shown over the striped content.

Returns

void

initializeAudio()

function initializeAudio(): Promise<void>;

Defined in: selkies-ws-core.js:4501

Builds the playback pipeline on the primary display: a 48 kHz AudioContext, the AudioWorklet that queues decoded PCM (drop-oldest at its cap, zero-fill on underrun, reporting depth and concealment counters on request), a gain node for volume, and the Opus decode worker, widened to the surround layout when the server streams one (best effort: the browser still downmixes to the device's layout).

Returns

Promise<void>

initializeDecoderAudio()

function initializeDecoderAudio(): Promise<void>;

Defined in: selkies-ws-core.js:4754

Reinitializes the audio decoder in its worker, building the whole pipeline first if it is missing.

Returns

Promise<void>

sendBackpressureAck()

function sendBackpressureAck(): void;

Defined in: selkies-ws-core.js:4800

Acks the newest video frame the client is done with, so the server can pace its sends against what this client actually keeps up with. The striped modes composite on the page and ack what reached the screen; a client whose rendering falls behind is then throttled instead of being sent frames it will never show. Full-frame h264enc presents through sinks the page cannot observe, so there the newest received id is the best the client knows.

Returns

void

sendClientMetrics()

function sendClientMetrics(): void;

Defined in: selkies-ws-core.js:4823

Metrics tick: refreshes the audio buffer depth the backpressure gates read and publishes window.fps — composites presented per second in the striped modes, and the wire's frame ids per second for full-frame h264enc, whose sinks present outside the page — independent of whether a dashboard is open.

Returns

void

reloadPossiblyFlippingMode()

function reloadPossiblyFlippingMode(): Promise<void>;

Defined in: selkies-ws-core.js:6178

Reloads the page, first switching the stored stream mode to WebRTC when the server is serving that transport: a plain GET on the transport endpoint answers 409 exactly then. One attempt per connect cycle, and only if this session never connected, so a client whose stored mode disagrees with the server converges instead of loop-reloading.

Returns

Promise<void>

cleanupVideoBuffer()

function cleanupVideoBuffer(): void;

Defined in: selkies-ws-core.js:6204

Closes every buffered VideoFrame and returns presentation to the canvas.

Returns

void

cleanupJpegStripeQueue()

function cleanupJpegStripeQueue(): void;

Defined in: selkies-ws-core.js:6224

Closes every queued JPEG stripe image and resets the frame-boundary blit latch, which stale would blit the previous mode's back-buffer once.

Returns

void

clearDecodedStripesQueue()

function clearDecodedStripesQueue(): void;

Defined in: selkies-ws-core.js:6244

Closes every decoded stripe awaiting the paint loop.

Returns

void

getAudioChannelCount()

function getAudioChannelCount(): number;

Defined in: selkies-ws-core.js:6271

The server's audio_channels setting, limited to the layouts the decoder handles.

Returns

number

1, 2, 6 or 8; 2 when unset or unknown.

buildMultiopusDescription()

function buildMultiopusDescription(channels): ArrayBuffer;

Defined in: selkies-ws-core.js:6284

Builds the OpusHead description for a surround layout: magic, version 1, channel count, a zero pre-skip (a live stream has nothing to trim), the 48 kHz input rate, zero output gain, mapping family 1 (multistream), then the stream and coupled counts and the channel mapping table.

Parameters

Returns

ArrayBuffer

null for a layout the client does not know.

startMicrophoneCapture()

function startMicrophoneCapture(): Promise<void>;

Defined in: selkies-ws-core.js:6501

Starts the microphone uplink: getUserMedia at 24 kHz mono with processing on, the capture worklet, and the encode worker whose Opus frames go straight onto the socket, so only encoded bytes cross the wire and the server decodes in pcmflux. Blocked for shared viewers.

Returns

Promise<void>

stopMicrophoneCapture()

function stopMicrophoneCapture(): void;

Defined in: selkies-ws-core.js:6584

Stops the microphone uplink and releases the stream, worklet, worker and context.

Returns

void

startWebcamCapture()

function startWebcamCapture(): void;

Defined in: selkies-ws-core.js:6646

Starts the webcam uplink (lib/webcam-capture.js): each encoded frame is sent as one binary [0x06][codec][flags][payload] message that the server's virtual camera decodes for the V4L2 device. Flags bit 0 marks a keyframe; bits 1 to 2 carry the frame's clockwise rotation in quarter turns and bit 3 a horizontal flip applied after it, the orientation metadata the encoder never bakes into the bitstream. Frames are dropped rather than queued while the socket is backed up (WEBCAM_QUEUE_MS). Blocked for shared viewers.

Returns

void

stopWebcamCapture()

function stopWebcamCapture(): void;

Defined in: selkies-ws-core.js:6684

Stops the webcam uplink.

Returns

void

cleanup()

function cleanup(): void;

Defined in: selkies-ws-core.js:6696

Tears everything down on unload: timers, capture, socket, audio, decoders and buffers, then resets the UI state.

Returns

void

performServerInitiatedVideoReset()

function performServerInitiatedVideoReset(reason?): void;

Defined in: selkies-ws-core.js:6766

Resets the video state after the server's PIPELINE_RESETTING: the shared keyframe gate, the frame id, every buffer and the decoders of the current mode, clearing the canvas for the modes that repaint it whole.

Parameters

Returns

void

requestKeyframe()

function requestKeyframe(): void;

Defined in: selkies-ws-core.js:6814

Asks the server for an IDR when a decoder waits for its first keyframe (a recreated stripe decoder, a shared viewer's closed gate). The GOP is infinite, so this is the only recovery path and shared viewers request too; debounced here, harder for shared viewers, and rate-limited server-side.

Returns

void

restartDecodersForAcceleration()

function restartDecodersForAcceleration(): void;

Defined in: selkies-ws-core.js:6830

Rebuilds every video decoder so a changed acceleration preference takes hold, then resyncs from a fresh IDR. The main decoder is only fed in shared mode; the stripe and worker decoders are rebuilt from the next keyframe by the paths that own them, and a worker decoder disqualified by the same broken path gets its turn back.

Returns

void

initiateFallback()

function initiateFallback(error, context): void;

Defined in: selkies-ws-core.js:6863

The decoder fallback ladder. A codec reclaimed by the browser is a soft error left to the tab-focus re-init. A decoder that accepted its config and then failed is the signature of a broken hardware path, so the first hard error retries the same encoder on software decode; errors from the decoders that switch replaced are absorbed for a settle period. A failure after that forgets the preference, counts a crash, and reloads: a shared viewer just resyncs, a controller resets its settings to safe defaults, stepping the encoder down to h264enc and, at three crashes, to jpeg. jpeg mode runs no VideoDecoder, so an error there is handover noise from a stream the server has yet to stop and never escalates.

Parameters

Returns

void

runPreflightChecks()

function runPreflightChecks(): boolean;

Defined in: selkies-ws-core.js:6938

Builds the UI and checks the engine: a secure context is required; without WebCodecs the stream is pinned to the jpeg encoder, which decodes through createImageBitmap, and a server-locked H.264 encoder is reported when it arrives rather than decoded into a crash loop.

Returns

boolean

False when the page cannot run.

pinJpegEncoder()

function pinJpegEncoder(): void;

Defined in: selkies-ws-core.js:6960

Pins the jpeg encoder, the fallback ladder's last rung.

Returns

void

showUndecodableEncoderNotice()

function showUndecodableEncoderNotice(encoderName): void;

Defined in: selkies-ws-core.js:6970

Reports a server-locked encoder this engine cannot decode instead of showing nothing.

Parameters

Returns

void

clearUndecodableEncoderNotice()

function clearUndecodableEncoderNotice(): void;

Defined in: selkies-ws-core.js:6980

Hides the undecodable-encoder notice once a decodable encoder is in use.

Returns

void

References

websockets

Re-exports websockets