OneVision Encoder

Input

Video

Pipeline

Visualization mode

Selection — kept patches in color, others fade to gray-white Heatmap — full-frame JET overlay (blue=low, red=high) Side-by-side — selection on the left, heatmap on the right

Sampled frames

4 64

Total patches budget (whole video)

The single budget shared across the whole video. The codec saliency picks these patches GLOBALLY — high-energy frames may contribute many, low-energy frames may contribute zero.

64 8192

Patch size (px)

14 16 28

GOP (group of pictures)

Splits sampled frames into groups; the patch budget is allocated equally across groups, top-K within each. Dynamic mode mirrors codec_tools' readiness grouping (equal-energy groups).

Global — one budget across the whole video GOP = 4 — fixed 4-frame groups GOP = 8 — fixed 8-frame groups GOP = 16 — fixed 16-frame groups Dynamic — adaptive groups by saliency energy

Quick presets

Detail: gradient · pct=99 · heatmap | Motion: frame_diff · pct=95 · heatmap | Balanced: combined · log · pct=96 · sbs

Patch selection visualization

Video

Codec selection vs uniform baseline

Same total patch budget on both sides. Left: where the codec saliency actually placed patches (often spiky, concentrated on important moments). Right: how a naive uniform frame sampler would spend the same budget — spread evenly in time, every patch in the chosen frames kept.

Plot

Packed canvas

Raw JSON

Full reproducible record of this run (params, frame ids, group spans). Collapsed by default — click to expand.

Built with Gradio logo