betaFeedback

WorksComputer Science

StreamEmbed: Real-Time Indexing from Pixels for Screenshot Retrieval

Abstract

Screen history can serve as searchable visual memory for people and agents, but unlike an offline corpus, it must be encoded and indexed as each screenshot arrives. OCR-based indexing is fast but loses visual information; visual encoding takes over two seconds. Yet screens change incrementally: most pixels repeat within and across screenshots. This raises a question: do visual encoders need to recompute their representations from scratch for every new screenshot? We present StreamEmbed, a system for incremental visual embedding that exploits this redundancy. Guided solely by pixel differences, it combines spatial token pruning, which skips blocks that repeat a neighbor, with temporal KV reuse, which reuses cached keys and values for blocks unchanged across screenshots. A small learned adapter compensates for pruning and reuse to preserve fidelity to full-screenshot embeddings. On ScreenMemEval, our benchmark of 10,426 questions over 184 hours of screen histories from two humans and a computer-use agent, StreamEmbed sets a new accuracy-latency Pareto frontier on an M2 Pro GPU: it encodes up to 3.7x faster than full-screenshot encoding without sacrificing accuracy and improves accuracy over OCR-based indexing by up to 6.3 Recall@10 points at matched latency. A memory-efficient vector index stores a simulated year of screenshots in 0.13 GB of RAM. Together, these results make real-time indexing of screen history in its native visual form practical, laying a foundation for fully local visual assistants.

Introduction

Much of daily computer work passes through laptop screens: reading papers, browsing products, and messaging colleagues. Yet people remember few of the details and may later ask: “Which paper had the pink and blue Pareto frontier curves I saw last week?” A searchable screen history could help people recover these details, and also provide persistent memory for locally running agents ([50] (Wu et al. 2025)). Because screenshots capture every application without specialized integration, they offer a unified interface across text and visual contents ((Hu & Lee, 2020)). Answering questions about past activity then becomes a retrieval problem over a continuously growing screenshot history.

The key challenge is maintaining this visual memory continuously without exhausting system resources. Unlike a fixed corpus that is indexed offline ([31]), this setting requires real-time indexing: encoding and indexing each screenshot as it arrives, ideally locally to keep private content on device, while consuming only a small fraction of the laptop’s capacity.

Converting each screenshot into OCR text, as existing open-source tools do ([49, 44]), is fast but loses what only pixels capture ([64]). Direct visual embedding preserves this information but takes 2.3 s per screenshot on the GPU. At one screenshot every 5 s, this uses the GPU for 28 minutes per hour of the user’s work, and at one every 2 s, it cannot keep pace. Even commercial systems, such as Microsoft Recall ((Microsoft)), require dedicated hardware like NPUs and at least 16 GB of RAM. Thus, the central question is: can we retain visual retrieval quality while keeping continuous encoding lightweight enough to run silently in the background?

Screen histories, however, offer a particular opportunity: they change incrementally and are largely redundant (Figure 2). Background elements often repeat within and across screenshots, and typing or moving the cursor changes only a few regions, leaving the rest untouched. This raises a question: do we need to compute representations from scratch for every screenshot?

Figure 2. Screenshots repeat most of their blocks, unlike natural video. (a) Blocks of st+1s_{t+1} that repeat their left or upper neighbor. (b) Blocks of st+1s_{t+1} unchanged from sts_t. (c) (a) and (b) together split st+1s_{t+1} into repeated, unchanged, and changed blocks. (d) Share of repeated, unchanged, and changed blocks per source ([8, 12] (Lu et al. 2024)) (Wiki.: Wikimedia Commons).

We propose STREAMEMBED, which encodes screenshots as a stream rather than as isolated images, exploiting this redundancy. Within a screenshot, it drops pixel blocks that merely duplicate a neighboring block (spatial pruning), cutting the computation by 33–41%. Across screenshots, it recomputes only the changed blocks and reuses cached keys and values for the rest in both the vision encoder and the language model (temporal reuse), cutting the remaining computation by a further 34–57%. These two techniques are largely complementary, and combining them further reduces the computation per screenshot by 57–71%. A small adapter merged into the weights compensates for pruning and reuse, keeping the resulting embedding close to that of the full screenshot.

Evaluating STREAMEMBED presents another challenge, since no suitable benchmark is available. Such a benchmark needs a screenshot history, questions about it, and the screenshots that answer each question, but existing benchmarks lack a screenshot history (Wu et al., 2026), questions about the past (Xie et al., [69]), or relevance labels and public data (Wang & Chen, [62]). We therefore build SCREENMEMEVAL (SME), consisting of 10,426 questions drawn from three screenshot histories, each labeled with the screenshots that answer it. Two histories come from human workers on Windows in a public dataset of computer-use recordings (Howland et al., [20]): SME-H1 spans 117 days with 88 hours of screen time, and SME-H2 spans 57 days with 18 hours. The third, SME-Agent, spans 43 days with 78 hours, recorded on Ubuntu from a computer-use agent (the frontier model GPT-6.1 Sol (OpenAI, [47])) performing the tasks of myPCBench (Jang et al., [25]).

On SME, STREAMEMBED on Qwen3-VL-Embedding-2B (Li et al., 2026) sets a new accuracy–latency Pareto frontier (Figure 1). Against baselines on the same backbone, fine-tuned on the same data, it is up to 6.3 points more accurate than an OCR-based index at similar latency, and up to 3.7× faster than a screenshot index while retaining over 98%

Figure 1. STREAMEMBED moves the accuracy–latency frontier. Recall@10 against indexing latency per screenshot (log scale) on an M2 Pro GPU, on the three histories of SCREENMEMEVAL. STREAMEMBED runs on Qwen3-VL-Embedding-2B at several input resolutions and on SigLIP 2. STREAMEMBED and the +LoRA baselines are fine-tuned on the training split of SME-H1, and all results are on held-out test sets (§ 5.3). We run baselines as released, while STREAMEMBED uses a linear patch embedding for Qwen-VL (§ 6.1).

of its accuracy. To cut latency, one can lower the input resolution or use a smaller encoder. When we apply these to both STREAMEMBED and the baselines, STREAMEMBED is more accurate at every latency, by up to 1.4 points (§ 6.4). We also show that binary codes of the embeddings reduce memory usage without sacrificing accuracy on SME. With them, an index of five simulated years of screenshots fits in under 1 GB of RAM, and search takes under 0.1 s. Together, this work makes real-time indexing of screen history in its native visual form practical and efficient, paving the way for fully local, visual computer-use assistants.

In summary, our contributions are as follows:

  • We introduce real-time indexing of screenshot streams: continuously building searchable visual memory on local hardware, where existing methods face a trade-off between lossy OCR text and slow screenshot embeddings.

  • We propose STREAMEMBED, which exploits redundancy within and across screenshots through spatial token pruning and temporal KV reuse. Both use pixels alone, and a small adapter merged into model weights preserves fidelity to full-screenshot embeddings.

  • We build SCREENMEMEVAL, consisting of 10,426 questions over 184 hours of screen histories from two humans and a computer-use agent.

  • On SCREENMEMEVAL, STREAMEMBED achieves the new latency-accuracy frontier among evaluated methods: it runs up to 3.7× faster than the base screenshot encoder at matched accuracy. We also show its index fits five years of screenshots in under 1 GB of RAM.

Problem Setup

Real-time indexing. While a human or an agent uses a computer, its screen history arrives as a stream of screenshots S=(s1,s2,…)\mathcal{S}=(s_1,s_2,\ldots), one every Δ\Delta seconds. Each screenshot sts_t is indexed as it arrives, building a memory for future questions. To keep private content local, indexing runs on the user’s device. Here, indexing sts_t means encoding it into a vector vt\mathbf{v}_t with an embedding model EE and adding vt\mathbf{v}_t to a growing index.

Indexing is real-time if it keeps pace with the stream, so that screenshots never accumulate faster than they are indexed. Let the indexing latency LL be the mean time to index one screenshot, including any preprocessing such as OCR. Keeping pace thus requires, at minimum, L≤ΔL \leq\Delta; practical use further calls for a small LL, since the device remains in use for other work during indexing.

Retrieval runs only when a question qq arrives: the same model EE embeds qq, and Tk(q)=(s(1),…,s(k))\mathcal{T}_k(q)=(s_{(1)},\ldots,s_{(k)}) returns the kk indexed screenshots with the highest cosine similarity to qq, as in dense retrieval ([31]). This takes about 0.4 s (Table 4) and runs once per question rather than every Δ\Delta seconds, so only indexing is time-constrained.

IndexB/entryR@10 SME-H1R@10 SME-AgentRAM (GB) 1 yrRAM (GB) 2 yrRAM (GB) 5 yrSearch (ms) 1 yrSearch (ms) 2 yrSearch (ms) 5 yrInsert (ms) 1 yrInsert (ms) 2 yrInsert (ms) 5 yr
Flat4,09682.157.68.617.2–440881–0.0020.002–
IVF4,10480.9−1.254.8−2.88.617.3–92184–0.090.09–
HNSW4,36881.7−0.456.2−1.49.218.4–0.961.3–0.550.67–
PQ (128 B)12882.0−0.157.5−0.10.270.541.35651293590.140.140.14
1-bit (all 2048 dims)25682.157.60.541.082.6924.844.11020.0070.0070.007
1-bit (first 512 dims)6482.157.60.130.270.6715.529.556.00.0100.0100.010

Table 4. Index types as the index grows. B/entry is RAM per screenshot. R@10 is on real histories, and subscripts give the change from Flat. RAM, search, and insertion on CPU are for 1–5 years of screenshots (2.1M/year), simulated from the real embeddings with Gaussian noise. PQ and 1-bit re-rank the top 1,000 with fp16 vectors on the SSD. Search times the index only; encoding a query adds 52 ms on average. – exceeds 24 GB of RAM.

The challenge thus lies in indexing, which has two competing goals: preserving in each vt\mathbf{v}_t whatever a future question may need, and keeping LL small. Accordingly, § 6.2 reports retrieval accuracy against LL for each method.

Questions and evaluation. Each history in SME consists of task recordings: contiguous spans of screenshots, each covering one task. A question qq is answered by nn task recordings, and all screenshots of the mm-th recording (1≤m≤n1 \leq m \leq n) form the relevant set Sm+(q)S_m^+(q). For evaluation, each history is fully indexed before any question is asked, and retrieval accuracy is the fraction of these recordings that Tk(q)\mathcal{T}_k(q) hits with at least one screenshot,

Recall⁡@k(q)=1n∣{m:Sm+(q)∩Tk(q)≠∅}∣,(1)\operatorname{Recall}@k(q)=\frac{1}{n}\left|\left\{m:S_m^+(q)\cap\mathcal{T}_k(q)\ne\varnothing\right\}\right|, \tag*{(1)}

averaged over test questions ([58]).

Text or screenshots. An index can apply EE either to the OCR text of each screenshot sts_t, as a text index, or to sts_t itself, as a screenshot index. Existing open-source tools build text indexes (screenpipe, 2024; OpenRecall, 2024; McGhee, 2023), but OCR text often discards much of what the screen shows ([18, 64] (Ma et al. 2024)): it mixes the answer with every other string on the screen, such as menu labels and text from other windows; it loses the layout that tells them apart; and it drops state that only the pixels show, such as which option is selected.

Visual Encoding

A screenshot index encodes each arriving screenshot with a vision-language embedding model. This section describes the challenge, encoding as the main cost of indexing (§ 3.1), and the opportunity, redundant blocks that account for most of it (§ 3.2).

Challenges

Indexing a screenshot has two steps: encoding and insertion. Insertion takes under a millisecond, so nearly all of the cost lies in encoding. A vision-language embedding model such as Qwen3-VL-Embedding encodes a screenshot with a vision encoder and passes the resulting hundreds of visual tokens through its language model, whereas a text index passes only the OCR text through the language model.

As Table 1 shows, encoding a screenshot takes 2.33 s on an M2 Pro GPU: 7.5× as long as encoding its OCR text, and 3.6× the total latency of a text index even with OCR included. At one screenshot every Δ=5\Delta= 5 s, encoding thus occupies the GPU nearly half of the time, on a laptop that remains in use for other work. This cost likely explains why existing tools index OCR text, despite a 10.1-point drop in R@10 (screenpipe, 2024; OpenRecall, 2024; McGhee, 2023). Moreover, it recurs for every screenshot, since screenshot indexes encode each one from scratch.

IndexR@10OCREnc.InsertTotal
Text64.00.340.31<0.0010.65
Screenshot74.1–2.33<0.0012.33

Table 1. Mean R@10 and per-screenshot latency (s) of Qwen3-VL-Embedding-2B indexes on the SME. Enc.: encoding; Insert: adding the embedding to the index; Total: OCR + Enc. + Insert.

Opportunities

Encoding each screenshot from scratch ignores how much of a screen repeats, both within a screenshot and across screenshots. (i) Spatial redundancy arises within a screenshot, where backgrounds, margins, and blank panels span large areas (Figure 2(a)). (ii) Temporal redundancy arises across screenshots, where typing or moving the cursor changes a few regions and leaves the rest of the screen as it was (Figure 2(b)). Natural video has little of either: texture varies within a frame, and camera motion alters most pixels between frames.

Figure 2(d) quantifies this. In screen recordings, 67–77% of the pixel blocks that become visual tokens repeat a neighbor exactly or are unchanged from the previous screenshot, against 6–19% in natural video. An encoder that treats every screenshot as new thus spends most of its work on redundant tokens. It could instead skip blocks that repeat a neighbor, reuse the computation of blocks unchanged since the previous screenshot, and recompute only the rest. STREAMEMBED builds on this idea (§ 4).

STREAMEMBED

STREAMEMBED exploits this redundancy (§ 3.2) by encoding screenshots as a stream, computing only what has not been shown before. Spatial pruning removes redundancy within a screenshot (§ 4.2), temporal reuse removes redundancy across screenshots by recomputing only changed blocks (§ 4.3), and a consistency adapter keeps the embedding of the reduced input close to that of the full screenshot (§ 4.4). Figure 3 shows how a screenshot is indexed.

Indexing with StreamEmbed

Figure 3. Indexing with StreamEmbed. A screenshot arrives every Δ=5\Delta= 5 s. Spatial pruning drops blocks that repeat their left or upper neighbors (§ 4.2), and change detection flags kept blocks that are new or have changed beyond a threshold (§ 4.3). The ViT and LM recompute only the flagged blocks (and the suffix), reusing cached keys and values for the rest. The last token’s final hidden state is the embedding of the screenshot.

Embedding a screenshot

A vision-language embedding model Ma et al., 2024, such as Qwen3-VL-2B Li et al., 2026, encodes a screenshot in three steps. First, a vision encoder (ViT) Dosovitskiy et al., [14] turns each 16×1616 \times16 patch into a vector. Second, each 2×22 \times2 group of patches, a 32×3232 \times32 pixel block, forms one visual token. Third, a language model (LM) reads these tokens between a fixed text prefix and suffix, and its normalized last-layer hidden state at the last token is the embedding E(s)E(s). The ViT and the LM stack self-attention layers, in which each token forms a query, key, and value and attends to the keys and values of all tokens, bidirectionally in the ViT and causally in the LM. A 1280×7201280 \times720 screenshot, resized to 1280×7041280 \times704, becomes a 22×4022 \times40 grid of 880 blocks and tokens.

Notation. We write bt,ib_{t,i} for the block at grid position ii of sts_t, so bt,ib_{t,i} and bt+1,ib_{t+1,i} cover the same screen region, and ∥b−b′∥∞\lVert b-b' \rVert_\infty for the largest absolute pixel difference between blocks bb and b′b' over all RGB channels.

Spatial Token Pruning

We prune before the ViT, so a dropped block is processed by neither the ViT nor the LM.

Neighbor rule. We write ℓ(i)\ell(i) and u(i)u(i) for the left and upper neighbors of block ii. A block is dropped if it repeats either neighbor within τ\tau, 8 of 255 levels, and kept otherwise,

Pt={i:∥bt,i−bt,ℓ(i)∥∞>τ and ∥bt,i−bt,u(i)∥∞>τ}.(2)\mathcal{P}_t = \left\{ i : \lVert b_{t,i} - b_{t,\ell(i)} \rVert_\infty> \tau\text{ and } \lVert b_{t,i} - b_{t,u(i)} \rVert_\infty> \tau\right\}. \tag*{(2)}

A block on the left or top edge is compared only with the neighbor it has, and the top-left block is always kept. Kept blocks keep their original position embeddings in the ViT and the LM, so each token still encodes where its block lies.

Grid keep. The neighbor rule reduces a large uniform region, such as a blank panel, to a few blocks, which shifts the embedding away from that of the full screenshot. We thus also keep Gt\mathcal{G}_t, one block of each uniform region in every fixed 3×33 \times3 tile of blocks, so the kept set is Kt=Pt∪Gt\mathcal{K}_t = \mathcal{P}_t \cup\mathcal{G}_t. Wang et al. [61] decide block by block in raster order, so each decision depends on all earlier ones. We instead compare each block with its neighbors in the original screenshot, so all blocks are decided independently and in parallel.

Temporal KV Reuse

We reuse the keys and values of earlier screenshots and recompute only changed blocks. Reuse is exact only if every token a reused token attends to is unchanged. Otherwise, it approximates, and periodic refresh bounds the error.

Change detection. Let b^i\hat{b}_i be block ii as it was when its cached keys and values were last computed. Among the kept blocks Kt\mathcal{K}_t, we recompute those that have no cache or have changed by more than δ\delta since then,

Ct={i∈Kt:b^i does not existor ∥bt,i−b^i∥∞>δ},(3)\mathcal{C}_t = \left\{ i \in\mathcal{K}_t : \hat{b}_i \text{ does not exist} \\ \text{or } \lVert b_{t,i} - \hat{b}_i \rVert_\infty> \delta\right\}, \tag*{(3)}

with δ=τ\delta= \tau. Comparing with b^i\hat{b}_i rather than bt−1,ib_{t-1,i} keeps every reused block within δ\delta of b^i\hat{b}_i, even when it changes slowly over many screenshots.

ViT. In every layer, we recompute the queries, keys, and values of the blocks in Ct\mathcal{C}_t and update their cache. They attend to all kept blocks, taking the keys and values of unchanged ones from the cache, so each changed block still sees every kept block. The outputs of the other kept blocks come from the cache. The ViT approximates in that an unchanged block does not see what changed elsewhere on the screen.

LM. In the LM, we likewise recompute, in every layer, the visual tokens of Ct\mathcal{C}_t and the fixed suffix and update their cache. They attend causally to all earlier tokens, taking the keys and values of the others from the cache. The suffix is always recomputed because it attends to every visual token, and its last token yields the embedding. The LM approximates in that unchanged visual tokens keep their cached keys and values even when earlier tokens have changed.

Refresh. Reused keys and values drift from those of a full encoding. We therefore re-encode all kept blocks, setting Ct←Kt\mathcal{C}_t \leftarrow\mathcal{K}_t, when

∣Ct∣>ρ∣Kt∣ort−tr≥R,(4)\lvert\mathcal{C}_t \rvert> \rho\lvert\mathcal{K}_t \rvert\quad\text{or} \quad t - t_r \ge R, \tag*{(4)}

where trt_r is the last refreshed screenshot, ρ=0.9\rho= 0.9, and R=10R = 10. The first condition catches a new screen, on which partial recomputation saves little, and the second bounds the drift.

Consistency Adapter

We fine-tune the embedding model with LoRA (Hu et al., 2022) on the training split of ScreenMemEval (§ 5.3) and use it to embed questions. Pruning and reuse, however, shift the embedding of a reduced screenshot away from that of the full one. We therefore train a consistency adapter, a second LoRA initialized from the first and used only to encode screenshots, by distilling the fine-tuned model ETE_T on full screenshots into the adapter ESE_S on pruned ones. For a screenshot ss and its pruned input s~\tilde{s}, the loss is

L=1−cos⁡(ES(s~),ET(s))+KL⁡(pT∥pS)+Lmargin,(5)\mathcal{L} = 1 - \cos(E_S(\tilde{s}), E_T(s)) + \operatorname{KL}(p_T \mathbin{\|} p_S) + \mathcal{L}_{\mathrm{margin}}, \tag*{(5)}

where pTp_T and pSp_S are softmax distributions of the teacher’s and student’s similarities over a question’s candidate screenshots, and Lmargin\mathcal{L}_{\mathrm{margin}} is Margin-MSE loss on a relevant screenshot and a hard negative.

Following multi-objective distillation (Sanh et al., [53]; Jiao et al., [29]), the three terms match the teacher’s embedding (Hwang et al., [24]), its ranking over candidates (Abbasi Kooppaei et al., 2020; Fang et al., [17]), and its score gap on hard pairs (Hofstätter et al., [19]). Merged into the screenshot encoder’s weights, the adapter adds no computation.

The ScreenMemEval Benchmark

Evaluating screenshot retrieval under real-time indexing (§ 2) requires a screenshot history, questions about its past, and relevance labels. The closest benchmark (Wang & Chen, [62]) asks questions over a person’s screenshot history, but scores generated answers without relevance labels, and its data is not public.

We therefore build ScreenMemEval (SME), with 10,426 test questions (plus 1,315 for training and validation) over three screenshot histories, two from human workers and one from a computer-use agent (Table 2). Each question is labeled with the task recordings that answer it. SME consists of three sets, SME-H1, SME-H2, and SME-Agent. The appendix gives further details.

SME-H1SME-H2SME-Agent
Recorded fromhumanhumanagent
Days1175743
Recordings2,105946787
Screenshots63.5k12.8k55.9k
Questions5,1113,5071,808
single / multi4,265 / 8463,096 / 4111,654 / 154

Table 2. Statistics of the three screenshot histories of SME.

Screen Histories

Histories from human workers. SME-H1 and SME-H2 each hold one worker’s task recordings, in time order, from Howland et al. [20], a public dataset of instructed tasks on websites and desktop applications on Windows. Screenshots are sampled every Δ=5\Delta= 5 s at 1280×\times720.

History from a computer-use agent. SME-Agent extends the SME to a computer-use agent’s screen. This reflects an emerging workflow: agents perform tasks, and humans later verify what they did; the history can also serve as the agent’s own memory. We run GPT-6.1-sol (OpenAI, [47]), a frontier model, on the 184 tasks of myPCBench (Jang et al., [25]). The tasks are built around one persona, whose Ubuntu virtual machine holds personal files, 17 self-hosted web applications, and desktop applications. Since each task involves many steps, we use an LLM to split them into 787 sub-tasks, which we then place on a 43-day calendar. Each sub-task forms one recording, captured at the same resolution and interval as the human histories.

Questions

SME has single- and multi-recording questions qq, answered by one (n=1n = 1) or several (n≥2n \ge2) recordings (Table 3).

Single-recording questions. A single-recording question asks where, when, what, or how a task was done, or which value was entered, and is phrased in the first person, as the worker or the persona would ask it. Opus 5.5 (Anthropic, [4]) writes one of each type for each recording, based on its annotations, which describe the task, application, and time. The generator never sees screenshots or OCR text, so it favors neither index type. Most screenshots in every history show a clock, so times can be matched on screen.

To keep each answer unique, the same model, as a judge, compares each question with similar recordings, discards it if another recording answers differently, and labels recordings with the same answer as relevant. For SME-Agent, GPT-6-astra (OpenAI, [48]) also checks that the recording’s screens and actions support the answer.

TypeExample question and answer
Single-recording
whereWhere did I look up highly rated Vienna hotels with parking and breakfast? → Booking.com
whenWhen did I check the airline’s size rules for flying with a bike? → Jul 29, 4:44 pm
whatWhat did I do on Rent.com on the afternoon of August 7? → Checked utility costs
howHow did I create the battery report? → Ran powercfg
valueWhat return date did I set for the JFK round-trip flight? → June 20th
Multi-recording
countHow many times in September did I search for places to stay on Airbnb? → 3
comparisonDid I search for more nights for the Sydney hotel or for the Jakarta hotel? → Sydney
orderDid I look up a pivot table tutorial first, or check the price of Bitcoin? → Pivot table
durationWhich took me longer: finding hiking trails nearby, or looking up the symbol for gold? → Hiking trails

Table 3. Question types in SME, with examples.

Multi-recording questions. A multi-recording question counts how often an event occurred, compares two values, orders two events, or compares two durations. Its answer is computed from the annotations or verified answers of single recordings, and an LLM only phrases the question. To ensure each question needs all its evidence, a comparison, order, or duration, a question is kept only if an LLM answers it correctly from the relevant recordings, but fails once any one is removed. A count must match two independent LLM recounts exactly. In SME-Agent, GPT-6-astra also checks that every recording a question combines shows its task done on screen.

Relevance Labels and Splits

Relevance labels. Questions ask about tasks, not single screenshots, so relevance is labeled per task recording. For a single-recording question (n=1n = 1), S1t(q)S_{1}^{t}(q) in Eq. (1) holds the screenshots of its recording and of every recording the judge labeled relevant. A multi-recording question has one set per piece of evidence, holding the screenshots of the recording that supplies it and of any recording judged to give the same evidence. As in parallel multi-hop search (Ko et al., 2026), each piece of evidence can be retrieved independently and then combined. Comparison, order, and duration questions have n=2n = 2, and for a count question nn equals its answer, from 2 to 8. Since a SME-Agent recording averages 71 screenshots, we also score SME-Agent on only the screenshots that show the evidence.

Experiments

We first compare STREAMEMBED with text and screenshot indexes (§ 6.2). We then index a long stream, simulating up to five years of screenshots to test how different index types scale (§ 6.3). Finally, we apply STREAMEMBED to other encoders (§ 6.4) and ablate its components (§ 6.5).

Setup

Datasets. We evaluate on the three histories of SME (§ 5). Fine-tuned models are trained on 1,000 questions about the earliest recordings of SME-H1, and all checkpoints and settings are selected on 315 questions about the next 79 recordings; neither set overlaps with the test questions. SME-H1 is thus in-domain for every fine-tuned model, SME-H2 tests transfer to another worker, and SME-Agent is out-of-domain, recorded from a computer-use agent on another operating system (§ 5.1).

Baselines. We compare STREAMEMBED with text and screenshot indexes. A text index extracts the OCR text of each screenshot with Apple Vision OCR (Apple Inc.) for GPU latency and with RapidOCR (RapidAI, [51]) for CPU latency, since Apple Vision OCR runs on the Neural Engine. It then ranks the text with BM25 (Robertson & Zaragoza, [52]), or embeds it with Qwen3-Emb-0.6B, -4B, or -8B (Zhang et al., [77]) (and gte-modernbert-base (Li et al., (2023); Zhang et al. [76]) in Figure 1) or with the embedding model Qwen3-VL-2B (Li et al., 2026). We also fuse the ranking of Qwen3-VL-2B with that of BM25 by reciprocal rank fusion (Cormack et al., [11]). A screenshot index embeds the full screenshot with the NaFlex variants of SigLIP2-B and SigLIP2-So400m (SigLIP2-So) (Tschannen et al., [59]), VultronRetriever-Flash (Vultron) (Vultr, [60]), or Qwen3-VL-2B or -8B (Li et al., 2026). Qwen3-VL-2B serves as both a text and a screenshot encoder in the main comparison, used base or fine-tuned with LoRA (FT).

Implementation. STREAMEMBED uses τ=δ=8\tau= \delta= 8, tiles of 3×33 \times3 blocks, ρ=0.9\rho= 0.9, and R=10R = 10, chosen on the SME-H1 validation questions, and encodes screenshots with the consistency adapter (§ 4.4). The Qwen3-VL screenshot baselines run as released, with a Conv3d patch embedding. STREAMEMBED computes it with an equivalent linear one (§ 6.5). LoRA and the consistency adapter are merged into the weights and do not change latency.

Metrics. We report (i) Recall@10 of each dataset for accuracy and (ii) indexing latency for cost, the mean time to encode a screenshot and add it to the index on the GPU and CPU of an Apple M2 Pro, including OCR for the text index. We measure latency over the first 50 consecutive screenshots, or all if fewer, of 30, 30, and 20 recordings sampled from SME-H1, SME-H2, and SME-Agent.

Main Results

Figure 4 compares STREAMEMBED against all text and screenshot indexes across the three histories, reporting R@10 and mean latency on the GPU and CPU.

Figure 4. Main results. STREAMEMBED approaches the accuracy of a screenshot index at close to the latency of a text index. Recall@10 and mean indexing latency per screenshot on the GPU and CPU of an M2 Pro (§ 6.1), latency on a log scale. Text indexes use Apple Vision OCR on the GPU and RapidOCR on the CPU, since Apple Vision OCR runs on the Neural Engine. OCR latency is almost equal to the BM25 latency, since BM25 adds under 0.1 ms.

STREAMEMBED is 2.8–3.7× faster than the screenshot index and stays within 1.3 R@10 of it. Against the fine-tuned Qwen3-VL-2B screenshot index, which takes 2.33–2.34 s per screenshot on the GPU, STREAMEMBED takes 0.66, 0.84, and 0.63 s on SME-H1, SME-H2, and SME-Agent and scores 1.3, 1.2, and 0.6 R@10R@10 lower (Figure 4). It is more accurate than the base screenshot index on SME-H1 and SME-H2 and 1.0 R@10R@10 below it on SME-Agent. Only Qwen3-VL-8B is clearly more accurate, on SME-Agent, at about eight times the latency.

STREAMEMBED is more accurate than every text index, at a similar latency. The strongest text index, the fine-tuned Qwen3-VL-2B fused with BM25, scores 3.5–4.6 R@10R@10 below STREAMEMBED, and larger text encoders such as Qwen3-Emb-8B do not close this gap while taking longer. On the GPU, STREAMEMBED nearly matches its latency on SME-H1 (0.66 vs. 0.62 s) and is faster on SME-Agent (0.63 vs. 0.69 s). On SME-H2 it takes 0.84 against 0.65 s, but it still fits within a capture interval of 1 s, which the screenshot index (2.34 s) does not. The same holds on the CPU. STREAMEMBED takes 1.87–2.61 s, close to the strongest text index (1.86–2.12 s), while the fine-tuned screenshot index takes 5.42–5.81 s, longer than even a 5 s capture interval.

STREAMEMBED moves the Pareto frontier beyond lower resolution and smaller encoders. A common way to cut indexing latency is to lower the input resolution or to use a smaller encoder. Whereas Figure 1 runs each baseline at its default resolution, Figure 5 revisits the comparison by applying both to the screenshot index as well as to STREAMEMBED, which can also raise its drop ratio. Even then, on both encoder families and all three histories, STREAMEMBED scores 2.6–11.4 R@10R@10 higher than the baseline at a similar or lower latency. The gain is largest at low latency, where the baseline must fall back to a small input.

Pareto frontiers comparing STREAMEMBED and baseline screenshot indexes across Qwen3-VL and SigLIP2 encoder sizes, resolutions, and latencies

Figure 5. STREAMEMBED with lower resolution and smaller encoders. Bands are the Pareto frontiers of Qwen3-VL (top) and SigLIP2 (bottom) over encoder size and resolution, with and without STREAMEMBED; arrows give the R@10R@10 gain of STREAMEMBED at a similar or lower latency. Labels give the encoder size and input width (SigLIP2: maximum patches, p) and, for STREAMEMBED, the fraction of dropped blocks. Gray crosses are other indexes, off both frontiers.

At 0.19 s on SME-H1, Qwen3-VL-2B with STREAMEMBED at 480×270480 \times270 reaches 75.3 R@10R@10, against 65.0 for the full screenshot at 320×180320 \times180. The gain narrows near 0.7 s, where both sides approach the accuracy of the full encoder, yet STREAMEMBED on Qwen3-VL-2B still scores 2.7 R@10R@10 above Qwen3-VL-8B at 480×270480 \times270 while running faster. Lowering the resolution or the encoder size applies to STREAMEMBED as well, and STREAMEMBED makes each resolution cheaper, so most points on its frontier pair STREAMEMBED with a lower resolution.

Indexing a Long Stream

So far, we have shown that STREAMEMBED makes real-time encoding practical, but on-device retrieval poses a second challenge: can a growing screenshot index fit within a laptop’s RAM? Memory already constrains deployment (e.g., Recall (Microsoft) requires at least 16 GB).

To quantify how far an index can go, we grow a simulated index from one to five years of screenshots (2.1M per year at one every 5 s over eight hours a day, whose fp16 embeddings take 8.6 GB) by adding Gaussian noise to copies of the real embeddings. We compare exact search (Flat), approximate search with an inverted file (IVF) (Jégou et al., [27]) or a graph (HNSW) (Malkov & Yashunin, [43]), and compressed codes kept in RAM whose top 1,000 candidates are re-ranked with the full embeddings on disk, namely product quantization (PQ) (Jégou et al., [27]) and 1-bit codes (Shakir et al., [55]; Yamada et al., [73]), all implemented in Faiss (Douze et al., [15]).

Table 4 reports the accuracy of each index on SME-H1 and SME-Agent, and its RAM, search time, and insertion time as the history grows, on the CPU of the M2 Pro.

Indexes that keep full embeddings in RAM may not scale to years of screenshots. Flat needs 8.6 GB of RAM per year and exceeds 24 GB at five years, and its search time grows with the index, from 440 ms at one year to 881 ms at two. IVF searches about five times faster than Flat but loses up to 2.8 R@10, since relevant screenshots can fall outside the clusters it searches. HNSW searches in about 1 ms but needs 7% more RAM than Flat for its graph and loses up to 1.4 R@10.

Compressed codes with re-ranking from disk keep R@10 at a small fraction of a second. PQ and 1-bit codes stay within 0.1 R@10 of Flat, since the codes only need to place the relevant screenshots among the top 1,000, and re-ranking with the full embeddings restores their order. 1-bit codes of the first 512 dimensions take 64 bytes per screenshot, 64 times less than Flat, and hold five years in 0.67 GB. Comparing bit codes is also cheap, so they search in 56 ms after five years, against 359 ms for PQ. With the 52 ms of encoding a question, retrieval takes about 0.1 s even after five years.

Insertion stays negligible even with five years of history. Inserting a screenshot takes at most 0.67 ms, for HNSW, and 0.01 ms for 1-bit codes, against 0.63–0.84 s to encode it with STREAMEMBED. This confirms the premise of §3.1 that the cost of real-time indexing lies in encoding, not insertion, even as the history grows, and it is this cost that STREAMEMBED reduces.

Generalization to Other Encoders

Table 5 applies STREAMEMBED with the same thresholds to four other screenshot encoders, each fine-tuned with LoRA on SME-H1, and compares it with each encoder on full screenshots.

EncoderSME-H1 R@10SME-H1 LatencySME-H2 R@10SME-H2 LatencySME-Agent R@10SME-Agent Latency
Qwen3-VL-8B84.05.15 / 14.8285.05.19 / 14.8861.95.16 / 14.68
STREAMEMBED83.01.59 / 6.3183.02.08 / 7.7860.71.49 / 5.93
Qwen3-VL-2B83.42.33 / 5.4284.52.34 / 5.4758.22.34 / 5.81
STREAMEMBED82.10.66 / 1.9783.30.84 / 2.6157.60.63 / 1.87
SigLIP2-So (1.1B)77.50.34 / 0.8178.70.34 / 0.8049.60.34 / 0.81
STREAMEMBED76.30.19 / 0.3978.80.23 / 0.5146.90.18 / 0.37
Vultron (0.8B)78.91.05 / 2.4780.71.05 / 2.5160.41.05 / 2.44
STREAMEMBED77.80.71 / 1.2279.70.79 / 1.4956.80.67 / 1.21
SigLIP2-B (0.4B)68.70.13 / 0.2271.40.13 / 0.2239.40.13 / 0.22
STREAMEMBED68.20.09 / 0.1270.30.10 / 0.1538.10.09 / 0.12

Table 5. STREAMEMBED on other encoders. Full screenshot and STREAMEMBED for each encoder, all fine-tuned with LoRA on SME-H1. Latency is GPU / CPU in seconds per screenshot. Darker blue marks higher R@10 and darker pink marks lower GPU latency, within each column.

STREAMEMBED generalizes to every encoder, staying close to its full-screenshot accuracy. It lowers the GPU latency of all five encoders, by 1.3–1.9× on SigLIP2 and Vultron and by 2.5–3.7× on Qwen3-VL, while losing only 1.3 R@10 on average.

The speedup grows with the computation that reuse can skip. STREAMEMBED speeds up the Qwen3-VL encoders by 2.5–3.7× on the GPU, since temporal reuse also saves computation in the language model. On SigLIP2, which has no language model, and on Vultron, whose language model mostly reuses attention, STREAMEMBED reuses keys and values only in the vision encoder and still speeds them up by 1.3–1.9× on the GPU and 1.5–2.2× on the CPU. On Vultron, which stores one vector per visual token, pruning also stores 23–30% fewer vectors per screenshot, trading some accuracy on SME-Agent for a smaller index.

Ablation

Table 6 adds the components of StreamEmbed one at a time to the released Qwen3-VL-2B, fine-tuned with LoRA, starting with the equivalent linear patch embedding that StreamEmbed uses for Qwen3-VL (§ 6.1).

StreamEmbed speeds up encoding by 2.8–3.7×\times over the released model, mostly through spatial pruning and temporal reuse. Spatial pruning removes the repetition within a screenshot and temporal reuse removes the repetition across screenshots, each speeding up encoding by about 1.5×\times at about 1 RR@10. Their savings thus multiply, to 2.0–2.6×\times together. Their overhead is small, as pruning and change detection add about 0.01 s per screenshot, and in the final configuration the vision encoder and the language model each run two to three times faster than without them.

Grid keep and the consistency adapter recover the accuracy lost due to pruning on SME-Agent. Pruning every redundant block costs far more on SME-Agent than on the human histories, suggesting that dropping every redundant block shifts the embedding. Grid keep recovers most of this loss for about 0.15 s per screenshot, and the consistency adapter recovers more at no latency cost, leaving StreamEmbed within 1.3 RR@10 of the full screenshot on all three histories.

Related Work

Screen history. Re-finding what was once on a computer screen has long been studied, with people searching the files, e-mail, and web pages they have opened ([16]) or browsing past screenshots ((Hu & Lee, 2020)). Such histories serve not only people but also agents, which complete more later tasks when they reuse past experience ([66, 2]) or see past screenshots rather than summaries ([40]). Recent systems build such histories using vision-language models (VLMs), turning screenshots into the text memory of an agent ([62]) or a log of each task ([75]), or answering questions over them ((Li et al., 2025)). These systems rely on large cloud models ([62] (Li et al. 2025)) or, to run on the device, process only some screenshots ([75]), and either way represent screenshots as text. Yet retrieving from pixels is more accurate than from text on documents and the web ([18] (Ma et al. 2024)). We thus keep every screenshot as an embedding of its pixels, computed on the device.

Growing indexes. Earlier work has studied indexes that grow over time, but it rarely treats the cost of encoding each new item as the bottleneck. Real-time search indexes text, which is cheap ([6]), and updatable vector indexes leave embedding as a separate step ((Xu et al., 2023)). Work on evolving corpora studies how to add documents to a trained retriever ([33]) and when to update it ([32] (Ko et al. 2025)). For screenshots, however, encoding is the bottleneck. Work that faces this cost encodes less or defers it, using cheaper models ([21]), fewer frames ([30]), early exit ([7]), smaller encoders ((Teletchea et al., 2025)), or lower resolution ([64]). Instead, we remove only the computation that screens repeat, which is combined with a smaller encoder, lower resolution, or a skip gate. A growing index must also stay small. Lyu et al. [41] rank compressed codes from disk, and Wang et al. [63] recompute embeddings at search time on a laptop. We study how to fit a screenshot index in a laptop’s memory.

Streaming VLMs. Streaming VLMs keep each arriving frame in their context to answer questions about the video at any time ([9]), so their main concern is what to keep. Instead, we store each screenshot in an index for later queries, as in Jeong et al. [28], so our main concern is how fast each one is encoded. This leads to three differences. These models keep tokens or their keys and values, which they evict, compress ((Xu et al., 2025a)), or retrieve for each question ([46] (Di et al. 2025)). Our index holds only one embedding per screenshot. Methods that cut computation across frames decide what to skip inside or after the vision encoder ([74, 65]). Others decide from pixels but reuse computation in at most one of the encoder and the LM ([10] (Xu et al. 2025b)). In contrast, our method decides fast from pixels before the encoder and reuses unchanged blocks in both. Finally, prior work keeps accuracy with modules added at inference ([3, 24, 56]). Our adapter is instead a LoRA merged into the weights, adding no latency.

Conclusion

We introduced StreamEmbed, which indexes a growing screenshot history in real time by encoding it as a stream. Since a rendered screen repeats most of its blocks, StreamEmbed prunes blocks that repeat a neighbor, reuses keys and values of unchanged blocks in both the vision encoder and the language model, and keeps the reduced embedding close to the full one at no added cost. On ScreenMemEval, built from the screenshot histories of human workers and a computer-use agent, StreamEmbed matches a text index in latency and a screenshot index in accuracy, while binary codes keep a year of screenshots within the laptop’s memory. Future work could learn what to recompute from the stream itself, lowering cost and raising accuracy further. Agents could also search a person’s screenshot history to complete long-horizon tasks that depend on screens seen days or weeks earlier.

Acknowledgments

We sincerely thank Donghyun Lee, Prasann Singhal, and Ryan Wang for their valuable feedback and support.

Benchmark Details

Table 7 compares SCREENMEMEEVAL with ScreenshotVQA [62], a screen benchmark.

ScreenshotVQASME-H1SME-H2SME-Agent
History
Days–1175743
Span (days)221727843
Screenshots9.8k63.5k12.8k55.9k
per recording–301371
Screen time (h)–881878
Questions
Questions875,1113,5071,808
Evaluation
Relevance×\times✓✓✓
Scored onanswerretrievalretrievalretrieval
Latency–indexingindexingindexing
Role–train + testtest (new worker)test (new source)
Released×\times✓✓✓

Table 7. SCREENMEMEVAL and ScreenshotVQA. Days counts days with at least one recording, as in Table 2, and Span counts calendar days from the first to the last recording, on the simulated calendar for SME-Agent. Screen time is the number of screenshots times the 5 s interval. Relevance marks recording-level relevance labels, and Role is the use of each history when encoders are fine-tuned. For ScreenshotVQA, span and screenshots are averaged over its three histories.

Questions. The generator writes questions for 12 recordings at a time from their annotations, which are the instruction, application, times, and planned steps for SME-H1 and SME-H2, and the sub-task instruction, the agent’s actions, and the scheduled time for SME-Agent. To keep answers unique, the judge compares each question with up to 30 recordings of the same history chosen from annotations alone, such as recordings of the same task or on the same site, and always every sibling sub-task in SME-Agent. No evaluated index chooses them, so the labels favor neither index type. Each count recount (§ 5.2) runs over a shuffled list of the candidate recordings.

Relevance and splits. For screenshot-level scoring on SME-Agent, GPT-6-astra marks the keyframes that show each question’s evidence, and each keyframe counts with its run of near-identical screenshots. Training uses the earliest 710 recordings of SME-H1, and the 1,121 questions about the 433 of them without training questions are test questions. Training and validation recordings stay in the index and are relevant to 54 test questions, and removing these questions changes R@10 by at most 0.03 for both the fine-tuned full screenshot and STREAMEMBED.

Data. SME-H1 and SME-H2 come from Howland et al. [20], recorded by consenting workers who authorized public release, under the MIT license. SME-Agent shows a synthetic persona and holds no data of real people. We release all three histories with their questions, relevance labels, and splits.

Full Results

Table 8 lists every system of the main results with R@10 split into single- and multi-recording questions, together with nDCG@10 [26]. For a multi-recording question, R@10 is the fraction of its evidence sets hit, as in Eq. (1). nDCG@10 gives each evidence set one unit of gain at the rank of its first retrieved screenshot and normalizes by the ideal ranking of min⁡(n,10)\min(n, 10) sets.

SME-H1SME-H2SME-Agent
AllSingleMultinDCGAllSingleMultinDCGAllSingleMultinDCG
Text index (Apple Vision OCR)
BM2558.664.329.951.056.758.642.347.135.336.027.326.2
Qwen3-Emb-0.6B69.275.039.961.269.372.148.860.140.141.129.628.3
Qwen3-Emb-4B73.880.042.366.271.575.045.363.049.050.434.136.0
Qwen3-Emb-8B75.081.144.568.272.876.049.164.446.047.331.133.9
Qwen3-VL-2B73.579.444.165.371.273.652.762.647.448.732.934.3
Qwen3-VL-2B (FT)77.483.347.770.477.480.157.268.451.352.934.137.4
Qwen3-VL-2B + BM2574.880.545.966.572.474.754.863.548.850.134.835.2
Qwen3-VL-2B (FT) + BM2578.684.449.471.378.781.160.068.753.254.737.038.3
Screenshot index
SigLIP2-B66.273.031.958.067.570.644.157.236.137.125.326.2
SigLIP2-So73.380.238.865.673.376.351.263.443.845.426.131.7
Vultron77.884.743.470.379.983.453.469.457.660.031.543.1
Qwen3-VL-2B81.487.650.373.882.385.260.572.258.660.538.344.5
Qwen3-VL-2B (FT)83.489.254.276.384.587.363.774.758.259.940.944.9
Qwen3-VL-8B82.288.451.275.983.186.259.674.061.964.434.547.0
STREAMEMBED (Qwen3-VL-2B)82.188.052.275.583.386.162.473.957.659.439.043.8

Table 8. Full main results. R@10R@10 over all questions (All), single-recording questions (Single), and multi-recording questions (Multi), and nDCG@10 over all questions. FT marks fine-tuning with LoRA on SME-H1, and other rows use base models.

OCR engines. Table 9 compares the text indexes on the two OCR engines, Apple Vision OCR used for GPU latency and RapidOCR used for CPU latency. The engine changes R@10 by at most 1.6, so every text index stays below STREAMEMBED with either engine.

Text encoderApple Vision OCRRapidOCR
SME-H1SME-H2SME-AgentSME-H1SME-H2SME-Agent
Qwen3-Emb-0.6B69.269.340.170.069.739.8
Qwen3-Emb-4B73.871.549.074.372.348.8
Qwen3-Emb-8B75.072.846.074.671.246.3
Qwen3-VL-2B73.571.247.473.571.746.8
Qwen3-VL-2B (FT)77.477.451.378.077.351.7

Table 9. Text indexes on two OCR engines. R@10R@10 with OCR text from Apple Vision OCR or RapidOCR. FT marks fine-tuning with LoRA on SME-H1.

Implementation Details

Training. Every fine-tuned encoder uses LoRA (Hu et al., 2022) with rank 32 on the attention projections of the vision encoder and of the LM or text tower. It is trained with InfoNCE on the 1,000 training questions of SME-H1, each with two hard negatives, for 300 steps at a learning rate of 7×10−67 \times10^{-6} for Qwen3-VL, 5×10−55 \times10^{-5} for SigLIP2, and 3×10−53 \times10^{-5} for Vultron. The consistency adapter is a second LoRA initialized from the first and distilled from the frozen fine-tuned model with the loss of Eq. (5), on screenshots pruned by the neighbor rule, for 300 steps at 10−510^{-5}. Text indexes use Apple Vision OCR at its accurate level with language correction, and the fused text index combines Qwen3-VL-2B and BM25 by reciprocal rank fusion with k=10k = 10 and weight 0.5 on BM25. All checkpoints, the settings of STREAMEMBED, and the fusion are chosen on the SME-H1 validation questions.

Latency. We time every index on an Amazon EC2 mac2-m2pro instance, an Apple M2 Pro with 32 GB of memory, left idle, with PyTorch on MPS in fp16 for the GPU and in fp32 on 8 threads for the CPU. Screenshots are processed one at a time in their recorded order, caches start empty at each recording, and the timer runs from reading the image to the embedding, including OCR for a text index.

Patch embedding. The Qwen3-VL screenshot baselines run the released Conv3d patch embedding, and STREAMEMBED computes the same map as one matrix multiplication, whose embeddings have a cosine similarity of at least

0.99986 in fp16. Vultron runs with an equivalent but faster layout of its own Conv3d, so this baseline is faster than released.

Resolution and Drop Variants

Figure 6 splits the frontiers of Figure 5 by model size, so each size of STREAMEMBED is compared with the same size of the base encoder at lower resolutions. STREAMEMBED uses the fine-tuned encoder with its consistency adapter. Latency of the lower-resolution baselines is measured on SME-H1 and reused for the other histories.

Figure 6. Frontiers per model size. R@10 against GPU latency per screenshot for Qwen3-VL (top) and SigLIP2 (bottom). Pink and purple mark STREAMEMBED on the small and large model, and light and dark slate the baselines. Solid lines are frontiers of STREAMEMBED over full resolution, lower resolutions, and drop rates of 50% and 60%, and dashed lines are frontiers of the base full screenshot over lower resolutions. Stars mark STREAMEMBED at full resolution. GTE-MB-149M is gte-modernbert-base on Apple Vision OCR text without language correction. In the Qwen3-VL row of SME-Agent, points below 42 R@10 are omitted.

Fine-tuned baseline. The baselines in Figure 5 and Figure 6 are base encoders. On Qwen3-VL, fine-tuning alone moves the full screenshot by −0.4 to 2.2 R@10R@10 (Table 8 and Table 5), less than every Qwen3-VL gain in Figure 5. At 960×540, the fine-tuned Qwen3-VL-2B needs half the computation of a full screenshot and takes 1.22 s per screenshot on the GPU with the released Conv3d, timed on SME-H1, where full-screenshot latency matches the other histories. STREAMEMBED is 1.4–1.9× faster than it and 2.0 R@10R@10 more accurate on SME-Agent, while staying within 1.3 R@10R@10 on the human histories.

Latency Breakdown

Table 10 splits the GPU latency of STREAMEMBED into its vision encoder (ViT) and language model (LM). With reuse in the ViT only, the LM reuses the prefix before the first changed token. Pruning and reuse make the ViT 2.8×2.8 \times and the LM 2.3×2.3 \times faster than with the linear patch embedding alone, and the ViT 4.9×4.9 \times faster than in the released model. On the CPU of SME-H1, the ViT runs 3.2×3.2 \times faster (3.72 to 1.16 s) and the LM 2.2×2.2 \times faster (1.69 to 0.78 s).

Latency (s)Comp.
TotalViTLMViTLM
Latency sample of the paper (994 screenshots)
Screenshot, Conv3d2.331.660.663.532.64
+ Linear patch embed.1.650.960.683.532.64
+ Spatial pruning1.000.520.451.871.60
+ Temporal reuse0.660.340.291.010.85
Separate set of 2,000 screenshots, no grid keep
Screenshot, Conv3d2.351.670.663.532.64
+ Linear patch embed.1.650.960.673.532.64
+ Neighbor rule0.880.450.401.561.37
+ Reuse in the ViT0.750.340.380.871.16
+ Reuse in the LM0.670.340.300.870.75

Table 10. Latency breakdown on SME-H1. Mean GPU latency per screenshot (s) of the fine-tuned Qwen3-VL-2B as parts of STREAMEMBED are added, with preprocessing in Total. The top block is the latency sample used throughout the paper, as in Table 6, with grid keep. The bottom block is a separate set of 2,000 SME-H1 test screenshots without grid keep. Comp. is computation in TFLOPs per screenshot.

SME-H1 R@10SME-H1 LatencySME-H2 R@10SME-H2 LatencySME-Agent R@10SME-Agent Latency
Screenshot (FT)83.42.33 / 5.4284.52.34 / 5.4758.22.34 / 5.81
+ Linear patch embed.83.41.65 / 5.4284.51.64 / 5.4758.21.66 / 5.81
+ Spatial pruning82.81.00 / 2.9683.81.17 / 3.6457.11.08 / 3.48
w/o grid keep82.60.84 / 2.4983.51.03 / 3.0755.00.91 / 2.60
+ Temporal reuse81.80.66 / 1.9782.90.84 / 2.6156.20.63 / 1.87
+ Consistency adapter82.10.66 / 1.9783.30.84 / 2.6157.60.63 / 1.87

Table 6. Ablation. Components of STREAMEMBED added one at a time to the fine-tuned Qwen3-VL-2B screenshot index. Latency is GPU / CPU in sec per screenshot.

Speedup and redundancy. STREAMEMBED computes 1.95, 2.67, and 1.79 TFLOPs per screenshot on SME-H1, SME-H2, and SME-Agent, or 32%, 43%, and 29% of the 6.17 TFLOPs of a full screenshot. This follows the redundancy of each history, where 72.4%, 63.3%, and 75.6% of blocks repeat a neighbor or are unchanged (Appendix § 16), and grid keep, refresh, and the suffix add a little work to the remaining 24–37%. On the GPU, STREAMEMBED is accordingly 3.5×3.5 \times, 2.8×2.8 \times, and 3.7×3.7 \times faster than the released model. Its 99th-percentile latency, 1.47–1.77 s, stays below the mean of the released full screenshot, 2.33–2.34 s.

Memory. By our count from the model configuration, the cache of temporal reuse holds at most about 0.46 GB in fp16 for Qwen3-VL-2B and does not grow with the history.

Sensitivity to Thresholds

Table 11 and Table 12 show how R@10 and computation change around the settings of STREAMEMBED, varying one at a time. STREAMEMBED is not sensitive to its thresholds. Changing the refresh settings moves R@10 by at most 0.2 and the tile size by at most 1.4, and even raising both pixel tolerances 16-fold costs at most 2.0 R@10 while cutting computation up to 2.0×2.0 \times. The settings are chosen on the SME-H1 validation questions (§ 6.1), and the tables report R@10 on the test questions. Every row uses the fine-tuned Qwen3-VL-2B with all parts of STREAMEMBED, and computation is in TFLOPs per screenshot (TF).

TileSME-H1 R@10SME-H1 TFSME-H2 R@10SME-H2 TFSME-Agent R@10SME-Agent TF
None82.41.6882.92.3755.51.59
2×282.02.1383.12.8856.91.92
3×3 (STREAMEMBED)82.11.9583.32.6757.61.79
4×4––––56.21.74

Table 11. Grid keep tile size. R@10 and computation (TF). None uses the neighbor rule alone, and – was not run.

SME-H2 R@10SME-H2 TFSME-Agent R@10SME-Agent TF
τ=δ=8\tau= \delta= 8 (STREAMEMBED)83.62.6757.41.79
τ=δ=16\tau= \delta= 1683.02.3456.41.59
τ=δ=32\tau= \delta= 3283.11.9955.81.31
τ=δ=64\tau= \delta= 6482.81.8255.81.04
τ=δ=128\tau= \delta= 12882.61.5555.40.88
δ=16\delta= 1683.42.45––
δ=32\delta= 3283.22.21––
δ=64\delta= 6482.52.13––
R=20R = 2083.52.61––
R=40R = 4083.42.58––
No ρ\rho refresh83.62.70––

Table 12. Pixel tolerances and refresh. R@10 and computation (TF). The top block raises τ\tau and δ\delta together, and the bottom block changes one reuse setting on SME-H2. – was not run.

Grid keep. Without grid keep, SME-Agent loses 2.1 R@10, while SME-H1 and SME-H2 change by at most 0.4. Tiles of 3×3 blocks match or exceed the accuracy of 2×2 tiles on every history with less computation, and 4×4 tiles lose 1.4 R@10 on SME-Agent. With 3×3 tiles, STREAMEMBED keeps 63%, 69%, and 71% of the blocks on SME-H1, SME-H2, and SME-Agent.

Pixel tolerances. The runs of Table 12 encode screenshots in fp16, so their reference row differs from the main results by up to 0.3 R@10. Raising both tolerances from 8 to 128 levels lowers computation 1.7× on SME-H2 and 2.0× on SME-Agent at 1.0 and 2.0 R@10. Over all settings we tried, R@10 changes by at most 1.1 on SME-H2 and 2.0 on SME-Agent. Changing RR or ρ\rho moves R@10 by at most 0.2 and computation by at most 4%.

Dropping more blocks. To drop a set fraction of blocks, as in Figure 5, we choose τ\tau for each screenshot by binary search. Drop rates of 50% and 60% give 81.9 and 81.2 R@10 on SME-H1 and 55.8 and 52.4 on SME-Agent, against 82.3 and 57.4 for the fp16 reference.

Redundancy by Source

Table 13 gives the redundancy of §3.2 per source, including the three histories of SCREENMEMEVAL, measured with the

SourcePairsSpatialTemporalCombined
Screen
SME-H164,85546.925.572.4
SME-H211,84541.421.963.3
SME-Agent6,09839.436.275.6
WebLINX2,97552.424.576.9
GUI-World19458.28.466.6
Natural video
Wikimedia Commons1,06517.61.318.9
YouTube47010.10.610.7
EPIC-KITCHENS3005.60.15.7

Table 13. Redundancy by source. Share of blocks (%) that repeat a neighbor (Spatial), or else are unchanged from the frame 5 s earlier (Temporal), and their sum (Combined), averaged over consecutive frame pairs. SME-Agent uses 142 of its 787 recordings.

same blocks, interval, and tolerance. The public sources are WebLINX (Lü et al., 2024), GUI-World (Chen et al., [8]), and natural video from Wikimedia Commons, YouTube, and EPIC-KITCHENS-100 (Damen et al., [12]).

Qualitative Examples

Table 14 shows the screenshot that each index ranks first for four single-recording questions, three from SME-H1 and one from SME-H2. All three indexes use the fine-tuned Qwen3-VL-2B, and the text index embeds the Apple Vision OCR text of each screenshot. We chose the questions among those on which the text index and STREAMEMBED rank different recordings first, so they are not a random sample; Figure 4 gives the accuracy over all questions.

Text indexScreenshot indexStreamEmbed
0.62–0.65 s per screenshot2.33–2.34 s per screenshot0.66–0.84 s per screenshot
Question. Which smartphone model did I add to my cart?
Question. Which work app did I update my profile photo in?
Question. Which all-in-one desktop did I look up the starting price of?
Question. Which television did I add to my newly created wishlist?

Table 14. Screenshots ranked first for four questions. For each single-recording question, the screenshot that each index ranks first, framed in green if it belongs to a gold recording and in red otherwise. All indexes use the fine-tuned Qwen3-VL-2B. Below each index is its mean GPU time to index one screenshot on SME-H1 and SME-H2 (Figure 4).

On all four questions, the screenshot index and STREAMEMBED rank a gold screenshot first, and STREAMEMBED indexes a screenshot in 0.66–0.84 s, against 2.33–2.34 s for the screenshot index. The text index ranks the gold recording 16th to 752nd and instead ranks first a screenshot of another task, such as setting a desktop background for the profile-photo question or adding an LG television to a Costco cart for the wishlist question.

References

  1. [2]Agashe, S., Han, J., Gan, S., Yang, J., Li, A., and Wang, X. Agent S: An open agentic framework that uses computers like a human. In International Conference on Learning Representations, volume 2025, pp. 22924–22946, 2025.arxiv.org/abs/2410.08164
  2. [3]Aggarwal, P. and Welleck, S. Visual token routing: Learning to keep tokens that matter, 2026. URL https://arxiv.org/abs/2611.06854.
  3. [4]Anthropic. System card: Claude Opus 5.5. https://www-cdn.anthropic.com/fc1b44717c85dc068bc6ba5024219938094694bd/Claude%20Opus%205.5%20System%20Card.pdf, September 2026. Accessed: 2026-10-19.
  4. [6]Busch, M., Gade, K., Larson, B., Lok, P., Luckenbill, S., and Lin, J. Earlybird: Real-time search at twitter. In 2012 ieee 28th international conference on data engineering, pp. 1360–1369. IEEE, 2012.
  5. [7]Cai, D., Wang, S., Peng, C., Zhang, Z., Lu, Z., Qi, T., Lane, N. D., and Xu, M. Ubiquitous memory augmentation via mobile multimodal embedding system. Nature Communications, 16(1):5339, 2025.DOI
  6. [8]Chen, D., Huang, Y., Wu, S., Tang, J., Zhou, H., Zhang, Q., He, Z., Bai, Y., Gao, C., Chen, L., Li, Y., Wang, C., Yu, Y., Zhou, T., Li, Z., Gui, Y., Wan, Y., Zhou, P., Gao, J., and Sun, L. Gui-world: A video benchmark and dataset for multimodal gui-oriented understanding. In International Conference on Learning Representations, volume 2025, pp. 22812–22864, 2025.arxiv.org/abs/2406.10819
  7. [9]Chen, J., Lv, Z., Wu, S., Lin, K. Q., Song, C., Gao, D., Liu, J.-W., Gao, Z., Mao, D., and Shou, M. Z. Videollm-online: Online video large language model for streaming video. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 18407–18418. IEEE, 2024.arxiv.org/abs/2406.11816
  8. [10]Choudhury, R., Zhu, G., Liu, S., Niinuma, K., Kitani, K. M., and Jeni, L. A. Don’t look twice: Faster video transformers with run-length tokenization. Advances in Neural Information Processing Systems, 37:28127–28149, 2024.arxiv.org/abs/2411.05222
  9. [11]Cormack, G. V., Clarke, C. L., and Buettcher, S. Reciprocal rank fusion outperforms concordet and individual rank learning methods. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pp. 758–759, 2009.DOI
  10. [12]Damen, D., Doughty, H., Farinella, G. M., Furnari, A., Kazakos, E., Ma, J., Moltisanti, D., Munro, J., Perrett, T., Price, W., and Wray, M. Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100. International Journal of Computer Vision, 130(1):33–55, 2022.DOI
  11. [14]Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021.
  12. [15]Douze, M., Guhzva, A., Deng, C., Johnson, J., Szilvasy, G., Mazaré, P.-E., Lomeli, M., Hosseini, L., and Jégou, H. The Faiss library. arXiv preprint arXiv:2401.08281, 2024.arxiv.org/abs/2401.08281
  13. [16]Dumais, S., Cutrell, E., Cadiz, J. J., Jancke, G., Sarin, R., and Robbins, D. C. Stuff i’ve seen: a system for personal information retrieval and re-use. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pp. 72–79, 2003.
  14. [17]Fang, Z., Wang, J., Wang, L., Zhang, L., Yang, Y., and Liu, Z. [SEED]: Self-supervised distillation for visual representation. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=AHm3dbp7D1D.arxiv.org/abs/2101.04731
  15. [18]Faysse, M., Sibille, H., Wu, T., Omrani, B., Viaud, G., Hudelot, C., and Colombo, P. Colpali: Efficient document retrieval with vision language models. In International Conference on Learning Representations, volume 2025, pp. 61424–61449, 2025.arxiv.org/abs/2407.01449
  16. [19]Hofstätter, S., Althammer, S., Schröder, M., Sertkan, M., and Hanbury, A. Improving efficient neural ranking models with cross-architecture knowledge distillation. arXiv preprint arXiv:2010.02666, 2020.arxiv.org/abs/2010.02666
  17. [20]Howland, A., Thinnappan, A., and Mohammed, J. S. Computer use data - paradigm shift ai, 2025. URL https://huggingface.co/datasets/anaisleila/computer-use-data-psai.
  18. [21]Hsieh, K., Ananthanarayanan, G., Bodik, P., Venkataraman, S., Bahl, P., Philipose, M., Gibbons, P. B., and Mutlu, O. Focus: Querying large video datasets with low latency and low cost. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pp. 269–286, 2018.arxiv.org/abs/1801.03493
  19. [24]Hwang, J., Kim, D., Lee, S., Kim, Y., Heo, G., Kim, H., Jeong, Y., Meaza, T., Park, E., Ahn, J., and Park, J. Déjà vu: Efficient video-language query engine with learning-based inter-frame computation reuse. arXiv preprint arXiv:2506.14107, 2025.arxiv.org/abs/2506.14107
  20. [25]Jang, L. K., Jang, A. K., Koh, J. Y., and Salakhutdinov, R. Mypcbench: A benchmark for personally intelligent computer-use agents. arXiv preprint arXiv:2606.16748, 2026.arxiv.org/abs/2606.16748
  21. [26]Järvelin, K. and Kekäläinen, J. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS), 20(4):422–446, 2002.DOI
  22. [27]Jégou, H., Douze, M., and Schmid, C. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1):117–128, 2011. doi: 10.1109/TPAMI.2010.57.DOI
  23. [28]Jeong, S., Kim, K., Baek, J., and Hwang, S. J. Videorag: Retrieval-augmented generation over video corpus. In Findings of the Association for Computational Linguistics: ACL 2025, pp. 21278–21298, 2025.arxiv.org/abs/2501.05874
  24. [29]Jiao, X., Yin, Y., Shang, L., Jiang, X., Chen, X., Li, L., Wang, F., and Liu, Q. Tinybert: Distilling bert for natural language understanding. In Findings of the association for computational linguistics: EMNLP 2020, pp. 4163–4174, 2020.arxiv.org/abs/1909.10351
  25. [30]Kang, D., Guibas, J., Bailis, P., Hashimoto, T., and Zaharia, M. Semantic indexes for machine learning-based queries over unstructured data. arXiv preprint arXiv:2009.04540, 2020.arxiv.org/abs/2009.04540
  26. [31]Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp. 6769–6781, 2020.arxiv.org/abs/2004.04906
  27. [32]Kim, J., Ko, D., and Kim, G. Dynamicer: Resolving emerging mentions to dynamic entities for rag. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 13752–13770, 2024.arxiv.org/abs/2410.11494
  28. [33]Kishore, V., Wan, C., Lovelace, J., Artzi, Y., and Weinberger, K. Q. Incdsi: Incrementally updatable document retrieval. In International conference on machine learning, pp. 17122–17134. PMLR, 2023.arxiv.org/abs/2307.10323
  29. [40]Luo, J., Liao, Z., Teng, J., and Wang, Y. Causalcache: Conditional high-fidelity restoration for long-horizon gui agents. arXiv preprint arXiv:2608.22577, 2026.arxiv.org/abs/2608.22577
  30. [41]Lyu, X., Duan, M., Shao, R., Koh, P. W., and Min, S. Frustratingly simple retrieval improves challenging, reasoning-intensive benchmarks. In International Conference on Learning Representations, volume 2026, pp. 10248–10285, 2026.arxiv.org/abs/2507.01297
  31. [43]Malkov, Y. A. and Yashunin, D. A. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(4):824–836, 2020. doi: 10.1109/TPAMI.2018.2889473.DOI
  32. [44]McGhee, J. remi: An open source approach to locally record and enable searching everything you view on your Mac. https://github.com/jasonjmcghee/rem, 2023. GitHub repository. Accessed: 2026-10-15.
  33. [46]Ning, Z., Liu, G., Jin, Q., Li, C., Ding, W., Guo, M., and Zhao, J. Livevlm: Efficient online video understanding via streaming-oriented kv cache and retrieval. In Proceedings of the 63rd ACM/IEEE Design Automation Conference, pp. 1–7, 2026.DOI
  34. [47]OpenAI. Introducing GPT-6.1 Sol. https://openai.com/index/introducing-gpt-6-1-sol/, 2026a. Released Sep. 29, 2026. Accessed: 2026-10-29.
  35. [48]OpenAI. GPT-6 Astra: A new generation of intelligence. https://openai.com/index/gpt-6-astra/, 2026b. Released Sep. 3, 2026. Accessed: 2026-10-29.
  36. [49]OpenRecall. OpenRecall: A fully open-source, privacy-first alternative to Microsoft’s Windows Recall. https://github.com/openrecall/openrecall, 2024. GitHub repository. Accessed: 2026-10-15.
  37. [50]Packer, C., Wooders, S., Lin, K., Fang, V., Patil, S. G., Stoica, I., and Gonzalez, J. E. Memgpt: Towards llms as operating systems. arXiv preprint arXiv:2310.08560, 2023.arxiv.org/abs/2310.08560
  38. [51]RapidAI, T. Rapid OCR: Ocr toolbox. https://github.com/RapidAI/RapidOCR, 2021.
  39. [52]Robertson, S. and Zaragoza, H. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4):333–389, 2009. doi: 10.1 561/1500000019.
  40. [53]Sanh, V., Debut, L., Chaumond, J., and Wolf, T. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019.arxiv.org/abs/1910.01108
  41. [55]Shakir, A., Aarsen, T., and Lee, S. Binary and scalar embedding quantization for significantly faster & cheaper retrieval. https://huggingface.co/blog/embedding-quantization, 2024. Hugging Face Blog.
  42. [56]Soldan, M., Heilbron, F. C., Ghanem, B., Sivic, J., and Russell, B. Residualvit for efficient temporally dense video encoding. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 22305–22315. IEEE, 2025.arxiv.org/abs/2509.13255
  43. [58]Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., and Gurevych, I. Beir: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. arXiv preprint arXiv:2104.08663, 2021.arxiv.org/abs/2104.08663
  44. [59]Tschannen, M., Gritsenko, A., Wang, X., Naeem, M. F., Alabdulmohsin, I., Parthasarathy, N., Evans, T., Beyer, L., Xia, Y., Mustafa, B., Hénaff, O., Harmsen, J., Steiner, A., and Zhai, X. SigLIP 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502.14786, 2025.arxiv.org/abs/2502.14786
  45. [60]Vultr. VultronRetrieverFlash-Qwen3.5-0.8B. https://huggingface.co/vultr/VultronRetrieverFlash-Qwen3.5-0.8B, 2026. Hugging Face model card. Accessed: 2026-10-28.
  46. [61]Wang, N., Jin, Z., Chen, C., and Lu, H. Pixelprune: Pixel-level adaptive visual token reduction via predictive coding. arXiv preprint arXiv:2604.00886, 2026a.arxiv.org/abs/2604.00886
  47. [62]Wang, Y. and Chen, X. Mirix: Multi-agent memory system for llm-based agents. arXiv preprint arXiv:2507.07957, 2025.
  48. [63]Wang, Y., Li, Z., Liu, S., Wu, Y., Mao, Z., Zhao, Y., Yan, X., Xu, Z., Zhou, Y., Stoica, I., Min, S., Zaharia, M., and Gonzalez, J. E. Leann: A low-storage overhead vector index. Proceedings of Machine Learning and Systems, 8:944–961, 2026b.arxiv.org/abs/2506.08276
  49. [64]Wang, Y., Li, Z., Wang, Z., Teiletche, P., Jin, L., Zaharia, M., Gonzalez, J. E., and Min, S. Efficient retrieval: Web screenshots beat text for retrieval-augmented generation. arXiv preprint arXiv:2606.28344, 2026c.
  50. [65]Wang, Y., Liu, X., Gui, X., Lin, X., Yang, B., Liao, C., Chen, T., and Zhang, L. Accelerating streaming video large language models via hierarchical token compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18523–18533, 2026d.arxiv.org/abs/2512.00891
  51. [66]Wang, Z. Z., Mao, J., Fried, D., and Neubig, G. Agent workflow memory. arXiv preprint arXiv:2409.07429, 2024.arxiv.org/abs/2409.07429
  52. [69]Xie, T., Zhang, D., Chen, J., Li, X., Zhao, S., Cao, R., Hua, T. J., Cheng, Z., Shin, D., Lei, F., Liu, Y., Xu, Y., Zhou, S., Savarese, S., Xiong, C., Zhong, V., and Yu, T. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments. Advances in Neural Information Processing Systems, 37:52040–52094, 2024.arxiv.org/abs/2404.07972
  53. [73]Yamada, I., Asai, A., and Hajishirzi, H. Efficient passage retrieval with hashing for open-domain question answering. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pp. 979–986, 2021.arxiv.org/abs/2106.00882
  54. [74]Yao, L., Li, Y., Wei, Y., Li, L., Ren, S., Liu, Y., Ouyang, K., Wang, L., Li, S., Li, S., Kong, L., Liu, Q., Zhang, Y., and Sun, X. Timechat-online: 80% visual tokens are naturally redundant in streaming videos. In Proceedings of the 33rd ACM International Conference on Multimedia, pp. 10807–10816, 2025.arxiv.org/abs/2504.17343
  55. [75]Yin, H., Wen, Z., Cao, J., Yuan, B., and Yang, R. Focal: Filtered on-device continuous activity logging for efficient personal desktop summarization. arXiv preprint arXiv:2604.19541, 2026.arxiv.org/abs/2604.19541
  56. [76]Zhang, X., Zhang, Y., Long, D., Xie, W., Dai, Z., Tang, J., Lin, H., Yang, B., Xie, P., Huang, F., Zhang, M., Li, W., and Zhang, M. mgte: Generalized long-context text representation and reranking models for multilingual text retrieval. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pp. 1393–1412, 2024.arxiv.org/abs/2407.19669
  57. [77]Zhang, Y., Li, M., Long, D., Zhang, X., Lin, H., Yang, B., Xie, P., Yang, A., Liu, D., Lin, J., Huang, F., and Zhou, J. Qwen3 Embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176, 2025.arxiv.org/abs/2506.05176

Paper details

vlnc:W_zdwqpy8k
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents