betaFeedback

WorksComputer Science

Enhancing Local and Global Structure in Compressed AI Agent Memory

Abstract

AI agents store text as embedding vectors and compress them at scale. A compression is accepted when each item's nearest neighbors survive, but agents also consolidate, delete, and compare memories, tasks that depend on which items lie far apart. We evaluate both ends with two label-free fidelities from our prior work: the local fidelity LL at 10 neighbors and the global fidelity GG at 30 to 50 percent of the memory. At 96 bytes per passage, rescored binary codes reach an LL of 0.97 yet move 10.1 percent of passages to another theme; PCA with 8-bit codes, at an LL of 0.77, moves 1.7 percent. Rescoring leaves the stored codes, and thus GG, unchanged. We prove that across the GG band, 1−νk1-ν_k is at most 2 to 2.18 times the square root of the near-far rank-violation rate, and that this rate follows the distance noise of the compression (correlation 0.979). Our locality-aware quantization (LAQ) turns the bound into a hinge objective on the codebooks of an OPQ index, keeping its bytes, rotation, and search. To our knowledge, LAQ is the first quantizer whose objective comes from a proven bound on the far end. Reconstruction alone raises GG in 0 of 52 held-out runs; the far hinge raises it in all 52. LAQ raises GG in all 52 (p=8.5×10−4p=8.5\times 10^{-4} over 13 memory-encoder groups), including the external FinDER memory, has the highest GG of seven equal-size quantizers in all 12 Multi-Doc-2025 runs, scales to a million passages, and keeps evidence recall.

Introduction

Language-model agents remember by embedding what they read, say, and do. When they need an item, they recall the nearest embeddings [1]. Managed agent memories extract records from conversations, retrieve them by semantic search, and consolidate them over time [2]. At millions of vectors, every byte per vector costs memory, latency, and money. The vectors are therefore compressed: by truncating Matryoshka embeddings [3], by projection, by scalar or binary codes, or by product quantization [4]. Cloud services offer each of these as a switch [5, 6].

How should an agent memory be compressed so that it keeps both what lies near and what lies far? Today, a compression is accepted by its nearest neighbors. Vendors report that binary embeddings retain 97 to 98.5 percent of retrieval accuracy [5] and evaluate quantized indexes by recall@100 [6]. At 96 bytes per passage on FinQA, rescored binary codes keep LL at 0.97 and top-10 agreement at 0.87, yet move 10.1 percent of passages to another theme (Section 6). Embedding compression studies report nDCG@10 [7] or benchmark task scores [8]. Yet an agent also acts on what lies far apart: it consolidates a year of filings into themes [9, 10], deletes every item of a client or a session on request, and maps how companies and sectors relate. Because these operations compare each item with the whole memory, they decide what a sector brief says and what a deletion removes. Keeping this far end is difficult for two reasons: a private agent memory has no labels to test it with, and quantizers are trained for reconstruction, nearest neighbors, or query relevance. Our idea is to measure the far end with the same statistic that measures the near end, prove which pairwise order controls it, and train the codebooks on that order.

At the same bytes and search, our proposed locality-aware quantization (LAQ) keeps the nearest neighbors of a held-out financial memory while moving far fewer passages to another theme (Figure 1a–c). In such a memory, passages from a single company form islands: in the outer fifth of the map, all 10 nearest neighbors of each passage belong to its own company. Passages from different banks meet in shared themes. At 128 bytes per passage, an optimized product quantization (OPQ) index [16] keeps local fidelity LL at 0.85 but moves 224 of the 2,688 passages to another theme. LAQ keeps the bytes, the search, and LL at 0.85, while moving only 94 passages.

Measurement. One label-free profile reads both ends of a compressed memory. Following our earlier work [17], we evaluate the two ends of a chance-corrected locality profile [18]: the near end (local structure) and the far end (global structure). Local fidelity LL is chance-corrected recall@10. Global fidelity GG averages, over 30 to 50 percent of the memory, the agreement on which items lie in each item’s nearer part. At its upper edge, the profile measures agreement on the near and far halves, which under a Gaussian copula equals Kendall’s τ\tau of the distances. Both fidelities require only the vectors before and after compression, and GG is estimated from a subsample of 2,000 items, however large the memory.

Learning. We prove that rank violations among cross pairs control the profile at every scale. A cross pair consists of one near item and one far item relative to an anchor. Let νk\nu_k be the profile at scale kk and N=n−1N = n - 1 for a memory of nn items. Then 1−νk1-\nu_k is at most N/k(N−k)N/\sqrt{k(N-k)} times the square root of the cross-pair violation rate. The leading factor

stays between 2 and 2.18 across the GG band. A hinge loss on distance gaps bounds this violation rate. LAQ lowers this hinge at middle and far scales by refining the codebooks of an OPQ index without labels or queries. The principal component analysis (PCA) basis, rotation, bytes per item, and search remain identical to those of the OPQ index.

What is new. To our knowledge, LAQ is the first quantizer trained on the far end of a memory, and its objective comes from a bound we prove. The closest learned quantizers train codebooks with pairwise or triplet losses. PQN uses triplets from class labels [19], and spreading vectors pair a nearest-neighbor triplet loss with an entropy term [20]. DPQ trains product codes end to end via straight-through assignments [21], and JPQ and Distill-VQ train with queries and relevance labels [22, 23]. LAQ differs in target, input, and output. Its target is the rank order at 30 to 50 percent of the memory, selected via our theoretical bound; its input is the unlabeled memory alone; and its output modifies only the codebook values of an OPQ index. We arrived at νk\nu_k from our DLP curve [18]. Rank-based quality criteria [24, 25] also count kept neighbors across scales and summarize them by an area that weights the near end. To our knowledge, the far end had no proven meaning until our earlier work showed that its midpoint is a rank correlation of distances [17] (Propositions 2–3). This paper takes the far end from a measure to a target. We prove that rank violations bound the profile at every scale (Theorem 1), that under noisy order the profile is fixed by one ratio, the distance noise of a compression against the spread of distances (Proposition 4, Corollary 2), and that at an OPQ start the far hinge falls at no first-order cost in reconstruction (Proposition 5). To our knowledge, no earlier quantizer, learned or not, trains its codes on the far end of this curve.

Contributions. Our contributions, in order of importance, are:

  1. A quantizer derived from a bound. A new rank-violation bound ties the profile at every scale to the order of cross pairs, and a hinge surrogate makes it trainable. A noisy-order proposition and its corollary predict the violation rate and the profile in the GG band from one ratio, the distance noise of the compression against the spread of distances. A further proposition shows that, at the OPQ start, the far hinge falls at no first-order cost in reconstruction, and the straight-through steps of training meet its condition at every start. Each result comes with a proof. LAQ turns the bound into a label-free objective on the codebooks of an OPQ index and replaces that index at the same bytes and search. The ablation confirms that the gain comes from the far hinge: reconstruction alone raises GG in 0 of 52 held-out runs, the near hinge raises it by 0.0058 on average, and the far hinge raises it in all 52, by 0.0127.

  1. Held-out evidence that it works. On held-out companies and conversations across three memories, LAQ raises GG in all 40 runs and LL in 34. The gain is largest with Amazon Titan V2 [5], where GG rises by 0.032 to 0.038 on every split, two to five times the gain with the three retrieval-tuned encoders. On FinDER [26], an external financial memory run once after the recipe was fixed, LAQ raises both fidelities in all 12 runs. On Multi-Doc-2025 it lifts the lowest theme agreement from 0.80 to 0.89 and has the highest GG among seven quantizers of equal size. Paired tests confirm the gain: GG rises in all 13 memory–encoder groups (p=8.5×10−4p = 8.5 \times10^{-4}, sign test), LAQ has a higher GG than each of six other quantizers in all 12 Multi-Doc-2025 runs, and evidence recall@10 equals that of OPQ (p=0.84p = 0.84). On a million passages, one training run takes 175 seconds and a second run reproduces the codebooks bit for bit.

  1. Why the gain matters. Each end governs its own operations. Across two agent memories and six encoders, retrieval tracks LL (partial rank correlation 0.83 to 0.99 given GG). With the retrieval-tuned encoders, consolidation and deletion track GG (0.55 to 0.93 given LL). Rescoring repairs the near end at query time but leaves the far end of the stored codes unchanged. LAQ applies wherever a vector index also serves clustering, deduplication, or mapping.

Section 2 reviews related work. Section 3 presents the measurement theory and the bound, and Section 4 introduces the LAQ quantizer. Section 5 describes the memories, encoders, and operations. Section 6 shows what each end measures, and Section 7 evaluates LAQ on held-out memories.

Related work

We draw on three lines of research and apply them to agent memory: vector quantization, the evaluation of compressed vectors, and local and global structure.

Vector quantization. Quantizers compress vectors through subspace decomposition, additive codewords, or binary projections. Product quantization (PQ) splits a vector into subvectors and codes each separately [4], while OPQ learns a rotation to balance variance across subvectors [16]. Residual quantization (RQ) [27] and local search quantization (LSQ++) [28] represent a vector as a sum of codewords, whereas ScaNN weights quantization error by its impact on query inner products [29]. For binary representations, iterative quantization (ITQ) rotates the data before taking signs [30], and RaBitQ generates randomized binary codes with a theoretical error bound on distance estimates [31]. Learned quantizers train codebooks end to end. PQN applies an asymmetric triplet loss on labeled images [19]. UNQ trains a nonlinear encoder for nearest-neighbor search [32]. Spreading vectors combine a nearest-neighbor triplet loss with an entropy regularizer [20]. DPQ trains product quantization end to end via soft assignments and straight-through estimators [21]. In dense retrieval, JPQ, RepCONC, and Distill-VQ optimize quantizers using queries and relevance labels or teacher rankings [22, 23, 33]. Unlike these methods, our proposed LAQ trains codebooks for far-end rank order using the unlabeled memory alone.

Evaluation of compressed vectors. Quantized indexes are typically evaluated by nearest-neighbor recall [6, 34], retrieval quality [5, 7], or downstream task scores [8]. Kisako et al. [8] conjecture that both clustering and retrieval depend on neighborhood geometry, and Kuffo and Boncz [35] find that one-bit codes support the internal clustering an index uses to partition its vectors. Earlier, Turaga et al. [36] showed that fitting a moment-preserving quantizer per cluster preserves a known kk-means partition. In contrast, our measure asks whether the compressed memory keeps both the partition and the global arrangement of the uncompressed memory, with no partition given in advance.

Local and global structure. Rank-based quality criteria evaluate neighborhood preservation across scales [24, 25]. Lueks et al. [37] split the co-ranking curve into local and global regimes, and recent visualization methods such as DREAMS and CMET balance or score both ends [38, 39]. Global embedding structure is also commonly scored by pairwise distance correlation and triplet accuracy [40, 41]. In our prior work, we introduced the degree of locality preservation [18]. In a subsequent study [17], we defined the local fidelity LL, global fidelity GG, and entity fidelity on its chance-corrected profile and proved that the profile’s midpoint converges to a rank correlation of distances. We also derived the subsample estimator of GG and applied these fidelities to two-dimensional maps and compressed agent memory. Building on that measurement framework, the present paper proves which pairwise order controls the profile, turns the resulting bound into a trainable quantizer, and evaluates both ends on held-out memories.

Agent memory. Benchmarks such as LoCoMo [15] and LongMemEval [42] evaluate whether agents can recall facts from long interactions. Agent memory systems consolidate new records against the most similar stored entries [2], while corpus-level frameworks cluster embeddings into themes and summaries [9, 10, 43]. Whereas these systems compress memory at the text level through summarization and consolidation, we study the compression of the underlying embedding vectors and the memory-wide operations that depend on them.

Two-end fidelity of a compressed memory

One chance-corrected profile measures both ends of a compressed memory, while the relative order of near and far items controls this profile at every scale. This section defines the profile, proves what its two ends measure, defines the local and global fidelities LL and GG, and proves the bound that LAQ optimizes in Section 4.

Setting

A memory holds nn items with vectors x1,…,xn∈Rdx_1,\ldots,x_n \in\mathbb{R}^d, where dd is the embedding dimension. A compression maps each xix_i to a stored code. We write ziz_i for the vector with which the memory computes distances: the decoded vector for scalar, PCA, and product codes, or the ±1\pm1 vector for binary codes compared by Hamming distance. All other distances are Euclidean. Let N=n−1N=n-1. For item ii, let rix(j)∈{1,…,N}r_i^x(j) \in\{1,\ldots,N\} be the rank of item j≠ij \ne i among the other items by increasing distance to xix_i, with ties broken by index, and let riz(j)r_i^z(j) be the corresponding rank among the compressed vectors. For 1≤k≤N−11 \le k \le N-1, the kk-neighborhoods are Nkx(i)={j:rix(j)≤k}\mathcal{N}_k^x(i)=\{j:r_i^x(j)\le k\} and Nkz(i)={j:riz(j)≤k}\mathcal{N}_k^z(i)=\{j:r_i^z(j)\le k\}.

The locality profile

At every scale, the profile counts how many neighbors a compression keeps.

Definition 1 (Locality profile). The degree of locality preservation at scale kk is

DLPk=1nk∑i=1n∣Nkx(i)∩Nkz(i)∣,(1)\mathrm{DLP}_k=\frac{1}{nk}\sum_{i=1}^{n}\left|\mathcal{N}_k^x(i)\cap\mathcal{N}_k^z(i)\right|, \tag*{(1)}

and its chance-corrected value is

νk=N DLPk−kN−k.(2)\nu_k=\frac{N\,\mathrm{DLP}_k-k}{N-k}. \tag*{(2)}

The locality profile is the curve k↦νkk\mapsto\nu_k, read over the relative scale k/nk/n.

We introduced DLPk\mathrm{DLP}_k in our earlier work on the explainability of manifold learning [18]. Rank-based quality criteria for dimensionality reduction [24, 25] also count kept neighbors across scales. At a single scale kk with the uncompressed memory as reference, DLPk\mathrm{DLP}_k is the recall@k used to evaluate approximate nearest-neighbor indexes [34]. Reading this quantity across every scale yields the profile, and the results below say what its two ends measure. Propositions 1–3 and Definition 2 are from our earlier work on nonlinear embeddings [17]; we give the proofs so that the paper is self-contained.

The next result gives the chance level and invariance of the profile: a compression that keeps neighbors at random scores 0, and codes whose distances differ in scale are compared on one statistic.

Proposition 1 (Chance level and invariance). (i) If each Nkz(i)\mathcal{N}_k^z(i) is a uniformly random kk-subset of the other NN items, independent of the uncompressed vectors, then DLPk=k/N\mathrm{DLP}_k=k/N and νk=0\nu_k=0. (ii) A compression that keeps every ranking has νk=1\nu_k=1. (iii) νk\nu_k is invariant under any rotation, reflection, translation, or uniform scaling of the ziz_i, and under any strictly increasing transformation of either distance.

All proofs except that of Theorem 1 are in Appendix A. Invariance (ii) allows one statistic to compare Euclidean distances between decoded vectors, Hamming distances between bits, and distances after a learned rotation. All 486 compressions evaluated in Section 6 lie above the chance level; the lowest GG among them, 0.04, belongs to UMAP [44] at 16 dimensions (MobileCLIP2 [45], FinQA).

The next result shows what the profile reads at its midpoint: for every item, which half of the memory lies near.

Proposition 2 (Near–far duality). Let Fkx(i)\mathcal{F}_k^x(i) and Fkz(i)\mathcal{F}_k^z(i) be the complements of Nkx(i)\mathcal{N}_k^x(i) and Nkz(i)\mathcal{N}_k^z(i) among the other NN items. Then ∣Fkx(i)∩Fkz(i)∣=N−2k+∣Nkx(i)∩Nkz(i)∣|\mathcal{F}_k^x(i)\cap\mathcal{F}_k^z(i)|=N-2k+|\mathcal{N}_k^x(i)\cap\mathcal{N}_k^z(i)|. For k∗=N/2k^*=N/2 with NN even,

νk∗=2A−1,\nu_{k^*}=2A-1,
A=1nN∑i=1n∣{j≠i:[rix(j)≤k∗]=[riz(j)≤k∗]}∣,(3)A=\frac{1}{nN}\sum_{i=1}^{n}\left|\left\{j\ne i:\left[r_i^x(j)\le k^*\right]=\left[r_i^z(j)\le k^*\right]\right\}\right|, \tag*{(3)}

where AA is the probability that the uncompressed and the compressed memory agree on whether a random other item lies in the nearer or the farther half.

While at small kk the profile asks whether an item keeps its nearest neighbors, at k∗k^* it asks which half of the memory is near and which is far for every item. That question is decided by large distances, and thus by the relative positions of themes, sessions, and companies. The median split also acts as a rank correlation of all distances to an item, and under a Gaussian copula the two coincide.

Proposition 3 (Median split and rank correlation). Fix item ii and let uu be a uniformly random other item. Suppose that, after strictly increasing transformations, the pair of distances (∥xi−xu∥,∥zi−zu∥)(\lVert x_i-x_u\rVert,\lVert z_i-z_u\rVert) is bivariate normal with correlation γi\gamma_i, that is, the pair has a Gaussian copula. Then for large nn the contribution 2Ai−12A_i-1 of item ii to νk⋆\nu_{k^\star}, with AiA_i its term in (3), has expectation

E(2Ai−1)=2πarcsin⁡γi,(4)\mathbb{E}(2A_i-1)=\frac{2}{\pi}\arcsin\gamma_i, \tag*{(4)}

which is Kendall’s τ\tau of the pair and a strictly increasing function of its Spearman correlation ρiS=6πarcsin⁡γi2\rho_i^{S}=\frac{6}{\pi}\arcsin\frac{\gamma_i}{2}.

Compressed memories follow this model closely. Across all 486 compressions, converting the Spearman correlation of all distances into 2πarcsin⁡γ\frac{2}{\pi}\arcsin\gamma via Proposition 3 yields a rank correlation of 0.98 with the measured median split, and the two differ by only 0.03 on average.

Local and global fidelity

Two numbers summarize the profile, one read at each end.

Definition 2 (Local and global fidelity). The local fidelity and the global fidelity of a compressed memory are

L=ν10,G=1∣K∣∑k∈Kνk,K={k:0.3≤k/n≤0.5}.(5)L=\nu_{10},\quad G=\frac{1}{|\mathcal{K}|}\sum_{k\in\mathcal{K}}\nu_k,\quad\mathcal{K}=\{k:0.3\leq k/n\leq0.5\}. \tag*{(5)}

Here, LL is chance-corrected recall@10, whereas GG averages the profile over the band whose upper edge is the median split of Proposition 2. Averaging over this band reduces the variance of a single νk⋆\nu_{k^\star} while preserving the meaning of the median split. Within each of the 18 runs in Section 6, GG and the median split have a rank correlation of 0.99 to 1.00 across the 27 compressions. Because both fidelities evaluate the same chance-corrected statistic at two scales, they can be compared directly. Every compression thus lies on the (L,G)(L,G) plane, where its vertical distance below the diagonal G=LG=L measures how much more of the far end it loses relative to the near end.

Because scale is defined relative to nn, GG can be estimated from a subsample: if n0n_0 items are drawn uniformly, then for a relative scale α=k/n\alpha=k/n the ⌊αn0⌋\lfloor\alpha n_0\rfloor nearest items within the subsample approximate the α\alpha-quantile neighborhood of the whole memory. We compute GG on five uniform subsamples of 2,000 items and report the mean. Across twelve datasets of up to 1.38 million items, subsamples of 2,000 items estimate GG within 0.010 of its value at 8,000 items [17]. By contrast, LL refers to the fixed scale k=10k=10 and is computed on the whole memory. For each pair of items we record the neighbor rank in the uncompressed and in the compressed memory. DLPk\mathrm{DLP}_k counts the pairs ranked within kk in both, so one running count over kk gives DLPk\mathrm{DLP}_k and νk\nu_k at every scale in O(n02log⁡n0)O(n_0^2\log n_0) time.

Which vectors each fidelity reads. Each fidelity is read from the vectors that its corresponding operations use. When a retrieval pipeline uses rescoring, it fetches the 40 nearest codes and rescores them using full-precision vectors read from disk. Rescoring modifies the lists returned to a query while leaving the stored codes unchanged. We therefore read LL and retrieval from the rescored lists seen by the query. We read GG and every other operation from the stored codes on which those operations run.

fig:1d, e shows the profiles of real compressions. On the two agent memories, scalar quantization to 8 bits keeps the profile near 1 at every scale. Four compressions to 96 bytes start at similar heights near k=10k=10 and separate as kk grows. Rotated binary codes fall fastest toward the GG band.

Rank violations bound the profile

The order of near and far items controls the profile at every scale. Whereas the profile counts kept neighbors, a learning objective requires a quantity that varies smoothly with the codes. Pairwise rank order provides such a quantity. For item ii, the anchor, and scale kk, we call (p,q)(p,q) a cross pair if p∈Nkx(i)p\in\mathcal{N}_k^x(i) and q∉Nkx(i)q\notin\mathcal{N}_k^x(i); that is, pp is near and qq is far in the uncompressed memory. Item ii has k(N−k)k(N-k) cross pairs, and a pair is violated if the compression reverses its order so that riz(q)<riz(p)r_i^z(q)<r_i^z(p). Let pk(i)p_k(i) be the fraction of violated cross pairs of ii, and let pˉk=1n∑ipk(i)\bar{p}_k=\frac{1}{n}\sum_i p_k(i).

The theorem below states that a memory cannot lose many neighbors at scale kk unless it reverses many cross pairs.

Theorem 1 (Rank-violation bound). For every compression and every 1≤k≤N−11\leq k\leq N-1,

1−νk≤Nk(N−k)⋅1n∑i=1npk(i)≤Nk(N−k)pˉk.(6)1-\nu_k\leq\frac{N}{\sqrt{k(N-k)}}\cdot\frac{1}{n}\sum_{i=1}^{n}\sqrt{p_k(i)}\leq\frac{N}{\sqrt{k(N-k)}}\sqrt{\bar{p}_k}. \tag*{(6)}

Proof. Fix ii and let mi=k−∣Nkx(i)∩Nkz(i)∣m_i=k-\lvert\mathcal{N}_k^x(i)\cap\mathcal{N}_k^z(i)\rvert. Exactly mim_i items are lost, in Nkx(i)\mathcal{N}_k^x(i) but outside Nkz(i)\mathcal{N}_k^z(i), and exactly mim_i are gained, in Nkz(i)\mathcal{N}_k^z(i) and outside Nkx(i)\mathcal{N}_k^x(i), because both sets have size kk. Take a lost pp and a gained qq. Then pp is near and qq is far in the uncompressed memory, so (p,q)(p,q) is a cross pair, and riz(q)<riz(p)r_i^z(q)<r_i^z(p), so it is violated. These mi2m_i^2 pairs are distinct, hence mi2≤pk(i)k(N−k)m_i^2\leq p_k(i)k(N-k) and mi≤pk(i)k(N−k)m_i\leq\sqrt{p_k(i)k(N-k)}. By (1), 1−DLPk=1nk∑imi1-\mathrm{DLP}_k=\frac{1}{nk}\sum_i m_i, and by (2), 1−νk=NN−k(1−DLPk)1-\nu_k=\frac{N}{N-k}(1-\mathrm{DLP}_k). Combining,

1−νk=Nnk(N−k)∑i=1nmi≤Nk(N−k)⋅1n∑i=1npk(i).1-\nu_k=\frac{N}{nk(N-k)}\sum_{i=1}^{n}m_i\leq\frac{N}{\sqrt{k(N-k)}}\cdot\frac{1}{n}\sum_{i=1}^{n}\sqrt{p_k(i)}.

The second inequality is Jensen’s inequality for the concave square root. □\square

The middle term of (6) is Nk(N−k)pˉk\frac{N}{\sqrt{k(N-k)}}\sqrt{\bar{p}_k}, with pˉk=1n∑ipk(i)\sqrt{\bar{p}_k}=\frac{1}{n}\sum_i\sqrt{p_k(i)}. This bound is strongest where GG is read. Because the factor N/k(N−k)N/\sqrt{k(N-k)} equals 1/(k/N)(1−k/N)1/\sqrt{(k/N)(1-k/N)}, it lies between 2 and 2.18 across the GG band (0.3≤k/n≤0.5)(0.3\leq k/n\leq0.5). Consequently, a small violation rate at these scales forces νk\nu_k close to 1, transferring the bound directly to GG. By contrast, at k=10k=10 the factor is approximately N/10\sqrt{N/10}, which reaches 14 for a memory of 2,000 items. Theorem 1 thus singles out the middle and far scales as the scales where ordering cross pairs controls the profile, motivating LAQ to train its codes at those scales. On the held-out memories of Section 7, the bound holds across all 160 combinations of memory, quantizer, and scale (Figure 3e).

Figure 3. At the same bytes, OPQ keeps the far end best, and LAQ raises it further on every held-out memory by lowering rank violations (LL, GG: local and global fidelity). (a, b) Seven compressions at d/8d/8 bytes on FinQA and LoCoMo with the retrieval-tuned encoders: OPQ has the highest GG (0.82 to 0.91), whereas rescoring (arrows to open markers) raises LL and top-10 agreement but leaves GG and the far-end operations unchanged. (c–f) Held-out Multi-Doc-2025 across four splits: LAQ lies above OPQ at every scale, most notably in the GG band (gray) and with Titan V2; the bound of Theorem 1 holds at all 160 points, with 1−νk1-\nu_k at 0.49 to 0.60 of it; and LAQ lowers the violation rate at k≈0.4nk \approx0.4n by 12 to 18 percent on average (10 to 20 percent per run). (g) Change in each operation across all 40 held-out runs; right: runs in which LAQ is higher. (h) LAQ (black) has the highest GG among seven quantizers in each of nine runs with the retrieval-tuned encoders. (i) The lowest theme agreement rises from 0.80 to 0.89.

Because rank violations are counted by a step function, we bound them with a hinge on distance gaps. Let Δx=∥xi−xq∥2−∥xi−xp∥2≥0\Delta^{x} = \lVert x_i-x_q\rVert^{2}-\lVert x_i-x_p\rVert^{2} \ge0 be the uncompressed gap of a cross pair, Δz=∥zi−zq∥2−∥zi−zp∥2\Delta^{z} = \lVert z_i-z_q\rVert^{2}-\lVert z_i-z_p\rVert^{2} be its compressed gap, and h=[ηΔx−Δz]+h = [\eta\Delta^{x}-\Delta^{z}]_{+} with margin η>0\eta> 0. Pushing compressed gaps above a fraction of the uncompressed gaps drives the violation rate down, as the following corollary shows.

Corollary 1 (Hinge surrogate). Let (p,q)(p,q) be a uniformly random cross pair of a uniformly random item at scale kk, and Hˉk=E h\bar{H}_{k} = \mathbb{E}\,h. For every δ>0\delta> 0,

pˉk≤Pr⁡(Δx<δ)+Hˉkηδ.(7)\bar{p}_{k} \le\Pr(\Delta^{x} < \delta) + \frac{\bar{H}_{k}}{\eta\delta}. \tag*{(7)}

The first term depends on the uncompressed memory alone, measuring the share of cross pairs whose two squared distances to ii lie within δ\delta of each other. The second term is the expected hinge, for which the average hinge over uniformly drawn cross pairs provides an unbiased estimate. Together, Theorem 1 and Corollary 1 turn the profile at a chosen scale into an objective that gradient descent can lower.

Predicting the far-end loss

Theorem 1 says that the far end is kept when few cross pairs are reversed. The next result says how many are reversed. For anchor ii, let Dj=∥xi−xj∥2D_j = \lVert x_i-x_j\rVert^{2} and D~j=∥zi−zj∥2\widetilde{D}_j = \lVert z_i-z_j\rVert^{2} for j≠ij \ne i, and let mim_i and sis_i be the mean and standard deviation of the DjD_j. Fit the compressed distances by the least-squares line D~j≈aiDj+bi\widetilde{D}_j \approx a_iD_j+b_i with ai>0a_i > 0. A common map of this form leaves every rank unchanged, so a pair can be reversed by the residual ej=(D~j−bi)/ai−Dje_j = (\widetilde{D}_j-b_i)/a_i-D_j alone. We call ωi=sd⁡(ej)/si\omega_i = \operatorname{sd}(e_j)/s_i the relative distance noise of anchor $i: the distortion of its distances measured against their spread. Let ϕ\phi and Φ\Phi be the density and distribution function of N(0,1)\mathcal{N}(0,1), and zα=Φ−1(α)z_{\alpha} = \Phi^{-1}(\alpha).

Proposition 4 (Noisy order). Fix α=k/n∈(0,1)\alpha= k/n \in(0,1). Suppose the standardized distances uj=(Dj−mi)/siu_j = (D_j-m_i)/s_i of anchor ii are independent N(0,1)\mathcal{N}(0,1), and the scaled residuals ej/sie_j/s_i are independent normal with mean 0 and variance ωi2\omega_i^{2}, independent of the uju_j. Then, as n→∞n \to\infty, the expected violation rate of anchor ii at scale kk converges to

π(α,ωi)=1α(1−α)∫−∞zα∫zα∞ϕ(u)ϕ(v)Φ(−u−v2ωi) du dv,(8)\pi(\alpha,\omega_i) = \frac{1}{\alpha(1-\alpha)} \int_{-\infty}^{z_{\alpha}}\int_{z_{\alpha}}^{\infty} \phi(u)\phi(v)\Phi\left(-\frac{u-v}{\sqrt{2}\omega_i}\right)\,du\,dv, \tag*{(8)}

and π(α,ω)=ϕ(zα)2ω2/(2α(1−α))+O(ω3)\pi(\alpha,\omega) = \phi(z_{\alpha})^{2}\omega^{2}/(2\alpha(1-\alpha)) + O(\omega^{3}) as ω→0\omega\to0.

Proposition 4 counts reversed pairs. The same model gives the profile itself.

Corollary 2 (Profile under noisy order). Under the assumptions of Proposition 4, as n→∞n \to\infty the contribution of anchor ii to νk\nu_k converges to

ν(α,ωi)=Φ2(zα,zα;ϱi)−α2α(1−α),ϱi=(1+ωi2)−1/2,(9)\nu(\alpha,\omega_i) = \frac{\Phi_{2}(z_{\alpha},z_{\alpha};\varrho_i)-\alpha^{2}}{\alpha(1-\alpha)}, \qquad\varrho_i = (1+\omega_i^{2})^{-1/2}, \tag*{(9)}

where Φ2(⋅,⋅;ϱ)\Phi_{2}(\cdot,\cdot;\varrho) is the bivariate normal distribution function with unit variances and correlation ϱ\varrho. It decreases strictly in ωi\omega_i, and at α=1/2\alpha= 1/2 it equals 2πarcsin⁡ϱi\frac{2}{\pi}\arcsin\varrho_i.

The far end thus depends on one ratio: how much the compression perturbs the distances from an anchor, against how much those distances spread. At the same distortion, a memory whose distances spread less loses more of its far end, and any quantizer that lowers ωi\omega_i in the GG band raises νk\nu_k there. The model rests on a normal approximation of each anchor’s distances. In the GG band, these distances are sums over many directions and lie in the bulk of their distribution. At k=10k = 10, they lie in the lower tail, where the local structure of the memory decides. The independence of the residuals also holds approximately: items that share a codeword have correlated errors, and the noise varies across items. Figure 12 tests the consequences of both approximations.

The held-out memories bear out both results (Figure 12 in Appendix C). Standardized by their normal mean and variance, the distances of each anchor match the normal quantile to within a median of 0.10 standard deviations at k=0.4nk = 0.4n and 0.06 at k=0.5nk = 0.5n, against 0.42 at k=10k = 10. Across all 52 held-out runs, for the OPQ start and LAQ, π(0.4,ωi)\pi(0.4,\omega_i) predicts the measured violation rate at k=0.4nk = 0.4n with correlation 0.979. The median error is 5.3 percent with the retrieval-tuned encoders, and the prediction is 25 to 31 percent low with Titan V2. These predictions use the full integral (8). Its leading term is larger, by a median of 25 percent with Titan V2, so the gap with Titan V2 lies in the assumptions of the model and not in the expansion. At k=10k = 10, the normal model predicts 2.8 times the measured rate (median), so the near end keeps more order than the model allows. Evaluated at the mean relative distance noise of each run, Corollary 2 gives νk\nu_k at k=0.4nk = 0.4n within a median of 0.005 of the measured value (correlation 0.982). The relative distance noise ranges from 0.16 to 0.37 with the retrieval-tuned encoders and from 0.47 to 0.52 with Titan V2, whose OPQ start keeps the least of the far end. The data entropy of a memory [46], h=−∑ℓuℓlog⁡2uℓh = -\sum_{\ell}u_{\ell}\log_{2}u_{\ell} with uℓu_{\ell} the singular values of the memory divided by their sum, tells how much LAQ lowers this noise. Within each retrieval-tuned encoder, across its 16 held-out runs, hh has a rank correlation of 0.77 to 0.81 with the drop in mean relative distance noise from OPQ to LAQ and of 0.82 to 0.84 with the gain in GG ($p \le 4 \times 10^{-4}$ in each case). Memories of higher data entropy spread their information over more directions, and LAQ removes more of their distance noise.

Locality-aware quantization

Locality-aware quantization (LAQ) trains the codebooks of an OPQ index to keep the order of cross pairs at the middle and far scales, the target identified by Theorem 1. Whereas product quantizers learn codebooks to reconstruct vectors or keep nearest neighbors, LAQ adds this far-end target, needs no labels, and keeps the rest of the OPQ index intact. Figure 1g shows the procedure.

Starting point. LAQ first fits PCA on the training memory in the full basis, followed by OPQ with M=d/8M = d/8 subspaces of eight dimensions and K=256K = 256 codewords each, storing every item in d/8d/8 bytes. Let μ\mu be the mean of the training vectors and RR be the orthogonal map combining the PCA basis with the OPQ rotation. Let yi=R(xi−μ)y_i = R(x_i-\mu) denote the rotated item, yi,ℓy_{i,\ell} its ℓ\ell-th subvector, and C=[cℓ,j  (ℓ=1,…,M,j=1,…,K)]C = [c_{\ell,j}\;(\ell= 1,\ldots,M, j = 1,\ldots,K)] the codebooks initialized at

Figure 1. At the same bytes and search, locality-aware quantization (LAQ) keeps the near end of an optimized product quantizer (OPQ) and moves fewer passages to another theme. LL: local fidelity (chance-corrected recall@10); GG: global fidelity (the profile averaged over 30 to 50 percent of the memory). (a) Multi-Doc-2025 [11], BGE-large [12]: 2,688 held-out passages of 22 companies, each joined to its 10 nearest neighbors and colored by theme (t-SNE layout [13]); companies form islands, and banks share themes. (b, c) At 128 bytes, both keep LL at 0.85; OPQ moves 8.3 percent of the passages to another theme (red) and LAQ 3.5 percent, on the split where OPQ has its lowest theme agreement; over 16 runs the mean falls from 2.8 to 2.1 percent. (d, e) Locality profiles (Definition 1) of five compressions of FinQA [14] and LoCoMo [15]: compressions of equal size agree near k=10k = 10 and part by up to 0.30 in the GG band (gray). (f) Every lost-gained neighbor pair is a violated cross pair, which bounds the profile (Theorem 1). (g) LAQ in four steps; gray: cost of each step (Section 4). (h) LAQ minus OPQ on all 52 held-out runs: GG rises in all 52 and LL in 46.

Figure 12. The normal model of Proposition 4 holds in the GG band and predicts its violations and its loss of profile. All 52 held-out runs; open markers, OPQ start; filled, LAQ. (a) Median error of each anchor’s standardized distance quantile against the normal quantile, in standard deviations, at the near end and in the GG band. (b) The full integral (8) against the measured violation rate at k=0.4nk = 0.4n. (c) The leading term at k=10k = 10 against the measured rate. (d) Measured 1−νk1 - \nu_k at k=0.4nk = 0.4n against the mean relative distance noise ω\omega; solid line, Corollary 2; dashed line, its leading term 0.64 ω0.64\,\omega (Appendix A).

OPQ values C0C_0. Each item is encoded by its nearest codeword in each subspace and decoded as the concatenation zi(C)z_i(C). Because RR is orthogonal, distances in the rotated basis equal those in the original space.

Objective. LAQ minimizes

J(C)=Lrec(C)Lrec(C0)+λmidLmid(C)+λfarLfar(C),(10)J(C)=\frac{\mathcal{L}_{\mathrm{rec}}(C)}{\mathcal{L}_{\mathrm{rec}}(C_0)}+\lambda_{\mathrm{mid}}\mathcal{L}_{\mathrm{mid}}(C)+\lambda_{\mathrm{far}}\mathcal{L}_{\mathrm{far}}(C), \tag*{(10)}

where Lrec=1n∑i∥yi−zi∥2\mathcal{L}_{\mathrm{rec}}=\frac{1}{n}\sum_i\lVert y_i-z_i\rVert^2 is the reconstruction error, and Lmid\mathcal{L}_{\mathrm{mid}} and Lfar\mathcal{L}_{\mathrm{far}} are hinge losses evaluated at two scale groups with weights λmid\lambda_{\mathrm{mid}} and λfar\lambda_{\mathrm{far}}. For a scale group S∈{mid,far}S\in\{\mathrm{mid},\mathrm{far}\} and a batch of sampled cross pairs (p,q)(p,q) with anchors ii,

LS=1sSmean⁡[ηΔx−Δz]+,(11)\mathcal{L}_S=\frac{1}{s_S}\operatorname{mean}\left[\eta\Delta^x-\Delta^z\right]_+, \tag*{(11)}

where Δx\Delta^x and Δz\Delta^z are the distance gaps defined in Corollary 1. The normalizer sSs_S is the median uncompressed gap of the group, computed once at initialization so that both groups contribute on a common scale. The middle group draws k/n∈{1%,5%}k/n\in\{1\%,5\%\}, while the far group draws kk uniformly from KK, the GG band. The normalized reconstruction term keeps the near end, which product quantization already serves well.

Sampling cross pairs. Each step draws 1,024 anchors per group, and for each anchor samples a scale kk and a cross pair. Half of the pairs are drawn uniformly: pp from the kk nearest items of ii, and qq from the remaining N−kN-k items. By Corollary 1, these pairs yield an unbiased estimate of Hˉk\bar{H}_k, which bounds the violation rate and, through Theorem 1, bounds 1−νk1-\nu_k. The other half straddle rank kk: pp is drawn from ranks k−w+1k-w+1 to kk and qq from ranks k+1k+1 to k+wk+w, with w=max⁡(3,⌊0.1k⌋)w=\max(3,\lfloor0.1k\rfloor). Because lost and gained neighbors arise at this boundary, as shown in the proof of Theorem 1, these boundary pairs focus half of the gradient where the neighborhood changes.

Straight-through codes. Because encoding is a hard assignment, LAQ uses hard assignments in the forward pass and the soft assignment softmax⁡(−∥yi,ℓ−cℓ,j∥2/T)\operatorname{softmax}(-\lVert y_{i,\ell}-c_{\ell,j}\rVert^2/T) in the backward pass [21]. It re-encodes all training items with the current codebooks at every step. The temperature at step tt decays as Tt=T0⋅0.05t/tmax⁡T_t=T_0\cdot0.05^{t/t_{\max}} over tmax⁡t_{\max} steps, where T0T_0 is the initial mean squared distance from a subvector to its nearest codeword. The codebooks are updated using Adam [47] with a cosine-decayed learning rate equal to 0.002 times the standard deviation of the entries of C0C_0.

Checkpoint on a monitor set. LAQ draws 2,000 training items as a monitor set. Every 250 steps, it evaluates on this monitor set both νk\nu_k at k/n=0.4k/n=0.4 on the set (the midpoint of the GG band) and the relative reconstruction error Lrec(C)/Lrec(C0)\mathcal{L}_{\mathrm{rec}}(C)/\mathcal{L}_{\mathrm{rec}}(C_0). It keeps the codebooks with the highest νk\nu_k among checkpoints whose relative reconstruction error is at most 1.15, falling back to C0C_0 if no checkpoint improves upon the start.

The far hinge at the OPQ start. At the OPQ start, the far hinge can fall while the reconstruction error holds to first order. Let nℓ,jn_{\ell,j} be the number of training items whose ℓ\ell-th subvector is assigned to cℓ,jc_{\ell,j}, and let ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle be the inner product of codebooks.

Proposition 5 (Far hinge at no first-order cost). Suppose every codeword of C0C_0 is the mean of the training subvectors assigned to it, no subvector is equidistant from its two nearest codewords, and Lfar\mathcal{L}_{\mathrm{far}}, computed with the assignments of C0C_0, has a gradient gg at C0C_0. Then, for every direction hh with ⟨g,h⟩>0\langle g,h\rangle>0 and all small enough ϵ>0\epsilon>0, the step Cϵ=C0−ϵhC_\epsilon=C_0-\epsilon h satisfies

Lrec(Cϵ)=Lrec(C0)+ϵ2∑ℓ,jnℓ,jn∥hℓ,j∥2,\mathcal{L}_{\mathrm{rec}}(C_\epsilon)=\mathcal{L}_{\mathrm{rec}}(C_0)+\epsilon^2\sum_{\ell,j}\frac{n_{\ell,j}}{n}\lVert h_{\ell,j}\rVert^2,
Lfar(Cϵ)=Lfar(C0)−ϵ⟨g,h⟩+o(ϵ).(12)\mathcal{L}_{\mathrm{far}}(C_\epsilon)=\mathcal{L}_{\mathrm{far}}(C_0)-\epsilon\langle g,h\rangle+o(\epsilon). \tag*{(12)}

Proposition 5 is a statement about the two training terms: along any direction that lowers the far hinge, the reconstruction error rises at second order in the step. This holds for h=gh=g and for the first step of Adam, which is h=sign⁡(g)h=\operatorname{sign}(g) up to scale, with ⟨g,sign⁡(g)⟩=∥g∥1>0\langle g,\operatorname{sign}(g)\rangle=\lVert g\rVert_1>0.

Straight-through steps. Training computes a different gradient. The straight-through codes use the hard assignments of Proposition 5 in the forward pass and the soft assignments at temperature TT in the backward pass, so the computed gradient gstg_{\mathrm{st}} differs from gg. As T→0T\to0 the soft assignments become hard and gst→gg_{\mathrm{st}}\to g. At the starting temperature T0T_0, we measured both gradients on the same 8,192 far cross pairs at all 52 OPQ starts of Section 7. The cosine between gstg_{\mathrm{st}} and gg is 0.56 to 0.69 (median 0.62), so h=gsth=g_{\mathrm{st}} meets the condition of Proposition 5 at every start. One first Adam step along gstg_{\mathrm{st}}, re-encoding all items, lowers the far hinge by 1.1 to 2.5 percent (median 1.6 percent) and raises the reconstruction error by at most 0.008 percent, while 0.05 to 0.11 percent of the subvectors change codeword.

From the hinge to GG. A lower far hinge lowers the bound of Corollary 1 on the violation rate in the GG band, and through Theorem 1 the bound on 1−νk1-\nu_k. The checkpoint rule keeps the gain on the monitor set: the start is itself a candidate, so the kept codebooks have a monitor νk\nu_k at least that of C0C_0, at a reconstruction error at most 1.15 times that of C0C_0. Section 7 tests the gain on held-out memories. The OPQ starts are close to the fixed point that Proposition 5 assumes. On all 52 training memories, one more centroid step lowers the reconstruction error by at most 0.032 percent (median 0.016 percent) and moves the codebooks by at most 0.6 percent of their norm. Corollary 2 also predicts the size of the gain. Evaluated at the mean relative distance noise of OPQ and of LAQ, it predicts a mean gain in νk\nu_k at k=0.4nk=0.4n of 0.0126, against 0.0124 measured on the held-out memories, with a rank correlation of 0.92 across runs and a predicted gain above zero in all 52.

What LAQ changes. LAQ modifies the codebook values and nothing else: the PCA basis, rotation, bytes per item, and asymmetric distance computation (ADC) search all remain identical to those of OPQ. An LAQ index therefore replaces an OPQ index directly and adds no operation at query time. A single recipe serves every memory and encoder: tmax⁡=1,000t_{\max}=1{,}000 steps, λmid=λfar=1\lambda_{\mathrm{mid}}=\lambda_{\mathrm{far}}=1, and η=1\eta=1 (Appendix B).

Complexity. Let nn be the number of training items, dd the dimension, K=256K=256 the codebook size, B=2,048B=2{,}048 the number of cross pairs per step (1,024 anchors in each of the two groups), and tmax⁡t_{\max} the number of steps. Ranking the training memory once costs O(n2(d+log⁡n))O(n^2(d+\log n)) time and O(n2)O(n^2) memory. Each step encodes all items with the current codebooks in O(nKd)O(nKd) time and evaluates the hinge on BB cross pairs in O(Bd)O(Bd) time. Training therefore costs O(n2(d+log⁡n)+tmax⁡(nKd+Bd))O(n^2(d + \log n) + t_{\max}(nKd + Bd)) time in total, while the query cost and the encoding of a new item, O(Kd)O(Kd), remain those of OPQ. The n×nn \times n matrix of 7,336 to 7,672 items fits on one 48 GB GPU, and the 1,000 steps take 35 to 112 seconds on Multi-Doc-2025. For a memory of a million items, a scalable variant ranks against m=8,192m = 8{,}192 reference items at each step and decodes only the items used in that step, with a reconstruction batch of b=16,384b = 16{,}384. This reduces the per-step cost to O(Bm(d+log⁡m)+(B+b)Kd)O(Bm(d + \log m) + (B + b)Kd) and memory to O(nd+Bm)O(nd + Bm) (§ 7.5).

Experimental setup

We evaluate the two fidelities across five memories and six encoders, and test LAQ on held-out splits of four memories as well as on a million-passage memory.

Memories. Table 1 lists five memories: three agent memories and two topic corpora. FinQA, LoCoMo, and the topic corpora serve the measurement, and Multi-Doc-2025 and two additional memories test LAQ. FinQA [14] contains pages from the annual reports of S&P 500 companies paired with numerical questions annotated with supporting facts. We retain the 106 companies in the current index and up to three report pages per company. Every sentence and table row is one item, prefixed by the company name and fiscal year. This yields 8,356 items across 11 Global Industry Classification Standard (GICS) sectors and 863 questions. Multi-Doc-2025 [11] is a financial question-answering benchmark spanning 179 10-K filings from 89 S&P 500 companies for fiscal years 2022 to 2024. Each paragraph and table row serves as a candidate item with the same prefix. Every filing retains the passages containing its questions’ evidence plus a random sample, up to 56 passages, yielding 10,024 items and 898 questions with located evidence. Multi-Doc-2025 is used to test LAQ. LoCoMo [15] is a long-term conversational memory benchmark for agents, comprising ten conversations of 19 to 32 sessions, 5,882 turns, and 1,536 answerable questions annotated with their evidence turns. Each turn, prefixed by its speaker, forms one item. 20 Newsgroups [48] and AG News [49] contribute 10,000 random documents each, spanning 20 and 4 topics, respectively. A sixth memory, FinDER [26], is reserved for testing LAQ: it pairs 5,703 queries written by financial professionals with expert-selected evidence from the 10-K filings of 497 S&P 500 companies. Following our Multi-Doc-2025 construction, we keep up to 30 passages per filing. Because FinDER provides the evidence text without specifying its company, we locate each excerpt in the filing that contains the most of its rarer words. When multiple filings match, the company named in the query breaks the tie, and any excerpt located in a company other than the one named is dropped. Taking the one or two passages in that filing whose words appear in the excerpt as the evidence passages yields 14,949 items and 5,129 questions. Finally, a seventh memory comprising 1,000,000 passages from MS MARCO [50] tests LAQ at scale (§ 7.5).

MemoryItemsQuestionsStructure used
FinQA8,356863company, sector, fiscal year
Multi-Doc-202510,024898company, sector, fiscal year
LoCoMo5,8821,536conversation, session
20 Newsgroups10,000–topic, topic family
AG News10,000–topic

Table 1. Five memories with company, session, or topic structure, three with annotated questions.

Encoders. Six encoders serve the fidelity measurement runs. Three are retrieval-tuned sentence encoders used with their query instructions: BGE-base-en-v1.5 (BGE-base) and BGE-large-en-v1.5 (BGE-large) [12] at 768 and 1,024 dimensions, and E5-base-v2 (E5-base) [51] at 768 dimensions. Three are encoders of other kinds: the text encoder of MobileCLIP2-L/14 [45] (768 dimensions), RoBERTa-base [52] with mean pooling (768), and averaged GloVe vectors [53] (300). A seventh encoder, the commercial Amazon Titan Text Embeddings V2 (Titan V2) [5] at 1,024 dimensions, is used on Multi-Doc-2025 to test LAQ. All vectors have unit length.

Compressions. There are 18 measurement runs: the six encoders on the two agent memories FinQA and LoCoMo, and the three encoders of other kinds on the two topic corpora. Each run applies 27 compressions fitted on the memory itself. Scalar codes comprise half precision along with 8- and 4-bit codes. Binary codes comprise 1-bit sign codes, the same codes with full-precision rescoring of the 40 nearest candidates, and 2- and 4-bit thermometer codes that set one bit per threshold passed on each coordinate. Truncation, PCA, and Gaussian random projection reduce vectors to d/2d/2, d/4d/4, d/8d/8, and d/16d/16 dimensions. In addition, PCA to d/4d/4 and d/8d/8 is stored with 8-bit codes, and PCA to dd and d/2d/2 is followed by a random rotation and binarization. Product quantization uses d/8d/8, d/16d/16, and d/32d/32 bytes, while UMAP [44] maps vectors to 16 dimensions. Finally, OPQ with and without rescoring is included at d/8d/8 bytes. For comparison, t-SNE [13], SR-tSNE [54], PHATE [55], Isomap [56], and nonnegative PCA (NPCA) [57] are fitted on FinQA and LoCoMo with BGE-base (Table 4). Binary codes are compared by Hamming distance and all other codes by Euclidean distance on the decoded vectors. In our figures, PCA with 8-bit codes is labeled PCA + int8 and rescored binary codes binary + rescoring. Binarization after PCA and a random rotation is labeled PCA + rotation + binary (PCA + rot. + binary where space is short).

MemoryMapBytesLLGGRetrievalThemes / deletion
FinQAUMAP, 16 dims640.5230.100.240.08
PCA to 50, UMAP to 16640.4740.180.160.21
PHATE, 16 dims640.5370.150.190.08
UMAP, 2 dims80.4530.070.180.07
PHATE, 2 dims80.2310.120.040.07
t-SNE, 2 dims80.5850.130.070.09
SR-tSNE, 2 dims80.5900.130.070.09
Isomap, 16 dims640.5600.250.280.15
NPCA, 16 dims640.2370.430.100.41
PCA, 16 dims (linear)640.3490.540.170.71
LoCoMoUMAP, 16 dims640.3620.190.300.08
PCA to 50, UMAP to 16640.3290.190.250.07
PHATE, 16 dims640.2920.220.240.08
UMAP, 2 dims80.2550.160.210.04
PHATE, 2 dims80.1410.190.140.05
t-SNE, 2 dims80.3880.200.150.05
SR-tSNE, 2 dims80.3920.200.150.05
Isomap, 16 dims640.3880.430.330.23
NPCA, 16 dims640.2200.440.180.11
PCA, 16 dims (linear)640.3490.540.260.14

Table 4. Neighborhood-embedding maps keep GG at 0.22 or less, and nonnegative PCA keeps more of the far end than of the near end. Maps fitted to the memory (BGE-base), kept in full precision; PCA to 16 dimensions is the linear reference for NPCA. Queries are mapped by the fitted transform (UMAP [44], PHATE [55], Isomap [56], NPCA [57], PCA) or placed at the inverse-distance-weighted mean of the positions of their 10 nearest stored items (t-SNE [13], SR-tSNE [54]). Top-10 agreement: overlap of the top-10 lists. Last column: theme agreement (FinQA) or agreement of session deletion (LoCoMo).

Operations. Each operation is executed on both the compressed and uncompressed memories, and we report their agreement. We model an agent’s memory operations directly on the vectors, which scores every operation on every compression without a language model in the loop.

  • Retrieval. Each question retrieves its 10 nearest items (within its conversation for LoCoMo). Agreement is the overlap of the two top-10 lists, while quality is evidence recall@10, the share of a question’s evidence among those 10 items.

  • Consolidation into themes. We run kk-means on the compressed memory, initialized from the centroids of the uncompressed partition, using 11 themes for the financial memories and 20 and 4 themes for the topic corpora. Agreement is measured by the adjusted Rand index (ARI) [58], which we call theme agreement.

  • Deletion. For each session or topic, the memory selects as many items as the session or topic contains, choosing those nearest its centroid. This models how a memory identifies deletion candidates by meaning when metadata are incomplete. Agreement is the Jaccard overlap of the deleted sets.

  • Company operations. The information barrier walls off a company by applying deletion to it, while peer groups consist of each company’s five nearest companies by centroid. The company map is the rank correlation between distances among compressed company centroids and the uncompressed ones; the sector, session, and topic maps are defined in the same way.

  • Global references. These use the subsamples of GG. All distances is the Spearman correlation across 200,000 pairwise distances [40]. Triplets is the accuracy on random triplets [41]. The median split is νk⋆\nu_{k}^{\star} from Proposition 2.

Because consolidation, deletion, the session, sector, and topic maps, and the global references compare each item against the whole memory, we call them the far-end operations. Across the 27 compressions of each run, we relate each operation’s agreement to LL and GG using Spearman correlation as well as the partial rank correlation with one fidelity given the other.

Held-out protocol for LAQ. LAQ is trained and tested on disjoint companies or conversations. For FinQA and Multi-Doc-2025, each split holds out a quarter of the companies (26 of 106 and 22 of 89). For LoCoMo it holds out two of the ten conversations. Three splits (seeds 2026 to 2028) were fixed in advance. A fourth split (seed 2029) was drawn after the LAQ recipe was fixed and run once with the recipe unchanged. PCA, OPQ, and LAQ are fitted on the training part only, and every reported number is computed on the held-out part using its own questions. Evaluating the three retrieval-tuned encoders across all three memories and Titan V2 on Multi-Doc-2025 yields 40 held-out runs, each comparing LAQ against its own OPQ start at d/8d/8 bytes. The other quantizers were evaluated on the first three splits. On Multi-Doc-2025, queries remain in full precision and are searched via ADC, just as an OPQ index serves them. LL, GG, and the far-end operations are evaluated on the stored codes. FinDER was run after all of the above, with the recipe unchanged. Each of its four splits (seeds 2026 to 2029) holds out 124 of its 497 companies, with queries searched by ADC. The three retrieval-tuned encoders thus provide 12 further runs, which we report separately from the 40.

Quantizers compared. On the held-out Multi-Doc-2025 memories, we compare LAQ at d/8d/8 bytes against OPQ and eight other quantizers. Five store codes of approximately the same size: PQ [4], RQ [27], LSQ++ [28], PCA to d/8d/8 dimensions with 8-bit codes, and RabitQ [31] with dd bits plus its per-vector factors (104 and 136 bytes). Sign and ITQ [30] binary codes are rescored on the 40 nearest candidates, and ScaNN uses anisotropic quantization [29]. All but ScaNN are fitted on the training companies.

What the two ends measure

Each end of the profile predicts its own operations: retrieval follows LL, consolidation and deletion follow GG, and compressions of equal size can keep very different far ends. Section 7 tests whether LAQ raises the far end at the same bytes.

Compressions separate on the (L,G)(L,G) plane

Compressions of equal size keep different ends of a memory. Figure 2 places every compression of FinQA with BGE-base on the (L,G)(L,G) plane; in Appendix § 12, Figure 5 shows both agent memories with the retrieval-tuned encoders, and Figure 6 shows the other three encoders. Within a given memory, the two ends correlate across compressions (Spearman 0.81 to 0.95 over the 18 runs) because fewer bytes lower both. Yet compressions with the same LL can differ widely in GG. With the retrieval-tuned encoders, half precision and 8-bit scalar codes keep both ends at 0.97 or above. PCA, PCA with 8-bit codes, and product quantization keep GG within 0.12 of LL, whereas binarization after PCA and a random rotation places GG 0.06 to 0.22 below LL. Among the linear compressions, rescored binary codes fall furthest below the diagonal, by 0.21 to 0.41. At 96 bytes per passage on FinQA with BGE-base, they keep LL at 0.97 but move 10.1 percent of the passages to another theme. PCA with 8-bit codes keeps LL at 0.77 and moves only 1.7 percent (Figure 2b, Figure 2c). Among nonlinear maps with BGE-base, UMAP to 16 dimensions keeps LL at 0.36 to 0.54 and GG at 0.10 to 0.20, while t-SNE and PHATE keep GG at 0.22 or less. Isomap keeps GG at 0.25 and 0.43. NPCA lies above the diagonal, with GG at 0.43 and 0.44 and LL at 0.22 and 0.24, as does PCA to the same 16 dimensions, with GG at 0.54 and LL at 0.35 (Figure 2a, Table 4). With the encoders of other kinds, PCA and product quantization lie on or above the diagonal on both the agent memories and the topic corpora (Figure 6 and Figure 10a–Figure 10f). One-bit codes of GloVe and RoBERTa vectors reach GG of 0.58 to 0.81 at LL of 0.26 to 0.51. Because these encoders have coordinates of highly unequal variance, taking the sign of a low-variance coordinate perturbs the nearest neighbors far more than the nearer half.

Compressions and nonlinear maps, fidelity comparisons, and rescoring results

Figure 2. At the same bytes, a compression that keeps more nearest neighbors can displace more of the overall memory, because retrieval follows LL whereas themes and deletion follow GG (local and global fidelity). (a) 27 compressions of FinQA (BGE-base) alongside four nonlinear maps (rings: Isomap [56], NPCA [57], PHATE [55], t-SNE); triangle: PCA to 16 dimensions. Rescored binary codes lie 0.32 below the diagonal G=LG = L, while t-SNE, PHATE, and UMAP keep GG at 0.22 or less. (b, c) Two 96-byte compressions plotted on the uncompressed memory layout; colored points mark passages moved to another theme: 1.7 percent for PCA with 8-bit codes versus 10.1 percent for rescored binary codes. (d, e) Across every compression and run, retrieval follows LL (median Spearman ρ\rho 0.99; partial 0.97 given GG), whereas themes and deletion follow GG (ρ\rho 0.95; partial 0.79 given LL). (f, g) Rescoring the 40 nearest binary codes raises LL and top-10 agreement while leaving GG, themes, and deletion unchanged at the binary values (black ticks). All runs: Appendix § 12.

Figure 5. Compressions of one memory spread off a single curve, and rescored binary codes lie furthest below the diagonal, by 0.21 to 0.41. LL and GG of 27 compressions of the two agent memories, three retrieval-tuned encoders; Figure 6 gives the other three. Dashed line: G=LG = L; shaded: less of the far end kept than of the near end.

Figure 6. With encoders of other kinds, PCA and product quantization lie on or above the diagonal, and one-bit codes of GloVe and RoBERTa keep more of the far end than of the near end. LL and GG of 27 compressions of the two agent memories; dashed line: G=LG = L.

Figure 10. On the topic corpora, PCA and product quantization lie on or above the diagonal (a–f). On FinQA, rescored binary codes keep neighbors yet move up to 39 percent of a company’s passages to another theme (g). (a–f) LL and GG of 27 compressions, three encoders of other kinds; dotted line: G=LG = L. (g) The two 96-byte compressions of FinQA (BGE-base) by company; diamonds: means; dotted line: 20 percent moved.

Each end predicts its own operations

Retrieval follows LL, whereas the far-end operations follow GG (Figure 2d, Figure 2e); Figure 7 in Appendix § 12 plots each operation against both fidelities. Top-10 agreement lies close to a single increasing curve in LL, with a median within-run Spearman correlation of 0.99 versus 0.89 for GG. Conversely, the far-end operations lie closer to a single curve in GG, with a median correlation of 0.95 versus 0.87 for LL. Because LL and GG fall together under more severe compression, partial correlations separate the two ends even more sharply.

Figure 7. Retrieval follows LL (ρ\rho 0.99, against 0.89 with GG), and the far-end operations follow GG (ρ\rho 0.95, against 0.87 with LL). Agreement with the uncompressed memory, every compression and run. (a, b) Retrieval on the two agent memories. (c, d) Consolidation into themes for FinQA, deletion for LoCoMo and the topic corpora. ρ\rho: median over runs of the Spearman correlation across the 27 compressions of a run.

Partial correlations. This separation holds when conditioning on the other end: retrieval follows LL given GG, and with the retrieval-tuned encoders, consolidation and deletion follow GG given LL. Figure 8 in Appendix § 12 gathers every operation across every run. Retrieval sits at the lower right, with a partial correlation with LL given GG of 0.83 to 0.99 (Table 5 and Table 6 in Appendix § 12). Consolidation, deletion, and the session and topic maps sit at the upper left. With the retrieval-tuned encoders, consolidating the financial memory into themes follows GG. Its partial correlations with GG given LL are 0.68, 0.76, and 0.66 for BGE-base, E5-base, and BGE-large, whereas with LL given GG they are 0.04, 0.01, and −0.02.

Figure 8. Each operation follows the end at which it compares items: retrieval lies in the LL region, consolidation and deletion in the GG region. Partial rank correlation with GG given LL against that with LL given GG, across the 27 compressions of each run. (a) Agent memories, (b) topic corpora; color: operation; shape: encoder. (c) Medians and ranges over the six runs of the retrieval-tuned encoders.

Retrieval follows LL, and consolidation and deletion follow GG. Spearman correlation across 27 compressions of each operation’s agreement with the uncompressed memory (label precision for the topic corpora) with LL and with GG, and the partial rank correlation with each end given the other (L∣GL \mid G: with LL given GG; G∣LG \mid L: with GG given LL). Agent memories, FinQA.

Memory, encoderOperationEndLLGGL∣GL \mid GG∣LG \mid L
FinQA, BGE-basesame top-10 retrievednear1.000.890.990.30
same themes (ARI)far0.800.900.040.68
same peer groupsfar0.870.830.530.26
information barrierfar0.910.840.650.21
company mapfar0.910.860.640.26
sector mapfar0.860.840.450.35
FinQA, E5-basesame top-10 retrievednear0.980.940.940.78
same themes (ARI)far0.830.930.010.76
same peer groupsfar0.890.900.450.51
information barrierfar0.880.880.440.44
company mapfar0.900.890.500.45
sector mapfar0.890.870.520.37
FinQA, BGE-largesame top-10 retrievednear1.000.920.970.31
same themes (ARI)far0.830.91-0.020.66
same peer groupsfar0.880.860.430.30
information barrierfar0.900.880.480.32
company mapfar0.940.910.630.34
sector mapfar0.870.860.390.32
FinQA, MobileCLIP2same top-10 retrievednear0.950.920.830.70
same themes (ARI)far0.770.900.020.74
same peer groupsfar0.880.900.490.63
information barrierfar0.880.880.550.54
company mapfar0.880.900.510.61
sector mapfar0.730.830.110.57
FinQA, RoBERTasame top-10 retrievednear0.980.880.93-0.24
same themes (ARI)far0.840.860.250.43
same peer groupsfar0.910.850.630.10
information barrierfar0.910.990.140.92
company mapfar0.950.930.670.46
sector mapfar0.920.870.640.15
FinQA, GloVesame top-10 retrievednear0.990.830.970.35
same themes (ARI)far0.770.860.260.63
same peer groupsfar0.900.720.77-0.01
information barrierfar0.920.930.780.81
company mapfar0.890.940.660.84
sector mapfar0.900.760.740.12

Table 5. Table 5

Memory, encoderOperationEndLLGGL∣GL \mid GG∣LG \mid L
LoCoMo, BGE-basesame top-10 retrievednear0.990.890.95-0.16
same memories deletedfar0.870.97-0.110.86
session mapfar0.890.970.100.82
LoCoMo, E5-basesame top-10 retrievednear0.990.890.960.18
same memories deletedfar0.870.900.360.55
session mapfar0.900.980.360.92
LoCoMo, BGE-largesame top-10 retrievednear0.990.900.97-0.35
same memories deletedfar0.900.99-0.140.93
session mapfar0.910.940.300.66
LoCoMo, MobileCLIP2same top-10 retrievednear1.000.880.99-0.25
same memories deletedfar0.880.960.190.84
session mapfar0.860.920.220.67
LoCoMo, RoBERTasame top-10 retrievednear0.970.890.87-0.36
same memories deletedfar0.920.98-0.030.83
session mapfar0.930.970.260.71
LoCoMo, GloVesame top-10 retrievednear0.990.850.97-0.57
same memories deletedfar0.920.960.430.78
session mapfar0.900.990.200.93
20 Newsgroups, MobileCLIP2label precision of 10 neighborsnear0.590.290.68-0.48
same memories deletedfar0.900.950.600.80
same themes (ARI)far0.770.810.250.48
topic mapfar0.680.790.030.56
20 Newsgroups, RoBERTalabel precision of 10 neighborsnear0.620.410.76-0.65
same memories deletedfar0.920.98-0.060.85
same themes (ARI)far0.910.880.530.12
topic mapfar0.930.950.380.56
20 Newsgroups, GloVelabel precision of 10 neighborsnear0.670.340.83-0.71
same memories deletedfar0.930.970.590.86
same themes (ARI)far0.870.890.400.52
topic mapfar0.800.880.090.61
AG News, MobileCLIP2label precision of 10 neighborsnear0.310.49-0.260.47
same memories deletedfar0.860.930.320.74
same themes (ARI)far0.730.830.030.59
AG News, RoBERTalabel precision of 10 neighborsnear0.620.460.65-0.51
same memories deletedfar0.930.980.030.83
same themes (ARI)far0.860.850.320.21
AG News, GloVelabel precision of 10 neighborsnear0.240.220.100.04
same memories deletedfar0.860.950.420.85
same themes (ARI)far0.620.80-0.140.65

Table 6. The same split holds on LoCoMo and the topic corpora. Continuation of Table 5.

Deleting a LoCoMo session has partial correlations of 0.86, 0.55, and 0.93 with GG given LL, versus −0.11, 0.36, and −0.14 with LL given GG. The session and topic maps follow GG at 0.56 to 0.93 given LL, and the two global references, all distances and triplets, follow GG at 0.78 to 1.00. The operations defined on companies sit between the two ends. Peer groups, the barrier, and the company and sector maps compare 106 company centroids, making each a nearest-neighbor question in the space of companies. With the retrieval-tuned encoders, they follow LL (0.39 to 0.65 given GG) at least as closely as GG (0.21 to 0.51 given LL). In short, an operation follows the end of the profile at which it compares items.

Rescoring repairs the near end and keeps the stored codes

Rescoring repairs the near end while leaving the far end tied to the stored codes. Disk-based vector search keeps binary codes in memory and rescores candidates using full-precision vectors read from disk; Figure 2f, Figure 2g reproduces this pipeline using the 40 nearest codes. On FinQA, rescoring raises LL from 0.74 to 0.97 with BGE-base, from 0.61 to 0.87 with E5-base, and from 0.77 to 0.98 with BGE-large. It lifts top-10 agreement from 0.57 to 0.87, 0.30 to 0.53, and 0.62 to 0.90. Yet GG remains fixed at the value of the binary codes, as does every operation computed on them. Theme agreement on the financial memory stays at 0.79, 0.63, and 0.82, and LoCoMo session deletion stays at 0.56, 0.35, and 0.57. Judged by a recall check, rescored binary codes appear close to the uncompressed memory; judged by GG, they keep only the far end of plain binary codes. The same pattern holds for every code family that preserves Euclidean distances (Table 7).

Rescoring raises LL and retrieval for every code family, and GG and the far-end operations keep the values of the stored codes. Code families that keep Euclidean distances, d/8d/8 bytes per item. The 40 nearest codes are rescored with the full-precision vectors; LL and retrieval come from the rescored lists, GG and consolidation or deletion from the stored codes. Top-10 agreement: overlap of the top-10 lists. Last column: theme agreement (FinQA) or agreement of session deletion (LoCoMo).

MemoryEncoderCompressionLLGGRetrievalThemes / deletion
FinQABGE-base (96 B)PCA + int80.770.740.600.96
FinQABGE-base (96 B)PCA + int8 + rescoring0.990.740.910.96
FinQABGE-base (96 B)PQ0.820.730.680.88
FinQABGE-base (96 B)PQ + rescoring0.990.730.950.88
FinQABGE-base (96 B)OPQ0.880.830.780.93
FinQABGE-base (96 B)OPQ + rescoring1.000.830.990.93
FinQAE5-base (96 B)PCA + int80.770.730.570.96
FinQAE5-base (96 B)PCA + int8 + rescoring0.990.730.890.96
FinQAE5-base (96 B)PQ0.820.730.640.89
FinQAE5-base (96 B)PQ + rescoring1.000.730.940.89
FinQAE5-base (96 B)OPQ0.880.820.760.93
FinQAE5-base (96 B)OPQ + rescoring1.000.820.990.93
FinQABGE-large (128 B)PCA + int80.820.770.680.97
FinQABGE-large (128 B)PCA + int8 + rescoring1.000.770.960.97
FinQABGE-large (128 B)PQ0.840.740.710.89
FinQABGE-large (128 B)PQ + rescoring1.000.740.970.89
FinQABGE-large (128 B)OPQ0.910.860.830.94
FinQABGE-large (128 B)OPQ + rescoring1.000.861.000.94
LoCoMoBGE-base (96 B)PCA + int80.760.810.690.59
LoCoMoBGE-base (96 B)PCA + int8 + rescoring1.000.810.990.59
LoCoMoBGE-base (96 B)PQ0.780.790.700.65
LoCoMoBGE-base (96 B)PQ + rescoring1.000.790.980.65
LoCoMoBGE-base (96 B)OPQ0.870.890.810.79
LoCoMoBGE-base (96 B)OPQ + rescoring1.000.891.000.79
LoCoMoE5-base (96 B)PCA + int80.720.770.630.43
LoCoMoE5-base (96 B)PCA + int8 + rescoring0.990.770.970.43
LoCoMoE5-base (96 B)PQ0.770.750.670.59
LoCoMoE5-base (96 B)PQ + rescoring0.990.750.960.59
LoCoMoE5-base (96 B)OPQ0.850.850.780.70
LoCoMoE5-base (96 B)OPQ + rescoring1.000.850.990.70
LoCoMoBGE-large (128 B)PCA + int80.820.850.750.68
LoCoMoBGE-large (128 B)PCA + int8 + rescoring1.000.851.000.68
LoCoMoBGE-large (128 B)PQ0.800.800.740.66
LoCoMoBGE-large (128 B)PQ + rescoring1.000.800.980.66
LoCoMoBGE-large (128 B)OPQ0.900.910.860.82
LoCoMoBGE-large (128 B)OPQ + rescoring1.000.911.000.82

Table 7.

Evidence and fiscal year. With the retrieval-tuned encoders, evidence recall follows LL, with partial correlations of 0.71 to 0.83 given GG on FinQA and 0.61 to 0.87 on LoCoMo (Figure 9c in Appendix C). Because financial text repeats across years, a retrieved passage can come from the right company but the wrong fiscal year. In the uncompressed memories with the retrieval-tuned encoders, 0.18 of the right-company passages come from another fiscal year, and every linear compression keeps this share between 0.15 and 0.24 (Figure 9d).

Figure 9. Rescoring raises LL and retrieval and leaves GG and the far-end operations at the values of the binary codes. (a, b) Binary codes before and after full-precision rescoring of the 40 nearest codes; arrows run from binary to rescored, and a marker without an arrow is unchanged. (c) Evidence recall@10 on FinQA rises with LL; dotted lines: uncompressed memories. (d) The share of right-company passages from another fiscal year stays near its uncompressed level.

Same bytes, different ends

Memories of the same size differ most at the far end. At d/8d/8 bytes per item, 32 times smaller than full precision, we compare seven compressions. Bytes per item count only the per-item codes. The PCA basis, the rotation, and the codebooks are shared across all items (5.5 MB with BGE-base and 9.4 MB with BGE-large, identical for OPQ and LAQ), while rescoring additionally retains the full vectors on disk. At these bytes, a memory can store one-bit codes with or without rescoring, rotated one-bit codes, product-quantization codes, d/8d/8 principal components in 8 bits, or OPQ codes with or without rescoring. Figure 3a, Figure 3b and Figure 11 in Appendix C compare these seven options, while Table 8 lists all 27 compressions on FinQA with BGE-base. On FinQA with BGE-base, rescored binary codes have the highest LL among the first five methods (0.97) but yield a theme agreement of 0.79. PCA with 8-bit codes keeps theme agreement at 0.96. The same ordering holds with E5-base.

Figure 11. At d/8d/8 bytes, OPQ keeps the highest GG of seven compressions, and rescored binary codes keep a high LL with a lower GG. Fidelity and agreement with the uncompressed memory, retrieval-tuned encoders; (a–c) FinQA, (d–f) LoCoMo.

CompressionBytesLLGGRetrievalEvidenceThemesPeersBarrierMap
float1615361.001.001.000.501.001.001.001.00
int87680.980.970.970.500.990.990.981.00
int43840.960.950.940.500.990.990.971.00
binary, 1 bit960.740.650.570.440.790.800.870.97
binary, 2 bit1920.830.650.700.480.900.830.890.97
binary, 4 bit3840.860.720.760.490.940.850.900.97
binary + rescoring960.970.650.870.490.790.800.870.97
truncation 38415360.860.760.740.490.770.750.890.96
truncation 1927680.760.620.580.460.620.590.810.90
truncation 963840.640.470.400.330.440.460.660.80
truncation 481920.470.330.220.210.220.310.490.64
PCA 38415360.970.970.960.491.001.000.981.00
PCA 1927680.890.860.810.440.990.920.920.99
PCA 963840.770.740.600.350.960.780.840.95
PCA 481920.630.650.390.240.920.610.600.88
random projection 38415360.800.660.640.470.670.670.820.91
random projection 1927680.730.570.510.390.590.590.720.87
random projection 963840.620.450.370.370.340.400.570.73
random projection 481920.450.350.200.180.210.330.370.67
PCA 192 + int81920.880.840.800.430.990.930.920.99
PCA 96 + int8960.770.740.600.350.960.780.840.95
PCA 768 + rotation + binary960.770.570.620.450.850.820.880.96
PCA 384 + rotation + binary480.720.510.520.410.790.820.880.96
PQ 96960.820.730.680.440.880.870.900.98
PQ 48480.710.620.490.330.790.780.850.95
PQ 24240.600.510.320.200.520.650.780.89
UMAP 16640.520.100.240.190.080.200.690.59

Table 8. At equal bytes, compressions of the financial memory differ most in GG and theme agreement. FinQA, BGE-base, 27 compressions: bytes per item, fidelity, and agreement with the uncompressed memory. Top-10 agreement: overlap of the top-10 lists for 863 questions. Evidence: evidence recall@10 (0.50 uncompressed). Themes: theme agreement over 11 themes. Peers: overlap of five-company peer groups. Barrier: Jaccard overlap of the memories removed to wall off a company. Map: rank correlation of distances among company centroids.

(0.96 against 0.63) and with BGE-large at 128 bytes (0.97 against 0.82). The gap also shows by company. Even when every company keeps at least 91 percent of its neighbors, rescored binary BGE-base codes move a fifth or more of the passages to another theme for 5 of the 106 FinQA companies (Figure 10g). With PCA and 8-bit codes, every company keeps at least 86 percent of its passages in their original theme. On LoCoMo, product quantization and PCA with 8-bit codes delete a session with an agreement of 0.43 to 0.68, compared with 0.35 to 0.57 for binary codes. Across every encoder on both memories, OPQ has the highest GG of the seven compressions, 0.82 to 0.86 on FinQA and 0.85 to 0.91 on LoCoMo. It deletes a LoCoMo session with an agreement of 0.70 to 0.82. When rescored, OPQ reaches a top-10 agreement of 0.99 to 1.00. OPQ thus keeps both ends best among the seven compressions at these bytes, making it the natural starting point for LAQ.

LAQ on held-out memories

LAQ raises the far end of every held-out memory, keeps the near end, and improves the far-end operations. It also has the highest GG among quantizers of the same size. These gains extend to an external financial memory and to a memory of one million passages.

LAQ raises the far end of every held-out memory

LAQ increases GG in all 40 held-out runs (Table 2, Figure 1h). With the retrieval-tuned encoders, the gain ranges from 0.005 to 0.017. Every gain exceeds the standard deviation of GG over subsamples, which is at most 0.0023. With Titan V2, whose OPQ start keeps the least of the far end, the gain reaches 0.032 to 0.038. LL increases in 34 of the 40 runs and stays within 0.0023 of OPQ in the rest, so the far end is gained with the near end kept.

Memoryhigher OPQLAQLAQ − OPQ
FinQAGG12/12.800.808+.005 to +.010
FinQALL10/12.861.864−.002 to +.008
LoCoMoGG12/12.845.853+.005 to +.012
LoCoMoLL8/12.820.822+.002 to +.009
Multi-Doc-2025GG12/12.830.842+.007 to +.017
Multi-Doc-2025LL12/12.823.828+.0003 to +.010
Multi-Doc-2025, Titan V2GG4/4.656.692+.032 to +.038
Multi-Doc-2025, Titan V2LL4/4.790.798+.006 to +.011
FinDERGG12/12.821.835+.010 to +.017
FinDERLL12/12.809.814+.003 to +.008

Table 2. LAQ raises GG over its OPQ start in every held-out run. d/8d/8 bytes; runs in which LAQ is higher, means over runs, and the range of the change. Retrieval-tuned encoders (BGE-base, E5-base, BGE-large), four splits each.

Per-company intervals. These gains hold across the companies of each memory. LL, GG, and evidence recall@10 are means over items or queries. We construct paired bootstrap intervals (95% confidence, 10,000 resamples) by resampling these terms by held-out company. On LoCoMo we resample by session, since the held-out part of each split holds two conversations. The interval for GG lies above zero in all 52 held-out runs, including FinDER. The interval for LL lies

above zero in 25 runs and below zero in none. LAQ keeps evidence recall@10: its interval contains zero in 47 of the 52 runs.

Mechanism. The gain spans the whole profile (Figure 3c–Figure 3f). Across the 16 held-out Multi-Doc-2025 memories, LAQ lifts νk\nu_k above OPQ at all scales from k=10k = 10 to k⋆k^\star. The gain grows toward the GG band, where LAQ trains. Theorem 1 holds at all 160 combinations of memory, quantizer, and scale, with 1−νk1-\nu_k at 0.49 to 0.60 of the bound. The theorem also explains the mechanism. At k≈0.4nk \approx0.4n, LAQ reduces the violation rate pˉk\bar{p}_k by 10 to 16 percent with the retrieval-tuned encoders and by 16 to 20 percent with Titan V2. This lowers the bound on 1−νk1-\nu_k in the GG band, and the measured νk\nu_k rises there as well.

Ablation. The far hinge LfarL_{\mathrm{far}} carries the gain in GG (Figure 4a, Figure 4b; Table 10 in Appendix D). We retrained every held-out run with the same recipe and checkpoint rule and changed only the terms of Eq. (10). With reconstruction alone, training keeps the OPQ start C0C_0 in all 52 runs, since OPQ already fits its codebooks to reconstruction. A near hinge, Eq. (11) at k=10k = 10 or 50 on stored and ADC distances, raises GG by 0.0058 on average. Reconstruction with LfarL_{\mathrm{far}} raises GG in all 52 runs, by 0.0127 on average, and above the near hinge in 48 of them (paired Wilcoxon test, p=6×10−9p = 6 \times10^{-9}). The full objective raises GG by 0.0124. With the retrieval-tuned encoders, its GG equals that of LfarL_{\mathrm{far}} alone (mean difference 0.0001), and the middle hinge adds to LL (0.0036 against 0.0028). With Titan V2, LfarL_{\mathrm{far}} alone raises GG by 0.041 and the full objective by 0.036, with the larger gain in LL (0.008 against 0.005). The far end is thus learned from the order of cross pairs in the GG band, the pairs that Theorem 1 links to νk\nu_k.

Multi-panel comparison of LAQ training objectives, hinge effects, paired tests, and company-bootstrap intervals

Figure 4. Reconstruction alone leaves GG at the OPQ start; the far hinge lifts it in all 52 held-out runs. Change in GG for each training objective, every held-out run. LAQ gains what Lfar\mathcal{L}_{\mathrm{far}} alone gains. The far hinge acts in the GG band, where Theorem 1 ties νk\nu_k to rank violations. Gain by scale, retrieval-tuned encoders. Violation rate cut at k≈0.4nk \approx0.4n. Two ends, two hinges. Paired tests: LAQ has higher GG than OPQ in 52 of 52 runs and than each other quantizer in 12 of 12. LAQ against OPQ, every held-out run. LAQ against other quantizers, GG. Company bootstrap, GG. BGE-base; E5-base; BGE-large; Titan V2. Fig. 4. The far hinge carries the gain in GG, and LAQ is better than OPQ and the other quantizers in paired tests. (a) Change in GG from the OPQ start for each training objective on all 52 held-out runs; bars are means. (b) LAQ against reconstruction with Lfar\mathcal{L}_{\mathrm{far}} alone, run by run. (c) Mean change in νk\nu_k by scale with the retrieval-tuned encoders; shading, interquartile range; gray, the GG band. (d) Reduction of the violation rate pˉk\bar{p}_k at k≈0.4nk \approx0.4n relative to OPQ. (e) Change in LL against change in GG; large markers are means. (f) One-sided Wilcoxon tests of LAQ against OPQ over runs, Holm-corrected, with the runs in which LAQ is higher; evidence recall@10 is tested two-sided. (g) GG of LAQ minus that of each quantizer on Multi-Doc-2025, 12 runs. (h) Paired company-bootstrap 95 percent intervals for GG, LAQ minus OPQ.

ObjectiveGG changerangehigherLL change
Lrec\mathcal{L}_{\mathrm{rec}}.0000.0000 to .00000/52.0000
Lrec\mathcal{L}_{\mathrm{rec}} + near hinge+.0058.0000 to +.012450/52+.0043
Lrec\mathcal{L}_{\mathrm{rec}} + Lfar\mathcal{L}_{\mathrm{far}}+.0127+.0022 to +.045352/52+.0030
LAQ, Eq. (10)+.0124+.0047 to +.037952/52+.0039

Table 10. The far hinge Lfar\mathcal{L}_{\mathrm{far}} carries the gain in GG. Change from the OPQ start over all 52 held-out runs, each objective trained with the same recipe and checkpoint rule; means, range of the change in GG, and runs in which GG is higher.

Where the far hinge acts. The far hinge acts in the GG band (Figure 4c–Figure 4e). The near hinge lifts νk\nu_k most near k=0.05nk = 0.05n, whereas LfarL_{\mathrm{far}} and LAQ lift it most in the GG band. At k≈0.4nk \approx0.4n, the near hinge reduces the violation rate by 6 percent with the retrieval-tuned encoders, and LfarL_{\mathrm{far}} and LAQ by 11 percent.

Paired tests. We test whether LAQ is better than OPQ with a one-sided Wilcoxon signed-rank test over the 52 held-out runs and correct each family of tests by the Holm method (Figure 4f). Runs that share a corpus are dependent, so we repeat each test as a sign test over the 13 combinations of memory and encoder, each the mean over its splits. LAQ has higher GG, LL, all-distance correlation, triplet agreement, and median split at p≤6.4×10−9p \le6.4 \times10^{-9} over runs and p=8.5×10−4p = 8.5 \times10^{-4} over the 13 combinations. Theme agreement and session deletion are higher at p=0.028p = 0.028 and p=0.046p = 0.046 over runs. Evidence recall@10 is equal to that of OPQ (p=0.84p = 0.84, two-sided). The company-bootstrap intervals for GG lie above zero in all 52 runs (Figure 4h).

The gain reaches the far-end operations

Global references and maps. The far-end operations of Section 6 move with GG (Figure 3g). Both global references and the median split improve in all 40 runs. The Spearman correlation of all distances increases by 0.002 to 0.016 with the retrieval-tuned encoders and by 0.046 to 0.063 with Titan V2. The median split increases by 0.004 to 0.019 and by 0.036 to 0.043, respectively. The company map improves in 27 of 28 runs.

The session map improves in 10 of 12 runs, and the remaining two stay within 0.001 of OPQ.

Themes and deletion. Consolidation into the themes gains the most where the OPQ start keeps it least. On Multi-Doc-2025 with the retrieval-tuned encoders, the mean theme agreement rises from 0.908 to 0.931. The lowest theme agreement across the 12 runs rises from 0.800 to 0.886 (Figure 3i). With Titan V2, both keep theme agreement at 0.98 or above. Across the 16 Multi-Doc-2025 runs, OPQ moves 1,203 passages to another theme, whereas LAQ moves 902. On FinQA, the mean theme agreement rises from 0.911 to 0.914. For conversational memory, deleting a LoCoMo session improves in 9 of 12 runs, with mean agreement rising from 0.686 to 0.695.

Retrieval. Retrieval is served by the near end, which LAQ keeps. Top-10 agreement increases in 27 of 40 runs. Mean evidence recall@10 stays within 0.005 of OPQ in every memory: 0.545 versus 0.546 on FinQA, 0.649 versus 0.649 on LoCoMo, 0.795 versus 0.796 on Multi-Doc-2025, and 0.749 versus 0.754 with Titan V2. Peer groups and the information barrier are nearest-neighbor questions among company centroids. As Section 6 found, they follow LL at least as closely as GG. LAQ raises peer agreement in 17 of 28 runs and keeps barrier agreement within 0.011 of OPQ in every run. All runs are listed in Appendix D (Table 9).

MemoryEncoderSplitGG OPQGG LAQLL OPQLL LAQ
FinQABGE-base20260.79270.80210.85370.8566
20270.79300.80200.85790.8570
20280.78850.79510.85340.8557
20290.79580.80350.84890.8514
E5-base20260.78720.79590.84130.8458
20270.78580.79520.84630.8544
20280.77810.78840.84400.8486
20290.78670.79440.84690.8450
BGE-large20260.82270.82800.88190.8840
20270.82090.82610.88440.8869
20280.81850.82600.88840.8910
20290.82660.83420.88790.8880
LoCoMoBGE-base20260.86670.87440.81860.8224
20270.85720.86410.83490.8358
20280.83190.84400.81480.8182
20290.87220.87770.83190.8302
E5-base20260.79790.80430.76510.7671
20270.76810.77850.78590.7946
20280.77030.78070.76120.7697
20290.80720.81560.78250.7806
BGE-large20260.89210.89940.84960.8574
20270.89280.89890.88130.8790
20280.86780.87590.85220.8518
20290.91280.91750.85870.8602
Multi-DocBGE-base20260.81960.83060.81320.8183
20270.81400.82660.81810.8208
20280.82740.83810.81020.8122
20290.81650.82710.81830.8207
E5-base20260.81540.83200.80180.8092
20270.81540.82900.80680.8122
20280.81670.83020.79440.7999
20290.81400.82820.80370.8132
BGE-large20260.85520.86470.85260.8572
20270.85400.86350.85640.8601
20280.86080.86860.84620.8524
20290.85450.86170.85570.8559
Titan V220260.66800.70250.79420.8010
20270.64340.68110.79620.8043
20280.66510.70300.78480.7958
20290.64800.68030.78520.7909
FinDERBGE-base20260.81390.83030.80460.8072
20270.81420.83120.80240.8055
20280.81420.83050.80290.8069
20290.81370.82890.80490.8076
E5-base20260.79820.81570.78090.7893
20270.80280.82020.77930.7854
20280.80750.82380.78120.7883
20290.80320.82000.78420.7905
BGE-large20260.84560.85590.84500.8480
20270.84580.85580.84450.8494
20280.84610.85630.84330.8478
20290.84390.85670.83930.8454

Table 9. LAQ raises GG over OPQ on every held-out memory. GG and LL of OPQ and LAQ, four splits.

LAQ against other quantizers at the same size

LAQ has the highest GG among the quantizers of its size (Table 3, Figure 3h). In each of the nine runs with the retrieval-tuned encoders, LAQ has both the highest GG and the highest LL of all codes without rescoring. LSQ++, the strongest additive quantizer tested, reaches a GG of 0.73 to 0.77, against 0.83 to 0.87 for LAQ. With Titan V2, LAQ has the highest GG in all three runs, and ScaNN, fitted on the held-out memory itself, has the highest LL. With rescoring of the 40 nearest codes, LAQ reaches an LL of 0.995 to 0.999 in all twelve runs. This is above rescored sign codes (0.861 to 0.996) and rescored ITQ codes (0.938 to 0.994) in each run, and the stored LAQ codes keep the highest GG. Against each of OPQ, PQ, LSQ++, RQ, PCA + int8, and RaBitQ, LAQ has higher GG and LL in all 12 runs (paired Wilcoxon test, p=2.9×10−3p = 2.9 \times10^{-3} after Holm correction; Figure 4g).

BGE-baseE5-baseBGE-largeTitan V2
LAQ.832 / .817.830 / .807.866 / .857.696 / .800
OPQ.820 / .814.816 / .801.857 / .852.659 / .792
PQ.754 / .743.759 / .749.769 / .771.661 / .787
LSQ++.759 / .725.730 / .689.764 / .744.543 / .679
RQ.693 / .592.693 / .552.697 / .598.429 / .511
PCA + int8.729 / .631.710 / .584.768 / .695.521 / .578
RaBitQ.677 / .666.561 / .545.712 / .704.656 / .776
ScaNN– / .615– / .507– / .578– / .837

Table 3. LAQ has the highest GG with every encoder. GG / LL on held-out Multi-Doc-2025 at d/8d/8 bytes, means over the first three splits. RaBitQ uses 104 and 136 bytes. ScaNN is fitted on the held-out memory and searched with full-precision queries; we report its LL.

An external financial memory

These gains hold on FinDER, a memory built by others from the queries of financial professionals. It was run once, with the recipe unchanged, on 124 held-out companies in each of four splits (Table 2). LAQ raises GG in all 12 runs by 0.010 to 0.017, lifting the mean from 0.821 to 0.835. Every gain exceeds the standard deviation of GG over subsamples, which is at most 0.0025 here. LL increases in all 12 runs by 0.003 to 0.008, and top-10 agreement rises in all 12. Mean evidence recall@10 is 0.458 for both, within 0.007 of OPQ in every run.

A memory of a million passages

LAQ also raises GG on a memory of one million passages. The memory holds 1,000,000 passages of MS MARCO [50], encoded with BGE-base and stored at 96 bytes each. OPQ and LAQ were fitted on 100,000 other passages, and the 6,980 development queries search the memory by ADC. At this size, training draws a new subsample of 8,192 reference items at each step and chooses the cross pairs by their ranks within it, the argument behind the estimator of GG. It decodes only the items a step uses and averages the reconstruction term over 16,384 items. All other settings are those of the recipe. Over three seeds, LAQ raises GG from 0.774 to 0.784 to 0.785. The gain of 0.010 to 0.011 is about three times the standard deviation of GG over subsamples (at most 0.0039). LL, computed on the 10 nearest neighbors of 20,000 random passages, stays within 0.002 of OPQ. Top-10 agreement is 0.818 for OPQ and 0.818 to 0.820 for LAQ. Evidence recall@10, over the 820 queries whose relevant passage is in the memory, is 0.860 for OPQ and within 0.008 of it for LAQ. One training run takes 175 seconds and at most 10.0 GiB of one 48 GB GPU, and a second run reproduces the codebooks bit for bit.

Discussion

Why the far end can be learned. Ordering cross pairs controls the far end, and this is what makes LAQ work. The bound has a leading factor near 2 across the GG band and near N/10\sqrt{N}/10 at k=10k = 10. Reconstruction and nearest-neighbor objectives serve the near end. LAQ combines the two through a hinge loss at middle and far scales, a normalized reconstruction term for the near end, and a checkpoint selected on the far end of a monitor set. The ablation of § 7 isolates the far hinge: reconstruction alone keeps the OPQ start, and LfarL_{\mathrm{far}} alone gives the full gain in GG in all 52 runs (Figure 4). The gain is largest where the OPQ codes keep the least of the far end, as with Titan V2, and it reaches the operations that follow GG.

Cost against other quantizers. LAQ trains in minutes, between PQ and the additive quantizers, and keeps the query cost of OPQ. Each LAQ step costs about one kk-means iteration of PQ and OPQ [4, 16], plus a one-time ranking of the training memory. In contrast, additive quantizers spend their cost on encoding. RQ keeps several partial codes per item in a beam search, and LSQ++ encodes vectors by joint local search over all codebooks [27, 28]. ITQ alternates a d×dd \times d rotation with sign codes [30], and RaBitQ applies a random rotation without training [31]. On the Multi-Doc-2025 training memories, the fast implementations [59] on CPU took 6 to 12 seconds to fit PQ, 146 to 517 seconds for ITQ, 511 to 1,193 seconds for RQ, and 8.2 to 17.1 hours for LSQ++. By comparison, LAQ took 35 to 112 seconds on one GPU after its OPQ start. LAQ and OPQ encode a new item in O(Kd)O(Kd) time, while RQ and LSQ++ repeat their search for every item added to the memory. LAQ thus has the highest GG among these quantizers (Table 3) at a training cost between PQ and the additive quantizers.

What to report. A compressed memory can be reported with (L,G)(L,G) against the memory it replaces, computed on the operator’s own vectors. LL says whether queries return the same items. GG says whether the memory keeps its far end: which items are consolidated or deleted together, and how sessions, companies, and themes relate. Neither needs labels, and GG is estimated from a subsample of 2,000 items. Where a memory serves consolidation or deletion, GG belongs in its acceptance test. The compression is then chosen as the one with the highest GG among those with the required LL at the bytes available. Theorem 1 adds a diagnostic: the violation rate of cross pairs at a scale shows which part of the profile a compression loses.

Financial memory. The consequences are clearest in financial memories. A consolidation that groups filings differently changes the sector briefs an analyst agent writes. A deletion that selects different passages changes what a client request removes. Peer groups, the information barrier, and the company map compare company centroids. A check of these operations combines LL with agreement measured on the firm’s own companies.

Extensions. Residual and additive codebooks are natural next targets for this objective, and sampling reference items already extends training to one million passages (§ 7.5).

Conclusion

Agent memories are compressed and accepted by their nearest neighbors, even though agents also consolidate, delete, and map what they hold. One label-free profile measures both ends. Its near end predicts retrieval, and with retrieval-tuned encoders its far end predicts consolidation and deletion, each beyond the other. Our main result is that the order of near and far items bounds the profile at every scale, and that this bound can be trained. Locality-aware quantization (LAQ) learns this order and raises the global fidelity of every held-out memory at the same bytes and search as the OPQ index it replaces. To our knowledge, LAQ is the first quantizer whose objective comes from a proven bound on the far end of a memory. It needs no labels and no change at query time, and it applies wherever embeddings are compressed and then clustered, deduplicated, or mapped. For agent memory, (L,G)(L,G) gives an acceptance test for both ends, and LAQ gives a way to pass it.

Data and code: https://github.com/hank08819/LAQ

Appendix A: Proofs

This appendix proves the results of Sections 3 and 4, except Theorem 1, whose proof is in Section 3.4.

Proof of Proposition 1. (i) Each of the kk members of Nkx(i)N_k^x(i) lies in a uniform kk-subset of NN items with probability k/Nk/N, so E∣Nkx(i)∩Nkz(i)∣=k2/N\mathbb{E}|N_k^x(i) \cap N_k^z(i)| = k^2/N. Substitution in (2) gives 0. (ii) A rotation, reflection, or translation keeps every distance, and a uniform scaling multiplies every distance by one positive constant. A strictly increasing transformation of a distance keeps its order. In each case every ranking rizr_i^z, and hence every Nkz(i)N_k^z(i), is unchanged.

Proof of Proposition 2. Inclusion–exclusion gives the identity. At 2k⋆=N2k^\star= N it reads ∣Fx∩Fz∣=∣Nx∩Nz∣|F^x \cap F^z| = |N^x \cap N^z|, so item ii has 2∣Nk⋆x(i)∩Nk⋆z(i)∣2|N_{k^\star}^x(i) \cap N_{k^\star}^z(i)| agreements out of NN and A=DLPk⋆A = \mathrm{DLP}_{k^\star}. (2) with N−k⋆=k⋆N-k^\star= k^\star gives νk⋆=2DLPk⋆−1\nu_{k^\star} = 2\mathrm{DLP}_{k^\star} - 1.

Proof of Proposition 3. The transformations map both medians to 0. For a bivariate normal pair with unit variances and correlation γ\gamma, Sheppard’s formula [60] gives Pr⁡(both coordinates on the same side of 0)=12+1πarcsin⁡γ\Pr(\text{both coordinates on the same side of 0}) = \frac{1}{2} + \frac{1}{\pi}\arcsin\gamma, so E(2Ai−1)=2πarcsin⁡γi\mathbb{E}(2A_i - 1) = \frac{2}{\pi}\arcsin\gamma_i. Kendall’s τ\tau and Spearman’s ρ\rho of the bivariate normal take the forms above [61], and both increase in γ\gamma.

Proof of Corollary 1. A violated pair has Δz≤0\Delta^z \le0, so h≥ηΔxh \ge\eta\Delta^x. If also Δx≥δ\Delta^x \ge\delta, then h≥ηδh \ge\eta\delta. Hence 1[violated]≤1[Δx<δ]+h/(ηδ)\mathbb{1}[\text{violated}] \le\mathbb{1}[\Delta^x < \delta] + h/(\eta\delta), and taking expectations gives (7). Rankings by squared distance equal rankings by distance, so Δz≤0\Delta^z \le0 for every violated pair, with equality when two compressed distances tie.

Proof of Proposition 4. As n→∞n \to\infty, the kk nearest items of ii are those with uj<zαu_j < z_\alpha, so a cross pair (p,q)(p,q) has v=up<zα<u=uqv = u_p < z_\alpha< u = u_q, with density ϕ(u)ϕ(v)/(α(1−α))\phi(u)\phi(v)/(\alpha(1-\alpha)) on this region. The pair is violated when D~q<D~p\widetilde{D}_q < \widetilde{D}_p, that is, when u+eq/si<v+ep/siu + e_q/s_i < v + e_p/s_i. The difference (ep−eq)/si(e_p-e_q)/s_i is normal with variance 2ωi22\omega_i^2, so this happens with probability Φ(−(u−v)/(2ωi))\Phi(-(u-v)/(\sqrt{2}\omega_i)). Integrating over the region gives (8). For small ω\omega, write u=zα+xu = z_\alpha+ x and v=zα−yv = z_\alpha-y with x,y≥0x,y \ge0. The factor Φ(−(x+y)/(2ω))\Phi(-(x+y)/(\sqrt{2}\omega)) confines the integral to x+y=O(ω)x+y = O(\omega), where ϕ(u)ϕ(v)=ϕ(zα)2(1+O(x+y))\phi(u)\phi(v) = \phi(z_\alpha)^2(1+O(x+y)). With t=x+yt=x+y, the area element of {x,y≥0,x+y∈dt}\{x,y \ge0, x+y \in dt\} is t dtt\,dt, and

∫0∞tΦ(−t2ω) dt=2ω2∫0∞sΦ(−s) ds=ω22,\int_0^\infty t\Phi\left(-\frac{t}{\sqrt{2}\omega}\right)\,dt =2\omega^2\int_0^\infty s\Phi(-s)\,ds =\frac{\omega^2}{2},

since ∫0∞sΦ(−s) ds=12∫0∞s2ϕ(s) ds=14\int_0^\infty s\Phi(-s)\,ds = \frac{1}{2}\int_0^\infty s^2\phi(s)\,ds = \frac{1}{4} by parts. The O(x+y)O(x+y) term contributes O(ω3)O(\omega^3).

Proof of Corollary 2. Write ϵj=ej/si\epsilon_j = e_j/s_i. The compressed distances rank the items as uj+ϵju_j+\epsilon_j does, and yj=(uj+ϵj)/1+ωi2y_j=(u_j+\epsilon_j)/\sqrt{1+\omega_i^2} is N(0,1)\mathcal{N}(0,1) with correlation ϱi\varrho_i to uju_j. As n→∞n \to\infty, the kk nearest items of ii are those with uj<zαu_j < z_\alpha before compression and those with yj<zαy_j < z_\alpha after it, so the share of the kk nearest items that both keep converges to Φ2(zα,zα;ϱi)/α\Phi_2(z_\alpha,z_\alpha;\varrho_i)/\alpha. Substituting this share for DLPk\mathrm{DLP}_k in (2), with k/N→αk/N \to\alpha, gives (9). Plackett’s identity ∂Φ2(z,z;ϱ)/∂ϱ=ϕ2(z,z;ϱ)\partial\Phi_2(z,z;\varrho)/\partial\varrho= \phi_2(z,z;\varrho), along with Φ2\Phi_2 the bivariate normal density, shows that ν\nu increases strictly in ϱi\varrho_i, which decreases strictly in ωi\omega_i. At α=1/2\alpha=1/2, zα=0z_\alpha=0 and Sheppard’s formula gives Φ2(0,0;ϱ)=14+12πarcsin⁡ϱ\Phi_2(0,0;\varrho)=\frac{1}{4}+\frac{1}{2\pi}\arcsin\varrho, so ν=2πarcsin⁡ϱi\nu=\frac{2}{\pi}\arcsin\varrho_i. Finally, 1−ν=(α−Φ2)/(α(1−α))1-\nu=(\alpha-\Phi_2)/(\alpha(1-\alpha)), and α−Φ2\alpha-\Phi_2 is the probability that uj<zα<uj+ϵju_j<z_\alpha<u_j+\epsilon_j after rescaling. As ωi→0\omega_i\to0, this probability is ϕ(zα)E[ϵj]+o(ωi)=ϕ(zα)ωi/2π+o(ωi)\phi(z_\alpha)\mathbb{E}[\epsilon_j]+o(\omega_i)=\phi(z_\alpha)\omega_i/\sqrt{2\pi}+o(\omega_i), so 1−ν=ϕ(zα)ωi/(2πα(1−α))+o(ωi)1-\nu=\phi(z_\alpha)\omega_i/(\sqrt{2\pi}\alpha(1-\alpha))+o(\omega_i), which is 0.64ωi0.64\omega_i at α=0.4\alpha=0.4.

Proof of Proposition 5. Because no subvector is equidistant from its two nearest codewords, the assignments of C0C_0 stay unchanged for small ϵ\epsilon, and every decoded vector is then a linear function of the codebooks. The reconstruction error is Lrec(C)=1n∑ℓ,j∑i:aℓ(i)=j∥yi,ℓ−cℓ,j∥2\mathcal{L}_{\mathrm{rec}}(C)=\frac{1}{n}\sum_{\ell,j}\sum_{i:a_\ell(i)=j}\|y_{i,\ell}-c_{\ell,j}\|^2, with aℓ(i)a_\ell(i) the assignment, a quadratic in CC. Its gradient with respect to cℓ,jc_{\ell,j} is 2n∑i:aℓ(i)=j(cℓ,j−yi,ℓ)\frac{2}{n}\sum_{i:a_\ell(i)=j}(c_{\ell,j}-y_{i,\ell}), which is zero when cℓ,jc_{\ell,j} is the mean of its subvectors, and its Hessian is 2nℓ,jn\frac{2n_{\ell,j}}{n} times the identity on cℓ,jc_{\ell,j}. A second-order expansion along −h-h, which has no remainder for a quadratic, gives the first line of (12). The second line is the first-order expansion of Lfar\mathcal{L}_{\mathrm{far}} along −h-h, whose directional derivative is −⟨g,h⟩-\langle g,h\rangle.

Appendix B: LAQ recipe

Every LAQ run uses one recipe: M=d/8M=d/8 subspaces of 8 dimensions with 256 codewords each, the full PCA basis, 1,000 steps, 1,024 anchors per scale group and step, λmid=λfar=1\lambda_{\mathrm{mid}}=\lambda_{\mathrm{far}}=1, margin η=1\eta=1, Adam with a learning rate of 0.002 times the standard deviation of C0C_0 and cosine decay, temperature from T0T_0 to 0.05T00.05T_0, a monitor set of 2,000 training items checked every 250 steps, and a reconstruction limit of 1.15 for a checkpoint. The scale normalizers smids_{\mathrm{mid}} and sfars_{\mathrm{far}} are medians of 16,384 uncompressed gaps sampled at the start.

Additional measurement results

The figures and tables of this appendix give every run behind the main figures. Figure 5 places the 27 compressions of the two agent memories on the (L,G)(L, G) plane for the retrieval-tuned encoders, and Figure 6 for the three encoders of other kinds. Figure 7 and 8 relate each operation to LL and GG, and Figure 9 gives rescoring. The topic corpora repeat the pattern of the agent memories: Figure 10 places their compressions on the plane and breaks the 96-byte comparison of FinQA down by company. Figure 11 gives the seven compressions at d/8d/8 bytes. Table 4–8 give the nonlinear maps, the correlations behind Figures 7 and 8, rescoring for every code family that keeps Euclidean distances, and every compression of the financial memory.

LAQ against OPQ by run

References

  1. [1]P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktaschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive NLP tasks,” in Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 9459–9474.
  2. [2]Amazon Web Services, “Building smarter AI agents: AgentCore long-term memory deep dive,” AWS Machine Learning Blog, Oct. 2025.
  3. [3]A. Kusupati, G. Bhatt, A. Rege, M. Wallingford, A. Sinha, V. Venkataraman, W. Howard-Snyder, K. Chen, S. Kakade, P. Jain, and A. Farhadi, “Matryoshka representation learning,” in Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 30 233–30 249.arxiv.org/abs/2205.13147
  4. [4]H. Jégou, M. Douze, and C. Schmid, “Product quantization for nearest neighbor search,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 1, pp. 117–128, 2011.DOI
  5. [5]Amazon Web Services, “Build cost-effective RAG applications with binary embeddings in Amazon Titan Text Embeddings V2, Amazon OpenSearch Serverless, and Amazon Bedrock Knowledge Bases,” AWS Machine Learning Blog, Nov. 2024.
  6. [6]A. Govindaraju and V. V. Nakkirtha, “Cost optimized vector database: Introduction to Amazon OpenSearch Service quantization techniques,” AWS Big Data Blog, Jan. 2025.
  7. [7]N. Huerga-Pérez, R. Álvarez, R. Ferrero-Guillén, A. Martínez-Gutiérrez, and J. Díez-González, “Optimization of embeddings storage for RAG systems using quantization and dimensionality reduction techniques,” arXiv preprint arXiv:2505.00105, 2025.
  8. [8]R. Kisako, H. Tsukagoshi, and R. Sasano, “When is 0.1% enough? Analyzing the combined effects of dimensionality reduction and quantization on text embedding compression,” arXiv preprint arXiv:2606.01074, 2026.arxiv.org/abs/2606.01074
  9. [9]P. Sarthi, S. Abdullah, A. Tuli, S. Khanna, A. Goldie, and C. D. Manning, “RAPTOR: Recursive abstractive processing for tree-organized retrieval,” in International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2401.18059
  10. [10]M. Grootendorst, “BERTopic: Neural topic modeling with a class-based TF-IDF procedure,” arXiv preprint arXiv:2203.05794, 2022.arxiv.org/abs/2203.05794
  11. [11]“Multi-Doc-2025: A financial question-answering benchmark over SEC 10-K filings,” Hugging Face dataset Anonymous-Team-HC-RAG/Multi-doc-2025, 2026.
  12. [12]S. Xiao, Z. Liu, P. Zhang, N. Muennighoff, D. Lian, and J.-Y. Nie, “Packed resources for general Chinese embeddings,” in Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2024, pp. 641–649.arxiv.org/abs/2309.07597
  13. [13]L. van der Maaten and G. Hinton, “Visualizing data using t-SNE,” Journal of Machine Learning Research, vol. 9, pp. 2579–2605, 2008.
  14. [14]Z. Chen, W. Chen, C. Smiley, S. Shah, I. Borova, D. Langdon, R. Moussa, M. Beane, T.-H. Huang, B. Routledge, and W. Y. Wang, “FinQA: A dataset of numerical reasoning over financial data,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 3697–3711.arxiv.org/abs/2109.00122
  15. [15]A. Maharana, D.-H. Lee, S. Tulyakov, M. Bansal, F. Barbieri, and Y. Fang, “Evaluating very long-term conversational memory of LLM agents,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024.
  16. [16]T. Ge, K. He, Q. Ke, and J. Sun, “Optimized product quantization,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 4, pp. 744–755, 2014.DOI
  17. [17]H. Han, “Quantifying local and global fidelity of nonlinear embeddings,” arXiv preprint arXiv:2611.00290v2 [cs.AI], 2026, doi: 10.48550/arXiv.2611.00290.arxiv.org/abs/2611.00290
  18. [18]H. Han, W. Li, J. Wang, Q. Qin, and X. Qin, “Enhance explainability of manifold learning,” Neurocomputing, vol. 500, pp. 877–895, 2022.DOI
  19. [19]T. Yu, J. Yuan, C. Fang, and H. Jin, “Product quantization network for fast image retrieval,” in European Conference on Computer Vision (ECCV), 2018, pp. 186–201.DOI
  20. [20]A. Sablayrolles, M. Douze, C. Schmid, and H. Jégou, “Spreading vectors for similarity search,” in International Conference on Learning Representations (ICLR), 2019.arxiv.org/abs/1806.03198
  21. [21]T. Chen, L. Li, and Y. Sun, “Differentiable product quantization for end-to-end embedding compression,” in International Conference on Machine Learning (ICML), ser. PMLR, vol. 119, 2020, pp. 1617–1626.arxiv.org/abs/1908.09756
  22. [22]J. Zhan, J. Mao, Y. Li, J. Guo, M. Zhang, and S. Ma, “Jointly optimizing query encoder and product quantization to improve retrieval performance,” in ACM International Conference on Information and Knowledge Management (CIKM), 2021, pp. 2487–2496.arxiv.org/abs/2108.00644
  23. [23]S. Xiao et al., “Distill-VQ: Learning retrieval oriented vector quantization by distilling knowledge from dense embeddings,” in ACM SIGIR Conference on Research and Development in Information Retrieval, 2022.arxiv.org/abs/2204.00185
  24. [24]J. A. Lee and M. Verleysen, “Quality assessment of dimensionality reduction: Rank-based criteria,” Neurocomputing, vol. 72, pp. 1431–1443, 2009.DOI
  25. [25]J. A. Lee, D. H. Peluffo-Ordóñez, and M. Verleysen, “Multi-scale similarities in stochastic neighbour embedding: Reducing dimensionality while preserving both local and global structure,” Neurocomputing, vol. 169, pp. 246–261, 2015.DOI
  26. [26]C. Choi, J. Kwon, J. Ha, H. Choi, C. Kim, Y. Lee, J.-y. Sohn, and A. Lopez-Lira, “FinDER: Financial dataset for question answering and evaluating retrieval-augmented generation,” arXiv preprint arXiv:2504.15800, 2025.arxiv.org/abs/2504.15800
  27. [27]Y. Chen, T. Guan, and C. Wang, “Approximate nearest neighbor search by residual vector quantization,” Sensors, vol. 10, no. 12, pp. 11 259–11 273, 2010.DOI
  28. [28]J. Martinez, S. Zakhmi, H. Hoos, and J. J. Little, “LSQ++: Lower running time and higher recall in multi-codebook quantization,” in European Conference on Computer Vision (ECCV), 2018.DOI
  29. [29]R. Guo, P. Sun, E. Lindgren, Q. Geng, D. Simcha, F. Chern, and S. Kumar, “Accelerating large-scale inference with anisotropic vector quantization,” in International Conference on Machine Learning (ICML), 2020.
  30. [30]Y. Gong, S. Lazebnik, A. Gordo, and F. Perronnin, “Iterative quantization: A Procrustean approach to learning binary codes for large-scale image retrieval,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 12, pp. 2916–2929, 2013.DOI
  31. [31]J. Gao and C. Long, “RaBitQ: Quantizing high-dimensional vectors with a theoretical error bound for approximate nearest neighbor search,” in ACM SIGMOD International Conference on Management of Data, 2024.arxiv.org/abs/2405.12497
  32. [32]S. Morozov and A. Babenko, “Unsupervised neural quantization for compressed-domain similarity search,” in IEEE/CVF International Conference on Computer Vision (ICCV), 2019.arxiv.org/abs/1908.03883
  33. [33]J. Zhan et al., “Learning discrete representations via constrained clustering for effective and efficient dense retrieval,” in ACM International Conference on Web Search and Data Mining (WSDM), 2022.arxiv.org/abs/2110.05789
  34. [34]M. Aumüller, E. Bernhardsson, and A. Faithfull, “ANN-Benchmarks: A benchmarking tool for approximate nearest neighbor algorithms,” Information Systems, vol. 87, p. 101374, 2020.
  35. [35]L. Kuffo and P. Boncz, “Stop indexing at full precision: Revisiting clustering for vector embeddings,” in VLDB Workshop on Vector Databases, 2026, arXiv:2608.14648.arxiv.org/abs/2608.14648
  36. [36]D. S. Turaga, M. Vlachos, and O. Verscheure, “On k-means cluster preservation using quantization schemes,” in IEEE International Conference on Data Mining, 2009.DOI
  37. [37]W. Lueks, B. Mokbel, M. Biehl, and B. Hammer, “How to evaluate dimensionality reduction? Improving the co-ranking matrix,” arXiv preprint arXiv:1110.3917, 2011.arxiv.org/abs/1110.3917
  38. [38]N. Kury, D. Kobak, and S. Damrich, “DREAMS: Preserving both local and global structure in dimensionality reduction,” Transactions on Machine Learning Research, 2026.arxiv.org/abs/2508.13747
  39. [39]S. Ghosh, C. Maitra, and R. K. De, “CMET: Clustering guided metric for quantifying embedding quality,” arXiv preprint arXiv:2507.04840, 2025.arxiv.org/abs/2507.04840
  40. [40]D. Kobak and P. Berens, “The art of using t-SNE for single-cell transcriptomics,” Nature Communications, vol. 10, p. 5416, 2019.DOI
  41. [41]Y. Wang, H. Huang, C. Rudin, and Y. Shaposhnik, “Understanding how dimension reduction tools work: An empirical approach to deciphering t-SNE, UMAP, TriMAP, and PaCMAP for data visualization,” Journal of Machine Learning Research, vol. 22, no. 201, pp. 1–73, 2021.arxiv.org/abs/2012.04456
  42. [42]D. Wu, H. Wang, W. Yu, Y. Zhang, K.-W. Chang, and D. Yu, “LongMemEval: Benchmarking chat assistants on long-term interactive memory,” in International Conference on Learning Representations, 2025.arxiv.org/abs/2410.10813
  43. [43]D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, and J. Larson, “From local to global: A graph RAG approach to query-focused summarization,” arXiv preprint arXiv:2404.16130, 2024.arxiv.org/abs/2404.16130
  44. [44]L. McInnes, J. Healy, and J. Melville, “UMAP: Uniform manifold approximation and projection for dimension reduction,” arXiv:1802.03426, 2018.arxiv.org/abs/1802.03426
  45. [45]F. Faghri, P. K. A. Vasu, C. Koc, V. Shankar, A. Toshev, O. Tuzel, and H. Pouransari, “MobileCLIP2: Improving multi-modal reinforced training,” Transactions on Machine Learning Research, 2025.arxiv.org/abs/2508.20691
  46. [46]H. Han, J. Y.-L. Forrest, J. Wang, S. Yuan, F. Han, and D. Li, “Explainable machine learning for high frequency trading dynamics discovery,” Information Sciences, vol. 684, p. 121286, 2024.
  47. [47]D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations (ICLR), 2015.arxiv.org/abs/1412.6980
  48. [48]K. Lang, “NewsWeeder: Learning to filter netnews,” in Proceedings of the Twelfth International Conference on Machine Learning, 1995, pp. 331–339.
  49. [49]X. Zhang, J. Zhao, and Y. LeCun, “Character-level convolutional networks for text classification,” in Advances in Neural Information Processing Systems, vol. 28, 2015, pp. 649–657.arxiv.org/abs/1509.01626
  50. [50]P. Bajaj, D. Campos, N. Craswell, L. Deng, J. Gao, X. Liu, R. Majumder, A. McNamara, B. Mitra, T. Nguyen, M. Rosenberg, X. Song, A. Stoica, S. Tiwary, and T. Wang, “MS MARCO: A human generated MAchine reading COmprehension dataset,” arXiv preprint arXiv:1611.09268, 2016.arxiv.org/abs/1611.09268
  51. [51]L. Wang, N. Yang, X. Huang, B. Jiao, L. Yang, D. Jiang, R. Majumder, and F. Wei, “Text embeddings by weakly-supervised contrastive pre-training,” arXiv preprint arXiv:2212.03533, 2022.arxiv.org/abs/2212.03533
  52. [52]Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “RoBERTa: A robustly optimized BERT pretraining approach,” arXiv preprint arXiv:1907.11692, 2019.arxiv.org/abs/1907.11692
  53. [53]J. Pennington, R. Socher, and C. D. Manning, “GloVe: Global vectors for word representation,” in Conference on Empirical Methods in Natural Language Processing, 2014, pp. 1532–1543.DOI
  54. [54]H. Han, L. Niu, D. Li, and J. Wang, “Differential rank preservation manifold learning,” 2026, under review at Pattern Recognition. [Online]. Available: http://arxiv.org/abs/2610.41107
  55. [55]K. R. Moon et al., “Visualizing structure and transitions in high-dimensional biological data,” Nature Biotechnology, vol. 37, pp. 1482–1492, 2019.DOI
  56. [56]J. B. Tenenbaum, V. de Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” Science, vol. 290, pp. 2319–2323, 2000.pubmed.ncbi.nlm.nih.gov/11125149
  57. [57]X. Han, “Nonnegative principal component analysis for cancer molecular pattern discovery,” IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 7, no. 3, pp. 537–549, 2010.DOI
  58. [58]L. Hubert and P. Arabie, “Comparing partitions,” Journal of Classification, vol. 2, no. 1, pp. 193–218, 1985.DOI
  59. [59]M. Douze, A. Guzhva, C. Deng, J. Johnson, G. Szilvasy, P.-E. Mazare, M. Lomeli, L. Hosseini, and J. Jégou, “The Faiss library,” arXiv:2401.08281, 2024.arxiv.org/abs/2401.08281
  60. [60]W. F. Sheppard, “On the application of the theory of error to cases of normal distribution and normal correlation,” Philosophical Transactions of the Royal Society A, vol. 192, pp. 101–167, 1899.
  61. [61]W. H. Kruskal, “Ordinal measures of association,” Journal of the American Statistical Association, vol. 53, no. 284, pp. 814–861, 1958.DOI

Paper details

vlnc:W_ev354n2y
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents