WorksComputer Science
Item Representations Matter: Bridging Hard Semantic IDs and Soft Prototype Mixtures for Generative Retrieval
Abstract
Modern recommender systems rely on generative retrieval techniques to predict the next item either as discrete codes or as a continuous vector. On the one hand, Semantic IDs (SIDs) give discrete models shared parameters and pretrained semantic structure, but discretizing items into hard codes discards some information from the original content embeddings. On the other hand, continuous models avoid quantization, yet typically use flat item lookup tables with no explicit parameter tying across items. We ask whether a continuous recommender can introduce shared structure without making it a discrete bottleneck. We propose CAP-Mix, a flow recommender that represents each item as a soft mixture of shared prototypes plus an item-specific vector. The assignments, prototypes, and item vectors are trained jointly, and the soft representation is retained at inference. Pretrained information is optional: SID responsibilities can serve as a prior over assignments, while a full content embedding can initialize the item-specific vector without changing the initial flat table. In matched comparisons on three datasets, CAP-Mix improves over a flat table on the two sparse Amazon datasets and yields a small gain on Steam when trained without pretrained information. Given only an offline SID, it outperforms a hard table built from the same tokenizer by 4.45-14.90% in HR@5. Given the full content embedding, it matches or improves a strong flat table while hard assignments regress on the sparse Amazon datasets. Overall, averaged across six benchmarks, CAP-Mix improves HR@5 and NDCG@5 by 6.9% and 13.6% relative to the best evaluated baseline for each dataset and metric, while delivering 1.4-161x the inference throughput of the discrete baselines in our experiments.
Introduction
Generative retrieval techniques enable modern recommender systems to perform large-scale next-item retrieval by learning the next-item distribution and generating recommendations from it, using either discrete tokens (Rajput et al., [25]) or continuous item embeddings (Li et al., [17]). Either way, a recommender needs an item representation table (or lookup table) as input, i.e., the map from a catalog to the representations it generates or scores against, and how that map is built is a lever in its own right, open to external information. Traditionally, it is flat: one independently parameterized vector per item, with no explicit parameter tying between item rows. Recently, discrete methods have replaced that default, whereas continuous methods have retained it.
Discrete models tokenize each item embedding into codes and emit those codes autoregressively ([7, 25]) or by masked diffusion ([26]). They owe much of their strength to those codes being Semantic IDs (SIDs) (Singh et al., 2024). A residual quantizer ([16] (van den Oord et al. 2017; Tong et al. 2026)) turns each item’s pretrained content embedding into a tuple of codes, so the item vocabulary becomes a clustering ([13]). Two things follow: items sharing a code share the parameters attached to it, so their gradients are aggregated through the shared codeword, introducing explicit cross-item coupling and thereby pooling training signal across items; and because the codes are quantized from a content embedding, that grouping imports semantic structure. However, tokenization has two costs: committing each item to a single codeword at each level generally introduces reconstruction error, and distinct items may also collide on the same code tuple. A recent line of work makes the tokenizer itself adaptive by regularizing it (Wang et al., 2024), coupling it to the recommender [21], or relaxing the assignment so gradients pass through it [17, 8], but recommendation is still organized around emitting discrete codes, and both costs remain: they stem from tokenization itself, not from when the tokenizer is fit.
Continuous models instead represent each item as a vector in and retrieve those nearest to the one they predict [22, 17] (Wang et al. 2023). They do not tokenize and are free to seed the content embedding straight into their item table, so semantic structure reaches them without any codes. What they do not obtain automatically is explicit cross-item parameter sharing: every item row remains independently parameterized. This raises our central question: can a continuous generative recommender obtain adaptive cross-item parameter sharing without converting items into discrete tokens?
A simple way to use an SID in a continuous recommender is to replace its flat lookup table with a hard SID table whose assignments are frozen. If the tokenizer assigns item the code at level , then its embedding is , where contains the codeword vectors and the item-specific vector can be initialized from the original content embedding. This preserves continuous retrieval while transferring the SID’s parameter sharing: items with the same code update a shared vector. However, our diagnostic comparison (Appendix § 12, Table 6) shows that, without content embeddings, semantic assignments outperform matched random assignments, indicating that the imported structure is useful. When the full content embedding initializes , by contrast, frozen hard semantic assignments hurt, whereas random assignments remain near the flat baseline. These observations suggest that sharing can help, but that a frozen semantic partition can become restrictive when its underlying continuous representation is already available. This motivates shared structure that adapts jointly with the recommender while retaining item-specific degrees of freedom.
| scenario | item table | Beauty | Industrial | Steam |
| FROM SCRATCH | FLAT | 5.77 | 2.75 | 8.29 |
| FROM SCRATCH | + HARD RQ-SID | +0.52% | +9.82% | +0.36% |
| FROM SCRATCH | + RANDOM SID | −1.73% | +0.73% | +0.12% |
| PCA-TEXT | FLAT, TRAINABLE | 6.69 | 3.81 | 8.13 |
| PCA-TEXT | FROZEN | −18.24% | −20.21% | −14.15% |
| PCA-TEXT | + HARD RQ-SID | −3.89% | −7.61% | +0.74% |
| PCA-TEXT | + RANDOM SID | +0.30% | −1.84% | +0.86% |
Table 6. Item-table variants for a continuous flow recommender. The flat table of each block is absolute test HR@5 ↑; every other cell is its % change relative to it. A hard SID beats random codes from scratch, but with the full content embedding the ordering reverses and, on the sparse catalogs, both fall below the trainable flat table.
Following these observations, we propose CAP-Mix, a flow recommender whose item table is a co-adaptive mixture of shared prototypes plus an item-specific vector. Rather than selecting one codeword per level, an item softly mixes over all of them, so the grouping is differentiable and the ranking objective moves it continuously instead of by discrete reassignment, while the item-specific vector keeps it from being reduced to the group it lands in. We call this a prototype-mixture item table:
with the shared prototypes of level , the weights item places on them, the item’s own vector, and a scale factor, all learned with the flow in a single stage.
Empirically, we evaluate CAP-Mix across six public benchmarks (Table 2), where CAP-Mix improves average HR@5 ↑ and NDCG@5 ↑ by +6.9% and +13.6%, respectively, relative to the strongest evaluated baseline (FlowRec, TIGER, or MaskGR) for each dataset and metric, while delivering 1.4–161× the inference throughput ↑ of the evaluated discrete baselines (Table 3).
| Beauty HR@5 ↑ | Beauty NDCG@5 ↑ | Indus. HR@5 ↑ | Indus. NDCG@5 ↑ | Steam HR@5 ↑ | Steam NDCG@5 ↑ | ML-1M HR@5 ↑ | ML-1M NDCG@5 ↑ | Yelp HR@5 ↑ | Yelp NDCG@5 ↑ | Office HR@5 ↑ | Office NDCG@5 ↑ | |
| FlowRec | 6.27p ±0.18 | 4.03 ±0.03 | 3.62p ±0.02 | 2.04 ±0.03 | 8.28s ±0.33 | 5.37 ±0.30 | 15.36p ±0.20 | 10.14 ±0.12 | 2.98p ±0.05 | 1.86 ±0.03 | 2.90p ±0.02 | 1.67 ±0.02 |
| TIGER | 3.51s ±0.07 | 2.54 ±0.05 | 2.05p ±0.03 | 1.34 ±0.02 | 8.43i ±0.10 | 5.01 ±0.03 | 12.59s ±0.40 | 8.47 ±0.28 | 1.29p ±0.07 | 0.80 ±0.04 | 1.52p ±0.01 | 0.98 ±0.01 |
| MaskGR | 5.06i ±0.13 | 3.28 ±0.12 | 3.46p ±0.03 | 2.24 ±0.03 | 8.64p ±0.00 | 5.57 ±0.01 | 15.64s ±0.40 | 10.23 ±0.28 | 2.99s ±0.04 | 1.87 ±0.03 | 2.29s ±0.02 | 1.53 ±0.01 |
| CAP-Mix | 6.79p ±0.06 | 4.73 ±0.04 | 3.79p ±0.01 | 2.34 ±0.03 | 8.69i ±0.29 | 5.57 ±0.19 | 17.25p ±0.13 | 11.71 ±0.11 | 3.18p ±0.02 | 2.03 ±0.02 | 3.22p ±0.01 | 2.28 ±0.01 |
| Δ (%) | +8.3% | +17.4% | +4.7% | +4.5% | +0.6% | 0.0% | +10.3% | +14.5% | +6.4% | +8.6% | +11.0% | +36.5% |
Table 2. Table 2 Main results, HR@5 ↑ and NDCG@5 ↑ (mean±std, ×100). For each model–dataset pair, both metrics come from one run selected by their validation geometric mean; s marks scratch, i offline SID, and p PCA-text. Bold/underline mark best/second; Δ compares CAP-Mix with the best baseline.
| baseline | hot-parameter ratio | MFLOP/query ratio | throughput ratio |
| TIGER | 0.9–10.0× | 23–265× | 1.9–7.1× |
| MaskGR | 1.4–5.4× | 3.7–2,654× | 1.4–161× |
Table 3. Inference efficiency across five datasets (min–max from unrounded values). Hot-parameter ratio and MFLOP/query ratio are baseline/CAP-Mix; throughput ratio is CAP-Mix/baseline.
Preliminaries
Flow-based generative retrieval for recommender systems
Flow matching [20] learns to transform samples from a simple source distribution into samples from a target distribution . It learns a velocity field that tells each sample how to move at time . Let be the item catalog, a user’s interaction history, and the next item. A flow-based generative recommender can be decomposed into three components. First, the item encoder maps each item to an embedding . Applied elementwise to , it yields . Second, the flow starts from a source and follows the learned velocity field toward a predicted next-item embedding by integrating from to . Writing for the exact velocity field, explicit Euler approximates this transport using evaluations of the learned field:
Here, denotes the numerical state at time , with . Depending on the architecture, the encoded history may determine the source, condition the velocity field, or both. Third, the decoder converts this endpoint into a distribution over the catalog by scoring it against the item table . Figure 1(a) illustrates this complete encoder–flow–decoder path. During training, the target endpoint is the next-item embedding . For a source and target , conditional flow matching commonly uses the straight path and regresses the conditional velocity

Figure 1. (a) A general flow-based retrieval pipeline has three components, and the item table is shared across them: the encoder builds it, the flow is trained to land on one of its rows, and the decoder retrieves against it. (b) CAP-Mix mainly changes the encoder of the pipeline: an attention block over the level tokens maps each item’s short feature to a soft assignment over the prototypes of level , which then blends the rows of that level’s codebook . Adding a trainable identity vector yields the item embedding . Nothing is quantized or frozen, so the clustering is shaped by the training objective and the assignments are kept soft during training and inference.
The same item table therefore embeds the history, defines the flow’s training targets, and supports retrieval. Existing flow-based recommenders ([22, 17]) focus primarily on the transport, whereas we study how the item-table parameterization can enable sharing across items.
Semantic IDs
A Semantic ID (SID) represents an item by an ordered tuple of codes, typically obtained by applying residual quantization to a pretrained content embedding ([25, 13]). Given codebooks , residual quantization decomposes the content embedding into selected codewords and a final residual :
The SID retains the selected indices but not . Methods differ in how they obtain these assignments: RQ-KMeans selects the nearest centroid ([13]), RQ-VAE learns its quantization space through reconstruction ([16]), and RQ-GMM assigns items using posterior mixture probabilities (Tong et al., 2026). Regardless of the fitting procedure, the SID used at inference is hard: each level selects one codeword, whose index becomes that level’s code. Items that share a code reuse its parameters, thereby pooling recommendation gradients across their interactions. Deriving the codes from content embeddings also transfers pretrained semantic structure.
Recent methods remove the need for a separately pretrained, frozen tokenizer by adapting it jointly with the recommender ([17, 8] (Wang et al. 2026)), but their deployed IDs remain hard and discard the residual . SIDs therefore share parameters through discrete code assignments at the cost of quantizing the item representation. Flat-table continuous recommenders avoid quantization but learn each item embedding independently. Our goal is to combine adaptive sharing with trainable continuous representations.
Method: CAP-Mix
Following § 2.1, CAP-Mix uses a continuous flow to predict a next-item embedding and a catalog decoder to model . Its main contribution is the item encoder , which replaces a flat item table with a prototype-mixture parameterization. This representation learns soft item groupings for cross-item sharing while retaining an item-specific vector, and can incorporate pretrained SIDs or content embeddings when available. Figure 1 provides an overview.
Prototype-mixture item representations
CAP-Mix maintains shared codebooks , whose rows are -dimensional prototype vectors. For item , is a soft assignment over the prototypes in codebook , and is an item-specific vector. The item embedding is then
where is a convex mixture of the prototypes, controls the contribution of the shared component, and keeps its scale stable as changes.
Cross-item sharing and item-specific memorization. Consider a differentiable objective whose dependence on the codebooks and item-specific vectors is only through the composed item table , and let . Differentiating Equation (5) gives
The prototype channel aggregates recommendation gradients across items by their assignment weights, providing parameter sharing that can improve generalization. Meanwhile, receives the unrestricted gradient , preserving item-specific memorization and uniqueness even when assignments coincide. Appendix § 11 derives the resulting coupling under fixed assignments.
Relation to flat and hard item tables. Equation (5) contains flat and hard item tables as special cases: yields the flat , while fixed one-hot selects one prototype per codebook as in a hard SID lookup. Additionally setting gives the fully quantized case and discards item information not captured by the selected codewords. CAP-Mix instead keeps assignments soft and retains for item-specific flexibility.
Learning soft assignments. Each item has a learned low-dimensional feature . We derive one token per codebook from and process the tokens jointly with a lightweight attention block:
Here, is a codebook-specific affine projection, is its contextualized output, and maps this output to assignment logits. This parameterization has three properties. First, because , serves as an assignment bottleneck: it stores compact information for selecting shared prototypes, while detailed item-specific information remains in . Second, attention lets the assignments interact: it is bidirectional across codebooks by default and, when an offline residual SID provides a level ordering, causally masked to respect that ordering. Third, no arg max or straight-through estimator is used: the assignments remain soft and differentiable during training and inference, and all representation parameters are learned jointly with the recommender.
Using optional pretrained information
Pretrained information is optional: CAP-Mix can learn entirely from recommendation interactions or incorporate an offline SID or a full content embedding when either is available.
From scratch. CAP-Mix requires neither a tokenizer nor content embeddings. All representation parameters are randomly initialized and learned solely from recommendation interactions. The resulting item groupings are therefore induced by the recommendation objective rather than imported from pretrained content information.
Offline Semantic ID. An offline residual tokenizer supplies both a distribution over codes at each depth and the level ordering induced by its residual construction. We follow this ordering with parallel causal chains of depth , sharing the tokenizer prior at each depth across chains:
Here, the sum ranges over and , and is a learned trust parameter. Because gives the prior full support, for any finite the learned logits can realize any distribution in the interior of the simplex. The assignments therefore remain soft and differentiable. Equation (5) is recovered when . Appendix § 10 gives the pooling derivation and causal attention structure.
Full content embedding. Let be the PCA-projected and coordinatewise-standardized content embedding. We initialize the item-specific vector from this embedding and set every prototype to zero:
Both and remain trainable after initialization. By Equation (5), initially, so CAP-Mix and the flat content-initialized baseline begin from the same unquantized item embeddings. The prototype channel then learns cross-item sharing from this common starting point.
Flow backbone and decoder
We build on the FlowRec backbone [17], retaining its behavior encoder, velocity field, and base flow objective while replacing its item encoder and decoder. Appendix § 8 provides the full specification. A behavior encoder maps the most recent item embeddings to a history-dependent source, while the next-item embedding supplies the target:
We train on the straight interpolant using the conditional flow-matching loss in Equation (3). Following FlowRec, we also form the one-step endpoint estimate
Decoder. For a query , the decoder combines dot-product scores against the shared item table with a history-dependent co-visitation prior through log-linear pooling:
The prior favors items that frequently co-occur with the items in the user’s recent history. We compute it once from the training interactions before model training and keep it fixed. The trust controls its influence. Setting defines the no-prior ablation with a pure dot-product decoder. Appendix § 10 gives the complete construction.
Training objectives and inference
Base flow objective. Using the decoder in Equation (12), we apply next-item classification losses to the behavior source and the one-step endpoint:
Together with the conditional flow-matching loss, the inherited FlowRec objective is
Here, makes the history-dependent source predictive of the next item, while applies the same supervision to the one-step endpoint estimate. The weights and are fixed throughout our experiments; their values are given in § 4.2.
Code matching. The item-level losses can be satisfied through the item-specific vectors alone, leaving the shared assignments uninformative for retrieval. We therefore train the predicted endpoint to recover the target item’s soft assignments, encouraging items retrieved in similar contexts to use compatible prototypes. Let collect all item assignments for chain at depth , and let . Here and below, ranges over all chains and depths, with outside the residual-SID setting. We define
Here, marginalizes the fixed item-space prior into code space. The stop-gradient operator prevents this computation from updating the assignments. The code-matching loss is
Gradients still pass through the target assignment and, through , to the prototypes and flow model, allowing these components to co-adapt.
Maintaining prototype utilization. Because the item-specific channel can support retrieval even when assignments concentrate on only a few prototypes, we penalize catalog-wide utilization below a target :
where is entropy and . Since is the effective number of utilized prototypes, the loss vanishes once it reaches . The loss depends only on the catalog-wide marginal , leaving individual assignments free to be soft or sharp without forcing discretization.
Full objective. All trainable components are optimized jointly. Given weights ,
Inference. After training, we materialize the item table from Equation (5), retaining the learned soft mixtures without discretization. Given a history , we look up its item embeddings and compute the behavior source . Starting from , we numerically integrate the learned velocity field as in Equation (2) to obtain . The decoder in Equation (12) then scores the catalog, and we return the top- items. Because the table is materialized once, the prototype-mixture encoder adds no per-query computation relative to a flat item table.
Experiments
Our experiments address three questions: (RQ1) when does a prototype-mixture item table improve over flat and hard SID alternatives under different levels of pretrained information; (RQ2) how does the complete CAP-Mix system compare with continuous and discrete generative recommenders in predictive performance and efficiency; and (RQ3) how do semantic initialization, the co-visitation prior, and target-item frequency affect the observed gains?
Setup
Setup
Datasets and preprocessing. We evaluate on six public benchmarks: Amazon Beauty (2014) ([24, 11]), Amazon Industrial & Scientific and Amazon Office (2023) ([12]), MovieLens-1M ([10]), Yelp-2022 (Yelp Inc., 2022), and Steam-2018 ([14]). All datasets are processed identically following prior work ([9, 25, 26]): iterative 5-core filtering, consecutive-repeat removal, chronological ordering, and leave-one-out splitting, with each user’s penultimate and final interactions used for validation and testing. Appendix § 7 reports dataset statistics, sources, metadata, and complete preprocessing details.
Evaluation protocol. All models are trained and evaluated on identical item sequences using full-catalog ranking without candidate sampling. We report HR@5 ↑ and NDCG@5 ↑ as three-seed means scaled by 100; Appendix § 14 additionally reports . We use early stopping, select the checkpoint with the best validation performance, and evaluate only that checkpoint on the test split.
Information regimes. To separate content information from parameter-sharing benefits, we evaluate three regimes that differ in the pretrained information available. From scratch provides none: item representations are initialized randomly and learned from interactions. Offline SID provides the stored codes and posterior responsibilities of a balanced RQ-GMM (see Appendix § 9), but not the content embeddings from which they were computed. PCA-text provides the full content embeddings, projected through PCA to dimensions and kept trainable.
Implementation and model selection. All controlled item-table variants use the same flow backbone, three-item histories, three Euler steps, and matched non-table hyperparameters. We encode item metadata once with frozen Qwen3-Embedding-4B (Zhang et al., 2025). In PCA-text, we initialize the trainable item-specific vectors with standardized -dimensional PCA projections of the content embeddings. CAP-Mix zero-initializes its prototypes, making its initial table identical to the flat baseline. We select configurations by the three-seed validation geometric mean of HR@5 ↑ and NDCG@5 ↑. Appendices § 7–§ 9 give the SID construction and remaining setup details.
RQ1: When do prototype mixtures help?
Our central hypothesis is that continuous item embeddings benefit from shared prototypes when that structure complements, rather than replaces, an item-specific continuous representation. We test this by holding the flow backbone and training protocol fixed while varying the item-table parameterization. Because the same table embeds histories, defines the flow targets, and supports retrieval, ranking differences measure the downstream utility of the learned item embeddings.
Item-table variants. The flat table uses . The hard SID table uses , with trainable codewords and item-specific vectors but frozen one-hot assignments. CAP-Mix instead uses the prototype-mixture representation of Equation (5). All variants share the same backbone and non-table hyperparameters.
Information available to each variant. Table 1 compares the item-table variants under three settings. In from scratch, the flat and CAP-Mix tables are randomly initialized and learn only from interactions. In offline SID, the hard table uses the tokenizer’s stored codes as fixed assignments, whereas CAP-Mix uses its posterior responsibilities as the assignment prior in Equation (8); neither receives the original content embedding. In PCA-text, every variant initializes its item-specific vectors from the same PCA projection. The hard table uses the tokenizer’s fixed codes, whereas CAP-Mix initializes its prototypes to zero, making its initial table identical to the flat table.
| dataset | from scratch: FLAT | from scratch: CAP-Mix | from scratch: vs. FLAT | offline SID: HARD | offline SID: CAP-Mix | offline SID: vs. HARD | PCA-text: FLAT | PCA-text: HARD | PCA-text: CAP-Mix | PCA-text: vs. FLAT |
| Beauty | 5.77<br>ref. | 5.89<br>+2.08% | +2.08% | 5.80<br>+0.52% | 6.32<br>+9.53% | +8.97% | 6.69<br>+15.94% | 6.43<br>+11.44% | 6.79<br>+17.68% | +1.49% |
| Industrial | 2.75<br>ref. | 2.88<br>+4.73% | +4.73% | 3.02<br>+9.82% | 3.47<br>+26.18% | +14.90% | 3.81<br>+38.55% | 3.52<br>+28.00% | 3.79<br>+37.82% | −0.52% |
| Steam | 8.29<br>ref. | 8.35<br>+0.72% | +0.72% | 8.32<br>+0.36% | 8.69<br>+4.83% | +4.45% | 8.13<br>−1.93% | 8.19<br>−1.21% | 8.30<br>+0.12% | +2.09% |
Table 1. Table 1 Item tables across three information regimes. Each cell reports test HR@5 ↑ (top) and change from the dataset’s scratch FLAT anchor (bottom; ref.). The final Δ compares CAP-Mix with FLAT (scratch), HARD (offline SID), or FLAT (PCA-text). HARD uses the same tokenizer; all variants have matched backbones and non-table hyperparameters.
Prototype sharing helps without pretrained content. In from scratch, CAP-Mix improves HR@5 ↑ over the flat table by +2.08% on Beauty, +4.73% on Industrial, and +0.72% on Steam. Thus, shared prototypes learn useful cross-item structure from interactions alone, producing item embeddings that are more effective for next-item ranking than independently parameterized rows.
An offline SID is more useful as a prior than as a fixed assignment. The hard SID table exceeds the scratch anchor on all three datasets, confirming that the tokenizer carries useful information. Given the same tokenizer,
CAP-Mix improves HR@5 ↑ over the hard table by 4.45%–14.90%, showing that differentiable assignments use this information more effectively than frozen one-hot assignments.
Prototype sharing complements the full content embedding. In PCA-TEXT, content seeding already strengthens the flat table on the Amazon datasets. Starting from the same embeddings, CAP-Mix improves it by +1.49% on Beauty and +2.09% on Steam while essentially preserving its Industrial result (−0.52%). In contrast, the hard SID table changes HR@5 ↑ by −3.89% and −7.61% relative to the seeded flat table on the Amazon datasets. Thus, prototype mixtures add cross-item sharing while retaining the unquantized initialization improvement.
Across all three regimes, prototype mixtures improve how continuous item embeddings use the information available by introducing cross-item pooling without quantizing their representations.
RQ2: Comparison with generative recommenders
We compare CAP-Mix with continuous FlowRec (Li et al., [17]), autoregressive TIGER (Rajput et al., [25]), and masked-diffusion MaskGR (Shah et al., [26]) in all three information regimes, selecting one configuration per model and dataset by the validation geometric mean of HR@5 ↑ and NDCG@5 ↑. CAP-Mix and FlowRec use three-item histories; TIGER and MaskGR retain their released 120-token limit and decode one top-20 list with beam search. Appendix § 8 specifies the tokenizers, initializations, resulting item-level history lengths, and decoding protocol.
Table 2 reports the complete-system comparison at , with results at and in Appendix § 14. CAP-Mix achieves the highest HR@5 ↑ on all six datasets and the highest or tied-for-highest NDCG@5 ↑ on all six. Averaged across datasets, it improves over the strongest evaluated baseline for each dataset and metric by +6.9% in HR@5 ↑ and +13.6% in NDCG@5 ↑. For CAP-Mix, validation selects PCA-TEXT on five of the six datasets and OFFLINE SID on Steam, suggesting that direct content initialization is generally preferable when the original embeddings are available.
Inference efficiency. At inference, CAP-Mix materializes the composed item embeddings once, so its soft assignments add no per-query computation. Table 3 shows that CAP-Mix consistently requires fewer MFLOP/query and achieves higher throughput than both discrete baselines across the five profiled datasets. The advantage is largest when the baselines generate multi-code SIDs and narrows for atomic-token configurations. The parameter advantage depends on catalog size: on Yelp, the continuous item table makes CAP-Mix slightly larger than TIGER. Appendix § 8 documents the measurement protocol, and Table 5 reports the full per-dataset measurements.
| ML-1M | Steam | Beauty | Indus. | Yelp | |
| 3.4k | 4.1k | 12k | 25k | 111k | |
| Hot params (M) ↓: table + backbone | |||||
| CAP-Mix | 0.4+0.9 | 0.5+0.9 | 1.5+0.9 | 3.3+0.9 | 14.2+0.9 |
| TIGER | 0.9+12.9 | 0.3+12.9 | 3.1+12.9 | 0.3+12.9 | 0.5+12.9 |
| MaskGR | 0.4+6.8 | 0.3+6.8 | 0.2+6.8 | 0.3+6.8 | 14.2+6.8 |
| MFLOP / query ↓ | |||||
| CAP-Mix | 6.4 | 6.6 | 8.6 | 12.2 | 34.0 |
| TIGER | 195 | 1,742 | 199 | 1,742 | 1,747 |
| MaskGR | 43.6 | 17,435 | 17,037 | 10,399 | 126 |
| Queries/s ↑ | |||||
| CAP-Mix | 26,934 | 28,752 | 29,240 | 27,864 | 7,688 |
| TIGER | 5,568 | 4,054 | 7,242 | 4,121 | 4,028 |
| MaskGR | 5,634 | 182 | 182 | 242 | 5,656 |
| Train GPU-h ↓ | |||||
| CAP-Mix | 1.46 | 0.21 | 0.06 | 0.08 | 1.99 |
| TIGER | 2.01 | 0.20 | 0.53 | 1.08 | 2.21 |
| MaskGR | 2.06 | 10 | 17 | 47 | 7.59 |
| Train examples/s ↑ | |||||
| CAP-Mix | 40,623 | 53,331 | 37,149 | 31,406 | 17,385 |
| TIGER | 1,616 | 2,796 | 4,541 | 4,551 | 1,311 |
| MaskGR | 5,525 | 5,547 | 1,344 | 5,552 | 5,995 |
Table 5. Inference and training cost at matched input and output: top-20 lists from 3 history items, with TIGER and MaskGR using beam search with width 20. Hot params are those a served model must hold, split as table + backbone (every backbone is constant across datasets). Subscripts give each baseline relative to CAP-Mix and are computed from unrounded measurements. Throughput is measured on one H200 with a batch size of 128 in FP32. Train GPU-h is the estimated cost of training the reported model, computed by dividing the optimizer-step count at the selected checkpoint by a separately measured training rate. the number of tokens per item. Hot params are the parameters a served model must hold, determined by which modules actually fire during an inference batch rather than by the checkpoint size: CAP-Mix materializes its item table once into and discards the codebooks, assignment head, and item features, so it serves at the size of a flat table, whereas a generative decoder has nothing to fold away. MFLOP/query is counted with PyTorch’s flop formulas. Queries/s is wall-clock throughput on one H200 with a batch size of 128 in FP32, measured the same way for every model. We run three independent processes per cell; each contributes one steady-state estimate after warm-up, and we report the median of the three. Within a process we time the decode alone, excluding import and checkpoint load, over 100 batches for the baselines and a 12 s window for CAP-Mix.
RQ3: Robustness and diagnostics
Motivating controls. Appendix § 12 compares hard semantic assignments with random assignments matched in depth, codebook size, and codeword vectors. Without content initialization, semantic assignments win on all three datasets, confirming that they carry useful structure beyond parameter sharing. With PCA initialization, they underperform the flat and random controls on both Amazon datasets, showing that this structure can become restrictive when frozen.
Complementarity with the retrieval prior. Without the co-visitation prior, prototype sharing is beneficial in all regimes on both Amazon datasets and retains a +6.91% HR@5 advantage over hard SID on Steam (Table 7), confirming that prototype sharing and the prior are complementary.
| dataset | from scratch FLAT | from scratch CAP-Mix | from scratch vs. | offline SID HARD | offline SID CAP-Mix | offline SID vs. | PCA-text FLAT | PCA-text HARD | PCA-text CAP-Mix | PCA-text vs. |
| Beauty | 5.61<br>ref. | 5.67<br>+1.07% | +1.07% | 5.76<br>+2.67% | 5.89<br>+4.99% | +2.26% | 6.27<br>+11.76% | 6.27<br>+11.76% | 6.50<br>+15.86% | +3.67% |
| Industrial | 2.64<br>ref. | 2.81<br>+6.44% | +6.44% | 2.85<br>+7.95% | 3.21<br>+21.59% | +12.63% | 3.62<br>+37.12% | 3.51<br>+32.95% | 3.68<br>+39.39% | +1.66% |
| Steam | 8.28<br>ref. | 8.26<br>−0.24% | −0.24% | 7.96<br>−3.86% | 8.51<br>+2.78% | +6.91% | 8.10<br>−2.17% | 8.21<br>−0.85% | 8.14<br>−1.69% | +0.49% |
Table 7. Controlled results without the co-visitation prior. All values are test HR@5 ↑ (×100). Each variant’s cell gives its absolute score over its relative change from the from-scratch flat table. Each vs. column compares CAP-Mix with that regime’s baseline.
How do gains vary with item frequency? In the FROM SCRATCH regime, the frequency-stratified results are mixed. However, when pretrained tokenizer or content information is available, relative gains are larger for rare targets than for frequent ones in 11 of 12 dataset–comparison–metric combinations for HR@5 and NDCG@10 , consistent with transfer to less-observed items (Table 8).
| scenario | baseline | dataset and metric | rare | medium | frequent | all | larger end |
| FROM SCRATCH | FLAT | Beauty HR@5 ↑ | +0.2% | −1.8% | +4.1% | +2.1% | frequent |
| Beauty NDCG@10 ↑ | −3.3% | −1.5% | +2.9% | +0.9% | frequent | ||
| Industrial HR@5 ↑ | +5.9% | +4.5% | +4.4% | +4.5% | rare | ||
| Industrial NDCG@10 ↑ | +4.6% | +1.8% | +5.1% | +4.5% | frequent | ||
| OFFLINE SID | HARD | Beauty HR@5 ↑ | +8.5% | +8.8% | +8.9% | +8.8% | frequent |
| Beauty NDCG@10 ↑ | +10.9% | +6.6% | +4.8% | +6.1% | rare | ||
| Industrial HR@5 ↑ | +49.7% | +26.6% | +10.9% | +14.7% | rare | ||
| Industrial NDCG@10 ↑ | +56.0% | +32.3% | +17.1% | +20.8% | rare | ||
| PCA-TEXT | FLAT | Beauty HR@5 ↑ | +5.2% | +3.6% | −0.3% | +1.4% | rare |
| Beauty NDCG@10 ↑ | +4.2% | +3.1% | −0.4% | +1.1% | rare | ||
| Industrial HR@5 ↑ | +7.1% | +2.6% | −1.4% | −0.3% | rare | ||
| Industrial NDCG@10 ↑ | +9.0% | +0.8% | −2.5% | −1.4% | rare | ||
| PCA-TEXT | HARD | Beauty HR@5 ↑ | +10.7% | +3.0% | +5.4% | +5.5% | rare |
| Beauty NDCG@10 ↑ | +5.3% | +3.2% | +5.1% | +4.6% | rare | ||
| Industrial HR@5 ↑ | +15.3% | +8.7% | +6.9% | +7.6% | rare | ||
| Industrial NDCG@10 ↑ | +18.8% | +9.7% | +8.3% | +9.0% | rare |
Table 8. Relative gains by target-item training frequency. Test users are grouped by their target item’s frequency in the training set. Entries are the relative improvement of CAP-Mix over the indicated baseline. Relative gains use unrounded scores and may differ slightly from calculations using displayed values.
Related Work
Discrete generative recommenders and semantic IDs. The discrete branch commonly replaces an atomic item identifier with a semantic ID (SID) by quantizing a pretrained content embedding ([13]). This is generally done through residual schemes that cluster either directly in the embedding space (RQ-KMeans and RQ-GMM ([7] (Tong et al. 2026))) or in a learned latent space (RQ-VAE ([16] (van den Oord et al. 2017))). A recommender then predicts the resulting tuple autoregressively, as in TIGER ([25]), OneRec ([7] (Zhou et al. 2025)), and MiniOneRec ([15]), or through masked discrete diffusion, as in MaskGR ([26]). SIDs provide two distinct benefits. The first, parameter sharing, arises because reusing codewords ties parameters across items and pools supervision from their interactions. The second, semantic transfer, comes from deriving the assignments from pretrained content embeddings, which places semantically related items under shared parameters. Together, these benefits improve generalization ([13] (Singh et al. 2024)). However, an SID exposes only hard code assignments and preserves a partition of the content space rather than its full continuous geometry.
Diffusion and flow recommenders. Continuous generative recommenders replace discrete token prediction with generation in a continuous space. DiffRec (Wang et al. (2023)) denoises a user’s interaction vector, whereas DreamRec (Yang et al. (2023)) and DiffuRec ([19]) diffuse a target item embedding for next-item prediction. FMRec ([22]) and FlowRec ([17]) use flow matching ([20]) to transport a source distribution to the target embedding, with FlowRec conditioning the source on the user’s history. These methods improve the generative dynamics while retaining a flat item table, thereby avoiding discretization into code sequences but forgoing the cross-item parameter sharing induced by shared SID codewords. CAP-Mix retains continuous generation while introducing that sharing through co-adaptive mixtures of prototypes.
Conclusion
We presented CAP-Mix, a flow-based recommender whose item table combines differentiable assignments over shared codebooks with an item-specific vector, all trained jointly with the recommendation objective. The shared codewords introduce explicit cross-item parameter coupling, while the item-specific vectors retain unrestricted per-item degrees of freedom. Across six public benchmarks, CAP-Mix improves average and relative to the strongest evaluated baselines, while providing higher throughput than the evaluated discrete baselines.
Dataset details
Sources. Most datasets are taken as RecBole atomic files (Zhao et al., 2021) from the RecSysDatasets1 distribution. Amazon’s Industrial & Scientific and Office are taken directly from the raw Amazon Reviews 2023 website2. Regardless of the source, all datasets are reduced to (user, item, timestamp) triples and then processed through the same code path.
| Dataset | #Users | #Items | #Interactions | Avg. length | Median inter./item | Sparsity (%) | Eff. rank | Var. kept (%) |
| Amazon Beauty | 22,363 | 12,101 | 198,502 | 8.9 | 9 | 99.927 | 219 | 73.2 |
| Amazon Industrial & Scientific | 51,322 | 25,923 | 417,422 | 8.1 | 9 | 99.969 | 240 | 73.5 |
| Amazon Office | 223,996 | 77,655 | 1,810,031 | 8.1 | 11 | 99.990 | 213 | 75.0 |
| MovieLens-1M | 6,040 | 3,416 | 999,611 | 165.5 | 146 | 95.155 | 326 | 66.0 |
| Yelp-2022 | 272,547 | 111,260 | 4,185,885 | 15.4 | 14 | 99.986 | 279 | 71.3 |
| Steam | 25,256 | 4,087 | 325,320 | 12.9 | 18 | 99.685 | 304 | 67.6 |
Table 4. Datasets after iterative 5-core filtering. Avg. length is interactions per user, Median inter./item is the median number of interactions a catalog item receives, and sparsity is . Eff. rank is over the normalized eigenvalue spectrum of the content embedding, out of 2560, and Var. kept is the share of that variance surviving the projection to .
Preprocessing. Every dataset follows the same procedure. First, we apply an iterative -core filter (). Removing a low-count user can push an item below the threshold and vice versa, so we alternately drop users with fewer than interactions and items with fewer than interactions until a pass removes nothing. Second, each user’s interactions are sorted chronologically by timestamp. Third, we collapse consecutive repeats of the same item and then re-run the -core filter, iterating both to a fixed point. Iterating guarantees every surviving user has at least distinct items and therefore appears in all three splits. In our experiments, datasets require between 1 and 4 collapse-and-re-core rounds. We then assign dense user and item IDs. Finally, each user’s sequence is split using a leave-one-out scheme: the test instance is the full sequence with target , the validation instance is with target , and training uses . All model families consume identical item sequences.
Text embedding. Every dataset has a deterministic per-item text template, filled from the item metadata and left empty where a field is missing: Amazon uses Title; Brand; Categories; Price, following TIGER [25]; MovieLens uses Title; Release year; Genres plus the most frequent community tags; Yelp uses Name; City, State; Categories; and Steam uses Title; Developer; Publisher; Genres; Tags. Each string is encoded once with a frozen Qwen3-Embedding-4B (Zhang et al., 2025) using last-token pooling, giving .
SID pretraining. The offline SIDs are produced by the balanced RQ-GMM of Appendix § 9. We fit 4 levels with , plus a fifth de-duplication digit that makes identifiers unique. The hard RQ-SID baseline consumes the resulting codes directly, while CAP-Mix consumes the per-level posterior responsibilities as the prior in Equation (8).
Evaluation. Evaluation ranks the target against the full catalog, with no candidate sampling and no filtering of previously seen items, so every model scores all items for every test user. We report hit rate and normalized discounted cumulative gain. Hit rate (HR@ ) is the fraction of users whose held-out item appears among the highest-ranked items. Normalized discounted cumulative gain (NDCG@ ) additionally rewards placing that item higher within the top . We report . The two generative baselines decode differently from CAP-Mix but are scored on the same ranked list: TIGER produces its top-20 by autoregressive beam search over code positions, and MaskGR by beam search over the positions it unmasks. Every number is a mean over three seeds, taken from the checkpoint with the best validation performance in each run.
Implementation details
Architecture and optimization. The history encoder is a bidirectional Transformer with rotary positions, RMSNorm, and pre-norm blocks (4 layers, 4 heads, and hidden dimension ), reading the most recent items as one token each. The time code is produced by passing a 128-dimensional sinusoidal embedding through a two-layer MLP with SiLU activations. The velocity field is a second two-layer SiLU MLP with hidden width , applied to the concatenation of the modulated state and . We use neither normalization nor FiLM conditioning:
where . The modulation vector is redrawn for every call during training and replaced by its mean during evaluation. Item features have dimension . The assignment head in Equation (7) uses a separate affine input projection and output projection for each codebook. With , the codebook tokens are processed jointly by a single-layer, single-head bidirectional self-attention block. With a residual SID, each of the chains contains tokens and is processed independently with causal attention following the tokenizer’s level order; the depth- prior is shared across chains. We optimize with AdamW at learning rate and no weight decay, use dropout 0.3 on item embeddings and 0.5 on hidden states, and stop early with patience 50. Inference uses Euler steps.
Hyperparameter search. We fix and , use Optuna’s TPE sampler (Bergstra et al., [3]; Akiba et al., [1]), and select configurations by the three-seed validation geometric mean of HR@5 and NDCG@5 . We independently sample log-uniformly from , log-uniformly from , and log-uniformly from . We search in all settings; by default, whereas is selected jointly with in the offline-SID setting.
PCA-text initialization. Within each run, we fit PCA on the catalog’s 2560-dimensional content embeddings and project them to . We then standardize each projected coordinate across the catalog to zero mean and unit variance, preventing any principal direction from dominating the initialization. For the continuous models, these standardized vectors initialize the trainable item-specific vectors . CAP-Mix initializes every prototype to zero, so Equation (5) gives the same initial item table as the flat PCA-initialized model. For TIGER and MaskGR, we apply the same fitted PCA projection and coordinatewise standardization to the RQ centroids and use the resulting vectors to initialize each level’s codeword embeddings. The de-duplication level has no centroid and remains randomly initialized.
Baseline implementations. We use MaskGR’s official implementation.3 Because its default hyperparameters differ from those reported in its paper, we train both configurations and select between them using the validation criterion from § 4.2. For TIGER, we use the implementation4 provided in GRID (Ju et al., [13]). We supply each with our own SIDs and interaction splits and otherwise keep the published training code untouched. We could not find an official implementation for FlowRec, so we use our own implementation following Li et al. [17] for the flow objective. For fair comparison, we use the same encoder, velocity field, and decoder as CAP-Mix. We fix for CAP-Mix and FlowRec and do not tune it. TIGER and MaskGR use their released implementations’ maximum input length of 120 tokens. Because an item occupies one token per SID level, including the de-duplication level, the corresponding maximum number of history items varies with the validation-selected SID depth.
Generative baselines across the three scenarios. TIGER and MaskGR consume items as tokens, so a scenario is a choice of tokenizer and of how its token table is initialized. OFFLINE SID uses the RQ-GMM tuple. FROM SCRATCH uses an atomic tokenizer, one level assigning item , the token , so the token table is rows learned from interactions alone. PCA-TEXT uses the RQ-GMM tuple with the PCA-initialized codeword rows described above.
Measuring inference cost. All five dataset columns of Table 5 describe the same query: every model is given the most recent items, which for a token-based baseline means an input sequence of length , where is
Measuring training cost. Table 5 also reports training cost. Optimizer steps come from the checkpoint that produced the reported result; the rate is measured separately from the elapsed-time difference between training-only runs stopped at and steps, excluding validation. GPU-hours are estimated by dividing the selected checkpoint’s optimizer-step count by this rate. All cells use one H200 with identical dataloader-worker and CPU allocations, and we report examples/s to account for differences in effective batch size. GPU-hours are implementation-bound: for TIGER and MaskGR, we use the unmodified implementations.
Balanced RQ-GMM
Why the balanced variant. RQ-GMM (Tong et al., 2026) fits each residual level by soft EM [6], which places no constraint on how item mass is distributed over the components, and in some of our experiments a few components absorb most of the catalog. Because an item’s identifier is the -tuple , an unbalanced level shrinks the effective identifier space, so more items collide on the same tuple and the clustering the SID is meant to provide degrades. We therefore replace the soft EM by a hard-assignment classification EM, which imposes the constraint where the decision is actually made. The soft responsibility then plays no part in the fit at all: it survives only as a read-time quantity, the prior in Equation (23).
The quantizer. Let be the item embeddings and . Level models the residual left by the levels above it with a -component diagonal Gaussian mixture whose means form that level’s codebook, and passes on . Writing for the mixing coefficients, the cost of placing a residual in component is its negative log-likelihood,
up to a constant that does not depend on . Each level is fitted by a classification EM (Celeux and Govaert, [5]), which alternates between a hard assignment and a maximum-likelihood update, and it is the assignment step that carries the size constraint:
with and . Without the size bounds, Equation (21) separates per item and returns the maximum a posteriori (MAP) code; with them it is a transportation problem, which we solve exactly as a minimum-cost flow. This extends the constrained -means of Bradley et al. [4], who impose a minimum cluster size and solve the resulting assignment step as a minimum-cost flow. We add the upper bound and use the mixture cost Equation (20) in place of squared Euclidean distance, so the assignment remains an exact minimum-cost flow while the cost stays a likelihood.
The M-step is the maximum-likelihood update of a diagonal mixture under the hard partition ,
with a covariance ridge added to every variance so Equation (20) does not send the cost to infinity on coordinates without spread. Each level is initialized with one short run of unconstrained -means (-means++ seeding (Arthur and Vassilvitskii, [2]), at most 50 Lloyd steps (Lloyd, [23])) and iterated until the assigned cost changes by less than relatively or fewer than 0.1% of items are reassigned, with a cap of 200 iterations. After reaching either stopping criterion or the iteration limit, we fix the resulting assignments, update the residuals, and fit the next level. Balance therefore holds by construction at every level: no code group differs from by more than one, whatever the scale of the embedding. A final de-duplication digit, ranking the items inside each colliding -tuple, makes the full identifier unique.
Use as a prior. The hard RQ-SID baseline consumes the codes directly. CAP-Mix instead consumes the level- responsibility, the posterior over the components given the residual,
where is the tempered version that Equation (8) uses. Since sums over the embedding dimensions, its spread across components grows with , so also scales with . We use at . The residual path in Equation (23) is the greedy one, with . Thus, the mode of is not always the stored code , although they agree for most items. For example, on Beauty, they coincide for 92–94% of items at each level. Feeding the responsibility rather than the code is what lets the tokenizer prior carry its uncertainty.
The two priors
CAP-Mix injects external knowledge in two places: a tokenizer prior on which codes are assigned to an item (Equation (8)) and a co-visitation prior on which item to retrieve (Equation (12)). Both follow the same rule, which this appendix states once, justifies, and then instantiates in both settings.
The rule. Let be a softmax over a finite set with learned logits , and let be a prior belief over that same set. Multiplying the two and renormalizing,
so adding a scaled log-prior to the logits is equivalent to multiplying the two beliefs, with weighting the prior. This is log-linear pooling. The trust interpolates from discarding the prior () to collapsing onto ().
Why not mix probabilities. What separates Equation (24) from the obvious alternative is that any finite leaves the hypothesis class unchanged. That is, provided the prior has full support, Equation (8) can be overridden by evidence: the learned logits can add an arbitrary vector, so every interior point of stays reachable. The persistent prior therefore biases the parameterization and optimization without restricting the reachable distributions. To ensure full support, we add inside the log of Equation (8) and mix the graph prior with the uniform distribution in Equation (25). A prior that is exactly zero on an outcome removes it at any . In contrast, the alternative linear mixing at a fixed restricts , since no matter the learned evidence.
Instance 1: the tokenizer prior on codes. At depth , the learned evidence for chain is . The tokenizer supplies the same prior to every chain at that depth: the responsibility that the fitted quantizer places on each of its Gaussians for item , denoted , (Appendix § 9). This is not a hand-built pseudo-count but an exact posterior under a fitted GMM generative model. It is read along a greedy residual path rather than the balanced assignment the quantizer was fitted with, so its mode need not be the stored code.
We use the tempered responsibility defined in Equation (23) as the prior in Equation (8), with controlling its influence.
Instance 2: the co-visitation prior on items. Here the set is the catalog , the evidence is the inner product against the shared item table, and the prior is with trust , yielding Equation (12). Equation (12) thus makes explicit a prior that a plain dot-product softmax leaves implicit. We build once from the training sequences in three steps:
First, counts how often items and occur in the same training sequence, and row-normalizing turns those counts into conditional frequencies: estimates how likely is to appear given that does, so every row is a distribution over the catalog and is the transition kernel of a random walk on the item graph. Second, we summarize the history by a recency-weighted distribution , which puts mass only on items with which the user interacted, weighted according to recency: the item at age (most recent ) gets weight , repeats accumulate, and the result is -normalized. Applying the kernel times lets that mass flow outward along co-occurrence edges as a -step random walk, so is large when co-occurs strongly with what the user recently touched. Third, a raw can place zero mass on an unseen (context, item) pair, which would veto that item outright, however strong the evidence. Thus, to ensure full support, we temper it toward the uniform distribution. For , this keeps finite; ignores the graph, whereas trusts it fully but can contain zero-probability outcomes.
Only and are tuned; all other settings use fixed defaults that are identical everywhere and never searched: whole-sequence symmetric counts with no self-loops, , (so only direct neighbors receive mass), and the same truncated history the encoder reads, which at gives from newest to oldest. The same log-prior is added to the training logits of both cross-entropies, so the model learns to complement the prior.
Gradient coupling induced by prototype sharing
Let , and stack the item embeddings and item-specific vectors into . For chain and depth , let and . To isolate the direct effect of prototype sharing, hold the assignments fixed and consider a loss component whose dependence on and is only through . Let .
Consider one gradient-descent step with learning rate , and let denote the resulting change in the item table. Equation (26) gives
Because the assignments are fixed, substituting these parameter updates into the composed table gives
where is the identity matrix. Consequently,
which couples item updates in proportion to assignment overlap, while the identity term preserves a direct item-specific update. This analysis isolates the direct representation path with fixed assignments, while the complete model additionally learns the assignments and contains losses that depend explicitly on the prototypes.
Diagnostic results
Table 6 separates the semantic information in the RQ-SID assignments from the parameter sharing induced by the codebook. The random control preserves the table structure, depth, codebook size, and codeword vectors while randomly permuting the item-to-code assignments.
In the FROM SCRATCH regime, the hard RQ-SID table improves HR@5 ↑ over the flat table on all three datasets, whereas matched random assignments yield smaller gains or a loss. This isolates useful semantic structure in the pretrained assignments beyond parameter sharing alone. With PCA-TEXT initialization, freezing the content embeddings causes losses ranging from 14.15% to 20.21%. On the two Amazon datasets, hard semantic assignments also underperform both the trainable flat table and the random-assignment control, suggesting
Full results
that a fixed semantic partition can conflict with the recommendation objective once the continuous content representation is available. This motivates using the tokenizer to initialize differentiable assignments that can be optimized jointly with the recommender. On Steam, PCA initialization itself reduces performance, while hard and random assignments produce similar gains below 1%; we hypothesize that the content embedding is less informative for next-item prediction on this dataset, causing its semantic partition to behave similarly to random sharing.
Removing the co-visitation prior. Table 7 repeats the scenario study with the co-visitation prior disabled for every variant. On Beauty and Industrial, CAP-Mix remains above the corresponding baseline in all three regimes. On Steam, it is essentially flat relative to the baseline in FROM SCRATCH and PCA-TEXT, but retains a +6.91% HR@5 ↑ advantage over the hard table in OFFLINE SID. These results show that the gradient-pooling benefit from our prototype mixtures is complementary to the retrieval prior.
Frequency stratification. We group test users by the training frequency of their target item, without splitting users whose targets have the same frequency. The resulting approximately equal-mass groups contain 22,363 Beauty users (rare: at most 6 interactions; medium: 7–22; frequent: more than 22) and 51,322 Industrial users (rare: at most 7; medium: 8–28; frequent: more than 28). Table 8 reports the relative gain within each group.
Full results
Table 9 summarizes all evaluation cutoffs for the validation-selected configuration of each model and dataset. Table 10–Table 15 then give the complete per-dataset results across all model families, initializations, and metrics.
| Beauty HR@5 | Beauty N@5 | Beauty HR@10 | Beauty N@10 | Beauty HR@20 | Beauty N@20 | Indus. HR@5 | Indus. N@5 | Indus. HR@10 | Indus. N@10 | Indus. HR@20 | Indus. N@20 | |
| FlowRec | ||||||||||||
| TIGER | ||||||||||||
| MaskGR | ||||||||||||
| CAP-Mix | ||||||||||||
| (%) | +8.3% | +17.4% | +2.4% | +12.2% | +0.1% | +9.2% | +4.7% | +4.5% | +4.2% | +5.7% | +4.1% | +7.4% |
| Steam HR@5 | Steam N@5 | Steam HR@10 | Steam N@10 | Steam HR@20 | Steam N@20 | ML-1M HR@5 | ML-1M N@5 | ML-1M HR@10 | ML-1M N@10 | ML-1M HR@20 | ML-1M N@20 | |
| FlowRec | ||||||||||||
| TIGER | ||||||||||||
| MaskGR | ||||||||||||
| CAP-Mix | ||||||||||||
| (%) | +0.6% | 0.0% | −3.4% | −2.3% | −0.7% | −1.0% | +10.3% | +14.5% | +4.7% | +10.3% | +0.1% | +6.8% |
| Yelp HR@5 | Yelp N@5 | Yelp HR@10 | Yelp N@10 | Yelp HR@20 | Yelp N@20 | Office HR@5 | Office N@5 | Office HR@10 | Office N@10 | Office HR@20 | Office N@20 | |
| FlowRec | ||||||||||||
| TIGER | ||||||||||||
| MaskGR | ||||||||||||
| CAP-Mix | ||||||||||||
| (%) | +6.4% | +8.6% | +2.9% | +6.3% | +0.5% | +3.9% | +11.0% | +36.5% | 0.0% | +23.7% | −6.3% | +15.7% |
Table 9. Full cross-dataset results, HR@k ↑ and NDCG@k ↑ (mean±std, ×100). Each model and dataset uses the configuration selected by the validation geometric mean of HR@5 ↑ and NDCG@5 ↑; the same run supplies every cutoff. Superscripts denote scratch, offline SID, and PCA-text. Bold = best, underline = 2nd per column, and compares CAP-Mix with the strongest baseline.
Table 10. Beauty — full results. All families initializations. Bold = best, underline = 2nd per column within an initialization block; CAP-Mix shows vs. the best baseline. FlowRec has no offline SID variant, since a flat table has no codes to consume, so the row marked repeats its from scratch numbers.
| beauty | model | HR@5 | HR@10 | HR@20 | N@5 | N@10 | N@20 |
| scratch | FlowRec | 5.610.02 | 8.010.05 | 10.900.20 | 3.860.02 | 4.630.03 | 5.360.06 |
| TIGER | 3.510.07 | 4.920.09 | 6.900.21 | 2.540.05 | 2.990.05 | 3.490.08 | |
| MaskGR | 4.380.16 | 6.480.19 | 9.330.09 | 2.990.14 | 3.660.14 | 4.380.12 | |
| CAP-Mix | 5.890.04 +4.99% | 8.280.05 +3.37% | 11.300.01 +3.67% | 4.210.01 +9.07% | 4.980.01 +7.56% | 5.750.01 +7.28% | |
| SID | FlowRec | 5.610.02 | 8.010.05 | 10.900.20 | 3.860.02 | 4.630.03 | 5.360.06 |
| TIGER | 3.480.14 | 5.330.15 | 8.000.13 | 2.300.14 | 2.890.14 | 3.560.14 | |
| MaskGR | 5.060.13 | 8.110.15 | 12.100.16 | 3.280.12 | 4.270.12 | 5.290.09 | |
| CAP-Mix | 6.320.10 +12.66% | 8.720.08 +7.52% | 11.960.10) −1.16% | 4.510.05 +16.84% | 5.280.03 +14.04% | 6.100.00 +13.81% | |
| PCA | FlowRec | 6.270.18 | 9.520.22 | 13.520.41 | 4.030.03 | 5.070.05 | 6.080.10 |
| TIGER | 3.390.09 | 5.450.19 | 8.300.18 | 2.240.06 | 2.900.09 | 3.620.08 | |
| MaskGR | 5.050.13 | 7.930.05 | 11.970.13 | 3.260.13 | 4.190.10 | 5.220.06 | |
| CAP-Mix | 6.790.06 +8.29% | 9.750.06 +2.42% | 13.530.17 +0.07% | 4.730.04 +17.37% | 5.690.04 +12.23% | 6.640.07 +9.21% |
| office | model | HR@5 | HR@10 | HR@20 | N@5 | N@10 | N@20 |
| scratch | FlowRec | ||||||
| TIGER | |||||||
| MaskGR | |||||||
| CAP-Mix | |||||||
| SID | FlowRec† | ||||||
| TIGER | |||||||
| MaskGR | |||||||
| CAP-Mix | |||||||
| PCA | FlowRec | ||||||
| TIGER | |||||||
| MaskGR | |||||||
| CAP-Mix |
Table 15. Amazon Office — full results. Conventions as in Table 10.
Table 11. Industrial & Scientific — full results. Conventions as in Table 10.
| industrial | model | HR@5 | HR@10 | HR@20 | N@5 | N@10 | N@20 |
| scratch | FlowRec | 2.640.00 | 4.040.03 | 5.910.09 | 1.490.01 | 1.940.01 | 2.410.02 |
| TIGER | 1.610.05 | 2.600.10 | 4.010.13 | 1.050.03 | 1.370.05 | 1.720.06 | |
| MaskGR | 2.360.02 | 3.630.02 | 5.470.02 | 1.580.01 | 1.990.01 | 2.450.01 | |
| CAP-Mix | 2.880.01 +9.09% | 4.150.07 +2.72% | 5.820.05 −1.52% | 2.000.00 +26.58% | 2.410.02 +21.11% | 2.830.02 +15.51% | |
| SID | FlowRec | 2.640.00 | 4.040.03 | 5.910.09 | 1.490.01 | 1.940.01 | 2.410.02 |
| TIGER | 2.010.05 | 3.200.03 | 4.620.06 | 1.320.03 | 1.700.02 | 2.060.02 | |
| MaskGR | 3.300.05 | 5.190.07 | 7.410.07 | 2.140.02 | 2.750.03 | 3.310.03 | |
| CAP-Mix | 3.470.02 +5.15% | 5.170.00 −0.39% | 7.500.04 +1.21% | 2.290.02 +7.01% | 2.840.02 +3.27% | 3.430.02 +3.63% | |
| PCA | FlowRec | 3.620.02 | 5.540.04 | 8.030.04 | 2.040.03 | 2.660.04 | 3.280.04 |
| TIGER | 2.050.03 | 3.250.03 | 4.610.01 | 1.340.02 | 1.730.02 | 2.080.01 | |
| MaskGR | 3.460.03 | 5.250.06 | 7.460.07 | 2.240.03 | 2.820.02 | 3.380.01 | |
| CAP-Mix | 3.790.01 +4.70% | 5.770.02 +4.15% | 8.360.08 +4.11% | 2.340.03 +4.46% | 2.980.02 +5.67% | 3.630.04 +7.40% |
Table 12. Steam — full results. Conventions as in Table 10.
| steam | model | HR@5 | HR@10 | HR@20 | N@5 | N@10 | N@20 |
| scratch | FlowRec | 8.280.33 | 13.540.31 | 20.610.35 | 5.370.30 | 7.060.29 | 8.840.30 |
| TIGER | 7.940.17 | 13.180.24 | 21.201.59 | 4.900.07 | 6.590.07 | 8.580.38 | |
| MaskGR | 8.520.17 | 14.210.19 | 20.940.05 | 5.580.18 | 7.400.19 | 9.110.16 | |
| CAP-Mix | 8.350.31 −2.00% | 14.150.19 −0.42% | 21.090.05 −0.52% | 5.160.17 −7.53% | 7.020.18 −5.14% | 8.770.14 −3.73% | |
| SID | FlowRec | 8.280.33 | 13.540.31 | 20.610.35 | 5.370.30 | 7.060.29 | 8.840.30 |
| TIGER | 8.430.10 | 13.280.63 | 20.461.07 | 5.010.03 | 6.570.19 | 8.360.24 | |
| MaskGR | 8.640.09 | 14.430.24 | 20.640.23 | 5.540.07 | 7.400.14 | 8.970.13 | |
| CAP-Mix | 8.690.09 +0.58% | 13.940.24 −3.40% | 20.820.04 +0.87% | 5.570.04 +0.54% | 7.250.20 −2.03% | 8.990.23 +0.22% | |
| PCA | FlowRec | 8.100.24 | 13.450.11 | 20.700.22 | 5.180.23 | 6.880.19 | 8.720.22 |
| TIGER | 8.220.31 | 13.580.33 | 18.650.63 | 5.000.07 | 6.720.07 | 8.000.17 | |
| MaskGR | 8.640.00 | 14.430.05 | 20.970.23 | 5.570.01 | 7.420.02 | 9.080.07 | |
| CAP-Mix | 8.300.20 −3.94% | 14.280.02 −1.04% | 21.500.06 +2.53% | 5.070.09 −8.98% | 6.990.03 −5.80% | 8.810.04 −2.97% |
| ml-1m | model | HR@5 | HR@10 | HR@20 | N@5 | N@10 | N@20 |
| scratch | FlowRec | ||||||
| TIGER | |||||||
| MaskGR | |||||||
| CAP-Mix | |||||||
| SID | FlowRec† | ||||||
| TIGER | |||||||
| MaskGR | |||||||
| CAP-Mix | |||||||
| PCA | FlowRec | ||||||
| TIGER | |||||||
| MaskGR | |||||||
| CAP-Mix |
Table 13. ML-1M — full results. Conventions as in Table 10.
| yelp | model | HR@5 | HR@10 | HR@20 | N@5 | N@10 | N@20 |
| scratch | FlowRec | ||||||
| TIGER | |||||||
| MaskGR | |||||||
| CAP-Mix | |||||||
| SID | FlowRec† | ||||||
| TIGER | |||||||
| MaskGR | |||||||
| CAP-Mix | |||||||
| PCA | FlowRec | ||||||
| TIGER | |||||||
| MaskGR | |||||||
| CAP-Mix |
Table 14. Yelp-2022 — full results. Conventions as in Table 10.
References
- [1]Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019. doi: 10.1145/3292500.3330701. https://doi.org/10.1145/3292500.3330701.
- [2]David Arthur and Sergei Vassilvitskii. k-means++: the advantages of careful seeding. In Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’07, page 1027–1035, USA, 2007. Society for Industrial and Applied Mathematics. ISBN 9780898716245. https://dl.acm.org/doi/10.5555/1282838.1283494.DOI
- [3]James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for hyper-parameter optimization. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K. Weinberger, editors, Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011. https://proceedings.neurips.cc/paper_files/paper/2011/file/86e8f7ab32cfd12577bc2619bc635690-Paper.pdf.
- [4]Paul S. Bradley, Kristin P. Bennett, and Ayhan Demiriz. Constrained k-means clustering. Technical Report MSR-TR-2000-65, Microsoft Research, 2000. https://www.microsoft.com/en-us/research/publication/constrained-k-means-clustering/.
- [5]Gilles Celeux and Gérard Govaert. A classification EM algorithm for clustering and two stochastic versions. Computational Statistics & Data Analysis, 14(3):315–332, 1992. doi: 10.1016/0167-9473(92)90042-E. https://doi.org/10.1016/0167-9473(92)90042-E.
- [6]A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B, 39:1–38, 1977. http://web.mit.edu/6.435/www/Dempster77.pdf.DOI
- [7]Jiaxin Deng, Shiyao Wang, Kuo Cai, Leijian Ren, Qigen Hu, Weifeng Ding, Qiang Luo, and Guorui Zhou. OneRec: Unifying retrieve and rank with generative recommender and iterative preference alignment, 2025. https://arxiv.org/abs/2502.18965.
- [8]Junchen Fu, Xuri Ge, Alexandros Karatzoglou, Ioannis Arapakis, Suzan Verberne, Joemon M. Jose, and Zhaochun Ren. Differentiable semantic ID for generative recommendation. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’26, page 369–379, New York, NY, USA, 2026. Association for Computing Machinery. ISBN 978400725999. doi: 10.1145/3805712.3809641. https://doi.org/10.1145/3805712.3809641.
- [9]Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. Recommendation as language processing (RLP): A unified pretrain, personalized prompt & predict paradigm (P5). In Proceedings of the 16th ACM Conference on Recommender Systems, RecSys ’22, page 299–315, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450392785. doi: 10.1145/3523227.3546767. https://doi.org/10.1145/3523227.3546767.
- [10]F. Maxwell Harper and Joseph A. Konstan. The MovieLens datasets: History and context. ACM Trans. Interact. Intell. Syst., 5(4), December 2015. ISSN 2160-6455. doi: 10.1145/2827872. https://doi.org/10.1145/2827872.
- [11]Ruining He and Julian McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the 25th International Conference on World Wide Web, WWW ’16, page 507–517, Republic and Canton of Geneva, CHE, 2016. International World Wide Web Conferences Steering Committee. ISBN 9781450341431. doi: 10.1145/2872427.2883037. https://doi.org/10.1145/2872427.2883037.
- [12]Yupeng Hou, Jiacheng Li, Xiangjun Fu, Zhankui He, An Yan, Xiusi Chen, and Julian McAuley. Bridging language and items for retrieval and recommendation: Benchmarking LLMs as semantic encoders. In Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens, editors, Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3251–3265, San Diego, California, United States, July 2026. Association for Computational Linguistics. ISBN 979-8-89176-390-6. doi: 10.18653/v1/2026.acl-long.147. https://aclanthology.org/2026.acl-long.147/.
- [13]Clark Mingxuan Ju, Liam Collins, Leonardo Neves, Bhuvesh Kumar, Louis Yufeng Wang, Tong Zhao, and Neil Shah. Generative recommendation with semantic IDs: A practitioner’s handbook. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management (CIKM), 2025. doi: 10.1145/3746252.3761612. https://doi.org/10.1145/3746252.3761612.
- [14]Wang-Cheng Kang and Julian McAuley. Self-Attentive Sequential Recommendation. In 2018 IEEE International Conference on Data Mining (ICDM), pages 197–206, Los Alamitos, CA, USA, November 2018. IEEE Computer Society. doi: 10.1109/ICDM.2018.00035. https://doi.ieeecomputersociety.org/10.1109/ICDM.2018.00035.DOI
- [15]Xiaoyu Kong, Leheng Sheng, Junfei Tan, Yuxin Chen, Jiancan Wu, An Zhang, Xiang Wang, and Xiangnan He. MiniOneRec: An open-source framework for scaling generative recommendation, 2025. https://arxiv.org/abs/2510.24431.
- [16]Doyup Lee, Chiheon Kim, Saeh oon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. https://openaccess.thecvf.com/content/CVPR2022/html/Lee_Autoregressive_Image_Generation_Using_Residual_Quantization_CVPR_2022_paper.html.arxiv.org/abs/2203.01941
- [17]Jialei Li, Yang Zhang, Yimeng Bai, Shuai Zhu, Ziqi Xue, Xiaoyan Zhao, Dingxian Wang, Frank Yang, Andrew Rabinovich, and Xiangnan He. UniGRec: Unified generative recommendation with soft identifiers for end-to-end optimization, 2026a. https://arxiv.org/abs/2601.17438.
- [19]Zihao Li, Aixin Sun, and Chenliang Li. DiffuRec: A diffusion model for sequential recommendation. ACM Trans. Inf. Syst., 42(3), December 2023. ISSN 1046-8188. doi: 10.1145/3631116. https://doi.org/10.1145/3631116.
- [20]Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. https://openreview.net/forum?id=PqvMrD0Jt9t.arxiv.org/abs/2210.02747
- [21]Enze Liu, Bowen Zheng, Cheng Ling, Lantao Hu, Han Li, and Wayne Xin Zhao. Generative recommender with end-to-end learnable item tokenization. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’25, page 729–739, New York, NY, USA, 2025a. Association for Computing Machinery. ISBN 9798400715921. doi: 10.1145/3726302.3729989. https://doi.org/10.1145/3726302.3729989.
- [22]Feng Liu, Lixin Zou, Xiangyu Zhao, Min Tang, Liming Dong, Dan Luo, Xiangyang Luo, and Chenliang Li. Flow matching based sequential recommender model. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI ’25, 2025b. ISBN 978-1-956792-06-5. doi: 10.24963/ijcai.2025/346. https://doi.org/10.24963/ijcai.2025/346.
- [23]S. Lloyd. Least squares quantization in PCM. IEEE Transactions on Information Theory, 28(2):129–137, 1982. doi: 10.1109/TIT.1982.1056489. https://doi.org/10.1109/TIT.1982.1056489.
- [24]Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. Image-based recommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’15, page 43–52, New York, NY, USA, 2015. Association for Computing Machinery. ISBN 9781450336215. doi: 10.1145/2766462.2767755. https://doi.org/10.1145/2766462.2767755.
- [25]Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost, Maciej Kula, Ed H. Chi, and Maheswaran Sathiamoorthy. Recommender systems with generative retrieval. In Advances in Neural Information Processing Systems, 2023. https://openreview.net/forum?id=BJ0fQU32w.arxiv.org/abs/2305.05065
- [26]Kulin Shah, Bhuvesh Kumar, Neil Shah, and Liam Collins. Masked diffusion for generative recommendation, 2025. https://arxiv.org/abs/2511.23021.