The Vocabulary Is Already Here: Adapting Pretrained Interpretable Features for Generative Recommendation

CCS Concepts

• Information systems → Information retrieval; • Computing methodologies → Natural language processing.

Relevance

Relevance: This work is relevant to the Track of User modeling, personalization and recommendation. This work contributes to recommendation via adapting the pretrained vocabulary and provides a practical path for applying interpretability to deployment.

Introduction

Generative recommendation systems (GRs) adopts the token generation paradigm of language models (LMs) to retrieve items conditioned on the user’s interaction history [12, 13, 16]. A widely used approach represents items as the semantic IDs (SIDs), each consisting of a short sequence of discrete tokens. The mapping from items to SIDs, termed item tokenization, affects the GR pipeline from two fundamental perspectives: item representation and prediction supervision [28]. The resulting SIDs determine which item distinctions are retained and define the token-level predictive targets used to train the recommendation model. Therefore, a central problem in tokenization is how to construct compact SIDs that preserve item distinctions and provide effective supervision for recommendation.

Existing approaches commonly construct SIDs by fitting a special discrete vocabulary over item representations produced by pretrained language models (PLMs) [5, 20, 31]. Inspired by the strong generalizability of unified natural language vocabularies in LMs, recent efforts have explored shared vocabularies for generalizable item tokenization [4, 26]. For example, UTGRec [30] learns a transferable tokenizer through multi-domain pretraining, while UniTok [6] combines shared and specific experts for unified tokenization. Though these advances broaden tokenizer generalizability, they still construct vocabulary through additional fitting on the recommendation-specific data. The semantic coverage of the learned vocabulary therefore depends on the fitting data, while vocabulary construction incurs additional optimization costs. Moreover, fitting a vocabulary to recommendation-specific data does not necessarily yield SID tokens with readily interpretable semantics [2]. These observations raise a natural question: Is there a free lunch for vocabulary construction in generative recommendation?

On the basis of recent advances in mechanistic interpretability, we answer this question in the affirmative. Specifically, pretrained sparse autoencoders (P-SAEs) already provide feature dictionaries for their paired PLMs, with many features admitting human-interpretable descriptions [3]. For each item, the P-SAE encodes its PLM representation into a sparse activation vector, whose nonzero coordinates identify the expressed features and whose values indicate their strengths. For example, a red T-shirt might activate a feature associated with "red" more strongly than one associated with "blue". As illustrated in Figure 1(b–Figure 1c), the indices of these features can serve as SID tokens, providing a natural vocabulary that can be reused without additional recommendation-specific vocabulary learning. This vocabulary also carries inspectable semantics that P-SAE features often respond to recognizable semantic patterns in their inputs, allowing the corresponding tokens to indicate which properties are expressed in an item.

However, bringing this vocabulary into effective and explainable GR still requires addressing two key challenges. First, recommendation adaptation requires preserving item distinctions in both SID construction and the supervision built on these SIDs. As illustrated in Figure 1(d), similar items can share dominant features while differing in their activation strengths and less active features. Representing items only by their most active feature indices can discard these distinctions, while predicting shared indices alone does not ensure that the target item outranks similar candidates. Moreover, while these features make item semantics inspectable, unleashing the potential of the vocabulary requires extending this interpretability to the retrieval. This motivates the second challenge, retrieval explanation, which is to trace feature contributions to items and explain why the model favors one item over another.

To address these challenges, we propose API, a generative recommendation framework that Adapting the Pretrained Interpretable vocabulary for effective and explainable generative recommendation. This adaptation spans two complementary levels: item representation through Pretrained interpretable ID (PID) construction, and recommendation learning through a Semantic Prediction Adapter (SPA). PID construction prioritizes strongly activated features while discounting those prevalent across the catalog, producing compact identifiers with accompanying activation evidence. Building on these representations, SPA incorporates feature identities and relative activation strengths into history modeling and aligns feature prediction of the target. To supervise these predictions, SPA combines sequence likelihood and strength agreement into retrieval and introduces preference alignment against competing items. API further traces feature-level contributions to retrieval, linking the semantics of each feature to the predicted evidence that favors a retrieved item. Our contributions are summarized as:

• We propose API, a generative recommendation framework that replaces recommendation-specific vocabulary learning with pretrained concept adaptation, opening a practical path for applying interpretability to practical deployment.

• We introduce PIDs that preserve item distinctions through feature selection and relative strengths, and theoretically characterize their separability under compact token budget.

• We develop SPA to effective and interpretable recommendation, aligning user preferences through item-level supervision and explain ranking through feature contributions.

• Experiments substantiate pretrained vocabulary reuse as a viable basis for GR, showing that API support competitive recommendation and enables explainable retrieval. Code is available at https://anonymous.4open.science/r/API-C1F9/.

Related Works

Generative recommendation and item tokenization. Generative recommendation (GR) makes item identifiers part of the prediction space, placing item tokenization at the interface between item representation and model supervision. Language-based approaches include the text-to-text formulation [4] and the learned textual identifiers [22]. For structured identifiers, TIGER constructs semantic IDs (SIDs) through residual quantization [9] of pretrained language model (PLM) representations [20] while SEATER combines balanced tree indexing with contrastive learning [21]. LC-Rec then incorporate collaborative semantics into item indexing [24, 29] and ETGRec couples tokenizer learning with recommendation through alternating optimization [11]. UTGRec and UniTok further extend vocabulary learning across domains through shared tokenization [6, 30]. Despite these advances, the vocabulary learning still relies on recommendation-specific data which requires additional optimization and its semantic organization remains limited by the training domains. Our work extends pretrained knowledge reuse from continuous representations to the vocabulary units, adapting an existing PLM feature dictionary for recommendation. Sparse autoencoders and pretrained feature reuse. Motivated by the mechanistic interpretability [1, 7], sparse autoencoders (SAEs) decompose language model activations into sparse combinations of dictionary features, exposing interpretable structure within dense representations [16]. Subsequent work scales these dictionaries [3, 23], improves reconstruction under sparsity constraints [19] and releases pretrained SAEs for reuse through resources such as Gemma Scope [10]. In recommendation, SAERec uses SAE-derived intent priors to enrich sequence modeling [25], but does not explore feature-level explanations of generative retrieval. In this work, we explores a practical route from mechanistic interpretability to recommendation systems, reusing a pretrained SAE feature dictionary as a shared vocabulary for GR. This further requires adapting reconstruction-oriented features for recommendation supervision and connecting feature-level readable concepts to item retrieval.

Preliminaries

Generative recommendation. Let $\mathcal{I}$ denote an item set, where each item $i \in\mathcal{I}$ is associated with a textual description $x_i$ [14, 15]. Given a user’s sequential interaction history $\mathcal{H}_u = (i_1, \ldots, i_T)$, the task is to predict the subsequent item $i^+ \in\mathcal{I}$. Generative recommendation (GR) approaches this task by generating a discrete token sequence conditioned on $\mathcal{H}_u$ and mapping the generating sequence to items. Autoregressive (AR) decoding generates tokens sequentially, conditioned on preceding tokens [20], whereas parallel (PR) decoding predicts multiple token in a single decoding step [5].

Item tokenization. For text-based item tokenization [4], a pretrained language model (PLM) (e.g., GPT-2 [17], T5 [18]) encodes the item description $x_i$ into continuous hidden representations $h_i$. The tokenizer then converts them into a semantic ID (SID), a compact sequence of discrete tokens drawn from a shared set $\mathcal{V}$, referred to as the vocabulary. Formally, the tokenizer $\tau$ is defined:

where $K$ is the length and $c_{i,r} \in\mathcal{V}$ denotes the token at position $r$. The vocabulary thus determine the information retained in these tokens and the token-level supervision used for recommendation.

Pretrained sparse autoencoders. A pretrained sparse autoencoders (P-SAE) decomposes continuous PLM representations into sparsely activated dictionary features [3]. Given a hidden representation $h$ from its paired PLM, the encoder $E$ produces a non-negative activation vector $z = E(h)$. The corresponding reconstruction is

where $D = [d_1, \ldots, d_m]$ is the decoder dictionary and $b_{\mathrm{dec}}$ is the decoder bias. Each column $d_j$ represents a feature identified by its index $j$ and the activation $z_j$ indicates how strongly that feature is expressed in $h$. The associated pretrained semantic annotation $m_j$ provides a human-readable description of the feature (e.g., "red"). These feature indices can thus serve as a reusable token vocabulary, accompanied by activation strengths and semantic annotations.

Methodology

In this section, we propose API, a generative recommendation framework comprising Pretrained Interpretable ID (PID) construction and the Semantic Prediction Adapter (SPA). As illustrated in Figure 2, PID construction selects pretrained features and retains their relative activation strengths to mitigate collisions (§ 4.1). Building on this representation, SPA then learns user preferences from interaction histories through PID prediction (§ 4.2), with item-level supervision encouraging the GR model to favor the target item over other candidates (§ 4.3). We further extend explanation from item representations to retrieval decisions, revealing the evidence behind the preference for one candidate over another (§ 4.4).

Pretrained Interpretable IDs Construction

To construct PIDs that preserve item distinctions, we first identify the pretrained features expressed in each item and quantify their strengths. For an item description sentence $x_i$, the frozen PLM–SAE pair produces sparse activations $z_i^{(t)}$ over token $t$. We then aggregate the activations into an item-level strength for feature $j$:

where $\mathcal{T}_i$ denotes the token positions in $x_i$, excluding special token. The resulting $a_{ij}$ quantifies the item-level activation strength of feature $j$, indicating how strongly the item expresses that feature.

However, selecting feature indices solely by activation strength can assign the same token to distinct items. Within a domain, shared dominant features can occupy the token budget, excluding less active features that distinguish individual items. To allow these features to compete for inclusion, we form a candidate pool $\mathcal{C}_i$ containing the $M$ features with the highest positive activation strengths. For each feature $j \in\mathcal{C}_i$, we define a selection priority that favors strong activations while discounting features prevalent:

where $\rho_j$ denotes the smoothed prevalence of feature $j$, estimated from its occurrence in candidate pools of training-visible items. Lower prevalence can compensate for weaker activation, allowing less active attributes to compete with shared dominant features. We then select the $K$ features with the highest priorities $\pi_{ij}$ and order their indices by decreasing activation strength $a_{ij}$. Let $c_{i,r}$ denote the feature index at position $r$. The resulting sequence $c_i = (c_{i,1}, \ldots, c_{i,K})$ thus forms the token component of the PID.

Although selection by priority mechanism mitigates collisions caused by shared dominant features, a compact token budget still leave semantically similar items with identical token sequences. We observe that the selected features can nevertheless exhibit different relative activation strengths across them. To preserve these differences, we retain the relative activation strengths in a vector $q_i = (q_{i,1}, \ldots, q_{i,K})$ aligned with the token sequence $c_i$. Each entry $q_{i,r}$ records normalized log strength of the feature indexed by $c_{i,r}$:

We retain the corresponding pretrained semantic concepts as $m_i = (m_{c_{i,1}}, \ldots, m_{c_{i,K}})$, associating each feature index with its human-readable meaning. The feature-index sequence, relative activation strengths, and aligned semantic annotations together form the PID:

By retaining relative activation strengths, PIDs can distinguish items that share the same feature-index sequence. The following proposition further bounds the number of such items.

Proof. See Appendix B.1. The bound relates collisions to reconstruction error and variation in summed strengths. When $\Delta> 2\epsilon+ \delta$, the strengths can distinguish items when their token sequence coincide.

SPA: PID Prediction

Although PIDs retain relative activation strengths alongside feature, predicting the indices alone does not explicitly model these strengths. The Semantic Prediction Adapter (SPA) therefore incorporates feature-index and activation strengths into history encoding and predicts the next item’s token sequence and relative strengths.

Encoding history based PIDs. Though the PID construction selects $K$ features for compact item discrimination, the excluded may still provide evidence of user preferences. To retain this, SPA encodes each historical item using the candidate pool $C_i$ from which $c_i$ is selected, together with the corresponding activation strengths. Specifically, we represent item $i$ by the feature–strength pairs $\mathcal{E}_i = \{(j,a_{ij}) : j \in C_i\}$. The feature encoder $\Psi_{\theta}$ combines trainable feature-index embeddings with activation-strength encodings and aggregates them into a fixed number of tokens per item. Given the interaction history $\mathcal{H}_u = (i_1,\ldots,i_T)$, a recommendation Transformer $F_{\theta}$ processes these tokens in interaction order:

Predicting target PIDs. Given the history representation $\mathbf{H}_u$, SPA models the next item’s feature-index sequence through the conditional distribution $p_{\theta}(c \mid\mathbf{H}_u)$. SPA also predicts a strength vector $\hat{\mathbf{q}}_u$ from $\mathbf{H}_u$, where $\hat{q}_{u,r}$ estimates $q_{i+,r}$ at activation-ranked position $r$. Its entries satisfy $\hat{q}_{u,r} \ge0$ and $\sum_{r=1}^{K}\hat{q}_{u,r}=1$. The semantic annotations are obtained by looking up $m_j$ for each predicted feature index $j$. Its entries are normalized across the $K$ positions, satisfying $\sum_{r=1}^{K}\hat{q}_{u,r}=1$. In practice, the feature-index sequence can be generated autoregressively [4] or in parallel [5]. The remaining problem is how to favor the next item over competing catalog items.

SPA: Item-Level Supervision

Building on the feature-index and strength predictions, we use the observed next item’s PID to construct item-level supervision for GR training. We combine sequence likelihood and strength agreement into the score to supervise PID prediction and candidates ranking.

Comparing candidate PIDs. For a item $i$, let $\ell_u(i) = \log p_{\theta}(c_i \mid\mathbf{H}_u)$ denote the log likelihood of its feature-index sequence. We then define the item score by balancing sequence likelihood $\ell_u(i)$ with agreement between the predicted and candidate relative strengths:

where $\beta\ge0$ controls the strength penalty. The squared Hellinger distance $D_{\mathrm{H}}$ compares the predicted strengths $\hat{\mathbf{q}}_u$ with the candidate’s strengths $\mathbf{q}_i$ at corresponding activation-ranked positions:

For these normalized vectors, the distance lies in $[0,1]$ and equals zero exactly when their relative strengths match.

Learning user preferences. For the observed next item $i^+$, minimizing $-s_\theta(u,i^+)$ supervises its feature-index sequence and relative strengths. However, a better match to the target PID still leave competing items with higher scores. To encourage the target item to outrank competing candidates, we introduce a preference alignment loss over the candidate set $\mathcal{J}_u \subseteq\mathcal{I}$ containing $i^+$:

We then minimize the joint loss over observed interactions:

where $\lambda\ge0$ weights preference alignment. The following proposition shows how the relative strengths in PIDs can help SPA rank the target above the competing items with the same feature sequence.

Proof. See Appendix B.2. Under the stated condition, the strength differences retained in PIDs yield a positive ranking margin for the target. At retrieval, SPA ranks items using the same score $s_\theta(u,i)$.

Retrieval Explanation

To explain why SPA favors one candidate over another, we decompose their margin into contributions from individual interpretable features. We then associate these contributions with their semantic annotations to explain the feature evidence behind the ranking.

Decomposing retrieval scores to pretrained features. To quantify each feature’s contribution to the item score, we decompose the sequence log likelihood as $\ell_u(i) = \sum_{r=1}^{K} \ell_{u,r}(i)$. Here, $\ell_{u,r}(i)$ denotes the log probability assigned by SPA to feature index $c_{i,r}$ at position $r$. Combining this term with the strength agreement at the same position gives the feature contribution:

where $\phi_{u,i}(c_{i,r})$ measures the contribution of feature $c_{i,r}$ to candidate $i$’s score. For features $j$ absent from $c_i$, we set $\phi_{u,i}(j)=0$.

Explaining candidate rankings. To explain the ranking of candidates $i$ and $i'$, we compare the contributions of each feature $j$ to their respective scores. Their score difference decomposes as:

Each term measures feature $j$’s contribution to this difference, with positive values favoring $i$ and negative values favoring $i'$. We pair these contributions with the pretrained semantic annotations $m_j$ to show which features each candidate and by how much.

Experiments

We evaluate pretrained vocabulary reuse and proposed API to generative recommendation through the following research questions:

• RQ1: How well does API recommend across decoding strategies?

• RQ2: How well can PIDs preserve item distinctions and content?

• RQ3: Which supervision signals improve item retrieval?

• RQ4: How effectively can the pretrained vocabulary be reused?

• RQ5: Can item retain readable meanings with the PID construction and GR model support targeted-feature steering to retrieval?

Experimental Setup

Benchmark. We use the Beauty, Sports and Toys subsets of the pre-processed Amazon datasets aligned with prior work [4, 8], retaining their dataset splits and evaluation protocol. Item descriptions combine the title, brand, category and price fields. Recommendation models are trained on training interactions; PID selection statistics and activation calibration are estimated from training-item contents. Dataset statistics and details are provided in Appendix A.

Compared methods. We organize the main comparisons at two levels: item tokenization in Table 2 and complete recommendation frameworks in Table 1. The tokenization comparison includes Random, OPQ, HiD-VAE and residual quantization baselines (RVQ, RQ-VAE and RKMeans) that aligned with GRiD [8]. We then evaluate them with T5- and GPT-2-style backbones, adopting AR and PR implementations based on TIGER [20] and RPG [5], respectively. Detailed baseline and configurations are provided in Appendix A.3.

Method	Beauty				Sports				Toys
	R@5	R@10	N@5	N@10	R@5	R@10	N@5	N@10	R@5	R@10	N@5	N@10
GRU4Rec	0.0433	0.0646	0.0304	_0.0372_	0.0222	0.0371	0.0141	_0.0189_	0.0384	0.0543	_0.0272_	_0.0324_
Bert4Rec	0.0281	0.0434	0.0184	0.0233	0.0129	0.0224	0.0079	0.0110	0.0201	0.0313	0.0133	0.0168
Tiger (2023)	0.0464	_0.0662_	0.0305	0.0369	_0.0237_	0.0341	_0.0156_	_0.0189_	_0.0402_	0.0584	0.0263	0.0322
LCRec (2024)	0.0228	0.0362	0.0159	0.0197	0.0082	0.0133	0.0073	0.0089	0.0174	0.0285	0.0119	0.0157
RPG (2025)	0.0322	0.0502	0.0210	0.0269	0.0176	0.0266	0.0124	0.0153	0.0306	0.0477	0.0192	0.0247
UTGRec (2026)	0.0193	0.0330	0.0122	0.0167	0.0126	0.0219	0.0080	0.0109	0.0177	0.0304	0.0107	0.0147
Ghost (2026)	0.0388	0.0590	0.0257	0.0322	0.0260	0.0422	0.0171	0.0223	0.0333	0.0543	0.021	0.0277
DIGR (2026)	0.0431	0.0605	0.0307	0.0362	0.0186	0.0291	0.0129	0.0161	0.0396	0.0571	0.0268	0.0319
TDPM (2026)	0.0418	0.0655	0.0279	0.0355	0.0216	0.0321	0.0140	0.0173	0.0398	_0.0591_	0.0259	0.0322
API (Ours)	_0.0457_	0.0694	_0.0305_	0.0389	0.0271	_0.0411_	0.0188	0.0239	0.0414	0.0616	0.0291	0.0353

Back.	Generation	Tokenization	Beauty R@5	Beauty R@10	Beauty N@5	Beauty N@10	Sports R@5	Sports R@10	Sports N@5	Sports N@10	Toys R@5	Toys R@10	Toys N@5	Toys N@10
T5	AR	Random	0.0310	0.0454	0.0216	0.0262	0.0082	0.0111	0.0051	0.0060	0.0188	0.0250	0.0140	0.0161
T5	AR	RVQ	0.0464	0.0662	0.0305	0.0369	0.0237	0.0341	0.0156	0.0189	0.0402	0.0584	0.0263	0.0322
T5	AR	RQVAE	0.0402	0.0602	0.0269	0.0334	0.0215	0.0310	0.0139	0.0170	0.0344	0.0492	0.0240	0.0290
T5	AR	RKMeans	0.0441	0.0667	0.0305	0.0378	0.0217	0.0330	0.0145	0.0181	0.0384	0.0564	0.0256	0.0314
T5	AR	HiD-VAE	0.0419	0.0644	0.0283	0.0355	0.0215	0.0328	0.0138	0.0175	0.0388	0.0595	0.0257	0.0324
T5	AR	PID	0.0435	0.0630	0.0299	0.0361	0.0210	0.0346	0.0135	0.0178	0.0401	0.0585	0.0269	0.0328
T5	AR	+ SPA PID	0.0476	0.0701	0.0323	0.0396	0.0243	0.0372	0.0157	0.0198	0.0419	0.0607	0.0287	0.0347
T5	PR	OPQ ($K = 4$)	0.0290	0.0466	0.0196	0.0256	0.0057	0.0105	0.0034	0.0049	0.0248	0.0373	0.0160	0.0200
T5	PR	OPQ ($K = 8$)	0.0368	0.0543	0.0247	0.0303	0.0080	0.0134	0.0052	0.0070	0.0320	0.0485	0.0210	0.0263
T5	PR	OPQ ($K = 16$)	0.0312	0.0484	0.0203	0.0259	0.0114	0.0199	0.0076	0.0102	0.0278	0.0424	0.0171	0.0218
T5	PR	PID ($K = 4$)	0.0278	0.0402	0.0194	0.0234	0.0049	0.0087	0.0029	0.0041	0.0294	0.0488	0.0182	0.0245
T5	PR	PID ($K = 8$)	0.0370	0.0541	0.0262	0.0316	0.0094	0.0158	0.0062	0.0083	0.0332	0.0547	0.0207	0.0276
T5	PR	PID ($K = 16$)	0.0340	0.0523	0.0225	0.0284	0.0132	0.0194	0.0086	0.0106	0.0348	0.0509	0.0209	0.0262
T5	+ SPA	PID ($K = 4$)	0.0388	0.0572	0.0264	0.0323	<u>0.0168</u>	<u>0.0267</u>	<u>0.0111</u>	<u>0.0142</u>	<u>0.0352</u>	<u>0.0517</u>	0.0242	0.0295
T5	+ SPA	PID ($K = 8$)	0.0401	0.0627	0.0274	0.0346	0.0192	0.0289	0.0126	0.0157	0.0354	<u>0.0517</u>	0.0229	0.0282
T5	+ SPA	PID ($K = 16$)	<u>0.0394</u>	<u>0.0575</u>	<u>0.0271</u>	<u>0.0329</u>	0.0162	0.0241	0.0106	0.0131	0.0319	0.0462	0.0210	0.0256
GPT2	AR	Random	0.0292	0.0401	0.0207	0.0247	0.0070	0.0122	0.0039	0.0056	0.0133	0.0176	0.0094	0.0108
GPT2	AR	RVQ	0.0394	0.0594	0.0255	0.0320	0.0197	0.0331	0.0126	0.0169	0.0327	0.0536	0.0206	0.0274
GPT2	AR	RQVAE	0.0362	0.0542	0.0242	0.0301	0.0166	0.0271	0.0103	0.0137	0.0296	0.0439	0.0192	0.0238
GPT2	AR	RKMeans	0.0423	0.0640	0.0278	0.0348	0.0182	0.0294	0.0110	0.0145	0.0313	0.0508	0.0195	0.0258
GPT2	AR	HiD-VAE	0.0398	0.0607	0.0261	0.0329	<u>0.0223</u>	<u>0.0360</u>	<u>0.0141</u>	<u>0.0185</u>	0.0335	<u>0.0567</u>	0.0211	0.0286
GPT2	AR	PID	0.0420	<u>0.0663</u>	<u>0.0282</u>	<u>0.0361</u>	0.0190	0.0292	0.0126	0.0159	0.0380	0.0529	<u>0.0265</u>	<u>0.0313</u>
GPT2	AR	+ SPA PID	0.0457	0.0694	0.0305	0.0389	0.0271	0.0411	0.0188	0.0239	0.0414	0.0616	0.0291	0.0353
GPT2	PR	OPQ ($K = 4$)	0.0292	0.0457	0.0197	0.0250	0.0100	0.0175	0.0065	0.0089	0.0267	0.0449	0.0164	0.0222
GPT2	PR	OPQ ($K = 8$)	0.0374	0.0549	0.0248	0.0305	0.0161	0.0241	0.0109	0.0135	0.0274	0.0441	0.0165	0.0219
GPT2	PR	OPQ ($K = 16$)	0.0322	0.0502	0.0210	0.0269	0.0176	0.0266	0.0124	0.0153	0.0306	0.0477	0.0192	0.0247
GPT2	PR	PID ($K = 4$)	0.0267	0.0393	0.0193	0.0234	0.0153	0.0234	0.0101	0.0127	0.0330	0.0483	0.0230	0.0279
GPT2	PR	PID ($K = 8$)	0.0379	0.0540	0.0270	0.0322	0.0162	0.0309	0.0118	0.0167	0.0307	0.0447	0.0214	0.0259
GPT2	PR	PID ($K = 16$)	0.0336	0.0525	0.0228	0.0288	0.0179	0.0281	0.0121	0.0156	0.0267	0.0383	0.0189	0.0226
GPT2	+ SPA	PID ($K = 4$)	0.0411	0.0618	0.0277	0.0343	0.0195	<u>0.0300</u>	0.0129	<u>0.0163</u>	0.0393	0.0559	0.0281	0.0334
GPT2	+ SPA	PID ($K = 8$)	<u>0.0395</u>	<u>0.0573</u>	<u>0.0274</u>	0.0331	<u>0.0186</u>	0.0294	0.0120	0.0154	<u>0.0384</u>	<u>0.0531</u>	<u>0.0271</u>	<u>0.0318</u>
GPT2	+ SPA	PID ($K = 16$)	<u>0.0398</u>	<u>0.0573</u>	0.0277	0.0333	0.0161	0.0253	0.0104	0.0133	0.0377	0.0515	0.0262	0.0306

Implementation. We extract item features using pretrained GPT-2-small [17] and its paired P-SAE with 24, 576 labeled dictionary features, keeping both models frozen. The T5-style and GPT-2-style recommendation backbones are trained from random initialization. We retain 32 candidate features per item and use $K \in\{4, 8, 16\}$ tokens. We optimize the recommendation models using Adam with a learning rate of $10^{-3}$ and weight decay of $10^{-4}$.

The loss and strength weights are set to $\lambda_{\mathrm{pref}} = 0.1$ and $\beta= 8$, respectively. Checkpoints are selected by Recall@5 on the validation split. Detailed configurations are provided in Appendix A.3.

Evaluation. We report Recall@5/10 and NDCG@5/10 on the complete test split, using the full catalog without sampled negatives. For PID discrimination, collision rate measures the fraction of items whose selected feature set is shared by another item. For content preservation, Overlap@20 measures the average fraction of shared neighbors between representation-based and reference top-20 lists. For steering, we report feature exposure@10, the fraction of top-10 recommendations whose PIDs contain the target feature. PR uses exact catalog scoring, while AR uses catalog-constrained beam search with a width of 50. Detailed are provided in Appendix A.1.

Findings for RQ1 (Recommendation Performance)

Finding 1: Effective vocabulary reuse. Pretrained features support competitive GR without additional vocabulary learning.

Finding 2: Benefits of recommendation adaptation. SPA improves over direct PID in all evaluated AR settings and most PR settings.

Finding 3: Competitive framework performance. API achieves the best reported results in nine of twelve dataset–metric comparisons.

Recommendation Performance (RQ1)

Table 1 compares the complete API framework with sequential and generative recommenders. API achieves the best reported result in nine of the twelve dataset–metric comparisons, including all four metrics on Toys. It ranks second on Beauty Recall@5/NDCG@5 and Sports Recall@10, with a tie on Beauty NDCG@5. Table 2 further examines how recommendation adaptation affects the use of the pretrained vocabulary. Direct PID does not uniformly outperform learned tokenizers: on Beauty with T5-PR and $K = 4$, its Recall@5 is 0.0278, compared with 0.0290 for OPQ, while adding SPA raises it to 0.0388. Across the two backbones and three datasets, SPA improves Recall@5 over direct PID in all six AR settings and 16 of the 18 PR settings. With T5-AR, the complete API pipeline exceeds the strongest learned-tokenizer baseline by 2.5%–4.2% in Recall@5. Both PR exceptions occur at $K = 16$, while the best API-PR Recall@5 for every backbone–dataset pair is attained at $K = 4$ or $K = 8$. These results also show that increasing identifier length does not consistently improve recommendation.

PID Quality (RQ2)

Table 3 compares feature selection policies within the same top-32 candidate pool, measuring collisions between selected feature sets. PID improves item discrimination under compact budgets: at $K = 4$, it reduces the mean collision rate across the three datasets from 80.76% under native selection to 17.09%. The policies coincide at $K = 32$, when the entire candidate pool is retained. Random selection yields even fewer collisions, but Figure 55 shows substantially weaker content alignment at $K = 4$ and $K = 8$. At these budgets, PID achieves the highest Overlap@20 among the compared methods on Beauty and Sports against both title-based and PLM-based references. These comparisons indicate that low collision rates alone do not establish content preservation. To further separate these properties, Table 4 permutes complete feature–activation profiles across items while keeping item content fixed. Collision rates remain unchanged, whereas Overlap@20 falls below 0.2% against both references. Finally, activation-aware neighborhoods show stronger content alignment: with the selected feature indices fixed at $K = 8$, activation-weighted cosine neighborhoods exceed index-only Jaccard neighborhoods by 1.03%, 2.60% and 4.41% in PLM-based Overlap@20 on all benchmarks.

Findings for RQ2 (PID Quality)

Finding 1: Compact discrimination. Priority-based feature selection reduces collisions within the same frozen vocabulary.

Finding 2: Discrimination do not guarantee content alignment.

Dataset	Selection	$K = 4$	$K = 8$	$K = 16$	$K = 24$	$K = 32$
Beauty	Native SAE	77.37	42.72	11.11	2.86	1.23
Beauty	Random ID	2.50	0.00	0.00	0.00	1.23
Beauty	PID	15.16	3.33	1.26	1.19	1.23
Sports	Native SAE	79.96	51.77	19.98	6.30	3.44
Sports	Random ID	3.67	0.00	0.00	0.00	3.44
Sports	PID	21.48	7.71	3.43	3.20	3.44
Toys	Native SAE	84.95	57.59	17.96	4.68	1.49
Toys	Random ID	2.59	0.00	0.00	0.00	1.49
Toys	PID	14.64	4.54	1.51	1.49	1.49

Dataset	Coll.	Index only: Title Orig.	Index only: Title Perm.	Index only: PLM Orig.	Index only: PLM Perm.	Index + Activation: Title Orig.	Index + Activation: Title Perm.	Index + Activation: PLM Orig.	Index + Activation: PLM Perm.
Beauty	3.33	24.15	0.19	33.02	0.16	25.54	0.17	34.05	0.14
Sports	7.71	24.83	0.12	37.01	0.16	26.21	0.11	39.61	0.15
Toys	4.54	25.90	0.12	38.81	0.12	27.47	0.15	43.22	0.14

SPA Supervision (RQ3)

Table 5 examines preference alignment and strength modeling under T5-PR with $K = 8$, fixing the SPA architecture, activation-aware history encoding, feature-index targets, and output ordering. Starting from SID prediction alone, we add preference alignment, strength prediction and scoring, or both. Preference alignment consistently improves retrieval, increasing Recall@10 by 13.3%, 45.9%, and 9.4% on Beauty, Sports, and Toys, respectively. Adding strength prediction and scoring alone lowers NDCG@10 on all three datasets. Once preference alignment is included, however, strength modeling further improves NDCG@10 by 4.0%–6.0%. Full SPA achieves the highest Recall@10 and NDCG@10 throughout this ablation. These results suggest that strength evidence is more effective when coupled with item-level preference alignment.

Findings for RQ3

Finding 1: Learning through item competition. Catalog-grounded supervision provides the most consistent gains in this ablation.

Finding 2: Complementary strength supervision. SPA provides additional ranking gains when combined with preference alignment.

Vocabulary Reuse (RQ4)

Figure 3 evaluates learned tokenizers fitted on Beauty or Toys and frozen across all three target catalogs. PID retains the frozen P-SAE dictionary and estimates IDF from target training-item contents; all recommenders are trained on target-domain interactions. On Beauty and Toys, several learned tokenizers incur more collisions and lower Recall@5 when fitted on the other domain. PID reuses the vocabulary without refitting, maintaining comparatively low collision rates and competitive Recall@5 across targets: it leads on Toys and Sports under both source configurations, while R-SAE remains ahead on Beauty. Native top-$K$ selection produces substantially more collisions than PID using the same dictionary, highlighting the importance of catalog-specific selection. Figure 6(b) extrapolates single-task timings to three equal-workload tokenization tasks, counting UTGRec pretraining once and applicable fitting and processing per task. Treating the pretrained PLM and P-SAE as available resources, cumulative time relative to one GPT-2 encoding task is 3.78× for API, 407.69× for RVQ, and 674.99× for

Supervision	Beauty R@10	Beauty N@10	Sports R@10	Sports N@10	Toys R@10	Toys N@10
Index-only	0.05080	0.02959	0.01980	0.01104	0.04523	0.02518
+ Strength	0.04905	0.02833	0.01893	0.01055	0.04533	0.02447
+ Preference	0.05755	0.03312	0.02888	0.01509	0.04950	0.02656
Full SPA	0.06265	0.03456	0.02891	0.01569	0.05172	0.02816

UTGRec. API thus avoids additional vocabulary-fitting costs, while retaining per-task feature extraction and encoding.

Retrieval Explanation (RQ5)

Figure 4 illustrates readable feature semantics through Sports feature $f_{4260}$, whose external descriptions refer to ammunition and weapons. Among training items whose $K = 4$ PIDs contain this feature, 77 of 78 match the hunting-and-shooting lexical rule, yielding 98.7% agreement and a 7.91× lift over the rule’s prevalence in the training catalog. The highest-activation titles also describe training ammunition. The Toys case in Figure 6 further provides traceable ranking evidence: a LEGO brick box outranks Bruder construction accessories by a margin of +0.2897, comprising +0.2993 from token likelihoods and −0.0096 from strength agreement. At PID position 1, the contrast between LEGO-related $f_{13542}$ and $f_{13209}$ favors the LEGO item, whereas position 2 opposes it. Finally, we edit existing history activations while keeping model parameters and candidate PIDs fixed to probe feature-dependent retrieval responses. For $f_{4260}$, a +0.50 shift in standardized log-activation increases exposure@10 by 1.365 percentage points relative to matched control edits on 332 eligible test users (Appendix § 9.§ 2).

Finding 1: Vocabulary sharing without refitting. Frozen pretrained dictionary supports competitive recommendation across benchmarks.

Finding 2: Adaptation within a shared vocabulary. PID reduces the collisions while preserving the pretrained feature-index identities.

Finding 1: Inspectable ranking evidence. Score decomposition links readable features to evidence supporting or opposing a candidate.

Finding 2: Local steering effects. Activation edits can change exposure, with responses varying across features and doses.

Conclusion

We propose API to address catalog-dependent vocabulary learning in generative recommendation by adapting a pretrained interpretable dictionary into a shared vocabulary. Through PID construction and SPA, API connects compact item representations and user preference learning with interpretable retrieval evidence, opening a practical path for applying mechanistic interpretability to recommendation. Learning more general-purpose interpretable vocabularies and fully realizing their potential for recommendation remain promising directions for future research.

LLM Usage. Assist with baseline reproduction, refining grammar, and reviewing mathematical proofs. The authors take full responsibility for the final manuscript and experimental results.

Experimental Details

Data and Recommendation Evaluation

We retain the supplied Amazon splits and item orderings, with one validation and one test target per user. Validation uses up to 20 preceding training interactions; test uses up to 20 preceding interactions, including the validation item. The TIGER sampler converts 32 user sequences into at most 512 contiguous next-item windows per micro-batch, each containing up to 29 history items and one target. Evaluation uses unaugmented queries in batches of eight, including the final partial batch.

For target rank $r_u$, Recall@$k$ and NDCG@$k$ average $\mathbb{1}[r_u \leq k]$ and $\mathbb{1}[r_u \leq k]/\log_2(r_u+1)$ over users, respectively, for $k \in\{5,10\}$;

Dataset	Users	Items	Interactions	Train items
Beauty	22,363	12,101	198,502	12,068
Sports	35,598	18,357	296,337	18,284
Toys	19,412	11,924	167,597	11,876

unretrieved targets contribute zero. Evaluation uses item identities, without sampled negatives or seen-item filtering. PR scores the full catalog exactly; AR uses beam search over the catalog trie with width 50. Checkpoints selected by full-validation Recall@5 are reloaded for full-test evaluation.

Pretrained Resources and PID Construction

We extract GPT-2-small [17] with Bloom’s SAE through SAELens (gpt2-small-res-jb, hook blocks.8.hook_resid_pre). Item text concatenates title, brand, category, and price, truncated to 128 tokens. We pool token-level SAE activations excluding special and padding tokens, with decoder-norm scaling as in (3). Recommendation backbones are randomly initialized.

Each item contributes its top-256 positive features. We estimate $\rho_j=(n_j+1)/(N_{\mathrm{tr}}+1)$ from training items, where $n_j$ counts candidate set $j$ and $N_{\mathrm{tr}}$ counts training items; compact remapping preserves original feature identities. SPA orders selected features by decreasing activation, breaking ties by index; direct PID-PR uses training-frequency ordering. Ordering is fixed within SPA ablations. A nonsemantic suffix disambiguates shared supports, yielding $\tilde{c}_i=(c_i,v_i)$; strength profiles and collision diagnostics exclude it.

History encoding standardizes $\log(1+a_{ij})$ using featurewise training means $\mu_j$ and standard deviations $\sigma_j$, smoothed with 16 global pseudo-observations and a standard-deviation floor of 0.05. Values clipped to $[-2,2]$ interpolate eight learned gain/shift embeddings that modulate feature embeddings before GELU. Four priority-selected anchors incorporate projected mean top-32 context and position embeddings; a fifth encodes the suffix. Layer normalization yields five history tokens per item for every output $K$. Output strengths normalize selected log activations without history calibration.

Backbones, Training, and Baselines

Setting	T5 AR/PR	GPT-2 PR	GPT-2 AR
Layers (E/D)	4/4	0/4	0/4
Width / heads	128/6	448/8	128/8
K/V per head	64	56	16
FFN widths	1,024/1,024	1,792	1,024
Activation	ReLU	SiLU	GELU
Dropout	0.15	0.15	0.15

T5 uses relative positions and a three-linear-layer FFN; GPT-2 uses learned positions (512 for PR, history-plus-target capacity for AR). RPG-style PR baselines flatten SIDs, use position-specific vocabularies and tied input/output embeddings, and apply normalized-dot-product prediction heads at temperature 0.07. T5-SPA-PR uses a noncausal query decoder; GPT-2-SPA-PR appends learned role queries under causal attention. Both predict all roles in one pass without target-token inputs. AR uses shifted teacher forcing and candidate-specific prefixes. The history-only strength prediction is shared across candidates.

The standard recommendation recipe uses seed 45, the gradient accumulation over 16 microbatches, without learning-rate warmup or scheduling. Full validation occurs every 1,600 microbatches; early stopping allows ten unimproved checks, with limits of ten epochs and 320,000 optimizer steps. These limits concern recommendation training, not tokenizer fitting. Equal data protocols do not imply equal parameter counts or early-stopped exposure.

Full PR-SPA uses $(\lambda, \beta) = (1, 8)$ in Eq. (12). The four supervision settings are $(0, 0)$, $(0, 8)$, $(1, 0)$, and $(1, 8)$; all retain activation-aware histories and SID prediction. PR preference loss is positive-item cross-entropy after a temperature-one softmax over all catalog scores. The AR implementation instead uses the positive and four distinct uniform negatives, scoring each under its own prefix; this five-item objective is not exact full-catalog cross-entropy. T5-SPA-AR settings use $(\lambda, \beta) = (0.25, 2)$ on Beauty and $(1, 8)$ on Sports/Toys. At inference, AR normalizes over the full token vocabulary before masking illegal trie edges and applies strengths at complete leaves.

OPQ rotates and product-quantizes mean-pooled PLM residuals; the matched PR recipe uses $K$ subquantizers, 256 centroids each, 25 rotation/PQ iterations, and no additional PCA or whitening. Residual-tokenizer baselines preserve stage-specific code identities: RVQ learns residual codebooks, RKMeans applies residual clustering, and RQ-VAE learns an encoder/decoder around residual quantization. The verified GPT-2 residual-AR assets use four semantic indices plus one suffix. Their original-recipe tokenizers fit all catalog item contents (without held-out interaction labels), with seed 42 and batch size 2,048. RVQ uses per-stage residual normalization, Adagrad at $10^{-3}$, commitment weight 0.25, and 3,000 updates; RKMeans uses the same residual normalization, SGD at 0.5, and 30 updates. RQ-VAE uses batch/L2-normalized inputs, a 768–768–256–128–64 encoder and mirrored decoder, no per-stage normalization, and the same Adagrad rate, commitment weight, and update budget as RVQ. Random recommendation codes are distinct from the random-positive-feature selection control used below. The source-transfer recipes in Appendix C.1 form a separate diagnostic cohort, not additional replications of the main-table tokenizer fits.

Baselines. (1) Item tokenization. Random assigns codes independently of item content. OPQ [5] jointly optimizes a rotation and subspace codebooks. RVQ learns successive residual codebooks, whereas R-KMeans clusters successive residuals; we follow their GRID implementations [8]. RQ-VAE [9] learns an encoder and decoder around residual quantization, and Hid-VAE [2] constructs hierarchical, disentangled semantic identifiers. Native P-SAE selects the strongest pretrained features, whereas R-SAE learns a sparse dictionary from item embeddings. The random-positive control samples from the active feature pool and differs from Random item codes. (2) Recommendation frameworks. Our tokenizer comparisons use TIGER-style AR [20] and RPG-style PR [5] with T5/GPT-2 backbones: the former predicts SID tokens sequentially, while the latter predicts the code positions in parallel. The complete-framework comparison additionally includes LC-Rec [29], which aligns collaborative semantics with language-model representations; UTGRec [30], which learns a shared tokenizer for transfer across recommendation domains; Ghost [27], which addresses popularity bias through asymmetric unlikelihood optimization and skeleton-founded tokenization. DiGR [28] instead uses SIDs as item representations for direct item-level scoring. GRU4Rec [12] and BERT4Rec [22] provide conventional sequential-recommendation references using recurrent and bidirectional masked-item modeling.

Proofs and Score Derivatives

Proof of Proposition 4.1

Proof. Fix a group $\mathcal{G}$ of items sharing the token sequence $\mathbf{c}=(c_1,\ldots,c_K)$. Let $\mathbf{h}_i=|\mathcal{T}_i|^{-1}\sum_{t\in\mathcal{T}_i}\mathbf{h}^{(t)}_i$ denote the mean PLM representation over the positions used in Eq. (3). We assume nonnegative activations with positive sums $\omega_i=\sum_{r=1}^K a_{i,c_r}$, ensuring that $\mathbf{q}_i$ is well-defined.

Write $\mathbf{u}_j=\mathbf{d}_j/\|\mathbf{d}_j\|_2$ for nonzero decoder columns, and set $\mathbf{u}_j=\mathbf{0}$ otherwise. By Eq. (3), the mean SAE reconstruction using only the selected features is

The assumptions in the proposition give $\|\mathbf{h}_i-\tilde{\mathbf{h}}_i\|_2\le\epsilon$ for every $i\in\mathcal{G}$, while the sums $\omega_i$ lie in an interval of width at most $\delta$. Consider $n_{\max}$ items sharing the same PID. The case $n_{\max}=1$ is immediate. For any two distinct items $i,i'$ in this set, $\mathbf{q}_i=\mathbf{q}_{i'}$. Equation (5) therefore implies that

for some $\alpha>0$. Since the log strengths are nonnegative and the logarithm is strictly increasing, all differences $a_{i,c_r}-a_{i',c_r}$ have the same sign. Using $\|\mathbf{u}_{c_r}\|_2\le1$ and the triangle inequality, we obtain

Thus, any two such items have activation sums separated by at least $\Delta-2\epsilon>0$. Order these sums as $\omega_{(1)}\le\cdots\le\omega_{(n_{\max})}$. Summing the lower bounds on adjacent gaps gives

Rearranging and using the integrality of $n_{\max}$ yields

In particular, $\delta< \Delta- 2\epsilon$ forces $n_{\max}=1$, so all items in $\mathcal{G}$ have distinct PIDs. □

Proof of Proposition 4.2

Proof. Fix a user $u$, a target item $i^{+}$, and a competing item $i^{-}$. Let $\hat{\mathbf{q}}_u$ denote SPA’s prediction of the next item’s relative strengths given $\mathbf{H}_u$, and let $\mathbf{q}_{i^{+}}$ and $\mathbf{q}_{i^{-}}$ be the strength vectors recorded in their PIDs.

Write $\mathbf{x}=\sqrt{\hat{\mathbf{q}}_u}$, $\mathbf{y}=\sqrt{\mathbf{q}_{i^{+}}}$, and $\mathbf{z}=\sqrt{\mathbf{q}_{i^{-}}}$, with square roots taken elementwise. Since the strength vectors are nonnegative and sum to one, $\lVert\mathbf{x}\rVert_2=\lVert\mathbf{y}\rVert_2=\lVert\mathbf{z}\rVert_2=1$. The definitions of $\Delta_q$ and $\epsilon_u$ therefore give

By (10), the difference between the two strength penalties is

Writing $\mathbf{x}=\mathbf{y}+(\mathbf{x}-\mathbf{y})$ and using $\mathbf{y}^{\mathsf{T}}(\mathbf{y}-\mathbf{z})=\Delta_q$, the Cauchy–Schwarz inequality yields

Substituting this bound into (9) gives

This inequality holds for general candidate pairs. Under the proposition’s assumption $c_{i^{+}}=c_{i^{-}}$, their sequence likelihoods are equal, so the bound reduces to (13).

Finally, $\epsilon_u<\Delta_q/4$ implies $\Delta_q-2\sqrt{\Delta_q\epsilon_u}>0$. Together with $\beta>0$, this gives $s_{\theta}(u,i^{+})>s_{\theta}(u,i^{-})$, establishing the claimed ranking. □

Diagnostic and Interpretation Protocols

Vocabulary Quality

For a catalog of $N$ items and a suffix-free key $g(i)$, let $n_i$ count items sharing that key. The collision rate is $N^{-1}\sum_i 1[n_i>1]$. SAE keys are unordered positive-feature supports; quantizer keys retain stage identity. Strengths are excluded, so this statistic measures support ambiguity rather than equality of complete PIDs. Native, priority-based, and random-positive selection share a top-32 candidate pool; random selection samples without replacement.

Content alignment uses 2,000 fixed queries per domain (seed 45), the full catalog excluding each query, and a shared random tie order (seed 46). For representation and reference top-20 neighborhoods $\mathcal{N}_i$ and $\mathcal{N}^{\mathrm{ref}}_i$,

References use cosine similarity of title TF–IDF (lowercase, English stop-word removal, unigrams/bigrams, sublinear TF, smoothed IDF) or mean-pooled PLM residuals. Index-only neighborhoods use support Jaccard; activation-aware neighborhoods use cosine similarity of sparse, decoder-rescaled raw activations, distinct from SPA’s normalized log strengths. Permutation moves complete feature–activation profiles across items with content fixed, preserving collision multiplicities.

Reuse diagnostics freeze source-fitted tokenizers across targets, using four semantic tokens and the same 10,000-item subset per target for collision evaluation. R-SAE uses 4,096 features and 3,000 updates; RQ-VAE uses 3,000 updates. RVQ/R-KMeans use globally RMS-scaled residuals without per-stage normalization, taking the first four stages of 32-stage encoders trained for 100 updates per stage. OPQ fits source rotation/codebooks. PID retains the frozen P-SAE dictionary and estimates IDF from target training-item contents. Comparisons match token counts, not bit rates; downstream recommenders are trained on target interactions.

Semantic Annotations and Steering

Neuropedia supplies external, automatically generated feature descriptions, unused as model inputs. Title agreement is the lexical-match rate among feature-containing training items; lift divides it by the training-catalog match rate. For $f_{4260}$, the case-insensitive whole-word rule matches hunt, hunting, rifle, rifles, gun, guns, pistol, shotgun, holster, ammo, ammunition, or scope.

Steering freezes T5-PR Full SPA ($K=8$) and edits six prespecified features per domain at doses $d\in\{0,\pm0.25,\pm0.50\}$. Existing positive history activations are shifted by $d$ in unclipped standardized log coordinates, bounded below by the positive-activation floor, clipped to $[-2,2]$, and re-encoded; supports, anchors, suffixes, and candidate PIDs stay fixed. Eligible users have the target feature in at least two history items and an anchor responsive in both directions. Within this cohort, three deterministic control features match history frequency, anchors, and activation statistics; per-query scaling matches post-clipping perturbation norms.

Exposure@10 measures the fraction of retrieved items whose PID contains the target feature. Changes from unedited histories are reported in percentage points; Toys equally averages six feature-specific effects. Doses are validation-selected to retain at least 98% Recall@10 on both the full split and matched cohort. Curve bands show min–max over three runs. Paired target-minus-control effects average counterfactuals within users; nominal 95% percentile intervals use 500 paired user-bootstrap resamples without correction across 36 feature–direction comparisons. For Sports $f_{4260}$ at $d=0.50$, the effect is +1.365 points on 332 test users, with interval [0.652, 2.189].

Additional Results

Vocabulary reuse. Figure 7 extends the source-domain comparison to Sports. Across the three target catalogs, PID has substantially fewer collisions than native P-SAE while using the same frozen dictionary. RQ-VAE attains lower collision rates in this comparison, while PID supports item distinction without extra vocabulary fitting.

Collisions under compact budgets. Figure 8 compares feature selection and quantization at $K\in\{4,8,16\}$. At $K=4$, PID reduces the collision-affected rate from roughly 80% under native selection to 14.64%–21.48% across datasets; the gap narrows as the budget increases. Random-positive selection and RQ-VAE often yield still fewer collisions. Together with the content-alignment results in the main text, this comparison highlights why low collision rates alone are insufficient to assess semantic item representations.

Additional semantic examples. The Beauty feature $f_{14730}$ and Toys feature $f_{13542}$ further connect pretrained descriptions with item content. At $K = 4$, title agreement reaches 92.4% for brushes and 99.6% for LEGO, with lifts of 18.31× and 13.77×, respectively. Their top-activating titles exhibit the same patterns, providing complementary evidence for interpreting the selected features.

Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009.