betaFeedback

WorksComputer Science

X-split Argus: Learning Reusable History Tokens for Efficient CTR Prediction

Abstract

Accurate click-through rate (CTR) prediction depends on user representations that capture both persistent preferences and recent intent, yet processing extensive behavioral histories for every ranking request is costly under production compute and latency constraints. We present X-split ARGUS, a framework for ad ranking whose history compressor and online ranker are trained end to end. The compressor encodes past interactions into a compact set of tokens that are computed offline and reused across candidates, while the online ranker scores ads through cross-attention to these tokens and representations of recent events. Training combines an autoregressive loss for click feedback with distillation of candidate-specific click logits from a larger teacher that scores each candidate using the uncompressed history. The number of compressed history tokens and the length of recent history can be adjusted to fit serving compute and latency budgets. Offline evaluation shows a 0.81% relative reduction in normalized entropy (NE) compared with the production baseline. Deployment in a large-scale real-time bidding system increases online inference cost by 15% relative to the production baseline. An online A/B test against this baseline yields relative gains of 4.65% in a business metric reflecting the monetary value of conversions generated for advertisers and 2.96% in clicks.

Introduction

Accurate click-through rate (CTR) prediction requires user representations that capture persistent preferences, respond to recent intent, and retain behavioral signals relevant to each candidate ad. Longer histories can provide additional evidence for these predictions, and recent sequential models demonstrate the benefits of scaling behavioral context and model capacity [1, 2]. However, encoding extensive histories for each ranking request increases GPU computation and memory traffic, while processing that depends on the candidate can limit reuse across ads. At production request rates, these costs must fit strict compute and response latency budgets, making the representation of historical context a central deployment decision.

Existing systems already address important parts of this problem. TransAct [3] and LLaTTE [4] combine precomputed user representations with online processing of recent behavior, while VISTA [5] learns reusable history summaries for candidate-aware scoring. TM2QK [6] further combines sequence compression with supervision from a teacher that retains the full history. These approaches motivate a concrete question for our deployment: how much of a teacher’s predictive benefit from uncompressed history can be retained when past interactions are represented by a compact token set that does not depend on the candidate? We study this question for a jointly trained compressor and ranker, with explicit control over the history processed online.

We present X-split ARGUS, a framework that uses autoregressive feedback prediction, as in ARGUS [7], while changing the interface between history modeling and candidate scoring. ARGUS uses two-tower fine-tuning to support scoring with precomputed user and item representations. In X-split ARGUS, a history compressor instead produces a set of tokens without conditioning on the candidate, and an online ranker scores ads through cross-attention to these tokens and representations of recent events. The compressor and ranker are trained end to end, while the history tokens are precomputed offline for reuse during serving. Their number and the length of recent history determine the amount of history processed online.

Observed click feedback and click logits for individual candidates from a larger teacher jointly supervise the compressor and online ranker. The teacher scores ads using the uncompressed history. The distillation objective reaches the history compressor through the online ranker, allowing observed feedback and teacher predictions to shape the information retained in the summaries.

For online evaluation, we use Ad Conversion Value (ACV), our primary internal metric for accepting advertising A/B experiments. ACV reflects the monetary value of advertiser conversions delivered at the same advertising budget, capturing the benefit of improved conversion delivery.

Our main contributions are as follows:

  • Joint offline–online history modeling. We introduce a jointly trained history compressor and candidate-aware ranker that combines reusable history tokens with recent events. The number of summary tokens and the length of recent history control the amount of history processed online.

  • Training and evaluation under history compression. We combine autoregressive prediction losses with supervision from the click logits of a teacher using uncompressed history and evaluate the predictive quality retained by the compressed student. In the main offline comparison, X-split ARGUS reduces normalized entropy (NE) by 0.81% relative to Context-aware ARGUS and retains over 70% of the teacher’s improvement over the same baseline.

  • Production deployment and measured impact. We deploy X-split ARGUS on an advertising surface of our real-time bidding system handling roughly a hundred thousand requests per second. The model meets the existing latency budget using 15% more online GPUs of the same type to serve the same traffic. An online A/B test against the production baseline yields relative gains of 4.65% in ACV and 2.96% in clicks.

Related Work

Sequential Modeling in Recommender Systems

Early attention-based sequential recommenders established complementary learning objectives: SASRec [8] predicts the next item using causal self-attention, BERT4Rec [9] reconstructs masked items from bidirectional context, and S³-Rec [10] learns associations among attributes, items, and subsequences through self-supervised pretraining. For CTR prediction, DIN [11] makes history aggregation dependent on the candidate through a local activation mechanism. This distinction between learning a reusable user state and extracting information for a particular candidate remains central to modern ranking architectures.

HSTU [1] scales retrieval and ranking as sequential transduction using pointwise aggregated attention and multiplicative feature transformations. ARGUS [7] combines autoregressive feedback and next-item prediction with impression-aware two-tower fine-tuning for offline representation computation. X-split ARGUS exposes multiple history tokens to an online ranker that conditions its predictions on each candidate. Kunlun [2] addresses compute allocation through hierarchical seed pooling and Computation Skip, which alternates self-attention and summarization modules across layers.

Industrial rankers also differ in how they combine history and nonsequential features. WHALE [12] connects Wukong feature interactions to HSTU sequence states through attention; MTGR [13] retains engineered cross features in candidate tokens within an encoder based on HSTU; and OneTrans [14] processes sequential and nonsequential tokens in a unified backbone. HyFormer [15] alternates attention from global queries to history representations with interactions among the queries. These designs concern the interaction between history and ranking features; the timing and reuse of history computation introduce an additional serving choice.

Sequential modeling also supports generative retrieval, which predicts item identifiers from user histories. TIGER [16] generates semantic item identifiers, while Gryphon [17] shares a history encoder between identifier generation and scoring individual items. These methods illustrate the use of autoregressive history representations across retrieval and ranking; our focus is the prediction of click feedback for supplied candidate ads under online serving constraints.

Efficient Modeling of Long Histories

Systems for long histories differ in what they retain, when they compute it, and how candidates access it. Event selection, representation compression, and computation reuse can coexist within the same architecture. Particularly relevant to our setting, TransAct [3] combines user embeddings computed in batches with a Transformer conditioned on the candidate to process recent actions. LLaTTE [4] scales an asynchronous user model whose cached representations augment a compact online ranker with fresh events and ad context. Both establish the use of separate paths for historical context and recent intent.

VISTA [5] learns summary tokens for downstream candidate attention and caches these tokens for reuse. Its reported variants include joint training with HSTU and an auxiliary objective for sequence reconstruction. SITA [18] organizes stored interest tokens through semantic identifiers and uses each candidate’s identifiers to select tokens before target attention. Our design uses cross-attention from candidates to learned history summaries and representations of recent events. A teacher using uncompressed history supervises the jointly trained compressor and ranker. We study the quality retained by this combination under production constraints.

Systems using retrieval retain selected events or historical embeddings. SIM [19] retrieves a subsequence relevant to the candidate before applying finer target attention, and TWIN [20] aligns the relevance measures of these two stages. TransAct V2 [21] selects nearest neighbors from lifelong, recent, and impression histories while explicitly keeping the freshest actions. Representation Memento [22] retrieves relevant and diverse temporally aggregated embeddings using a modified maximal marginal relevance criterion. These approaches select context according to the candidate, controlling what is passed to the ranker.

Other methods reduce the history representation through aggregation. TWIN V2 [23] uses offline hierarchical clustering and cluster-aware target attention; CAUSE [24] forms history tokens by category while retaining recent events; and LONGER [25] combines local token merging with cross-attention into a shorter sequence. More general attention bottlenecks, including Perceiver’s latent array [26] and the learned queries of BLIP-2’s Q-Former [27], provide architectural precedents for extracting compact representations from larger inputs.

Representation reduction and computation reuse also interact within serving requests. HELIX [28] constructs compressed history states independently of the candidate once per request and lets tokens conditioned on each candidate repeatedly attend to them. This execution strategy differs from computing summaries of long histories offline, as in our deployment. Make It Long, Keep It Fast [29] uses stacked cross-attention from targets to history and batching within each request to share history encoding; OneTrans [14] reuses causal sequence computations; and MARM [30] caches intermediate history states for multilayer target attention. These reuse strategies reduce repeated computation without necessarily imposing a compact summary representation.

Distillation under Serving Constraints

Knowledge distillation supplements observed labels with teacher predictions to train a constrained student [31]. Ranking Distillation [32] transfers a teacher’s top-ranked unobserved items as weighted supervision, while Collaborative Distillation [33] combines soft targets with rank-aware sampling. Privileged Features Distillation [34] addresses a different asymmetry: teachers can use predictive features unavailable to the deployed student. In our setting, historical interactions remain available to the offline compressor; the constraint concerns their representation and processing during online ranking.

TM20K [6] is a close precedent for distillation under history compression: a teacher retains the full sequence, while students merge tokens according to item identity, position, and layer depth. X-split ARGUS learns summary tokens for offline precomputation and reuse alongside recent events. Gryphon-v2 [35] distills scores for generated and logged candidates into both a ranking module and its shared history encoder. Meta Lattice [36] combines logit supervision with asynchronously precomputed teacher embeddings for user–item pairs, which students consume during both training and serving. Our stored summaries do not depend on the candidate; teacher logits for individual candidates supervise their learning through the online ranker. The focus of our evaluation is the predictive quality retained by this representation and training design under production serving constraints.

X-split ARGUS Design

X-split ARGUS combines precomputed historical context with online modeling of recent behavior, while allowing supervision for individual candidates to train both parts end to end. Let H=(e1,…,en)H = (e_1, \ldots, e_n) be a chronological user history with n≤Nn \le N, where et=(xt,yt)e_t = (x_t, y_t) contains item features xtx_t and observed binary click feedback yty_t. The item features comprise an item identifier and tokenized metadata. For a target event at position tt, the model predicts yty_t using xtx_t and information available before that event. We use MM representations of recent events and KK compressed history tokens as the candidate’s historical memory. The student consists of a history compressor, a runtime user encoder, a candidate encoder, and a decoder with cross-attention, as shown in Figure 1. A teacher that processes the history without compression provides additional training supervision. Numerical hyperparameters are reported in Appendix § 7. Throughout this section, [⋅;⋅][\cdot;\cdot] denotes concatenation along the sequence dimension, while [⋅∥⋅][\cdot\mathbin{\Vert}\cdot] denotes concatenation along the feature dimension. We omit model parameter subscripts for readability.

Figure 1. X-split ARGUS architecture: jointly trained history compression and candidate-aware ranking with recent events, supervised click prediction, and knowledge distillation.

Student Architecture and Serving

History compressor. The compressor is a Transformer [37] with causal self-attention over historical events, including their observed feedback. It produces one contextualized state per input event:

(c1,…,cn)=C(H),ci depends only on e1,…,ei.(1)\begin{aligned} (c_1,\ldots,c_n) &= C(H),\\ c_i &\text{ depends only on } e_1,\ldots,e_i. \tag*{(1)} \end{aligned}

For a prefix ending at event bb, where K≤b≤nK \le b \le n, its compressed representation consists of the last KK states of that prefix:

Z(b)=[cb−K+1;…;cb].(2)Z(b) = [c_{b-K+1};\ldots;c_b]. \tag*{(2)}

Selecting the final KK contextualized states gives a compact history representation while allowing compressed representations for multiple training cutoffs to be extracted from a single causal pass. Each retained state carries information from its causal prefix, and the final state can attend to every event up to bb. The student prediction objectives train these states to preserve information useful for subsequent candidate scoring. Their computation does not depend on the candidates scored against them.

Runtime user encoder. The runtime encoder processes recent events and their observed feedback using causal self-attention with a sliding window. The window at each layer controls the temporal receptive field of the output states; stacked layers with local attention can propagate information beyond a single layer’s window [38]. For a serving request, let HrecentH^{\mathrm{recent}} contain up to the most recent MM events available in the user profile. The online representations are

U=E(Hrecent).(3)U = E(H^{\mathrm{recent}}). \tag*{(3)}

The encoder uses neither candidate features nor decoder outputs. The use of a shared sequence pass during training is described in § 3.3.

Candidate encoder and decoder. The candidate encoder maps an ad’s item identifier and metadata to a single item token qa=A(xa)q_a = A(x_a), which excludes the candidate’s click outcome. The student decoder receives this token without introducing a separate feedback token. It conditions the item representation on both sources of user context through cross-attention, with candidate states supplying queries and history states supplying keys and values [37]. Its historical memory is

V=[Z(b);U],(4)V = [Z(b); U], \tag*{(4)}

where bb is the cutoff of the cached history representation available for the request. The two history streams have the same feature dimension for this concatenation. Thus, the compressed tokens join the states of recent events after the runtime encoder, and the decoder attends to K+MK + M history states for a full window.

The optional decoder self-attention (SA) operates only over these K+MK + M history states, using full attention within the historical memory. Candidate tokens do not participate in this operation, so the resulting history representations remain independent of the candidates being scored. All states in the memory precede the prediction target; allowing them to attend to one another does not expose the target’s feedback or future events. Candidate scores remain independent when evaluated against the same memory.

As shown in Figure 1, a second concatenation combines the decoder output da=D(qa,V)d_a = D(q_a, V) with the candidate representation along the feature dimension, yielding ra=[da∥qa]r_a = [d_a \mathbin{\|} q_a]. This gives both prediction heads access to the representation conditioned on user history and the original candidate representation. The supervised head produces the serving CTR prediction.

Computation reuse. The architecture supports three scopes of execution (Figure 1). The compressor outputs are precomputed and cached across requests. Within each request, the runtime encoder and any self-attention over the decoder’s historical memory are independent of the candidates and can be shared across them. Candidate encoding, cross-attention to this memory, and the prediction heads provide computation for individual candidates. The absence of a dependency from candidate scoring back to either history encoder enables this reuse. The budgets KK and MM control the decoder memory size, while MM also controls how many raw events are encoded online.

Snapshot and recent history boundaries. For a request with nn available historical events, let s=max⁡(0,n−M)s = \max(0, n - M). The runtime input contains events s+1,…,ns + 1, \ldots, n, whereas the cached compressor states summarize a prefix ending at b≤nb \le n. These boundaries need not coincide: b=sb = s gives adjacent coverage, b<sb < s leaves events b+1,…,sb + 1, \ldots, s outside both branches, and b>sb > s represents events s+1,…,bs + 1, \ldots, b in both branches. This distinction concerns which events are available to the two branches; it is separate from the difference in the runtime encoder’s receptive field between training and serving, described in § 3.3.

Teacher with Uncompressed History

The teacher is a larger Transformer that processes the retained event history HH with full causal self-attention, without compressing it into summary states. For a target at position tt, it uses the current item features xtx_t and the same retained events preceding tt; its access to these events is not restricted to the student’s KK compressed states and MM recent states. It follows the autoregressive formulation of feedback prediction used in HSTU and ARGUS [1, 7]: item information precedes its observed feedback, allowing the model to predict the latter from the former and the preceding history. In our teacher, each target event is represented by an item token followed by a feedback token. The item token contains the item features but excludes yty_t; the feedback token carries the observed click outcome. This separation into two tokens is specific to the teacher. The student decoder uses the single item token described above, while both student history encoders receive observed feedback as an event feature.

Let H~\widetilde{H} denote this token sequence and π(t)\pi(t) the position of the item token for target event tt. The teacher’s click logit is

(v1T,…,v∣H~∣T)=T(H~),ztT=gT(vπ(t)T).(5)\begin{aligned} (v_1^T,\ldots,v_{\lvert\widetilde{H}\rvert}^T) &= T(\widetilde{H}), \\ z_t^T &= g_T(v_{\pi(t)}^T). \tag*{(5)} \end{aligned}

Here vjTv_j^T is the teacher state at token position jj, gTg_T is its click prediction head, and the superscript TT identifies teacher quantities. The feedback token for event tt occurs after π(t)\pi(t) and is unavailable to its own click prediction. Once present in the prefix, the event and its feedback can contribute to subsequent predictions. Teacher and student outputs are aligned by target event rather than by token index, since their sequence representations differ. The teacher is trained separately from the student, and its checkpoint provides fixed logit targets during student updates. No student gradient updates the teacher, which is absent from the online scoring path.

End-to-End Autoregressive Training

Sequence construction and temporal boundaries. We train on complete retained user histories. Let T(H)\mathcal{T}(H) denote the set of event positions selected for click supervision in history HH. For a target event t∈T(H)t \in\mathcal{T}(H), the student decoder receives an item token computed from xtx_t, and yty_t serves only as the prediction target. Historical events supplied to the compressor and runtime encoder include their observed feedback, but the decoder memory for event tt is assembled exclusively from states preceding tt.

For the default construction at target position tt, the boundary separating the selected recent events from the older prefix is

b(t)=max⁡(0,t−1−M).(6)b(t) = \max(0,t-1-M). \tag*{(6)}

This training boundary is defined for each target; the cached cutoff bb used at serving time is determined by the available offline snapshot and need not follow this rule. Both history encoders process the sequence in a single causal pass. Writing (u1,…,un)=E(H)(u_1,\ldots,u_n)=E(H), for a full memory window with b(t)≥Kb(t)\ge K the target’s decoder memory is assembled as

Ut=[ub(t)+1;…;ut−1],U_t = [u_{b(t)+1};\ldots;u_{t-1}],
Vt=[Z(b(t));Ut].(7)V_t = [Z(b(t));U_t]. \tag*{(7)}

All selected states precede the target event. Consequently, neither its feedback nor future events enter its click prediction through the history branches. The input event sequence is shared across prediction positions, avoiding a separate pass over the historical prefix for every target. Each supervised event contributes a prediction target at its own position and historical context at later positions. In the history budget experiments of § 4.4, we hold the boundary at its reference value while varying the numbers of retained compressor and runtime states.

Runtime context during training and serving. Independently of the cached prefix boundary, the runtime encoder’s shared training pass and its truncated serving input have different left boundaries. During training, early states within the selected recent block can incorporate older events through local attention. During serving, only the most recent MM raw events are supplied to the encoder, truncating this preceding context. Thus, MM specifies the selected runtime states during training and the raw online input length during serving; it does not impose an identical total receptive field in both modes.

Click prediction. For target event tt, let qt=A(xt)q_t = A(x_t) and dt=D(qt,Vt)d_t = D(q_t, V_t). The two prediction heads receive the shared representation rt=[dt∥qt]r_t = [d_t \mathbin{\|} q_t] shown in Figure 1. The supervised head produces ztGT=gGT(rt)z_t^{\mathrm{GT}} = g_{\mathrm{GT}}(r_t) and pt=σ(ztGT)p_t = \sigma(z_t^{\mathrm{GT}}), where σ\sigma is the logistic sigmoid. The binary cross-entropy for this event is

ℓGT,t=−ytlog⁡pt−(1−yt)log⁡(1−pt).(8)\ell_{\mathrm{GT},t} = -y_t \log p_t - (1-y_t)\log(1-p_t). \tag*{(8)}

Optional smoothing of negative targets. Some configurations regularize the supervised objective with asymmetric label smoothing [39, 40]. The training target for an impression without a click is raised from zero to ε∈[0,1)\varepsilon\in[0,1), while an observed click retains a target of one:

y~t=yt+ε(1−yt).(9)\widetilde{y}_t = y_t + \varepsilon(1-y_t). \tag*{(9)}

For these configurations, y~t\widetilde{y}_t replaces yty_t in Equation (8). Smoothing applies only to the supervised loss and leaves the teacher logits used for distillation unchanged. The value of ε\varepsilon is reported in Appendix § 7.

This regularization is motivated by the limited evidence about user interest provided by an impression without a click: the user may not have seen the ad. An unclicked impression remains a valid negative outcome for CTR prediction; smoothing softens the training objective without treating that observation as an incorrect label. We evaluate this optional training choice in § 4.4.

Knowledge distillation. Teacher supervision supplements observed clicks through knowledge distillation [31]. We use separate supervised and distillation heads over the same student representation, an arrangement also explored in distillation for industrial sequence models [6]. The auxiliary head produces ztKD=gKD(rt)z_t^{\mathrm{KD}} = g_{\mathrm{KD}}(r_t) and matches the teacher’s raw click logit using an L1L_1 objective:

ℓKD,t=∣ztKD−stopgrad⁡(ztT)∣.(10)\ell_{\mathrm{KD},t} = \left|z_t^{\mathrm{KD}} - \operatorname{stopgrad}\left(z_t^{\mathrm{T}}\right)\right|. \tag*{(10)}

The L1L_1 loss penalizes discrepancies between raw student and teacher logits linearly. The supervised head is trained with the click prediction loss, while the auxiliary distillation head is trained to match teacher logits. Both losses update the shared student components, allowing teacher supervision to shape the representations used by the serving head. At serving time, predictions are obtained exclusively from gGTg_{\mathrm{GT}}; the distillation head is used only during training.

Joint objective and parameter updates. Let LGT\mathcal{L}_{\mathrm{GT}} and LKD\mathcal{L}_{\mathrm{KD}} denote the corresponding aggregate losses over positions in T(H)\mathcal{T}(H). The combined click supervision and distillation objective is

L=LGT+λKDLKD.(11)\mathcal{L} = \mathcal{L}_{\mathrm{GT}} + \lambda_{\mathrm{KD}}\mathcal{L}_{\mathrm{KD}}. \tag*{(11)}

The nonnegative weight λKD\lambda_{\mathrm{KD}} controls the contribution of distillation and is specified in Appendix A. For each training sequence, we obtain teacher logits for the target events, compute both student history streams, select the prefix and recent states for each target, and evaluate the decoder objectives. The student components and prediction heads are optimized jointly. In particular, gradients of the student prediction objectives propagate through the decoder into both the runtime encoder and the history compressor. Teacher outputs remain detached during these updates. Offline precomputation is therefore an execution choice for serving, while the compressor remains part of the differentiable student during training.

Offline Evaluation

Experimental Setup and Metrics

Dataset and global temporal split. We use a large internal industrial dataset containing all data collected by our real-time bidding system. Table 1 summarizes the dataset, retained user histories, and temporal split. We use the same training and test time boundaries across users. The gap between the two periods emulates the delay between the completion of model training and the model’s actual deployment.

ParameterValue
SourceInternal RTB logs
Data and history horizon365 days
Training unitUser history
Training targets (including repeats)∼109\sim10^{9}
History cap8,192 events
TargetBinary click feedback
SplitGlobal temporal
Train–test gap1 day
Test period3 days

Table 1. Dataset and temporal split. History lengths count original events; the cap corresponds to the 99.5th percentile of user event counts.

Prediction context and model checkpoints. Each model’s checkpoint remains fixed throughout the test period, while both compressed history and runtime history are updated as evaluation progresses. Context uses events available before the impression being scored, including events from the gap and the elapsed portion of the test period. For X-split ARGUS predictions, the runtime encoder is recomputed on the last MM raw events preceding the target. Its states are obtained from the restricted recent input, rather than selected from a shared pass over the full history. All configurations are evaluated on the same set of target impressions.

Evaluated configurations. We compare X-split ARGUS with Context-aware ARGUS, described in § 4.2, and the uncompressed teacher from § 3.2. The X-split ARGUS configuration in this comparison uses target attention, knowledge distillation, self-attention over the decoder’s historical memory, and negative target smoothing. The deployed configuration and its serving costs are discussed in § 5.

The teacher provides a quality reference using the retained lifetime history without compression. Serving it would require processing extensive event sequences and keeping those sequences accessible in runtime user profiles. The resulting inference workload and profile storage costs make this configuration impractical under our production resource and latency constraints. X-split ARGUS uses precomputed history states and a bounded recent history during online ranking.

Normalized entropy. We use normalized entropy (NE), a standard metric for assessing the quality of predicted click probabilities [41]. Let E\mathcal{E} denote the evaluated impressions, yiy_i the observed binary click feedback, and ℓi\ell_i the binary cross-entropy of the predicted probability for impression ii. Sampling correction is applied through impression weights wiw_i, which are used in both the test log loss and the background CTR p0p_0:

Ltest=∑i∈Ewiℓi∑i∈Ewi,p0=∑i∈Ewiyi∑i∈Ewi.(12)\begin{aligned} \mathcal{L}_{\mathrm{test}} &= \frac{\sum_{i\in\mathcal{E}} w_i\ell_i}{\sum_{i\in\mathcal{E}} w_i},\\ p_0 &= \frac{\sum_{i\in\mathcal{E}} w_i y_i}{\sum_{i\in\mathcal{E}} w_i}. \tag*{(12)} \end{aligned}

Both quantities are computed over the entire test set. NE normalizes the weighted log loss by the binary entropy of this background CTR:

NE=Ltest−p0log⁡p0−(1−p0)log⁡(1−p0).(13)\mathrm{NE} = \frac{\mathcal{L}_{\mathrm{test}}}{-p_0\log p_0-(1-p_0)\log(1-p_0)}. \tag*{(13)}

Lower NE indicates better prediction quality. For a model with NE vv and a reference with NE vrefv_{\mathrm{ref}}, we report the relative reduction

ΔNE(v;vref)=100vref−vvref%.(14)\Delta\mathrm{NE}(v;v_{\mathrm{ref}})=100\frac{v_{\mathrm{ref}}-v}{v_{\mathrm{ref}}}\%. \tag*{(14)}

A positive reduction denotes an improvement. Each table explicitly identifies its reference. NE is displayed to four decimal places; relative reductions and retained teacher gains are computed from unrounded values. On this test set, a relative NE change of 0.1% is statistically significant.

Baseline

Context-aware ARGUS is a two-tower ranker based on ARGUS [7] whose user tower contains a history compressor and a runtime encoder, while its item tower represents the candidate ad. It scores ads with a dot product between the two tower outputs and therefore already models both historical context and recent behavior. The Context-aware variant separates each event into context and item components. This separation allows the runtime user encoder to receive all available contextual information when scoring the corresponding item. At serving time, context enters the user encoder while the candidate item is represented by the item tower. The baseline therefore combines historical behavior with the context available at ranking time.

Comparison with the Baseline and Teacher

Table 2 shows that X-split ARGUS improves NE over Context-aware ARGUS, while the uncompressed teacher achieves the lowest NE. The result supports the combined design of target attention and teacher supervision for ranking with compressed history.

ModelNE ↓\downarrowΔ\DeltaNE (%) ↑\uparrowρ\rho (%) ↑\uparrow
Context-aware ARGUS0.72040.000—
X-split ARGUS0.7146+0.81471.6
Teacher0.7123+1.136100.0

Table 2. Overall comparison of complete model configurations on the common RTB test set. Relative reductions use Context-aware ARGUS as the reference. ρ\rho is the share of the teacher’s NE improvement retained, defined in Equation 15.

ConfigurationNE ↓\downarrowΔ\DeltaNE (%) ↑\uparrow
X-split ARGUS baseline0.71550.000
Compressed history tokens
K=2K = 20.7168−0.193
K=32K = 320.7149+0.075
Runtime history states
M=8M = 80.7179−0.342
M=64M = 640.7157−0.029
Layers in each history encoder
L=8L = 80.7148+0.091
L=10L = 100.7137+0.245

Table 3. Configuration sensitivity relative to the X-split ARGUS baseline defined in Table 7 in Appendix A. The history split is fixed across token budget experiments; the runtime attention window at each layer is fixed across depth experiments. Negative reductions denote higher NE than the ablation baseline.

To quantify how much of the teacher’s improvement a student retains, we compare their gains over the same Context-aware ARGUS baseline:

ρ(v)=100vbase−vvbase−vT%,(15)\rho(v) = 100 \frac{v_{\mathrm{base}} - v}{v_{\mathrm{base}} - v_{\mathrm{T}}}\%, \tag*{(15)}

where vbasev_{\mathrm{base}} and vTv_{\mathrm{T}} are the NE of the baseline and teacher, respectively. X-split ARGUS retains more than 70% of the teacher’s improvement. This shows that substantial predictive benefit can be retained in a student that uses compact history representations and does not execute the teacher using the full history during serving.

Ablation Study

To complement the overall comparison, we study how the X-split ARGUS baseline responds to changes in compressed and recent context, encoder depth, and negative training targets. Table 3 reports changes to the compressed history budget KK, runtime history budget MM, and encoder depth LL relative to this ablation baseline. In the history budget experiments, the split between the compressed prefix and recent history is fixed at its reference value: we vary KK or MM independently, without moving the split or changing other hyperparameters. In the depth experiments, both the compressor and the runtime encoder have LL layers, while the runtime attention window at each layer remains fixed. The smoothing comparison holds K=2K = 2 in both configurations.

Compressed history tokens. We compare compressed token budgets of K=2K = 2, 8, and 32. Increasing the number of compressor states available to candidate attention improves prediction quality across these settings. Reducing the budget below the ablation baseline causes a larger quality loss than the gain from increasing it above the baseline.

Runtime history tokens. We compare runtime history budgets of M=8M = 8, 64, and 256 events. For each target and budget, we recompute the runtime encoder on the last MM raw events and pass their output states to the decoder. Earlier events are excluded from this encoder pass, while the compressed prefix remains the same across configurations. The smallest budget produces the largest loss in prediction quality, while the intermediate budget is closer to the ablation baseline. Retaining more recent events in the runtime branch therefore improves prediction quality with the same compressed historical context.

Encoder depth. We increase both history encoders from six to eight and ten layers. Deeper encoders improve prediction quality across the tested configurations without increasing the number of history tokens exposed to the decoder. With a fixed window at each layer, additional layers can also expand the context propagated through the runtime encoder stack. The experiment therefore measures the combined effect of depth in both branches, without separating their individual contributions or the effects of capacity and context propagation.

Smoothing negative targets. We test whether regularizing negative training targets with the asymmetric smoothing defined in § 3.3 improves prediction quality. The paired comparison uses the same K=2K = 2 configuration with binary or smoothed targets, keeping all other settings unchanged.

Table 4 shows that smoothing produces a statistically significant improvement in prediction quality over binary supervision without changing training or online inference cost.

TargetsNE ↓ΔNE (%) vs. binary
Binary0.71680.000
Smoothed negatives0.7155+0.179

Table 4. Negative target smoothing at fixed K=2K = 2. Relative NE reductions use the configuration with binary targets as the reference.

Production Deployment

We deploy X-split ARGUS on the largest advertising surface of our real-time bidding system. This section reports its online impact, the resources required to serve it, and the processes used to refresh historical representations and model parameters. The deployed model configuration is listed in Table 6 in Appendix § 7.

HyperparameterValue
Maximum history horizon1 year
Maximum history length NN8192
Recent history budget MM240
Compressed history tokens KK8
Compressor layers6
Compressor hidden dimension1024
Runtime encoder layers LrL_{\mathrm{r}}6
Runtime attention window ww40
Runtime encoder hidden dimension1024
Decoder layers2
Decoder hidden dimension256
Distillation weight λKD\lambda_{\mathrm{KD}}0.15
Smoothed negative target ε\varepsilon0.01

Table 6. Model and training hyperparameters.

Online Experiment and Deployment Impact

The surface accounts for approximately 50% of total RTB traffic and handles roughly a hundred thousand requests per second. We conducted an A/B test lasting seven days against the prior production ranker, Context-aware ARGUS (§ 4.2). Users were assigned to treatment and control by hashing their identifiers with an experiment salt, keeping assignment consistent across requests. Each group received 6% of the surface’s traffic; the request rate above refers to the entire surface.

Table 5 reports relative changes against the control. X-split ARGUS increases ACV by 4.65% and the number of clicks by 2.96%. ACV is our primary internal metric for advertising A/B acceptance and captures the monetary value of advertiser conversions delivered at the same advertising budget. Both improvements are statistically significant (p<0.001p < 0.001).

MetricRelative change
ACV+4.65%
Number of clicks+2.96%

Table 5. Online A/B results relative to the Context-aware ARGUS production baseline. Each group received 6% of the evaluated surface’s traffic during the test, which lasted seven days.

A reverse A/B test conducted one month after deployment confirmed the gains.

At this request volume, deployment must satisfy both the latency budget imposed by user response requirements and the throughput supported by the available GPUs. The ML service has a latency budget in the hundreds of milliseconds. X-split ARGUS meets the same budget as Context-aware ARGUS while requiring 15% more online GPUs of the same type to serve the same traffic volume. The computation reuse described in Section 3.1 keeps long history processing outside the request path and shares recent history encoding across candidates. The online gains are thus achieved within the existing latency constraint, with a measured increase in online GPU capacity. Precomputation, storage, and model updates incur additional resources, described below.

History Precomputation and Storage

A daily batch pipeline collects new user events, appends them to the retained user profiles, and runs the history compressor for users with new events. For each such user, it publishes KK contextualized history states to an embedding store. The online ranker retrieves these states and combines them with representations of recent events, as described in Section 3.1. Historical representations are therefore reused across requests, while recent behavior remains available through the runtime user profile.

We store the compressor states in BF16. For a stored state dimension dcd_c, the uncompressed embedding payload per user is

Suser=2Kdc bytes.(16)S_{\mathrm{user}} = 2K d_c \text{ bytes}. \tag*{(16)}

With standard storage compression, the database of precomputed embeddings occupies hundreds of gigabytes. This footprint refers to the computed embeddings, separately from the retained event histories used by the batch pipeline. The GPU compute required for daily precomputation is small relative to continuous online inference; storing the resulting representations is an additional infrastructure requirement.

Model Freshness and Daily Fine-Tuning

To inform the update schedule, we evaluate a fixed X-split ARGUS checkpoint on test windows spanning one day, with gaps of zero to four days between the latest training event and the start of each window. User histories and precomputed compressor embeddings are refreshed using events available before each prediction. Figure 2 shows the change in relative NE reduction over the production system evaluated on each test day, with the value at a gap of one day set to zero. This advantage decreases approximately linearly, by about 0.05 percentage points per day, despite the refreshed historical context.

Figure 2. Relative prediction quality deteriorates as the gap between the latest training event and the test day increases. Histories and compressor embeddings are refreshed while model weights remain fixed.

Based on this observation, we fine-tune the deployed student daily using prediction targets from the most recent three days. Data preparation, training, and deployment yield a gap of approximately one to two days between the latest training events and serving predictions. Daily fine-tuning requires additional GPU compute, but this workload is small relative to continuous online inference. The larger improvement with no gap suggests that more frequent or online model updates could yield further gains.

Conclusion

We presented X-split ARGUS, a framework for CTR prediction that combines compressed historical context with recent user behavior and candidate scoring through cross-attention. Its history compressor and online ranker are trained end to end using observed click feedback and distillation from a teacher that processes the uncompressed history. Precomputed compressor states are reused across requests, while runtime history representations are shared across candidates, keeping extensive history processing outside the online ranking path. In the offline comparison, the compressed student improves normalized entropy over Context-aware ARGUS and retains more than 70% of the uncompressed teacher’s improvement over the same baseline.

Deployment on the largest advertising surface of our RTB system confirms the practical value of this design. An online A/B test against the production baseline yields relative gains of 4.65% in ACV and 2.96% in clicks. X-split ARGUS serves the same traffic volume within the existing latency budget using 15% more online GPUs of the same type. Daily history precomputation and model fine-tuning support continued operation, with the latter motivated by the observed decline in the relative advantage of a fixed checkpoint. These results demonstrate that reusable history compression and candidate attention can bring the benefits of richer user context to production ranking within practical latency and resource constraints.

X-split ARGUS Configuration

Table 6 reports the model configuration and loss weights used in the deployment described in this paper. Sequence lengths count original events, before the teacher represents target events as separate item and feedback tokens. The maximum history length corresponds to the 99.5th percentile of the distribution of event counts. The deployed runtime encoder uses a window parameter w=40w = 40 at each of its Lr=6L_{\mathrm{r}} = 6 layers and a configured recent history budget M=wLr=240M = wL_{\mathrm{r}} = 240.

Table 7 defines the X-split ARGUS baseline used for the offline ablation study. The X-split ARGUS configuration in the main offline comparison additionally enables self-attention over history and negative target smoothing.

SettingValue
Compressed history tokens KK8
Recent history budget MM256
Compressor layers6
Runtime encoder layers6
Decoder self-attention over historyDisabled
Supervised targetsBinary
Knowledge distillationEnabled

Table 7. X-split ARGUS baseline settings for the offline ablation study.

For the X-split ARGUS baseline, the runtime encoder uses a window parameter at each layer of w=⌊S/Lr⌋w = \left\lfloor S/L_{\mathrm{r}}\right\rfloor, where SS is the configured visibility budget (seqlen). In the depth experiments of § 4.4, this window remains fixed as the number of layers increases; it is not recomputed for each depth. The window bounds the context propagated through the encoder stack. It does not make the last MM states of a full training sequence identical to states recomputed from only the last MM raw events during serving, as discussed in § 3.3.

References

  1. [1]Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Jiayuan He, Yinghai Lu, and Yu Shi. Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 58484–58509. PMLR, 2024. URL https://proceedings.mlr.press/v235/zhai24a.html.arxiv.org/abs/2402.17152
  2. [2]Bojian Hou, Xiaolong Liu, Xiaoyi Liu, Jiaqi Xu, Yasmine Badr, Mengyue Hang, Sudhanshu Chanpuriya, Junqing Zhou, Yuhang Yang, Han Xu, Quiling Suo, Laming Chen, Yuxi Hu, Jiasheng Zhang, Huaqing Xiong, Yuzhen Huang, Chao Chen, Yue Dong, Yi Yang, Shuo Chang, Xiaorui Gan, Wenlin Chen, Santanu Kolay, Darren Liu, Jade Nie, Chunzhi Yang, Ellie Wen, Jiyan Yang, and Huayu Li. Kunlun: Establishing scaling laws for massive-scale recommendation systems through unified architecture design, 2026. URL https://arxiv.org/abs/2602.10016.
  3. [3]Xue Xia, Pong Eksombatchai, Nikil Pancha, Dhruvil Deven Badani, Po-Wei Wang, Neng Gu, Saurabh Vishwas Joshi, Nazanin Farahpour, Zhiyuan Zhang, and Andrew Zhai. TransAct: Transformer-based realtime user action model for recommendation at Pinterest. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’23, pages 5249–5259. Association for Computing Machinery, 2023. doi: 10.1145/3580305.3599918. URL https://doi.org/10.1145/3580305.3599918.
  4. [4]Lee Xiong, Zhirong Chen, Rahul Mayuranath, Shangran Qiu, Arda Ozdemir, Lu Li, Yang Hu, Dave Li, Jingtao Ren, Howard Cheng, Fabian Souto Herrera, Ahmed Agiza, Baruch Epstein, Anju Aggarwal, Julia Ulziisaiikhan, Chao Wang, Dinesh Ramasamy, Parshva Doshi, Sri Reddy, and Arnold Overwijk. LaTTe: Scaling laws for multi-stage sequence modeling in large-scale ads recommendation, 2026. URL https://arxiv.org/abs/2601.20083.
  5. [5]Zhimin Chen, Chenyu Zhao, Ka Mo, Yunj iang Jiang, Jane Lee, Khushhall Chandra Mahajan, Ning Jiang, Kai Ren, Charlie Li, and Wen-Yun Yang. Massive memorization with hundreds of trillions of parameters for sequential transducer generative recommenders. In International Conference on Learning Representations, 2026. URL https://proceedings.iclr.cc/paper_files/paper/2026/hash/41bba7b0f5c81e789a20bb16a370aeeb-Abstract-Conference.html.arxiv.org/abs/2510.22049
  6. [6]Xinchun Li, Duoru Zheng, Wenlin Zhao, Haoran Ding, Ziyi Zhou, Jingxuan Tan, Huizhi Yang, Yuchen Jiang, Zhe Chen, Yuchao Zheng, Linlan Chen, Dongjian Wang, Dongyue Wang, Xiaosong Li, Hongyue Mao, and Yaocheng Tan. Teacher retains full tokens, student merges efficiently: TM20K for e-commerce sequence modeling in ad recommendation, 2026. URL https://arxiv.org/abs/2608.07055.
  7. [7]Kirill Khrylchenko, Artem Matveev, Sergei Makeev, and Vladimir Baikalov. Scaling recommender transformers to one billion parameters. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, pages 2255–2265. Association for Computing Machinery, 2026. doi: 10.1145/3770854.3783916. URL https://doi.org/10.1145/3770854.3783916.
  8. [8]Wang-Cheng Kang and Julian McAuley. Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining (ICDM), pages 197–206. IEEE, 2018. doi: 10.1109/ICDM.2018.00035. URL https://doi.org/10.1109/ICDM.2018.00035.
  9. [9]Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM ’19, pages 1441–1450. Association for Computing Machinery, 2019. doi: 10.1145/3357384.3357895. URL https://doi.org/10.1145/3357384.3357895.
  10. [10]Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. S³-Rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM ’20, pages 1893–1902. Association for Computing Machinery, 2020. doi: 10.1145/3340531.3411954. URL https://doi.org/10.1145/3340531.3411954.
  11. [11]Guorui Zhou, Xiaoqiang Zhu, Chengru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’18, pages 1059–1068. Association for Computing Machinery, 2018. doi: 10.1145/3219819.3219823. URL https://doi.org/10.1145/3219819.3219823.
  12. [12]Renqin Cai, Dawei Sun, Yuanjun Yao, Zhiyong Wang, Yelvin Fu, Maggie Zhuang, Yu Shi, Zhongnan Fang, Xuan Cao, Jing Qian, and Rui Li. WHALE: A scalable unified model for recommendation with Wukong-HSTU architecture, 2026. URL https://arxiv.org/abs/2607.17017.
  13. [13]Ruidong Han, Bin Yin, Shangyu Chen, He Jiang, Fei Jiang, Xiang Li, Chi Ma, Mincong Huang, Xiaoguang Li, Chunzhen Jing, Yueming Han, MengLei Zhou, Lei Yu, Chuan Liu, and Wei Lin. MTGR: Industrial-scale generative recommendation framework in Meituan. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, CIKM ’25, pages 5731–5738. Association for Computing Machinery, 2025. doi: 10.1145/3746252.3761565. URL https://doi.org/10.1145/3746252.3761565.
  14. [14]Zhaoqi Zhang, Haolei Pei, Jun Guo, Tianyu Wang, Yufei Feng, Hui Sun, Shaowei Liu, and Aixin Sun. OneTrans: Unified feature interaction and sequence modeling with one transformer in industrial recommender. In Proceedings of the ACM Web Conference 2026, WWW ’26, pages 8162–8170. Association for Computing Machinery, 2026. doi: 10.1145/3774904.3792838. URL https://doi.org/10.1145/3774904.3792838.
  15. [15]Yunwen Huang, Shiyong Hong, Xijun Xiao, Jinqiu Jin, Xuanyuan Luo, Zhe Wang, Zheng Chai, Shikang Wu, Yuchao Zheng, and Jingjian Lin. HyFormer: Revisiting the roles of sequence modeling and feature interaction in CTR prediction, 2026. URL https://arxiv.org/abs/2601.12681.
  16. [16]Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Tran, Jonah Samost, Maciej Kula, Ed Chi, and Maheswaran Sathiamoorthy. Recommender systems with generative retrieval. In Advances in Neural Information Processing Systems, volume 36, pages 10299–10315. Curran Associates, Inc., 2023. doi: 10. 52202/075280-0452. URL https://proceedings.neurips.cc/paper/2023/hash/20dcab0f14046a5c6b02b61da9f13229-Abstract.html.arxiv.org/abs/2305.05065
  17. [17]Daria Tikhonovich, Oleg Sorokin, Vladislav Dodonov, Mariia Ulianova, and Ilya Murzin. Gryphon: A unified architecture for Semantic-ID generation and item-level scoring in industrial recommendations, 2026. URL https://arxiv.org/abs/2606.08604.
  18. [18]Rui Zhou, Bo Chen, Qinglin Jia, Jiezho​u Ji, Chaoyi Ma, Ruiming Tang, Hao Wang, and Enhong Chen. SITA: Semantic interest tokens for target-aware compression in long-sequence recommendation, 2026. URL https://arxiv.org/abs/2608.03692.
  19. [19]Qi Pi, Guorui Zhou, Yuijing Zhang, Zhe Wang, Lejian Ren, Ying Fan, Xiaoqiang Zhu, and Kun Gai. Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM ’20, pages 2685–2692. Association for Computing Machinery, 2020. doi: 10.1145/3340531.3412744. URL https://doi.org/10.1145/3340531.3412744.
  20. [20]Jianxin Chang, Chenbin Zhang, Zhiyi Fu, Xiaoxue Zhang, Lin Guan, Jing Lu, Yiqun Hui, Dewei Leng, Yanan Niu, Yang Song, and Kun Gai. TWIN: TWo-stage interest network for lifelong user behavior modeling in CTR prediction at Kuaishou. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’23, pages 3785–3794. Association for Computing Machinery, 2023. doi: 10.1145/3580305.3599922. URL https://doi.org/10.1145/3580305.3599922.
  21. [21]Xue Xia, Saurabh Joshi, Kousik Rajesh, Kangnan Li, Yangyi Lu, Nikil Pancha, Dhruvil Badani, Jia​jing Xu, and Pong Eksombatchai. TransAct V2: Lifelong user action sequence modeling on Pinterest recommendation. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, CIKM ’25, pages 6881–6882. Association for Computing Machinery, 2025. doi: 10.1145/3746252.3761433. URL https://doi.org/10.1145/3746252.3761433.
  22. [22]Xiaoyu Chen, Ruichen Wang, Jieming Di, Suofei Feng, Nafis Abrar, Lilly Kumari, Tony Tsui, Yilin Liu, Yu Lu, Sowmya Patapati, Junwei Xiong, Qiao Yang, Dorothy Sun, Yang Cao, Victor Chen, Pan Chen, Ramsundar Sundarkumar, Shivendra Pratap Singh, Arnold Overwijk, Ling Leng, Dinesh Ramasamy, Sri Reddy, Robert Malkin, and Sandeep Pandey. Memento: Personalized RAG-style long-retention data scaling for META ads recommendation, 2026. URL https://arxiv.org/abs/2605.24051.
  23. [23]Zihua Si, Lin Guan, Zhongxiang Sun, Xiaoxue Zang, Jing Lu, Yiqun Hui, Xingchao Cao, Zeyu Yang, Yichen Zheng, Dewei Leng, Kai Zheng, Chenbin Zhang, Yanan Niu, Yang Song, and Kun Gai. TWIN V2: Scaling ultra-long user behavior sequence modeling for enhanced CTR prediction at Kuaishou. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, CIKM ’24, pages 4890–4897. Association for Computing Machinery, 2024. doi: 10.1145/3627673.3680030. URL https://doi.org/10.1145/3627673.3680030.
  24. [24]Qijiong Liu, Lu Fan, Zhongzhou Liu, Xiaoyu Dong, Yuankai Luo, Guoyuan An, Nuo Chen, Wei Guo, Yong Liu, and Xiao-Ming Wu. Accelerating generative recommendation via simple categorical user sequence compression. In Proceedings of the Nineteenth ACM International Conference on Web Search and Data Mining, WSDM ’26, pages 1195–1199. Association for Computing Machinery, 2026. doi: 10.1145/3773966.3779399. URL https://doi.org/10.1145/3773966.3779399.
  25. [25]Zheng Chai, Qin Ren, Xijun Xiao, Huizhi Yang, Bo Han, Sijun Zhang, Di Chen, Hui Lu, Wenlin Zhao, Lele Yu, Xionghang Xie, Shiru Ren, Xiang Sun, Yaocheng Tan, Peng Xu, Yuchao Zheng, and Di Wu. LONGER: Scaling up long sequence modeling in industrial recommenders. In Proceedings of the Nineteenth ACM Conference on Recommender Systems, RecSys ’25, pages 247–256. Association for Computing Machinery, 2025. doi: 10.1145/3705328.3748065. URL https://doi.org/10.1145/3705328.3748065.
  26. [26]Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira. Perceiver: General perception with iterative attention. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 4651–4664. PMLR, 2021. URL https://proceedings.mlr.press/v139/jaegle21a.html.arxiv.org/abs/2103.03206
  27. [27]Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 19730–19742. PMLR, 2023. URL https://proceedings.mlr.press/v202/li23q.html.arxiv.org/abs/2301.12597
  28. [28]Yuntao Zheng, Miao Zhang, Yadong Ding, Yanchuan Tang, Liyixu Chen, Hao Wang, Quan Li, Shiying Cai, Yue Lin, Jiayu Li, Yu Feng, Wentao Yang, Rongkun Xing, Jiekai Wang, Mingge Zhang, Feiling Gong, Xiang Gao, Jinyu Dong, Yajing Zhang, Pengfei Ren, and Yinzhou Wang. HELIX: Purified and unified – rethinking feature interaction and sequence modeling for large-scale recommendation, 2026. URL https://arxiv.org/abs/2609.37183.
  29. [29]Lin Guan, Jia-Qi Yang, Zhishan Zhao, Beichuan Zhang, Bo Sun, Xuanyuan Liao, Jinan Ni, Xiaowen Li, Yuhang Qi, Zhifang Fan, Hangyu Wang, Qiwei Chen, Yi Cheng, Feng Zhang, and Xiao Yang. Make it long, keep it fast: End-to-end 10k-sequence modeling at billion scale on Douyin. In Proceedings of the ACM Web Conference 2026, WWW ’26, pages 7989–7998. Association for Computing Machinery, 2026. doi: 10.1145/3774904.3792811. URL https://doi.org/10.1145/3774904.3792811.
  30. [30]Xiao Lv, Jiangxia Cao, Shijie Guan, Xiaoyou Zhou, Zhiguang Qi, Yaqiang Zang, Ben Wang, and Guorui Zhou. MARM: Unlocking the recommendation cache scaling-law through memory augmentation and scalable complexity. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, CIKM ’25, pages 2022–2031. Association for Computing Machinery, 2025. doi: 10.1145/3746252.3761015. URL https://doi.org/10.1145/3746252.3761015.
  31. [31]Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network, 2015. URL https://arxiv.org/abs/1503.02531.
  32. [32]Jiaxi Tang and Ke Wang. Ranking distillation: Learning compact ranking models with high performance for recommender system. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’18, pages 2289–2298. Association for Computing Machinery, 2018. doi: 10.1145/3219819.3220021. URL https://doi.org/10.1145/3219819.3220021.
  33. [33]Jae-woong Lee, Minjin Choi, Jongwuk Lee, and Hyunjung Shim. Collaborative distillation for top-N recommendation. In 2019 IEEE International Conference on Data Mining (ICDM), pages 369–378. IEEE, 2019. doi: 10.1109/ICDM.2019.00047. URL https://doi.org/10.1109/ICDM.2019.00047.
  34. [34]Chen Xu, Quan Li, Junfeng Ge, Jinyang Gao, Xiaoyong Yang, Changhua Pei, Fei Sun, Jian Wu, Hanxiao Sun, and Wenwu Ou. Privileged features distillation at Taobao recommendations. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’20, pages 2590–2598. Association for Computing Machinery, 2020. doi: 10.1145/3394486.3403309. URL https://doi.org/10.1145/3394486.3403309.
  35. [35]Anna Lipkina, Daria Tikhonovich, Viktor Yanush, Mariia Ulianova, Oleg Sorokin, Vladislav Dodonov, Ilya Murzin, Denis Burshtein, and Nikolay Savushkin. Gryphon-v2: One model in place of a cascade – generate-and-rank recommender with rollout distillation, 2026. URL https://arxiv.org/abs/2608.06213.
  36. [36]Liang Luo, Yuxin Chen, Zhengyu Zhang, Mengyue Hang, Andrew Gu, Buyun Zhang, Boyang Liu, Chen Chen, Chengze Fan, Dong Liang, Fan Yang, Feifan Gu, Huayu Li, Jade Nie, Jiayi Xu, Jiyan Yang, Jongsoo Park, Laming Chen, Longhao Jin, Qianru Li, Qin Huang, Shali Jiang, Shiwen Shen, Shuawei Wang, Sihan Zeng, Siyang Yuan, Tongyi Tang, Weilin Zhang, Wenjun Wang, Xi Liu, Xiaohan Wei, Xiaozhen Xia, Yuchen Hao, Yunlong He, Yasmine Badr, Zeliang Chen, Maxim Naumov, Yantao Yao, Wenlin Chen, Santanu Kolay, GP Musumeci, and Ellie Dingqiao Wen. Meta Lattice: Model space redesign for cost-effective industry-scale ads recommendations, 2025. URL https://arxiv.org/abs/2512.09200.
  37. [37]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, 2017. URL https://papers.nips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html.arxiv.org/abs/1706.03762
  38. [38]Iz Beltagy, Matthew E. Peters, and Arman Cohan. Longformer: The long-document transformer, 2020. URL https://arxiv.org/abs/2004.05150.
  39. [39]Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the Inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pages 2818–2826. IEEE, 2016. doi: 10.1109/CVPR.2016.308. URL https://openaccess.thecvf.com/content_cvpr_2016/html/Szegedy_Rethinking_the_Inception_CVPR_2016_paper.html.DOI
  40. [40]Rafael Müller, Simon Kornblith, and Geoffrey E. Hinton. When does label smoothing help? In Advances in Neural Information Processing Systems, volume 32, pages 4694–4703. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper/2019/hash/f1748d6b0fd9d439f71450117eba2725-Abstract.html.arxiv.org/abs/1906.02629
  41. [41]Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, and Boaquin Quiñonero Candela. Practical lessons from predicting clicks on ads at Facebook. In Proceedings of the Eighth International Workshop on Data Mining for Online Advertising, ADKDD’14, pages 1–9, New York, NY, USA, 2014. Association for Computing Machinery. doi: 10.1145/2648584.2648589. URL https://doi.org/10.1145/2648584.2648589.

Paper details

vlnc:W_hj3y8sd3
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 8

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents