Rethinking Positional Encoding and Interaction Granularity for CTR Prediction

Introduction

Click-through rate (CTR) prediction estimates the probability that a user clicks a target item based on historical behaviors, user and item features, and contextual information [19, 30]. By modeling heterogeneous feature interactions and historical behavior sequences, CTR prediction provides engagement estimates that guide candidate ranking and exposure [19, 22, 30]. CTR models are widely used in e-commerce recommendation [9], personalized content delivery [24], and online advertising [30]. These applications support personalized information access and discovery on the Web [6, 24].

Transformer-based modeling has achieved substantial progress and become an important paradigm for CTR prediction [2, 4, 10, 19, 28].

Recent approaches unify feature interaction and behavior-sequence modeling within shared Transformer architectures [12, 13, 16, 28, 31]. Current methods typically inherit index-based positional encodings and token-level causal attention from language modeling [4, 28, 31]. Positional encodings represent behaviors by their physical indices in the input sequence, while causal attention restricts historical interactions to preceding context. For example, OneTrans [28] processes historical behaviors and non-sequential features as a unified token sequence with learned absolute positional embeddings (APE) through stacked causal Transformer blocks. However, the suitability of these positional encodings and interaction granularities for CTR prediction remains unclear.

Empirical Observations. We examine positional encoding and interaction granularity in OneTrans [28] and observe two phenomena. (1) As shown in Figure 1(a), none of the six positional encoding configurations of OneTrans yields better performance than its NoPE counterpart on KuaiRand. (2) Figure 1(b) shows that bidirectional interaction within target-relative temporal groups outperforms both event-level causal and global bidirectional interaction in AUC under all four encodings. Together, these findings motivate reconsidering: What positional information should CTR models encode, and at what granularity should historical behaviors interact?

Understanding the Observations. To understand these observations, we examine historical behavior sequences in the context of the current prediction target. Figure 2 provides an illustrative example for the following two-part analysis.

• Positional information. As illustrated in Figure 2(a), historical behaviors in CTR prediction have three positional views: sequence rank, event timestamp, and target-relative time. Sequence rank characterizes a behavior’s ordinal context, whereas its timestamp specifies when it occurred. For a fixed history, ranks and timestamps remain unchanged across prediction requests, but the target-relative coordinates change from $T_1-t_i$ to $T_2-t_i$ with the prediction timestamp. Moreover, the same rank can correspond to different target-relative times, while temporally adjacent behaviors can occupy distinct ordinal positions. Thus, effective CTR positional modeling should jointly capture behavioral order, temporal information, and target-relative context, rather than relying on any single positional signal alone.

• Interaction granularity. As illustrated in Figure 2(b), the same history is partitioned into different temporal groups as the prediction time changes, and is thus modeled at different scales for different targets. Within a group, behaviors mutually enrich one another to characterize the user’s interest during that period, while successive groups reveal how interests evolve toward the current request. Event-level causal interaction limits this within-group enrichment, whereas global bidirectional interaction removes the directed order across groups.

Our Solution. Based on the above observations and analysis, we propose TRIP, which combines target-aware rank–time positional encoding with temporal-group interaction modeling. Factorized Rank Encoding (FRE) represents each behavior’s ordinal position through a low-rank factorization with rank-specific coefficients over shared basis vectors. This preserves rank distinctions with a lightweight parameterization. Target-Relative Time Encoding (TRTE) encodes each behavior’s temporal distance to the target on a coarse-to-fine scale. Based on these target-anchored coordinates, it constructs density-corrected learnable temporal potentials as attention biases to guide time-aware history reading. Using the same target-relative temporal reference, Temporal Group-Causal Interaction (TGCI) partitions the history into temporal groups, where behaviors enrich one another through bidirectional interaction, while causal interaction across groups preserves interest evolution toward the current request. Furthermore, we use masked item reconstruction to strengthen contextual representations within each temporal group. Finally, contextualized behavior states are condensed into compact group summaries for efficient modeling of interest evolution in subsequent layers.

We evaluate TRIP against eleven baselines on four public CTR benchmarks. TRIP outperforms the best-performing baseline on each dataset, achieving an average relative AUC improvement of 1.47% and a maximum of 4.55% on Taobao. The joint target-aware rank–time encoding also transfers to other Transformer-based CTR models, achieving relative AUC improvements of up to 4.20% over the best of eight alternative positional encoding configurations on the same backbone. Ablation studies further confirm the importance of behavior rank and target-relative time, as well as the effectiveness of temporal-group interaction modeling. The main contributions of this work are as follows:

• We identify limitations of existing positional and temporal encodings and uniform interaction structures in CTR prediction through empirical comparisons and behavioral analysis.

• We propose target-aware rank–time positional encoding through FRE and TRTE, coupling lightweight ordinal representations with target-relative temporal potentials.

• We develop TGCI for within-group mutual enrichment and cross-group causal interest evolution, with masked reconstruction for contextual learning and group summaries for efficient modeling.

Preliminaries

CTR prediction is a supervised binary classification task that estimates the probability of a click on a displayed target item. Each sample corresponds to an impression of item $v_*$ to user $u$ at prediction time $T$. Let $\boldsymbol{x}^{\mathrm{NS}}=[\boldsymbol{x}^{(1)},\ldots,\boldsymbol{x}^{(F)}]$ denote the associated user, target-item, and contextual features across $F$ fields. These heterogeneous fields include categorical identifiers and attributes, as well as numerical features. The user’s observed behavior history is

where $n$ is the history length, and $v_i$, $f_i$, and $t_i$ denote the item, associated features, and timestamp of the $i$-th event, respectively. Historical behaviors provide sequential evidence of user interests, while the target item and impression context specify the prediction instance. The label $y\in\{0,1\}$ indicates whether the displayed item is clicked. Conditioned on these inputs, the model estimates

The model parameters $\theta$ are learned from labeled interactions by minimizing the binary cross-entropy loss. Following unified CTR models [12, 13, 28], we map each historical event to a token $h_i=\operatorname{Tokenizer}(v_i,f_i,t_i)$ and tokenize the embedded user, target-item, and contextual fields into $N_{\mathrm{NS}}$ non-sequential tokens. The resulting matrices are $\boldsymbol{H}_{\mathrm{S}}\in\mathbb{R}^{L\times D}$ and $\boldsymbol{H}_{\mathrm{NS}}\in\mathbb{R}^{N_{\mathrm{NS}}\times D}$, where $L$ is the history capacity and $D$ the hidden dimension. Non-sequential tokens can combine multiple fields, so $N_{\mathrm{NS}}$ need not equal the field count $F$.

Methodology

Figure 3 illustrates TRIP, a unified CTR framework for joint behavior-sequence modeling and non-sequential feature interaction. TRIP jointly represents behavioral rank and target-relative time, then organizes interactions within target-relative temporal groups. FRE injects a lightweight ordinal representation into each history token. TRTE uses the event’s distance to the target to guide attention through either learnable temporal potentials or temporal rotations. TGCI contextualizes events within temporal groups, supplies masked-item reconstruction supervision, and compresses the history into group summaries for subsequent modeling.

Factorized Rank Encoding

Observations. In CTR prediction, sequence rank provides an item’s ordinal context, which elapsed time alone cannot fully characterize under irregular user activity. Since the predictive role of a historical behavior can vary across sequence positions, we adopt learnable rank embeddings that allow CTR supervision to determine position-specific representations without imposing a predefined positional geometry. Interestingly, we observe that these learned embeddings exhibit an approximately low-rank structure. As shown in Figure 4, reconstructing them with only a few principal directions largely preserves prediction performance. Motivated by this observation, we design FRE with rank-specific coefficients over shared basis vectors, enabling flexible yet compact ordinal representations.

FRE parameterizes the rank representation with a common vector $\mu\in\mathbb{R}^{D}$, position-specific coefficients $A \in\mathbb{R}^{L \times K}$, and shared basis vectors $B \in\mathbb{R}^{K \times D}$. Here, $K$ is the factorization width. We center the coefficients across ranks to separate the shared offset from rank-specific variation and define the rank representation as

where $a_r^c$ is row $r$ of $A_c$, and $\gamma_{\mu}$ and $\gamma_R$ control the shared and rank-dependent components, respectively. The parameters $\mu$, $A$, and $B$ are jointly learned end-to-end, allowing each rank to learn a distinct linear combination of shared basis vectors for CTR prediction.

For event $i$ with sequence rank $r_i$, FRE adds the corresponding rank embedding to its input representation:

where $P$ collects the rank embeddings, and concatenation is along the token axis. To represent ordinal context within the behavior sequence, we only apply FRE to the history tokens. Compared with full learned APE, FRE reduces the positional parameter count from $LD$ to $D + LK + KD$, corresponding to a 3.8–21.6× reduction under our benchmark configurations. This lightweight design provides effective rank representations within the behavior history.

Target-Relative Time Encoding

Sequence rank captures behavioral order but overlooks temporal intervals, while event timestamps alone do not explicitly characterize temporal relationships to the current prediction target. Moreover, the temporal relevance of a historical behavior varies with the current target, even when the history remains unchanged. Therefore, we propose TRTE to encode each behavior’s temporal position relative to the target.

Target-Anchored Temporal Coordinates. Temporal distinctions are not equally informative across the history: recent behaviors often reflect immediate interests and benefit from finer temporal resolution, whereas distant behaviors can be modeled at coarser scales. We therefore adopt a fine-grained logarithmic time scale to preserve detailed temporal distinctions near the target while compactly representing the distant past. The temporal coordinate is defined as

where $\delta_i = T - t_i$ is the elapsed time in seconds from event $i$ to the prediction timestamp $T$, $m \ge1$ specifies the number of subdivisions per logarithmic interval, and $u_i$ is the resulting discretized logarithmic time coordinate. Increasing $m$ refines temporal resolution while preserving the logarithmic scaling.

Learnable Temporal Potentials. We map each target-relative coordinate $u_i$ to a discrete temporal interval index $j_i$ and introduce a head-specific learnable function $\phi_h(j_i; \theta_h)$. This allows different attention heads to respond differently to the same temporal interval. Combined with the fine-grained logarithmic time scale, these head-specific preferences enable flexible aggregation of behaviors from both recent and distant history. Historical behaviors are unevenly distributed across temporal intervals. Since softmax assigns weights to individual events, intervals containing more behaviors may accumulate greater attention mass simply through event multiplicity. We therefore correct the temporal potential using the event distribution within each sample. For positive elapsed times, the corrected potential is

where $n_j$ counts valid history events in interval $j$ for the current sample, and $\lambda_{\mathrm{cnt}}$ controls the correction strength. This correction mitigates quantity-driven amplification while retaining content relevance and learned temporal preferences.

For prediction-side queries $q \in\mathrm{NS}$ and history keys $k \in S$, we add the corrected potential to the attention scores:

Here, $q_{h,q}$ and $k_{h,k}$ denote query and key vectors, $\langle\cdot,\cdot\rangle$ is their inner product, and $d_h = D/H$ is the dimension of each of $H$ heads. Other query–key pairs retain the standard content-based scores. The attention weights are obtained by softmax:

where $\mathcal{V}(q)$ contains all valid keys visible to query $q$, including permitted prediction-side keys. The corrected scores guide history reading through head-specific target-relative temporal preferences. This learned-potential variant is denoted TRIP-B.

Target-Relative Temporal Rotations. As an alternative to learnable temporal potentials, we introduce a rotary formulation, denoted TRIP-R, that incorporates target-relative temporal information into query/key interactions. Reusing the logarithmic coordinate in (5), we place historical behaviors with $\delta_i > 0$ on the negative time axis and prediction-side tokens at the origin:

Here, $u_i$ is evaluated with resolution $m = m_R$, and $C_{\mathrm{rot}}$ bounds the coordinate magnitude. This arrangement expresses each historical behavior’s temporal position relative to the current prediction time. Following RoPE [20], we rotate queries and keys:

where $R(\cdot)$ denotes the rotary transformation. The resulting attention scores are

and are normalized using (8). This formulation encodes target-relative temporal relationships through query/key rotations, while FRE retains ordinal context in the input representations. TRIP-B and TRIP-R are alternative variants of TRTE. TRIP-B biases prediction-side history reading only in the first Transformer block, while TRIP-R applies temporal Q/K rotations in all Transformer blocks. Implementation details are provided in Appendix A.2.

Temporal Group-Causal Interaction

TGCI organizes the history into temporal groups defined by distance to the current prediction, so that behaviors within a group exchange context bidirectionally while information across groups flows causally toward the target. Sharing the target-relative reference of TRTE with its own resolution, we partition historical behaviors as

where $C_{\mathrm{grp}}$ caps the group index and $\mathcal{G}_c = \{i : g_i = c\}$ denotes a nonempty temporal group. For valid history events ordered chronologically, we define the additive attention mask as

The mask is added to the attention scores before softmax. It permits bidirectional context exchange within a group while retaining causal information flow across groups. Because groups are defined relative to $T$, the same history is regrouped as the target changes.

Masked Contextual Reconstruction. To encourage learning semantic relationships within temporal groups, we introduce masked-item reconstruction as auxiliary supervision. During training, we select historical events for masking at a nominal ratio $\rho\in(0, 1)$ and replace their item embeddings with a learnable mask embedding. The masked set $\mathcal{M}$ is selected within temporal groups. We reconstruct the original item embeddings from contextualized event states:

where $\mathcal{M}$ is the set of masked events and $\mathbf{h}_i^{(0)}$ is the event representation after the first Transformer block. The reconstruction head $f_{\mathrm{rec}}$ consists of layer normalization and a two-layer MLP with a SiLU activation. It predicts the original item embedding from each masked event’s contextualized state before group compression. The auxiliary objective encourages recovery of masked behavior content using local context. Reconstruction and CTR prediction share the same masked input and backbone pass during training. At inference, masking and the reconstruction head are disabled.

Group-Level Interest Modeling. After within-group contextualization, we aggregate event representations into a group summary:

Each summary represents the user’s interest during the corresponding period. The summaries are arranged chronologically and replace the original events, reducing the history length from $L$ events to $G$ nonempty group summaries, while prediction-side states are retained. Subsequent causal Transformer blocks model interest evolution across these group-level representations more effectively and at lower computational cost.

The CTR head predicts the click probability from the mean-pooled prediction-side states:

where $\mathbf{z}_q^{\mathrm{final}}$ is the final state of prediction-side token $q$, $f_{\mathrm{CTR}}$ is an MLP, and $\sigma$ is the sigmoid function.

Optimization

We train TRIP by jointly minimizing the CTR prediction loss and an auxiliary masked-item reconstruction loss. For a minibatch $\mathcal{B}$,

Experimental Setup

the CTR objective is binary cross-entropy:

The reconstruction objective encourages contextualized event representations to recover masked item content:

where $\mathcal{M}_{\mathcal{B}}$ denotes the masked events in the minibatch, $\hat{\mathbf{e}}_i$ is the reconstructed item embedding, and $\operatorname{sg}$ stops gradients through the original embedding used as the target. The joint objective is

where $\lambda_{\mathrm{rec}}$ controls the auxiliary supervision strength. The model parameters are jointly optimized under this objective. Training and implementation details are provided in Appendix A.

Limitations. TRIP defines temporal groups using fixed logarithmic intervals relative to the prediction time, which may not align with individual users’ interest transitions. In addition, mean-pooling contextualized events into group summaries can obscure fine-grained event distinctions in subsequent layers. Adaptive group boundaries and selective retention of event-level representations could address these limitations.

Experiments

Experimental Setup

Datasets. We evaluate on four public benchmarks. KuaiRec [7] contains short-video viewing feedback with a compact catalog; KuaiRand-1K [8] provides short-video logs with randomized exposure interventions and a larger catalog. TAAC-1M [18] contains Tencent advertising sequences, while TaobaoAd [1] combines display-ad impressions with shopping histories. These settings cover different recommendation domains, catalog sizes, history lengths, and feedback distributions (Table 1).

Statistic	KuaiRec	KuaiRand	TAAC	TaobaoAd
Train samples	12,190,187	10,911,770	4,714,200	20,015,245
Val. samples	181,130	390,961	148,699	3,234,051
Test samples	159,040	410,314	92,495	3,308,665
User ID vocab.	7,176	1,000	1,001,845	1,141,729
Item ID vocab.	10,728	4,369,953	4,783,154	12,975
User fields	31	31	9	9
Target fields	45	50	14	5
Max. history	200	200	100	300
Train pos. rate	7.44%	37.83%	10.92%	5.20%

Baselines. We use two sets of baselines to evaluate overall model performance and the contribution of positional encoding.

• Overall model performance. We compare TRIP with eleven models: (1) feature-interaction models, including DeepFM [11], DCNv2 [22], Wukong [27], and RankMixer [32]; (2) behavior-sequence models, including DIN [30], DIEN [29], and STCA [10]; and (3) unified feature and sequence models, including TokenFormer [31], OneTrans [28], HyFormer [13], and MixFormer [12].

• Positional encoding. We compare FRE + TRTE-B and FRE + TRTE-R with eight configurations in four categories: (1) physical-position configurations, including the NoPE reference, learned APE, sinusoidal APE, and rank RoPE [20]; (2) time-based encodings, including T-RoPE [17] and ClockRoPE [5]; (3) target-aware encoding, represented by CAPE [25]; and (4) joint time–position encoding, represented by TO-RoPE [23]. We evaluate them on OneTrans, HyFormer, and MixFormer to assess the contribution and transferability of FRE and TRTE independently of TGCI.

Training and Evaluation. We implement all models in PyTorch and use the same NVIDIA H20 hardware, optimizer, learning rate, batch sizes, and training duration for fair comparison. All baselines retain raw event-time and prediction-time features, so the comparisons do not withhold timestamp information from existing methods. We evaluate predictive performance using AUC and LogLoss, reporting the mean and sample standard deviation over three random seeds. Further training, model, and evaluation details are provided in Appendix A.1.

Overall Performance

Table 2 shows that TRIP-B consistently outperforms the eleven baselines in AUC across all four datasets, with an average relative improvement of 1.47% and a maximum of 4.45% on TAAC. TRIP also achieves the lowest LogLoss on KuaiRec, TAAC, and TaobaoAd, demonstrating strong ranking discrimination and probabilistic prediction performance. These gains reflect the complementary roles of FRE and TRTE in modeling behavioral rank and target-relative time, while TGCI enriches behaviors within target-relative temporal groups and preserves interest evolution across groups. Together, these components provide richer positional context and more effective historical interaction. On KuaiRand, despite its AUC advantage, TRIP-B exhibits a higher LogLoss. KuaiRand contains heterogeneous exposure scenarios with distinct feedback definitions and positive rates, and analysis reveals that TRIP-B substantially overestimates positive-feedback probabilities in certain low-positive-rate scenarios, accounting for much of the LogLoss gap (Appendix B.5).

Evaluation of Positional Encoding

Table 3 evaluates FRE + TRTE against eight positional and temporal configurations on three CTR backbones, without using TGCI. The additive and rotary variants outperform the strongest competing encoding in 10 and 11 of the 12 settings, respectively, with at least one achieving the highest AUC in every setting. On TAAC, FRE + TRTE-R improves AUC by 4.20%, 3.78%, and 3.65% on OneTrans, HyFormer, and MixFormer, respectively. These results demonstrate that the proposed rank–time representation is transferable across backbones independently of the complete TRIP pipeline. The comparisons further highlight the limitations of existing positional encodings. On KuaiRand, learned APE, sinusoidal APE, and rank RoPE underperform NoPE across all three backbones, while time-based T-RoPE and ClockRoPE also trail NoPE on OneTrans. On TAAC, target-aware CAPE and joint order–time TO-RoPE remain below FRE + TRTE across all three backbones. These findings suggest that incorporating positional cues does not by itself guarantee effective modeling. FRE + TRTE instead couples task-adaptive rank representations with prediction-time-anchored temporal encoding, jointly interpreting sequence rank and event time relative to the current prediction.

Rank Encoding Replacements. Table 4 replaces FRE in TRIP-B with other rank encodings while keeping TRTE fixed. The contrast is sharpest on TAAC. Rank encoding alone does not improve over

Positional Encoding	KuaiRec OneTrans	KuaiRec HyFormer	KuaiRec MixFormer	KuaiRand OneTrans	KuaiRand HyFormer	KuaiRand MixFormer	TAAC OneTrans	TAAC HyFormer	TAAC MixFormer	TaobaoAd OneTrans	TaobaoAd HyFormer	TaobaoAd MixFormer
NoPE	0.8443	0.8449	0.8438	0.7694	0.7683	0.7754	0.6067	0.6050	0.6042	0.6276	0.6254	0.6265
Learned APE	0.8445	0.8466	0.8439	0.7589	0.7621	0.7660	0.6058	0.6037	0.6034	0.6281	0.6278	0.6275
Sinusoidal APE	0.8457	0.8450	0.8456	0.7648	0.7641	0.7645	0.6061	0.6038	0.6063	0.6273	0.6276	0.6279
Rank RoPE	0.8434	0.8432	0.8442	0.7616	0.7623	0.7026	0.6057	0.6018	0.6058	0.6280	0.6267	0.6273
CAPE	0.8424	0.8421	0.8189	0.7641	0.7641	0.6418	0.6055	0.6059	0.5729	0.6266	0.6248	0.6106
TO-RoPE	0.8474	0.8462	0.8471	0.7668	0.7640	0.6742	0.6062	0.6031	0.6061	0.6283	0.6260	0.6269
T-RoPE	0.8497	0.8453	0.8452	0.7668	0.7687	0.7675	0.6054	0.6035	0.6021	0.6292	0.6261	0.6225
ClockRoPE	0.8488	0.8464	0.8496	0.7658	0.7643	0.7648	0.6056	0.6036	0.6053	0.6284	0.6274	0.6267
FRE + TRTE-B	0.8504	0.8502	0.8495	0.7756	0.7693	0.7758	0.6319	0.6137	0.6277	0.6291	0.6279	0.6286
FRE + TRTE-R	0.8502	0.8490	0.8528	0.7747	0.7708	0.7719	0.6322	0.6288	0.6284	0.6296	0.6285	0.6282

NoPE there, and removing FRE from TRIP-B returns AUC to a similar level, yet with TRTE in place, learned APE and FRE outperform Rank RoPE by about 0.027. Rank RoPE still exposes recency, because histories are right-aligned and prediction-side tokens share a fixed rotary coordinate, but it only changes attention scores. Additive encodings instead write rank into the token representations that TRTE aggregates by target-relative time, consistent with rank and time being complementary when both enter the representations. FRE performs comparably to or better than full learned APE on all four datasets while using 3.8–21.6× fewer positional parameters, since each rank is expressed through a few shared basis vectors.

Encoding	KuaiRec	KuaiRand	TAAC	TaobaoAd
FRE	0.8512 ± 0.0016	0.7778 ± 0.0008	0.6337 ± 0.0017	0.6299 ± 0.0007
Learned APE	0.8493 ± 0.0009	0.7787 ± 0.0012	0.6323 ± 0.0015	0.6297 ± 0.0013
Sinusoidal APE	0.8504 ± 0.0010	0.7782 ± 0.0009	0.6240 ± 0.0121	0.6292 ± 0.0012
Rank RoPE	0.8496 ± 0.0014	0.7753 ± 0.0010	0.6061 ± 0.0021	0.6287 ± 0.0017

Ablation and Mechanism Analysis

Figure 5 evaluates nine ablated variants of TRIP-B. The full model achieves the highest mean AUC among the displayed variants on all four datasets, demonstrating the benefits of combining rank–time positional modeling with temporal-group interaction.

Ordinal and Target-Relative Information. Removing either FRE or TRTE lowers AUC across all four datasets, indicating that each contributes beyond the other. The effect is particularly pronounced on TAAC, where AUC drops from 0.6337 to 0.6066 without FRE and 0.6073 without TRTE. Replacing the prediction-time reference while keeping the other components fixed also reduces AUC across datasets. These results support retaining both ordinal and temporal information, with the current prediction timestamp providing an effective reference for temporal modeling.

Temporal Preferences and Density. Removing density correction lowers mean AUC across all four datasets, supporting occupancy adjustment to counteract the attention advantage of densely populated intervals. Figure 6 further shows dataset- and head-specific temporal patterns: the learned potentials assign higher scores to minute-scale intervals on KuaiRec, exhibit a broad recency preference on KuaiRand, and differ across heads on TAAC. TaobaoAd shows separated high-potential windows rather than uniform recency decay. These curves describe temporal biases, not final attention weights, which also depend on content scores and density correction. Together, these observations support flexible temporal preferences with sample-wise occupancy adjustment.

Within-Group Interaction and Group-Level Modeling. Replacing group-causal interaction with token-level causal attention lowers mean AUC on every dataset even when temporal grouping and summaries are retained, supporting the contribution of within-group bidirectional context. Removing summaries also reduces performance, indicating the value of group-level aggregation after event contextualization. Removing reconstruction while retaining masking further lowers AUC, supporting the contribution of auxiliary reconstruction supervision. These controlled comparisons support the roles of within-group enrichment, group summaries, and reconstruction within TGCI. Variant definitions and parameter sensitivity results are given in Appendices B.1 and B.4.

Grouping Strategies. Figure 7 compares temporal grouping with Random and Rank-contiguous alternatives. All strategies preserve per-sample group counts and sizes, keeping the summary compression ratio unchanged. Random redistributes group membership, whereas Rank-contiguous partitions consecutive ranks using reordered temporal-group sizes. Under these matched conditions, temporal grouping achieves the highest mean AUC on all four datasets. This suggests that grouping behaviors by their distance to the prediction is more effective than arbitrary or rank-based partitions, beyond the benefit of compression alone. Construction details are given in Appendix B.1.

Efficiency of Group-Level Modeling

Table 5 shows two benefits of group-summary compression: higher AUC and 6.3–32.8% lower per-step training time. On TaobaoAd, AUC improves from 0.6297 to 0.6304 while time falls from 141.31 to 95.01 ms/step; KuaiRand gains AUC from 0.7753 to 0.7778 with 20.7% time savings. The AUC gains suggest that aggregation suppresses redundant or noisy event-level variation while retaining useful interest signals. Preserving event-level inputs in the first Transformer block allows the model to learn contextual representations from the original behavior features before compression. Subsequent aggregation can therefore consolidate the contextualized interest of each group rather than discard event details before they are encoded. The shorter group-level sequence also reduces later-layer computation, improving accuracy and efficiency together. Further timing comparisons are provided in Appendix B.3.

Dataset	AUC (w/o → with)	Saved	ms/step
KuaiRec	0.8501 → 0.8508	20.0%	113.95
KuaiRand	0.7753 → 0.7778	20.7%	116.54
TAAC	0.6355 → 0.6359	6.3%	102.13
TaobaoAd	0.6297 → 0.6304	32.8%	95.01

Related Work

CTR Modeling

CTR modeling has developed along two lines: feature interaction and sequence modeling. Feature models have progressed from factorization-based and explicit crosses [11, 22] to attention-based interactions [19]. Wukong [27] and RankMixer [32] scale interactions through stacked factorization and token mixing, respectively. Sequence models have evolved from target-conditioned interest aggregation and evolution [29, 30] toward efficient long-history processing with LONGER [3] and STCA [10]. These separate modules increasingly converge toward unified architectures for joint optimization and sequence–feature interaction [12, 13, 16, 28, 31]. OneTrans [28] processes historical and non-sequential tokens with causal Transformer blocks. HyFormer [13] and MixFormer [12] further explore the integration and joint scaling of sequence modeling and feature interaction. However, suitable positional encoding and interaction granularity for Transformer-based Unified CTR prediction remain unclear.

Position Encoding for Sequential Recommendation

Positional encodings represent sequence indices, event time, target relevance, or joint order–time relationships [15, 20, 23, 25]. Index-based learned or rotary encodings remain common [14, 20, 21, 28, 31]. Other methods incorporate event timing or intervals to account for irregular behavior histories [5, 15, 17, 23, 26]. For example, T-RoPE [17] uses multiscale timestamp rotations with prediction-time query alignment, while TO-RoPE [23] combines index and time rotations within shared or separate rotary subspaces. In addition, context-aware CAPE [25] constructs soft positions from target–history item dissimilarity, expressing item relevance rather than elapsed time. TRIP combines task-adaptive rank representations with prediction-time-anchored temporal encoding and target-relative temporal grouping.

Conclusion

In this work, we found that existing positional encodings do not consistently improve CTR prediction, while bidirectional interaction within target-relative temporal groups can outperform both token-level causal and global bidirectional attention. These observations reveal the need to reconsider both historical position representations and interaction granularity. Based on these findings, we proposed TRIP, which jointly models sequence rank, event time, and the prediction-time reference, and organizes history into target-relative temporal groups with intra-group bidirectional and inter-group causal interaction. Experiments on four public datasets demonstrate TRIP’s effectiveness, while positional-encoding transfer experiments across three unified backbones validate the portability of its rank–time design. Detailed ablations and representation analyses further support the component designs.

References

Implementation Details

Training and Evaluation Settings

Common Training Settings. We implement all models in PyTorch and train each run on one NVIDIA H20 GPU. TRIP and all baselines share AdamW, a fixed learning rate of $10^{-3}$, weight decay $10^{-2}$, training/evaluation batch sizes of 1024/2048, and 64-dimensional sparse feature embeddings. All models are trained for one epoch with BF16 mixed precision and gradient-norm clipping at 1.0. This single-pass protocol limits the risk of overfitting sparse embeddings through repeated exposure to the same impressions. Hyperparameters are selected using only training and validation data, and checkpoints are selected by validation AUC before final test evaluation. We report the mean and sample standard deviation over seeds 41, 42, and 43, counting only real samples. These shared training and evaluation settings ensure a consistent comparison. The TRIP implementation uses PyTorch 2.5.1, CUDA 12.1, and cuBLAS 12.8.5; timing protocols are detailed in Appendix B.3.

Model and Hyperparameter Settings. TRIP uses four Transformer blocks, hidden dimension 64, four attention heads, and FFN dimension 256. The prediction head mean-pools the final prediction-side states and uses a two-layer MLP. The non-sequential token counts $N_{\mathrm{S}}$ are 12, 18, 12, and 12 for KuaiRec, KuaiRand, TAAC, and TaobaoAd, respectively. Most Transformer-based baselines share these block, hidden, attention-head, and FFN settings while retaining their respective architectures. RankMixer [32] retains its 16-head configuration, while MixFormer [12] uses a per-head dimension of 64. Method-specific hyperparameters follow the original model designs and are tuned on validation data. Table 6 lists TRIP’s selected settings. FRE and reconstruction settings are shared by both variants, whereas $m_{\mathrm{B}}$, $J_{\max}$, and $\lambda_{\mathrm{cnt}}$ apply to TRIP-B and $m_{\mathrm{R}}$ to TRIP-R. Temporal grouping uses $C_{\mathrm{grp}} = 32$ independently of encoding resolution, and TRIP-R uses $C_{\mathrm{rot}} = 32$.

Dataset	$K$	$\gamma_{\mu}$	$\gamma_{R}$	$\rho$	$\lambda_{\mathrm{rec}}$	$m_{\mathrm{B}}$	$J_{\max}$	$\lambda_{\mathrm{cnt}}$	$m_{\mathrm{R}}$
KuaiRec	4	1.0	1.0	0.15	$10^{-4}$	6	192	0.50	6
KuaiRand	2	0.75	1.0	0.05	0.01	8	256	0.75	6
TAAC	10	0.25	1.0	0.15	0.01	1	32	0.50	2
TaobaoAd	4	1.0	1.0	0.05	0.01	1	32	0.50	1

Data and Sequence Processing. Histories retain positive-feedback events on KuaiRec and KuaiRand and all events on TAAC and TaobaoAd. KuaiRec uses watch ratio $\ge2$ for positive feedback;

TaobaoAd uses category IDs for history–target matching, with ad-group ID as a separate feature. For each target impression, valid history events are ordered chronologically and truncated to the most recent $L$ events. Shorter histories are left-padded, with valid events right-aligned: the event nearest the prediction-side tokens is the most recent, and events farther to the left occurred earlier. Padding is excluded by the validity mask. All models use the same dataset-specific history construction.

Module Implementation

Encoding Initialization. FRE initializes $\boldsymbol{\mu}$ to zero, $A$ from a normal distribution with standard deviation $1/\sqrt{K}$, and $B$ from a truncated normal distribution with standard deviation 0.02, without PCA initialization. TRIP-B uses $j_i = \min(J_{\max}, 1 + \lfloor m_{\mathrm{B}}\log_2(1+\delta_i)\rfloor)$ to index a learnable embedding table $\Theta\in\mathbb{R}^{(J_{\max}+1)\times H}$, with $\phi_h(j_i;\theta_h) = \Theta_{j_i,h}$. Each head shares its temporal potential across prediction-side queries. The table is initialized as $\Theta_{j,h}=-0.005j$, and occupancy is counted over valid events before compression. TRIP-R uses fixed RoPE frequencies with base $10^4$.

Reconstruction and Summary Details. Temporal groups with at least two events retain both masked and visible items. The reconstruction weight increases linearly to $\lambda_{\mathrm{rec}}$ over the first 5% of training steps. FRE is not reapplied after summary compression. For TRIP-R, each summary’s representative timestamp is the rounded mean of its constituent event timestamps, from which target-relative coordinates are recomputed for subsequent blocks.

Supplementary Experiments

Additional Ablation Results

Tables 2 and 4 report overall performance and rank replacements in the main text. Here we specify the variants for ablation analysis.

Variant Definitions. TGCA removal restores event-level causal attention while retaining groups, summaries, and reconstruction. Reconstruction removal retains masking; the joint removal disables both. Target-awareness removal changes only the prediction-time reference while keeping all other components unchanged. All ablated variants are compared with the full TRIP-B model.

Grouping Controls. For the comparison in Figure 7, both alternatives begin with the original temporal grouping and retain each group label’s event count. Random permutes group labels among valid events using a deterministic fixed-seed hash, so an individual sample’s assignment remains unchanged across training steps. Rank-contiguous reorders the original group sizes using a fixed permutation of group labels, then partitions consecutive valid ranks according to those sizes. Its segments are neither equal-sized nor fixed-width; all strategies retain the same number of group summaries.

Learned Positional Representations

Learned Rank Geometry. Figure 8 and Figure 9 show that FRE does not impose a uniform distance-decay geometry. On TaobaoAd, TRIP-B learns nonuniform rank relationships, whereas TRIP-R exhibits more regular diagonal bands and pronounced rank-distance oscillations. The other datasets show weaker oscillatory structure. These differences illustrate how low-rank sharing accommodates dataset-specific positional relationships and rank representations adapted to the temporal formulation.

Temporal Preference and Density Correction. Figure 10 reports biases averaged over events within each interval using 8,192 test samples per dataset. Compared with the raw potentials in Figure 6, occupancy correction changes the relative prominence of temporal windows, showing that learned preference alone does not determine the applied bias. Since the same interval penalty is subtracted across heads, correction preserves their preference differences while reducing event-multiplicity effects. Thus, TRTE separates head-specific temporal preference from sample-specific history density.

Efficiency Analysis

On TaobaoAd, TRIP-B takes 72.63 ms/step, compared with 73.78 for OneTrans, 147.23 for HyFormer, 124.20 for MixFormer, and 123.88 for TokenFormer. This measurement uses an optimized temporal-bias implementation; Table 5 compares runs with and without compression under the same standard implementation, so its relative savings are unaffected. TRIP retains OneTrans’s dense-attention formulation: its full-history first block adds positional-encoding and reconstruction overhead, offset by shorter group-summary sequences in later blocks. For $N_{\mathrm{blk}}$ blocks, the dense attention cost is $O((L+N_{\mathrm{ns}})^2D)+(N_{\mathrm{blk}}-1)O((G+N_{\mathrm{ns}})^2D)$. For candidates sharing a request timestamp and candidate-independent historical inputs, history-side states and key/value projections can in principle be reused, since historical tokens do not attend to NS tokens.

Hyperparameter Analysis

Figure 11 examines seven TRIP-B hyperparameters across four datasets, varying one at a time with the others fixed. Each sweep, including its reference point, is independently rerun with three seeds; reference means may therefore differ across sweeps and from Table 2. Validation-selected settings in Table 6 need not coincide with these test-set peaks. Branch-scale axes retain the original configuration values.

Model	KuaiRec AUC $\uparrow$	KuaiRec LogLoss $\downarrow$	KuaiRand AUC $\uparrow$	KuaiRand LogLoss $\downarrow$	TAAC AUC $\uparrow$	TAAC LogLoss $\downarrow$	TaobaoAd AUC $\uparrow$	TaobaoAd LogLoss $\downarrow$
DeepFM	0.8329 $\pm$0.0021	0.2261 $\pm$0.0015	0.7362 $\pm$0.0016	0.5838 $\pm$0.0008	0.5729 $\pm$0.0047	0.1914 $\pm$0.0030	0.5902 $\pm$0.0024	0.2119 $\pm$0.0009
DCNv2	0.8445 $\pm$0.0016	0.2262 $\pm$0.0185	0.7730 $\pm$0.0012	0.5487 $\pm$0.0018	0.6018 $\pm$0.0016	0.1838 $\pm$0.0042	0.6232 $\pm$0.0013	0.2173 $\pm$0.0086
WuKong	0.8448 $\pm$0.0012	0.2191 $\pm$0.0015	0.7739 $\pm$0.0028	0.5506 $\pm$0.0040	0.6021 $\pm$0.0027	0.1820 $\pm$0.0054	0.6251 $\pm$0.0003	0.1974 $\pm$0.0012
RankMixer	0.8449 $\pm$0.0001	0.2197 $\pm$0.0031	0.7752 $\pm$0.0008	0.5481 $\pm$0.0002	0.6021 $\pm$0.0021	0.1862 $\pm$0.0043	0.6241 $\pm$0.0003	0.1983 $\pm$0.0013
DIN	0.8447 $\pm$0.0009	0.2154 $\pm$0.0004	0.7705 $\pm$0.0003	0.5518 $\pm$0.0010	0.6042 $\pm$0.0019	0.1875 $\pm$0.0052	0.6235 $\pm$0.0008	0.1977 $\pm$0.0011
DIEN	0.8449 $\pm$0.0015	0.2142 $\pm$0.0011	0.7686 $\pm$0.0007	0.5593 $\pm$0.0032	0.6018 $\pm$0.0011	0.1795 $\pm$0.0029	0.6257 $\pm$0.0009	0.1965 $\pm$0.0011
STCA	0.8440 $\pm$0.0023	0.2161 $\pm$0.0004	0.7705 $\pm$0.0005	0.5629 $\pm$0.0106	0.6014 $\pm$0.0008	0.1888 $\pm$0.0026	0.6228 $\pm$0.0005	0.1974 $\pm$0.0006
TokenFormer	0.8437 $\pm$0.0011	0.2181 $\pm$0.0007	0.7593 $\pm$0.0013	0.5794 $\pm$0.0062	0.5981 $\pm$0.0060	0.1838 $\pm$0.0046	0.6263 $\pm$0.0005	0.1964 $\pm$0.0007
OneTrans	0.8443 $\pm$0.0012	0.2163 $\pm$0.0005	0.7694 $\pm$0.0013	0.5554 $\pm$0.0020	0.6067 $\pm$0.0011	0.1857 $\pm$0.0061	0.6276 $\pm$0.0015	0.1966 $\pm$0.0002
HyFormer	0.8449 $\pm$0.0005	0.2151 $\pm$0.0006	0.7683 $\pm$0.0016	0.5559 $\pm$0.0008	0.6050 $\pm$0.0009	0.1826 $\pm$0.0023	0.6254 $\pm$0.0012	0.1962 $\pm$0.0005
MixFormer	0.8438 $\pm$0.0034	0.2210 $\pm$0.0058	0.7754 $\pm$0.0006	0.5490 $\pm$0.0014	0.6042 $\pm$0.0012	0.1793 $\pm$0.0038	0.6265 $\pm$0.0008	0.1977 $\pm$0.0004
TRIP-B	0.8512 $\pm$0.0016	0.2114 $\pm$0.0007	0.7778 $\pm$0.0008	0.5692 $\pm$0.0076	0.6337 $\pm$0.0017	0.1802 $\pm$0.0084	0.6299 $\pm$0.0007	0.1964 $\pm$0.0002
TRIP-R	0.8506 $\pm$0.0012	0.2116 $\pm$0.0011	0.7731 $\pm$0.0008	0.5653 $\pm$0.0024	0.6331 $\pm$0.0013	0.1753 $\pm$0.0087	0.6292 $\pm$0.0014	0.1958 $\pm$0.0009

Factorization Width. FRE benefits from sufficient rank-specific capacity, but larger $K$ is not uniformly better. On TAAC, mean AUC rises from 0.6095 at $K = 1$ to 0.6340 at $K = 10$, then declines at $K = 16$. KuaiRec is much less sensitive, with AUC ranging from 0.8502 to 0.8508 across the tested widths; on KuaiRand, $K = 2$ already approaches the best result. These differences favor a compact, dataset-adaptive factorization: shared basis vectors capture useful ordinal structure, while the required capacity for distinguishing ranks varies across histories.

Temporal Resolution. Finer target-relative coordinates improve temporal discrimination on KuaiRand: increasing $m$ from 1 to 8 raises mean AUC from 0.7711 to 0.7781. In contrast, TAAC and TaobaoAd attain their best results at $m = 1$, and KuaiRec shows only small fluctuations. Thus, increasing resolution is useful when finer distinctions help history reading, but does not yield universal gains. Here, $m$ subdivides TRT’s logarithmic time scale without changing TGCI’s temporal grouping, separating the granularity of temporal reading from that of group-level interaction.

Density-Correction Strength. All four datasets achieve their highest observed AUC with nonzero $\lambda_{\mathrm{cnt}}$. The peaks occur at 0.5 on KuaiRec, TAAC, and TaobaoAd, and at 0.75 on KuaiRand. For example, KuaiRec improves from 0.8488 without correction to 0.8510 at 0.5. Moderate correction counteracts the attention-mass advantage of densely populated intervals, allowing temporal preference and content relevance to guide reading. Further increasing the penalty brings no consistent benefit, indicating that occupancy should be corrected without excessively suppressing dense but potentially informative history.

FRE Branch Scales. The shared-offset scale has a comparatively small effect: the AUC range across its tested values is below 0.001 on every dataset. Rank-specific variation is more consequential, particularly on TAAC and KuaiRand. Setting the configured rank-branch scale to zero yields AUCs of 0.6068 and 0.7669, compared with 0.6340 and 0.7777 at configuration scale one. Unlike a shared offset, the rank-specific branch distinguishes ordinal contexts. These results support FRE’s separation of shared structure and rank-dependent variation, with the latter providing the essential positional distinctions on these datasets.

Masking and Reconstruction. The masking-ratio sweeps peak at $\rho= 0.1$, 0.05, 0.15, and 0.3 on KuaiRec, KuaiRand, TAAC, and TaobaoAd, respectively. Raising $\rho$ to 0.5 reduces AUC relative to these peaks, particularly on KuaiRec and TAAC. Masking encourages contextual recovery, but excessive removal leaves less item content for both reconstruction and CTR prediction. The reconstruction-weight sweeps similarly favor balanced supervision: $\lambda_{\mathrm{rec}} = 0.01$ gives the highest AUC on KuaiRand, TAAC, and TaobaoAd, whereas KuaiRec peaks at 0.05. Increasing the weight to 0.05 reduces AUC on the former three datasets. Together, these patterns favor auxiliary supervision that strengthens contextual learning without overwhelming the prediction objective.

KuaiRand Probability Calibration

Probability diagnostics on KuaiRand (seed 42) show that TRIP-B predicts a mean positive-feedback probability of 46.74%, above the observed rate of 37.67%. This bias is particularly pronounced in $tab=0$, which contains 19.03% of test samples: its observed positive rate is 11.64%, compared with a mean prediction of 36.05%. This scenario contributes 69.93% of the sample-weighted net LogLoss gap relative to MixFormer. Such probability overestimation increases the loss on negative examples, despite improved ranking in the dominant scenario. Positive-slope affine logit calibration fitted only on validation data reduces TRIP-B’s LogLoss from 0.5778 to 0.5471 without changing AUC. Applying validation-fitted calibration to both models reduces TRIP-B’s LogLoss gap to MixFormer from 0.0300 to 0.0011 while preserving its AUC advantage. This suggests that the higher uncalibrated LogLoss largely reflects correctable probability miscalibration rather than weaker ranking discrimination.