betaFeedback

WorksComputer Science

PEETS: Preference-aware Embeddings with Explicit Tree Structure for Auxiliary Product Bundling

Abstract

Auxiliary products such as flight protection and flexible-change services are recommended alongside primary travel purchases. This setting creates a granularity ambiguity: SKUs within one product type share semantics but differ in price and coverage, while rejecting one offer does not imply disinterest in the entire type. We present PEETS (Preference-aware Embeddings with Explicit Tree Structure), a training-time framework for auxiliary product bundling. PEETS regularizes an existing click–conversion predictor through two conditional comparisons. Hierarchical Contrastive Loss uses the pair's common ancestor as a reference; Negative Contrastive Loss conditions structural sampling and comparison shifts on logged rejection events. Both act on shared SKU representations during training. This separates catalog relatedness from individual preference supervision and leaves the predictive serving path intact. PEETS achieves 78.20% AUC, exceeding TWIN V2 by 0.90 percentage points and the matched base model by 1.15 points. In a 14-day online evaluation, gross auxiliary GMV per exposed user-day improves by 8.20% over Policies, 3.06% over Base, and 0.84% over TWIN V2, with all three user-bootstrap 95% intervals above zero. Bundling rate is 0.51 points below Policies. The cancellation interval's upper limit exceeds the prespecified margin. PEETS adds no auxiliary-loss computation to serving and records 42.1~ms p95 latency, compared with 42.0~ms for Base.

Introduction

Travel purchases involve choosing an itinerary, an auxiliary product type, and an offer with specific price and coverage. Ranking must distinguish catalog-related offers that serve different needs. Figure 1 shows the selection and replacement interfaces. Here, auxiliary product bundling ranks eligible add-ons for a selected itinerary; joint flight–add-on optimization is outside our scope.

Schematic auxiliary-product interfaces

Figure 1. Schematic auxiliary-product interfaces: (a) bundling, (b) details, and (c) replacement offers.

We call this difficulty granularity ambiguity. An auxiliary product type, or xitem, groups related stock keeping units (SKUs), but its SKUs are not interchangeable: rejecting limited coverage can coexist with interest in a comprehensive offer. Representations must preserve shared semantics and decision-relevant distinctions, while distinguishing explicit rejection from an unchosen offer.

PEETS uses catalog structure to condition training comparisons while retaining clicks and purchases as preference supervision. Hierarchical Contrastive Loss (HCL) references a pair’s type-specific common ancestor; Negative Contrastive Loss (NCL) conditions comparisons on logged rejections and sampled reference buckets. Both update shared SKU representations without changing the predictive serving path.

We specify these similarities, sampling rules, and gradient paths, evaluate matched-backbone and NCL component controls, and assess hierarchy sensitivity and online commercial outcomes.

Related Work

Recommendation and Multi-task Learning

Feature interaction and behavior modeling are central to industrial recommendation. Wide & Deep [1] combines linear and nonlinear features; DIN [2] and DIEN [3] model target-related interest and its evolution. SIM [4], TWIN [5], and TWIN V2 [6] address long-history retrieval and interest extraction. SRP4CTR [7] transfers sequential pre-training into CTR prediction. Flight-itinerary ranking [8] and nearby-flight recommendation [9] provide related travel applications, with different candidate spaces and prediction targets.

ESMM [10] models click–conversion dependencies, while MMoE [11] and PLE [12] control information sharing across tasks. AITM [13], HoME [14], and DTN [15] further study task dependencies and interactions. These architectures are relevant backbones or comparators. A comparison with them alone, however, cannot isolate the effect of the tree or the contrastive objectives; matched-backbone ablations are necessary.

Hierarchical and Contrastive Representations

Contrastive learning shapes representations through positive and negative comparisons [16], [17]. Supervised and label-aware variants exploit additional relational information [18, 19, 20]. Hierarchical methods include multi-label penalties [21], hierarchical prototypes [22], and category-aware query classification [23]. HAKG [24] incorporates hierarchical knowledge into recommendation. Related work also studies cross-domain matching [25], positive selection in sequential recommendation [26], and inventory-add prediction at Fliggy [27].

PEETS places hierarchical references inside the contrastive similarity: HCL conditions on the SKU pair, and NCL on a rejection event. We therefore compare established recommenders with backbone-matched controls receiving the same catalog information and ordinary contrastive supervision.

Problem Formulation

Decision Scope and Feedback

Let u∈Uu \in\mathcal{U} denote a user and q=(o,d,t)q=(o,d,t) an origin–destination–departure-time query. The information available before a recommendation consists of the query, historical behavior Hu\mathcal{H}_u, and context C\mathcal{C}. Let I(q)\mathcal{I}(q) be the eligible auxiliary SKUs for the selected itinerary. Each i∈I(q)i \in\mathcal{I}(q) belongs to an xitem c(i)c(i) and has catalog attributes, including price and coverage. Eligibility and availability are determined before ranking.

A training record distinguishes offer exposure, an xitem click, and a SKU purchase, with labels yiclicky_i^{\mathrm{click}} and yibuyy_i^{\mathrm{buy}}. Explicit rejection is a separately logged action, such as replacing or removing a displayed offer. Non-purchase alone is insufficient. The event definitions and attribution windows appear in § 7-A1; they define which interactions can supervise prediction and which can trigger rejection-conditioned regularization.

The hierarchy is a rooted tree with SKU leaves and attribute-bucket internal nodes. Shared ancestors encode relatedness, not identical user preference: rejecting one SKU does not label every descendant of its ancestor as undesirable.

Prediction and Commercial Ranking

For the fixed itinerary, the commercial ranking objective combines relevance and price. Let piclickp_i^{\mathrm{click}} be the probability of clicking the relevant product type and pibuy∣clickp_i^{\mathrm{buy|click}} the conditional probability of buying SKU ii. If purchase follows that click event, an expected gross transaction-value score is

pGMVi=piclickpibuy∣clickPricei.(1)\mathrm{pGMV}_i=p_i^{\mathrm{click}}p_i^{\mathrm{buy|click}}\mathrm{Price}_i. \tag*{(1)}

This quantity is distinct from net platform revenue, which can depend on commissions, refunds, and settlement rules. The revenue definition used in the online evaluation is specified in § 7-D.

Bundling rate measures the number of auxiliary units purchased per primary ticket purchased within the same observation window:

BR=auxiliary units purchasedprimary tickets purchased.(2)\mathrm{BR}=\frac{\text{auxiliary units purchased}}{\text{primary tickets purchased}}. \tag*{(2)}

When multiple auxiliary units can accompany one ticket, this is a unit ratio rather than the fraction of tickets with at least one attachment. The same denominator is used across experiment arms.

A price-weighted serving rule then trades off purchase probability and transaction value. We write the log-price surrogate as

Scorei=pc(i)clickpibuy∣click[log⁡(Pricei/CNY)]α,(3)\mathrm{Score}_i=p_{c(i)}^{\mathrm{click}}p_i^{\mathrm{buy|click}}\left[\log(\mathrm{Price}_i/\mathrm{CNY})\right]^\alpha, \tag*{(3)}

where c(i)c(i) is the xitem containing SKU ii and α≥0\alpha\ge0. Prices are expressed in CNY, and the logarithm takes the dimensionless ratio Pricei/CNY>1\mathrm{Price}_i/\mathrm{CNY}>1. (3) is a ranking surrogate, not (1). Model comparisons hold its configuration fixed, while a separate experiment studies α\alpha.

PEETS

Overview

Figure 2(a) shows the predictive and auxiliary branches. The predictive branch contains xitem-click and SKU-conversion heads. The auxiliary branch compares shared SKU representations under hierarchical and rejection-conditioned transformations. Bucket lookup and contrastive sampling are training operations; online scoring uses the predictive heads and the configured ranking rule.

Figure 2. PEETS framework. (a) Predictor and training-only auxiliary branch; dashed links denote sharing and training dependencies. (b) Simplified hierarchy (five levels in experiments). (c) Ancestor-conditioned comparisons. (d) Rejected and reference buckets; structural pairs are not preference labels. (e) Rejection-conditioned comparisons; “bad bucket” is event-specific. In (c,e), ordinary comparisons are on the left. Here bb denotes a bucket and L1 the root. Geometries are schematic, not measured projections.

Candidate-aware Prediction

The inputs contain five groups: user profile, itinerary query, historical behavior, candidate product, and context. Let their embeddings be eu,eq,eHu,ea,ece_u,e_q,e_{\mathcal{H}_u},e_a,e_c, where candidate aa denotes either an xitem or a SKU. The candidate-specific input is

xa=eu⊕eq⊕eHu⊕ea⊕ec.(4)x_a=e_u\oplus e_q\oplus e_{\mathcal{H}_u}\oplus e_a\oplus e_c. \tag*{(4)}

Target-attention backbones also condition the history encoder on aa. The embeddings follow standard dense representation learning [28]; the experimental backbone and shared layer are a shared MLP with 256/128 hidden units and the 128-dimensional SKU embedding table.

For task t∈{xitem,sku}t \in\{\text{xitem},\text{sku}\}, let hth_t denote its encoder and σ\sigma the sigmoid function. The prediction is

y^t,a=σ(wt⊤ht(xa)+bt).(5)\hat{y}_{t,a}=\sigma\left(w_t^\top h_t(x_a)+b_t\right). \tag*{(5)}

The figure’s pXitemSku is the product of these outputs. Auxiliary gradients reach the candidate path through the shared SKU representation in Equation (4); no auxiliary features are added at serving time.

Let St\mathcal{S}_t be the eligible labeled records for task tt. The normalized binary cross-entropy is

Lt=−1∣St∣∑n∈St[ynlog⁡y^n+(1−yn)log⁡(1−y^n)].(6)\mathcal{L}_t=-\frac{1}{|\mathcal{S}_t|}\sum_{n\in\mathcal{S}_t}\left[y_n\log\hat{y}_n+(1-y_n)\log(1-\hat{y}_n)\right]. \tag*{(6)}

The conditional conversion head is trained on exposed records with an xitem click, without sampling or propensity weighting. A clicked-but-unpurchased record is positive for the click task and negative for conditional conversion; the two tasks therefore use separate labels and eligibility masks.

Reproducible Bucket Hierarchy

Figure 2(b) shows the rooted tree T=(V,E)\mathcal{T} = (V, \mathcal{E}): SKU leaves and internal type/attribute nodes, each non-root node having one parent. The root is level one. The SKU-to-xitem mapping, attribute order, interval boundaries, and parent map determine assignment and ancestor lookup.

The experimental hierarchy uses training-catalog price quantiles, attribute order xitem, coarse price, fine price, SKU, and maximum level 5. Its reference number of leaf-level buckets is 15, counted per xitem. Boundaries are estimated from the training-period catalog; duplicate values and interval endpoints follow merged duplicate cut points and right-closed intervals.

Adjacent fine price intervals merge into coarser intervals; categories follow the catalog taxonomy. Each bucket has an independently trainable embedding, distinct from its descendant SKU vectors. HCL uses the bucket parameter; NCL separately computes the rejected bucket’s SKU mean.

The bucket count is selected using only the validation set; the test set is reserved for final evaluation. Tree boundaries use the training catalog only, and each checkpoint records its tree version.

Hierarchical Contrastive Loss

For a SKU anchor ii, let p(i)≠ip(i) \ne i be a structural positive drawn from the designated bucket. Let Ni\mathcal{N}_i contain distinct negative candidates excluding ii and p(i)p(i), and let Ai={p(i)}∪Ni\mathcal{A}_i = \{p(i)\} \cup\mathcal{N}_i. With a similarity function ss and temperature τ>0\tau> 0, define

ℓi(s)=−log⁡exp⁡(s(ei,ep(i))/τ)∑j∈Aiexp⁡(s(ei,ej)/τ).(7)\ell_i(s) = - \log\frac{\exp(s(e_i,e_{p(i)})/\tau)}{\sum_{j \in\mathcal{A}_i} \exp(s(e_i,e_j)/\tau)}. \tag*{(7)}

The positive is sampled uniformly within the finest bucket and included in the denominator. Negative candidates comprise 128 distinct in-batch SKUs after excluding the anchor and positive; singleton anchors are skipped. Same-bucket membership expresses structural relatedness, not an observed click or purchase label.

Let b−(i,j)b^{-}(i,j) be the common xitem ancestor and b+(i,j)b^{+}(i,j) the deepest common price-bucket ancestor. If the required ancestor type is absent, that component uses the root embedding. These type-specific references belong to the same configured tree. With ε>0\varepsilon> 0 for numerical stability, write

cos⁡ε(a,b)=aTbmax⁡(∥a∥2∥b∥2,ε).(8)\cos_{\varepsilon}(a,b) = \frac{a^{\mathsf{T}}b}{\max(\lVert a\rVert_2\lVert b\rVert_2,\varepsilon)}. \tag*{(8)}

The two transformed similarities are

s±(ei,ej)=cos⁡ε(ei±eb±(i,j),ej±eb±(i,j)).(9)s^{\pm}(e_i,e_j) = \cos_{\varepsilon}(e_i \pm e_{b^{\pm}(i,j)}, e_j \pm e_{b^{\pm}(i,j)}). \tag*{(9)}

The subtraction form re-centers embeddings around shared category information; the addition form emphasizes a common attribute reference. As illustrated in Figure 2(c), the transformation depends on the pair’s structural relationship rather than treating all comparisons identically. For a valid anchor set B\mathcal{B}, each HCL component is

L±=1∣B∣∑i∈Bℓi(s±).(10)\mathcal{L}^{\pm}=\frac{1}{|\mathcal{B}|}\sum_{i\in\mathcal{B}}\ell_i(s^{\pm}). \tag*{(10)}

The xitem component uses subtraction and the price component addition, with LH=L−+L+\mathcal{L}_{H}=\mathcal{L}^{-}+\mathcal{L}^{+}. Each component uses its type-specific reference above; an empty valid-anchor set contributes zero.

For either component’s selected reference ebe_b, write ei=eb+rie_i=e_b+r_i and ej=eb+rje_j=e_b+r_j. The transformed comparison is then between residuals rir_i and rjr_j; addition instead compares 2eb+ri2e_b+r_i with 2eb+rj2e_b+r_j. When ancestors differ across pairs, a single global embedding bias cannot generally absorb the transformation. These are origin-dependent angular comparisons, not projections or guarantees of semantic disentanglement. The sign choice is an inductive bias whose value is assessed against ordinary contrastive learning and bucket-feature controls.

Rejection-conditioned Contrastive Loss

An explicit rejection belongs to a user decision, not permanently to a product. Write an observed rejection as r=(u,q,kr−)r=(u,q,k_r^{-}), where kr−k_r^{-} is the rejected displayed SKU. A subsequently selected replacement is recorded separately.

Define bˉr=parent⁡(kr−)\bar{b}_r=\operatorname{parent}(k_r^{-}) and sample a reference bucket brrefb_r^{\mathrm{ref}} from

{b:level⁡(b)=level⁡(bˉr), kr−∉leaves⁡(b)}.(11)\{b:\operatorname{level}(b)=\operatorname{level}(\bar{b}_r),\ k_r^{-}\notin\operatorname{leaves}(b)\}. \tag*{(11)}

The reference bucket is sampled uniformly at the same level, excluding the rejected bucket. A sampled bucket is a structural reference, not evidence that its unexposed offers are preferred by the user.

Let Pr\mathcal{P}_r contain the valid anchor–positive pairs for event rr. Anchors are sampled uniformly within the reference bucket, each with one same-bucket positive and 128 negative SKUs. The anchor ii is distinct from the rejected SKU kr−k_r^{-}; in Figure 2(d), the pair lies in the reference bucket.

We write the rejected-bucket aggregate and the shift as

eneg(r)=sg⁡(∑k∈leaves⁡(bˉr)ek∣leaves⁡(bˉr)∣),e_{\mathrm{neg}}(r)=\operatorname{sg}\left(\frac{\sum_{k\in\operatorname{leaves}(\bar{b}_r)}e_k}{|\operatorname{leaves}(\bar{b}_r)|}\right),
Δr=ebˉr−ebrref−eneg(r).(12)\Delta_r=e_{\bar{b}_r}-e_{b_r^{\mathrm{ref}}}-e_{\mathrm{neg}}(r). \tag*{(12)}

Here sg⁡\operatorname{sg} denotes stop-gradient: the SKU mean is recomputed each batch, but this aggregate does not backpropagate into its descendant vectors. For any comparison attached to the same event, the similarity is

sr′(ei,ej)=cos⁡ε(ei+Δr,ej+Δr).(13)s'_r(e_i,e_j)=\cos_{\varepsilon}(e_i+\Delta_r,e_j+\Delta_r). \tag*{(13)}

Let ℓr,i\ell_{r,i} denote (7) with the positive and candidate set for event rr. The event-normalized objective is

LN=1∣R∣∑r∈R1∣Pr∣∑(i,p)∈Prℓr,i(sr′).(14)\mathcal{L}_{N}=\frac{1}{|\mathcal{R}|}\sum_{r\in\mathcal{R}}\frac{1}{|\mathcal{P}_r|}\sum_{(i,p)\in\mathcal{P}_r}\ell_{r,i}(s'_r). \tag*{(14)}

Here R\mathcal{R} contains eligible rejection events with valid pairs. If a batch has no such event, its NCL contribution is defined as zero. The implementation averages anchors within each event, then averages across events.

Figure 2(e) illustrates the rejection-conditioned comparison.

Gradient paths. The trainable ebr−e_{b_r^{-}} and detached eneg(r)e_{\mathrm{neg}}(r) do not algebraically cancel. Shifted similarities update compared SKU vectors and bucket parameters; only the aggregate is detached, not every use of its descendant SKU vectors.

Geometry and conditioning. NCL changes comparison angles, not the pair’s Euclidean difference: ∥(ei+Δr)−(ej+Δr)∥2=∥ei−ej∥2\|(e_i+\Delta_r)-(e_j+\Delta_r)\|_2=\|e_i-e_j\|_2. NCL conditions shared representation learning on rejection-bucket statistics; user- and query-specific prediction remains in the supervised branch. False-negative correction [29] is not part of this structural sampler.

Joint Objective and Training

With the overall scale fixed to one, the joint objective is

L=Lxitem+rLsku+βLH+γLN.(15)\mathcal{L}=\mathcal{L}_{\mathrm{xitem}}+r\mathcal{L}_{\mathrm{sku}}+\beta\mathcal{L}_{H}+\gamma\mathcal{L}_{N}. \tag*{(15)}

Weight comparisons hold the optimizer and HCL aggregation fixed.

Training computes predictions and task masks, constructs structural and rejection-conditioned pairs under a fixed tree version, and updates shared parameters using the normalized losses. Serving retains the candidate encoder and predictive heads while omitting pair construction, contrastive ancestor lookup, and bucket aggregation. End-to-end latency still depends on the deployed encoder and feature pipeline.

Computational cost. With BB anchors, MM comparisons per anchor, dimension dd, and tree height hh, transformed similarities cost O(BMd)O(BMd) beyond the predictive encoder. Walking parent paths for ancestor lookup adds up to O(BMh)O(BMh) work. The rejected-SKU mean costs O(∣leaves⁡(bˉr)∣d)O(|\operatorname{leaves}(\bar{b}_r)|d) per uncached bucket and is recomputed from current embeddings each batch with stop-gradient. Table VIII reports training and serving costs separately.

MethodTrain timeMeanp50p95
Policies–21.83.2
GBDT34m 19s1413.221.6
WDL58m 28s2726.142.8
Base67m 50s2625.442.0
PLE73m 28s2726.543.7
TWIN V277m 33s2625.542.3
PEETS73m 30s2625.542.1

Table VIII. Training and serving efficiency on 11M records. Neural models train for 3 epochs; latencies are in ms.

Gradient and sampling controls. Comparisons are identified by event, anchor, positive, candidate set, and tree version. The ablations share pair construction, seed policy, and training budget. Each HCL anchor has 129 candidates (one positive and 128 negatives); similarity computation uses L2 normalization and ε=10−8\varepsilon=10^{-8}, with gradient norms capped at 5. Temperature τ\tau scales logits and does not replace norm stabilization.

Experiments

The evaluation tests ranking gains beyond identical catalog features, sensitivity to hierarchy and rejection sampling, and online value alongside user experience.

Evaluation Setup

  1. Data and temporal protocol. The industrial data come from Fliggy’s auxiliary-selection workflow, collected from June 1 to October 1, 2025. The test set covers October 1, 2025; the exact training and validation windows are June 1–September 28, 2025 and September 29–30, 2025. Historical features and catalog snapshots use information available before prediction time. The records cover 4.2 million users, 960

SKUs, and 6 xitems; repeated sessions follow one record per displayed SKU per session.

Table I reports sample and event counts. Positives are clicked-and-booked products; negatives are the remaining sampled records. Click and purchase attribution windows are 30 minutes and 24 hours. Explicit replacement or removal after exposure yields 0.48 million rejection events across the splits. Negative-to-positive ratios are 10:1, 6.5:1, and 17:1 for train/validation/test. Unexposed offers are not treated as observed non-clicks.

RecordsTrainValid.Test
All samples11M1.5M1.8M
Positive1M0.2M0.1M
Negative10M1.3M1.7M
Exposures11M1.5M1.8M
Clicks3.30M0.46M0.52M
Purchases1.00M0.20M0.10M
Rejections0.36M0.05M0.07M

Table I. DATASET STATISTICS (M: MILLION). POSITIVE SAMPLES DENOTE CLICKED-AND-BOOKED PRODUCTS; TASK-EVENT COUNTS ARE REPORTED SEPARATELY.

2) Baselines and implementation.: The PEETS project URL is https://github.com/BOTAK0803/PEETS. The data are de-identified, and data use and online experimentation were internally authorized by the company. Raw industrial data cannot be publicly released.

The comparison set includes price-ordering rules and the production Policies control; LR [30], GBDT [31], WDL, DIN, and ECAU-Net [32]; ESMM, MMoE, and PLE; HAKG; SRP4CTR; and TWIN/TWIN V2. Baselines use public implementations adapted to the auxiliary SKU task. Cheapest and Priciest are evaluated on their respective policy-induced exposure sets, so their AUCs are contextual results rather than reversed rankings on one common test set.

Learned-model comparisons share the candidate pool, labels, temporal split, and available raw features, with at most 30 validation trials per method. Neural models use matched embedding dimensions and prediction-task definitions; these settings do not apply to rule or tree-based methods. Matched neural controls use 20 history events, while TWIN retains native retrieval. Price-rule exposure sets are excluded from claims about common-test-set performance.

3) Metrics and optimization.: We report AUC [33] and request-level NDCG@10/@15 [34] using binary purchase labels and the probability score pclickpbuy∣clickp^{\mathrm{click}p^{\mathrm{buy}}\mid\mathrm{click}}, without price weighting. Each request ranks its eligible displayed SKUs and may contain multiple purchased items. Let nqn_q be its candidate count, mqm_q its number of positives, and πq(j)\pi_q(j) the item at rank jj. For requests Q+Q_+ with mq>0m_q > 0,

NDCG⁡@K=1∣Q+∣∑q∈Q+∑j=1min⁡(K,nq)yq,πq(j)/log⁡2(j+1)∑j=1min⁡(K,mq)1/log⁡2(j+1)(16)\operatorname{NDCG}@K = \frac{1}{|Q_+|} \sum_{q \in Q_+} \frac{\sum_{j=1}^{\min(K,n_q)} y_{q,\pi_q(j)} / \log_2(j+1)}{\sum_{j=1}^{\min(K,m_q)} 1 / \log_2(j+1)} \tag*{(16)}

Requests without a positive item are excluded from NDCG but retained for AUC.

PEETS and the neural controls use Adam [35] with learning rate 0.0001, batch size 512, and 3 training epochs. PEETS uses dimension 128, temperature 0.07, and history limit 20. Learned-model results average 5 independent training seeds: 42, 1024, 2024, 2025, and 2026. Table II gives AUC and NDCG@10 standard deviations for Base, TWIN V2, and PEETS. Validation selects checkpoints by AUC, breaking ties by NDCG@10.

Loss-weight selection. Relative weights are r=1r = 1, β=0.05\beta= 0.05, and γ=0.02\gamma= 0.02, with scale c=1c = 1. The validation search uses r∈{0.5,1,2}r \in\{0.5,1,2\}; β,γ∈{0.005,0.01,0.02,0.05,0.1}\beta,\gamma\in\{0.005,0.01,0.02,0.05,0.1\} and selection procedure 3 task-ratio trials followed by 25 auxiliary-weight trials, holding loss normalization, backbone, and training budget fixed.

Offline Evaluation

1) Overall comparison.: Table 2 compares predictive-ranking performance. PEETS attains the highest AUC, NDCG@10, and NDCG@15 among the learned models evaluated on the common test set. Relative to TWIN V2, the strongest comparator on all three metrics, it improves AUC by 0.90 percentage points and NDCG@10 by 0.63% relative. The respective NDCG@15 values are 0.9667 and 0.9598. The AUC gap exceeds the observed seed SDs of 0.038 and 0.045 points for TWIN V2 and PEETS. These SDs summarize training variation on the fixed test set. Matched-backbone controls further isolate the auxiliary objectives.

GroupMethodN@10N@15AUCGroupMethodN@10N@15AUC
RulesCheapest†0.84350.914677.15Multi-taskESMM0.88010.951275.77
Priciest†0.83570.912073.73MMoE0.88240.953676.12
Policies0.85580.923177.17PLE0.88450.955676.45
PredictiveLR0.85630.930175.71HierarchyHAKG0.87500.942176.20
GBDT0.86610.931676.30Pre-trainingSRP4CTR0.87900.946876.50
WDL0.86970.933176.07Two-stageTWIN0.88620.954576.95
DIN0.87150.938076.15TWIN V20.89000.959877.30
ECAU-Net0.87320.941376.28OursPEETS0.89560.966778.20
AUC (mean ± SD)Base 77.05 ± 0.041; TWIN V2 77.30 ± 0.038; PEETS 78.20 ± 0.045.
N@10 (mean ± SD)Base 0.8774 ± 0.0012; TWIN V2 0.8900 ± 0.0009; PEETS 0.8956 ± 0.0010.

Table 2. Offline comparison. N@10/N@15 denote NDCG@10/NDCG@15. Learned-model rows report five-seed means; the bottom rows show variability for three key comparators. AUC is in percent and its SD in percentage points. †Price-rule rows use distinct exposure sets, unlike common-test-set learned models.

Figure 3 contrasts accuracy with commercial outcomes for comparators shared by the offline and online tables. PEETS has the highest AUC and GMV per exposed user-day among these rows, while its bundling rate is above Policies. Thus, value uplift should not be interpreted as an attachment-rate improvement.

Figure 3. Selected results from Tables II and VI. GMV per exposed user-day change is relative to Policies; BR change is in percentage points (pp). The panels show point estimates under their respective evaluation settings; overall GMV intervals appear in § 7-D.

2) Matched-backbone ablation.: Table 3 compares bucket features, ordinary contrastive learning, HCL, and NCL on the same predictive backbone. PEETS improves AUC by 1.15 percentage points and NDCG@10 by 2.07% relative to Base. Its AUC exceeds the bucket-feature and ordinary-CL controls by 0.88 and 0.74 points, respectively. Thus the observed benefit extends beyond adding catalog features. All ablations share data splits, raw inputs, catalog information, the MLP, training epochs, and seeds. Derived bucket features and auxiliary objectives are enabled only in the named variants. Ordinary CL retains the HCL/NCL pair samplers, loss reductions, and weights, replacing their transformed similarities with ordinary cosine similarity.

VariantN@10N@15AUC (%)
Base0.87740.948477.05
Base + bucket features0.88210.953877.32
Base + ordinary CL0.88450.955677.46
Base + NCL0.88920.960177.72
Base + HCL0.89180.962577.89
PEETS0.89560.966778.20
NCL component controlN@10AUC (%)
HCL + NCL, no shift0.892977.98
HCL + NCL, no mean0.892777.97

Table 3. Matched-backbone five-seed means. Top: feature and objective controls. Bottom: NCL component controls, retaining HCL and the original NCL sampling while changing only the shift.

Both single-component variants improve all three metrics over Base. HCL reaches 77.89% AUC and NCL reaches 77.72%; combining them yields 78.20%, a further 0.31 and 0.48 points, respectively. The combination has the highest means on all three metrics. Figure 4 shows the same component AUCs at 15 buckets.

Figure 4. Bucket-count sensitivity. PEETS includes validation and test AUC at seven settings; HCL/NCL are shown at five test settings. Base is a fixed reference. Lines connect the evaluated settings.

3) Price-policy sensitivity.: Table 4 varies α\alpha over the same 14-day window with equal exposure across settings and a fixed predictive model. GMV increases across the listed α\alpha values, reaching CNY 173260 at α=6\alpha= 6, 22.25% above α=0\alpha= 0. Over the same comparison, tickets rise by 0.79% while BR falls by 0.91 percentage points. BR first declines and then recovers, so the trade-off is not monotonic in every outcome. Across this sweep, the highest GMV accompanies a lower BR than at α=0\alpha= 0.

αTicketsBR (%)Revenue
0869734.23141725
1869034.11143457
2848232.17153671
3856032.55160420
4862532.84165800
5869833.12170150
6876633.32173260

Table 4. Price-policy sensitivity. Revenue is gross auxiliary GMV in CNY. The sweep varies the exposure penalty α.

4) Hierarchy and rejection sensitivity. Figure 4 shows AUC across 5–30 buckets. PEETS peaks at 15 buckets,

selected using validation AUC (78.28%) before final test evaluation (78.20%). Nearby settings at 12 and 18 buckets reach 78.01% and 78.05% test AUC. The 10/20-bucket results are 77.62%/77.58%. At 5 and 30 buckets, AUC falls to 76.52% and 76.45%, below Base’s 77.05%. The tested 12–18-bucket settings remain above Base, whereas 5 and 30 buckets fall below it.

For 15 buckets, multiplying each internal quantile probability by an independent U(0.95,1.05)U(0.95, 1.05) draw, sorting, and rebuilding boundaries yields 78.16% validation and 78.08% test AUC. Equal-width bins yield 78.10%/78.02%. These two settings lower test AUC by 0.12 and 0.18 points relative to the default quantile partition.

The additional controls in Table III fix HCL, rejection events, anchor–positive–negative pairs, reductions, weights, and seeds. Removing the SKU mean gives Δr=ebr−ebref\Delta_r = e_{b_r} - e_{b_{\mathrm{ref}}} and 77.97% AUC; setting Δr=0\Delta_r = 0 gives 77.98%. Full PEETS exceeds them by 0.23 and 0.22 points, with NDCG@10 gains of 0.0029 and 0.0027. No shift retains the NCL loss and its rejection-conditioned sampling; unlike ordinary CL, it preserves transformed HCL. These comparisons report differences between the five-seed means.

Targeted Revenue Analysis

Table V reports PEETS’s relative improvement in GMV per request over TWIN V2 on selected online subsets. For each arm and subset, GMV is divided by its request count before computing the relative change. The Requests column totals requests from both arms. The ambiguous-candidate subset is defined by requests containing at least 3 SKUs of one xitem using pre-outcome information. Low- and high-frequency groups use the prior-activity threshold at most 2 versus more than 2 interactions in 30 days. The 95% intervals use 1000 bootstrap resamples of randomized users and the percentile method, keeping each user’s requests together.

SubsetRequestsΔ\Delta (%)95% CI
Ambiguous350000+3.6[3.12,4.08]
Low-frequency420000+4.2[3.65,4.75]
High-frequency980000+1.1[0.82,1.38]

Table 5. SELECTED ONLINE SUBSETS: RELATIVE CHANGE IN GMV PER REQUEST OVER TWIN V2, WITH USER-BOOTSTRAP 95% INTERVALS. REQUESTS ARE POOLED ACROSS THE TWO ARMS.

All three subset intervals lie above zero. Low- and high-frequency groups total 1.4 million filtered requests. Ambiguity is defined separately from activity, so its request count is reported separately. Subset results use GMV per request, while the overall comparison uses GMV per exposed user-day.

Online Evaluation

The online experiment spans 14 days, with user-level randomized assignment to seven arms. Mean daily exposed-user counts range from 20371 to 20442 across arms. The experiment uses 10% of traffic from September 1 to September 14, 2025. Exposure is counted per user-day; Table VI reports the daily mean. Policies is the production reference. The initial checkpoint uses data through August 31, 2025, with daily updates using records available by the preceding day. The October offline retrospective evaluation uses separately trained models.

Revenue is gross auxiliary GMV in CNY before refunds. Table VI gives daily means over 14 days. GMV per exposed user-day is mean daily GMV divided by mean daily exposed users. Because exposures differ across arms, its relative change differs from that of daily total GMV. Users may recur across days. Sample-ratio and cross-arm contamination checks found no anomalies.

ArmExposedTicketsBR (%)Revenue
Policies20382856133.83159764
WDL20435860233.27164378
ESMM20419862733.25164635
HAKG20371861133.22169922
TWIN V220442867133.32171927
Base20395864033.28167850
PEETS20428876633.32173260

Table 6. ONLINE RESULTS: PER-ARM DAILY MEANS OVER 14 DAYS. REVENUE IS GROSS AUXILIARY GMV IN CNY BEFORE REFUNDS.

PEETS improves GMV per exposed user-day by 8.20% over Policies (95% CI [6.84, 10.12]), 3.06% over Base ([1.85, 4.61]), and 0.84% over TWIN V2 ([0.16, 1.41]). Each interval uses 1000 within-arm user bootstrap resamples, retaining all 14 days per sampled user and recomputing total GMV divided by exposed user-days before the relative comparison. Percentile limits are 2.5% and 97.5%. All three pointwise intervals exclude zero. Unnormalized daily GMV increases by 8.45%, 3.22%, and 0.78%, respectively. Relative to Policies, tickets rise by 2.39% while BR falls by 0.51 points. The absolute daily GMV difference is CNY 13496 per arm in the measured traffic.

User-experience guardrails. Table VII compares PEETS with Policies: auxiliary-tagged complaints per exposed user, all-cause cancellations per booked primary ticket, and abandonment per initiated checkout. Post-booking outcomes use seven-day follow-up. Differences are treatment minus control, so positive values indicate deterioration. The 95% intervals use 1000 user-level bootstrap resamples and the percentile method. The non-inferiority criterion is an upper confidence limit below the acceptable increase specified before the experiment.

MetricCtrl.PEETSΔ\Delta95% CI (Δ\Delta)Margin
Complaints0.120.13+0.01[-0.009,0.028]0.03
Cancellations1.801.83+0.03[-0.076,0.137]0.10
Abandonment18.6018.55-0.05[-0.251,0.152]0.20

Table VII. User-experience guardrails. Rates are in percent; treatment-minus-control differences, intervals, and margins are in percentage points.

The upper limits for complaints (0.028) and abandonment (0.152) are below their respective margins of 0.03 and 0.20 points. The cancellation interval’s upper limit is 0.137 points, above the 0.10-point margin; the prespecified cancellation non-inferiority criterion is not met.

Session-Level Diagnostic

We retrospectively score a logged Beijing–Singapore session with both models; only the served policy’s feedback is observed. The fixed insurance candidates A–D cost CNY 30, 60, 68, and 25: basic accident, premium accident plus delay, comprehensive travel, and delay-only coverage. The user removed A and purchased B.

For A–D, the probability scores are 0.25, 0.15, 0.12, 0.28 (Base) and 0.04, 0.34, 0.31, 0.11 (PEETS). Equation (3) with the fixed α=6\alpha= 6 yields Base’s order B>C>A>D and PEETS’s C>B>D>A. In this replay, B moves from third to first in probability rank and from first to second in price-weighted rank.

Deployment and Practical Considerations

PEETS trains on Alibaba’s AI Platform (PAI) and serves through Real-Time Prediction (RTP) and Personalization Platform (TPP). Table VIII reports training time and server-side latency from request receipt to response, including feature processing and model computation. The configuration uses 8 A10 training workers and 16-vCPU serving instances with FP32 and batch size 1.

Catalog versions. Model checkpoints record their tree versions. Lookup failure uses the previous valid model and catalog snapshot.

Cost and serving audit. PEETS adds 5m 40s (8.35%) to Base’s training time. Both record 26 ms mean latency, while PEETS’s p50 and p95 are 0.1 ms higher. Latency is measured over the same 14-day window at 300 requests/s per serving instance.

Conclusion

PEETS combines ancestor-conditioned and rejection-conditioned training with the existing serving path. Five-seed results and controlled NCL comparisons support the observed ranking gains; the tested 12/18-bucket and alternative partitions remain within 0.19 points of the default test AUC. GMV per exposed user-day improves by 8.20%, 3.06%, and 0.84% over Policies, Base, and TWIN V2, with positive user-bootstrap intervals. PEETS records 42.1 ms p95 latency under the measured load. Its BR is below Policies, and the cancellation interval’s upper limit exceeds the prespecified margin.

References

  1. [1]H.-T. Cheng, L. Koc, J. Harmsen, T. Shaked, T. Chandra, H. Aradhye, G. Anderson, G. Corrado, W. Chai, M. Ispir, R. Anil, Z. Haque, L. Hong, V. Jain, X. Liu, and H. Shah, “Wide & Deep learning for recommender systems,” in Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, 2016, pp. 7–10.arxiv.org/abs/1606.07792
  2. [2]G. Zhou, X. Zhu, C. Song, Y. Fan, H. Zhu, X. Ma, Y. Yan, J. Jin, H. Li, and K. Gai, “Deep interest network for click-through rate prediction,” in Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, 2018, pp. 1059–1068.arxiv.org/abs/1706.06978
  3. [3]G. Zhou, N. Mou, Y. Fan, Q. Pi, W. Bian, C. Zhou, X. Zhu, and K. Gai, “Deep interest evolution network for click-through rate prediction,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, pp. 5941–5948, 2019.arxiv.org/abs/1809.03672
  4. [4]Q. Pi, G. Zhou, Y. Zhang, Z. Wang, L. Ren, Y. Fan, X. Zhu, and K. Gai, “Search-based user interest modeling with lifelong sequential behavior data for click-through rate prediction,” in Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020, pp. 2685–2692.arxiv.org/abs/2006.05639
  5. [5]J. Chang, C. Zhang, Z. Fu, X. Zhang, L. Guan, J. Lu, Y. Hui, D. Leng, Y. Niu, Y. Song, and K. Gai, “TWIN: Two-stage interest network for lifelong user behavior modeling in CTR prediction at Kuaishou,” in Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023, pp. 3785–3794.arxiv.org/abs/2302.02352
  6. [6]Z. Si, L. Guan, Z. Sun, X. Zhang, J. Lu, Y. Hui, X. Cao, Z. Yang, Y. Zheng, D. Leng, K. Zheng, C. Zhang, Y. Niu, Y. Song, and K. Gai, “TWIN V2: Scaling ultra-long user behavior sequence modeling for enhanced CTR prediction at Kuaishou,” in Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, 2024, pp. 4890–4898.DOI
  7. [7]R. Han, Q. Li, H. Jiang, R. Li, Y. Zhao, X. Li, and W. Lin, “Enhancing CTR prediction through sequential recommendation pre-training: Introducing the SRP4CTR framework,” in Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, 2024, pp. 3777–3781.DOI
  8. [8]J. Huang, Y. Li, S. Sun, B. Zhang, and J. Huang, “Personalized flight itinerary ranking at Fliggy,” in Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020, pp. 2541–2545.DOI
  9. [9]M. Huang, D. Lv, Y. Yu, S. Song, D. Li, and Z. Zhuang, “A context based personalized deep network for nearby flight recommendation,” in Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, 2025, pp. 4533–4542.
  10. [10]X. Ma, L. Zhao, G. Huang, K. Zhang, Z. Hu, X. Zhu, and K. Gai, “Entire space multi-task model: An effective approach for estimating post-click conversion rate,” in The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, 2018, pp. 1137–1140.
  11. [11]J. Ma, Z. Zhao, X. Yi, J. Chen, L. Hong, and E. H. Chi, “Modeling task relationships in multi-task learning with multi-gate mixture-of-experts,” in Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, 2018, pp. 1930–1939.DOI
  12. [12]H. Tang, J. Liu, M. Zhao, and X. Gong, “Progressive layered extraction (PLE): A novel multi-task learning (MTL) model for personalized recommendations,” in Proceedings of the 14th ACM Conference on Recommender Systems, 2020, pp. 269–278.
  13. [13]D. Xi, Z. Chen, P. Yan, Y. Zhang, Y. Zhu, F. Zhuang, and Y. Chen, “Modeling the sequential dependence among audience multi-step conversions with multi-task learning in targeted display advertising,” in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021, pp. 3745–3755.arxiv.org/abs/2105.08489
  14. [14]X. Wang, J. Cao, Z. Fu, K. Gai, and G. Zhou, “HoME: Hierarchy of multi-gate experts for multi-task learning at Kuaishou,” in Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1, 2025, pp. 2638–2647.arxiv.org/abs/2408.05430
  15. [15]Y. Bi, Y. Lian, J. Cui, J. Liu, P. Wang, G. Li, X. Chen, J. Zhao, H. Wen, J. Zhang, Z. Zhang, W. Song, Y. Sun, W. Zhang, M. Cai, J. Dong, and G. Zhang, “DTN: Deep multiple task-specific feature interactions network for multi-task recommendation,” arXiv preprint arXiv:2408.11611, 2024.arxiv.org/abs/2408.11611
  16. [16]T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in Proceedings of the 37th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 119. PMLR, 2020, pp. 1597–1607.arxiv.org/abs/2002.05709
  17. [17]T. Wang and P. Isola, “Understanding contrastive representation learning through alignment and uniformity on the hypersphere,” in Proceedings of the 37th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 119. PMLR, 2020, pp. 9929–9939.arxiv.org/abs/2005.10242
  18. [18]P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, “Supervised contrastive learning,” in Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 18661–18673.arxiv.org/abs/2004.11362
  19. [19]L. Pan, C.-W. Hang, A. Sil, and S. Potdar, “Improved text classification via contrastive adversarial training,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 10, pp. 11130–11138, 2022.arxiv.org/abs/2107.10137
  20. [20]H. Liu, F. Zhang, X. Zhang, S. Zhao, J. Sun, H. Yu, and X. Zhang, “Label-enhanced prototypical network with contrastive learning for multi-label few-shot aspect category detection,” in Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, 2022, pp. 1079–1087.arxiv.org/abs/2206.13980
  21. [21]S. Zhang, R. Xu, C. Xiong, and C. Ramaiah, “Use all the labels: A hierarchical multi-label contrastive learning framework,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 16660–16669.arxiv.org/abs/2204.13207
  22. [22]Y. Guo, M. Xu, J. Li, B. Ni, X. Zhu, Z. Sun, and Y. Xu, “HCSC: Hierarchical contrastive selective coding,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 9706–9715.arxiv.org/abs/2202.00455
  23. [23]L. Zhu, K. Zhang, H. Chen, C. Wei, W. Zhang, H. Tang, and X. Li, “HCLA4QC: Incorporating hierarchical category structures into contrastive learning for e-commerce query classification,” in Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023, pp. 3647–3656.DOI
  24. [24]Y. Du, X. Zhu, L. Chen, B. Zheng, and Y. Gao, “HAKG: Hierarchy-aware knowledge gated network for recommendation,” in Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022, pp. 1390–1400.arxiv.org/abs/2204.04959
  25. [25]H. Chang, L. Gu, C. Hu, Z. Zhang, H. Zhu, Y. Xu, Y. Fang, and Z. Chen, “Separated contrastive learning for matching in cross-domain recommendation with curriculum scheduling,” in Companion Proceedings of the ACM on Web Conference 2025, 2025, pp. 114–123.DOI
  26. [26]Z. Wang, Y. Shen, Z. Zhang, L. He, Y. Li, H. Gu, and Y. Zhang, “Relative contrastive learning for sequential recommendation with similarity-based positive sample selection,” in Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, 2024, pp. 2493–2502.DOI
  27. [27]M. Li, D. Lv, Y. Yu, and Z. Jiao, “Contrastive learning for inventory add prediction at Fliggy,” in Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1, 2025, pp. 2383–2392.DOI
  28. [28]T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” in Advances in Neural Information Processing Systems, vol. 26, 2013.arxiv.org/abs/1310.4546
  29. [29]C.-Y. Chuang, J. W. Robinson, Y.-C. Lin, A. Torralba, and S. Jegelka, “Debiased contrastive learning,” in Advances in Neural Information Processing Systems, vol. 33. Curran Associates, Inc., 2020, pp. 8765–8775.arxiv.org/abs/2007.00224
  30. [30]M. P. LaValley, “Logistic regression,” Circulation, vol. 117, no. 18, pp. 2395–2399, 2008.
  31. [31]G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, “LightGBM: A highly efficient gradient boosting decision tree,” in Advances in Neural Information Processing Systems, vol. 30, 2017, pp. 3146–3154.
  32. [32]W. Tao, H. Liu, X. Li, Q. Dai, H. Wen, and Z. Chen, “Event-aware adaptive clustering uplift network for insurance creative ranking,” in Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2023, pp. 1966–1970.DOI
  33. [33]T. Fawcett, “An introduction to ROC analysis,” Pattern Recognition Letters, vol. 27, no. 8, pp. 861–874, 2006.DOI
  34. [34]Y. Wang, L. Wang, Y. Li, D. He, and T.-Y. Liu, “A theoretical analysis of NDCG type ranking measures,” in Proceedings of the 26th Annual Conference on Learning Theory, ser. Proceedings of Machine Learning Research, vol. 30. PMLR, 2013, pp. 25–54.arxiv.org/abs/1304.6480
  35. [35]D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations, 2015. [Online]. Available: https://arxiv.org/abs/1412.6980

Paper details

vlnc:W_grbafjf2
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents