WorksComputer Science
ID2Pattern: Pretraining Generalizable Identity Behavior Patterns for Transaction Risk Foundation Models
Abstract
Large-scale behavioral pretraining offers a promising route to generalizable representations in online payment systems. However, high-cardinality identity fields such as card, device, and account IDs, though rich in behavioral signals, remain underexploited. Their continually growing vocabularies and limited semantic meaning make existing vocabulary-based and semantics-based encodings ill-suited. Their generalizable signal lies not in specific values, but in the identity behavior patterns they exhibit across a transaction sequence. Building on this insight, we propose ID2Pattern, a pretraining framework that brings these patterns into transaction foundation models. ID2Pattern first encodes identity fields with a Local Anonymous ID (LAID) tokenizer, which captures how each identity appears and recurs within the sequence, independent of its specific value. It then pretrains a multi-scale pattern codebook on unlabeled sequences, which abstracts LAID sequences into discrete behavior patterns for downstream prediction. Experiments on an industrial dataset with over 100M sequences and six public datasets show that ID2Pattern consistently outperforms strong baselines, improving industrial by 4.13% and public datasets by 8.83% on average. Notably, ID2Pattern remains equally effective even when all identity values are unseen during training, highlighting its potential to generalize to new users and markets.

Figure 1. Identity fields vs. semantic attributes on our industrial payment data: (a) continual vocabulary growth; (b) limited semantic meaning.
Introduction
Behavioral pretraining offers a promising route to transaction foundation models by learning generalizable representations from massive interaction histories. As online payment becomes core infrastructure of the Web economy, payment and e-commerce platforms on the Web continuously accumulate longitudinal interactions among users, payment instruments, merchants, and other entities, providing rich behavioral signals beyond individual transactions [16, 23]. Recent studies have therefore pretrained behavioral models on large-scale event sequences and adapted them to downstream tasks such as fraud detection and risk assessment [2, 18, 23]. These models characterize each transaction event by semantic attributes with relatively stable vocabularies, such as payment types and regions, and learn temporal dependencies across events.
Beyond semantic attributes, identity fields such as card, device, and account IDs also carry rich behavioral signals, yet remain underexploited. These fields characterize users’ transaction habits, such as which cards and devices they regularly use, and are therefore critical for risk management in payment systems. However, as illustrated in Figure 1, identity fields exhibit two characteristics that distinguish them from semantic attributes. First, continual vocabulary growth: identity fields are high-cardinality, with long-tailed vocabularies that keep expanding as transactions accumulate, whereas semantic vocabularies stabilize. Second, limited semantic meaning: identity values are arbitrary identifiers rather than interpretable categories like payment types or regions.
These characteristics make existing encoding methods in behavioral sequence models ill-suited to identity fields. Such methods fall into two categories: vocabulary-based and semantics-based. Vocabulary-based methods learn an embedding for each identity value [29, 40]. They rely on a fixed vocabulary and generalize poorly to unseen identities, while hashing variants introduce collisions [17, 30]. Semantics-based methods encode entity content with pretrained text models [11, 25]. They avoid vocabulary growth, but identity fields carry little content to encode. Consequently, neither category effectively represents identity fields.
For high-cardinality identity fields, the behavior signal lies not in specific identity values, but in the identity behavior patterns they exhibit across a transaction sequence. For example, when a user who has long paid with the same card and device suddenly switches to a new device, the change may indicate account takeover, regardless of the specific device ID. Such patterns are precisely the behavioral signals that payment risk systems seek from identity fields. Capturing these patterns from continually growing and semantically opaque identity fields is therefore essential for transaction foundation models to generalize across scenarios and downstream tasks. However, learning these patterns raises two key challenges:
Value-independent identity representation. How can each identity be represented by its relative identity within the sequence, rather than by its specific value?
Sequence-level pattern abstraction. How can event-level relative identities be abstracted into discrete behavior patterns across multiple temporal ranges?
To address these challenges, we propose ID2PATTERN, a pretraining framework that brings identity behavior patterns into transaction foundation models. Specifically, ID2PATTERN encodes identity fields as relative identities and pretrains on large-scale unlabeled transaction sequences to abstract them into discrete behavior patterns. For the first challenge, we propose the Local Anonymous ID (LAID) tokenizer, which represents each identity by how it appears and recurs within the sequence, yielding an encoding invariant to specific identity values. For the second challenge, we design a multi-scale pattern codebook, which abstracts LAID sequences over different temporal ranges into discrete behavior patterns. Experiments on our 100M-scale industrial payment-risk dataset and public financial and commerce datasets show that ID2PATTERN improves 4.13% on the industrial data and 8.83% on average across the public datasets. Notably, ID2PATTERN performs consistently even when all identity values are entirely unseen, demonstrating its potential to generalize to new users and markets. In summary, the main contributions of our work are as follows:
We identify identity behavior patterns, rather than specific identity values, as the generalizable signal in identity fields, and propose ID2PATTERN to bring them into transaction foundation models.
We propose the LAID tokenizer, which encodes identity fields as relative identities, and a multi-scale pattern codebook, which abstracts them into discrete behavior patterns through self-supervised pretraining.
Experiments on our industrial dataset with over 100M sequences and six public datasets show consistent improvements and stable performance on unseen identity values.
Related Work
Transaction Risk Modeling
Transaction risk models have moved from hand-crafted features toward sequential and graph-based learning. Early systems feed manually designed statistics, such as velocity features, into tabular learners such as gradient-boosted trees [3, 24]. Later work models transaction histories directly with sequential networks [16, 42], or connects accounts, devices, and merchants into graphs and applies graph neural networks [6, 20, 21, 32, 36]. Graph methods are where identity fields are used most directly, since shared cards and devices define the edges. The cost is a global entity graph that must be built and kept up to date as new entities appear. Our method avoids this by reading identity patterns from each user’s own sequence.
Pretraining Behavioral Foundation Models
Pretraining on large-scale behavior sequences has become a common route to foundation models outside language. In recommendation, BERT4Rec [29] and HSTU [41] train sequence models with masked and generative objectives, while industrial systems use pretrained knowledge for CTR prediction [8], user targeting [4], and generative retrieval [10]. Financial applications follow the same trend, from tabular and contrastive pretraining on transaction sequences [2, 23] and pretrained models over blockchain transactions [13] to transaction foundation models built for payment fraud detection [5, 18, 22, 39, 43]. In these models, however, card and device IDs are typically encoded like any other categorical attribute. Their continually growing vocabularies and limited semantic meaning receive no special treatment, and this is the gap we address.
High-Cardinality ID Representation
How to represent high-cardinality IDs has been studied mostly in recommendation, where item and user IDs are as arbitrary as transaction identities [40]. The standard approach learns one embedding per ID [29, 41], and hashing [30, 38], compositional embeddings [27], deep hash embeddings [17], and frequency-aware compression [12] cut the memory cost. These variants still tie each representation to a specific ID value. Semantic IDs [11, 25, 28] take another route and quantize item content into shared codes, but this works only because items have titles, images, and descriptions; a card number has none. The idea closest to ours comes from graph learning, where anonymous walks and related encodings [15, 19, 37] describe nodes by their position in a local structure rather than by their identities. We carry this relative view over to identity fields in transaction sequences and go one step further, abstracting relative identities into discrete behavior patterns through multi-scale pretraining.
Preliminaries
Transaction Events and Identity Fields. Each transaction event consists of identity fields, semantic attributes, and numerical features. We write the -th event as , where collects the values of the identity fields , such as card, device, and account IDs. Each value identifies the entity associated with field in the event. The vocabulary grows as new entities emerge, and its values are arbitrary identifiers with little intrinsic meaning.
Identity Encoding Paradigms. Approaches to encoding an identity value fall into three paradigms, which differ in what the embedding depends on. Vocabulary-based encoding depends on the value itself:
where maps an ID to one of rows of the embedding table . A one-to-one vocabulary lets grow with and maps unseen IDs to a shared out-of-vocabulary row, while hashing fixes at the cost of collisions; in both cases, what the model learns about one ID does not transfer to another ID that behaves the same way. Semantics-based encoding depends on the content associated with the ID:
where returns its textual description or attributes and is a semantic encoder. Opaque identifiers such as card and device IDs, however, carry little such content. Behavior-based encoding depends only on how the identity relates to others in the sequence:
where records which events up to share the same identity. Since is unchanged under any one-to-one relabeling of identity values, such encodings apply equally to unseen IDs. Hand-crafted statistics such as velocity features are predefined choices of , whereas ID2Pattern learns through LAID and multi-scale pretraining. Appendix A compares these paradigms in detail.
Problem Formulation. For each target transaction, we construct a chronological event sequence , where is the target and is its history. The task is to predict its risk label using only information available up to time :
Methodology
Overview
To capture identity behavior patterns from high-cardinality identity fields, we propose ID2Pattern, which converts raw identity values into discrete behavior patterns and models them together with transaction events. As illustrated in Figure 2, ID2Pattern consists of two modules. The LAID tokenizer encodes each identity by its relative identity within the sequence, yielding a fixed vocabulary that is invariant to specific identity values. The multi-scale pattern codebook then abstracts LAID sequences over different windows into discrete pattern tokens, and is pretrained on unlabeled sequences by recovering masked states. Finally, the pattern tokens are combined with event tokens for downstream risk prediction.
Figure 2. Overview of ID2Pattern. (A) Pattern tokens and event tokens are jointly modeled for risk prediction. (B) The LAID tokenizer encodes each identity by its relative identity within the sequence. (C) The multi-scale pattern codebook quantizes LAID sequences over short, medium, and long windows into discrete pattern tokens.
Local Anonymous Identity Tokenizer
The LAID tokenizer replaces each identity value with a token describing how the identity appears and recurs among other identities in the sequence. Each identity field is processed independently. For event , let
denote the position where the current identity first appears. The identity is then assigned a local ID
namely its order of first appearance in the sequence.
The local ID tells which events share the same identity, but not how the identity behaves over time. We therefore add three behavior descriptors that record its novelty, occurrence, and recurrence:
Here, indicates whether the identity appears for the first time, counts its occurrences so far, and is the number of events since its last occurrence, set to 0 for a new identity. The four channels form the LAID token .
Relabeling Invariance. All four channels are computed from the equality relations and event positions alone, so LAID is a behavior-based encoding in the sense of Eq. (3). Hence, for any one-to-one relabeling of the values of field ,
Multi-Scale Pattern Codebook
A sequence whose identity values are all unseen in training thus receives the same LAID tokens as a seen sequence with the same identity behavior. Since all channels are bounded by the sequence length, the LAID vocabulary stays fixed as new entities emerge.
The behavior of an identity depends on the temporal range in which it is viewed: a device that is new within the last few transactions may be familiar over the whole history. We therefore view each sequence at three scales , corresponding to short, medium, and long windows. For scale with window length , the window keeps the most recent events,
and LAID is recomputed within the window, , so the same identity can receive different LAID tokens at different scales.
Behavior sequence encoding. Within each window, the four channels of every LAID token are embedded separately and summed over channels and identity fields, giving an event-level behavior embedding
where indexes the four channels and is a channel-specific embedding table. A Transformer encoder [35] shared across scales encodes the behavior sequence of each window, and its output at the current event, denoted , summarizes the identity behavior within scale .
Pattern abstraction. The continuous representation still varies from sequence to sequence. To abstract it into a finite set of behavior patterns shared across users, each scale maintains its own codebook , and is replaced by its nearest code:
denotes the pattern token of scale . The three tokens summarize identity behavior from complementary temporal ranges.
Self-Supervised Pattern Pretraining
To learn reusable behavior patterns without labels, we pretrain the pattern branch, i.e., the behavior embeddings, pattern encoder, and codebooks, on training-period sequences by recovering the hidden identity states of the current transaction. This exploits the correlation among identities within a transaction: whether a device is new is closely related to whether the accompanying card is familiar.
Masked identity-state recovery. For each sequence, we mask the LAID tokens of a random nonempty field subset at in all three windows. With the whole token masked, including its local ID, the model must infer these states from how the other identities of relate to the history. For each scale and field , a field-specific head predicts the novelty , occurrence , and recurrence from , where is the position of in , so each code must capture the behavior needed for recovery.
Pretraining objective. Novelty is binary and optimized with binary cross-entropy, while occurrence and recurrence are heavy-tailed and regressed in log space, :
where and denote predictions in log space. The recovery loss averages over masked fields and scales:
The codebooks are trained with the VQ-VAE objective [34], using straight-through gradients for the nearest-code selection:
where is the stop-gradient operator and weights the commitment term. The pretraining loss is , plus a code-balancing regularizer against codebook collapse (Appendix § 10). The recovery head is then discarded, and the pattern branch is transferred to downstream modeling.
Pattern-Augmented Downstream Modeling
To exploit the pretrained patterns in risk prediction, we combine the pattern tokens with event tokens that retain transaction-level details.
Event representation. ID2PATTERN encodes every field of an event through a finite vocabulary. Each semantic attribute has a stable vocabulary and is embedded by its own table, . Each identity value is instead replaced by its LAID token computed over the full sequence , whose bounded channels form fixed vocabularies:
Numerical features are linearly projected, , and the event token concatenates and projects all field embeddings:
This event branch keeps transaction-level details, such as amounts and categories, that the pattern tokens do not capture.
Pattern-augmented prediction. The pattern tokens are prepended to the event sequence and modeled jointly by a Transformer backbone:
where Readout takes the output at the target event . The model is trained with binary cross-entropy between and . The event encoder, backbone, and head are randomly initialized, while the pattern branch is initialized from pretraining and fine-tuned jointly; the LAID tokenizer is deterministic and shared by both stages.
Complexity
ID2PATTERN adds a constant-factor overhead to the backbone and is independent of the number of identities. The LAID tokenizer computes the four channels in a single pass over each window with a hash map, taking time; Algorithm 1 in Appendix C summarizes the full procedure. The shared pattern encoder costs , dominated by the long window, and the codebook lookup costs . The backbone processes only three additional tokens, so its cost remains . None of these terms depends on the vocabulary size , which keeps both the computation and the parameters of identity representation fixed as new entities emerge.
Experiments
We evaluate the effectiveness of ID2Pattern through extensive experiments, guided by the following research questions:
RQ1: How effective is ID2Pattern compared with strong baselines on industrial and public benchmarks?
RQ2: How does each component of ID2Pattern contribute to its overall effectiveness?
RQ3: How robust is ID2Pattern to entirely unseen identities, and what behavior patterns do the codebooks learn?
RQ4: Do the learned identity behavior patterns generalize across different backbones and transaction datasets?
Experimental Setup
Datasets. We evaluate ID2Pattern on one industrial payment-risk dataset and six public datasets covering financial risk and commerce behavior. The industrial dataset contains over 100M transaction sequences from a real-world payment platform with nine identity fields, such as card, device, and account IDs, and is split chronologically into training, validation, and test periods. The public financial-risk datasets are CCT [23], IBM AML [1], and Elliptic++ [7], which label each transaction as fraudulent, laundering, or illicit. The commerce datasets are Taobao [33], RetailRocket [26], and Synerise [31], which test whether identity behavior patterns remain useful beyond risk detection: Taobao and RetailRocket predict whether the current event is a purchase, and Synerise predicts whether an add-to-cart event converts into a purchase. Table 1 summarizes the statistics, and Appendix § 9 details the preprocessing.
| Dataset | Domain | #Sequences | #ID Fields | Pos. Rate |
| Industrial | Risk | 100.0M+ | 9 | – |
| Elliptic++ | Risk | 37.3K | 2 | 5.69% |
| IBM AML | Risk | 1.54M | 2 | 0.20% |
| CCT | Risk | 7.44M | 2 | 0.12% |
| RetailRocket | Commerce | 540.4K | 1 | 2.78% |
| Synerise | Commerce | 6.12M | 2 | 24.44% |
| Taobao | Commerce | 39.26M | 2 | 1.88% |
Table 1. Statistics of evaluated datasets. #Sequences counts the supervised targets, each with its own history sequence; the positive rate is computed on the test set.
Baselines. Industrial evaluation. We use the existing internal production sequence foundation model as the baseline, which encodes semantic attributes and numerical features but no identity fields, denoted as Semantic-attribute-only in Table 2. To isolate the effect of identity representation, we add identity fields to this model through hashed embeddings (+ Hash ID) or LAID (+ LAID), and ID2Pattern further adds the pretrained pattern tokens. The relabeled variants evaluate the same trained models after replacing every test-set identity value with an unseen one, while preserving which events share the same identity.
| Method | AUC-ROC (%) | T1 (%) | AddCB (%) |
| Semantic-attribute-only | — | — | — |
| + Hash ID | −5.87 | −11.91 | −8.07 |
| + Hash ID (relabeled) | −8.00 | −50.53 | −46.96 |
| + LAID | +1.11 | +6.96 | +6.09 |
| + LAID (relabeled) | +1.11 | +6.96 | +6.09 |
| ID2Pattern | +4.13 | +22.64 | +19.67 |
Table 2. Relative performance changes on the industrial payment-risk test set, using the semantic-attribute-only baseline as the reference.
Public benchmarks. We compare ID2Pattern with three groups of baselines: (1) sequential models, including LSTM, GRU, and Transformer; (2) tabular and boosting models, including XGBoost [3], CatBoost [24], and Temporal Boosting [9]; and (3) pretraining-based and specialized models, including BERT4Rec [29], TabBERT [23], HSTU [41], PANTHER [18], IBM AI-on-Z GRU [14], NVIDIA TFM [39], FRiskGPT [13], and PRAGMA [22]. All baselines are trained on our datasets, and Appendix § 11 provides further details.
Metrics and implementation. On the public datasets, we report PR-AUC as the primary metric and [email protected]%, the recall at a false positive rate of at most 0.5% on the test set. On the industrial dataset, we report AUC-ROC and two business metrics: T1, the recall among the top 1% highest-scored transactions, and AddCB, the additional chargeback recall among transactions not intercepted by the existing risk system. ID2Pattern uses windows of 7, 30, and events with codes per scale, and each public experiment is repeated with three random seeds. Appendix § 9.§ 2 details the industrial metrics and online protocol, and Appendix § 10 lists the remaining hyperparameters.
Industrial Payment-Risk Evaluation (RQ1)
We first evaluate ID2Pattern on industrial payment risk through offline experiments on over 100M transaction sequences and a two-month online shadow evaluation on live traffic. Offline evaluation. As illustrated in Table 2, ID2Pattern achieves the best results on all three metrics, improving AUC-ROC, T1, and
AddCB by 4.13%, 22.64%, and 19.67% over the semantic-attribute-only model. The controlled variants show that these gains depend on how identity fields are represented. Adding identity fields through hashing degrades all metrics below the reference model, and relabeling test identities with unseen values further reduces its T1 by 43.8%, revealing its reliance on specific identity values. In contrast, LAID improves over the reference model and remains unchanged under relabeling, and the pattern tokens bring further gains on top of LAID.
Online shadow evaluation. We further evaluate ID2Pattern in shadow mode on live production traffic from July to August. Since risk labels such as chargebacks arrive weeks after transactions, we compute the metrics after the labels mature rather than measuring real-time business gains (Appendix B.2). As shown in Figure 3, the offline gains persist on live traffic: ID2Pattern achieves the highest AUC on every day, and the variants keep the same order as offline, with LAID above hashing and the pattern tokens adding a consistent margin over LAID.
Figure 3. Online shadow evaluation from July to August. AUC is normalized by the average AUC of the semantic-attribute-only model over the first three days.
Public Benchmark Evaluation (RQ1)
To evaluate whether the gains of ID2Pattern generalize beyond our platform, we compare it with 14 baselines on three financial-risk and three commerce datasets. As shown in Table 3, we make three observations.
Table 3: Performance on six public datasets in terms of PR-AUC (PR) and recall at a false positive rate of at most 0.5% ([email protected]%). Best results are in bold, and second-best results are underlined.
| Risk | Commerce | |||||||||||
| Method | Elliptic++ | IBM AML | CCT | RetailRocket | Synerise | Taobao | ||||||
| PR | [email protected]% | PR | [email protected]% | PR | [email protected]% | PR | [email protected]% | PR | [email protected]% | PR | [email protected]% | |
| Sequential Models | ||||||||||||
| LSTM | .2410 | .0697 | .2014 | .3436 | .0318 | .0665 | .7255 | .5168 | .4411 | .0341 | .2078 | .1528 |
| GRU | .2428 | .0860 | .1600 | .3594 | .0315 | .0675 | .7358 | .5685 | .4487 | .0337 | .2102 | .1553 |
| Transformer | .2333 | .0540 | .2129 | .3800 | .0303 | .0873 | .7518 | .5695 | .4138 | .0214 | .2026 | .1480 |
| Tabular / Boosting Models | ||||||||||||
| XGBoost | .1691 | .0409 | .1269 | .4393 | .0097 | .0287 | .6398 | .4841 | .4382 | .0276 | .2368 | .1656 |
| CatBoost | .1851 | .0456 | .1481 | .3973 | .0158 | .0439 | .6476 | .5096 | .4373 | .0273 | .2012 | .1405 |
| Temporal Boosting | .1614 | .0566 | .0826 | .0233 | .0130 | .0164 | .5962 | .4636 | .4251 | .0242 | .2078 | .1507 |
| Pretrained / Foundation Models | ||||||||||||
| BERT4Rec | .2414 | .0781 | .2398 | .4280 | .0562 | .2040 | .7651 | .6239 | .4142 | .0230 | .2029 | .1482 |
| TabBERT | .2125 | .0613 | .1316 | .2264 | .0375 | .1513 | .7604 | .6088 | .3950 | .0194 | .2260 | .1471 |
| HSTU | _ .2489_ | _ .0917_ | .0955 | .2962 | .0228 | .0581 | .7608 | .6148 | .4176 | .0230 | .2050 | .1514 |
| PANTHER | .1963 | .0697 | .2543 | .1486 | .0250 | .0569 | .6629 | .5793 | .4471 | .0266 | .1837 | .1178 |
| IBM AI-on-Z GRU | .2381 | .0786 | _ .2588_ | .3902 | .0291 | .0595 | .7472 | .5710 | .3551 | .0260 | .1932 | .1441 |
| NVIDIA TFM | .1622 | .0225 | .1430 | .2711 | .0483 | _ .2044_ | .7531 | .5999 | .4198 | .0221 | .1988 | .1443 |
| FRiskGPT | .1553 | .0142 | .1982 | .3673 | .0487 | .1903 | _ .7698_ | _ .6239_ | .4227 | .0230 | .2024 | .1473 |
| PRAGMA | .2409 | .0204 | .1774 | .1067 | .0197 | .0339 | .6181 | .4262 | .3476 | .0167 | .2245 | .1570 |
| ID2Pattern | .2758 | .1001 | .2761 | .4951 | .0607 | .2161 | .8283 | .6903 | .4776 | .0464 | .2686 | .1845 |
Table 3. Performance on six public datasets in terms of PR-AUC (PR) and recall at a false positive rate of at most 0.5% ([email protected]%). Best results are in bold, and second-best results are underlined.
Overall performance. First, ID2Pattern consistently outperforms all baselines, while no baseline is consistently strong. We achieve the best PR-AUC and [email protected]% on all six datasets, improving PR-AUC by 8.83% on average over the strongest baseline on each dataset. The strongest baseline differs on every dataset, and generic pretraining alone is insufficient: HSTU performs well on Elliptic++ but falls below GRU and CatBoost on IBM AML. These results suggest that the gains of ID2Pattern come from how identity fields are represented rather than from pretraining or model capacity.
Financial-risk tasks. Second, ID2Pattern is effective at ranking the riskiest transactions. On the three risk datasets, it improves PR-AUC by 6.7–10.8% and [email protected]% by 5.7–12.7% over the strongest baseline, where [email protected]% reflects how many risky transactions are caught while at most 0.5% of legitimate ones are flagged. Tabular models, which summarize the history into aggregated statistics, fall far behind on CCT, suggesting that the order in which identities are reused or switched carries risk signals that aggregation discards.
Commerce tasks. Third, identity behavior patterns remain useful beyond risk detection. On the three commerce datasets, ID2Pattern improves PR-AUC by 6.4–13.4% over the strongest baseline, with the largest gain on Taobao. Purchase intent is often revealed by repeated visits to the same item within a short period, as shown by the revisit pattern on Taobao in Section 5.5, which the short-window codes of ID2Pattern capture directly.
Ablation Study (RQ2)
To understand how each component contributes, we compare our method with six variants on two risk and two commerce datasets. w/o LAID replaces LAID with hash-based identity encoding; Local ID only keeps only the local-ID channel of LAID; w/o Pattern Branch removes the pattern tokens and keeps only the event branch; Single-scale uses only the long window; w/o Codebook feeds continuous representations instead of discrete pattern tokens; and w/o Pretraining trains the full model from scratch.
As shown in Table 4, every component contributes, and LAID contributes the most. Removing any component lowers PR-AUC on all four datasets, and replacing LAID with hashing causes the largest average drop, reaching 44.3% on CCT. Even within LAID, the novelty, occurrence, and recurrence channels improve over local IDs alone on all four datasets. The pattern branch, multi-scale windows, codebook, and pretraining each bring further gains on top of LAID. Notably, discrete pattern tokens outperform their continuous counterparts on all four datasets, which supports abstracting identity behavior into a finite set of shared patterns rather than keeping sequence-specific representations.
| Ablation Variants | Risk | Risk | Commerce | Commerce |
| CCT | IBM AML | RetailRocket | Synerise | |
| w/o LAID | .0338 | .2018 | .7814 | .4228 |
| Local ID Only | .0600 | .2695 | .8152 | .4718 |
| w/o Pattern Branch | .0418 | .1880 | .8064 | .4734 |
| Single-scale | .0526 | .2263 | .8087 | .4700 |
| w/o Codebook | .0419 | .2047 | .8092 | .4630 |
| w/o Pretraining | .0451 | .2351 | .8011 | .4561 |
| ID2Pattern | .0607 | .2761 | .8283 | .4776 |
Table 4. Ablation study of different components (PR-AUC).
The pattern branch matters more for risk tasks than for commerce tasks. On IBM AML, removing the pattern branch, multi-scale windows, codebook, or pretraining reduces PR-AUC by 15% to 32%, whereas on RetailRocket and Synerise the drops stay within 5%. In contrast, replacing LAID with hashing still costs 5.7% and 11.5% on these two commerce datasets. This contrast is consistent with our motivation: how cards, devices, and accounts are reused or switched carries strong risk signals that call for multi-scale pattern abstraction, while in commerce the LAID-encoded event branch already captures most of the purchase signal.
Identity Generalization and Pattern Analysis (RQ3)
To evaluate whether ID2Pattern generalizes to new identities and what identity behavior patterns it learns, we conduct a generalization study under identity relabeling and a case study on the learned codebooks.
Generalization to unseen identities. To test whether models rely on specific identity values, we relabel identity values in the test set with values never seen during training, while keeping which events share the same identity unchanged. For this study, we additionally train five representative baselines with hashed identity fields as inputs, keeping all other settings unchanged. We relabel either all identities or a growing fraction of them, from 0% to 100%, and evaluate all models without retraining. As shown in Figure 4, ID2PATTERN keeps exactly the same PR-AUC under every ratio, as guaranteed by the relabeling invariance of LAID. In contrast, the baselines already lose 53.4–87.9% of their PR-AUC on CCT and 4.7–7.4% on Taobao at 25% relabeling; some partly recover at 100% on CCT, most notably HSTU, as no memorized identity remains to conflict with the unseen ones. The drop is larger on CCT, where fraud depends more on whether merchants and cities are familiar.
Figure 4. Generalization to unseen identities on CCT and Taobao. Left: PR-AUC with original and fully relabeled identities. Right: PR-AUC under increasing relabeling ratios.
Case study on learned codebooks. To examine whether the codes carry behavioral meaning, we group test samples by the code assigned at each scale and compare the positive rate of each group with the dataset average. As shown in Figure 5 (left), several codes concentrate positive samples far above the base rate, reaching 24.6× the fraud rate on CCT and 6.2× the purchase rate on Taobao. The most informative scale also differs by task: long-window codes are the strongest on CCT and short-window codes on Taobao, suggesting that fraud deviates from long-term habits while purchases reflect recent intent.
Figure 5. Case study on learned codebooks. Left: positive-rate lift of codes at each scale. Right: multi-scale code combinations describing behavior patterns on CCT and Taobao.
Combining codes across scales further reveals interpretable behavior patterns (Figure 5, right). On CCT, the codes describe a city rare in the long history but accounting for 30% of the last 30 events and 57% of the last 7, followed by a payment to an unseen merchant; this combination has a fraud rate 222× the base rate. On Taobao, they describe an item whose share rises from 4% to 13% and 43% as the window shrinks, followed by its purchase, with a purchase rate 9.1× the base rate. These patterns capture two distinct identity behaviors, emerging novelty and repeated revisits, and depend only on how identities appear relative to the history, not on which city or item is involved.
Representation Generalizability (RQ4)
The identity modeling of ID2Pattern generalizes beyond the architecture and dataset on which it is developed. We examine this property from two aspects: generalization across sequence backbones and transfer across datasets.
Generalization across Backbones. ID2Pattern improves diverse sequence backbones without modifying their architectures. To test this, we attach the identity modeling of ID2Pattern, namely LAID encoding and the pretrained pattern tokens, to four representative backbones, GRU, LSTM, Transformer, and Mamba, and compare each with its original version. As shown in Table 5, ID2Pattern improves PR-AUC in 15 of the 16 backbone–dataset combinations, covering recurrent, attention-based, and state-space architectures. The gains are consistent on the commerce datasets, ranging from 4.6% to 11.4% on RetailRocket and from 26.9% to 32.6% on Taobao. On the risk datasets, the gains are larger but less uniform, ranging from −15.7% to +100.3%, as PR-AUC fluctuates more at the low positive rates of IBM AML and CCT.
| Backbone | IBM AML | CCT | RetailRocket | Taobao |
| GRU | .1600 | .0315 | .7358 | .2102 |
| + ID2Pattern | .2397 | .0345 | .7700 | .2756 |
| Improv. | +49.8% | +9.5% | +4.6% | +31.1% |
| LSTM | .2014 | .0318 | .7255 | .2078 |
| + ID2Pattern | .2543 | .0268 | .7763 | .2678 |
| Improv. | +26.3% | −15.7% | +7.0% | +28.9% |
| Transformer | .2129 | .0303 | .7518 | .2026 |
| + ID2Pattern | .2761 | .0607 | .8283 | .2686 |
| Improv. | +29.7% | +100.3% | +10.2% | +32.6% |
| Mamba | .1999 | .0285 | .7033 | .1991 |
| + ID2Pattern | .2133 | .0390 | .7833 | .2526 |
| Improv. | +6.7% | +36.8% | +11.4% | +26.9% |
Table 5. PR-AUC of different backbones with and without the identity modeling of ID2Pattern, averaged over 3 seeds.
Generalization across Datasets. Pattern encoders pretrained on other datasets transfer to a new dataset and remain as effective as in-domain pretraining. We use the three commerce
datasets, whose events share a similar user–item structure, take each as the target in turn, and pretrain the pattern encoder on the target alone or on the other two datasets before the same fine-tuning on the target. As shown in Table 6, cross-dataset pretraining matches or slightly exceeds in-domain pretraining on all three targets, and both outperform training without pretraining by 3.0–6.0%. The identity behavior patterns learned on some platforms are therefore reusable on others, which allows a single pattern encoder to serve multiple scenarios.
| Pretraining data | Taobao | RetailRocket | Synerise |
| Without pretraining | .2608±.0042 | .8011±.0055 | .4561±.0036 |
| Target only | .2686±.0022 | .8283±.0067 | .4776±.0047 |
| Other two datasets | .2698±.0025 | .8313±.0016 | .4836±.0010 |
Table 6. PR-AUC with the pattern encoder pretrained on the target dataset alone or on the other two commerce datasets.
Efficiency Analysis
Since identity vocabularies keep growing in online payment systems, an identity representation must remain affordable at scale. To evaluate whether LAID meets this, we compare it with ID embeddings and hashing in representation size and inference time.
Representation size. LAID is the only representation whose size does not grow with the number of identities and that never merges distinct identities within a sequence. As shown in Figure 6(a), per-ID embeddings grow linearly with the number of identities, while hashing fixes the size by packing hundreds of identities into each bucket on large datasets. The LAID vocabulary is bounded by the window length, so its size stays constant as new identities emerge.
Figure 6. Efficiency of identity representations. (a) Parameters per identity field; numbers give identities per hash bucket. (b) Inference time per sample.
Inference time. LAID is also cheaper to compute than hashing. As shown in Figure 6(b), including the time for identity encoding, LAID is faster than hashing on all datasets, by up to 3.2× on CCT, and adds little cost over the semantic-attribute-only model. This is because LAID only checks whether identity values are equal within a window, without mapping each value into a global code space.
Conclusion
In this paper, we propose ID2PATTERN, a behavioral pretraining framework that learns reusable patterns from high-cardinality identities without relying on their specific values or semantics. Specifically, the Local Anonymous Identity Tokenizer preserves identity
Ethical Considerations
All identity values in the proprietary dataset were anonymized before modeling, and we report only relative improvements on it to protect commercially sensitive information. The online evaluation was conducted in shadow mode, so model outputs did not affect any real user or transaction. Since LAID represents identities by how they recur rather than by their values, it also reduces the reliance of downstream models on specific personal identifiers.
A Comparison of Identity Encoding Strategies
Identity fields differ from ordinary categorical features in two ways: their vocabularies grow without bound as new cards, devices, and accounts appear, and their raw values carry little semantic meaning. Table 7 compares existing strategies along three properties that follow from these differences: whether model size grows with the number of identities, how unseen identities are handled, and whether identity behavior is represented. LAID is the only strategy that keeps a fixed size, treats unseen identities in the same way as seen ones, and learns identity behavior from data.
| Strategy | Size grows with IDs | Unseen IDs | Identity behavior |
| Vocabulary-based | Linear | OOV vector | Value-tied |
| Hashing | No | Arbitrary code | Value-tied |
| Hand-crafted | No | Supported | Predefined |
| Semantics-based | No | If content | Not captured |
| LAID (ours) | No | Invariant | Learned |
Table 7. Comparison of strategies for encoding high-cardinality identity fields.
Vocabulary-based encoding. Vocabulary-based encoding assigns each identity value its own embedding, as in ID-based sequential models and pretrained models over blockchain addresses [13, 29, 41]. The embedding table grows linearly with the number of identities, and identities absent from training share a single out-of-vocabulary vector. Because each embedding is tied to one value, what the model learns about one card or device does not transfer to another that behaves the same way.
Hashing. Hashing maps identity values into a fixed number of buckets, which bounds memory regardless of vocabulary size [38]. Hash embeddings, compositional embeddings, deep hash embeddings, and frequency-aware compression further reduce collisions or memory [12, 17, 27, 30]. An unseen identity, however, receives an arbitrary code determined by its hash value, and distinct identities may collide into the same code. The learned embeddings thus remain tied to values, and collisions can merge identities that a risk model needs to keep apart.
Hand-crafted statistics. Velocity features and other hand-crafted statistics, such as the number of distinct devices linked to a card within a time window, describe identity behavior independently of raw values and therefore apply to unseen identities. Each statistic, however, is predefined by domain experts, and covering a new behavior requires designing a new feature. LAID can be viewed as a learnable generalization of these statistics: its channels record when and how often each identity appears, and the model learns which combinations matter.
Semantics-based encoding. Semantics-based encoding derives representations from item content, either by quantizing content embeddings into semantic IDs or by encoding descriptions with pretrained text encoders [11, 25, 28, 40]. These representations generalize to unseen items that come with descriptive content. Identity fields such as card numbers, device fingerprints, and IP addresses have no such content, and even when a value can be embedded, its embedding says nothing about how the identity behaves within a sequence.
LAID. LAID carries the value-independent view of anonymous walks and related graph encodings [15, 19, 37] over to transaction sequences. It represents each identity by how it appears and recurs within the sequence, so its vocabulary is bounded by the sequence length, its tokens are invariant to relabeling, and it requires no identity content. Within a sequence, LAID never merges distinct identities, which preserves the sharing structure that hashing may corrupt, and the pattern codebook abstracts this structure into reusable behavior patterns that can be pretrained without labels. Knowledge about specific identities, such as known fraudulent devices, is typically maintained by rule-based blocklists in deployed risk systems; LAID instead covers what such lists cannot, namely the behavior of identities that are new or not yet flagged.
Dataset Details
Public Datasets
Datasets and prediction tasks. We evaluate ID2PATTERN on six public datasets spanning two task families: financial risk detection (CCT, IBM AML, and Elliptic++) and commerce behavior prediction (Taobao, RetailRocket, and Synerise). The financial datasets identify fraudulent, money-laundering, or illicit transactions, whereas the commerce datasets focus on purchase-related prediction. Table 8 summarizes the positive class and identity fields for each dataset. Since public datasets rarely release opaque identifiers such as card or device IDs, we use their high-cardinality entity-like fields as identity fields. Some of these fields, such as city or category, carry partial semantics, but LAID uses only their recurrence structure, so the results reflect the value of identity behavior alone. For Synerise, which lacks event-level labels, we construct a seven-day conversion task from the RecSys 2025 event logs: an add-to-cart event is labeled positive if the same user purchases the same SKU within the following seven days. Targets whose conversion windows extend beyond their respective data splits are excluded.
| Dataset | Positive class | Identity fields | Subjects | Train | Valid | Test | |
| CCT | fraud | merchant name, city | 6,139 | 125,304 | 3,657,794 | 3,656,690 | 512 |
| IBM AML | laundering | receiving bank, account | 496,999 | 15,936 | 760,071 | 760,345 | 512 |
| Elliptic++ | illicit | two output addresses | 46,564 | 18,360 | 7,765 | 11,184 | 256 |
| Taobao | purchase | item, category | 987,991 | 2,683,554 | 10,944,062 | 25,628,669 | 100 |
| RetailRocket | purchase | item | 58,653 | 428,761 | 55,209 | 56,467 | 100 |
| Synerise | 7-day conversion | SKU, URL | 2,032,633 | 4,944,430 | 529,289 | 642,336 | 512 |
Table 8. Details of the public datasets: positive class, identity fields, number of sequence subjects and supervised targets in each split, and maximum history length .
Sequence construction. For each target event, we construct a chronological sequence from the preceding events of the same subject (e.g., a user, card, or account), truncated to the maximum history length given in Table 8. The current event is appended as the prediction target, with its label excluded from the model input; for the commerce tasks, its behavior type is additionally masked. Since Elliptic++ provides no subject-level sequences, we construct each target sequence by tracing the fund flow backward to obtain up to 256 ancestor transactions, with the target transaction itself serving as the sequence subject.
Temporal splits. All datasets are split chronologically into training, validation, and test periods rather than by disjoint entities. A validation or test target may therefore use earlier events of the same subject, including those from preceding periods, but never their labels. Split boundaries follow the native time granularity of each dataset. CCT and IBM AML use approximately 70%/15%/15% of the global event timeline, while Synerise is split at 80% and 90% of its time span. Elliptic++ uses time steps 1–31, 32–39, and 40–49 for training, validation, and testing, respectively. Taobao uses 2017-12-01 and 2017-12-02 as split boundaries, while RetailRocket uses 2015-08-15 and 2015-09-01.
Preprocessing and sampling. The baselines in Table 3 do not use identity fields, whereas ID2Pattern encodes them with LAID as sequence-local identifiers; the hash-augmented baselines in the relabeling study are described in Appendix § 12. For the financial datasets, all positive training targets are retained and paired with five randomly sampled negatives per positive. To keep Taobao training tractable, its training targets are subsampled at a rate of 1/20 using a deterministic, label-independent hash. Validation and test sets preserve the natural label distributions, and all reported metrics are computed on these sets.
Industrial Dataset
Data and sequence construction. The industrial dataset contains over 100M transaction sequences sampled from a real-world online payment platform, whose full traffic is considerably larger. Each transaction includes semantic attributes, numerical features such as the amount, and nine high-cardinality identity fields: card, device, seller, client IP address, billing email, account email, shipping phone number, shipping contact name, and shipping postal code. Sequences are constructed at the user level in chronological order, with each target transaction appended to its preceding history and a maximum sequence length of 300. All identity values are anonymized before modeling, and due to confidentiality, we report only relative improvements over the reference model.

Figure 1 (PDF p. 11). Algorithm 1 Training procedure of ID2Pattern.
Offline evaluation. Transactions from March to June are split chronologically into training, validation, and test periods, with the test period strictly following those used for training and model selection. Positive labels correspond to confirmed fraudulent transactions. Besides AUC-ROC, we report two business metrics. T1 is the recall among the top 1% highest-scored transactions. AddCB measures the additional chargeback recall among transactions not rejected or challenged by the existing risk system, under a fixed intervention budget. Both metrics focus on the high-risk region where operational decisions are made.
Online shadow evaluation. We further evaluate all variants in shadow mode on sampled production traffic from July to August, following the offline period. All variants score the same transactions in parallel with the production model. Since risk labels take several weeks to mature, we compute the metrics after the labels become available, using the same transactions and labels for all variants. AUC is computed daily and normalized by the average AUC of the semantic-attribute-only model over the first three days. This enables a paired comparison on production traffic without affecting actual risk decisions.
Implementation Details
Model configuration. We use three windows of lengths , , and , where is dataset-specific sequence length, and each scale-specific codebook contains codes.
Pretraining. The recovery losses in Eq. (13) use , with occurrence and recurrence targets transformed by and normalized. The codebook objective uses and , together with a usage-balancing regularizer with weight that encourages all codes to be used and a routing temperature that controls the soft code assignment during training.
Optimization. We use AdamW for both pretraining and fine-tuning with a learning rate of and weight decay of , and fine-tune for 100 epochs with early stopping on the validation set. Hyperparameters are selected on the validation set, and all experiments are conducted on NVIDIA H200 GPUs.
Baseline Details
Industrial Baselines
The reference model is the platform’s Transformer-based production transaction foundation model, which models transaction histories using semantic attributes and numerical features. To isolate the effect of identity representation, two variants additionally incorporate identity fields through hashing or LAID, and ID2Pattern further adds the pretrained multi-scale pattern tokens. For hashing, each identity value is prefixed with its field name and hashed once with 128-bit MurmurHash3; four positions extracted from the hash each select one of 32 slots, yielding a 128-dimensional multi-hot vector that is fed to a field-specific MLP. This encoding needs no vocabulary but distinguishes at most codes, so distinct identities may collide. The relabeled variants are evaluated without retraining following Appendix § 12.
Public Baselines
We compare against fourteen baselines from three modeling paradigms, following the grouping in Table 3. Baselines marked with † are reimplemented following their papers, as no official code is available; all others are built on publicly available implementations.
Sequential models.
LSTM. A recurrent encoder with gated memory cells that encodes the event history and is trained end to end with the classification objective.
GRU. A recurrent encoder with simpler gating than LSTM, trained with the same inputs and objective.
Transformer. A self-attention encoder with positional encodings over the event history, serving as the basic attention-based baseline.
Tabular and boosting models.
XGBoost. A gradient-boosted tree model on the fields of the current event and aggregated statistics of its history.
CatBoost. A gradient-boosted tree model with native handling of categorical fields, using same features as XGBoost.
Temporal Boosting.† A gradient-boosted tree model that alternates leaf refreshing and tree growth over chronological training shards to follow temporal shifts in the data.
Pretraining-based and specialized models.
BERT4Rec. A bidirectional Transformer originally proposed for sequential recommendation; we pretrain it by reconstructing masked historical events and then fine-tune it for classification.
TabBERT. A hierarchical Transformer for tabular event sequences that encodes fields within each event and events within each sequence.
HSTU. A generative sequential model for user behavior that combines gated attention with relative position bias.
PANTHER. A generative pretraining framework for sequential user behavior that pretrains on behavior sequences and is fine-tuned for payment risk detection.
IBM AI-on-Z GRU. A GRU architecture from IBM’s open-source fraud-detection project that classifies each transaction from a short history of six events.
NVIDIA TFM. A transaction foundation model [39] that tokenizes each event into field tokens and pretrains a causal decoder with next-token prediction.
RiskGPT.† A generative financial risk model that treats each event as a behavior token and pretrains a causal decoder with next-behavior prediction.
PRAGMA. A transaction foundation model that pretrains on users’ financial event histories and is fine-tuned for downstream risk tasks.
Implementation. All baselines are trained on our datasets with the same splits, as no public checkpoints are available for these tasks. Each baseline is adapted to the binary classification of the current event, retains its original architecture and history length, and uses only training-period histories for pretraining. Neural baselines are averaged over three seeds.
Identity Relabeling Protocol
For the experiments in § 5.5, we train GRU, Transformer, TabBERT, RiskGPT, and HSTU with hashed identity fields as additional inputs. Identity fields are hashed into multi-hot vectors as in Appendix § 11, and all other settings, including training steps, seeds, and history lengths, follow Table 3. We then fix the trained models and test targets and replace a fraction of identity values with values never seen during training. For each identity field, a random one-to-one mapping relabels 0%, 25%, 50%, 75%, or 100% of its distinct values, and the same mapping is applied to both historical and current events. Event order, numerical features, ordinary categorical features, and labels remain unchanged, so the relabeling changes which values identities take while preserving which events share the same identity. Metrics are averaged over three model seeds. The industrial relabeled variants in Table 2 follow the same procedure with all identity values relabeled.
References
- [1]Erik Altman, Jovan Blanuša, Luc von Niederhäusern, Beni Egressy, Andreea Anghel, and Kubilay Atasu. 2023. Realistic Synthetic Financial Transactions for Anti-Money Laundering Models. In Advances in Neural Information Processing Systems, Vol. 36. Curran Associates, Inc., 29851–29874. doi:10.52202/075280-1300
- [2]Dmitrij Babayev, Nikita Ovsov, Ivan Kireev, Maria Ivanova, Gleb Gusev, Ivan Nazarov, and Alexander Tuzhilin. 2022. COLeS: Contrastive Learning for Event Sequences with Self-Supervision. In Proceedings of the 2022 International Conference on Management of Data (SIGMOD ’22). Association for Computing Machinery, 1190–1199. doi:10.1145/3514221.3526129
- [3]Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). 785–794. doi:10.1145/2939672.2939785DOI
- [4]Bin Dou, Baokun Wang, Yun Zhu, Xiaotong Lin, Yike Xu, Xiaorui Huang, Yang Chen, Yun Liu, Shaoshuai Han, Yongchao Liu, et al. 2025. Transferable and Forecastable User Targeting Foundation Model. In Companion Proceedings of the ACM Web Conference 2025 (WWW Companion). 181–190. doi:10.1145/3701716.3715266DOI
- [5]Yingting Dou, Zhimeng Jiang, Tianyi Zhang, Mingzhi Hu, Zhichao Xu, Shubham Jain, Uday Singh Saini, Xiran Fan, Jiarui Sun, Menghai Pan, et al. 2025. TransactionGPT. arXiv preprint arXiv:2511.08399 (2025).
- [6]Yingtong Dou, Zhiwei Liu, Li Sun, Yutong Deng, Hao Peng, and Philip S. Yu. 2020. Enhancing Graph Neural Network-based Fraud Detectors against Camouflaged Fraudsters. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM).DOI
- [7]Youssef Houmy and Ling Liu. 2023. Demystifying Fraudulent Transactions and Illicit Nodes in the Bitcoin Network for Financial Forensics. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’23). Association for Computing Machinery, 3979–3990. doi:10.1145/3580305.3599803
- [8]Yuanbo Gao, Peng Lin, Dongyue Wang, Feng Mei, Xiwei Zhao, Suxiong Xu, and Jinghe Hu. 2024. PPM: A Pre-trained Plug-in Model for Click-through Rate Prediction. In Companion Proceedings of the ACM Web Conference 2024 (WWW Companion). doi:10.1145/3589335.3648329DOI
- [9]Rahul Gupta, Payal Pote, Bhanurpiya Pegu, Maneet Singh, Josh Allbright, and Chris Merz. 2025. Temporal Boosting for Incremental Tree-based Learning on Tabular Data. In Proceedings of the 13th ACM IKDD International Conference on Data Science (CODS ’25). Association for Computing Machinery, 297–305. doi:10.1145/3799830.3799877DOI
- [10]Ruining He, Lukasz Heldt, Lichan Hong, Raghunandan Keshavan, Shifan Mao, Nikhil Mehta, Zhengyang Su, Alicia Tsai, Yueqi Wang, Shao-Chuan Wang, Xinyang Yi, Lexi Baugher, Baykal Cakici, Ed Chi, Cristos Goodrow, et al. 2025. PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations. arXiv preprint arXiv:2510.07784 (2025).DOI
- [11]Yupeng Hou, Zhankui He, Xuehan McAuley, and Wayne Xin Zhao. 2023. Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders. In Proceedings of the ACM Web Conference 2023 (WWW). doi:10.1145/3543507.3583434DOI
- [12]Sihao Hu, Tiansheng Huang, Ka-Ho Chow, Wenqi Wei, Yanzhao Wu, and Ling Lu. 2024. ZipZap: Efficient Training of Language Models for Large-Scale Fraud Detection on Blockchain. In Proceedings of the ACM Web Conference 2024 (WWW). doi:10.1145/3589334.3645352DOI
- [13]Sihao Hu, Zhen Zhang, Bingqiao Luo, Shengliang Lu, Bingsheng He, and Ling Lu. 2023. BERT4ETH: A Pre-trained Transformer for Ethereum Fraud Detection. In Proceedings of the ACM Web Conference 2023 (WWW). doi:10.1145/3543507.3583345DOI
- [14]IBM. [n. d.]. ai-on-z-fraud-detection: Credit Card Fraud Detection Notebooks and ONNX Models. GitHub repository. Accessed 22 October 2026. https://github.com/IBM/ai-on-z-fraud-detection
- [15]Sergey Ivanov and Evgeny Burnaev. 2018. Anonymous Walk Embeddings. In Proceedings of the 35th International Conference on Machine Learning (ICML) (Proceedings of Machine Learning Research, Vol. 80). 2186–2195.
- [16]Johannes Jurgovsky, Michael Granitzer, Konstantin Ziegler, Sylvie Calabretto, Pierre-Edouard Portier, Liyun He-Gueltou, and Olivier Caelen. 2018. Sequence Classification for Credit-Card Fraud Detection. Expert Systems with ApplicationsDOI
- [17]Wang-Cheng Kang, Derek Zhiyuan Cheng, Tiansheng Yao, Xinyang Yi, Ting Chen, Lichan Hong, and Ed H. Chi. 2021. Learning to Embed Categorical Features without Embedding Tables for Recommendation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD).arxiv.org/abs/2010.10784
- [18]Guilin Li, Yun Zhang, Xiyuan Chen, Chengqi Li, Bo Wang, Linghe Kong, Wenjia Wang, Weiran Huang, and Matthias Hwai Yong Tan. 2025. PANTHER: Generative Pretraining Beyond Language for Sequential User Behavior Modeling. In Advances in Neural Information Processing Systems 38 (NeurIPS).
- [19]Pan Li, Yanbang Wang, Hongwei Wang, and Jure Leskovec. 2020. Distance Encoding: Design Provably More Powerful Neural Networks for Graph Representation Learning. In Advances in Neural Information Processing Systems 33 (NeurIPS).
- [20]Yang Liu, Xiang Ao, Zidi Qin, Jianfeng Chi, Jinghua Feng, Hao Yang, and Qing He. 2021. Pick and Choose: A GNN-based Imbalanced Learning Approach for Fraud Detection. In Proceedings of the Web Conference 2021 (WWW).DOI
- [21]Ziqi Liu, Chaochao Chen, Xue Gong, Yang Zhu, Xiaolong Li, and Le Song. 2018. Heterogeneous Graph Neural Networks for Malicious Account Detection. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management (CIKM). 2077–2085. doi:10.1145/3269206.3272010DOI
- [22]Maxim Ostroukhiv, Ruslan Mikhailov, Vladimir Iashin, Artem Sokolov, Andrei Akshonov, Vitaly Protasov, Dmitrii Beloborodov, V ince Mullin, Roman Yokunda Enzmann, Georgios Kolovos, Jason Renders, Pavel Nesterov, and Anton Repushko. 2026. PRAGMA: Revolut Foundation Model. arXiv preprint arXiv:2604.08649 (2026).arxiv.org/abs/2604.08649
- [23]Inklt Padhi, Yair Schiff, Igor Melnyk, Mattia Rigotti, Youssef Mroueh, Pierre Dognin, Jerret Ross, Ravi Nair, and Erik Altman. 2021. Tabular Transformers for Modeling Multivariate Time Series. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).arxiv.org/abs/2011.01843
- [24]Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2018. CatBoost: Unbiased Boosting with Categorical Features. In Advances in Neural Information Processing Systems 31 (NeurIPS).arxiv.org/abs/1706.09516
- [25]Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikak Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Tran, Jonah Samost, Maciej Kula, Ed Chi, and Maheswaran Sathiamoorthy. 2023. Recommender Systems with Generative Retrieval. In Advances in Neural Information Processing Systems 36 (NeurIPS).arxiv.org/abs/2305.05065
- [26]Retail Rocket. [n. d.]. Retailrocket Recommender System Dataset. Kaggle dataset. Accessed 22 October 2026. https://www.kaggle.com/datasets/retailrocket/ecommerce-dataset
- [27]Hao-Jun Michael Shi, Dheevarsha Mudigere, Maxim Naumov, and Jiyan Yang. 2020. Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems. In Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD). doi:10.1145/3394486.3403059DOI
- [28]Anima Singh, Trung Vu, Nikhil Mehta, Raghunandan Keshavan, Maheswaran Sathiamoorthy, Yilin Zheng, Lichan Hong, Lukasz Heldt, Li Wei, Dewansh Tandon, Ed H. Chi, et al. 2024. Better Generalization with Semantic IDs: A Case Study in Ranking for Recommendations. In Proceedings of the 18th ACM Conference on Recommender Systems (RecSys). doi:10.1145/3640457.3688190DOI
- [29]Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM). 1441–1450. doi:10.1145/3357384.3357895DOI
- [30]Dan Tito Svenstrup, Jonas Meinertz Hansen, and Ole Winther. 2017. Hash Embeddings for Efficient Word Representations. In Advances in Neural Information Processing Systems 30 (NeurIPS).
- [31]Synerise SA. 2025. Universal Behavioral Modeling Dataset. RecSys Challenge 2025 official dataset. Accessed 22 October 2026. https://recsys.synerise.com/data-set
- [32]Jianheng Tang, Jiajin Li, Ziqi Gao, and Jia Li. 2022. Rethinking Graph Neural Networks for Anomaly Detection. In Proceedings of the 39th International Conference on Machine Learning (ICML) (Proceedings of Machine Learning Research, Vol. 162). 21076–21089.arxiv.org/abs/2205.15508
- [33]Tianchi. [n. d.]. User Behavior Data from Taobao for Recommendation. Tianchi dataset 649. Accessed 22 October 2026. https://tianchi.aliyun.com/dataset/649
- [34]Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. 2017. Neural Discrete Representation Learning. In Advances in Neural Information Processing Systems, Vol. 30. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2017/hash/7a98af17e63a0ac09ce2e96d03992fbc-Abstract.htmlarxiv.org/abs/1711.00937
- [35]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. In Advances in Neural Information Processing Systems, Vol. 30.
- [36]Daixin Wang, Jianbin Lin, Peng Cui, Quanhui Jia, Zhen Wang, Yanming Fang, Quan Yu, Jun Zhou, Shuang Yang, and Yuan Qi. 2019. A Semi-supervised Graph Attentive Network for Financial Fraud Detection. In Proceedings of the IEEE International Conference on Data Mining (ICDM). doi:10.1109/ICDM.2019.00070DOI
- [37]Yangbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. 2021. Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks. In International Conference on Learning Representations (ICLR).arxiv.org/abs/2101.05974
- [38]Kilian Q. Weinberger, Anirban Dasgupta, John Langford, Alexander J. Smola, and Josh Attenberg. 2009. Feature Hashing for Large Scale Multitask Learning. In Proceedings of the 26th International Conference on Machine Learning (ICML), 1113–1120. doi:10.1145/1553374.1553516DOI
- [39]Benjamin Wang. 2026. Build Your Own Transaction Foundation Model for Financial Intelligence. NVIDIA Technical Blog. Published 16 June 2026; accessed 22 October 2026. https://developer.nvidia.com/blog/build-your-own-transaction-foundation-model-for-financial-intelligence/
- [40]Zheng Yang, Faie Yuan, Yu Song, Youhua Li, Junchen Fu, Fei Yang, Yunzhu Pan, and Yongxin Ni. 2023. Where to Go Next for Recommender Systems? ID-vs. Modality-based Recommender Models Revisited. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR).arxiv.org/abs/2303.13835
- [41]Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Jiayuan He, Yinghai Lu, and Yu Shi. 2024. Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 (Proceedings of Machine Learning Research, Vol. 235). PMLR / OpenReview.net, 58484–58509. https://proceedings.mlr.press/v235/zhai24a.htmlarxiv.org/abs/2402.17152
- [42]Dongyu Zhang, Liang Wang, Xin Dai, Shubham Jain, Junpeng Wang, Yujie Fan, Chin-Chia Michael Yeh, Yan Zheng, Zhongfang Zhuang, and Wei Zhang. 2023. FATA-Trans: Field And Time-Aware Transformer for Sequential Tabular Data. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (CIKM). doi:10.1145/3583780.3614879DOI
- [43]Zhongjian Zhang, Mengmei Zhang, Dehua Xu, Rongjun Shi, Jianfeng Liu, Fuli Meng, Huaijian Xu, Xiao Wang, Ruijia Wang, Junze Chen, Minwei Tang, and Chuan Shi. 2026. RiskGPT: A Generative Foundation Model for Financial Risk Detection. In Proceedings of the ACM Web Conference 2026 (WWW), 7733–7744. doi:10.1145/3774904.3792832




