PANTHER: Hybrid Pretraining for Billion-Scale Risk Detection via Sequential User Behavior Modeling
Guilin Li, Yun Zhang, Xiuyuan Chen, Chengqi Li, Weiran Huang, Linghe Kong, Wenjia Wang, Bo Wang, Matthias Tan
Large-scale payment platforms face fraud detection challenges due to high-cardinality transactions, strict latency, and limited labeled data. As a result, traditional models relying on extensive handcrafted features struggle with generalization. We propose PANTHER, a hybrid framework combining self-supervised generative pretraining with lightweight discriminative modeling for real-time fraud detection. Leveraging abundant unlabeled data, PANTHER learns compact user behavior representations via a predict-next-behavior task, capturing sparse interactions and transactional dynamics. PANTHER addresses four key challenges: (1) Structured Tokenization to compress multi-dimensional transaction attributes into an interpretable vocabulary; (2) Sequence Pattern Recognition Module (SPRM) for modeling periodic transaction motifs; (3) a Unified User-Profile Embedding that fuses static demographics with dynamic transaction histories, enabling both personalized predictions and population-level knowledge transfer; and (4) Real-time scalability enabled by offline caching of pre-trained embeddings for millisecond-level inference. Fully deployed and operational online at WeChat Pay, PANTHER delivers a 38.6\% relative improvement in fraud detection recall and a 25.6\% boost in next-transaction prediction HR@1, significantly outperforming transformer baselines. Cross-domain evaluations on public benchmarks (CCT, MBD, MovieLens-1M, Yelp) show strong generalization, achieving up to 21\% HR@1 gains over transformer baselines, establishing PANTHER as a scalable, high-performance framework for industrial sequential behavior modeling.