Context-Conditioned Autoregression: A Hybrid Framework for Autoregressive De Novo Sequencing
Xiang Zhang, Jiaqi Wei, Zijie Qiu, Sheng Xu, Zhi Jin, ZhiQiang Gao, Nanqing Dong, Siqi Sun
Autoregressive (AT) models, common in sequence generation, are limited in biological tasks like \textit{de novo} peptide sequencing by their unidirectionality, failing to capture crucial global bidirectional dependencies. Non-Autoregressive Transformers (NATs) offer holistic, bidirectional processing but face challenges with generative coherence, scalability, and complex optimization.To transcend this, we propose a hybrid framework enhancing AT generation by dynamically integrating rich contextual information from non-autoregressive mechanisms. Our approach couples a shared spectrum encoder with two decoders: a non-autoregressive one learning latent bidirectional features, and an AT decoder synthesizing the sequence by leveraging these features. A novel cross-decoder attention module enables the AT decoder to iteratively query and integrate these bidirectional features, enriching its predictions. This synergy is cultivated via a tailored training strategy with importance annealing for balanced objectives and \cross-decoder gradient blocking for stable, focused learning.Evaluations on a demanding 9-species benchmark show our model substantially surpasses AT and NAT baselines. It uniquely harmonizes AT stability with NAT contextual awareness, delivering robust, superior performance on diverse proteomic data. This research advances de novo peptide sequencing and contributes a novel architectural paradigm for augmenting AT models with enhanced bidirectional understanding for complex sequence modeling.Our code is available for reproduction: https://anonymous.4open.science/r/CrossNovo-E263