TrajSelector: Harnessing Latent Representations for Efficient and
  Effective Best-of-N in Large Reasoning Model
Large language models (LLMs) have shown remarkable progress in complex
reasoning tasks, largely enabled by test-time scaling (TTS) paradigms that
allocate additional compute during inference. Among these, external TTS
(particularly the Best-of-N selection paradigm) yields scalable performance
improvements by selecting from multiple independently generated reasoning
trajectories. However, this approach faces key limitations: (i) the high
computational overhead of deploying process reward models, (ii) the
underutilization of the LLM's intrinsic latent representations. We introduce
TrajSelector, an efficient and effective Best-of-N framework that exploit the
hidden states in the sampler LLM for process-level scoring. A lightweight
verifier (with only 0.6B parameters) evaluates the quality of step-wise
trajectory, and then aggregates these scores to identify the optimal reasoning
trajectory. Our framework employs a fully data-driven, end-to-end training
recipe that eliminates reliance on massive step-level annotations. Experiential
results across five benchmarks demonstrate that TrajSelector delivers
consistent performance gains. In Best-of-32 settings, it surpasses majority
voting by 4.61% accuracy and outperforms existing process reward models by
4.31% to 12.21%, all while maintaining lower inference costs.
68f6f28324c44893631118ac
Bin Yu
claimed_verified
2025-10-21T14:03:45.205Z
63d3b5f1640bb0f77173baea
https://cdn-avatars.huggingface.co/v1/production/uploads/1674819020331-noauth.jpeg
yubin
user
VLyb
68f6f28324c44893631118ad
Xinming Wang
68f6f28324c44893631118ae
Shijie Lian
68f6f28324c44893631118af
Haotian Li
68f6f28324c44893631118b0
Changti Wu
68f6f28324c44893631118b1
Ruina Hu
68f6f28324c44893631118b2
Bailing Wang
68f6f28324c44893631118b3
Yuliang Wei
68f6f28324c44893631118b4
Kai Chen
TrajSelector: Harnessing Latent Representations for Efficient and
  Effective Best-of-N in Large Reasoning Model
Large language models (LLMs) have shown remarkable progress in complex
reasoning tasks, largely enabled by test-time scaling (TTS) paradigms that
allocate additional compute during inference. Among these, external TTS
(particularly the Best-of-N selection paradigm) yields scalable performance
improvements by selecting from multiple independently generated reasoning
trajectories. However, this approach faces key limitations: (i) the high
computational overhead of deploying process reward models, (ii) the
underutilization of the LLM's intrinsic latent representations. We introduce
TrajSelector, an efficient and effective Best-of-N framework that exploit the
hidden states in the sampler LLM for process-level scoring. A lightweight
verifier (with only 0.6B parameters) evaluates the quality of step-wise
trajectory, and then aggregates these scores to identify the optimal reasoning
trajectory. Our framework employs a fully data-driven, end-to-end training
recipe that eliminates reliance on massive step-level annotations. Experiential
results across five benchmarks demonstrate that TrajSelector delivers
consistent performance gains. In Best-of-32 settings, it surpasses majority
voting by 4.61% accuracy and outperforms existing process reward models by
4.31% to 12.21%, all while maintaining lower inference costs.
TrajSelector is an efficient Best-of-N framework that leverages hidden states for process-level scoring, improving LLM performance with lower computational costs.
large language models test-time scaling external TTS Best-of-N selection process reward models latent representations hidden states sampler LLM lightweight verifier data-driven training step-wise trajectory optimal reasoning trajectory inference costs