Enhancing Zero-Shot Black-Box Optimization via Pretrained Models with Efficient Population Modeling, Interaction, and Stable Gradient Approximation
Muqi Han, Xiaobin Li, Kai Wu, Xiaoyu Zhang, Handing Wang
Zero‑shot optimization, which solves new black‑box problems without task‑specific tuning, is critical for efficient decision making across science and engineering. Pre‑trained optimization models (POMs) address this need by leveraging large‑scale learned priors to optimize unseen objective functions out of the box, reducing reliance on expert knowledge. However, existing POMs suffer from three key weaknesses: (1) limited population modeling that hinders convergence and generalization, (2) static information exchange that restricts solution diversity, and (3) noisy gradient estimates that destabilize training. To overcome these challenges, we introduce Efficient Pretrained Optimization Models, featuring: (1) LMM/LCM Tokenizer, which embeds landscape cues into population tokens for richer modeling and seamless feature fusion; (2) Efficient LMM, a deformable attention module enabling adaptive, diversity‑promoting interactions; (3) Differentiable LCM, which yields precise gradient approximations for stable, reliable training. Our model sets a new benchmark in zero‑shot black‑box optimization by delivering faster convergence, higher diversity, and robust performance.