Entropy Regularizing Activation: Boosting Continuous Control, Large
  Language Models, and Image Classification with Activation as Entropy
  Constraints
We propose ERA, a new paradigm that constrains the sampling entropy above
given thresholds by applying specially designed activations to the outputs of
models. Our approach demonstrates broad effectiveness across different domains:
1) for large language models(LLMs), boosting the AIME 2025 score for
Qwen2.5-Math-7B by 37.4%; 2) for continuous control reinforcement learning
agents, improving performance by more than 30% over strong baselines such as
SAC on the challenging HumanoidBench; 3) for image classification, enhancing
ImageNet top-1 accuracy by 0.69% for ResNet-50. These gains are achieved with a
computational overhead of less than 7%. Our work validates output activation as
a powerful tool for entropy control, opening a new direction for designing
simpler and more robust algorithms.
68e86fbb95e8e6771df3894f
Zilin Kang
claimed_verified
2025-10-10T04:32:30.481Z
64fefd879fee065c899c4977
https://cdn-avatars.huggingface.co/v1/production/uploads/64fefd879fee065c899c4977/jyHEOMtHFNMUq9wHEjGr4.png
Zilin Kang
user
zilinkang
68e86fbb95e8e6771df38950
Chonghua Liao
claimed_verified
2025-10-10T04:32:32.996Z
667fdaee20ee9ac417c7708c
/avatars/69dfba6ff392643af1dcfe8af0a42ae9.svg
Chonghua Liao
user
ChonghuaLiao
68e86fbb95e8e6771df38951
Tingqiang Xu
claimed_verified
2025-10-10T09:12:12.803Z
665697ead156bf7c364f0505
/avatars/b043889791fd8c2c5810d4c95663d131.svg
Tingqiang Xu
user
xtqqwq
68e86fbb95e8e6771df38952
Huazhe Xu
Entropy Regularizing Activation: Boosting Continuous Control, Large
  Language Models, and Image Classification with Activation as Entropy
  Constraints
We propose ERA, a new paradigm that constrains the sampling entropy above
given thresholds by applying specially designed activations to the outputs of
models. Our approach demonstrates broad effectiveness across different domains:
1) for large language models(LLMs), boosting the AIME 2025 score for
Qwen2.5-Math-7B by 37.4%; 2) for continuous control reinforcement learning
agents, improving performance by more than 30% over strong baselines such as
SAC on the challenging HumanoidBench; 3) for image classification, enhancing
ImageNet top-1 accuracy by 0.69% for ResNet-50. These gains are achieved with a
computational overhead of less than 7%. Our work validates output activation as
a powerful tool for entropy control, opening a new direction for designing
simpler and more robust algorithms.
ERA, a new paradigm using specially designed activations, enhances performance across LLMs, reinforcement learning, and image classification with minimal computational overhead.
ERA sampling entropy activations large language models LLMs AIME 2025 Qwen2.5-Math-7B continuous control reinforcement learning SAC HumanoidBench image classification ImageNet ResNet-50 entropy control