Has Your Pretrained Model Improved? A Multi-head Posterior Based
  Approach
The emergence of pretrained models has significantly impacted from Natural
Language Processing (NLP) and Computer Vision to relational datasets.
Traditionally, these models are assessed through fine-tuned downstream tasks.
However, this raises the question of how to evaluate these models more
efficiently and more effectively. In this study, we explore a novel approach
where we leverage the meta features associated with each entity as a source of
worldly knowledge and employ entity representations from the models. We propose
using the consistency between these representations and the meta features as a
metric for evaluating pretrained models. Our method's effectiveness is
demonstrated across various domains, including models with relational datasets,
large language models and images models.
659cbe46442979361cd46580
Prince Aboagye
admin_assigned
2024-01-09T09:07:54.590Z
60db7be9bf02367a70af5bb9
/avatars/6bddc3e0d04775aaff9fe71c5f237baa.svg
Prince Aboagye
user
paboagye
659cbe46442979361cd46581
Yan Zheng
659cbe46442979361cd46582
Junpeng Wang
659cbe46442979361cd46583
Uday Singh Saini
admin_assigned
2024-01-09T09:09:31.635Z
634ee7a64677a5891c0d6cba
/avatars/df548841cbf388f1de2f0245fa818fef.svg
Uday Singh Saini
user
lazypikachu23
659cbe46442979361cd46584
Xin Dai
659cbe46442979361cd46585
Michael Yeh
admin_assigned
2024-01-09T09:10:12.679Z
646dbc8b212146121301212c
/avatars/bffd1cc6e4920422ed577d4421057ca3.svg
Michael Yeh
user
NCGTrigger
659cbe46442979361cd46586
Yujie Fan
admin_assigned
2024-01-09T09:10:22.966Z
6267b82333e1845bf1fe5d7a
/avatars/43d3b12c4df7db2f3fadc2c911a59908.svg
fanyujie
user
fanyujie
659cbe46442979361cd46587
Zhongfang Zhuang
659cbe46442979361cd46588
Shubham Jain
claimed_verified
2024-01-10T08:38:55.246Z
64ef7b470af6d9bfbc8b976f
/avatars/8be23c2f74f94ee2aa2ffa78dddaf2e9.svg
shubham jain
user
shubhjai
659cbe46442979361cd46589
Liang Wang
659cbe46442979361cd4658a
Wei Zhang
Has Your Pretrained Model Improved? A Multi-head Posterior Based
  Approach
The emergence of pretrained models has significantly impacted from Natural
Language Processing (NLP) and Computer Vision to relational datasets.
Traditionally, these models are assessed through fine-tuned downstream tasks.
However, this raises the question of how to evaluate these models more
efficiently and more effectively. In this study, we explore a novel approach
where we leverage the meta features associated with each entity as a source of
worldly knowledge and employ entity representations from the models. We propose
using the consistency between these representations and the meta features as a
metric for evaluating pretrained models. Our method's effectiveness is
demonstrated across various domains, including models with relational datasets,
large language models and images models.
A new evaluation method for pretrained models uses the consistency between entity representations and meta features across various domains.
pretrained models natural language processing computer vision relational datasets fine-tuning meta features entity representations consistency