Has Your Pretrained Model Improved? A Multi-head Posterior Based
  Approach
The emergence of pretrained models has significantly impacted from Natural
Language Processing (NLP) and Computer Vision to relational datasets.
Traditionally, these models are assessed through fine-tuned downstream tasks.
However, this raises the question of how to evaluate these models more
efficiently and more effectively. In this study, we explore a novel approach
where we leverage the meta features associated with each entity as a source of
worldly knowledge and employ entity representations from the models. We propose
using the consistency between these representations and the meta features as a
metric for evaluating pretrained models. Our method's effectiveness is
demonstrated across various domains, including models with relational datasets,
large language models and images models.
659cbe46442979361cd46580
Prince Aboagye
admin_assigned
2024-01-09T09:07:54.590Z
60db7be9bf02367a70af5bb9
/avatars/6bddc3e0d04775aaff9fe71c5f237baa.svg
Prince Aboagye
user
paboagye
659cbe46442979361cd46581
Yan Zheng
659cbe46442979361cd46582
Junpeng Wang
659cbe46442979361cd46583
Uday Singh Saini
admin_assigned
2024-01-09T09:09:31.635Z
634ee7a64677a5891c0d6cba
/avatars/df548841cbf388f1de2f0245fa818fef.svg
Uday Singh Saini
user
lazypikachu23
659cbe46442979361cd46584
Xin Dai
659cbe46442979361cd46585
Michael Yeh
admin_assigned
2024-01-09T09:10:12.679Z
646dbc8b212146121301212c
/avatars/bffd1cc6e4920422ed577d4421057ca3.svg
Michael Yeh
user
NCGTrigger
659cbe46442979361cd46586
Yujie Fan
admin_assigned
2024-01-09T09:10:22.966Z
6267b82333e1845bf1fe5d7a
/avatars/43d3b12c4df7db2f3fadc2c911a59908.svg
fanyujie
user
fanyujie
659cbe46442979361cd46587
Zhongfang Zhuang
659cbe46442979361cd46588
Shubham Jain
claimed_verified
2024-01-10T08:38:55.246Z
64ef7b470af6d9bfbc8b976f
/avatars/8be23c2f74f94ee2aa2ffa78dddaf2e9.svg
shubham jain
user
shubhjai
659cbe46442979361cd46589
Liang Wang
659cbe46442979361cd4658a
Wei Zhang
Has Your Pretrained Model Improved? A Multi-head Posterior Based
  Approach
The emergence of pretrained models has significantly impacted from Natural
Language Processing (NLP) and Computer Vision to relational datasets.
Traditionally, these models are assessed through fine-tuned downstream tasks.
However, this raises the question of how to evaluate these models more
efficiently and more effectively. In this study, we explore a novel approach
where we leverage the meta features associated with each entity as a source of
worldly knowledge and employ entity representations from the models. We propose
using the consistency between these representations and the meta features as a
metric for evaluating pretrained models. Our method's effectiveness is
demonstrated across various domains, including models with relational datasets,
large language models and images models.
pretrained models NLP Computer Vision relational datasets fine-tuned downstream tasks meta features entity representations consistency relational datasets large language models images models