Vera: A General-Purpose Plausibility Estimation Model for Commonsense
  Statements
Despite the much discussed capabilities of today's language models, they are
still prone to silly and unexpected commonsense failures. We consider a
retrospective verification approach that reflects on the correctness of LM
outputs, and introduce Vera, a general-purpose model that estimates the
plausibility of declarative statements based on commonsense knowledge. Trained
on ~7M commonsense statements created from 19 QA datasets and two large-scale
knowledge bases, and with a combination of three training objectives, Vera is a
versatile model that effectively separates correct from incorrect statements
across diverse commonsense domains. When applied to solving commonsense
problems in the verification format, Vera substantially outperforms existing
models that can be repurposed for commonsense verification, and it further
exhibits generalization capabilities to unseen tasks and provides
well-calibrated outputs. We find that Vera excels at filtering LM-generated
commonsense knowledge and is useful in detecting erroneous commonsense
statements generated by models like ChatGPT in real-world settings.
645847698eef89b5c3c1d268
Jiacheng Liu
claimed_verified
2023-05-10T10:36:54.632Z
635f46d1928a42bc95cfcf7c
https://cdn-avatars.huggingface.co/v1/production/uploads/635f46d1928a42bc95cfcf7c/5KF8aLiDCJdl7B1SdJ-7V.png
Jiacheng Liu
user
liujch1998
645847698eef89b5c3c1d269
Wenya Wang
admin_assigned
2023-06-02T13:56:07.443Z
630ca0b2b66f088d54761836
https://cdn-avatars.huggingface.co/v1/production/uploads/1661771932639-noauth.jpeg
wang
user
wenya
645847698eef89b5c3c1d26a
Dianzhuo Wang
645847698eef89b5c3c1d26b
Noah A. Smith
645847698eef89b5c3c1d26c
Yejin Choi
admin_assigned
2023-08-10T08:38:59.687Z
64d42729f63b01b7f676b176
/avatars/52e54bdd6a1fb6c774a40cd70f3d7925.svg
Yejin Choi
user
yejinchoinka
645847698eef89b5c3c1d26d
Hannaneh Hajishirzi
Vera: A General-Purpose Plausibility Estimation Model for Commonsense
  Statements
Despite the much discussed capabilities of today's language models, they are
still prone to silly and unexpected commonsense failures. We consider a
retrospective verification approach that reflects on the correctness of LM
outputs, and introduce Vera, a general-purpose model that estimates the
plausibility of declarative statements based on commonsense knowledge. Trained
on ~7M commonsense statements created from 19 QA datasets and two large-scale
knowledge bases, and with a combination of three training objectives, Vera is a
versatile model that effectively separates correct from incorrect statements
across diverse commonsense domains. When applied to solving commonsense
problems in the verification format, Vera substantially outperforms existing
models that can be repurposed for commonsense verification, and it further
exhibits generalization capabilities to unseen tasks and provides
well-calibrated outputs. We find that Vera excels at filtering LM-generated
commonsense knowledge and is useful in detecting erroneous commonsense
statements generated by models like ChatGPT in real-world settings.
Vera, a model trained on commonsense statements, effectively verifies and separates correct from incorrect statements across diverse domains, outperforming existing models and aiding in detecting erroneous statements from language models.
commonsense verification Vera declarative statements commonsense knowledge training objectives general-purpose model well-calibrated outputs LM-generated commonsense knowledge ChatGPT