RaTEScore: A Metric for Radiology Report Generation
This paper introduces a novel, entity-aware metric, termed as Radiological
Report (Text) Evaluation (RaTEScore), to assess the quality of medical reports
generated by AI models. RaTEScore emphasizes crucial medical entities such as
diagnostic outcomes and anatomical details, and is robust against complex
medical synonyms and sensitive to negation expressions. Technically, we
developed a comprehensive medical NER dataset, RaTE-NER, and trained an NER
model specifically for this purpose. This model enables the decomposition of
complex radiological reports into constituent medical entities. The metric
itself is derived by comparing the similarity of entity embeddings, obtained
from a language model, based on their types and relevance to clinical
significance. Our evaluations demonstrate that RaTEScore aligns more closely
with human preference than existing metrics, validated both on established
public benchmarks and our newly proposed RaTE-Eval benchmark.
667a32e17dc34fe17cef3a84
Weike Zhao
claimed_verified
2024-07-01T07:47:05.570Z
64365addfae287005149dd24
/avatars/bd6d4512d66fd9fd7fd5476ea7a44b46.svg
Weike Zhao
user
Angelakeke
667a32e17dc34fe17cef3a85
Chaoyi Wu
admin_assigned
2024-07-01T15:04:29.973Z
6436aaaa0c77d7c5036abdbd
https://cdn-avatars.huggingface.co/v1/production/uploads/6436aaaa0c77d7c5036abdbd/C9A276yEeAPkKLEqUcjl_.jpeg
Chaoyi Wu
user
chaoyi-wu
667a32e17dc34fe17cef3a86
Xiaoman Zhang
admin_assigned
2024-07-01T15:04:37.223Z
664dbb92724eaeb47777ded9
/avatars/f507ebddee54b35c14c96658c1858428.svg
Xiaoman Zhang
user
XiaomanZhang
667a32e17dc34fe17cef3a87
Ya Zhang
667a32e17dc34fe17cef3a88
Yanfeng Wang
667a32e17dc34fe17cef3a89
Weidi Xie
admin_assigned
2024-07-01T15:05:32.447Z
63f1841f2f7c0152e8748ec4
/avatars/5398658c7d0cf556531a625a4ca5d18a.svg
Xie
user
Weidi
RaTEScore: A Metric for Radiology Report Generation
This paper introduces a novel, entity-aware metric, termed as Radiological
Report (Text) Evaluation (RaTEScore), to assess the quality of medical reports
generated by AI models. RaTEScore emphasizes crucial medical entities such as
diagnostic outcomes and anatomical details, and is robust against complex
medical synonyms and sensitive to negation expressions. Technically, we
developed a comprehensive medical NER dataset, RaTE-NER, and trained an NER
model specifically for this purpose. This model enables the decomposition of
complex radiological reports into constituent medical entities. The metric
itself is derived by comparing the similarity of entity embeddings, obtained
from a language model, based on their types and relevance to clinical
significance. Our evaluations demonstrate that RaTEScore aligns more closely
with human preference than existing metrics, validated both on established
public benchmarks and our newly proposed RaTE-Eval benchmark.
A novel metric, RaTEScore, evaluates AI-generated medical reports by assessing the quality through medical entity recognition and embedding similarity, demonstrating better alignment with human preferences.
radiological reports entity-aware metric RaTEScore diagnostic outcomes anatomical details medical NER entity embeddings language model clinical significance RaTE-NER RaTE-Eval