Weakly supervised information extraction from inscrutable handwritten
  document images
State-of-the-art information extraction methods are limited by OCR errors.
They work well for printed text in form-like documents, but unstructured,
handwritten documents still remain a challenge. Adapting existing models to
domain-specific training data is quite expensive, because of two factors, 1)
limited availability of the domain-specific documents (such as handwritten
prescriptions, lab notes, etc.), and 2) annotations become even more
challenging as one needs domain-specific knowledge to decode inscrutable
handwritten document images. In this work, we focus on the complex problem of
extracting medicine names from handwritten prescriptions using only weakly
labeled data. The data consists of images along with the list of medicine names
in it, but not their location in the image. We solve the problem by first
identifying the regions of interest, i.e., medicine lines from just weak labels
and then injecting a domain-specific medicine language model learned using only
synthetically generated data. Compared to off-the-shelf state-of-the-art
methods, our approach performs >2.5x better in medicine names extraction from
prescriptions.
6487ca50536e4b60679170d9
Sujoy Paul
claimed_verified
2024-07-30T08:44:06.626Z
66a8a7968268a5b1a0819668
/avatars/a8cdca874279beeef1023c8ecb10403a.svg
Sujoy Paul
user
sujoyp
6487ca50536e4b60679170da
Gagan Madan
6487ca50536e4b60679170db
Akankshya Mishra
6487ca50536e4b60679170dc
Narayan Hegde
6487ca50536e4b60679170dd
Pradeep Kumar
6487ca50536e4b60679170de
Gaurav Aggarwal
Weakly supervised information extraction from inscrutable handwritten
  document images
State-of-the-art information extraction methods are limited by OCR errors.
They work well for printed text in form-like documents, but unstructured,
handwritten documents still remain a challenge. Adapting existing models to
domain-specific training data is quite expensive, because of two factors, 1)
limited availability of the domain-specific documents (such as handwritten
prescriptions, lab notes, etc.), and 2) annotations become even more
challenging as one needs domain-specific knowledge to decode inscrutable
handwritten document images. In this work, we focus on the complex problem of
extracting medicine names from handwritten prescriptions using only weakly
labeled data. The data consists of images along with the list of medicine names
in it, but not their location in the image. We solve the problem by first
identifying the regions of interest, i.e., medicine lines from just weak labels
and then injecting a domain-specific medicine language model learned using only
synthetically generated data. Compared to off-the-shelf state-of-the-art
methods, our approach performs >2.5x better in medicine names extraction from
prescriptions.
The method extracts medicine names from handwritten prescriptions using weakly labeled data and a synthesized domain-specific language model, outperforming state-of-the-art methods significantly.
OCR information extraction handwritten documents domain-specific training weakly labeled data regions of interest medicine language model synthetic data