rednote-hilab

/

dots.ocr

like 1.1k

Follow

rednote-hilab 603

I want to know how the model extract markdown for OCR. does it based on the json like (bounding box, context,...) then use LLM to generate the markdown.

It just serializes the output JSON into markdown

1. It sorts the segments in reading order (top-to-bottom, left-to-right)
2. Formats the text according to its category
3. Skips page footers and page headers (optionally)

Basically, you can write down your own markdown serializer

Sources:
https://github.com/rednote-hilab/dots.ocr/blob/master/dots_ocr/utils/format_transformer.py#L145
https://huggingface.co/spaces/MohamedRashad/Dots-OCR/blob/main/app.py#L240