--- language: - en license: apache-2.0 library_name: llm2ner base_model: EleutherAI/pythia-6.9b tags: - ner - span-detection - llm - pytorch pipeline_tag: token-classification model_name: ToMMeR-pythia-6.9b_L5_R64 source: https://github.com/VictorMorand/llm2ner paper: https://arxiv.org/abs/2510.19410 --- # ToMMeR-pythia-6.9b_L5_R64 ToMMeR is a lightweight probing model extracting emergent mention detection capabilities from early layers representations of any LLM backbone, achieving high Zero Shot recall across a wide set of 13 NER benchmarks. ## Checkpoint Details | Property | Value | |-----------|-------| | Base LLM | `EleutherAI/pythia-6.9b` | | Layer | 5| | #Params | 528.4K | # Usage ## Installation Our code can be installed with pip+git, Please visit the [repository](https://github.com/VictorMorand/llm2ner) for more details. ```bash pip install git+https://github.com/VictorMorand/llm2ner.git ``` ## Fancy Outputs ```python import llm2ner from llm2ner import ToMMeR tommer = ToMMeR.from_pretrained("llm2ner/ToMMeR-pythia-6.9b_L5_R64") # load Backbone llm, optionnally cut the unused layer to save GPU space. llm = llm2ner.utils.load_llm( tommer.llm_name, cut_to_layer=tommer.layer,) tommer.to(llm.device) text = "Large language models are awesome. While trained on language modeling, they exhibit emergent Zero Shot abilities that make them suitable for a wide range of tasks, including Named Entity Recognition (NER). " #fancy interactive output outputs = llm2ner.plotting.demo_inference( text, tommer, llm, decoding_strategy="threshold", # or "greedy" for flat segmentation threshold=0.5, # default 50% show_attn=True, ) ```

Large PRED language PRED models are awesome . While trained on language PRED modeling , they exhibit emergent PRED abilities that make them suitable for a wide range of tasks PRED , including Named PRED Entity Recognition ( NER PRED ) .

## Raw inference By default, ToMMeR outputs span probabilities, but we also propose built-in options for decoding entities. - Inputs: - tokens (batch, seq): tokens to process, - model: LLM to extract representation from. - Outputs: (batch, seq, seq) matrix (masked outside valid spans) ```python tommer = ToMMeR.from_pretrained("llm2ner/ToMMeR-pythia-6.9b_L5_R64") # load Backbone llm, optionnally cut the unused layer to save GPU space. llm = llm2ner.utils.load_llm( tommer.llm_name, cut_to_layer=tommer.layer,) tommer.to(llm.device) #### Raw Inference text = ["Large language models are awesome"] print(f"Input text: {text[0]}") #tokenize in shape (1, seq_len) tokens = model.tokenizer(text, return_tensors="pt")["input_ids"].to(device) # Output raw scores output = tommer.forward(tokens, model) # (batch_size, seq_len, seq_len) print(f"Raw Output shape: {output.shape}") #use given decoding strategy to infer entities entities = tommer.infer_entities(tokens=tokens, model=model, threshold=0.5, decoding_strategy="greedy") str_entities = [ model.tokenizer.decode(tokens[0,b:e+1]) for b, e in entities[0]] print(f"Predicted entities: {str_entities}") >>> Input text: Large language models are awesome >>> Raw Output shape: torch.Size([1, 6, 6]) >>> Predicted entities: ['Large language models'] ``` Please visit the [repository](https://github.com/VictorMorand/llm2ner) for more details and a demo notebook. ## Evaluation Results | dataset | precision | recall | f1 | n_samples | |---------------------|-------------|----------|--------|-------------| | MultiNERD | 0.2321 | 0.9522 | 0.3732 | 154144 | | CoNLL 2003 | 0.2954 | 0.6994 | 0.4154 | 16493 | | CrossNER_politics | 0.3186 | 0.9398 | 0.4759 | 1389 | | CrossNER_AI | 0.3658 | 0.9029 | 0.5207 | 879 | | CrossNER_literature | 0.354 | 0.8738 | 0.5039 | 916 | | CrossNER_science | 0.3683 | 0.9109 | 0.5245 | 1193 | | CrossNER_music | 0.3867 | 0.9057 | 0.542 | 945 | | ncbi | 0.1268 | 0.8529 | 0.2207 | 3952 | | FabNER | 0.3089 | 0.6339 | 0.4154 | 13681 | | WikiNeural | 0.2202 | 0.9277 | 0.356 | 92672 | | GENIA_NER | 0.2589 | 0.9177 | 0.4039 | 16563 | | ACE 2005 | 0.2533 | 0.2962 | 0.2731 | 8230 | | Ontonotes | 0.2605 | 0.6408 | 0.3704 | 42193 | | Aggregated | 0.2407 | 0.8658 | 0.3767 | 353250 | | Mean | 0.2884 | 0.8041 | 0.415 | 353250 | ## Citation If using this model or the approach, please cite the associated paper: ``` @misc{morand2025tommerefficiententity, title={ToMMeR -- Efficient Entity Mention Detection from Large Language Models}, author={Victor Morand and Nadi Tomeh and Josiane Mothe and Benjamin Piwowarski}, year={2025}, eprint={2510.19410}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2510.19410}, } ``` ## License Apache-2.0 (see repository for full text).