OpenMed NER: Open-Source, Domain-Adapted State-of-the-Art Transformers
  for Biomedical NER Across 12 Public Datasets
Named-entity recognition (NER) is fundamental to extracting structured
information from the >80% of healthcare data that resides in unstructured
clinical notes and biomedical literature. Despite recent advances with large
language models, achieving state-of-the-art performance across diverse entity
types while maintaining computational efficiency remains a significant
challenge. We introduce OpenMed NER, a suite of open-source, domain-adapted
transformer models that combine lightweight domain-adaptive pre-training (DAPT)
with parameter-efficient Low-Rank Adaptation (LoRA). Our approach performs
cost-effective DAPT on a 350k-passage corpus compiled from ethically sourced,
publicly available research repositories and de-identified clinical notes
(PubMed, arXiv, and MIMIC-III) using DeBERTa-v3, PubMedBERT, and BioELECTRA
backbones. This is followed by task-specific fine-tuning with LoRA, which
updates less than 1.5% of model parameters. We evaluate our models on 12
established biomedical NER benchmarks spanning chemicals, diseases, genes, and
species. OpenMed NER achieves new state-of-the-art micro-F1 scores on 10 of
these 12 datasets, with substantial gains across diverse entity types. Our
models advance the state-of-the-art on foundational disease and chemical
benchmarks (e.g., BC5CDR-Disease, +2.70 pp), while delivering even larger
improvements of over 5.3 and 9.7 percentage points on more specialized gene and
clinical cell line corpora. This work demonstrates that strategically adapted
open-source models can surpass closed-source solutions. This performance is
achieved with remarkable efficiency: training completes in under 12 hours on a
single GPU with a low carbon footprint (< 1.2 kg CO2e), producing permissively
licensed, open-source checkpoints designed to help practitioners facilitate
compliance with emerging data protection and AI regulations, such as the EU AI
Act.
68916f8df01a094725f8347f
Maziyar Panahi
claimed_verified
2025-08-06T19:22:17.859Z
5fd5e18a90b6dc4633f6d292
https://cdn-avatars.huggingface.co/v1/production/uploads/5fd5e18a90b6dc4633f6d292/gZXHW5dd9R86AV9LMZ--y.png
Maziyar Panahi
user
MaziyarPanahi
OpenMed NER: Open-Source, Domain-Adapted State-of-the-Art Transformers
  for Biomedical NER Across 12 Public Datasets
Named-entity recognition (NER) is fundamental to extracting structured
information from the >80% of healthcare data that resides in unstructured
clinical notes and biomedical literature. Despite recent advances with large
language models, achieving state-of-the-art performance across diverse entity
types while maintaining computational efficiency remains a significant
challenge. We introduce OpenMed NER, a suite of open-source, domain-adapted
transformer models that combine lightweight domain-adaptive pre-training (DAPT)
with parameter-efficient Low-Rank Adaptation (LoRA). Our approach performs
cost-effective DAPT on a 350k-passage corpus compiled from ethically sourced,
publicly available research repositories and de-identified clinical notes
(PubMed, arXiv, and MIMIC-III) using DeBERTa-v3, PubMedBERT, and BioELECTRA
backbones. This is followed by task-specific fine-tuning with LoRA, which
updates less than 1.5% of model parameters. We evaluate our models on 12
established biomedical NER benchmarks spanning chemicals, diseases, genes, and
species. OpenMed NER achieves new state-of-the-art micro-F1 scores on 10 of
these 12 datasets, with substantial gains across diverse entity types. Our
models advance the state-of-the-art on foundational disease and chemical
benchmarks (e.g., BC5CDR-Disease, +2.70 pp), while delivering even larger
improvements of over 5.3 and 9.7 percentage points on more specialized gene and
clinical cell line corpora. This work demonstrates that strategically adapted
open-source models can surpass closed-source solutions. This performance is
achieved with remarkable efficiency: training completes in under 12 hours on a
single GPU with a low carbon footprint (< 1.2 kg CO2e), producing permissively
licensed, open-source checkpoints designed to help practitioners facilitate
compliance with emerging data protection and AI regulations, such as the EU AI
Act.
OpenMed NER, a suite of open-source transformer models using DAPT and LoRA, achieves state-of-the-art performance on diverse biomedical NER benchmarks with high efficiency and low computational cost.
transformer models lightweight domain-adaptive pre-training (DAPT) parameter-efficient Low-Rank Adaptation (LoRA) DeBERTa-v3 PubMedBERT BioELECTRA micro-F1 scores BC5CDR-Disease gene clinical cell line corpora open-source checkpoints EU AI Act