GuideX: Guided Synthetic Data Generation for Zero-Shot Information
  Extraction
Information Extraction (IE) systems are traditionally domain-specific,
requiring costly adaptation that involves expert schema design, data
annotation, and model training. While Large Language Models have shown promise
in zero-shot IE, performance degrades significantly in unseen domains where
label definitions differ. This paper introduces GUIDEX, a novel method that
automatically defines domain-specific schemas, infers guidelines, and generates
synthetically labeled instances, allowing for better out-of-domain
generalization. Fine-tuning Llama 3.1 with GUIDEX sets a new state-of-the-art
across seven zeroshot Named Entity Recognition benchmarks. Models trained with
GUIDEX gain up to 7 F1 points over previous methods without humanlabeled data,
and nearly 2 F1 points higher when combined with it. Models trained on GUIDEX
demonstrate enhanced comprehension of complex, domain-specific annotation
schemas. Code, models, and synthetic datasets are available at
neilus03.github.io/guidex.com
6842edd6a4e3571765bb4916
Neil De La Fuente
claimed_verified
2025-06-07T05:45:23.265Z
64fee0a7a10455384ebba184
https://cdn-avatars.huggingface.co/v1/production/uploads/64fee0a7a10455384ebba184/KIzdANoxlbWLLqP2pEaka.jpeg
Neil de la fuente
user
neildlf
6842edd6a4e3571765bb4917
Oscar Sainz
claimed_verified
2025-06-07T05:45:25.760Z
6253d0cc27a8414d3bbea683
https://cdn-avatars.huggingface.co/v1/production/uploads/6253d0cc27a8414d3bbea683/fRck2wXEg-PS11ph9u8ZI.jpeg
Oscar Sainz
user
OSainz
6842edd6a4e3571765bb4918
Iker García-Ferrero
6842edd6a4e3571765bb4919
Eneko Agirre
GuideX: Guided Synthetic Data Generation for Zero-Shot Information
  Extraction
Information Extraction (IE) systems are traditionally domain-specific,
requiring costly adaptation that involves expert schema design, data
annotation, and model training. While Large Language Models have shown promise
in zero-shot IE, performance degrades significantly in unseen domains where
label definitions differ. This paper introduces GUIDEX, a novel method that
automatically defines domain-specific schemas, infers guidelines, and generates
synthetically labeled instances, allowing for better out-of-domain
generalization. Fine-tuning Llama 3.1 with GUIDEX sets a new state-of-the-art
across seven zeroshot Named Entity Recognition benchmarks. Models trained with
GUIDEX gain up to 7 F1 points over previous methods without humanlabeled data,
and nearly 2 F1 points higher when combined with it. Models trained on GUIDEX
demonstrate enhanced comprehension of complex, domain-specific annotation
schemas. Code, models, and synthetic datasets are available at
neilus03.github.io/guidex.com
GUIDEX enhances zero-shot Named Entity Recognition by automatically defining schemas and inferring guidelines, setting new benchmarks without extensive human-labeled data.
Large Language Models zero-shot IE domain-specific schemas Named Entity Recognition fine-tuning Llama 3.1 synthetic labeled instances F1 points