BERT-VBD: Vietnamese Multi-Document Summarization Framework
In tackling the challenge of Multi-Document Summarization (MDS), numerous
methods have been proposed, spanning both extractive and abstractive
summarization techniques. However, each approach has its own limitations,
making it less effective to rely solely on either one. An emerging and
promising strategy involves a synergistic fusion of extractive and abstractive
summarization methods. Despite the plethora of studies in this domain, research
on the combined methodology remains scarce, particularly in the context of
Vietnamese language processing. This paper presents a novel Vietnamese MDS
framework leveraging a two-component pipeline architecture that integrates
extractive and abstractive techniques. The first component employs an
extractive approach to identify key sentences within each document. This is
achieved by a modification of the pre-trained BERT network, which derives
semantically meaningful phrase embeddings using siamese and triplet network
structures. The second component utilizes the VBD-LLaMA2-7B-50b model for
abstractive summarization, ultimately generating the final summary document.
Our proposed framework demonstrates a positive performance, attaining ROUGE-2
scores of 39.6% on the VN-MDS dataset and outperforming the state-of-the-art
baselines.
66ec6d245a65f26be78a1e9a
Tuan-Cuong Vuong
66ec6d245a65f26be78a1e9b
Trang Mai Xuan
66ec6d245a65f26be78a1e9c
Thien Van Luong
BERT-VBD: Vietnamese Multi-Document Summarization Framework
In tackling the challenge of Multi-Document Summarization (MDS), numerous
methods have been proposed, spanning both extractive and abstractive
summarization techniques. However, each approach has its own limitations,
making it less effective to rely solely on either one. An emerging and
promising strategy involves a synergistic fusion of extractive and abstractive
summarization methods. Despite the plethora of studies in this domain, research
on the combined methodology remains scarce, particularly in the context of
Vietnamese language processing. This paper presents a novel Vietnamese MDS
framework leveraging a two-component pipeline architecture that integrates
extractive and abstractive techniques. The first component employs an
extractive approach to identify key sentences within each document. This is
achieved by a modification of the pre-trained BERT network, which derives
semantically meaningful phrase embeddings using siamese and triplet network
structures. The second component utilizes the VBD-LLaMA2-7B-50b model for
abstractive summarization, ultimately generating the final summary document.
Our proposed framework demonstrates a positive performance, attaining ROUGE-2
scores of 39.6% on the VN-MDS dataset and outperforming the state-of-the-art
baselines.
A Vietnamese MDS framework combines extractive and abstractive methods using modified BERT and VBD-LLaMA2 models, achieving superior performance on the VN-MDS dataset.
BERT siamese network triplet network VBD-LLaMA2-7B-50b ROUGE-2