LLaMo: Large Language Model-based Molecular Graph Assistant
Large Language Models (LLMs) have demonstrated remarkable generalization and
instruction-following capabilities with instruction tuning. The advancements in
LLMs and instruction tuning have led to the development of Large
Vision-Language Models (LVLMs). However, the competency of the LLMs and
instruction tuning have been less explored in the molecular domain. Thus, we
propose LLaMo: Large Language Model-based Molecular graph assistant, which is
an end-to-end trained large molecular graph-language model. To bridge the
discrepancy between the language and graph modalities, we present the
multi-level graph projector that transforms graph representations into graph
tokens by abstracting the output representations of each GNN layer and motif
representations with the cross-attention mechanism. We also introduce
machine-generated molecular graph instruction data to instruction-tune the
large molecular graph-language model for general-purpose molecule and language
understanding. Our extensive experiments demonstrate that LLaMo shows the best
performance on diverse tasks, such as molecular description generation,
property prediction, and IUPAC name prediction. The code of LLaMo is available
at https://github.com/mlvlab/LLaMo.
672aed3f44b2453edfdbeafd
Jinyoung Park
claimed_verified
2024-11-15T09:37:04.989Z
64b6eae88ba7d6c922c0434a
/avatars/09977e1f12bbbf0f44c106b6f6b1f0df.svg
Jinyoung Park
user
jinypark
672aed3f44b2453edfdbeafe
Minseong Bae
claimed_verified
2025-01-02T10:19:29.383Z
64549b4cc13cdb83f1100b94
/avatars/4e809bed509e6ce2dd5532730af81a12.svg
Minseong Bae
user
KyleBae1017
672aed3f44b2453edfdbeaff
Dohwan Ko
claimed_verified
2025-03-26T20:50:10.384Z
62bab5a69f045fd1fb9f760e
/avatars/3f12a908faec1135e0221444f0769496.svg
Dohwan Ko
user
ikodoh
672aed3f44b2453edfdbeb00
Hyunwoo J. Kim
claimed_verified
2025-06-16T15:51:30.858Z
63183a30a760ff8cff318be5
/avatars/1b314afd8d3319423915ffc2ae56d567.svg
Hyunwoo Kim
user
hkimcvml
LLaMo: Large Language Model-based Molecular Graph Assistant
Large Language Models (LLMs) have demonstrated remarkable generalization and
instruction-following capabilities with instruction tuning. The advancements in
LLMs and instruction tuning have led to the development of Large
Vision-Language Models (LVLMs). However, the competency of the LLMs and
instruction tuning have been less explored in the molecular domain. Thus, we
propose LLaMo: Large Language Model-based Molecular graph assistant, which is
an end-to-end trained large molecular graph-language model. To bridge the
discrepancy between the language and graph modalities, we present the
multi-level graph projector that transforms graph representations into graph
tokens by abstracting the output representations of each GNN layer and motif
representations with the cross-attention mechanism. We also introduce
machine-generated molecular graph instruction data to instruction-tune the
large molecular graph-language model for general-purpose molecule and language
understanding. Our extensive experiments demonstrate that LLaMo shows the best
performance on diverse tasks, such as molecular description generation,
property prediction, and IUPAC name prediction. The code of LLaMo is available
at https://github.com/mlvlab/LLaMo.
LLaMo, a large language model for molecular graphs, uses a multi-level graph projector and cross-attention to achieve top performance in tasks like molecular description, property prediction, and IUPAC name generation.
Large Language Models Large Vision-Language Models LLaMo molecular graph assistant end-to-end trained multi-level graph projector graph tokens GNN layer motif representations cross-attention mechanism machine-generated molecular graph instruction data instruction-tuning molecular description generation property prediction IUPAC name prediction