IntrEx: A Dataset for Modeling Engagement in Educational Conversations
Engagement and motivation are crucial for second-language acquisition, yet
maintaining learner interest in educational conversations remains a challenge.
While prior research has explored what makes educational texts interesting,
still little is known about the linguistic features that drive engagement in
conversations. To address this gap, we introduce IntrEx, the first large
dataset annotated for interestingness and expected interestingness in
teacher-student interactions. Built upon the Teacher-Student Chatroom Corpus
(TSCC), IntrEx extends prior work by incorporating sequence-level annotations,
allowing for the study of engagement beyond isolated turns to capture how
interest evolves over extended dialogues. We employ a rigorous annotation
process with over 100 second-language learners, using a comparison-based rating
approach inspired by reinforcement learning from human feedback (RLHF) to
improve agreement. We investigate whether large language models (LLMs) can
predict human interestingness judgments. We find that LLMs (7B/8B parameters)
fine-tuned on interestingness ratings outperform larger proprietary models like
GPT-4o, demonstrating the potential for specialised datasets to model
engagement in educational settings. Finally, we analyze how linguistic and
cognitive factors, such as concreteness, comprehensibility (readability), and
uptake, influence engagement in educational dialogues.
68bfe4ea207285de11b07e16
Xingwei Tan
claimed_verified
2025-09-09T13:46:14.706Z
643d0a4d8a55b2bbf4f2a90e
/avatars/9534aaf81cbf12f015c6826b682fdb84.svg
Xingwei Tan
user
XingweiT
68bfe4ea207285de11b07e17
Mahathi Parvatham
68bfe4ea207285de11b07e18
Chiara Gambi
68bfe4ea207285de11b07e19
Gabriele Pergola
IntrEx: A Dataset for Modeling Engagement in Educational Conversations
Engagement and motivation are crucial for second-language acquisition, yet
maintaining learner interest in educational conversations remains a challenge.
While prior research has explored what makes educational texts interesting,
still little is known about the linguistic features that drive engagement in
conversations. To address this gap, we introduce IntrEx, the first large
dataset annotated for interestingness and expected interestingness in
teacher-student interactions. Built upon the Teacher-Student Chatroom Corpus
(TSCC), IntrEx extends prior work by incorporating sequence-level annotations,
allowing for the study of engagement beyond isolated turns to capture how
interest evolves over extended dialogues. We employ a rigorous annotation
process with over 100 second-language learners, using a comparison-based rating
approach inspired by reinforcement learning from human feedback (RLHF) to
improve agreement. We investigate whether large language models (LLMs) can
predict human interestingness judgments. We find that LLMs (7B/8B parameters)
fine-tuned on interestingness ratings outperform larger proprietary models like
GPT-4o, demonstrating the potential for specialised datasets to model
engagement in educational settings. Finally, we analyze how linguistic and
cognitive factors, such as concreteness, comprehensibility (readability), and
uptake, influence engagement in educational dialogues.
IntrEx, a large dataset annotated for interestingness in educational conversations, shows that fine-tuned LLMs can predict human judgments of interestingness better than larger proprietary models, highlighting the role of linguistic and cognitive factors in engagement.
large language models fine-tuning reinforcement learning from human feedback RLHF IntrEx Teacher-Student Chatroom Corpus TSCC sequence-level annotations concreteness comprehensibility uptake