Rethinking Interpretability in the Era of Large Language Models
Interpretable machine learning has exploded as an area of interest over the
last decade, sparked by the rise of increasingly large datasets and deep neural
networks. Simultaneously, large language models (LLMs) have demonstrated
remarkable capabilities across a wide array of tasks, offering a chance to
rethink opportunities in interpretable machine learning. Notably, the
capability to explain in natural language allows LLMs to expand the scale and
complexity of patterns that can be given to a human. However, these new
capabilities raise new challenges, such as hallucinated explanations and
immense computational costs.
  In this position paper, we start by reviewing existing methods to evaluate
the emerging field of LLM interpretation (both interpreting LLMs and using LLMs
for explanation). We contend that, despite their limitations, LLMs hold the
opportunity to redefine interpretability with a more ambitious scope across
many applications, including in auditing LLMs themselves. We highlight two
emerging research priorities for LLM interpretation: using LLMs to directly
analyze new datasets and to generate interactive explanations.
65c1b38ec79ecb36bea97912
Chandan Singh
claimed_verified
2024-02-06T16:22:09.793Z
62b0bee117fa24819dd0999c
https://cdn-avatars.huggingface.co/v1/production/uploads/62b0bee117fa24819dd0999c/eJ7ryFQ3_ZbqV29qf4KyM.jpeg
Chandan Singh
user
csinva
65c1b38ec79ecb36bea97913
Jeevana Priya Inala
65c1b38ec79ecb36bea97914
Michel Galley
claimed_verified
2025-02-11T10:36:11.363Z
5f6bed855e78cc6b0ed31dee
/avatars/d67edb079fa3fe7ea6f2081f7a3afe9e.svg
Michel Galley
user
mgalley
65c1b38ec79ecb36bea97915
Rich Caruana
65c1b38ec79ecb36bea97916
Jianfeng Gao
Rethinking Interpretability in the Era of Large Language Models
Interpretable machine learning has exploded as an area of interest over the
last decade, sparked by the rise of increasingly large datasets and deep neural
networks. Simultaneously, large language models (LLMs) have demonstrated
remarkable capabilities across a wide array of tasks, offering a chance to
rethink opportunities in interpretable machine learning. Notably, the
capability to explain in natural language allows LLMs to expand the scale and
complexity of patterns that can be given to a human. However, these new
capabilities raise new challenges, such as hallucinated explanations and
immense computational costs.
  In this position paper, we start by reviewing existing methods to evaluate
the emerging field of LLM interpretation (both interpreting LLMs and using LLMs
for explanation). We contend that, despite their limitations, LLMs hold the
opportunity to redefine interpretability with a more ambitious scope across
many applications, including in auditing LLMs themselves. We highlight two
emerging research priorities for LLM interpretation: using LLMs to directly
analyze new datasets and to generate interactive explanations.
Large language models offer new opportunities in interpretable machine learning, enabling more complex explanations but also introducing challenges like hallucinations and high computational costs.
large language models LLM interpretation LLMs interpretability auditing LLMs interactive explanations