Language Models Model Language
Linguistic commentary on LLMs, heavily influenced by the theoretical
frameworks of de Saussure and Chomsky, is often speculative and unproductive.
Critics challenge whether LLMs can legitimately model language, citing the need
for "deep structure" or "grounding" to achieve an idealized linguistic
"competence." We argue for a radical shift in perspective towards the
empiricist principles of Witold Ma\'nczak, a prominent general and historical
linguist. He defines language not as a "system of signs" or a "computational
system of the brain" but as the totality of all that is said and written. Above
all, he identifies frequency of use of particular language elements as
language's primary governing principle. Using his framework, we challenge prior
critiques of LLMs and provide a constructive guide for designing, evaluating,
and interpreting language models.
68f5d9e88589920bf4d3220b
Łukasz Borchmann
claimed_verified
2025-10-24T16:14:06.207Z
600b381d3cc3b87db94bc0ce
https://cdn-avatars.huggingface.co/v1/production/uploads/600b381d3cc3b87db94bc0ce/I3xpr4gzcG1uXawXBpWpD.jpeg
Łukasz Borchmann
user
Borchmann
Language Models Model Language
Linguistic commentary on LLMs, heavily influenced by the theoretical
frameworks of de Saussure and Chomsky, is often speculative and unproductive.
Critics challenge whether LLMs can legitimately model language, citing the need
for "deep structure" or "grounding" to achieve an idealized linguistic
"competence." We argue for a radical shift in perspective towards the
empiricist principles of Witold Ma\'nczak, a prominent general and historical
linguist. He defines language not as a "system of signs" or a "computational
system of the brain" but as the totality of all that is said and written. Above
all, he identifies frequency of use of particular language elements as
language's primary governing principle. Using his framework, we challenge prior
critiques of LLMs and provide a constructive guide for designing, evaluating,
and interpreting language models.
The paper advocates for an empiricist approach to evaluating language models, emphasizing frequency of use over traditional theoretical frameworks.