How Programming Concepts and Neurons Are Shared in Code Language Models
Several studies have explored the mechanisms of large language models (LLMs)
in coding tasks, but most have focused on programming languages (PLs) in a
monolingual setting. In this paper, we investigate the relationship between
multiple PLs and English in the concept space of LLMs. We perform a few-shot
translation task on 21 PL pairs using two Llama-based models. By decoding the
embeddings of intermediate layers during this task, we observe that the concept
space is closer to English (including PL keywords) and assigns high
probabilities to English tokens in the second half of the intermediate layers.
We analyze neuron activations for 11 PLs and English, finding that while
language-specific neurons are primarily concentrated in the bottom layers,
those exclusive to each PL tend to appear in the top layers. For PLs that are
highly aligned with multiple other PLs, identifying language-specific neurons
is not feasible. These PLs also tend to have a larger keyword set than other
PLs and are closer to the model's concept space regardless of the input/output
PL in the translation task. Our findings provide insights into how LLMs
internally represent PLs, revealing structural patterns in the model's concept
space. Code is available at https://github.com/cisnlp/code-specific-neurons.
683e923450c43488f7ad5e02
Amir Hossein Kargaran
claimed_verified
2025-06-03T08:39:12.648Z
61bf84c8ca59d6d196a1b4e8
https://cdn-avatars.huggingface.co/v1/production/uploads/61bf84c8ca59d6d196a1b4e8/L_NvUwlMYcye9X35z6f7e.jpeg
Amir Hossein Kargaran
user
kargaranamir
683e923450c43488f7ad5e03
Yihong Liu
claimed_verified
2025-06-05T07:28:34.979Z
653f7e569e84d1e8b6a66e70
/avatars/24eaa6434508a162c349aebfc51990ff.svg
Yihong Liu
user
yihongLiu
683e923450c43488f7ad5e04
François Yvon
683e923450c43488f7ad5e05
Hinrich Schütze
How Programming Concepts and Neurons Are Shared in Code Language Models
Several studies have explored the mechanisms of large language models (LLMs)
in coding tasks, but most have focused on programming languages (PLs) in a
monolingual setting. In this paper, we investigate the relationship between
multiple PLs and English in the concept space of LLMs. We perform a few-shot
translation task on 21 PL pairs using two Llama-based models. By decoding the
embeddings of intermediate layers during this task, we observe that the concept
space is closer to English (including PL keywords) and assigns high
probabilities to English tokens in the second half of the intermediate layers.
We analyze neuron activations for 11 PLs and English, finding that while
language-specific neurons are primarily concentrated in the bottom layers,
those exclusive to each PL tend to appear in the top layers. For PLs that are
highly aligned with multiple other PLs, identifying language-specific neurons
is not feasible. These PLs also tend to have a larger keyword set than other
PLs and are closer to the model's concept space regardless of the input/output
PL in the translation task. Our findings provide insights into how LLMs
internally represent PLs, revealing structural patterns in the model's concept
space. Code is available at https://github.com/cisnlp/code-specific-neurons.
LLMs representing multiple programming languages in their concept space tend to cluster closer to English and exhibit distinct neuron activations for specific languages, particularly in the upper layers, with highly aligned languages sharing similar representations.
large language models LLMs programming languages PLs concept space few-shot translation embeddings intermediate layers neuron activations language-specific neurons