The Dragon Hatchling: The Missing Link between the Transformer and
  Models of the Brain
The relationship between computing systems and the brain has served as
motivation for pioneering theoreticians since John von Neumann and Alan Turing.
Uniform, scale-free biological networks, such as the brain, have powerful
properties, including generalizing over time, which is the main barrier for
Machine Learning on the path to Universal Reasoning Models.
  We introduce `Dragon Hatchling' (BDH), a new Large Language Model
architecture based on a scale-free biologically inspired network of \n
locally-interacting neuron particles. BDH couples strong theoretical
foundations and inherent interpretability without sacrificing Transformer-like
performance.
  BDH is a practical, performant state-of-the-art attention-based state space
sequence learning architecture. In addition to being a graph model, BDH admits
a GPU-friendly formulation. It exhibits Transformer-like scaling laws:
empirically BDH rivals GPT2 performance on language and translation tasks, at
the same number of parameters (10M to 1B), for the same training data.
  BDH can be represented as a brain model. The working memory of BDH during
inference entirely relies on synaptic plasticity with Hebbian learning using
spiking neurons. We confirm empirically that specific, individual synapses
strengthen connection whenever BDH hears or reasons about a specific concept
while processing language inputs. The neuron interaction network of BDH is a
graph of high modularity with heavy-tailed degree distribution. The BDH model
is biologically plausible, explaining one possible mechanism which human
neurons could use to achieve speech.
  BDH is designed for interpretability. Activation vectors of BDH are sparse
and positive. We demonstrate monosemanticity in BDH on language tasks.
Interpretability of state, which goes beyond interpretability of neurons and
model parameters, is an inherent feature of the BDH architecture.
68dc87954159d1f2418f9a13
Adrian Kosowski
68dc87954159d1f2418f9a14
Przemysław Uznański
admin_assigned
2025-10-01T15:32:27.465Z
6763fd76d63e4b348eac8290
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/4ZtRLbXBmrXVQgpQzPpDm.png
Przemysław Uznański
user
izulin
68dc87954159d1f2418f9a15
Jan Chorowski
claimed_verified
2025-10-01T10:22:24.524Z
651310e3b7994cff61d71320
/avatars/161cb412c7cb284fa7483e790807e4d3.svg
Jan Chorowski
user
janchorowski
68dc87954159d1f2418f9a16
Zuzanna Stamirowska
68dc87954159d1f2418f9a17
Michał Bartoszkiewicz
claimed_verified
2025-10-23T15:02:49.073Z
68ac8c07747bd02a9fcfb967
https://cdn-avatars.huggingface.co/v1/production/uploads/68ac8c07747bd02a9fcfb967/xm8uyJqDCqhQBQoaVN_RX.png
Michał Bartoszkiewicz
user
embe-pw
The Dragon Hatchling: The Missing Link between the Transformer and
  Models of the Brain
The relationship between computing systems and the brain has served as
motivation for pioneering theoreticians since John von Neumann and Alan Turing.
Uniform, scale-free biological networks, such as the brain, have powerful
properties, including generalizing over time, which is the main barrier for
Machine Learning on the path to Universal Reasoning Models.
  We introduce `Dragon Hatchling' (BDH), a new Large Language Model
architecture based on a scale-free biologically inspired network of \n
locally-interacting neuron particles. BDH couples strong theoretical
foundations and inherent interpretability without sacrificing Transformer-like
performance.
  BDH is a practical, performant state-of-the-art attention-based state space
sequence learning architecture. In addition to being a graph model, BDH admits
a GPU-friendly formulation. It exhibits Transformer-like scaling laws:
empirically BDH rivals GPT2 performance on language and translation tasks, at
the same number of parameters (10M to 1B), for the same training data.
  BDH can be represented as a brain model. The working memory of BDH during
inference entirely relies on synaptic plasticity with Hebbian learning using
spiking neurons. We confirm empirically that specific, individual synapses
strengthen connection whenever BDH hears or reasons about a specific concept
while processing language inputs. The neuron interaction network of BDH is a
graph of high modularity with heavy-tailed degree distribution. The BDH model
is biologically plausible, explaining one possible mechanism which human
neurons could use to achieve speech.
  BDH is designed for interpretability. Activation vectors of BDH are sparse
and positive. We demonstrate monosemanticity in BDH on language tasks.
Interpretability of state, which goes beyond interpretability of neurons and
model parameters, is an inherent feature of the BDH architecture.
BDH, a biologically inspired Large Language Model, combines scale-free network architecture with Hebbian learning to achieve Transformer-like performance while maintaining interpretability.
Large Language Model scale-free network Hebbian learning synaptic plasticity spiking neurons graph model GPU-friendly Transformer-like scaling laws monosemanticity interpretability