Open-Endedness is Essential for Artificial Superhuman Intelligence
In recent years there has been a tremendous surge in the general capabilities
of AI systems, mainly fuelled by training foundation models on internetscale
data. Nevertheless, the creation of openended, ever self-improving AI remains
elusive. In this position paper, we argue that the ingredients are now in place
to achieve openendedness in AI systems with respect to a human observer.
Furthermore, we claim that such open-endedness is an essential property of any
artificial superhuman intelligence (ASI). We begin by providing a concrete
formal definition of open-endedness through the lens of novelty and
learnability. We then illustrate a path towards ASI via open-ended systems
built on top of foundation models, capable of making novel, humanrelevant
discoveries. We conclude by examining the safety implications of
generally-capable openended AI. We expect that open-ended foundation models
will prove to be an increasingly fertile and safety-critical area of research
in the near future.
66627275648349b3930079ba
Edward Hughes
admin_assigned
2024-06-07T12:54:03.783Z
64dcad22249785efad8da97b
/avatars/11ebe5dec1e5d2e3c7cd00ec684e2319.svg
Edward Hughes
user
visible-ed
66627275648349b3930079bb
Michael Dennis
66627275648349b3930079bc
Jack Parker-Holder
66627275648349b3930079bd
Feryal Behbahani
admin_assigned
2024-06-07T12:54:36.327Z
64c197b7ec3c6181351b2c5d
/avatars/f81fe98f68720b95885d4968b0e7cc0f.svg
Feryal Behbahani
user
ferya
66627275648349b3930079be
Aditi Mavalankar
66627275648349b3930079bf
Yuge Shi
66627275648349b3930079c0
Tom Schaul
66627275648349b3930079c1
Tim Rocktaschel
Open-Endedness is Essential for Artificial Superhuman Intelligence
In recent years there has been a tremendous surge in the general capabilities
of AI systems, mainly fuelled by training foundation models on internetscale
data. Nevertheless, the creation of openended, ever self-improving AI remains
elusive. In this position paper, we argue that the ingredients are now in place
to achieve openendedness in AI systems with respect to a human observer.
Furthermore, we claim that such open-endedness is an essential property of any
artificial superhuman intelligence (ASI). We begin by providing a concrete
formal definition of open-endedness through the lens of novelty and
learnability. We then illustrate a path towards ASI via open-ended systems
built on top of foundation models, capable of making novel, humanrelevant
discoveries. We conclude by examining the safety implications of
generally-capable openended AI. We expect that open-ended foundation models
will prove to be an increasingly fertile and safety-critical area of research
in the near future.
foundational models open-endedness ASI (artificial superhuman intelligence) novelty learnability open-ended systems human-relevant discoveries safety implications