Language agents reduce the risk of existential catastrophe
Journal article
Goldstein, Simon and Kirk-Giannini, Cameron Domenico. (2023). Language agents reduce the risk of existential catastrophe. AI & Society. pp. 1-11. https://doi.org/10.1007/s00146-023-01748-4
Authors | Goldstein, Simon and Kirk-Giannini, Cameron Domenico |
---|---|
Abstract | Recent advances in natural-language processing have given rise to a new kind of AI architecture: the language agent. By repeatedly calling an LLM to perform a variety of cognitive tasks, language agents are able to function autonomously to pursue goals specified in natural language and stored in a human-readable format. Because of their architecture, language agents exhibit behavior that is predictable according to the laws of folk psychology: they function as though they have desires and beliefs, and then make and update plans to pursue their desires given their beliefs. We argue that the rise of language agents significantly reduces the probability of an existential catastrophe due to loss of control over an AGI. This is because the probability of such an existential catastrophe is proportional to the difficulty of aligning AGI systems, and language agents significantly reduce that difficulty. In particular, language agents help to resolve three important issues related to aligning AIs: reward misspecification, goal misgeneralization, and uninterpretability. |
Keywords | language agents; existential risk; reward misspecification; goal misgeneralization; interpretable AI |
Year | 2023 |
Journal | AI & Society |
Journal citation | pp. 1-11 |
Publisher | Springer |
ISSN | 0951-5666 |
Digital Object Identifier (DOI) | https://doi.org/10.1007/s00146-023-01748-4 |
Scopus EID | 2-s2.0-85168370049 |
Page range | 1-11 |
Funder | The Center for AI Safety |
Publisher's version | License All rights reserved File Access Level Controlled |
Output status | In press |
Publication dates | |
Online | 19 Aug 2023 |
Publication process dates | |
Accepted | 03 Aug 2023 |
Deposited | 08 Nov 2023 |
https://acuresearchbank.acu.edu.au/item/8zy79/language-agents-reduce-the-risk-of-existential-catastrophe
Restricted files
Publisher's version
45
total views0
total downloads0
views this month0
downloads this month