Shutdown-seeking AI
Journal article
Goldstein, Simon David and Robinson, Pamela. (2024). Shutdown-seeking AI. Philosophical Studies : an international journal for philosophy in the analytic tradition. pp. 1-13. https://doi.org/10.1007/s11098-024-02099-6
Authors | Goldstein, Simon David and Robinson, Pamela |
---|---|
Abstract | We propose developing AIs whose only final goal is being shut down. We argue that this approach to AI safety has three benefits: (i) it could potentially be implemented in reinforcement learning, (ii) it avoids some dangerous instrumental convergence dynamics, and (iii) it creates trip wires for monitoring dangerous capabilities. We also argue that the proposal can overcome a key challenge raised by Soares et al. (2015), that shutdown-seeking AIs will manipulate humans into shutting them down. We conclude by comparing our approach with Soares et al.'s corrigibility framework. |
Keywords | AI safety; Instrumental convergence; Reward misspecification |
Year | 01 Jan 2024 |
Journal | Philosophical Studies : an international journal for philosophy in the analytic tradition |
Journal citation | pp. 1-13 |
Publisher | Springer Science and Business Media B.V. |
ISSN | 0031-8116 |
Digital Object Identifier (DOI) | https://doi.org/10.1007/s11098-024-02099-6 |
Web address (URL) | https://link.springer.com/article/10.1007/s11098-024-02099-6 |
Open access | Published as non-open access |
Research or scholarly | Research |
Page range | 1-13 |
Publisher's version | License File Access Level Open |
Output status | Published |
Publication dates | |
Online | 06 Jun 2024 |
Publication process dates | |
Accepted | 30 Dec 2023 |
Deposited | 04 Oct 2024 |
Additional information | © The Author(s) 2024 |
This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Com- | |
Place of publication | Netherlands |
https://acuresearchbank.acu.edu.au/item/90zy8/shutdown-seeking-ai
Download files
Publisher's version
OA_Goldstein_2024_Shutdown_seeking_AI.pdf | |
License: CC BY 4.0 | |
File access level: Open |
11
total views3
total downloads0
views this month0
downloads this month