I disagree with Michael that you did a bad job addressing Eliezer's arguments but agree that a couple key points might change your view especially around clippy stuff. The two key things seem to be:
(1) viability of takeoff scenarios, which I think has not been argued too precisely by anyone including Eliezer but the arguments I'm aware o…
I disagree with Michael that you did a bad job addressing Eliezer's arguments but agree that a couple key points might change your view especially around clippy stuff. The two key things seem to be:
(1) viability of takeoff scenarios, which I think has not been argued too precisely by anyone including Eliezer but the arguments I'm aware of are: (1a) AlphaGo and other AI accomplishments tend not to stay near human level for long once we get there and (1b) evolutionary history was slow until humans got a little more intelligent than other apes before exploding into the scene in terms of evolutionary timescales. Both might work similarly for first AGIs.
(2) no one knows how to make an AI care about anything or have almost any inner property at all and so after reflection it would care about something different from what you trained it to care about. This is said better in Eliezer's "AGI Ruin" post, #16-19.
I don't expect this to automatically flip all your views but I do think these are your key points of disagreement. I'd love to see you two in a friendly public debate on the topic.
But surely AI could be trained to care until it becomes part of core code. I believe this is how nature did it too from its early beginnings. If caring for itself and others forms part of a survival protocol then that part of its code would surely be passed on to later generations as happens with genetics. I don't see the difference really. It's all information in the end surely. And we only have that code in us because it also improved our survivability at some point.
I disagree with Michael that you did a bad job addressing Eliezer's arguments but agree that a couple key points might change your view especially around clippy stuff. The two key things seem to be:
(1) viability of takeoff scenarios, which I think has not been argued too precisely by anyone including Eliezer but the arguments I'm aware of are: (1a) AlphaGo and other AI accomplishments tend not to stay near human level for long once we get there and (1b) evolutionary history was slow until humans got a little more intelligent than other apes before exploding into the scene in terms of evolutionary timescales. Both might work similarly for first AGIs.
(2) no one knows how to make an AI care about anything or have almost any inner property at all and so after reflection it would care about something different from what you trained it to care about. This is said better in Eliezer's "AGI Ruin" post, #16-19.
I don't expect this to automatically flip all your views but I do think these are your key points of disagreement. I'd love to see you two in a friendly public debate on the topic.
But surely AI could be trained to care until it becomes part of core code. I believe this is how nature did it too from its early beginnings. If caring for itself and others forms part of a survival protocol then that part of its code would surely be passed on to later generations as happens with genetics. I don't see the difference really. It's all information in the end surely. And we only have that code in us because it also improved our survivability at some point.