> Supraintelligent AI being evil is FUD imho because we can't reason about supraintelligent AI.
There is a difference between being evil and incomprehensible intelligence. You are not being evil when you accidentally step on an ant or dig up an ant-hill to build your shed. The ants won't be able to understand what you're doing, or why.
Well that's what I'm saying: We can't know if it's being evil unless we know everything about it, and if we knew that, we'd be the supraintelligent beings in the equation. Thinking that it'll go off and dominate the world is thinking about the worst case. So why bother since it's not likely we'd be able to do much about it anyway?
Maybe there's a second AI on the same level as the first and thinks the first AI is evil. We're still dumb as rocks compared to them, but something certainly has that opinion.
There is a difference between being evil and incomprehensible intelligence. You are not being evil when you accidentally step on an ant or dig up an ant-hill to build your shed. The ants won't be able to understand what you're doing, or why.