>I am much more disturbed by the prospect of a tyrannical or even "Friendly" AI some of them seem to be fond of.
Same. Even if such a thing is possible, the probability that it's done right the first time is close to nil. And since a recursively improving singularity A.I can be assumed to irreversibly take control of the balance of power, it's not really something that you could afford to screw up.
Of course, Yudowsky (To the extent that he's doing anything.) seems to be working off the assumption that if he doesn't do it someone else will.
Same. Even if such a thing is possible, the probability that it's done right the first time is close to nil. And since a recursively improving singularity A.I can be assumed to irreversibly take control of the balance of power, it's not really something that you could afford to screw up.
Of course, Yudowsky (To the extent that he's doing anything.) seems to be working off the assumption that if he doesn't do it someone else will.