I don't think the speakers themselves would want to play the role of 'designated asshole'. After all, the problems with this are largely the same as the ones with having the current speaker cut off bad questions - they need to be polite as speakers.

As you said, I think this works better in mathematical conferences and academic scenarios because people must be willing to accept objective criticism of their ideas.