A similar thing occurred to me not too long ago, when I was learning about type theory: if you have the type of some function, then the set of possible implementations of that function is (usually) countably infinite, so the type doesn’t really tell you anything. But of course you probably care only about one of the simplest, so the type really does tell you an awful lot.

Anyway, I’m with you on the business of IQ tests. The problem is that as answers become increasingly open-ended, they are both more representative of a person’s actual intelligence, and more difficult to score objectively.