That is one reason they're problematic, but not the problem I described. Public IDs would make the problem I am describing worse, as even more businesses would unashamedly ask for them. People's feeling of "oh isn't that private" is one of the few things holding back everyday retail businesses from asking for them.
I don't think it would be any worse than the status quo. Some businesses might not ask for your SSN in particular, but adtech can identify you pretty uniquely regardless. And in the meantime, a million businesses have your sensitive, private SSN, and a relatively high risk of data breach.