Doctors, lawyers, and scientists hate talking about work, but they love talking about medicine, law, and science. Talking about a subject they love is different from talking about why their boss is mad at them and why their five o'clock meeting ran over.
But then again, maybe that's because there are a lot of nerds in medicine, law, and science.
There's a huge difference between talking about a subject in general, and talking about a subject that you work with all day long. The former is preferable to the latter. That's obvious.
But even so, you're still falling into the obsessive nerd trap: most people -- especially the really smart people -- love it when you talk with them about something other than what they do for a living, even if they're otherwise content to talk to you about the general aspects of their profession. It's the difference between being a forgettable conversationalist and a memorable one.
But then again, maybe that's because there are a lot of nerds in medicine, law, and science.