To me, the biggest difference between DH and RSA is that DH is stateful and RSA is stateless. Next up is that with DH you're generating random exponents each time instead of doing it once during key generation.

The number of primes involved is way down on my list of important differences between DH and RSA.