What is the difference in the purpose of DH and RSA? Aren't they both public-key encryption?
The difference is subtle.
RSA is used to come up with a public/private key pair for asymmetric ("public-key") encryption:
It just so happens that -- in practice -- RSA's results are subsequently used to generate a symmetric key.
Yes, they're both public key systems. The difference in the way that you're asking is that Diffie-Hellman relies on the hardness of taking logarithms (actually discrete logs, but just don't worry about that for now). RSA relies on the hardness of factoring.
Interestingly, the two problems are related. There are mathematical theorems that say that a structural problem in one means there's a structural problem in the other. But they are two distinct families of public key crypto, the logarithm family and the factoring family.
Elliptic curve crypto, by the way, is just logarithm-family crypto on a different finite field than modular arithmetic. If that just sailed over your head, I can explain later.