In computer security, we know that weak points in software are called vulnerabilities (if related to security). And once the vulnerability is found, theoretically it requires a piece of code as proof of concept (this is called an exploit). In this context, the term payload is also mentioned.

Then, what is the difference between 'payload' and 'exploit'?

The exploit is what delivers the payload. 
Take a missile as an analogy. You have the rocket and fuel and everything else in the rocket, and then you have the warhead that does the actual damage. Without the warhead, the missile doesn't do very much when it hits. Additionally, a warhead isn't much use if it goes off in your bunker without a rocket delivering it.

The delivery system(missile) is the exploit and the payload (warhead) is the code that actually does something.

Exploits give you the ability to 'pop a shell/run your payload code'.

Example payloads are things like Trojans/RATs, keyloggers, reverse shells etc.

Payloads are only referred to when code execution is possible and not when using things like denial of service exploits.

 
