Artificial intelligence website defines off-policy and on-policy learning as follows: 


  "An off-policy learner learns the value of the optimal policy independently of the agent's actions. Q-learning is an off-policy learner. An on-policy learner learns the value of the policy being carried out by the agent including the exploration steps."


I would like to ask your clarification regarding this, because they don't seem to make any difference to me. Both the definitions seem like they are identical. What I actually understood are the model-free and model-based learning, and I don't know if they have anything to do with the ones in question. 

How is it possible that the optimal policy is learned independently of the agent's actions? Isn't the policy learned when the agent performs the actions? 

First of all, there's no reason that an agent has to do the greedy action;  Agents can explore or they can follow options.   This is not what separates on-policy from off-policy learning.

The reason that Q-learning is off-policy is that it updates its Q-values using the Q-value of the next state $s'$ and the greedy action $a'$.  In other words, it estimates the return (total discounted future reward) for state-action pairs assuming a greedy policy were followed despite the fact that it's not following a greedy policy.

The reason that SARSA is on-policy is that it updates its Q-values using the Q-value of the next state $s'$ and the current policy's action $a''$.  It estimates the return for state-action pairs assuming the current policy continues to be followed.

The distinction disappears if the current policy is a greedy policy.  However, such an agent would not be good since it never explores.

Have you looked at the book available for free online?  Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. Second edition, 
MIT Press, Cambridge, MA, 2018.
