I was taught that something which reflects all the colors of light is white. The function of a mirror is the same, it also reflects all light. What's the difference?

Update:
But what if the white object is purely plain and does not scatter light? Will it be a mirror?

The difference is the direction the light is emitted in. Mirrors 'bounce' light in a predictable direction, white objects scatter light.