To main page
is an concept iun programming when many processes work simultaneously to achieve the same task. The opposite (and most common one) is linear programming.
GPU is the excellent example of parallel programming concept. At any given time there can be tens of thousands of parallel processes working to render your primitives
in the scene. Contrary to that, "usual" linear programming uses one process.
Excellent demonstration by MythBusters