<p>Task/Data parallelism is a simple classification that lies at the algorithm-level of a computation.</p>

<p>Flynn's taxonomy describes low-level machine architectures or models.</p>

<p>Trying to draw lines between both completely ignores the vast sea of complexity that lies between those two levels.  Using an example; You can do task, data and pipeline parallelism perfectly on a SISD for example. Stating that you cannot ignores the vast complexity of today's operating systems.</p>

CC BY-SA 3.0
2011-08-18T13:18:54.603Z
2011-08-18T13:18:54.603Z
Beef
<p>Task parallelism and data parallelism are different approaches to handling MIMD.  Thus Flynn's taxonomy is irrelevant.</p>

<p>The difference is simple.  In data parallelism you hand out data to different CPUs that are doing the same thing with their data.  In task parallelism you hand tasks out to different machines and send data where it needs to go.</p>

<p>Of course in the real world you don't do one or the other, you do <em>both</em>.  For instance you run a <a href="http://en.wikipedia.org/wiki/MapReduce" rel="nofollow">MapReduce</a>.  In your map, sort and reduce phases there is a lot of data parallelism as many different nodes are doing the same thing with the data that they are receiving.  However there is also task parallelism because you might have a central supervisor, nodes doing mapping, nodes doing sorting, and nodes who are ready to reduce, all at the same time.</p>

CC BY-SA 3.0
2011-07-18T17:48:13.750Z
Thanks! About "Task parallelism and data parallelism are different approaches to handling MIMD. Thus Flynn's taxonomy is irrelevant.", in data parallelism http://en.wikipedia.org/wiki/Data_parallelism, there is SIMD
Tim
CC BY-SA 3.0
2011-07-18T19:04:27.657Z
@Tim True, good point.  And there is no way to do task parallelism with SIMD.  But my point remains that Flynn's taxonomy is addressing the architecture of the machine(s) you have available, while task vs data parallelism is discussing how you are handling parallelism in your software solution.  Also Flynn's taxonomy is pretty much binary, while task/data parallelism are two poles in a continuum.
btilly
CC BY-SA 3.0
2011-08-18T13:14:31.957Z
Just want to stress here that Data/Task parallelism is a very primitive and naive view on the parallel world. In practice you indeed often do combinations, but you also have pipeline and tournament parallelism. (tournament parallelism is: whoever finishes first returns final result, > linear speedup)
Beef
CC BY-SA 3.0
2011-07-18T17:00:49.387Z
2011-07-18T17:00:49.387Z
btilly
<p>Quoted from <a href="http://en.wikipedia.org/wiki/Task_parallelism" rel="nofollow">http://en.wikipedia.org/wiki/Task_parallelism</a>:</p>

<blockquote>
  <p><strong>Task parallelism</strong> (also known as function parallelism and control
  parallelism) is a form of parallelization of computer code across
  multiple processors in parallel computing environments. Task
  parallelism focuses on distributing execution processes (threads)
  across different parallel computing nodes. It contrasts to <strong>data
  parallelism</strong> as another form of parallelism.</p>
</blockquote>

<ol>
<li><p>I was wondering if there is some correspondence between  Flynn's
taxonomy (SISD, SIMD, MISD and MIMD), and "task parallelism and data parallelism"? For example,</p>

<p>does task parallelism mean multiple-instruction in Flynn's taxonomy,
so one can say "task parallelism = MISD+MIMD"?</p>

<p>does data parallelism mean multiple-data in Flynn's taxonomy, so one
can say "data parallelism = SIMD+MIMD"?</p></li>
<li>If there is no exact correspondence between the two classification
methods, how shall one understand their differences and relations?</li>
</ol>

<p>Thanks and regards!</p>

<dc.parallel-comp>
Flynn's taxonomy, and "task parallelism and data parallelism"
Tim
# Flynn's taxonomy, and "task parallelism and data parallelism"

(Asked by: username_0 on 2011-07-18)

Quoted from <http://en.wikipedia.org/wiki/Task_parallelism>:

> **Task parallelism** (also known as function parallelism and control
> parallelism) is a form of parallelization of computer code across
> multiple processors in parallel computing environments. Task
> parallelism focuses on distributing execution processes (threads)
> across different parallel computing nodes. It contrasts to **data
> parallelism** as another form of parallelism.

1. I was wondering if there is some correspondence between Flynn's
   taxonomy (SISD, SIMD, MISD and MIMD), and "task parallelism and data parallelism"? For example,

   does task parallelism mean multiple-instruction in Flynn's taxonomy,
   so one can say "task parallelism = MISD+MIMD"?

   does data parallelism mean multiple-data in Flynn's taxonomy, so one
   can say "data parallelism = SIMD+MIMD"?
2. If there is no exact correspondence between the two classification
   methods, how shall one understand their differences and relations?

Thanks and regards!

**Tags:** dc.parallel-comp


---

## 2 answers

### Answer by username_1 on 2011-08-18. Score: 2

Task/Data parallelism is a simple classification that lies at the algorithm-level of a computation.

Flynn's taxonomy describes low-level machine architectures or models.

Trying to draw lines between both completely ignores the vast sea of complexity that lies between those two levels. Using an example; You can do task, data and pipeline parallelism perfectly on a SISD for example. Stating that you cannot ignores the vast complexity of today's operating systems.


---

### Answer by username_2 on 2011-07-18. Score: 1

Task parallelism and data parallelism are different approaches to handling MIMD. Thus Flynn's taxonomy is irrelevant.

The difference is simple. In data parallelism you hand out data to different CPUs that are doing the same thing with their data. In task parallelism you hand tasks out to different machines and send data where it needs to go.

Of course in the real world you don't do one or the other, you do *both*. For instance you run a [MapReduce](http://en.wikipedia.org/wiki/MapReduce). In your map, sort and reduce phases there is a lot of data parallelism as many different nodes are doing the same thing with the data that they are receiving. However there is also task parallelism because you might have a central supervisor, nodes doing mapping, nodes doing sorting, and nodes who are ready to reduce, all at the same time.

> **Comments:**
> **username_0**: Thanks! About "Task parallelism and data parallelism are different approaches to handling MIMD. Thus Flynn's taxonomy is irrelevant.", in data parallelism http://en.wikipedia.org/wiki/Data_parallelism, there is SIMD  
> **username_2**: @username_0 True, good point.  And there is no way to do task parallelism with SIMD.  But my point remains that Flynn's taxonomy is addressing the architecture of the machine(s) you have available, while task vs data parallelism is discussing how you are handling parallelism in your software solution.  Also Flynn's taxonomy is pretty much binary, while task/data parallelism are two poles in a continuum.  
> **username_1**: Just want to stress here that Data/Task parallelism is a very primitive and naive view on the parallel world. In practice you indeed often do combinations, but you also have pipeline and tournament parallelism. (tournament parallelism is: whoever finishes first returns final result, > linear speedup)  

---

