<p>I agree with @pmw1234, there is no theory. The intrsuction sets on CPU and GPU are both Turing complete, therefore CPU/GPU can do, in theory, whatever the other can. I found a <a href="https://www.quora.com/Since-GPUs-are-Turing-complete-does-that-mean-you-can-program-an-OS-similar-to-Windows-that-can-run-on-just-a-GPU-without-a-CPU" rel="nofollow noreferrer">dicussion for this</a> on Quora, maybe you will be interested.</p>
<p>Note that, for one single thread, GPU processes slower than CPU (on average, device dependent) and it is much less flexible. GPU achieve its acceleration via a large number of concurrent threads (organized in warps -&gt; blocks -&gt; grids) processing the same or similar tasks, which resembles to <a href="https://en.wikipedia.org/wiki/Single_instruction,_multiple_data" rel="nofollow noreferrer">SIMD</a> in the CPU context. There are a few things that could make GPU much faster than CPU for some of the tasks:</p>
<ul>
<li>Large scale GPU-thread-based parallelism (therefore, SIMD-like processing)</li>
<li>Latency hiding, I believe this would be a good reference: <a href="https://news.ycombinator.com/item?id=26444397" rel="nofollow noreferrer">latency hiding</a></li>
<li>Memory management (shared / texture memory) / asynchronous load &amp; store</li>
<li>Stream-based multiprocessing</li>
</ul>
<p>Actually to the best of my knowledge, GPU threads can be very limitted for some tasks. For example, it does not perform as efficient as CPU threads when the program focuses more on branching logics than data-driven computation and large-scale similar simple tasks. There is a concept called &quot;warp divergence&quot; describing that when different threads in a GPU warp diverge during if-else branching, these threads could be serialized. Therefore, if your code (task) relies heavily on branching, you should be very careful when coding on GPU. A simple example is &quot;select-like&quot; function.</p>
<p>CPU however, is well-designed for handling context changing, the switch between multiple processes, branch prediction and out-of-order execution (therefore well pipelined) therefore each core can process very sophisticated tasks independently. As a simple analogy, a CPU is like a handful of extremely efficient workers who can work independently while a GPU is like a large number of normal workers who work well in groups. Both of them are similarly capable while they focus differently.</p>

CC BY-SA 4.0
2023-08-03T15:22:11.700Z
Thanks a lot, very useful insight... I always though that a different instruction set meant different capabilities although I still dont get who is responsible for the SW being able to run in a CPU or GPU at the same time
Raul Lapeira Herrero
CC BY-SA 4.0
2023-08-03T05:18:31.490Z
2023-08-03T05:18:31.490Z
Enigmatisms
<p>GPUs have a smaller instruction set than a CPU does, but that means that from an architecture POV all the instructions in a GPU have a combination of slower CPU equivalent instructions?</p>

CC BY-SA 4.0
2023-07-31T11:32:56.590Z
There is no theory, any overlap is out of necessity.
pmw1234
<gpu>
What is the theory behind a CPU being able to do everything a GPU can? Can I have something GPU based that can not run on a CPU?
Raul Lapeira Herrero
# What is the theory behind a CPU being able to do everything a GPU can? Can I have something GPU based that can not run on a CPU?

(Asked by: username_0 on 2023-07-31)

GPUs have a smaller instruction set than a CPU does, but that means that from an architecture POV all the instructions in a GPU have a combination of slower CPU equivalent instructions?

**Tags:** gpu

> **Comments:**
> **username_1**: There is no theory, any overlap is out of necessity.  

---

## 1 answers

### Answer by username_2 on 2023-08-03. Score: 1 | Accepted answer

I agree with @username_1, there is no theory. The intrsuction sets on CPU and GPU are both Turing complete, therefore CPU/GPU can do, in theory, whatever the other can. I found a [dicussion for this](https://www.quora.com/Since-GPUs-are-Turing-complete-does-that-mean-you-can-program-an-OS-similar-to-Windows-that-can-run-on-just-a-GPU-without-a-CPU) on Quora, maybe you will be interested.

Note that, for one single thread, GPU processes slower than CPU (on average, device dependent) and it is much less flexible. GPU achieve its acceleration via a large number of concurrent threads (organized in warps -> blocks -> grids) processing the same or similar tasks, which resembles to [SIMD](https://en.wikipedia.org/wiki/Single_instruction,_multiple_data) in the CPU context. There are a few things that could make GPU much faster than CPU for some of the tasks:

* Large scale GPU-thread-based parallelism (therefore, SIMD-like processing)
* Latency hiding, I believe this would be a good reference: [latency hiding](https://news.ycombinator.com/item?id=26444397)
* Memory management (shared / texture memory) / asynchronous load & store
* Stream-based multiprocessing

Actually to the best of my knowledge, GPU threads can be very limitted for some tasks. For example, it does not perform as efficient as CPU threads when the program focuses more on branching logics than data-driven computation and large-scale similar simple tasks. There is a concept called "warp divergence" describing that when different threads in a GPU warp diverge during if-else branching, these threads could be serialized. Therefore, if your code (task) relies heavily on branching, you should be very careful when coding on GPU. A simple example is "select-like" function.

CPU however, is well-designed for handling context changing, the switch between multiple processes, branch prediction and out-of-order execution (therefore well pipelined) therefore each core can process very sophisticated tasks independently. As a simple analogy, a CPU is like a handful of extremely efficient workers who can work independently while a GPU is like a large number of normal workers who work well in groups. Both of them are similarly capable while they focus differently.

> **Comments:**
> **username_0**: Thanks a lot, very useful insight... I always though that a different instruction set meant different capabilities although I still dont get who is responsible for the SW being able to run in a CPU or GPU at the same time  

---

