<blockquote>
  <p>Are there algorithms that cannot be implemented this way and have to be implemented in OpenCL, and if so, how do I know if my algorithm is one of them?</p>
</blockquote>

<p>Yes. Generally <a href="https://en.wikipedia.org/wiki/Gather-scatter_(vector_addressing)" rel="nofollow noreferrer">scatter-gather algorithms</a> work poorly in OpenGL as the data locality is poor. When you have to grab bits from all over the input image for each output pixel, it's going to be slow or nearly impossible to implement in a typical GL shader. Likewise, when a calculation on an input pixel results in data that needs to be spread out to several output pixels, it's a poor fit.</p>

<p>It's hard to tell just from the abstract, but it looks to me like this wouldn't be a great fit for a typical OpenGL shader. I think you'll end up needing to sample an area of the input, classify it, then sample from one or more of a large set of small exemplars.</p>

<p>I mainly work on iOS and macOS which both come with OpenGL and OpenCL. Not sure how Windows and Linux handle that.</p>

CC BY-SA 3.0
2017-02-06T07:42:54.883Z
Thanks. So do you mean that scatter-gather algorithms work better in OpenCL than in OpenGL?
HelloGoodbye
CC BY-SA 3.0
2017-02-06T07:58:19.903Z
Which part of the algorithm don't you think would be well suited for an OpenGL shader? The algorithm uses interpolation by combining the pixels in a small neighborhood linearly with coefficients that it gets from a lookup table. The table key is obtained by looking at the strength, angle and spread respectively of the gradients in a local neighborhood and combining the quantizations of those values. It also calculates the local binary patterns feature.
HelloGoodbye
CC BY-SA 3.0
2017-02-06T17:18:56.600Z
The part you described sounds like it involves sampling in an area around the pixel currently being operated on to calculate a value to then use in a lookup table. So you your "gathering" by sampling in an area around the current pixel. Then your doing a dependent texture lookup. Both of those can be expensive depending on the hardware you're running on. If the area sampled in both the input texture and the lookup table is small enough, it might not be a bottleneck, but I suspect that's where it would happen. It seems OpenCL might be a better fit for that reason (to me).
user1118321
CC BY-SA 3.0
2017-02-03T03:29:57.760Z
2017-02-03T03:29:57.760Z
user1118321
<p>I recently ran across the paper <a href="https://arxiv.org/abs/1606.01299" rel="nofollow noreferrer">RAISR: Rapid and Accurate Image Super Resolution</a> and thought it seemed like an interesting technique for scaling up images. However, I haven't seen any implementation of it, and I would like to see what it would be like to use it for scaling up a video in realtime, if that is possible. So I'm thinking of implementing my own version of it, but I will need to use GPU acceleration in some way if I want to at least give it a chance to run in realtime. Ultimately, I would like to see if I could successfully implemented this in an open source media player.</p>

<p><strong>So</strong>, what library should I use to implement this algorithm? DirectX and CUDA are out of the question, since I want to make it run under any (major) OS and on any graphics card (well, on most of the popular ones, at least), so I'm thinking that my only options are</p>

<ol>
<li><p>OpenGL (ES) + GLSL (ES), and</p></li>
<li><p>OpenCL.</p></li>
</ol>

<p>(Vulkan also exists but I don't know whether this API is mature enough yet.)</p>

<p>If I choose OpenGL + GLSL, is this approach flexible enough to allow this algorithm to be implemented? Are there algorithms that cannot be implemented this way and have to be implemented in OpenCL, and if so, how do I know if my algorithm is one of them?</p>

<p>If I on the other hand choose OpenCL, will it be possible for other people to use my implementation without installing OpenCL on their computer? </p>

CC BY-SA 3.0
2017-11-07T14:28:38.273Z
"Vulkan also exists but I don't know whether this API is mature enough yet." It's mature but Apple doesn't support it, so you won't meet your goal of running on any OS.
Dan Hulme
CC BY-SA 4.0
2018-12-12T20:35:28.617Z
I implemented a CUDA version of RAISR and am working on an OpenGL implemention. The hashing step isn't much of a problem -- just cache things in shared memory. You'll get garbage performance on older hardware, but on newer low-end cards, it's totally fine. The tricky part is the filter lookup, and there's quite a few methods available to deal with that problem (shared memory, some compression tricks, etc). But you'll probably get decent performance if you just store the filters in an RGBA texture.
AnimatedRNG
CC BY-SA 3.0
2017-02-03T01:54:14.327Z
"*If I on the other hand choose OpenCL, will it be possible for other people to use my implementation without installing OpenCL on their computer?*" That's just as true of OpenGL. The only difference being that GL typically comes with graphics drivers, while CL typically requires an explicit download.
Nicol Bolas
CC BY-SA 3.0
2017-02-03T18:27:44.983Z
Why was my question downvoted?
HelloGoodbye
CC BY-SA 3.0
2017-02-03T18:34:35.123Z
@NicolBolas That may be true. So what frameworks do media players and other video software usually use to accelerate video rendering?
HelloGoodbye
<opengl><glsl><real-time><gpgpu><opencl>
What framework shoud I use to implement the RAISR upscaling algorithm on the GPU?
HelloGoodbye
# What framework shoud I use to implement the RAISR upscaling algorithm on the GPU?

(Asked by: username_0 on 2017-02-03)

I recently ran across the paper [RAISR: Rapid and Accurate Image Super Resolution](https://arxiv.org/abs/1606.01299) and thought it seemed like an interesting technique for scaling up images. However, I haven't seen any implementation of it, and I would like to see what it would be like to use it for scaling up a video in realtime, if that is possible. So I'm thinking of implementing my own version of it, but I will need to use GPU acceleration in some way if I want to at least give it a chance to run in realtime. Ultimately, I would like to see if I could successfully implemented this in an open source media player.

**So**, what library should I use to implement this algorithm? DirectX and CUDA are out of the question, since I want to make it run under any (major) OS and on any graphics card (well, on most of the popular ones, at least), so I'm thinking that my only options are

1. OpenGL (ES) + GLSL (ES), and
2. OpenCL.

(Vulkan also exists but I don't know whether this API is mature enough yet.)

If I choose OpenGL + GLSL, is this approach flexible enough to allow this algorithm to be implemented? Are there algorithms that cannot be implemented this way and have to be implemented in OpenCL, and if so, how do I know if my algorithm is one of them?

If I on the other hand choose OpenCL, will it be possible for other people to use my implementation without installing OpenCL on their computer?

**Tags:** opengl, glsl, real-time, gpgpu, opencl

> **Comments:**
> **username_3**: "*If I on the other hand choose OpenCL, will it be possible for other people to use my implementation without installing OpenCL on their computer?*" That's just as true of OpenGL. The only difference being that GL typically comes with graphics drivers, while CL typically requires an explicit download.  
> **username_0**: Why was my question downvoted?  
> **username_0**: @username_3 That may be true. So what frameworks do media players and other video software usually use to accelerate video rendering?  
> **username_1**: "Vulkan also exists but I don't know whether this API is mature enough yet." It's mature but Apple doesn't support it, so you won't meet your goal of running on any OS.  
> **username_2**: I implemented a CUDA version of RAISR and am working on an OpenGL implemention. The hashing step isn't much of a problem -- just cache things in shared memory. You'll get garbage performance on older hardware, but on newer low-end cards, it's totally fine. The tricky part is the filter lookup, and there's quite a few methods available to deal with that problem (shared memory, some compression tricks, etc). But you'll probably get decent performance if you just store the filters in an RGBA texture.  

---

## 1 answers

### Answer by username_4 on 2017-02-03. Score: 2

> Are there algorithms that cannot be implemented this way and have to be implemented in OpenCL, and if so, how do I know if my algorithm is one of them?

Yes. Generally [scatter-gather algorithms](https://en.wikipedia.org/wiki/Gather-scatter_(vector_addressing)) work poorly in OpenGL as the data locality is poor. When you have to grab bits from all over the input image for each output pixel, it's going to be slow or nearly impossible to implement in a typical GL shader. Likewise, when a calculation on an input pixel results in data that needs to be spread out to several output pixels, it's a poor fit.

It's hard to tell just from the abstract, but it looks to me like this wouldn't be a great fit for a typical OpenGL shader. I think you'll end up needing to sample an area of the input, classify it, then sample from one or more of a large set of small exemplars.

I mainly work on iOS and macOS which both come with OpenGL and OpenCL. Not sure how Windows and Linux handle that.

> **Comments:**
> **username_0**: Thanks. So do you mean that scatter-gather algorithms work better in OpenCL than in OpenGL?  
> **username_0**: Which part of the algorithm don't you think would be well suited for an OpenGL shader? The algorithm uses interpolation by combining the pixels in a small neighborhood linearly with coefficients that it gets from a lookup table. The table key is obtained by looking at the strength, angle and spread respectively of the gradients in a local neighborhood and combining the quantizations of those values. It also calculates the local binary patterns feature.  
> **username_4**: The part you described sounds like it involves sampling in an area around the pixel currently being operated on to calculate a value to then use in a lookup table. So you your "gathering" by sampling in an area around the current pixel. Then your doing a dependent texture lookup. Both of those can be expensive depending on the hardware you're running on. If the area sampled in both the input texture and the lookup table is small enough, it might not be a bottleneck, but I suspect that's where it would happen. It seems OpenCL might be a better fit for that reason (to me).  

---

