<h3>Intro</h3>
<p>I'm trying to render multi-channel images (more than 4 channels, for the purposes of feeding it to a Neural Network). Since OpenGL doesn't support it natively, I have multiple 4-channel render buffers, into which I render a corresponding portion of channels.</p>
<p>For example, I need multi-channel image of size <code>512 x 512 x 16</code>, in OpenGL I have 4 render buffers of size <code>512 x 512 x 4</code>. Now the problem is that the Neural Network expects the data with strides <code>512 x 512 x 16</code>, i.e. 16 values of channels of one pixel are followed by 16 values of channels from the next pixel. However currently I can efficiently read my 4 render buffers via 4 calls to <code>glReadPixels</code>, basically making the data having strides <code>4 x 512 x 512 x 4</code>. Manual reordering of data on the client side <strong>will not suffice me</strong> as it's too slow.</p>
<h3>Main question</h3>
<p>I've got an idea to render to a single 4-channel render buffer of size <code>512 x 512*4 x 4</code>, because stride-wise it's equivalent to <code>512 x 512 x 16</code>, we just treat a combination of 4 pixels in a row as a single pixel of 16-channel output image. Let's call it an &quot;interleaved rendering&quot;</p>
<p>But this requires me to magically adjust my fragment shader, so that every group of consequent 4 <em>fragments</em> would have exactly the same interpolation of vertex attributes. Is there any way to do that?</p>
<p>This bad illustration with 1 render buffer of <code>512 x 1024</code> 4-channel image, is an example of how it should be rendered. With that I can in 1 call <code>glReadPixels</code> extract the data with stride <code>512 x 512 x 8</code> <a href="https://i.sstatic.net/Py6fX.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Py6fX.png" alt="enter image description here" /></a></p>
<p>EDIT: better pictures
What I have now (4 render buffers)
<a href="https://i.sstatic.net/WQeZY.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/WQeZY.png" alt="enter image description here" /></a></p>
<p>What I want to do natively in OpenGL (this image is done in Python offline)
<a href="https://i.sstatic.net/n0UaB.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/n0UaB.png" alt="enter image description here" /></a></p>

CC BY-SA 4.0
2021-10-14T12:57:39.217Z
Have you looked into 64 bit images? 4 channels of 64 bits would give you the number of channels needed, but it leaves the packing and unpacking of the data up to the shaders/code. But writing a function to pack/unpack the channels shouldn't be to difficult.
pmw1234
CC BY-SA 4.0
2021-10-14T13:27:09.313Z
@pmw1234 do you mean making a texture with internal format `GL_RGBA32UI` and pack 4 one-byte values per each component, having a total of 16 values per pixel? I like this idea. But I worry if endianness of OpenGL is universal across versions and hardware
Alexey S. Larionov
CC BY-SA 4.0
2021-10-14T13:46:38.060Z
yes, I wouldn't expect byte ordering to be an issue since the code is essentially taking over control of the packing. Also the market is dominated by little endian machines but even if it comes up specializing for machine endianness is just a matter of having the appropriate functions which would be simple to write, especially once there is a functional algorithm.
pmw1234
<opengl><rendering><efficiency>
"Interleaved rendering" in fragment shader
Alexey S. Larionov
# "Interleaved rendering" in fragment shader

(Asked by: username_0 on 2021-10-13)

### Intro

I'm trying to render multi-channel images (more than 4 channels, for the purposes of feeding it to a Neural Network). Since OpenGL doesn't support it natively, I have multiple 4-channel render buffers, into which I render a corresponding portion of channels.

For example, I need multi-channel image of size `512 x 512 x 16`, in OpenGL I have 4 render buffers of size `512 x 512 x 4`. Now the problem is that the Neural Network expects the data with strides `512 x 512 x 16`, i.e. 16 values of channels of one pixel are followed by 16 values of channels from the next pixel. However currently I can efficiently read my 4 render buffers via 4 calls to `glReadPixels`, basically making the data having strides `4 x 512 x 512 x 4`. Manual reordering of data on the client side **will not suffice me** as it's too slow.

### Main question

I've got an idea to render to a single 4-channel render buffer of size `512 x 512*4 x 4`, because stride-wise it's equivalent to `512 x 512 x 16`, we just treat a combination of 4 pixels in a row as a single pixel of 16-channel output image. Let's call it an "interleaved rendering"

But this requires me to magically adjust my fragment shader, so that every group of consequent 4 *fragments* would have exactly the same interpolation of vertex attributes. Is there any way to do that?

This bad illustration with 1 render buffer of `512 x 1024` 4-channel image, is an example of how it should be rendered. With that I can in 1 call `glReadPixels` extract the data with stride `512 x 512 x 8` [![enter image description here](https://i.sstatic.net/Py6fX.png)](https://i.sstatic.net/Py6fX.png)

EDIT: better pictures
What I have now (4 render buffers)
[![enter image description here](https://i.sstatic.net/WQeZY.png)](https://i.sstatic.net/WQeZY.png)

What I want to do natively in OpenGL (this image is done in Python offline)
[![enter image description here](https://i.sstatic.net/n0UaB.png)](https://i.sstatic.net/n0UaB.png)

**Tags:** opengl, rendering, efficiency

> **Comments:**
> **username_1**: Have you looked into 64 bit images? 4 channels of 64 bits would give you the number of channels needed, but it leaves the packing and unpacking of the data up to the shaders/code. But writing a function to pack/unpack the channels shouldn't be to difficult.  
> **username_0**: @username_1 do you mean making a texture with internal format `GL_RGBA32UI` and pack 4 one-byte values per each component, having a total of 16 values per pixel? I like this idea. But I worry if endianness of OpenGL is universal across versions and hardware  
> **username_1**: yes, I wouldn't expect byte ordering to be an issue since the code is essentially taking over control of the packing. Also the market is dominated by little endian machines but even if it comes up specializing for machine endianness is just a matter of having the appropriate functions which would be simple to write, especially once there is a functional algorithm.  

---

## 0 answers

