<p>The shaders themselves have the structure memory layout in them and indexing into the array is essentially <code>(sizeof(material)+alignment) * offset</code> So what the listed pseudocode is already doing is pretty close to what is needed to index directly. The <code>sizeof(material)</code> would also need to be carefully computed since data alignment inside a struct follows the buffer type layout rules. For example the two vec3's actually align to vec4 boundaries. So internally at least 8 bytes are lost for alignment. This complicates writing the data since it is up to the program to place the data at the correct offset for both the structure alignment and the internal alignment, which is basically identical to what the shader is going to do to access it. For a vec3 25% of the memory is being lost to alignment.</p>
<p>As an example the listed struct layout would look something like the following in C++: (note that I am doing this from memory so please don't write these numbers in stone)</p>
<pre><code>struct phong {
   vec3 diffuse; float alignment;
   vec3 specular; float alignment2;
   int intensity;
};
</code></pre>
<p>Where the actual size of the struct would add another 12 bytes to the end of the <code>int</code> so that <code>sizeof(phong) == 48</code> instead of the expected <code>28</code>.</p>
<p>The unusable memory issue can be overcome with careful design of materials, like folding the specular intensity into the specular color to align it on a vec4 boundary, etc.</p>
<p>So this is all doable but automating it so that the programmer doesn't have to hand tune every shader is more challenging.</p>

CC BY-SA 4.0
2023-09-08T13:11:02.913Z
But this solves only the C++ part. I have that down however, the access in GLSL is the part I'm struggling with, seeing as the buffer contains various materials with arbitrary members, all of them tightly packed.
Tare
CC BY-SA 4.0
2023-09-08T13:21:40.337Z
The entire point here is that you can't just pack the memory any old way you feel like it.  Doing it the way I described packs the memory about as tightly as it can be packed. The data has to be put in the buffers where it meets the requirements of GLSL. Then in the shader access is simply "params_buffer[passed_in_index]".
pmw1234
CC BY-SA 4.0
2023-09-08T13:23:47.747Z
I can live with accessing it as a vec4 rather than a vec3 and a float, but this is not the problem I'm trying to solve. The problem might be that my data starts at position 2 in the buffer. I will add an image to hopefully clarify my issue.
Tare
CC BY-SA 4.0
2023-09-09T12:51:44.893Z
You should consider switching to an array of vec4's for the SBO or using dynamic offsets.  Dynamic offsets set the start of the array where you want it (but there are still alignment rules). GPU's rely heavily on streaming data, to do that they depend on alignment, you can't just force your own rules on the system and expect performant shaders. The problem you are trying to solve and the solution I described are one and the same, with alignment rules applied. Good luck.
pmw1234
CC BY-SA 4.0
2023-09-08T12:12:59.940Z
2023-09-08T12:12:59.940Z
pmw1234
<p><em><strong>TL;DR</strong></em></p>
<p>In a Vulkan / GLSL ray tracing program, I have several shader files, all accessing the same material buffer of arbitrary data. All materials have an arbitrary layout, e.g. one might be a float, one might contain two vec3s and a float.
Can I access a buffer (i.e. an array of uint type) with just an offset <em>within the shader (i.e. the same buffer binding is shared, I cannot use dynamic offsets set on the CPU side)</em> and read out a struct without having to read every member of the struct individually? I am aware of not being able to read a vec3 from a buffer as just 3 values, but that is not my problem. Just assume I only read good data types that don't waste space.</p>
<p>See in the image, I want to access at the given green position with just knowing the size of the struct that will follow at that position.</p>
<p><a href="https://i.sstatic.net/27ZwK.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/27ZwK.png" alt="Image of buffer data and where to access it" /></a></p>
<p>In C++ it would be something like this:</p>
<pre><code>uint32_t* buffer; //assume this has all the data in it
struct mat2
{
    glm::vec3 data1;
    float data2;
}

//the following line is what I try to achieve
mat2 matData = *((mat2)(buffer+offset);
</code></pre>
<p><em><strong>Long explanation</strong></em></p>
<p>I am trying to setup a buffer that contains material parameters.
The problem is that I have to support a large number of materials, that may differ drastically in number of parameters, e.g. there might be an entry with two uints and then an entry with five floats.</p>
<p>Since I assume using one buffer per material is bad (due to the number of layouts bindings I'd have to use) I wanted to tightly pack the material parameters into a single byte buffer and readout from the shader with a known offset to the material for the hit geometry.
I have a shader file per material, so no material is aware of other material structs - it just knows the offset into the array for its own data.</p>
<p>While I have basically set up the buffer (though apperantly the smallest chunks I am allowed to use are 4 bytes), I have trouble actually reading out of this buffer in GLSL. Given a simple struct for one material:</p>
<pre><code>struct BlinnPhongParams
{
    vec3 diffuse;
    vec3 specular;
    float specIntensity;
};
</code></pre>
<p>And the material buffer:</p>
<pre><code>layout(binding = 123, set = 0) readonly buffer MaterialParamsBuffer
{
    uint[] data;
}uMaterialParamsBuffer;
</code></pre>
<p>Currently I am reading every member from the buffer individually:</p>
<pre><code>BlinnPhongParams getMaterialParams(uint offset)
{
    BlinnPhongParams bpParams = BlinnPhongParams(
        //DiffuseCol
        vec3(
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset]),
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+1]),
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+2])),
        //SpecularCol
        vec3(
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+3]),
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+4]),
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+5])),
        //SpecIntensity
        uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+6])
    );
    return bpParams;
}
</code></pre>
<p>I would like to make reading from this buffer as easy as possible, ideally just say &quot;at this offset, I have a struct that looks like this&quot;, e.g. I would like to have something like this:</p>
<pre><code>BlinnPhongParams getMaterialParams(uint offset)
{
    // here is the problem
    BlinnPhongParams params = BlinnPhongParams(&amp;uMaterialParamsBuffer.data[offset]);
}
</code></pre>
<p>Is this possible?</p>

CC BY-SA 4.0
2023-09-07T14:11:46.560Z
What API is this code using to access the data? OpenGL, Vulkan, etc...
pmw1234
CC BY-SA 4.0
2023-09-08T06:30:15.567Z
I'm using Vulkan / GLSL
Tare
<glsl><vulkan><buffers>
Is it possible to read arbitrary data at arbitrary array offset in glsl?
Tare
# Is it possible to read arbitrary data at arbitrary array offset in glsl?

(Asked by: username_0 on 2023-09-07)

***TL;DR***

In a Vulkan / GLSL ray tracing program, I have several shader files, all accessing the same material buffer of arbitrary data. All materials have an arbitrary layout, e.g. one might be a float, one might contain two vec3s and a float.
Can I access a buffer (i.e. an array of uint type) with just an offset *within the shader (i.e. the same buffer binding is shared, I cannot use dynamic offsets set on the CPU side)* and read out a struct without having to read every member of the struct individually? I am aware of not being able to read a vec3 from a buffer as just 3 values, but that is not my problem. Just assume I only read good data types that don't waste space.

See in the image, I want to access at the given green position with just knowing the size of the struct that will follow at that position.

[![Image of buffer data and where to access it](https://i.sstatic.net/27ZwK.png)](https://i.sstatic.net/27ZwK.png)

In C++ it would be something like this:

```
uint32_t* buffer; //assume this has all the data in it
struct mat2
{
    glm::vec3 data1;
    float data2;
}

//the following line is what I try to achieve
mat2 matData = *((mat2)(buffer+offset);

```

***Long explanation***

I am trying to setup a buffer that contains material parameters.
The problem is that I have to support a large number of materials, that may differ drastically in number of parameters, e.g. there might be an entry with two uints and then an entry with five floats.

Since I assume using one buffer per material is bad (due to the number of layouts bindings I'd have to use) I wanted to tightly pack the material parameters into a single byte buffer and readout from the shader with a known offset to the material for the hit geometry.
I have a shader file per material, so no material is aware of other material structs - it just knows the offset into the array for its own data.

While I have basically set up the buffer (though apperantly the smallest chunks I am allowed to use are 4 bytes), I have trouble actually reading out of this buffer in GLSL. Given a simple struct for one material:

```
struct BlinnPhongParams
{
    vec3 diffuse;
    vec3 specular;
    float specIntensity;
};

```

And the material buffer:

```
layout(binding = 123, set = 0) readonly buffer MaterialParamsBuffer
{
    uint[] data;
}uMaterialParamsBuffer;

```

Currently I am reading every member from the buffer individually:

```
BlinnPhongParams getMaterialParams(uint offset)
{
    BlinnPhongParams bpParams = BlinnPhongParams(
        //DiffuseCol
        vec3(
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset]),
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+1]),
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+2])),
        //SpecularCol
        vec3(
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+3]),
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+4]),
            uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+5])),
        //SpecIntensity
        uintBitsToFloat(uMaterialParamsUintBuffer.data[offset+6])
    );
    return bpParams;
}

```

I would like to make reading from this buffer as easy as possible, ideally just say "at this offset, I have a struct that looks like this", e.g. I would like to have something like this:

```
BlinnPhongParams getMaterialParams(uint offset)
{
    // here is the problem
    BlinnPhongParams params = BlinnPhongParams(&uMaterialParamsBuffer.data[offset]);
}

```

Is this possible?

**Tags:** glsl, vulkan, buffers

> **Comments:**
> **username_1**: What API is this code using to access the data? OpenGL, Vulkan, etc...  
> **username_0**: I'm using Vulkan / GLSL  

---

## 1 answers

### Answer by username_1 on 2023-09-08. Score: 0

The shaders themselves have the structure memory layout in them and indexing into the array is essentially `(sizeof(material)+alignment) * offset` So what the listed pseudocode is already doing is pretty close to what is needed to index directly. The `sizeof(material)` would also need to be carefully computed since data alignment inside a struct follows the buffer type layout rules. For example the two vec3's actually align to vec4 boundaries. So internally at least 8 bytes are lost for alignment. This complicates writing the data since it is up to the program to place the data at the correct offset for both the structure alignment and the internal alignment, which is basically identical to what the shader is going to do to access it. For a vec3 25% of the memory is being lost to alignment.

As an example the listed struct layout would look something like the following in C++: (note that I am doing this from memory so please don't write these numbers in stone)

```
struct phong {
   vec3 diffuse; float alignment;
   vec3 specular; float alignment2;
   int intensity;
};

```

Where the actual size of the struct would add another 12 bytes to the end of the `int` so that `sizeof(phong) == 48` instead of the expected `28`.

The unusable memory issue can be overcome with careful design of materials, like folding the specular intensity into the specular color to align it on a vec4 boundary, etc.

So this is all doable but automating it so that the programmer doesn't have to hand tune every shader is more challenging.

> **Comments:**
> **username_0**: But this solves only the C++ part. I have that down however, the access in GLSL is the part I'm struggling with, seeing as the buffer contains various materials with arbitrary members, all of them tightly packed.  
> **username_1**: The entire point here is that you can't just pack the memory any old way you feel like it.  Doing it the way I described packs the memory about as tightly as it can be packed. The data has to be put in the buffers where it meets the requirements of GLSL. Then in the shader access is simply "params_buffer[passed_in_index]".  
> **username_0**: I can live with accessing it as a vec4 rather than a vec3 and a float, but this is not the problem I'm trying to solve. The problem might be that my data starts at position 2 in the buffer. I will add an image to hopefully clarify my issue.  
> **username_1**: You should consider switching to an array of vec4's for the SBO or using dynamic offsets.  Dynamic offsets set the start of the array where you want it (but there are still alignment rules). GPU's rely heavily on streaming data, to do that they depend on alignment, you can't just force your own rules on the system and expect performant shaders. The problem you are trying to solve and the solution I described are one and the same, with alignment rules applied. Good luck.  

---

