<blockquote>
  <p>The question is: is there a way to pass this type of texture to glTexImage2D as-is?</p>
</blockquote>

<p>Yes; you can just pass it as-is, but (looking at <a href="https://www.opengl.org/sdk/docs/man/html/glTexImage2D.xhtml" rel="nofollow">the docs</a>) it is not supported directly.  You'll need a custom shader to <em>interpret</em> it.</p>

<p>Depending on your needs, expanding the pixels first (or even doing the full conversion CPU-side) may be worthwhile.  At a guess, I'd say that expanding the pixels (as you copy from the camera's driver buffer into a staging buffer for upload to the GPU) is a pretty consistently good idea, unless you're extremely bandwidth-saturated.  I'd add the color-space conversion for easier debugging, or if the image doesn't update frequently.</p>

CC BY-SA 3.0
2016-07-27T14:20:20.693Z
Yes, I've decided it's best to unpack the data on the CPU (a task that lends itself well to multi-threading), then do color conversion in a shader.
001
CC BY-SA 3.0
2016-07-22T21:46:50.497Z
2016-07-22T21:46:50.497Z
geometrian
<p>I am grabbing video from a web cam using Microsoft Media Foundation. The image data is in YUY2 format. The pixels are packed in a 4:2:2 ratio. The color format is YCbCr. The color data gets repeated 2x. Ex the packed pixel data is:</p>

<pre><code>y0 u y1 v
</code></pre>

<p>Gets unpacked to 2 pixels:</p>

<pre><code>y0 u v and y1 u v
</code></pre>

<p>I want to use this data as an OpenGL texture. The question is: is there a way to pass this type of texture to <code>glTexImage2D</code> as-is? Or do I need to unpack it first? I would like you keep the YCbCr color and convert to RGBA in a shader.</p>

<p>(I know I can use OpenCV to get video - I've already done that. I am trying a more low level solution mostly as a learning experience).</p>

CC BY-SA 3.0
2016-07-21T04:36:26.513Z
Normally you would use YUV semi-planar for this, which is a seperate single channel Y texture and a packed UV channel.  You could also unpack inside a fragment shader by testing if the Y texel you are reading is on a odd or even column. So say you read Y0,U for the first pixel (Assuming a 2 channel texture), you test if Y0 is on a even-column, which means you need to read the texel to right to get the missing V. For odd-columns you would read the texel to the left to get the missing U.
PaulHK
CC BY-SA 3.0
2016-07-21T09:04:57.653Z
Just did a little googling, you want NV12 pixel format in Microsoft Media Foundation if you want the semi-planar format. Should be the fastest format for doing colour space conversion on the GPU.
PaulHK
CC BY-SA 3.0
2016-07-22T19:57:25.430Z
@PaulHK The camera only supports YUY2 and MJPG formats. I tried unpacking in the shader - the results were close to correct but there were vertical lines. I assume this is because I am incorrectly fetching the next pixel: `texture(tex, vec2(Texcoord.x + 1, Texcoord.y));` I'm guessing this is due to scaling.
001
CC BY-SA 3.0
2016-07-25T02:25:10.113Z
It might be simpler to deinterleave it into 2 buffers on the CPU ( Y & CrCb ), then the GPU can do CSC.
PaulHK
<opengl><texture><pixels>
Pixel packed texture source
001
# Pixel packed texture source

(Asked by: username_0 on 2016-07-20)

I am grabbing video from a web cam using Microsoft Media Foundation. The image data is in YUY2 format. The pixels are packed in a 4:2:2 ratio. The color format is YCbCr. The color data gets repeated 2x. Ex the packed pixel data is:

```
y0 u y1 v

```

Gets unpacked to 2 pixels:

```
y0 u v and y1 u v

```

I want to use this data as an OpenGL texture. The question is: is there a way to pass this type of texture to `glTexImage2D` as-is? Or do I need to unpack it first? I would like you keep the YCbCr color and convert to RGBA in a shader.

(I know I can use OpenCV to get video - I've already done that. I am trying a more low level solution mostly as a learning experience).

**Tags:** opengl, texture, pixels

> **Comments:**
> **username_1**: Normally you would use YUV semi-planar for this, which is a seperate single channel Y texture and a packed UV channel.  You could also unpack inside a fragment shader by testing if the Y texel you are reading is on a odd or even column. So say you read Y0,U for the first pixel (Assuming a 2 channel texture), you test if Y0 is on a even-column, which means you need to read the texel to right to get the missing V. For odd-columns you would read the texel to the left to get the missing U.  
> **username_1**: Just did a little googling, you want NV12 pixel format in Microsoft Media Foundation if you want the semi-planar format. Should be the fastest format for doing colour space conversion on the GPU.  
> **username_0**: @username_1 The camera only supports YUY2 and MJPG formats. I tried unpacking in the shader - the results were close to correct but there were vertical lines. I assume this is because I am incorrectly fetching the next pixel: `texture(tex, vec2(Texcoord.x + 1, Texcoord.y));` I'm guessing this is due to scaling.  
> **username_1**: It might be simpler to deinterleave it into 2 buffers on the CPU ( Y & CrCb ), then the GPU can do CSC.  

---

## 1 answers

### Answer by username_2 on 2016-07-22. Score: 3 | Accepted answer

> The question is: is there a way to pass this type of texture to glTexImage2D as-is?

Yes; you can just pass it as-is, but (looking at [the docs](https://www.opengl.org/sdk/docs/man/html/glTexImage2D.xhtml)) it is not supported directly. You'll need a custom shader to *interpret* it.

Depending on your needs, expanding the pixels first (or even doing the full conversion CPU-side) may be worthwhile. At a guess, I'd say that expanding the pixels (as you copy from the camera's driver buffer into a staging buffer for upload to the GPU) is a pretty consistently good idea, unless you're extremely bandwidth-saturated. I'd add the color-space conversion for easier debugging, or if the image doesn't update frequently.

> **Comments:**
> **username_0**: Yes, I've decided it's best to unpack the data on the CPU (a task that lends itself well to multi-threading), then do color conversion in a shader.  

---

