<p>I don't know if there is a smarter/more efficient way to do it, but a possible approach is the following:</p>
<p><strong>Step 1 - Projection volume</strong></p>
<p>First, you calculate the &quot;projection volume&quot; where its near plane corners are identical to the texture corners on your screen. You can do this by transforming the screen coordinates of the textures' corners into the range [-1, 1]. Then you use those coordinates to create a quad in NDC by combining the transformed x-y coordinates with z=-1 (near plane) and z=1 (far plane).</p>
<p><strong>Step 2 - Collision test</strong></p>
<p>Now you need to have both, your model and your &quot;projection volume&quot;, in the same space. You can either transform the &quot;projection volume&quot; into your model space or your model into NDC. I think the second choice makes the next step a little bit easier.</p>
<p>Now you have to find all triangles that are fully or partially inside the &quot;projection volume&quot;. This is basically a collision detection check (triangle vs. quad or frustum) and I won't go into the details here. For all triangles that are fully inside the volume, you now have a connection between each vertex and its position inside the texture you want to project. Simply transform the vertex position into screen space and from there to the textures pixel coordinates. The second transformation depends on where you positioned your texture on the screen and how you have scaled it.</p>
<p>For all partially covered triangles, the basic procedure is the same, but you have to introduce extra &quot;helper vertices&quot; and to transform them too to separate the covered from the uncovered areas. Doesn't mean that you have to really split your triangles into multiple smaller ones, but at least you need the extra vertices as intermediate data for the final step.</p>
<p><strong>Step 3 - Adjust model data</strong></p>
<p>In the last step, you have to bake this information somehow into your model. This totally depends on your model's data structure and layout. You can for example just split partially covered triangles into new ones and then set the <code>texCoord</code> of every vertex that is inside the &quot;projection volume&quot; accordingly.</p>
<p>If you have some kind of model-specific  UV-mapped textures, you would now use the information of the previous step to interpolate and copy the data of your projected texture to your model texture. For each vertex, you know where it is located inside the projected texture and inside its models UV texture. Now you basically cut out each triangle from your projected texture, and transform it into the corresponding triangle in your UV texture. Notice that this is not trivial since the triangle usually gets distorted and scaled. However, you can use the graphics pipeline for that by rendering directly to your UV-texture.</p>
<p>There are probably a lot of other data representations that might require a different approach for this final step. However, the key here is the second step which establishes the connection between your vertices and the texture. Once you have this, it shouldn't be too hard to come up with a method to add the projected texture to your model.</p>
<p><strong>Further information</strong></p>
<p>During the second step, you might also need to perform some kind of depth testing, so that only the closest area gets affected and not all sides that are inside the projection volume.</p>

CC BY-SA 4.0
2020-07-17T14:01:40.983Z
Hey thanks for a very detailed answer! Regarding the collision detection, does it have to be performed on the CPU? Do we cut the partially covered triangles, create a temporary mesh including the new vertices and pass it into vertex buffer?
Lenny White
CC BY-SA 4.0
2020-07-17T19:27:05.260Z
Also do you know if projective texture mapping is a viable technique?
Lenny White
CC BY-SA 4.0
2020-07-17T21:36:49.903Z
It should also be possible to do the collision detection in a compute shader on the GPU, but it depends on the complexity of your model if it is worth it. Regarding the second part of your question, that depends on how you are going to store the results in your model. In the first case, I mentioned you need to update your whole mesh with the new vertices. If you use a UV texture for your model, you can create a temporary mesh to render the triangles to the UV texture.
wychmaster
CC BY-SA 4.0
2020-07-17T21:50:14.980Z
"Also do you know if projective texture mapping is a viable technique?" --- Yes, why shouldn't it be? However, keep in mind that you should use the approach I described only if you want to include the projected texture directly into your model. If you just want to project a texture somewhere into your scene, there are easier, more effective ways to do that since you do not need to find a connection between model surface coordinates and texture coordinates. Just search for "Projective texture mapping" and you should find some material on how to implement it. It's similar to shadow mapping.
wychmaster
CC BY-SA 4.0
2020-07-17T22:33:28.037Z
Oh okay, I need the projected texture to be directly painted onto the model and its associated texture. So I guess projective texture mapping wouldn't work for me. When it comes to drawing onto models with brushes, I imagine the same technique (the original one you proposed) could work. However not sure if it would be too slow with continuous brush strokes. Something like this https://imgur.com/a/McpJy7b
Lenny White
CC BY-SA 4.0
2020-07-17T08:33:04.307Z
2020-07-17T08:39:06.813Z
2020-07-17T08:39:06.813Z
wychmaster
<p>In most modeling software you can texture paint onto 3d models using so called stencil textures. They basically project a texture from the viewport view onto the model as seen for instance in the image below from Blender.</p>
<p>What is the general technique used to project textures like this onto 3d models?</p>
<p><a href="https://i.sstatic.net/ByHK2.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ByHK2.png" alt="enter image description here" /></a></p>
<p><a href="https://i.sstatic.net/EFrxb.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/EFrxb.png" alt="enter image description here" /></a></p>

<opengl><projections>
How are textures projected onto 3d models in texture painting applications
Lenny White
# How are textures projected onto 3d models in texture painting applications

(Asked by: username_0 on 2020-07-17)

In most modeling software you can texture paint onto 3d models using so called stencil textures. They basically project a texture from the viewport view onto the model as seen for instance in the image below from Blender.

What is the general technique used to project textures like this onto 3d models?

[![enter image description here](https://i.sstatic.net/ByHK2.png)](https://i.sstatic.net/ByHK2.png)

[![enter image description here](https://i.sstatic.net/EFrxb.png)](https://i.sstatic.net/EFrxb.png)

**Tags:** opengl, projections


---

## 1 answers

### Answer by username_1 on 2020-07-17. Score: 1 | Accepted answer

I don't know if there is a smarter/more efficient way to do it, but a possible approach is the following:

**Step 1 - Projection volume**

First, you calculate the "projection volume" where its near plane corners are identical to the texture corners on your screen. You can do this by transforming the screen coordinates of the textures' corners into the range [-1, 1]. Then you use those coordinates to create a quad in NDC by combining the transformed x-y coordinates with z=-1 (near plane) and z=1 (far plane).

**Step 2 - Collision test**

Now you need to have both, your model and your "projection volume", in the same space. You can either transform the "projection volume" into your model space or your model into NDC. I think the second choice makes the next step a little bit easier.

Now you have to find all triangles that are fully or partially inside the "projection volume". This is basically a collision detection check (triangle vs. quad or frustum) and I won't go into the details here. For all triangles that are fully inside the volume, you now have a connection between each vertex and its position inside the texture you want to project. Simply transform the vertex position into screen space and from there to the textures pixel coordinates. The second transformation depends on where you positioned your texture on the screen and how you have scaled it.

For all partially covered triangles, the basic procedure is the same, but you have to introduce extra "helper vertices" and to transform them too to separate the covered from the uncovered areas. Doesn't mean that you have to really split your triangles into multiple smaller ones, but at least you need the extra vertices as intermediate data for the final step.

**Step 3 - Adjust model data**

In the last step, you have to bake this information somehow into your model. This totally depends on your model's data structure and layout. You can for example just split partially covered triangles into new ones and then set the `texCoord` of every vertex that is inside the "projection volume" accordingly.

If you have some kind of model-specific UV-mapped textures, you would now use the information of the previous step to interpolate and copy the data of your projected texture to your model texture. For each vertex, you know where it is located inside the projected texture and inside its models UV texture. Now you basically cut out each triangle from your projected texture, and transform it into the corresponding triangle in your UV texture. Notice that this is not trivial since the triangle usually gets distorted and scaled. However, you can use the graphics pipeline for that by rendering directly to your UV-texture.

There are probably a lot of other data representations that might require a different approach for this final step. However, the key here is the second step which establishes the connection between your vertices and the texture. Once you have this, it shouldn't be too hard to come up with a method to add the projected texture to your model.

**Further information**

During the second step, you might also need to perform some kind of depth testing, so that only the closest area gets affected and not all sides that are inside the projection volume.

> **Comments:**
> **username_0**: Hey thanks for a very detailed answer! Regarding the collision detection, does it have to be performed on the CPU? Do we cut the partially covered triangles, create a temporary mesh including the new vertices and pass it into vertex buffer?  
> **username_0**: Also do you know if projective texture mapping is a viable technique?  
> **username_1**: It should also be possible to do the collision detection in a compute shader on the GPU, but it depends on the complexity of your model if it is worth it. Regarding the second part of your question, that depends on how you are going to store the results in your model. In the first case, I mentioned you need to update your whole mesh with the new vertices. If you use a UV texture for your model, you can create a temporary mesh to render the triangles to the UV texture.  
> **username_1**: "Also do you know if projective texture mapping is a viable technique?" --- Yes, why shouldn't it be? However, keep in mind that you should use the approach I described only if you want to include the projected texture directly into your model. If you just want to project a texture somewhere into your scene, there are easier, more effective ways to do that since you do not need to find a connection between model surface coordinates and texture coordinates. Just search for "Projective texture mapping" and you should find some material on how to implement it. It's similar to shadow mapping.  
> **username_0**: Oh okay, I need the projected texture to be directly painted onto the model and its associated texture. So I guess projective texture mapping wouldn't work for me. When it comes to drawing onto models with brushes, I imagine the same technique (the original one you proposed) could work. However not sure if it would be too slow with continuous brush strokes. Something like this https://imgur.com/a/McpJy7b  

---

