<p>I think the problem comes from this starting assumption:</p>
<blockquote>
<p>If we have a quad that is non-planar.</p>
</blockquote>
<p>A non-planar quad <em>does not have</em> a normal. It's not a flat surface, so you can't talk about what its normal would be. You can't talk about the normal of a sphere, a cone, or any other non-flat surface. You can talk about what the normal might be at any particular point on that surface, but there is no normal for the surface in its entirety.</p>
<p>You are asking a question for which there is no correct answer. But there are many incorrect answers, and they're all <em>equally</em> wrong. So there's no reason to pick one wrong answer over another.</p>
<p>If you're talking about computing vertex normals, a non-planar quad is still not a valid surface. The two triangulations for a non-planar quad represent <em>very</em> different surfaces, which would have very different vertex normals. Neither answer can claim legitimacy over the other, so both answers are equally wrong.</p>
<p>So why choose that particular method? Because it's no more wrong than any other method you might pick.</p>
<p>It does not make sense to talk about the vertex or face normals for non-planar polygons. So if you want an answer that makes sense, fix <em>that</em> problem first.</p>

CC BY-SA 4.0
2021-03-18T15:07:27.003Z
So the implementation in most 3d modeling software is flat out wrong mathematically? And if it's wrong why do they do it that way? For flat shading at least if you have a non-planar polygon, a single normal is computed for it, and the normals for vertices belonged to it are split and assigned the polygon normal. Is the correct strategy than as I suggested to instead triangulate the polygon if it's non-planar, and calculate individual normals for each triangle?
Lenny White
CC BY-SA 4.0
2021-03-18T15:12:35.477Z
@LennyWhite: When I said "there is no correct answer," I wasn't prevaricating. As I pointed out, there are multiple triangulations for polygons, and for non-planar polygons they will give different answers with significantly different visual results. No triangulation is more correct than another. So there is no right answer. Your strategy is no less or more valid than theirs.
Nicol Bolas
CC BY-SA 4.0
2021-03-18T15:27:07.973Z
Oh I see, thank you! In that case my question becomes what are some of the popular strategies used (no matter if mathematically incorrect) for calculating vertex normals for a mesh that includes non-planar polygons. Specifically to achieve smooth shading. I guess I'll create a separate question for this, more on point.
Lenny White
CC BY-SA 4.0
2021-03-18T01:19:06.740Z
2021-03-18T01:19:06.740Z
Nicol Bolas
<p>If we have a quad that is non-planar. Why do most 3d modeling packages choose to use a single normal for the quad (Specifically for Blender3d <a href="https://blender.stackexchange.com/questions/47656/how-does-blender-implement-flat-shading-non-planar-polygons">by using cross product of two edge's vectors that do not share a vertex</a>, <a href="https://math.stackexchange.com/questions/3049969/calculate-surface-normal-and-area-for-a-non-planar-quadrilateral">here's the math</a>). Instead of viewing it as two triangles, and using a separate normal for each triangle?</p>
<p>Normally we calculate a vertex normal as the normalized average of all polygons the vertex is part of. If we have a quad consisting of two triangles, then the two vertices that are part of both triangles, get the average of the two triangle normals. This results in smooth shading across the non-planar quad. Instead of the uniform shading we would get if we were to use a single normal for the whole quad.</p>
<p>This also extends to polygons in general. In 3d modeling sofware a single normal is calculated for the whole polygon (for Blender3d <a href="https://blender.stackexchange.com/questions/47656/how-does-blender-implement-flat-shading-non-planar-polygons">with Newell's method</a>), instead of individual normals for each triangle it consists of.</p>
<p>To build on top of this, if we have a mesh with flat shading, each polygon gets a single normal as described above. Flat shading means that the shared vertices are split. And each vertex gets the normal of the polygons it's part of. However what happens if we want to apply a smooth shading on the mesh? Are the vertex normals calculated as average of all <em>polygons</em> it is part of? This doesn't seem right. Do we instead <em>triangulate the mesh</em>, and calculate the vertices as the average of all triangles it's part of?</p>

CC BY-SA 4.0
2021-03-19T07:18:51.497Z
The question is uncomplete. What you do is dependent on what you intend the nonplanar ngon to model. If its just for fancy graphics any method probably works fine. But if it has some simulation reason then you need to think about it in that contex. Since we dont know the context we can not answer.
joojaa
<shading><polygon>
Why do 3d modeling packages use a single normal per polygon and how is this viable for smooth shading
Lenny White
# Why do 3d modeling packages use a single normal per polygon and how is this viable for smooth shading

(Asked by: username_0 on 2021-03-17)

If we have a quad that is non-planar. Why do most 3d modeling packages choose to use a single normal for the quad (Specifically for Blender3d [by using cross product of two edge's vectors that do not share a vertex](https://blender.stackexchange.com/questions/47656/how-does-blender-implement-flat-shading-non-planar-polygons), [here's the math](https://math.stackexchange.com/questions/3049969/calculate-surface-normal-and-area-for-a-non-planar-quadrilateral)). Instead of viewing it as two triangles, and using a separate normal for each triangle?

Normally we calculate a vertex normal as the normalized average of all polygons the vertex is part of. If we have a quad consisting of two triangles, then the two vertices that are part of both triangles, get the average of the two triangle normals. This results in smooth shading across the non-planar quad. Instead of the uniform shading we would get if we were to use a single normal for the whole quad.

This also extends to polygons in general. In 3d modeling sofware a single normal is calculated for the whole polygon (for Blender3d [with Newell's method](https://blender.stackexchange.com/questions/47656/how-does-blender-implement-flat-shading-non-planar-polygons)), instead of individual normals for each triangle it consists of.

To build on top of this, if we have a mesh with flat shading, each polygon gets a single normal as described above. Flat shading means that the shared vertices are split. And each vertex gets the normal of the polygons it's part of. However what happens if we want to apply a smooth shading on the mesh? Are the vertex normals calculated as average of all *polygons* it is part of? This doesn't seem right. Do we instead *triangulate the mesh*, and calculate the vertices as the average of all triangles it's part of?

**Tags:** shading, polygon

> **Comments:**
> **username_1**: The question is uncomplete. What you do is dependent on what you intend the nonplanar ngon to model. If its just for fancy graphics any method probably works fine. But if it has some simulation reason then you need to think about it in that contex. Since we dont know the context we can not answer.  

---

## 1 answers

### Answer by username_2 on 2021-03-18. Score: 1 | Accepted answer

I think the problem comes from this starting assumption:

> If we have a quad that is non-planar.

A non-planar quad *does not have* a normal. It's not a flat surface, so you can't talk about what its normal would be. You can't talk about the normal of a sphere, a cone, or any other non-flat surface. You can talk about what the normal might be at any particular point on that surface, but there is no normal for the surface in its entirety.

You are asking a question for which there is no correct answer. But there are many incorrect answers, and they're all *equally* wrong. So there's no reason to pick one wrong answer over another.

If you're talking about computing vertex normals, a non-planar quad is still not a valid surface. The two triangulations for a non-planar quad represent *very* different surfaces, which would have very different vertex normals. Neither answer can claim legitimacy over the other, so both answers are equally wrong.

So why choose that particular method? Because it's no more wrong than any other method you might pick.

It does not make sense to talk about the vertex or face normals for non-planar polygons. So if you want an answer that makes sense, fix *that* problem first.

> **Comments:**
> **username_0**: So the implementation in most 3d modeling software is flat out wrong mathematically? And if it's wrong why do they do it that way? For flat shading at least if you have a non-planar polygon, a single normal is computed for it, and the normals for vertices belonged to it are split and assigned the polygon normal. Is the correct strategy than as I suggested to instead triangulate the polygon if it's non-planar, and calculate individual normals for each triangle?  
> **username_2**: @username_0: When I said "there is no correct answer," I wasn't prevaricating. As I pointed out, there are multiple triangulations for polygons, and for non-planar polygons they will give different answers with significantly different visual results. No triangulation is more correct than another. So there is no right answer. Your strategy is no less or more valid than theirs.  
> **username_0**: Oh I see, thank you! In that case my question becomes what are some of the popular strategies used (no matter if mathematically incorrect) for calculating vertex normals for a mesh that includes non-planar polygons. Specifically to achieve smooth shading. I guess I'll create a separate question for this, more on point.  

---

