Problem using Decaling with Stencil to avoid Z-fighting

It sounds like this code is using the stencil buffer to mask off the locations to draw the line. This isn't a "typical" decal approach. Normally the decal is draw directly into the scene and "depth bias" is turned on for the graphics state (OpenGL) or the graphics pipeline (Vulkan). Then turned off when done, again for OpenGL.
A solution that doesn't change the current rendering approach (which i guessed at) is to use a depth buffer when rendering the terrain. Then pass that depth buffer as an input to your decal shader, read the depth buffer and compare it to the depth of your current fragment. Then discard the fragment if the depth test fails. Basically, it is doing depth buffering by hand. It isn't to hard to set up and there are lots of tutorials out there to help get you going.
It also might be possible to just change the stencil buffer based on depth values this would be dependent on how the stencil is being generated. If it is regenerated each frame this could be a quick and easy solution.
On the other hand, if you are drawing lines or polygons directly, then you really should look into using depth bias settings. Assuming you are using OpenGL this is turned on before rendering the decal then turned off after rendering it. The hardware then computes an offset depending on how far the camera is away from the object up to some limit (which is also programable). This approach would allow the depth testing to be left on, it is view independent so it works from any direction, and it would allow the decal to be textured.
Also: The issue described here isn't really z fighting. Z fighting usually occurs when two fragments are being rendered to the same location and quantization causes the fragments to "fight" for visibility. As the camera moves one of the two fragments wins the "fight", but the winner keeps changing depending how much floating point error there is.


Difference between using + and * operator after tex2D to add albedo color?

In a shader, colors in the viewable range are assumed to be in the range 0 to 1. If you take a value that's between 0 and 1 and add a positive value to it, it will get brighter. However, if you take a value between 0 and 1 and multiply it by a value between 0 and 1, it will get darker. To understand, think about multiplying 0.5 by 0.5. The result is 0.25.

More generally, any value between 0 and 1 can be thought of as a percentage. 0.05 = 5%. 0.25  = 25%. So multiply any value between 0 and 1 by a percentage between 0 and 1, and it will be less than the original value. (50% of 50% is 25% in the earlier example.)

If you have a photo editing application like Photoshop or GIMP, you can see the difference between the 2 by creating 2 layers with image data in them, and setting the blend mode of the top layer to either Add or Multiply. It's particularly revealing if you use a left-to-right gradient of black to white over the same gradient rotated 90°. You'll see how all the values react to the current blend mode.


Compensation for energy loss in single-scattering microfacet BSDF models

To my knowledge, there is no easy and analytic way of recovering the energy lost in single-scattering models. The previous techniques precompute the energy loss and reinject it in the BRDF as a diffuse-like component:

http://sirkan.iit.bme.hu/~szirmay/scook.pdf
http://www.cs.cornell.edu/projects/layered-sg14/layered.pdf

What they propose is energy conservative and reciprocal, and it is probably the simplest way of fixing the visual darkening due to the energy loss. The main drawback is that since the energy loss does not have an analytic expression, it has to be precomputed and stored in a look-up table.


Confusion about data types in Jensen's subsurface scattering paper

The heading row of Figure 5(b) says that the units of $\sigma_s'$ and $\sigma_a$ are inverse millimetres, that is, inverse distance. Hence, $1/\sigma_t'$ has units of distance.

If you send a beam of photons through a medium with scattering coefficient $\sigma$ and measure it at distance $l$, then the probability that a photon has made it that far without being scattered is $e^{-\sigma l}$. Since $l$ has units of distance, $\sigma$ must have units of inverse distance.


procedurally generating a texture overlay simulating shrink wrap

I finally arrived at a solution that is not perfect, but seems to be good enough for my application right now.

The pipeline is as follows:

-> grayscale perlin noise with affine transformed (stretched and rotated) coordinate system
-> gaussian blur (sigma 3.0)
-> normalize (range 0.0 to 1.0)
-> pixels = pixels - 0.5
-> pixels = pixels * 2
-> threshold (change everything below 0 to 0)
-> add pixels to original image with an alpha value of 0.7

Here are some of the results:


Why does monte carlo ray tracing perform better than distributed ray tracing?

The term "distributed ray tracing" was originally coined by Robert Cook in this 1984 paper. His observation was that in order to perform anti-aliasing in a ray-tracer, the renderer needs to perform spatial upsampling - that is, to take more samples (i.e. shoot more rays) than the number of pixels in the image and combine their results. One way to do this is to shoot multiple rays within a pixel and average their color values, for example. However, if the renderer is already tracing multiple rays per pixel anyway to obtain an anti-aliased image, then these rays can also be "distributed" among additional dimensions than just the pixel position to sample effects that could not be captured by a single ray. The important bit is that this comes without any additional cost on top of spatial upsampling, since you're already tracing those additional rays anyway. For example, if you shoot multiple rays within your pixel to compute an anti-aliased result, you can get motion blur absolutely for free if you also use a different time value for each ray (or soft shadows if they connect to a different point on the light source, or depth of field if they use a different starting point on the aperture, etc.).

Monte Carlo ray tracing is a term that is slightly ambiguous. In most cases, it refers to rendering techniques that solve the rendering equation, introduced by Jim Kajiya in 1986, using Monte Carlo integration. Practically all modern rendering techniques that solve the rendering equation, such as path tracing, bidirectional path tracing, progressive photon mapping and VCM, can be classified as Monte Carlo ray tracing techniques. The idea of Monte Carlo integration is that we can compute the integral of any function by randomly choosing points in the integration domain and averaging the value of the function at these points. At a high level, in Monte Carlo ray tracing we can use this technique to integrate the amount of light arriving at the camera within a pixel in order to compute the pixel value. For example, a path tracer does this by randomly picking a point within the pixel to shoot the first ray, and then continues to randomly pick a direction to continue on the surface it lands on, and so forth. We could also randomly pick a position on the time axis if we want to do motion blur, or randomly pick a point on the aperture if wanted to do depth of field, or...

If this sounds very similar to distributed ray tracing, that's because it is! We can think of distributed ray tracing as a very informal description of a Monte Carlo algorithm that samples certain effects like soft shadows. Cook's paper lacks the mathematical framework to really reason about it properly, but you could certainly implement distributed ray tracing using a simple Monte Carlo renderer. It's worth noting that distributed ray tracing lacks any description of global illumination effects, which are naturally modeled in the rendering equation (it should be mentioned that Kajiya's paper was published two years after Cook's paper).

You can think of Monte Carlo ray tracing as being a more general version of distributed ray tracing. Monte Carlo ray tracing contains a general mathematical framework that allows you to handle practically any effect, including those mentioned in the distributed ray tracing paper.

These days, "distributed ray tracing" is not really a term that's used to refer to the original algorithm. More often you will hear it in conjunction with "distribution effects", which are simply effects such as motion blur, depth of field or soft shadows that cannot be handled with a single-sample raytracer.


Downscaling texture via mipmap [DirectX 11]

I found the problem. By zeroing the sampler description I implicitly set LOD bounds to 0. After adding

descSampler.MinLOD = 0;
descSampler.MaxLOD = D3D11_FLOAT32_MAX;

the mipmap works as expected. Thank you for your attention :-)


What is the correct way to put different textures on different parts of an object?

In vertex, put  vPos as varying. Then instead of doing the height check in vertex, do it in fragment.

if(outVPos.y > 1.2)
{
   textureColor = texture2D(urooftext, outTexCoords);
}
else
{
   discard;
}

By the way, varying qualifier is deprecated and replaced with out in vertex and in in fragment.


What is a Gaussian Lobe concerning BRDF and NDF?

In a BRDF or NDF this most likely refers to a spherical Gaussian distribution, also known as a von Mises–Fisher distribution.

It's similar to the usual Gaussian, but using the dot product between two unit vectors as a measure of distance to the mean:

$$ p(\hat v) = a \exp(\lambda(\hat \mu \cdot \hat v))$$

where $a$ is a normalization factor, $\lambda$ is a number controlling the width of the lobe (roughly corresponds to $1/2\sigma^2$ in the usual Gaussian), $\hat \mu$ is the unit direction vector for the mean of the lobe, and $\hat v$ is the unit direction vector at which we're evaluating the lobe.

(You'll also sometimes see it defined with $\lambda(\hat \mu \cdot \hat v - 1)$ in the exponent, but the $-1$ term can be pulled out and rolled into the normalization factor.)


What is required to limit the amount of draw calls in Unity?

There's quite a bit of information about this:

Static/dynamic batching: basically combining objects into one super-object
Atlas textures: combining textures into one large texture than only uses one material
Switch to a deferred renderer if you are using many lights

  From my understanding to render an object that contains exceptional amount of vertices, this would require a lot of draw calls.

The vertex amount doesn't directly determine the number of draw calls. There also isn't really a guide on how many draw calls is too much. The best thing to do is to profile your application and see where your performance problems come from.


Understanding lighting for physically based rendering

Here you have a full repository with a lot of papers about light transport. There's also a folder named 'theory' where you can get both intro and advanced refs regard light physics. And much more. I'm personally curating this and keeping up to date with latest advancements.

https://drive.google.com/drive/folders/16A6Yj2esRTk0FP-ALr6XfxkGQvTb_1dD?usp=sharing

However here are the main books/papers I suggest to read :

1985 - Physically Based Lighting Calculations For Computer Graphics

1986 - The Rendering Equation

1997 - A Framework for Realistic Image Synthesis

1997 - Robust MonteCarlo Methods For Light Transport Simulation

2011 - The State of the Art in Interactive Global Illumination

2014 - The Path to Path-Traced Movies


Do normals in obj files have to be changed into eye coordinate before being used?

Yes, all vectors used for lighting need to be in the same coordinates. If you do lighting in eye space, the normals would need to be transformed to eye space as well. Lighting calculations can also be done in world space, or any other space that's convenient.

You're right that normal vectors shouldn't receive translation; likewise other vectors that represent directions or displacements rather than points in space. If you represent transformations as 4×4 matrices, it's simple to skip the translation by just leaving off the last column.

Also, one subtlety that comes up is that if your transformations contain non-uniform scaling (different scale factors along different axes) or shear, then you need to use the inverse transpose matrix for normals. This is to ensure that the normals stay perpendicular to the polygons. (The inverse transpose will be the same as the original matrix if neither scaling nor shearing is present.)


How to send shadow rays to window light source with glazing

Most renderers allow you to set a flag on an object to make shadow rays ignore it (so it won't cast shadows). You could set this flag on your glazing.

That said, it's not unusual to make it so that refraction is disabled on shadow rays: that is, if they would be transmitted by an object, a child ray is traced in the same direction, ignoring any refraction that would be caused by the ior of the material. Although this gives wrong results for objects such as crystal balls, you need photon mapping or similar to get good results for that situation anyway. Just tracing a child ray is an easy way of making objects like stained-glass windows give simple coloured shadows.


How to Calculate Surface Normals for Generated Geometry

I see mainly three ways of computing normals for a generated shape.
Analytic normals
In some cases you have enough information about the surface to generate the normals. For example, the normal of any point on a sphere is trivial to compute. Put simply, when you know the derivative of the function, you also know the normal.
If your case is narrow enough to allow you to use analytic normals, they will probably give the best result in terms of precision. The technique doesn't scale too well though: if you also need to handle cases where you cannot use analytic normals, it may be easier to keep the technique that handles the general case and drop the analytic one altogether.
Vertex normals
The cross product of two vectors gives a vector perpendicular to the plane they belong to. So getting the normal of a triangle is straightforward:
vec3 computeNormal(vec3 a, vec3 b, vec3 c)
{
    return normalize(crossProduct(b - a, c - a));
}

Moreover, in the above example, the length of the cross product is proportional to the area inside abc. So the smoothed normal at a vertex shared by several triangles can be computed by summing up the cross products and normalizing as a last step, thus weighting each triangle by its area.
vec3 computeNormal(vertex a)
{
    vec3 sum = vec3(0, 0, 0);
    list<vertex> adjacentVertices = getAdjacentVertices(a);
    for (int i = 1; i < adjacentVertices; ++i)
    {
        vec3 b = adjacentVertices[i - 1];
        vec3 c = adjacentVertices[i];
        sum += crossProduct(b - a, c - a);
    }
    if (norm(sum) == 0)
    {
        // Degenerate case
        return sum;
    }
    return normalize(sum);
}

If you are working with quads, there is a nice trick you can use: for a quad abcd, use crossProduct(c - a, d - b) and it will handle nicely cases where the quad is in fact a triangle.
Iñigo quilez wrote a few short articles on the topic: clever normalization of a mesh, and normal and area of n sided polygons.
Normals from partial derivatives
Normals can be computed in the fragment shader from the partial derivatives. The math behind is the same, except this time it is done in screen space. This article by Angelo Pesce describes the technique: Normals without normals.


How to render the wireframe of a cube without the diagonals on the faces?

Things like that are usually done using an index buffer. The idea is that you have you have two buffers: One for all the vertices and one that determines the topology of what to draw. Then you can draw all the lines at once with a single glDrawElements call, using GL_LINES as mode.


Combine material coefficients and textures in Phong Shading

When applying various types of attenuation you are just multiplying by some number smaller than 1.  This is akin to every atom of your material having some probability of absorbing the light that hits it.

Something to note is that these calculations should be performed ( in the context of gamma correction and colour spaces ) in a linear space.

If you don't perform the calculations in linear space you might end up with overly bright, overly dark, overly saturated or under saturated colours.


How to know when rendering is complete in WebGL?

You can utilize ARB_sync or in other words sync objects which have been core since 3.2.

From the point-of-view of OpenGL then right after issuing a specific command (you want to know whether has finished), you'd insert a fence.

Say you want to check if a glDrawArrays() call has been executed.

glDrawArrays(...);
GLsync sync = glFenceSync(GL_SYNC_GPU_COMMANDS_COMPLETE, 0);

Note that if glFenceSync() fails it will return 0.

You can now check whether it has finished by doing:

GLint signaled = 0;
glGetSynciv(sync, GL_SYNC_STATUS, 1, NULL, &signaled);

if (signaled == GL_SIGNALED)
    // Done
else
    // Not done

You can then of course repeatably call glGetSynciv() and check when you receive GL_SIGNALED.

If you want to completely block until it's done then you can do:

GLenum status = glClientWaitSync(sync, 0, 0);

if (status == GL_CONDITION_SATISFIED)
    // Done

It blocks and waits for up to timeout nanoseconds (the last parameter).

Remember to delete the sync object again:

glDeleteSync(sync);

When it comes to WebGL. Then note that sync objects are not supported in WebGL 1.0. They are however in WebGL 2.0 in the form of WebGLSync.

The equivalent to the above code in WebGL 2.0 would be the following:

var sync = gl.fenceSync(gl.SYNC_GPU_COMMANDS_COMPLETE, 0);

var signaled = gl.getSyncParameter(sync, gl.SYNC_STATUS);

if (signaled == gl.SIGNALED)
    // Done
else
    // Not done

var status = gl.clientWaitSync(sync, 0, 0);

if (status == gl.CONDITION_SATISFIED)
    // Done

gl.deleteSync(sync);


Directly use Transform Feedback primitive count as instance count for drawing

The complexity of this makes it seem like you don't really want to use transform feedback; you want to use a compute shader. Then, through indirect rendering and SSBOs, you can write whatever vertex commands data you want.

The idea is that you have some number of indirect rendering commands (one command per set of instances). And for each instance you create, the CS invocation needs to bump the instance count of each of those commands (atomically, of course).

Now, because there are state changes between the commands, you wouldn't be able to use a multi-draw indirect command. But by using indirect commands and a CS, you can at least keep all of the data on the GPU, without having to do things like buffer copies to multiple locations.

Transform feedback is merely an ad-hoc mechanism for something that modern OpenGL allows you to just do directly. If you need to broadcast the "primitive" count to multiple locations (or really, if you're doing anything besides rendering the feedbacked data), then you should be using a compute shader, not TF.


Why does graphics speak of "dielectrics" rather than insulators?

Strictly speaking, dielectrics are not necessarily insulators. For example, salt water is a reasonable conductor but also a dielectric.

The term "dielectric" tends to show up in discussion of the Fresnel effect—how reflectance and transmittance vary with angle. Dielectric materials (i.e. nonmetals) are contrasted with metallic materials as they have different Fresnel behavior, which can be traced back to how the materials react on a microscopic scale to the electromagnetic field of an incident light wave.

So, for computer graphics purposes, the axis "dielectric/metallic" is more relevant than that of "insulator/conductor", since the former directly affects the appearance of materials.


using importance sampling to reduce the error of outscattered radiance

In general, the idea of importance sampling is to distribute samples in a way that matches the function being integrated (or more practically, matching some factor in it). This reduces the variance of the output values, allowing the Monte Carlo integration to converge faster.

In volume scattering, when a medium is homogeneous, we have the Beer–Lambert law, which states that the transmittance falls off exponentially with depth. That's the source of factors like $e^{-\sigma x}$ in those equations. So when raymarching through a medium, the regions closer to the start of the ray generally contribute more light to the output, while the farther regions have more of their light absorbed along the way.

So, if you sample points along the ray from an exponential distribution, the samples cluster toward the closer, more important regions. Then you just have to evaluate the scattering at those points and average over them.

BTW, importance sampling in the context of surface scattering is explained pretty well by this answer to another question.

On your second question, "where does single scattering fit into the general rendering equation?", well, it doesn't! What people call the "general rendering equation" actually only includes surface scattering, not volume scattering. So it's not really that "general". A general rendering equation for volume scattering might look something like:

$$ \nabla_\omega \, L(x, \omega) = \sigma_e(x) L_e(x, \omega) - \sigma_t(x) L(x, \omega) + \sigma_s(x) \int p(\omega, \omega') \, L(x, \omega') \, d\omega' $$

This states that the derivative of radiance as you move along a ray is given by the sum of emitted light from the volume, minus extinction, plus in-scattering.

To combine surface and volume scattering, we usually just say that the surface version of the equation applies at points on surfaces, and the volume version applies everywhere else.


How can I improve the performance of my custom Global Illumination approach?

You should always profile to determine where your slowdown occurs. From your question, it's unclear whether your work is CPU bound or GPU bound.

  But I don't really understand what is does: Does this render a multiple time the same mesh ? different mesh but with one shader ? different mesh with different shader but just with low CPU overhead ?

The glMultiDraw* draw commands are for allowing you to render multiple objects in one draw call. In terms of OpenGL 3.0 and later, this means that you could specify a VBO holding a bunch of objects and use glMultiDraw* to render them. You would use this with the same shader. If you want to render the same object multiple times, then using something such as instancing would be a good solution.

This all concerns draw calls though. If you have one million triangles for only a few meshes, you won't see a massive speedup with rendering. You didn't specify your framebuffer size, but if it's small enough, you can cause severe overdraw from many small triangles. If the images are really that small, maybe you should think about adding an LOD system or use lower polygon meshes and tessellate.

  trying to reduce the render time while rendering multiple time the same mesh in "Bounces" using the same technique than this

Stereo instanced rendering is specifically a technique for VR. It relies on the assumption that the scene is pretty much identical in each eye, and so it uses instancing to render to two sides of the screen at once. However, if your bounces render similar geometry, then you may be able to render multiple bounces onto a larger framebuffer provided that the bounces are independent. There is potential for speedup from avoiding framebuffer switching (which is a rather expensive process).

In newer OpenGL versions (4.0 and above), there is added support for indirect rendering, which allows you to determine parameters through GPU computing, but it's unclear if that will help you in this case.


Using a LUT to speed up a trig heavy shader for mobile devices

It has been suggested in comments repeatedly, but noone felt the need to give a proper answer, so for the sake of completeness, a straight-forward and common solution to this problem might be to use a texture as lookup table, specifically a 1D texture that contains all the values of your function for the possible input range (i.e. $[0,360)$ / $[0,2\pi)$). This has various advantages:

It uses normalized coordinates, i.e. you access the texture by mapping your angles from $[0,360]$ to $[0,1]$. This means your shader doesn't actually have to care about the specific amount of values. You can adjust its size according to whatever memory/speed vs. quality tradeoff you want (and especially on older/embedded hardware you might want a power of two as texture size anyway).
You get the additional benefit of not having to do your loop-like interval adjustment (although, you wouldn't need loops anyway and could just use a modulus operation). Just use GL_REPEAT as wrapping mode for the texture and it will automatically start at the beginning again when accessing with arguments > 1 (and similarly for negative arguments).
And you also get the benefit of linearly interpolating between two values in the array basically for free (or let's say almost free) by using GL_LINEAR as texture filter, this way getting values you didn't even store. Of course linear interpolation isn't 100% accurate for trigonometric functions, but it's certainly better than no interpolation.
You can store more than one value in the texture by using an RGBA texture (or however many components you need). This way you can get e.g. sin and cos with a single texture lookup.
For sin and cos you only need to store values in $[-1,1]$ anyway, which you can naturally upscale from the normalized $[0,1]$ range of a common 8-bit fixed-point format. However, that might not be enough precision for your needs. Some people suggested using 16-bit floating point values, as they're more precise than the usual 8-bit normalized fixed point values but less memory intensive than real 32-bit floats. But then again, I also don't know if your implementation supports floating point textures to begin with. If not, then maybe you can use 2 8-bit fixed point components and combine them into a single value with something like float sin = 2.0 * (texValue.r + texValue.g / 256.0) - 1.0; (or even more components for finer grain). This lets you profit from multi-component textures yet again.

Of course it still has to be evaluated if this is a better solution, since texture access isn't entirely free either, as well as what the best combination of texture size and format would be.

As to filling the texture with data and adressing one of your comments, you have to consider that texture filtering returns the exact value at the texel center, i.e. a texture coordinate off by half the texel size. So yes, you should generate values at .5 texels, i.e. something like this in application code:

float texels[256];
for(unsigned int i = 0; i < 256; ++i)
    texels[i] = sin((i + .5f) / 256.f) * TWO_PI);
glTexImage1D(GL_TEXTURE_1D, 0, ..., 256, 0, GL_RED, GL_FLOAT, texels);

You might, however, still want to compare this approach's performance against an approach using a small uniform array (i.e. uniform float sinTable[361], or maybe less in practice, keep an eye on your implementation's limit on uniform array size) which you just load with the respective values using glUniform1fv and access by adjustig your angle to $[0,360)$ using the mod function and rounding it to the nearest value:

angle = mod(angle, 360.0);
float value = sinTable[int(((angle < 0.0) ? (angle + 360.0) : angle) + 0.5)];


WebGL - issues doing a simple texture downsample

The downscaled picture is not smooth because you use "nearest" filtering.

Each block of 2x2 pixels in the original image is transformed into a single pixel in the downscaled image. Put another way, each pixel in the downscaled image represents the contents of 4 original pixels. Your current solution considers just one of those 4 pixels and ignores the others. This is what produces blockiness in the downscaled picture.

To fix this, each downscaled pixel must contain information from more than one original pixel. The simplest way to do this is by using a "box filter" where each downscaled pixel is computed as the mean of the 4 original pixels it covers. This can be achieved in a single texture fetch with linear filtering and the proper sampling position:

+---+---+
| A | B |  // ABCD => original pixels
+---X---+  // X => sample position in the middle of the 4 pixels (texels)
| C | D |
+---+---+

Linear filtering sampled at position 0 will give (A+B+C+D) / 4 or the average value.

Another computer graphic term for what you're doing is "mipmap generation". There are other filters which give better quality than the simple box filter I described.


Should new graphics programmers be learning Vulkan instead of OpenGL?

If you're getting started now, and you want to do GPU work (as opposed to always using a game engine such as Unity), you should definitely start by learning Vulkan. Maybe you should learn GL later too, but there are a couple of reasons to think Vulkan-first.

GL and GLES were designed many years ago, when GPUs worked quite differently. (The most obvious difference being immediate-mode draw calls vs tiling and command queues.) GL encourages you to think in an immediate-mode style, and has a lot of legacy cruft. Vulkan offers programming models that are much closer to how contemporary GPUs work, so if you learn Vulkan, you'll have a better understanding of how the technology really works, and of what is efficient and what is inefficient. I see lots of people who've started with GL or GLES and immediately get into bad habits like issuing separate draw calls per-object instead of using VBOs, or even worse, using display lists. It's hard for GL programmers to find out what is no longer encouraged.
It's much easier to move from Vulkan to GL or GLES than vice-versa. Vulkan makes explicit a lot of things that were hidden or unpredictable in GL, such as concurrency control, sharing, and rendering state. It pushes a lot of complexity up from the driver to the application: but by doing so, it gives control to the application, and makes it simpler to get predictable performance and compatibility between different GPU vendors. If you have some code that works in Vulkan, it's quite easy to port that to GL or GLES instead, and you end up with something that uses good GL/GLES habits. If you have code that works in GL or GLES, you almost have to start again to make it work efficiently in Vulkan: especially if it was written in a legacy style (see point 1).

I was concerned at first that Vulkan is much harder to program against, and that while it would be OK for the experienced developers at larger companies, it would be a huge barrier to indies and hobbyists. I posed this question to some members of the Working Group, and they said they have some data points from people they've spoken to who've already moved to Vulkan. These people range from developers at Epic working on UE4 integration to hobbyist game developers. Their experience was that getting started (i.e. getting to having one triangle on the screen) involved learning more concepts and having longer boilerplate code, but it wasn't too complex, even for the indies. But after getting to that stage, they found it much easier to build up to a real, shippable application, because (a) the behaviour is a lot more predictable between different vendors' implementations, and (b) getting to something that performed well with all the effects turned on didn't involve as much trial-and-error. With these experiences from real developers, they convinced me that programming against Vulkan is viable even for a beginner in graphics, and that the overall complexity is less once you get past the tutorial and starting building demos or applications you can give to other people.

As others have said: GL is available on many platforms, WebGL is a nice delivery mechanism, there's a lot of existing software that uses GL, and there are many employers hiring for that skill. It's going to be around for the next few years while Vulkan ramps up and develops an ecosystem. For these reasons, you'd be foolish to rule out learning GL entirely. Even so, you'll have a much easier time with it, and become a better GL programmer (and a better GPU programmer in general), if you start off with something that helps you to understand the GPU, instead of understanding how they worked 20 years ago.

Of course, there's one option more. I don't know whether this is relevant to you in particular, but I feel I should say it for the other visitors anyway.

To be an indie games developer, or a game designer, or to make VR experiences, you don't need to learn Vulkan or GL. Many people get started with a games engine (Unity or UE4 are popular at the moment). Using an engine like that will let you focus on the experience you want to create, instead of the technology behind it. It will hide the differences between GL and Vulkan from you, and you don't need to worry about which is supported on your platform. It'll let you learn about 3D co-ordinates, transforms, lighting, and animation without having to deal with all the gritty details at once. Some game or VR studios only work in an engine, and they don't have a full-time GL expert at all. Even in larger studios which write their own engines, the people who do the graphics programming are a minority, and most of the developers work on higher-level code.

Learning about the details of how to interact with a GPU is certainly a useful skill, and one that many employers will value, but you shouldn't feel like you have to learn that to get into 3D programming; and even if you know it, it won't necessarily be something you use every day.


Derivative maps vs. Tangent Space Normal maps

After some researches and some answers from professionals here is my conclusion.

Pros

Don’t require tangents or binormals. Less interpolators.
Only need two channels. less texture memory.
Don’t suffer from tangent seams.
Can be blended using alpha blending, without renormalization.
Less mesh memory: We don’t need to store a tangent vector.
Fast implementation.

Cons

More ALU
Less flexible. A normal map can represent any derivative map, but the reverse is not true. As an example, sharp edges can be difficult to represent.

So, that's a lot of pros compared to cons. But the major problem is that it's not an industry standard.
There are almost no content authoring tools nor artist know-how.

Here's is a quote from Bart Wronski that illustrates well the current status of derivative maps :

  Sadly in life / technology not always best solution wins / not even
  gets deserved attention... It's more about standards and inertia.

If you are interested in knowing more about derivative maps here are some interesting articles.

Mikkelsen and 3D Graphics: Derivative Maps | Morten Mikkelsen Blog
Derivative Maps | CodeItNow
Derivative Maps vs Normal Maps | CodeItNow
Triplanar Texturing with Derivative Maps | The Robot Seven
Derivative Normal Maps - What are they? | polycount.com
Derivative Maps | Knald Technologies Documentation

If I forgot something or if your don't agree feel free to tell in the comments, I would be glad to improve this answer.


Worst Case Scenario for Cohen-Sutherland Line Clipping Algorithm

Maybe it helps to draw out the steps of the algorithm. Here is an example for TBLR

The red is the worst case: clip top, clip bottom, clip left, clip right. Four steps!

The green is the best case: clip top, clip bottom, done. Two steps.

The blue is a middle case: clip top, clip bottom, clip left, done. Three steps.


Passing parameters in Metal Compute Kernel using Swift 4

The float2 type in the Metal structure requires an alignment of 8 bytes. The stride of the struct then becomes 24 so every value is aligned properly if you use it in an array.  Your Swift structure only uses float which requires an alignment of 4 bytes, and an overall stride of 20.

This presentation by Apple goes into some detail (starting at 30:20 of the video). The Metal Shading Language
Specification  lists the alignment of vector and matrix data types in sections 2.2 and 2.3.

Similarly, float3x3 has a size of 48 and an alignment of 16 in Metal. But an array of 9 floats would only have a size of 36 and an alignment of 4. Using the simd module can provide you data types in Swift with the same size and alignment requirements as those used in Metal.

As a side note, you should be using MemoryLayout<>.stride instead of MemoryLayout<>.size when you call setBytes.  The following Swift structure has a stride of 24 but a size of only 20.

struct BoundingBoxParameters {
    var topLeft: float2 = float2(0)   // float2 is from "import simd"
    var size: float2 = float2(0)
    var levelOfDetail: Float = 1.0
}


I changed the core profile to 4.1 by setting the window hints.

glfwWindowHint(GLFW_CONTEXT_VERSION_MAJOR, 3);
glfwWindowHint(GLFW_CONTEXT_VERSION_MINOR, 2);
glfwWindowHint(GLFW_OPENGL_FORWARD_COMPAT, GL_TRUE);
glfwWindowHint(GLFW_OPENGL_PROFILE, GLFW_OPENGL_CORE_PROFILE);


GPU not reaching 100% usage when frame rate is dropping

Generally, with VSYNC, your CPU is effectively doing nothing 99% of the time, as it merely sent the simple vertex buffer batch (few triangles, as you said) to the driver. If the shader is bandwidth-heavy, it takes multiple frames to render. In your case, you said it's 12 fps, e.g. 60/12 = 5 frames. So, the CPU (or the core that runs your thread), is literally doing nothing for 5 frames (other than waiting for vsync). That's an egregious waste of performance for a 3d engine.

Why is the 'utilization' statistics important, anyway ? What matters is your framerate, that's how you know how well you're doing, not some arbitrary 'utilization' number in the OS, that god-knows-how it got assembled in the first place.


Anisotropic GGX BRDF implementation. How is it related to isotropic GGX BRDF?

After coming back to the problem now with a fresh pair of eyes, I have managed to find my mistake.

Ad Question 1

Yes, the formula is correct. The problem is hidden in the tangent  and bi-tangent . When performing normal mapping, the tangent frame has to be perpendicular to the mapped normal, otherwise the term
 will not carry any information about the normal mapping.

Another issue here might be that once the normal is mapped, the tangent frame is not uniquely determined. Some more info on that can be found here: http://www.thetenthplanet.de/archives/1180. I have decided to solve this the same way as Blender does (look for make_orthonormals_tangent() in their repo) because I need to be compatible with it. Here is my algorithm for that:

bitangent_ortho = norm(cross(normal, tangent))
tangent_ortho = cross(bitangent_ortho, normal)

(Assuming that the normal is mapped and has unit length and the tangent was a valid tangent before the normal was mapped.)

Ad Question 2

This was indeed well answered by user @lightxbulb in a comment to my question. My only problem with the "Physically based shading at Disney" course notes was that the variable symbols weren't very clearly explained. I have since learned that the convention is the following:

 is the angle between the normal and
 is the azimuthal angle between the tangent and

Ad Question 3

The only pieces of advice that were useful for me here were the following:

Always come up with the simplest toy examples possible
Verify your assumptions
Visualize everything

Conclusion

I hope this is helpful for somebody else than just me. Good luck implementing BRDFs!


How to clip multiple tiled textures to polygon in Webgl / opengl

Quick idea:

render platforms color to screen quad
render mesh and platforms to screen quad, but as a mask. For example mesh is white, rest is black.
in another pass sample these textures and if mask is white, color it with sample from first texture. Render to screen.


How to get camera Right vector when Front vector and WorldUp vector are collinear

You can't have a camera with collinear look/front and up vectors. It doesn't make geometric sense. The whole point of the up vector is to disambiguate roll.

As russ says, for a "first-person" camera it's reasonable to always use world-up for the up vector and simply lock the pitch angle to within (say) 85 degrees of horizontal, to avoid degenerate cases.

For a CAD viewer where the user would expect to pitch the camera arbitrarily, you should rotate both the up and front vectors about the right vector when the user pitches the camera, to ensure they are always perpendicular. Beware that when you do this kind of arbitrary rotation, it's easy for the user to get stuck in unfortunate roll angles. You need to do at least one of the following to let them get themselves out:

Have a "reset up" button that applies the pitch angle limits and resets the up vector to world up.
Give the user arbitrary control over camera roll. (This can be confusing, so it's best to offer the first option as well if you do this.)
Constrain the up vector so that its dot product with the world up is non-negative. That is, if the user pitches so far up or down that they are upside-down, roll the camera 180 degrees.


How to dynamically calculate texture coordinates in a vertex shader with OpenGL?

To be able to give a good answer, we need to know what is the compiler error you are referring to.

At first sight your shader looks ok, although:

It always sets texture_coord.y to 0, which might not be what you want.
The .f notation (in orientation*(1/6.f)) doesn't exist in GLSL. It should be 1./6.. Moreover, I am not entirely sure if strict GLSL allows to multiply a uint with a float; a cast might be necessary.
You also need to make sure that on the receiving end, the fragment shader is declaring the texture coordinate as needed: in vec2 texture_coord;.

It is fairly common to set texture coordinates dynamically in the vertex shader, so you should be able to do so too.

Update to your additional questions and details:

  OpenGL throws an INVALID_OPERATION error

OpenGL error handling is quite tricky, since you have to poll for errors yourself, and it uses the same few error codes for wildly different functions. You have to identify exactly which function yielded the error, then refer to the documentation for the list of reasons this error can happen for that function. To make that task easier, the best way I have found so far is to always surround OpenGL calls with a macro doing an error check. You can find an implementation of a macro like that in the source code of bgfx for example.

  The error comes from my face type array being too big, but now I don't know how to pass the face type information into the shader, since I know I need 2048 elements at most

The layout declared in the shader must match the one declared through the OpenGL calls. The number of vertices is irrelevant on the shader side, but it must be consistent on the OpenGL declaration side. Unless you are using different indexes for the vertex attributes, you should have as many positions, normals, texture coordinates and face types.

The following snippet from your question seems to be declaring face types for only three vertices though:

glBindBuffer(GL_ARRAY_BUFFER, experiment);
glVertexAttribPointer(3, 1, GL_INT, GL_FALSE, sizeof(Face), (void*)0);
glBufferData(GL_ARRAY_BUFFER, face_types.size()*sizeof(Face), face_types.data(), GL_DYNAMIC_DRAW);
glVertexAttribDivisor(3, 1);

I am not familiar with glVertexAttribDivisor, but from what I read, it seems to be limited to instanced rendering.

You can find a simple example of binding in the examples of Dear ImGui.


How is anisotropic filtering typically implemented in modern GPUs?

The texture filtering hardware takes several samples of the various mipmap levels (the maximum amount of samples is indicated by the anisotropic filtering level, though the exact amount of samples taken in a given filtering operation will depend on the proportion between the derivatives on the fragment.) If you project the cone viewing a surface at an oblique angle onto the texture space, it will result in approximately an oval shaped projection, which is more elongated for more oblique angles. Extra samples are taken along the axis of this oval (from the correct mip levels, to take advantage of the pre-filtering they offer) and combined to give a sharper texture sample.

Another technique know as rip-mapping (mentioned in the Wikipedia article on Mipmapping), which is not commonly found in contemporary GPUs, uses prefiltering of textures. In contrast to mips, the texture is not scaled down uniformly but using various height-width-ratios (up to a ratio dependent on your chosen anisotropic filtering level). The variant - or maybe two variants if using trilinear filtering - of the texture is then chosen based on the angle of the surface to minimize distortion. Pixel values are fetched using default filtering techniques (bilinear or trilinear). Rip-maps are not used in any hardware that I know of due to their prohibitive size: while mipmaps use additional 33% storage, ripmaps use 300%. This can be verified by noting that texture usage requirements don't increase when using AF, rather, only bandwidth does.

For futher reading, you might want to take a look at the specification for the EXT_texture_filter_anisotropic OpenGL extension. It details the formulas used to calculate samples and how to combine them when using anisotropic filtering.


Alternatives to Clipping in avoiding display problem

"Go not to the CG Elves for counsel for they will say both no and yes".
Yes and No. As noted in another post, these issues don't exist with ray tracing but I'll assume you are interested in standard rasterisation.

It's possible to dispose of near clipping by using homogeneous rendering but that can introduce additional cost. (e.g more expensive triangle set up, more difficulty in identifying which screen regions are covered in tile-based renderers)

You could eliminate XY clipping by using float representations for the X and Y coordinates in the rasteriser but that can lead to catastrophic cancellation when computing triangle determinants which, AFAICS, is a necessary step. In my experience, hardware rasterisers* usually work in fixed-point so that everything is 'exact'.

As a compromise to XY clipping, guard-band clipping was introduced so that clipping only needs to be done on triangles that extend a 'long'** way off screen. Culling, however, is still be done with the standard frustum planes to eliminate those that are trivially off-screen.

*Dreamcast being a notable exception.
**for certain definitions of 'long', e.g. a small integer multiple of the screen width/height.


How can I intercept and filter all frames coming out of SteamVR?

So far, I've been able to attach to vrcompositor.exe with RenderDoc and intercept the D3D calls that do the composition. According to Baldur Karlsson (the author of RenderDoc), it's not that hard to hack RenderDoc to use its hooks to run arbitrary code on each Present call of the chain I'm interested in.

I've not had a chance to try that out yet, so this isn't a final answer! I'll update this post with any further progress.


You will need a new chunk of memory for the new surface anyway, so why not read from the original and store directly in the result.

Having precomputed normals will reduce a lot of the computation requirements though (especially needing to find&read each neighbour), the user action of the extrusion will probably be interactive so you will have multiple frames with different extrusion amounts but the normals of the original surface won't change. Having the normals be around will likely be helpful for other operations as well. You could make the temporary transformed geometry just a vertex shader trick as well and only commit the result once needed.

Computing normals is much simpler by going face per face and then summing the the non-normalized normals per vertex and normalizing after.


How Sunlight is added in Bidirectional Path tracing?

This can be done in two ways - either use a sky with a sun in it as an cubemap-type light source outside your building, or use the windows as light sources. With BDPT, you could sort of do both, and consider the windows area light sources where the light differs depending upon the angle, using a sun and sky image/model, and then you could implement importance sampling so that you usually get light rays going from the sun instead of the darker sky. This way you get all the accuracy of a full sun and sky model, and it'll still look good at low sample rates.


Determining slopes of lines in perspective projection

You definitely need to know distances and angle of view of the camera.  For example, if you add lines that are slightly further apart, they will be at different angles.  Like a multi-lane highway.


What kind of denoising must follow locally adaptive binarization?

Besides the suggestions in comments, I would suggest a high pass filtering to get rid of the low frequency noise, before running your local thresholding. Here an example with GIMP, that you can probably easily reproduce with opencv as well.


gl.vertexAttribPointer is used when different vertices should have different values, and gl.vertexAttrib is used when all vertices should have the same values, and you don't want to use a special shader with a uniform.


I would be tempted to use a solution which involves decoding a single rectangular video stream so you don't need any special video decoder. As you have said there will be performance penalties for decoding multiple seperate streams.

Something like octohedran cube maps might be a good offline way to convert 6 video streams into a single video stream while minimizing distortion. A relatively cheap pixel shader can be used to project the 'cube' map back to camera space.

Further reading here > http://www.vis.uni-stuttgart.de/~engelhts/paper/vmvOctaMaps.pdf


Solid angle and surface area values in (photometric) raytracing

You have a lot of questions so let me take a shortcut and just try to explain how this is properly done (:

What you are interested in calculating is luminance (cd/m^2) for a pixel. A simple way you can do this is by using brute-force Monte Carlo integration over the hemisphere defined by the normal of a pixel, i.e. cast few thousand randomly distributed rays over the hemisphere. For each of those sample rays you evaluate the BRDF (1/sr) and multiply it with the luminance of the area light source the ray hits, or 0 if it doesn't hit anything. Sum up all these results and multiply by 2*pi/num_samples in the end, i.e. by the solid angle of each sample.

This operation gives you the the luminance for the pixel, which you then multiply by camera exposure factor to map to a proper range which you can tone map, i.e. calculate EV100 from camera aperture, shutter time and ISO-settings as follows:

float ev100=log2((aperture*aperture/shutter_time)*100.0f/iso);

And then calculate exposure factor:

float exposure_factor=1.2f*pow(2.0f, ev100);

For IES light profiles you can normalize the profile and modulate light luminance with the IES factor for the given direction the ray hits the light.


The question is: is there a way to pass this type of texture to glTexImage2D as-is?

Yes; you can just pass it as-is, but (looking at the docs) it is not supported directly.  You'll need a custom shader to interpret it.

Depending on your needs, expanding the pixels first (or even doing the full conversion CPU-side) may be worthwhile.  At a guess, I'd say that expanding the pixels (as you copy from the camera's driver buffer into a staging buffer for upload to the GPU) is a pretty consistently good idea, unless you're extremely bandwidth-saturated.  I'd add the color-space conversion for easier debugging, or if the image doesn't update frequently.


Consistent animation speed across systems

It works exactly the way you describe. Most 3D games work on keyframe animations, where the human animator sets the important positions (for each joint or whatever is being animated) and the times they should be shown. The times are usually a number of "frames", but these are just intervals of 1/60 s, regardless of what frame rate is actually achieved on the client machine. The computer interpolates between those keyframe positions to decide what to draw at any given time.

For example, let's say an animation has keyframes on frame 0, 6, and 12. These are the positions to show at the beginning of the animation, 6/60 s later (that's 0.1 s), and 6/60 s after that. The first frame is drawn with the pose from the first keyframe.

Let's say the game is running really slowly, and the next frame starts being assembled 0.05 s later (which would give a 20 fps frame rate). The animation system uses interpolation to make a pose that's halfway between the first and second keyframes, because the time is halfway between 0 and 0.1. Next, imagine things are speeding up a bit, and the next frame is drawn 0.04 s later. It's 0.09 s since the start of the animation, so the animation system makes a pose that's 90% from the first keyframe to the second keyframe.

If this frame rate continues, the next frame will be generated at 0.13 s. This time the animation system has to interpolate between the second and third keyframes, taking 70% of the second and 30% of the third. Note that the second keyframe was never drawn in this example, but it contributed to the poses that were drawn, through interpolation.

Now to finish off the example quickly, let's assume the computer really slows down so the next frame is another 0.07 s later, making 0.2 s since the start of the animation. This is exactly the time that the third keyframe was set for, so the animation system just uses that pose. Although the animation was designed to take 13 frames (including the one at the start), we've finished it in just 5 frames of jerkiness, but it still takes the same length of time, 0.2 s.

In summary, it doesn't matter what time each frame actually comes out on the end system, because we can make whatever pose we like via interpolation.


Choosing Reflection or Refraction in Path Tracing

TL;DR

Yes, you can do it like that, you just have to divide the result by the probability of choosing the direction.

Full Answer

The topic of sampling in path tracers allowing materials with both reflection and refraction is actually a little bit more complex.

Let's start with some background first. If you allow BSDFs - not just BRDFs - in your path tracer, you have to integrate over the whole sphere instead of just the positive hemisphere. Monte Carlo samples can be generated by various strategies: for the direct illumination you can use BSDF and light sampling, for the indirect illumination the only meaningful strategy usually is the BSDF sampling. The sampling strategies themselves usually contain the decision about which hemisphere to sample (e.g. whether reflection or refraction is computed).

In the simplest version, the light sampling usually doesn't take care much about reflection or refraction. It samples the light sources or the environment map (if present) with respect to the light properties. You can improve sampling of environment maps by picking just the hemisphere in which the material has non-zero contribution, but the rest of the material properties is usually ignored. Note that for and ideally smooth Fresnel material the light sampling doesn't work.

For BSDF sampling, the situation is much more interesting. The case you described deals with an ideal Fresnel surface, where there are only two contributing directions (since Fresnel BSDF is in fact just a sum of two delta functions). You can easily split the integral into a sum of two parts - one reflection and one for refraction. Since, as you mentioned, we don’t want to go in both directions in a path tracer, we have to pick one. This means that we want to estimate the sum of numbers by picking just one of them. This can be done by discrete Monte Carlo estimation: pick one of the addends randomly and divide it by the probability of it being picked. In an ideal case you want to have the sampling probability proportional the the addends, but since we don't know their values (we wouldn't have to estimate the sum if we knew them), we just estimate them by neglecting some of the factors. In this case, we ignore the incoming light amount and use just the Fresnel reflectance/transmittance as our estimates.

The BSDF sampling routine for the case of smooth Fresnel surface is, therefore, to pick one of the directions randomly with probability proportional to the the Fresnel reflectance and, at some point, divide the result for that direction by probability of picking the direction. The estimator will look like:

$$
\frac
   {L_{i}\left(\omega_{i}\right)F\left(\theta_{i}\right)}
   {P\left(\omega_{i}\right)} =
\frac
   {L_{i}\left(\omega_{i}\right)F\left(\theta_{i}\right)}
   {F\left(\theta_{i}\right)} =
L_{i}\left(\omega_{i}\right)
$$

Where $\omega_{i}=\left( \phi_{i}, \theta_{i} \right)$ is the chosen incident light direction, $L_{i}\left(\omega_{i}\right)$ is the amount of incident radiance, $F\left(\theta_{i}\right)$ is either the Fresnel reflectance for the reflection case or 1 - Fresnel reflectance for the refraction case, $P\left(\omega_{i}\right)$ is the discrete probability of picking the direction and is equal to $F\left(\theta_{i}\right)$.

In case of more sophisticated BSDF models like those based on microfacet theory, the sampling is slightly more complex, but the idea of splitting the whole integral into a finite sum of sub-integrals and using discrete Monte Carlo afterwards can usually be applied too.


Energy Conservation in Lighting Equation

Comment: if you make references to book, then it would be good if you could provide references.

To answer your questions:

First, it is quite unusual these days to see people representing colors using a computer byte that is integer values within the range [0:255]. Most people these days use floats, and you will understand why in a moment. When you look at a material in general you can see a shiny part and a diffuse part. Shininess and diffuse appearance are computed using two different types of equation. Let's call the functions that compute the diffuse part diffuse() and the function that compute the specular component specular(). So now you can write:

ObjectColor = Ks * specular() + Kd * diffuse();

Ks and Kd control how shiny or/and diffuse your object is.

About conservation of energy. In short an object can not reflect more light that it receives. So that means that technically if the specular() and diffuse() function do the right thing Ks + Kd ought to be 1. If that wasn't true you would violate the principle of conservation of energy. In practice, it is generally good to follow this rule though most of the basic shading models are based on simple diffuse and specular models (for example the Phong model) and are not themselves energy conserving, so trying to be purely energy conserving at this level, is like trying to say you can paint a miniature with a paint roller. Forget about it, just try to be more or less in the right range.

ks and kd generally stand for the specular and diffuse coefficients.

if you don't do path tracing (aka don't simulate global illumination) the object won't reflect light from other objects in the scene, just light from direct light sources. So in the old time, in order to compensate for that, they would add an ambient term. In these days in age, this is considered as 'heresy'. Though if you account for the ambient term still then Ka + Kd + Ks should be again equal to 1 (same principle).
Not sure to understand what you say here but all these components are additive. You compute the diffuse multiplied by Kd and then you multiply the specular multiplied by Ks and then you add the two resulting numbers together. And that's your object color at the intersection point.

For an introduction to these topics I recommend you read:

Introduction Ray-Tracing

Introduction to Shading

The Phong Model

In order for you to progress, I personally recommend you proceed by step and don't try to understand everything at once. The problem of rendering and shading is very complex. You need to first understand the principle of light-matter interactions, what is diffuse what is specular and how are objects illuminated (direct and indirect lighting). Once you have these notions you can jump onto shading and understand or learn about the concept of shading model, and what the flaws and strengths of existing shading models (many of them exist).


I read that paper as well last year. There's a strong assumption made by authors which is the "stationary". What this means is that in absence of light in given 3000x2000 pixels image the material is more or less the same in each tile of 192x192. The reason for this is because we the surface is lit by white light is like they have a sample of BRDF for a set of angles.

The first shot is to get the stationarity of the material. The second shot is to get how each tile reacts to the interaction with the light from a different angle.

They collect the set of all tiles with the light and they later use their BRDF model to perform the fitting.

The fitting, as you say, is done by using the L-M algorithm. There're few inconsistenties with the formula they use to perform the fitting, if I remember correctly the use non differentiable functions and they still perform the fitting.

If I remember correctly the actual fitting algorithm is constructed by one hand differentiating part of their cost function, while from the other hand they use a numerical approximation of the derivative/gradient/Jacobian.

The BRDF they use is parametrized using the several maps you mention in your question, therefore when they minimize the cost function they get optimal maps.


How to scale and offset a cubemap using sphere mapping

This doesn't really answer the question but is a workaround. Beware it's hack time :)

Instead of using the texCUBE and samplerCube I'll be using the tex2D method and a simple sampler to sample the texture.

As I am using a sphere mesh, I generate 2D UVs like this.

float x = atan2(vertex.x, vertex.z) / PI;
x = x / 2.0 + 0.5; // To remap from 0 to 1
float y = vertex.y / 2.0 + 0.5; // Also remap from 0 to 1
float3 color = tex2D(sampler, float2(x, y) * scale.xy + offset.xy);

It's a bit hacky but using this I have 2D texture coordinates for my texture.


How can i wrap the earth image around a 3D Sphere using OpenGL, GLFW, GLAD, GLM?

I'm not going to post proper working code here just giving you a guideline. There might be an easier way but what I did back in my days was to create a sphere using Triangle Strips. There is an icosahedron approach as well but I haven't looked into that.

So first of all look up on GL_TRIANGLE_STRIPS and how they work. Secondly you need to look up on Spherical coordinates and how to convert them to Cartesian Coordinates.

A brief overview is that Spherical coordinates have parameters (radius, zenith, azimuth). Radius is pretty much self explanatory. Zenith angle is the inclination angle from the pole of the sphere (lines of longitude) While the Azimuth angle runs along the latitude. Zenith runs from 0-PI (assuming 0 starts from 1 pole and PI is the second pole) or -PI/2 - PI/2 (assuming the extreme at both poles and 0 is the center) and Azimuth from 0-2PI (all the way around the sphere).

Now after that you can think of the sphere forming up from concentric Rings. At each fraction of the Zenith angle there is ring that runs around the sphere (the latitude lines). Then you create triangle strips between those rings to fill the sphere up.

So basically you want something like,

zenith = azimuth = 0
ring_increment = PI/num_rings;
sector_inrement = 2PI/num_sectors;

for(i=0; i < PI; i+= ring_increment)
{
    for(j=0; j < 2PI; j+= sector_increment)
    {
        vertex_list.add(convertToCartesianCoord(radius, zenith, azimuth))
        zenith += ring_increment
        vertex_list.add(convertToCartesianCoord(radius, zenith, azimuth))
        zenith -= ring_increment
    }
}

The more the number of rings and the number of sectors the smoother the sphere will be.


Color transform algorithm (that also works on saturation and value/lightness)?

Very often, the Lab color space is used, since it is close to how humans perceive color. As to which distance to apply: This wikipedia article mentions a Delta E distance of 1.0 for a just noticeable difference. There are a lot of different formulas resulting in different thresholds.


Which effects do I need to take into account for the iris of the eye?

For a realistic Iris, you need:

Geometry: Irides are very different for different people and their appearance changes drastically depending on whether the pupil is dilated (sphicter muscle relaxed) or not. You will not get anywhere with primitives only. The whole structure is like a thick stretchy wrinkly cloth suspended in water. Wrinkles appear and disappear as it moves. Use a sculpting software and an anatomy textbook.
Material: There is almost no specular on the iris since it is suspended in (basically) water which has a similar refractive index as tissue. I do not know the roughness values since I have never seen a dry Iris. With the GGX specular model, roughness 0.4 looks okay. Brown irides seem to be more rough, almost like very fine sand. A shading model that uses roughness in the diffuse component would be nice.
Textures: While some are colorful, the vast majority of irides in the world are brown. You don't need too much variation inside the texture, it can be quite uniform. Make it relatively saturated and don't try to hint at geometry inside the texture by drawing dark areas for shadows or occlusion. Have a normal map. If the eye is animated, you need to blend between at least two normal maps for the dilated and undilated pupil. I don't think you need a roughness map, but especially for blue irides it might help.
Refraction and reflection at the cornea: The cornea (the fluid layer coating it) acts as a lens. This distorts the image of the iris as seen by the camera. Without this, it will not look real, especially when looking at the eye from the side. The unusual thing about the eye is that, from a graphics point of view, the object is inside the lens. So there's no thin lens equation going to come to your rescue. You have to refract light once, at the interface between the tear layer and air.
The other interfaces (tear layer to outside of cornea, layers and membranes inside the cornea, inside of cornea to chamber fluid) are not relevant since the indices of refraction are all very similar or the structures are extremely thin. The only other optically important structure is the lens, which is behind the iris and can be ignored.
The shape of the cornea is critical, as it will distort everything. People with cornea deformations such as a keratoconus have weird-looking eyes. Also the whole scene must reflect on the cornea/liquid. We expect to see reflections of light sources over the iris/pupil.
The limbus: Looking frontally at the eye, there is a greyish ring around the Iris. This is the part where the cornea merges into the sclera (the "white" of the eye), transitioning from transparent to nontransparent. Behind the limbus, anatomically, is the iridocorneal angle, where the iris meets the rest of the eye. This area is impossible to see due to the refraction mentioned above, it is literally a place that has never seen any light. Light emitted from there experiences total internal refraction at the interface of the cornea, you can't look at it from the outside. Thus, it looks very very weird when the transition from iris to the rest of the eye is visible. You need both the refraction and the gradual transition from cornea to sclera to get this transition right.
Subsurface scattering: You might possibly get away without it on the iris itself since there will never be light coming from the back, especially with a brown iris (more heavily pigmented -> more attenuation during scattering). For some light directions, it will make a difference though. And you need it on the surrounding skin and sclera anyway. But it is not the most important effect.
Ambient occlusion: You need good SSAO (or GI) to make the fine structure stand out. There's almost no specular, so you need other visual cues for the geometry. Choose a method that uses your normals from the normal map. HBAO will work well. For offline rendering, the GI will be done by your renderer of choice.
Lighting: Using some physically based method goes without saying. Image based lighting, especially for the cornea reflection. Under certain angles, refraction of the light at the cornea may become relevant as well (only for analytic lights). Postprocessing (bloom), especially for the cornea reflex.

Backscattered light from the retina, although it can be seen through the iris in very extreme viewing conditions during some medical examinations, is completely irrelevant for normal purposes. Basically, paint the retina black. If you want to go that far, you can add a little orange-red to the retina color whenever a strong light comes from a direction very very similar to the camera and the pupil is dilated (like a flash from a camera). But it will still not be strong enough to shine through the iris, by far. Black is an okay approximation.

That list looks big, but you asked for photorealism ;) It will still look good if you only do the easy half and fake the rest.

(Source: I do graphics programming for medical simulations with a focus on the human eye)

Material:

Anatomically accurate modeling and rendering of the human eye (2007)
High-Quality Capture of Eyes (Disney Research, Siggraph Asia 2014)


Why is this not a proper solution to handling transparency in deferred rendering?

PaulHK is right in what he said: you have to consider that there may be more than 2 transparent objects behind each other.

Also, the idea of deferred shading is to render the geometry only once to be more efficient. If you render the geometries multiple times, you lose (part of) your efficiency. Moreover, the lighting is deferred, thus you'd need to do lighting calculations for all your geometries before you can do the correct color calculations from transparency. So what you're really doing is going back to forward shading.

That being said, one solution is to first render any opaque geometry with deferred shading and afterwards render the transparent objects with forward shading, so maybe you weren't too far off. Those transparent objects however shouldn't be affected by too many lights to still render frames "fast".


How Image Based Lighting is applied to indoor scene?

There's two main things it seems like you've not understood correctly.

IBL is for indoor scenes and outdoor scenes

In the early days when IBL was hot, the Uffizi light-probe was probably the most popular for researchers. IBL works really well for indoor scenes with strong light coming from multiple directions, such as churches with stained glass. That kind of scene was the "killer app", because for most outdoor scenes, skylight simulation was already a popular, fast, and useful method. Environment maps were originally popular for outdoor scenes because simulated clouds didn't look right, even though the lighting was good.

The big limitation for indoor scenes is if the room is small. A light probe taken in the middle of a room won't be accurate for the corners of that room, because the light is coming from different directions. Just like with a reflection map, if your objects are moving far enough for that to be visible, you need to take multiple light probes and blend between them. This can also be a limit for outdoor scenes if the lighting is dominated by (say) the shadow of a building or tree, or a nearby artificial light.

IBL is not a global illumination technique

It's more helpful to think of it as pre-baked lighting. Just as you might use a light map to bake GI computed from your static lights (with some offline technique) into the textures for your static objects, you can also have that pre-computed GI affect your dynamic objects. But it's not an approximation to full GI, because dynamic lights and objects don't contribute at all.

So how do I use it indoors?

There's two ways. If you're trying to place CG objects into a real-world indoor scene, just capture a light probe of that scene exactly the same way as a real-world outdoor scene. If you're trying to light a CG scene (i.e. a room that only exists in your game), then put your scene into an offline renderer (such as a path-tracer with full GI) to create the light probe, then apply that light probe in your real-time engine.


A BRDF is a 4D function, but as we cannot really visualize a 4D function in a comprehensible way, if we want to understand its behavior it can be instructive to fix 2 or 3 of the parameters and graph the others. This kind of graph can be called a "slice". (The mental image I have is like: the BRDF is some oddly shaped fruit, and we're taking a knife and making a slice through it, to examine the shape of the cut area.)
In particular, a common way to do this is to fix the incoming light direction; then the BRDF can be graphed as a function of the outgoing light direction, like this:

(image from http://brdflab.sourceforge.net/)
The orange line on the left represents the fixed incoming light direction. This is a polar graph, so the magnitude of outgoing light in each direction is graphed as the radius of the shape.
In this case we can see that most of the light is going out along the specular reflection direction, to the upper right; but some is heading back toward the source as well (retroreflection). These are the "lobes". Lobes are not precisely defined, but intuitively they are the visible features in the shape of the function. Here, we would say there is a "specular lobe" and a "retroreflection lobe". In fact, this BRDF looks like it has two specular lobes, one long skinny one (which represents a tightly concentrated, bright highlight) and another short and fat one (a more spread-out, soft/rough highlight). There appears be a diffuse lobe as well, which shows up as a hemisphere around the origin in this type of graph.
It's also common to simplify the graph even more by constraining the outgoing light directions to be coplanar with the normal and the incoming light, so now you're only graphing the (N, V) angle. That gets you a graph like this:

(image from Physically Based Rendering in Filament)
This is the one that's most likely to be called a "slice". This diagram also shows two "lobes", a specular one and a diffuse-ish one.
Synthetic BRDFs (what we mostly use in graphics) are usually constructed by taking a weighted average of various lobes generated by different models, e.g. a Lambert diffuse lobe with one or more microfacet specular lobes, etc. When looking at measured BRDFs, we could try to identify the "lobes" by fitting some model lobes to the measured data.
To sum up, "slice" is a type of graph that allows us to visualize the features of a BRDF, and "lobe" is the name for the typical kind of features that we see in the BRDF shape, or that we use to construct the BRDF in the first place.


How would I convert a picture of a slanted surface into a straight one?

As @user1118321 mentioned, as well as @NathanReed in this answer to another question, you can do a perspective transform to transform any convex quadrilateral to any other.

OpenCV has a good library for doing perspective transformations if you don't want to roll your own.


Generation of stratified cosine weighted samples on hemisphere

The classic method is to uniformly sample the disc at the base of your hemisphere and to project your samples upwards on the hemisphere (eg. compute z from x and y). This yields a cosine weighted distribution.

As the projection preserves stratification, you need only use stratified sampling of the disc to get a stratified cosine distribution.


It basically gives you control of the semantics of the individual attribute indices. This mostly pays off in bigger projects, where you have many different shaders which, however, often use rather common attributes, like e.g. positions or texture coordinates. With explicit attribute locations you can establish a semantic for those attributes and enforce that on all your shaders, e.g. positions are always in attribute 0, texture coordinates always in attribute 1, ...

First, this has the advantage of not having to query the attribute index for every shader. Of course you could query it once and sotre it in e.g. a map from attribute names/semantics to indices, but establishing this beforehand is even better. And it brings us to a much bigger advantage, without explicit attribute binding you would have to bind vertex arrays of a model to the individual attribute channels not only for each model, but also for each shader, since your position array might go to attribute 0 in one shader and attribute 1 in another one.

This is not only a huge hassle, but also basically destroys the advantage of Vertey Array Objects (VAO), which are specfically there to capture all the vertex attribute state, including the enabled attribute indices and their buffer sources so that you can draw your object by just binding the VAO and calling a draw function. Without explicit attribute binding you would still have to reenable/bind the individual arrays at least for each different shader you use. With explicit attribute binding you know that your model's positions always go into attribute index 0.


BUT the alpha component will be 0.3×0.3 + 0×(1 - 0.3) = 0.3, so we then have a color of (0.3, 0, 0, 0.09) which is not correct, what we want to see on the screen is rather (0.3, 0, 0, 1).

Your blending function did not use the destination alpha. So "correct" or "not correct", its value is entirely irrelevant.
However, if you want to make the alpha more accurate in some way, you can provide a different set of blending factors, and even a different blend equation, for the alpha component.


GLUT OpenGL - Drawing Shapes using only GL_POINTS

I reckon your instructor is encouraging you to implement low-level rendering algorithms.

The obstacle to draw shapes using just points is the contradiction between continuous and discrete. Shapes are logically continuous if in its area every pixel is shaded. But you can not draw an actually continuous line using discrete points because infinite points are needed. If you draw points directly in the "world", you can not guarantee every pixel on the screen can be covered after projection, unless you set the interval small enough(which may be of quite low performance).

My advice is to maintain the world coordinates of the vertices of the shape but not draw it. Get the "projected" coordinates on screen and fill the area later using DDA or some other interpolation algorithms. "Projected" is the coordinates obtained by the three matrices(Projection, View, Model), I think you've heard of these. So in the filling process GLUT is used to draw points on the 2D screen, and cover every pixel in the area.


Avoiding z-fighting with coincident surfaces

If the surfaces are exactly co-planer, your fate is up to the FPU gods; you'll more than likely have Z-fighting. If the triangles are identical and you do the exact same math to each, you will end up with the same Z-values for both. But again, this will only happen if your math operations are identical for both. (Since, in general, FPU operations are not commutative)

One possible solution is to not use the Z-buffer at all. Rather you can use the Painter's Algorithm. Granted, this comes with all the problems of the painters algorithm as well. But, it would solve the Z-fighting.

In your example case of Screen Space Decals (SSDs), the common solution is to use an offset, aka a simple thin cube. See Warhammer 40k's presentation about SSDs for reference. Or Bart Wronski's post which addresses some other issues with decals, but also links to a few other presentations about SSDs


B-Spline patches on the tessellation shader. How to get the control points

For regular quad faces, what you want to do is send the vertices for the quad you're evaluating and the eight neighboring quads around it: a total of 4x4 vertices. The B-spline needs to know where the neighboring vertices are in order to generate a surface patch that connects smoothly to the neighboring patches.

In the case that the adjacent faces aren't quads, I don't think you can use the straightforward B-spline evaluation. In Catmull-Clark subdivision, everything is quads after the first level of subdivision, though you'll still have extraordinary vertices to deal with.


Mipmap sampler in Metal compute kernel (not vertex or fragment shader)

You havent' set the mipFilter property for your samplerDescriptor.  The default value for mipFilter is notMipped which means the sampler will never access anything other than level 0. Set it to nearest or linear.


General answer is that creating a window as a GUI element with some GUI designer library involves subclassing some basic GUI element (like widget in QtCreator for example) with class providing handling GL context in more or less user friendly way (like QOpenGLWidget in QtCreator which provide some user friendly wrapper or on the other hand basic Win32 API handle) and override methods for handling its framebuffer drawing or resizing, swapping buffers etc. with usual gl extensions provided by some lib you use.

This will work perfectly fine with other GUI elements.

I don't know which GUI lib or IDE with GUI designer or language you prefer but most good libs can handle this. You can even do it in Assembler with WinApi GUI elements.


How to convert Euler angles to Quaternions and get the same Euler angles back from Quaternions?

I had to change the order of inputs to def euler_to_quaternion as follow and have to convert the inputs to radians:

def euler_to_quaternion(roll, pitch, yaw):

        qx = np.sin(roll/2) * np.cos(pitch/2) * np.cos(yaw/2) - np.cos(roll/2) * np.sin(pitch/2) * np.sin(yaw/2)
        qy = np.cos(roll/2) * np.sin(pitch/2) * np.cos(yaw/2) + np.sin(roll/2) * np.cos(pitch/2) * np.sin(yaw/2)
        qz = np.cos(roll/2) * np.cos(pitch/2) * np.sin(yaw/2) - np.sin(roll/2) * np.sin(pitch/2) * np.cos(yaw/2)
        qw = np.cos(roll/2) * np.cos(pitch/2) * np.cos(yaw/2) + np.sin(roll/2) * np.sin(pitch/2) * np.sin(yaw/2)

        return [qx, qy, qz, qw]

Also, quaternion_to_euler was returning degrees and I had to change it to radians:

def quaternion_to_euler(x, y, z, w):

    t0 = +2.0 * (w * x + y * z)
    t1 = +1.0 - 2.0 * (x * x + y * y)
    roll = math.atan2(t0, t1)
    t2 = +2.0 * (w * y - z * x)
    t2 = +1.0 if t2 > +1.0 else t2
    t2 = -1.0 if t2 < -1.0 else t2
    pitch = math.asin(t2)
    t3 = +2.0 * (w * z + x * y)
    t4 = +1.0 - 2.0 * (y * y + z * z)
    yaw = math.atan2(t3, t4)
    return [yaw, pitch, roll]


light probe (for global illumination) placement techniques?

Looking for answer for similar question, I found that work: http://www.cse.chalmers.se/~uffe/xjobb/bowald_final_master_thesis_v2.pdf

It is based on automatic grid placement which depends on scene voxelization. Inspired by this I developed similar technique:

First scene is voxelized with small voxel size (binary voxelization for memory saving)
Result is downsampled to desired grid size (size of each cube with eight probes on each corner)
Based on result - data structures are prepared
Empty areas are optionally computed - cubes size can be bigger to save memory of volumes and probes buffers but it must be always multiplication of smallest cube size to fit in the grid
Lighting for each probe is precomputed

Data structures are:

Volume map buffer - mapping world position to volume index
Volumes buffer - each is array of eight probe indices
Probes buffer - SH data of each probe

This way performance is constant, fast and independent of quality (size of cube) unlike in tetrahedra grid searching algorithm.

However I don't know if there is more recent advancement in that field. You might check http://jcgt.org if there are any new research papers.


Perspective correct interpolation of normal values

Perspective correct interpolation of normals works just as it does any color or coordinate or other linearly varying attribute. Each component of each varying vector is interpolated independently as if they were scalars, using exactly the same equation.


Changing image so it would look like through colorful glasses

(XYZ) can be the RGB colour you want to tint your scene by. For the above scene it can be a red colour (1.0, 0.0, 0.0) or something similar with a strong red component.

Bear in mind that since you are applying the colour in a multiplicative way it will act as a filter suppressing original colour components. So if your scene is mostly green but you apply (i.e. multiply it with) a red filter (1,0,0) the resulting scene will be very dark.

A trick you could potentially do in such cases is to desaturate the original scene and then multiply it by the tint colour. That way you will keep the overall image intensity and achieve the colour tint you require.

For example, here is the image multiplied by (1, 0, 0), (1, 0.2, 0.2), and (1, 0.5, 0.5) from left to right:


As other people have mentioned, due to network latency issues, I think that full rendering on a cloud server and streaming video to a client device is unlikely to be workable. Even if the latency can be kept low enough in a best-case scenario, it would only work for a very limited number of users with very high-quality internet connections. (Don't forget that e.g. in the USA, there are still tens of millions of people on dialup or DSL, with no access to cable internet, let alone fiber. It will be decades still before fiber is widespread.) Also, note that many cloud gaming services do exist, but that according to players, input latency is noticeably worse on these services than playing locally on a PC.

However, all that being said, rendering entirely on a cloud server is not the only way to approach the problem. The most latency-sensitive aspect of VR is head tracking. Oculus's "timewarp" technology shows that you can quickly and cheaply update a previously rendered image to respond to head rotation, as a 2D image operation: if the user turns their head to the right, you shift the image to the left, and so on. With a depth buffer, you can do positional timewarp as well, although this creates various artifacts with parallax because it's based on a single-layered 2D image.

Timewarp is certainly cheap enough to run on a mobile device. So, one approach is to render frames in the cloud, stream the video to the client, then timewarp the video on the client device to obtain very low-latency head tracking. This wouldn't help the latency of other inputs such as keyboard/mouse/controller inputs, but those don't induce nausea when they're delayed, the way delayed head-tracking does.

There are also other possibilities along the lines of splitting rendering between the cloud and the client. For example, object-space lighting and shading could be done on the cloud and streamed to the client as textures; then the client only needs to render simple textured geometry. This would work better than positional timewarp for getting correct parallax when you move your head around, though you'd still have some artifacts with specular shading and reflections. There are a lot of possibilities for offloading rendering to the cloud in VR-friendly ways.


Calculate the position and rotation of a quad in 3d space given a 2d projection of that quad from a camera

Hello Djgaven588 and welcome to the forums!
This is not a computer graphics question, but a computer vision one. The answer to it is a little bit involved and I cannot make it justice on a forum post. However, I can point you in the right direction so that you are able to find the answer by yourself.
Here's a helpful link: https://docs.opencv.org/master/d9/db7/tutorial_py_table_of_contents_calib3d.html
The process is somewhat straightforward, but it depends on how much you know about projections and solving systems of equations.
First you need to find key points on a known object in your image through some kind of image-processing algorithm. It seems you already did that. By known object, I mean that you know its exact measurements in the real world with regards to the selected key points.
Secondly, you need to "unproject" your 2D image points onto 3D space, by reversing the projection math from 3D to 2D. This requires building a projection matrix containing intrinsic and extrinsic camera parameters, as well as finding coefficients modelling the camera lens distortion. This process is known as camera calibration.
Thirdly, when using a single camera, you can unproject the points only up to a scale factor, because the depth coordinate has been lost during the original projection. In order to figure out the scale, rotation, and position of the object, you can set up a system of equations relating the image key points to the known key points in the real world at a specific position and rotation of the known object. Solving this system will let you figure out the transform. Note that it is important to have a reasonable amount of key points (even redundant ones) so that the error can be minimised.
OpenCV has methods and functions for doing all of these procedures, and there are many tutorials on the web, like the one I linked above.
I hope you can figure it out! :)


As you can see in the figure above, during a perspective projection, the goal is to essentially "splat" points within the view frustum onto the near frustum plane. That means that the final z coordinate is actually just fixed at $z_{vp}$.

Now, if we did this poorly, we could just map $z$ to $z_{vp}$ directly in our projection matrix by setting the third row of the matrix to $\left[0, 0, \frac{z_{vp}}{z_{prp}}, 0\right]$ and setting the final homogenous coordinate to $1$ (so no divide by $w$ happens at the end). This approach is problematic because squashing all this depth information to $z_{vp}$ loses information and we can no longer do things like a depth test.

This is why, instead of mapping all $z$ values to the same fixed distance from the camera, we opt to map it into a range of $z$ values instead. This is the "normalization" that the text is referring to. Furthermore, we aren't preserving the "values" at all. The wording in the text is confusing, but what we are preserving is the information about what points lie further away from the view origin. When it says that we have a choice about how to do this normalization, well, we can choose to map $z$ into any range that we wish, and in graphics contexts, we normally want something that preserves precision well enough that we can do nice things like get crisp shadows and such.

If you look at standard texts, the $z$ value will be mapped via the matrix row:

$$\left[ 0, 0, \frac{-(f + n)}{f - n}, \frac{-2fn}{f - n} \right]$$

and the $w$ coordinate is mapped to $-z_{prp}$. This should look similar to the other entries that map $x$ and $y$ between the right-left and top-bottom frustum planes respectively. In this case, we're mapping $z_{prp}$ between $n$ and $f$. Now, we have a $z$ coordinate that is mapped in a range such that after the divide-by-$w$ to get back to normal 3-coordinate space from homogeneous space, maps back to $z_{vp}$ as expected, but until that occurs, can be used to perform depth comparisons. The choice of $n$ and $f$ is arbitrary, but too narrow and you end up clipping $z$ values that lie outside the range, and too wide and you lose precision. Some people use a $-\log$ depth computation instead to gain as much precision as possible, but I digress.


What does "6-separating" and "26-separating" voxelization mean?

The terms have to do with the "thickness" of the voxelization. I'll illustrate with the help of a diagram about 2D line rasterization (from this unrelated question).

On the right is the typical line rasterization: the algorithm finds the one pixel nearest the line within each row (or column, depending on slope). This produces what we usually think of as a "1-pixel-thick" line. On the left is a conservative rasterization, which finds every pixel whose rectangle is touched by the line, and it produces a thicker line.

6-separating voxelization is like the thin line on the right, and 26-separating is like the thick line on the left, but in 3D. If you imagine the line is actually a triangle viewed on-edge, this is analogous to what the voxelization would look like.

Different types of voxelization may be better depending on what you're going to do with the voxelized data later. If you're using the voxels as a spatial hierarchy to find triangles that intersect a given region, you probably want the thick voxelization, as it's conservative. The thick voxelization might also be preferable for ray-marching, as the thin voxelization could be missed by a diagonal ray. On the other hand, the thin voxelization is a more faithful representation of the original surface, which is probably better for visibility tests, collision detection, fluid simulation, and suchlike.

The "n-separating" terminology is a bit unfortunate, but here's what it's getting at. Imagine you're doing a 3D flood-fill in the voxel grid, but in the flood-fill you only look at the 6 direct neighbors of each voxel (±1 step along each axis). Then the "6-separating" (thin) voxelization will stop the flood-fill: it suffices to separate the two sides of the surface, if only 6 neighbors are considered. On the other hand, suppose your flood-fill was allowed to go to diagonal neighbors as well—26 neighbors in all (3×3×3 neighborhood of voxels). Then the 6-separating voxelization wouldn't stop the flood fill, but the 26-separating (thick) one would.


How to build a decent lens/camera objective model for path tracing?

The next step up from a pinhole camera model is a thin lens model, where we model the lens as being an infinitely thin disc. This is still an idealization that pretty far from modeling a real camera, but it will give you basic depth of field effects.

The image above, from panohelp.com, shows the basic idea. For each point on the image, there are multiple rays arriving at that image point, via every point on the 2D lens surface. Therefore, generating an image like this using Monte Carlo will require picking, for each ray, both a 2D sample point on the image plane and an independent 2D sample point on the lens surface.

The user-facing parameters to set will be the lens radius (as a physical radius in scene units), which controls how shallow the focus range is (larger lens = shallower focus range), and the distance at which you want objects to be in focus.

To generate eye rays into the scene, you can calculate the position and direction of rays leaving the lens surface; in this model there's no need to explicitly simulate the image plane and the refraction through the lens.  Basically, think of the lens as centered at the camera position and oriented to face the camera direction.

Based on the image location, construct a ray from the camera position (lens center) into the scene, just as you would in the pinhole model; then find its intersection with the focal plane. That's where all the rays from that image location should converge.  Now you can offset the starting point of the ray to a randomly chosen point on the lens, and set its direction to be toward the convergence point.

You can generalize this a bit by allowing the focal plane to be something other than a plane, or the lens to be something other than a circular disc, and following the same process. That can produce some interesting if not-quite-physical effects.  It's also possible to go beyond this simple model with a more physically realistic simulation of a camera's lens elements—but that's beyond my expertise.


Why are quads used in filmmaking and triangle in gaming?

For 3D modeling, the usual reason to prefer quads is that subdivision surface algorithms work better with them—if your mesh is getting subdivided, triangles can cause problems in the curvature of the resulting surface. For an example, take a look at these two boxes:

The left one is all quads; the right one has the same overall shape, but one corner is made of triangles instead of quads.

Now see what happens when the boxes get subdivided:

Different, yeah? Note the way the edge loops have changed from being roughly equidistant on the left box to a more complicated, pinched-and-stretched arrangement on the right. Now let’s turn off the wireframe and see how it’s getting lit.

See the weird pinching in the highlight on the right box? That’s caused by the messy subdivision. This particular one is a pretty benign case, but you can get way more messed-up-looking results with more complex meshes with higher subdivision levels (like the ones you’d usually use in film).

All of this still applies when making game assets, if you’re planning to subdivide them, but the key difference is that the subdivision happens ahead of time—while you’re still in quad-land—and then the final subdivided result gets turned into triangles because that’s what the graphics hardware speaks (because, as mentioned in the comments above, it makes the math easier).


How do I make sure my cosine-weighted sampling works fine?

The code you've shown here looks right to me. In order to verify the implementations, you could try writing some code that just integrates a few test functions over the hemisphere using both methods and verify that you get the same result (within some sampling error). Also if you haven't already, verify that the outputs from sample_unit_square and sample_unit_disc are doing what they're supposed to.


Worsening sampling quality of raytracing with increasing distance from world origin

Problem was that perspective matrix had very short Z_NEAR and Z_FAR values (0.01f and 200.0f). By changing it to much higher range (10.0f, 20000.0f), problem with jaggies and artifacts disappears.


Why does my code to plot a cone actually generate a roof?

You need to change the call to sqrt from:

np.sqrt(xx**2, yy**2)

to:

np.sqrt(xx**2 + yy**2)

Otherwise you're passing an array of values to the function and will be returned an array of values for the answer.

See here for details:

  >>> np.sqrt([1,4,9])

  array([ 1.,  2.,  3.])


Bresenham's Algorithm not drawing horizontal lines

e = (float)(delta_x / delta_y) - 1.0;

e += (float)delta_x / delta_y;

Either of these lines will cause a divide-by-zero error when delta_y is zero, which is when the line is horizontal (or zero-length). You need to check for this condition and have some special-case code.

As an aside, I don't know why people still teach Bresenham's algorithm to new CG students. It's a good example of an algorithm but it's not very useful knowledge in itself: you're unlikely to need to know it unless it's for a job interview.