Why do most photorealistic renderers have similar material test scenes?

While teapot, spheres and trusty Suzanne are not per se bad test scenes for materials, here are some things to consider. You can apply them to said examples to forge your own opinion.

First and foremost, the viewer needs to be able to examine the behavior of the BRDF. Since it is dependent on both the direction of incident light and the direction from surface to viewer, you need to have as many combinations as possible in your test scene. A spheroid nicely covers all possible viewer angles.

Notice that all three scenes feature some sharper edges or creases too. Differences in reflective behavior across a smooth surface (like a sphere) may be hard to conceive, so these edges put emphasis on this.

Since the observed brightness and color is a combination of the material (i.e. the BRDF) and the lighting environment, it may be hard to tell one from another. The Vray test scene takes care of this. The 100% white 'core' and the 25% black floor help you get an idea of what the lighting in the scene is like.

Blender and Vray both feature a grid that helps you estimate the total size of the scene. This helps with evaluating grain or surface texture.

Here is a side-by-side comparison of a cube, notched sphere and Suzanne for a glossy test material with a slight normal map (rendered in Blender with Cycles, lit with the Dutch Skies 360 - Free 002 HDR):

The cube does a pretty good job on depicting the normal map. The sphere, as expected, is reasonably good for normals and gloss. Suzanne, arguably, makes both a bit harder to perceive than the spheroid.


Why are Homogeneous Coordinates used in Computer Graphics?

They simplify and unify the mathematics used in graphics:

They allow you to represent translations with matrices.
They allow you to represent the division by depth in perspective projections.

The first one is related to affine geometry.
The second one is related to projective geometry.


It sounds like you want to know how to ray trace a bilinear surface patch.

A quick search turned up this page by Ramsey, Potter & Hansen, which includes a paper and, probably better still, source code!

However, if you just want a "cheap and cheerful" hack approach, you could try replacing the quad with 4 triangles. That is, by a new vertex at the 'centre' (i.e. computing the average of the corners) and joining each edge to that new vertex. This is generally produces a better looking, and order-independent result than just using a pair of triangles.


Solving a set of Geometry Constraints - What's that Method called?

Its called a geometric constraints solver (a good primer on subject). You can find a open source solver as part of Open Cascade but its a bit convoluted to get going. A simpler solution for just solving, but also 3D solver capable, is geosolver.

Making your own (algebraic solution being easiest to write) is also not that hard, just a bit of work to make it automatically write the equations (and be efficient at solving). Take the following drawing as a example (made and solved with Mathematica):

You can simply conclude that its a sum of vectors that has to loop back at the origin thus:

$$
7\begin{pmatrix}\sin(15^\circ)\\ \cos(15^\circ)\end{pmatrix}+5\begin{pmatrix}\sin(120^\circ)\\ \cos(120^\circ)\end{pmatrix}+2\begin{pmatrix}\sin(195^\circ)\\ \cos(195^\circ)\end{pmatrix}+4\begin{pmatrix}\sin(\delta)\\ \cos(\delta)\end{pmatrix}+x\begin{pmatrix}\sin(-90^\circ)\\ \cos(-90^\circ)\end{pmatrix}=0
$$

And solve the 2 unknowns from 2 equations, either algebraically or by using a gradient descent solver of some kind.


Should you measure mesh complexity in triangles or vertices?

Since you're talking about "vertex shaders" and "fragments", I assume that your question is to be interpreted in the context of real-time rendering using the graphics pipeline on a modern GPU through some modern graphics API. Even if we narrow down the question like that, I don't think a general answer can be given to what you're asking. It depends a lot on what kind of meshes/scenes/application you're dealing with. In general, it's not even clear what "number of vertices" or "number of triangles" should mean exactly. There isn't really the number of vertices and the number of triangles. If you're rendering a non-indexed triangle strip, then there's a direct relation between the number of vertices and the number of triangles. If you're rendering an indexed triangle list, then do we count the number of vertices that were referenced, the number of vertices that were shaded, or do we count the number of indices that were used to form triangles? If you're rendering patches with tessellation, then is the number of vertices the number of control points or is it the number of vertices produced by the tessellation? What if I have a geometry shader that eats a few points, and turns them into loads of overlapping triangles? What if my geometry shading streams the odd triangle to a stream output buffer but doesn't send anything to the rasterizer at all!? The point is, there's a great many different places in a graphics pipeline where something that one might call a "vertex" or a "triangle" could pop up…

In a typical triangle mesh that is supposed to represent the boundary surface of some physical object, there will be a direct relationship between the numbers of vertices and triangles. In such cases, either the number of vertices or the number of triangles is arguably as good a measure as the other when it comes to giving an estimate of mesh complexity. But then, what "complexity" are we talking about exactly? Complexity in terms of storage can probably be estimated from either number of vertices or triangles as explained above. Complexity in terms of rendering time is a whole different story. Depending on your application, it might be that neither the number of triangles nor the number of vertices is a good predictor for performance at all. It's actually trivial to write a program that makes even the latest and greatest graphics card sweat by rendering only a single triangle using no vertex buffer. Because your bottleneck is rasterization, fragment processing, or the framebuffer, for example. In such a case, you'd probably be more interested in the number of fragments produced, or the depth complexity…


How to remove internal geometry programmatically in all meshes in .OBJ file?

This would not be perfect but should probably work well enough for your use case - you could render the objects from lots of different angles with a different solid color for each mesh component (without lighting) then remove any meshes that didn't appear in the final image by counting the pixel colours.
If the objects aren't split up into separate meshes then do this by repeatedly merging groups of triangles which share a vertex into the same group, starting with one group per triangle or polygon.


How mesh geometry data (vertex coordinates) stored in glTF?

The min and max values provide an outer bounding box for all POSITION data within a given accessor.  Accessors typically contain multiple vertices, for example all of the vertices of a particular primitive, and then min: [x, y, z] and max: [x, y, z] will offer the bounding box for that primitive.

For your second question, let's look at the mesh structure from BoxTextured.gltf:

"meshes": [
    {
        "primitives": [
            {
                "attributes": {
                    "NORMAL": 1,
                    "POSITION": 2,
                    "TEXCOORD_0": 3
                },
                "indices": 0,
                "mode": 4,
                "material": 0
            }
        ],
        "name": "Mesh"
    }
],

In the above example, mode: 4 is an enum, where 4 means TRIANGLES.  The indices are stored in accessor 0, and the POSITION data is stored in accessor 2.  So, accessor 2 will contain a list of vertex positions, but each vertex is allowed to be referenced by multiple triangles.  The indices in accessor 0 declare the triangles by indexing into the list of vertices.

Here's a sample of the start of accessor 0 from this model:

0
1
2

3
2
1

4
5
6

7
6
5
...

And this is a sample of the contents from the start of accessor 2 in this model:

-0.50000    -0.50000     0.50000
 0.50000    -0.50000     0.50000
-0.50000     0.50000     0.50000
 0.50000     0.50000     0.50000
 0.50000     0.50000     0.50000
 0.50000    -0.50000     0.50000
 0.50000     0.50000    -0.50000
 0.50000    -0.50000    -0.50000
 ...

In this manner, the sample model builds a cube by winding triangles around vertices at the corners.

For a more graphical explanation of this, check out the glTF Overview Card.


Distributed parallel rendering in Gaming or VR rendering

You are essentially correct in assuming that it is technically feasible to distribute the rendering computation workload. It is, essentially, a computation workload like any other; possibly even better-suited for parallelization due to its very nature: many similarly structured units running the same code path (i.e. vertices, pixels).

What you are forgetting, however, is latency. Having all the render power in the world is useless if the image does not arrive to the HMD fast enough not to cause discomfort to the user. Here is an article that explains the importance of motion-to-photon latency:

http://www.chioka.in/what-is-motion-to-photon-latency/


Calculate a rotation around an arbitrary axis

Take a point $P$ and it's rotated point $P'$.

Find the plan that runs through the middle between them $C = \frac{P+P'}{2}$ and is perpendicular to the line connecting them.

Do this for all 3 of them and find the line of the planes' intersection. That will be the rotation axis.

If all mid-planes are coplanar then you can use the planes of the triangles themselves.

To get the angle you can take a $P$ and $P'$ again and project them onto the axis.

Take a point $A$ on the axis and the direction $v$ of the axis. The projected point is $P_p=A+\frac{v \cdot (P-A)}{v\cdot v} v$. And then the angle is $acos(\frac{dot(P-P_p, P'-P_p)}{|P-P_p|* |P'-P_p|})$


There are lots of ways to fake SSS with greater or lesser fidelity. A few recent-ish methods:

Screen-space blurring of object lighting (described in detail here)

Using “interior” ambient-occlusion maps generated from a normal-flipped version of the geometry to approximate local thickness, providing an easy way to fake light transmission (more information here, with an addendum here)

There’s also the assortment of techniques in the GPU Gems article PaulHK linked to. One thing I’d add is about the light-falloff-texture approach mentioned at the beginning of it, which Valve used for a couple of interesting effects in Source games in addition to fake SSS—see their $lightwarptexture documentation for some information about that.


Visualize the output of a Trowbridge-Reitz Half Vector Sampling Function

So for anyone who was interested - after studying the source material by Walter, I became convinced the error had to be in the rotation of the slopes back into "normal shading space".  PBRT does this in a one-liner: "CosPhi(wiStretched) * slope_x - SinPhi(wiStretched) * slope_y;" etc etc.  I had split it out into separate sinTheta, sinPhi, sinPhiX steps.  One of those steps wasn't correct so the rotation wasn't unstreching correctly.

So the correct visualization for a GGX normal distribution of half vectors is this:


What wavelengths(dominant) the primary RGB colors are supposed to be in sRGB color space in nanometers?

The primaries of sRGB are not pure wavelengths, they are color mixtures. Their true definition is in terms of CIE chromaticity coordinates. That said, for sRGB, the red primary is close-ish to 610nm, green is very roughly 555nm, and blue is close-ish to 465nm.
The Rec. 2020 RGB space, by the way, is actually defined using pure wavelengths for the primaries: R=630nm, G=532nm, and B=467nm.


An idea for an implementation of continuous collision detection of circles

I think you should determine the "next collision point" in your whole 16-ball-and-table-border system and adjust your simulation 'Delta t" depending on it. Rendering should be de-coupled from this anyway, or it will show irregular ball movements... .

Some further thoughts:

Collinear velocity vectors do not mean there is no collision - if v1 != v2 ...
In good pool games you can define the 'hit point', where the queue hits the ball. Then "curvy" ball trajectories are generated, etc. etc.
Your system should also consider friction, therefore: v->0 over time. This should also affect your collision calculations.


Why do animated CG characters always feel fake?

Because animators haven't fully bridged the uncanny valley.

There are a ton of little details you notice subconsciously that hasn't been fully mapped out.

Things like how muscles and tendons move under the surface, how clothing and hair behaves as you move around. Even how you adjust your stance as you move your arms around, prepare to move, etc.

Even with mocap it is difficult to capture and recreate all that. It is also not trivial to capture the movement of the fingers.


OpenGL: Purpose of transform matrix in Vertex shader where transform*vertex

The transform matrix will be a uniform which is easier to update and upload to the gpu (just a single glUniform call) than applying the transform to all vertices on the cpu and uploading the new vertices. Especially with large models.


How do you compute the winding number of a closed poly curve?

Winding number as in https://en.wikipedia.org/wiki/Winding_number?

Unless I'm mistaken, one way is to fire, say, a horizontal 'ray' out from your point towards, say, $+\infty$. Each time you cross an edge, $(V_n V_{n+1})$ that is going up (i.e.$V_n[y] <V_{n+1}[y]$) , increment a counter, and decrement if the edge is going down.

The special cases of horizontal edges and vertices that lie on the ray are left as a (not too difficult) exercise for the reader.

If you want to do it quickly/approximately with a graphics API, you can use stencil buffers.


How to correctly render visible spectrum with uniform intensity?

I guess that this is related to the sum of the three functions having a minimum there. See the sum plotted against wavelength

This is actually a wrong way to sum these functions: sRGB model specifies that to get luminosity one must weigh the RGB components with appropriate weights. Then luminosity, unsurprisingly, will become equal to the Y component of the XYZ representation.

  the problem with it is that the image seems to have a dip in intensity somewhere around 490 nm

I can actually see the dip not only near 490 nm, but also a smaller one near 570 nm.

Anyway, I think I've found the reason for this. First of all, let's plot the desaturated colors used in the rendering on the chromaticity diagram:

The dashed triangle is the sRGB gamut, and the cusp formed by the lower two lines of the green curve corresponds to the D65 white (gray) point. Notice that there are three "bumps" away from the white point, corresponding to violet, bluish-green and red. Or, that there is a large dip in saturation in the direction of 490 nm and a moderate decrease in the direction of around 570 nm. Let's characterize this graphically, plotting the norm of difference of the rendered points from white point:

This is exactly what I perceive visually in the image in terms of saturation and thus blending with the background.

Where does this strange shape come from, you may wonder. And the answer is, from the spectral locus in full XYZ space. See the following figure: the blue curve is the spectral locus, and the orange is its projection onto a plane normal to vector {1,1,1}.

So, the shape we got on the chromaticity diagram is something like a planar unfolding of the spectral locus.

As to realism of this rendering, I'm reasonably sure that if we try projecting real spectrum onto a gray non-fluorescent surface, we'll also see these two "strange" dips. And they will become less noticeable as you increase brightness of the spectrum relative to the brightness of the surface.


For a polygon to be convex the outside angle of the polygon has to be more than or equal to 180 degrees. Now at intersection of 2 lines the outermost angle has to be less than 180 degrees for the lines to intersect between the endpoints.

Now the answer to this question depends on how you define what is inside of the polygon. If you consider some a even odd filling rule then the answer is No such shape exists. But if you consider only the outermost curve to do filling then yes, its possible if you intersect at the endpoint of a line. But to me this sounds more like splitting hairs as that shape reduces to a simple polygon.


Prove radiance contributions of deeper paths fade away

If you are asking whether:

$$
\int fr(...) cos\theta_1Le_1 > \int fr(...)cos\theta_1 fr(...)cos\theta_2 Le_2
$$

(simplified terms from the render equation you posted above), I don't think you could prove this. Le_2 could just be much greater than Le_1.

For example you could have a set of mirrors which only hit a light source after two bounces. Then all the light would come from the second integral.

EDIT: Updating answer after your response.

If you are trying to prove that, ignoring the layout and lights of the scene, deeper paths contribute less to the overall equation, then you could try ignoring Le and assuming it is always 1. You are then trying to prove:

$$
\int_\Omega fr(...)cos\theta_1  > \int_{\Omega^2}fr(...)cos\theta_1fr(...)cos\theta_2
$$
or
$$
\int_{\Omega_1} fr(...)cos\theta_1 dw_1 > \int_{\Omega_1} fr(...)cos\theta_1\int_{\Omega_2} fr(...)cos\theta_2 dw_2 dw_1
$$

We also know that if the fr is energy conserving:

$$
0 \leq \int fr(...)  \leq 1
$$

And if 0 < cos < 1 we can say that:

$$
0 \leq \int fr(...)cos\theta \leq 1
$$

Substituting this knowlege into the equation into the inner integration of the right hand side doesnt give us much, as it could always be equal to 1 which would give us:

$$
\int_{\Omega_2} fr(...)cos\theta_2 dw_2  = 1
$$
$$
\int_{\Omega_1} fr(...)cos\theta_1 dw_1 > \int_{\Omega_1} fr(...)cos\theta_1 dw_1
$$

which disproves your idea. However if your bxdf is never perfectly energy conserving, eg:

$$
0 \leq \int fr(...)  < 1
$$

I think subsitituting this in does prove the above as we get something like:

$$
0 \leq n(w) < 1
$$
$$
\int_{\Omega_1} fr(...)cos\theta_1 dw_1 > \int_{\Omega_1} fr(...)cos\theta_1 n(w)dw_1
$$

As each term can only be less than the integral on the left.

Hopefully my math is okay here I might have made a mistake. You may need to clean it up and do some more formal proofs. I am sorry if this still doesnt help answer your question.


Materials for research on spherical projection

You need to use a cylindrical projection. So the key issue is how to prepare data for a cylindrical projection. Basically what you do is:

You project your data on a sphere, it may be that this process was already done for you.
And then you project it back to a cylinder

Both steps can be easily taken care of by your cartography software. Will the data distort... yes but since you project information onto a sphere from whatever source onto a sphere and to cylindrical coordinates it will just interpolate the distortion away. If anything your just going to have slight over resolution at poles.

Quite a lot of data is in fact in this format to begin with. But nearly all 3D applications can use projected textures and texture baking to do these steps for you. Also most 2D applications like Photoshop have tools for turning rectangular coordinates to polar coordinates which do the same thing

The real question is what your original image data is like?


A seam can be defined as a line along the mesh where the vertices are doubled each with different texture coordinates.

So that's what you should look for. The set of vertices where there is a twin in the same position with different texture coordinates.

Depending on how the obj file was built that will be simple if they didn't duplicate the position when a vertex became part of a seam. Or it's not as simple because you need to deduplicate positions again.


Sharing memory between compute shader and pixel shader

well, if anyone is stumbling across this and needs to know what solved it for me: We can bind a compute buffer to global shader memory if we just know what exactly we need to do

within the Compute shader the RWstructuredBuffer is declared as (without the "\" infront of float):

uniform RWStructuredBuffer<\float3> buffer : register(u1);

(not certain whether register(u1) is really needed

within the vert/frag shader the buffer is defined as:

uniform StructuredBuffer<\float3> buffer;

In a c# script before the first kernel dispatch (we only need to assign this buffer once), I assign the buffer via

buffer = new ComputeBuffer(size, sizeof(float)*3,ComputeBufferType.Default);
Shader.SetGlobalBuffer(Shader.PropertyToID("buffer"), buffer);

It took me ages to find this and it works as far as I can tell. The article I finally stumbled over that did something similar. I do not have to read the data back to cpu with buffer.GetData(container) and put it into the material manually.
If the compute shader changes the buffer data this will be reflected in the vert/frag shader immediately.


When and who introduced the idea of a scene graph?

Not 3D as such, but it seems in 1963 Ivan Sutherland's seminal "Sketchpad" (and wikipedia) had instancing. eg. See "Chapter VI RECURSIVE FUNCTIONS" of the 2nd reference.
IIRC somewhere there's a video of it in action - it made me annoyed as to how some of today's vector drawing tools don't seem to be able to do some of what sketchpad appeared to support.
Update: @Joojaa has kindly located a video   or this


Maybe you should try to find Virtual Tangent Plane like in Surface Normal In the Wild


What is the name of this moving color pixels effect, for animating a painting?

This effect is called Color Cycling (or Palette Shifting). Mark Ferrari has a great GDC talk where he demonstrates and explains the entire technique. You can view a bunch of his artwork based on this technique here.
The main idea is that each pixel is painted not with a single color, but as values of a look-up table to a palette of colors. The entire image then cycles through the palettes frame by frame. Assigning colors to each palette to create the illusion of movement requires a fair bit of artistry. Mark Ferrari does this in a software called Pro Motion, but this can be coded in a shader pretty easily.


Blender appears to have a .X importer.  It's a plugin that ships with Blender, but is disabled by default, so you have to enable it in User Preferences.  See: https://blender.stackexchange.com/a/8749/18161

Blender also has a glTF exporter.  For now that's a separate download, from here: glTF-Blender-Exporter


Am I missing something for ambient occlusion?

It's a bit hard to tell from your image, but it does look a bit faint. When debugging these kinds of things, it's always useful to strip down your scene as much as possible to remove any unnecessary complexity from the picture. In your case, try only creating a single diffuse sphere that touches the ground in one point. Give the ground and the sphere an albedo of 1. If ambient occlusion is implemented correctly, the point where the sphere touches the ground should have a pixel value of 0; the further away from the sphere you get, the closer the pixel values should go to 1 (not above - make sure nothing gets clamped when you output the image). Make sure to gamma correct your images.

The one thing that sticks out from your description is

  Rays reflect [...] in any direction from the hemisphere of possible directions when hitting the floor

That by itself is fine, but you need to make sure to multiply the rays by the Lambertian BRDF (i.e. dot(normal, ray)/Pi). Even better is to directly sample from a cosine hemisphere, in which case all factors cancel out. This blog post has all the important info in one place.


Here is the way I calculate the refraction vector according to Snell's law.

float cosine = dot(viewDir, worldNormal);
float sine = sqrt(1 - cosine * cosine);

float sine2 = (_IOR * sine);
float cosine2 = sqrt(1 - sine2 * sine2);

float3 x = -worldNormal;
float3 y = normalize(cross(cross(viewDir, worldNormal), worldNormal));
float3 refractedW = x * cosine2 + y * sine2;

This is integrated with the Physically based Refraction and it gives interesting results but I still don't know how to get rid of the high parallax scale issue which appear at grazing angles...

Any idea how to get rig of that ?


It turns out that the ring was happening from a negative divided by a negative, so adding nom = max(nom, 0.0); fixed the problem. The new highlight, amplified:


Handling Projective Aliasing In Shadow Mapping

Shadow mapping with acceptable quality is quite a journey. So you implemented first step - basic shadow map that encompass whole scene statically. This means that shadow map texel size in light view space to texel size of rendered scene in camera view space ratio is quite big, resulting in aliasing. To reduce this ratio closer to 1:1 there are techniques you mentioned like:

the easiest way is increasing size of your shadow map, to find the value with still optimal performance,
align shadow map with camera view frustum. Then shadow map encompass smaller part of the scene, so it reduces aliasing,
this can be improved to CSM, which does the same but you need to render scene for each of cascade with different center position of light view matrix based on center point of each slice of frustum.

Some of the basic techniques of smoothing the edge of shadow:

obviously PCF, but is it recommended to combine your PCF implementiation with built in hardware PCF (in Opengl - using sampler2DShadow). This would result in less loop iterations from your side and cheaper PCF,
blockiness of edges can be exchanged for noise using rotated Poisson Disk. Samples used to sample from shadow map are rotated randomly with each iteration of PCF loop.

You can also research the edge smoothing techniques like:

rendering shadow map to MSAA applied framebuffer,
further enhancing PCF by interpolation,
increasing number of samples with temporal blend.

PCF basic technique suffers from shadow acne which can be fixed by applying some offset in shadow test. Calculating the size of this offset is also area of research.

Other more advanced shadow map methods include:

Exponential Shadow Maps and Variance Shadow Maps - in most cases do not suffer from shadow acne and offer smooth edges without blockiness, but in cost of some light leaking situations,
Exponential Variance Shadow Maps - probably most advanced technique of 2D depth map shadow mapping, combines great smoothness of VSM and resolves light leaks, in a cost of memory usage doubling,
Percentage Closer Soft Shadows - basic technique of enhancing traditional shadow maps with varying penumbra size,
Multiview Shadow Maps - multiple shadow maps of some technique, arranged on area light, for computing realistic soft shadows with varying penumbra size,
Deep Shadow Maps - for enhancing volumetric scattering with shadows,
Real time ray traced shadows - probably the future.

You didn't fail with PCF, it looks like that:)

I wish there was a way to apply gaussian blur on shadow map in camera view space, but it doesn't work as expected.


Image regeneration with nearest neighbor search

I would have done it with a script and NETPBM like so:

1) reducing Image1 (the tiles) to size X by X -> Image1R

2) recording the mapping from pixel values of Image1R to coordinates (no need to worry about duplicates as an exact RGB match of two or more averaged pixels is unlikely)

3) remapping Image2 using Image1R as the color map -> Image2M

4) reading pixels of Image2M, consulting the map, and picking the tile corresponding to the coordinates

5) concatenating the selected tiles across and down

There is a separate tool for all the above image manipulations in NETPBM.

Any scripting language that allows spawning other processes and supports associative arrays, would do.


Importance Sampling in Unreal Engine Formula Derivation (Irradiance term)

I just read notes on moving frostbite to pbr and I found the derivation of the method above. So I will just show the derivation here and quote some of the explanation.

  One can notice an extra〈n·l〉in the LD term as well as a different
  weighting 1/(∑Ni〈n·l〉). These empirical terms have been introduce by
  Karis to allows to improve the reconstructed lighting integral which
  suffers from coarse hypothesis of separability of this integral.
  There is no mathematical derivation for these terms, goal was to have
  an exact match with a constant L(l).

So it turns out the pdf is weighted on the DFG term. As for dot(N,l), the term is introduced to minimize the error that is caused by split sum approximation. But I am actually still wondering what is the intuition on that empirical term.


How to use Monotone cubic interpolation in 3D?

Ok, monotonic interpolation depends on what you are monotonic about. For a simple 1D function interpolation monotonicity is easy to define. But for a 2D and 3D dataset its not so self evident what the situation would be.

First you could interpolate along a independent variable t in which case your monotonicity is most probably in relation to t. This is the same as interpolating each direction separately.
Second you could interpolate along one of the axes, so that your interpolant becomes a function of position on that axis. Reducing a 2D case into a 1D case and a 3D case to two separate 2D cases.
Third you could interpolate on some other spatial variable.

So you see the question is a bit open ended and is hard to say for sure.


$$
\def\mvec#1{\begin{pmatrix}#1\end{pmatrix}}
\def\ivec#1{\left(\begin{smallmatrix}#1\end{smallmatrix}\right)}
\def\mmat#1{\begin{bmatrix}#1\end{bmatrix}}
\def\vec#1{\mathrm{\mathbf{#1}}}
\def\mat#1{\mathrm{\mathbf{#1}}}
$$
If you have the screen-space x and y coordinates as well as the depth of your fragment, you can directly compute the coordinates of the corresponding point in world-space. To do so, first compute the clip-space coordinates of your fragment
\begin{equation}
 \vec p'' = \mvec{
  2\cdot\frac{x}{r_x} - 1 \\
  2\cdot\frac{y}{r_y} - 1 \\
  2 \cdot z - 1 \\
  1
 }
\end{equation}
where $x$ and $y$ are the screen-space (pixel) coordinates of the fragment, $z$ is the fragment depth, and $r_x$ and $r_y$ are the width and height (resolution) of the viewport. The above computation is assuming that you're using OpenGL with the default depth range. If you're using Direct3D, use $z$ directly without applying a scaling and offset. If you're using a depth range different from the default, apply whatever is the the appropriate inverse transformation in your case to get from screen-space to clip-space $z$.

Once you have the clip-space coordinates of your fragment, you can simply unproject the fragment by transforming with the inverse view-projection matrix:
\begin{equation}
 \vec p' = \mat M^{-1} \vec p''
\end{equation}
where $\mat M$ would be your view-projection matrix. (Note: you will most likely want to pass the inverse view-projection matrix to the shader as a uniform as inverting the view-projection matrix in the shader for every fragment would most likely introduce quite some unnecessary overhead.)

The final step is then to homogenize to get the world-space coordinates of the point
\begin{equation}
 \vec p = \frac{1}{p'_w} \cdot \mvec{
  p'_x \\
  p'_y \\
  p'_z
 }.
\end{equation}

By the way: It would seem to me that rendering this "infinite plane" should be possible using just a single triangle with all corners at infinity in different directions instead of four triangles. Since the boundary of your polygon is pushed to infinity, it shouldn't matter how many edges it has…


Difference between BVH and Octree/K-d trees

Well I researched quite a lot after that and this paper helped a lot.

"Space Subdivision algorithms" by Macdonald 1988.

So just summing what I understood. Some of them are obvious reasons but after reading the paper make much more sense and I'll try to do the same here.

1) In BVH we are subdividing objects into smaller pieces. So for example a model of dragon consisting of thousands of triangles, We first start with a Bounding Volume covering the whole dragon. Then we subdivide again and again until our conditions are met. In BVHs, the leafnodes usually have 1 object in contrast to octrees which can have many. However this isn't clear whether it's referring to an object (sphere) or a single triangle since spheres can contain hundreds of triangles if triangulated. Here is a picture explaining BVH.

In contrast, Octrees/K-d trees and other space subdivision, divide the space recursively. A simple picture of an octree given in the paper.

2) The above pictures show another difference. That is, in Space subdivisions we have disjoint sets of space regions or voxels or cells whatever you wanna call them. In contrast, BVHs have disjoint sets of objects.

This means in octrees one object may be assigned to more than 1 voxel. And thus we may have to check for that object again and again though this can be avoided using techniques like mailboxing etc.  In BVHs, however, this isn't the case as an object may be inside only 1 bounding volume at a time, thus it may be checked for intersection only once which is when the ray intersects the bounding volume it's currently in.

3) The other difference the paper mentions is that in Space subdivision traversal algorithms usually calculate the exit point of the ray from the current intersected child node and calculate a new point just past this exit point and from this information the next child node is found. The benefit of this is that traversal may stop as soon as it hits an object.

On the other hand, BVHs can't do this since in BVHs we may have object overlapping in nodes. In this case we will have to maintain a list of the intersected objects and find which is the closest one. A good example of this is given in "Raytracing Complex Scenes by Kajiya"

As we can see although the ray first intersects the upper BV, the closest object hit is actually inside the lower BV which the upper one has no information of unlike space subdivision structures.

And now on to the part of how I implemented the structure, it seems what Kajiya suggested was to use the octree scheme to define a traversing algorithm for the BVH. In the end when I recalculated the bounding volumes of each cells, this relaxed many voxels boundaries causing several to overlap. After this operation the structure didn't resemble "Disjoint sets of spaces" rather it resembled "disjoint sets of objects" hence it's essentially a BVH.

Do note that this operation may sound like somewhat an efficient optimization to the space subdivision structure, that is relaxing the bounds of voxels to avoid having all the voxels keep track of the object that overlaps them. This way we assign 1 object to 1 voxel and relax it bounds.

But doing so we lose the traversing efficiency of space subdivision structure where we could have stopped after intersecting only 1 object. Other pros may also have been lost. That's what I have understood so far.

If anybody would like to add some more feel free to do so.


Why different result when change input order in GL_LINES?

The difference in which pixels are covered depending on the vertex order is related to rasterization rules. They're the rules that the GPU hardware uses to determine exactly which pixels are covered by a primitive.

Rasterization rules are a bit subtle. One of their goals is to ensure that whenever you draw multiple primitives that are connected water-tightly, the rasterization never produces any cracks between them, nor are any pixels covered twice (that's important for blending). To achieve that, the rules have some very specific edge and corner cases. Literally...they're cases having to do with primitive edges and corners. :)

In the case of lines, it works as follows. There's a diamond-shaped region around each pixel center, as shown in this snippet of the diagram from the MSDN article linked above.

The rule is that a pixel is covered if the line exits the diamond, when tracing from the start to the end of the line. Note that a pixel doesn't get covered if the line enters, but doesn't exit, the diamond. This ensures that if you have two line segments connected end to end, that the pixel at their shared endpoint only belongs to one of the segments and thus doesn't get rasterized twice.

You can also see in the above diagram how changing the order of the vertices can affect which pixels are covered (which doesn't happen with triangles, BTW). The diagram shows two line segments that are identical except for swapping the endpoint order, and you can see it makes a difference which pixel gets covered.

Your line segment is somewhat analogous to this. The left end, at (0.5, 0.999) is inside the diamond of the (0, 0) pixel, so it gets covered when it's the first vertex (the line starts inside the diamond, then exits it) and not when it's the second vertex (the line enters the diamond, and ends inside it, so it never exits). Actually, the vertices are snapped to fixed-point with 8 subpixel bits before rasterization, so this one ends up rounded to (0.5, 1.0), which is exactly on the top corner of the diamond. Depending on the rasterization rules, this might or might not be considered inside the diamond; it seems as if on your GPU it's considered inside, but this can vary between implementations, as the GL spec doesn't nail down the rules entirely.


How can you interpolate over an array of say 5 colors?

Say you have colors

$[c_1, c_2, c_3, c_4, c_5]$

And $t$ values at which each color should be purely displayed.

$[t_1, t_2, t_3, t_4, t_5]$

Now your problem is given $t$ which color do I have to display?

Find the t-values $t_i$ and $t_{(i+1)}$ between which $t$ lies.
Calculate a 0 to 1 ratio where $t$ lies between them

$ param = (t - t_i) / (t_{(i+1)} - t_i) $

Interpolate the color via

$ c = param * c_i + (1-param) * c_{(i+1)}$

If I haven't messed up something in my mind, this should work for any piecwise linear function, and not only if your $t_i$ values lie between 0 and 1.


What is Illustrator's vector rasterization process?

As as far as I can tell Illustrator has 2 or 3 different rasterizer. The onscreen preview is also prone to the same gap artifacting as your show though its tuned to minimize the effect. Your post seems to imply that your interested in the "art optimized" output.

Image 1: The different render modes of illustrator. Art optimized on left and hinted on right. Note that hinted shows slight background through. I dont have the new CC version so i can not show the third.

How does art optimized work then

Art optimized render mode is simply a un-antialiased image rendered at higher resolution, that has then been subjected to a box filtering. This is called super sampling which is costly. A box filter is simply the mean value of the samples in the box (pixel). This is indeed a similar technology as what the multi pixel filter used in your blender render uses. You can do the same thing manually and get same results.

In fact the render you get form illustrator is not really good. Its rendered in nonlinear space and not corrected for and you can get better results by swapping to higher order filters and computing in linear color space. All multi sample methods exhibit the same correct rendering (given enough samples even mathematically correct).

Image 2: Multi pixel filtering, on left illustrator native 'Art Optimized'. On right same image at 16x16 pixels with multi lobed Lanczos filter in linear space. Make sure your zoom is 1:1.

The problem

You have in fact come at terms with the real problem of current generation of onscreen renderers. They try to avoid multi sampling in favor for coverage calculation. But it simply can not work perfectly! Simply because alpha blend is:

$$c_0= c_1*\alpha + c_2*(1-\alpha)$$

When chained never reaches 0 when chained over semitransparent surfaces so background show trough. Also it does not know how the coverage is distributed so it will do it wrong. There's simply no way around this in a non-overlapping case like this. but if you allow the shapes overlap then the coverage calculation can be solved (See here).

Other questions You can get same result in any other engine by just simply multisampling a non-antialiased image. You can not get the effect in on screen renderings without doing the overlap trick because they optimize for speed. Does illustrator use 3D? New CC might depends on your interpretation of what 3D is. This issue has nothing to do with 3D just about signal reconstruction methods so its not applicable to the question really.

Related post how-do-i-accurately-compute-coverage-of-overlapping-analytical-curves


How to triangulate a simple 2d surface to generate a constrained surface trimesh

The fastest and easiest way to do this is using blender. Simply import the svg (supported out of the box) into blender. The svg will be imported as a "curve"-type geometry. Optionally do some clean up/refinement. The curves can then be converted to a mesh. The mesh defaults to a trimesh. Voila - svg to trimesh in a couple of seconds.

Mesh refinement is not very fine-grained, but there is some measure of control via the "beauty-fill", "subdivide" and "triangulate" as well as "merge vertices by distance" tools from within blender.

The IBO (vertices + triangle vertex indices) can then be extracted using a short python script:

import bpy
obdata = bpy.context.object.data # selected object
for v in obdata.vertices:
    fh.write('({}, {}),\n'.format(v.co.x, v.co.y))
for f in obdata.polygons:
    for v in f.vertices:
        print('{} '.format(v))


Volume Rendering (Path Tracing & Real-Time)

Ray marching is a ray tracing method where you take multiple steps along a ray to find intersection with geometry or to perform integration of in-scattered light from participating media (fog, clouds, water, etc.) along the ray. Signed distance fields doesn't really help you with participating media rendering since it's a method of finding the ray intersection instead of helping with volume integration.

So when you perform ray marching along the ray for volume integration, for each step you need to calculate incoming light to that point and evaluate the "phase function" to calculate how much of that light scatters towards the eye. This phase function can be simple isotropic function (i.e. constant $\frac{\rho}{4\pi}$) or more complex anisotropic function such as Mie phase function shown below, depending on the modeled participating media.

Additionally you need to take the extinction coefficient of the media into account, which is spatially varying for heterogeneous media, for calculating transmittance and to properly "attenuate" the in-scattered light as it's being blocked by media between eye and the point (Beer-Lambert law). This approach assumes homogeneous media between steps and for proper physically based integration you could take a look at SIGGRAPH 2015 presentation "Physically-based & Unified Volumetric Rendering in Frostbite" by Sebastien Hillaire.

To know how much light reaches a point in space for each of those ray steps, you can use path tracing. This can get quite expensive though since light can take random paths in the media until it reaches the point. This is known as "multiple scattering", which is analogous to indirect illumination. When performing the integration over the sphere for each step along the ray, you can use Woodcock tracking to get unbiased results for heterogeneous media. Also if you use Mie phase function you should use importance sampling as the phase function has very strong forward peak.


How does GLSL code get “loaded” and “compiled”?

No, g++/gcc are not involved.

In OpenGL (prior to Vulkan), GLSL code is submitted to the driver as source code strings via a glShaderSource call. The driver is fully responsible for compilation, i.e. the driver must provide a built-in GLSL compiler for that GPU architecture. This would typically happen on startup of an app, or during level loading for a game.

With Vulkan, SPIR-V was introduced, which is a device-independent bytecode for shaders (vaguely based on LLVM IR). Now, GLSL code can be pre-compiled to SPIR-V offline using the shaderc toolchain. The developers would do this ahead of time and ship the SPIR-V bytecode with their app. Then at runtime the SPIR-V gets submitted to the Vulkan driver and compiled the rest of the way to GPU machine code.

It's also possible to compile other languages to SPIR-V; for instance, Microsoft's dxc compiler for HLSL can optionally output SPIR-V, so you can use this to write Vulkan shaders in HLSL. Also, SPIR-V support has been backported to OpenGL 4.6, so you can now also load precompiled SPIR-V shaders in OpenGL.


How can I raytrace a scene that does not fit into memory?

If the scene does not entirely fit into memory, you are entering the field of out-of-core rendering. There are essentially two approaches here: a) Generate your scene on-demand b) Load your scene on-demand

The former approach aligns well with most animation workflows, where models are heavily subdivided using e.g. Catmull-Clark and can become very memory-intensive, but the base meshes themselves easily fit into memory. Pixar have a few papers about this (e.g. Ray Differentials and Multiresolution Geometry Caching
for Distribution Ray Tracing in Complex Scenes), but the gist of it is that models are only subdivided when they are hit by a ray, and only subdivided as much as is reasonable for such a ray (e.g. diffuse interreflection need less accuracy than mirror reflections). The rest is handled by a geometry cache, which keeps the subdivided models in memory and hopefully makes the process efficient by a good eviction strategy.

As long as all your base meshes comfortably fit into memory, you can easily go out-of-core and render meshes at subdivision levels that would never fit into memory. The geometry cache also scales nicely with the amount of memory you have, allowing you to weigh RAM vs. render times. This was also used in Cars I believe.

The second approach is more general and does not rely on heavy use of subdivision. Instead, it relies on the fact that your scene was most likely made by an artist and already comes partitioned into reasonably small objects that fit into memory individually. The idea is then to keep two hierarchies (kD-tree or bounding volume hierarchy): A top-level hierarchy that only stores bounding boxes of the objects in your scene, and a low-level hierarchy that stores the actual geometry. There is one such low-level hierarchy for each object.

In this approach, you ideally already store a bounding box along with each object on disk. As the scene is loaded, you only build the top-level hierarchy initially, meaning you only have to look at the bounding boxes and not the geometry. You then start tracing rays and traverse them through the hierarchy. Whenever a ray hits a leaf node in the top-level hierarchy (i.e. it hits the bounding box of an object), that object is loaded into memory and its low-level hierarchy is built. The ray then continues down into tracing that object. Combined with an object cache that keeps as much of the low-level hierarchy in memory as possible, this can perform reasonably well.

The first benefit of such an approach is that objects that are never hit are never loaded, meaning that it automatically adapts to the visibility in your scene. The second benefit is that if you are tracing many rays, you don't have to load an object immediately as it is hit by a ray; instead, you can hold that ray and wait until enough rays have hit that object, amortizing the load over multiple ray hits.

You can also combine this approach with a ray sorting algorithm such as Sorted Deferred Shading for Production Path Tracing to avoid thrashing due to incoherent rays. The mentioned paper describes the architecture of Disney's Hyperion renderer, used for Big Hero 6 I believe, so it most likely can handle scenes at production scale.


Is back-face culling considered a technique/optimization of ray tracing?

I would exclude (A) for the following reasons:

In any modern ray-tracer, the acceleration structure (BVH, k-d tree, etc) holding your geometry is the primary source of culling. The final list of primitives to trace for a given ray may include non-visible triangles, but ideally there should be very few with good acceleration structures.
Ray tracers support many more primitives than just triangles. (How do you back-face cull a sphere?)

The following problems with back-face culling in general are also more relevant to ray tracing:

Transparent materials (such as glass) will make certain objects render incorrectly with back-face culling applied.
Non-closed meshes (like plants) need to render the back-facing triangles.


Does programs like After effects and Unity engine use graphics library to render custom guis?

So:

All GUI are somehow graphics. They don't have to be accelerated by the GPU but these days most are. In the old times, they could be rendered on the CPU (X11 times).
You need to be careful between making a distinction between the frameworks/libraries that are potentially being used for building up UIs and what these libraries use to render the graphics making up these UIs.

A lot of applications like Maya for instance use libraries for creating their UIs (Maya uses Qt so as most products developed by the Foundry: Nuke, Marri, etc.) which is why a lot of these applications look the same ... (sadly).

Qt and most UI library/framework use a graphics API to accelerate the rendering of widgets.

Some applications like Unity do not use "commercial" libraries like Qt. They develop their own, but in the end they also use the GPU to render their GUIs.


What is the difference between glossy and specular reflection?

The charts you show aren't showing two different phenomena - "glossy reflection" and "specular reflection" - they're showing two parameters of specular reflection. One is the specularity or specular colour and gives the amount or brightness of the specular reflection. The other is the glossiness or roughness and shows how sharp the specular reflection is.

Put another way, if you picture the curve of the specular reflection over changing angle, the specularity (the vertical axis on your charts) gives the height of the peak - the maximum brightness - and the glossiness gives the width of the peak - the size of the highlight. The highlight gets narrower with increasing glossiness, whereas roughness is the inverse of this.

BRDF is more general: it's a function that tells you how much light is reflected in a given direction (view direction) from a given direction (light direction). Usually (for most materials) we approximate a BRDF using a diffuse term and a specular term.


How can I verify if a selection area is over a shape in SVG?

In pseudocode

For each rectangle:
    If the rectangle overlaps the selection area, highlight the rectangle

This is all that is required for a correctly working approach, but there are a number of other things you might need to take into account if this approach performs too slowly. I recommend implementing the simple approach, and once it works correctly you can make gradual improvements if it is not sufficiently smooth and responsive (this may not be necessary unless you have a large number of rectangles or restrictive hardware/software).

Determining whether a rectangle overlaps the selection rectangle

As you point out, using the point in polygon algorithm is impractical for this purpose - each rectangle is made up of an infinite number of points. So we need a completely different approach.

One way to check if a candidate rectangle overlaps the selection rectangle is to check for the following two cases:

Check if any edge of the candidate rectangle intersects an edge of the selection rectangle.
Check if either the candidate rectangle or the selection rectangle is completely contained within the other.

If either of these are true, there is some overlap. If neither of these are true, there is no overlap.

The complete containment check is simplified by the fact that the selection rectangle is never rotated - always remaining aligned to the x and y axes:

To check if a candidate rectangle is completely contained within the selection rectangle, check that all 4 of its corners are between the x boundaries of the selection rectangle, and between the y boundaries of the selection rectangle.
To check if a candidate rectangle completely contains the selection rectangle, check that it has one corner outside the bounds of the selection rectangle in each of the 4 orthogonal directions (left, right, up, and down). If so, it either contains the selection rectangle completely, or one or more of the edges intersect (but edge intersection was checked for in the previous stage, so we're only checking for complete containment if we already know there is no edge intersection).

Line segment intersection

To check whether any of the edges of a candidate rectangle intersects with any of the edges of the selection rectangle, you can either use a general line segment intersection algorithm and apply it to each pair of candidate edge and selection edge, or you can take advantage of the fact that the selection rectangle is never rotated,as follows:

Parallel case

If the candidate rectangle has edges parallel to the x and y axes (that is, it is rotated by 0, 90, 180 or 270 degrees), check each of its edges:

If the edge is vertical, check if its x position is between the x coordinates of the selection rectangle, and if one of the following holds then there is an intersection:

The y coordinate of one of its end points is between the y coordinates of the selection rectangle AND the y coordinate of the other end point is not.
The end points are both outside the y coordinates of the selection rectangle but in opposite directions.

Similarly, if the edge is horizontal, check if its y position is between the y coordinates of the selection rectangle, and if one of the following holds then there is an intersection:

The x coordinate of one of its end points is between the x coordinates of the selection rectangle AND the x coordinate of the other end point is not.
The end points are both outside the x coordinates of the selection rectangle but in opposite directions.

Non-parallel case

Check each of the edges of the candidate rectangle as follows:

First find the points of intersection of the infinite line on which the candidate edge lies, with the infinite lines forming the boundaries of the selection rectangle.

If either of the intersections with the y boundaries of the selection rectangle has an x coordinate between the x boundaries of the selection rectangle, then there is an intersection of one of the horizontal line segment edges of the selection rectangle with the candidate infinite line (so potentially with the candidate line segment too).

Similarly, if either of the intersections with the x boundaries of the selection rectangle has a y coordinate between the y boundaries of the selection rectangle, then there is an intersection of one of the vertical line segment edges of the selection rectangle with the candidate infinite line (so potentially with the candidate line segment too).

To then check if there is an intersection with the candidate line segment, check if one of the following is true:

An intersection with the y boundary of the selection rectangle is between the x boundaries of the selection rectangle AND between the y coordinates of the candidate line segment.
An intersection with the x boundary of the selection rectangle is between the y boundaries of the selection rectangle AND between the x coordinates of the candidate line segment.

Examples of refinements if speed is unsatisfactory

Acceleration data structures such as a grid or space partitioning hierarchy so that not every rectangle needs to be checked.
Only checking the changed parts of the selection (usually one corner of the selection stays fixed while the opposite corner moves).
Tracking which rectangles are already selected to reduce the checking required each time the selection changes:

No need to check unselected rectangles if the selection has purely shrunk.
No need to check already selected rectangles if the selection has purely grown.
It will only be necessary to check both selected and unselected rectangles if the selection has grown in one direction and shrunk in the other.


How to do heterogeneous volume absorption using path tracing?

Your equation for the absorption-only model is correct: You multiply the luminance $L_x$ at the surface by the transmittance of the participating media between eye and the surface. For homogeneous media you can calculate this analytically using Beer-Lambert law:
$$L_y=L_xe^{-\mu|x-y|}$$
For heterogeneous medium you can calculate the transmittance by ray march through the medium using for example Woodcock tracking explained in the paper you referred.


Traditional rendering solutions do not do account for secondary light bounces (called indirect light). Even with strategically placed fill lights you still have areas where none of the direct light hits.

Ambient light tries to solve this problem by shining by a constant amount in all directions. In practice this means that light position or surface normal has no meaning, one just adds some of the shaders color multiplied by ambient light color to the shading result.

Ambient light has a tendency to look artificial when overused. But the opposite problem is that surfaces look like they are on outer space. Ambient light also makes the difference between dark materials and light materials more apparent.

Image 1: Image without ambient light (left) looks like it was shot in space. Image with ambient (right) looks more natural, although possibly a bit flat if overused.

The real problem is that ambient light does not really exist. Even if youd argue that its useful model, it certainly is not uniform. Its just a quick fix. Therefore all kinds of tricks, like ambient occlusion, have been proposed to enhance the quality of ambient light.


Generally a non-rigid transformation is motion that doesn't preserve the shape of objects. If you look at a typical transformation matrix, rigid transformations would include translation, rotation, and reflection. Things like shear, uniform or non-uniform scale, and perspective would be non-rigid. Shear and scale would fall into the category of affine transformations, and perspective is of course perspective transformation. So there's a hierarchy of types of transformations:

Rigid transforms - translation, reflection, and rotation. These perserve the distances between every pair of points on objects.
Affine transforms - rigid + shear and (possibly non-uniform) scale
Perspective transforms - Affine plus perspective transforms (think of a rectangle rotated around the upward pointing axis - the part closer to the camera/viewer may be larger than the pre-transformed object, and the part farther away may be smaller)

I haven't done a lot with optical flow, but I would guess that as you figure out the path of motion of various points on an object, you would be able to classify them into one of the above categories. If all the points in an object move in tandem, then you probably have rotation and/or translation only (rigid transforms), but if they don't then you may have an affine or perspective transform between the points.


Fastest Simulation Of Mass Spring Systems

Time integration with implicit Euler is unconditionally stable. This means you can choose arbitrary large time steps without worrying that your solution "explodes". In contrast to explicit Euler, it requires you to solve a linear equation system at least once, which is generally much slower than simple matrix-vector multiplication. If the problem is non-linear, you also need to iterate until your error is minimized. This obviously takes even longer.
However, stability issues can be a pain to deal with and all explicit methods have limitations regarding their stability. Solving stability problems might force you to reduce your time step size drastically and to calculate so many intermediate steps that the implicit version gets more effective. An additional factor is the size of the system you want to simulate. If I remember correctly, the finer your mesh (smaller cells), the smaller are the timesteps you need to maintain stability.

I'd like to know which algorithm uses the least amount of operations per frame / vertex.

I think there is no general answer to the question because it depends on many factors like the mesh size, target FPS and the maximum local velocity that might occur. You need to evaluate all relevant factors and pick a method based on your exact problem.
In case you want raw speed and do not encounter any stability problems, the explicit Euler method should be one of the fastest methods but it usually blows up rather quickly.
Additional info
A time-integration method that is quite popular in real-time engines is Runge-Kutta 4. It is also an explicit method and doesn't require you to solve a linear system. It has a good balance of stability and speed, so you might have a look at it. I have never simulated cloth myself, nor have I programmed an RK4, so I can't tell if it is a good choice here.


Keep constant number of visible circles in 3D animation

You don't seem to be setting circle.position.z


How are the colors for each pixel from a Ray Tracer stored in an Image?

It doesn't matter. All that matters is that you can store information per pixel.

In most cases this means allocating a x*y array of ints (for 3 byte-sized channels of RGB) to store the pixel values and then feed it into your image compression library of choice to save it out to disk.


Can you generate your own HDRI maps without using photos?

Yes, it is possible to do this. If you use a tool like Photoshop in 32-bit per channel mode, any values less than 0 or greater than 1 should be usable as a high dynamic range values. I don't know that I agree with your contention that "people don't seem to do this". Given that the tools exist, they must be supporting some workflow.


Do operating programs such as Microsoft Office or Windows Explorer use OpenGL to draw?

In general, no. Most applications, unless they are games, or 3D applications like CAD or animation software, will use OS APIs to do their drawing. On Windows, this can be things like GDI or GDI+. On macOS you would usually use CoreGraphics or one of its derivatives.

At a lower level than the application level, the operating system talks to the graphics driver to tell  it to draw things on the screen. They may do so using OpenGL or a proprietary API that's not usually exposed directly to applications. (For example on older macOS versions, the Window Server used OpenGL directly to do its compositing. I believe it uses Metal now.) Additionally, some drawing may not be done by the graphics card at all, but the results may be sent to the graphics card for display. For example, the original Windows GDI did its drawing using the CPU, as hardware accelerated graphics were not included on every machine.

So really, there are a variety of ways for applications to do their drawing. They rarely use OpenGL, DirectX, etc. directly unless they have heavy graphics requirements. However, under the hood, the OS may be using those technologies or something similar, or something even lower-level.


Finding vertices of the outer contour of intersecting polygons

Sounds like you’re looking for a way to do a Boolean union operation. There’s a couple of algorithms linked from that article that should do the trick.


Should JPEG encoding take into account gamma?

The short answer is "no", for reasons covered in Alvy Ray Smith's memo, Gamma Correction. Gamma is not about nonlinearity in human perception, it's about nonlinearity in display devices (and, I suppose, acquisition devices too).


LWJGL/OpenGL get BufferedImage from texture ID

To make it a more complete example, let's also consider loading a texture from a BufferedImage as well.

First let's assume:

int texture = glGenTextures();
glBindTexture(GL_TEXTURE_2D, texture);

First we need to load an image. For this we can use ImageIO.read(). Then we put all the pixels from the BufferedImage into a ByteBuffer.

BufferedImage image = ImageIO.read(new File("image.png"));
int width = image.getWidth(), height = image.getHeight();

int[] pixels = new int[width * height];
image.getRGB(0, 0, width, height, pixels, 0, width);

ByteBuffer buffer = BufferUtils.createByteBuffer(width * height * 4); // 4 because RGBA

for(int y = 0; y < height; ++y) {
    for(int x = 0; x < width; ++x) {
        int pixel = pixels[x + y * width];
        buffer.put((byte) ((pixel >> 16) & 0xFF));
        buffer.put((byte) ((pixel >> 8) & 0xFF));
        buffer.put((byte) (pixel & 0xFF));
        buffer.put((byte) ((pixel >> 24) & 0xFF));
    }
}

buffer.flip();

glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, buffer);

Before we go ahead and call glGetTexImage() lets assume that the only thing we have is the texture name (texture). Thus we consider that we don't know the size or format of the texture. We can get all this by doing:

int format = glGetTexLevelParameteri(GL_TEXTURE_2D, 0, GL_TEXTURE_INTERNAL_FORMAT);
int width = glGetTexLevelParameteri(GL_TEXTURE_2D, 0, GL_TEXTURE_WIDTH);
int height = glGetTexLevelParameteri(GL_TEXTURE_2D, 0, GL_TEXTURE_HEIGHT);

Using that information we again create a ByteBuffer as well as a BufferedImage. Reading the pixels from the ByteBuffer and placing them on the BufferedImage.

int channels = 4;
if (format == GL_RGB)
    channels = 3;

ByteBuffer buffer = BufferUtils.createByteBuffer(width * height * channels);
BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);

glGetTexImage(GL_TEXTURE_2D, 0, format, GL_UNSIGNED_BYTE, buffer);

for (int x = 0; x < width; ++x) {
    for (int y = 0; y < height; ++y) {
        int i = (x + y * width) * channels;

        int r = buffer.get(i) & 0xFF;
        int g = buffer.get(i + 1) & 0xFF;
        int b = buffer.get(i + 2) & 0xFF;
        int a = 255;
        if (channels == 4)
            a = buffer.get(i + 3) & 0xFF;

        image.setRGB(x, y, (a << 24) | (r << 16) | (g << 8) | b);
    }
}

Now go ahead and save that BufferedImage to a file.

ImageIO.write(image, "PNG", new File("out.png"));

Last but not least, you can get rid of all those GL11. prefixes, by statically importing GL11. You do that by doing:

import static org.lwjgl.opengl.GL11.*;


A common interpretation of surfaces for rendering is that of a BSDF (Bidirectional Scattering Distribution Function) made up of a BRDF (R stands for Reflectance) and a BTDF (T stands for Transmittance).

In such a model, the reflectance part is the light that 'bounces' off the surface, and the transmittance part is the light that penetrates the surface.

When modeling the latter there are two common looks that we try to replicate. One is that of pure refraction (light goes through and bends at the surface), and the other is that of diffusion (light bounces around within the surface and is re-emitted (this is commonly called diffuse reflection). What determines which one of these happens is the turbidity of the material, which is analogous to the probability of a light ray changing its path while inside the material.

These two phenomena are opposite ends of a spectrum and therefore there cannot, on real materials, be a blend (e.g. alpha blend) between the two. though one could concieve a model where turbidity of the material is not at either end but somewhere in-between, however that would require taking into account the volume of a 3d model and not only the surfaces (this often called Volume Scatter or Sub Surface Scatter).

Thus, what you are looking for is probably refraction. There are many ways to aproximate it. One such way that will get you the expected color but not the distortion you might want is multiplicative blending.


Which graphics APIs texture formats are HDR texture formats?

Also consider these alternatives for reducing bandwidth in comparison to R16G16B16A16.

DXGI_FORMAT_BC6H_UF16 is block compressed fp16 (half) RGB.

  https://docs.microsoft.com/en-us/windows/desktop/direct3d11/bc6h-format

DXGI_FORMAT_R11G11B10 unsigned small float

  https://bartwronski.com/2017/04/02/small-float-formats-r11g11b10f-precision/

Common image file formats for HDR are exrs or dds. They're pretty trivially encoded as 32-bit or 16-bit per channel floats.


Projection of a Polyhedron on xy Plane with CGAL

A possible way to do this is tracing the silhouette edges of the polyhedron and projecting them to a 2d polygon only at the end of the process.

A silhouette edge (in your context) is defined by its neighboring facets having one upwards and one downwards normal (i.e., one positive normal z-coordinate and one negative). This can be checked using Polyhedron_traits_with_normals_3 or you can compute the normals yourself on-the-fly.

If you only need the outside polygon, you can start with an extreme vertex (e.g., the one with minimal x-coordinate), circulate over its incident halfedges (using Halfedge_around_vertex_circulator), take the one that is a silhouette edge and continue tracing with its next vertex until you close the cycle.

If you also need inside (hole) polygons, a possible solution can be to gather the silhouette edges from all the polyhedron, and trace the cycles from this set.


Wondering if it's possible to 'extend' a texture beyond the mesh without altering the mesh itself

Textures are applied to the faces of a mesh, so textures cannot be anywhere there isn't a face for it to appear on. "Adding more length" to a mesh pretty much implies moving some of the vertices. I don't think you're going to get around that.
Now if you are assembling a game world, it's quite common to have modular mesh pieces that can be aligned end-to-end, or placed intersecting each other, to form larger structures—like a wall panel that can be duplicated several times to form a longer wall, for instance, without needing a new mesh to be built specifically for that wall.


How to perform additive blending in Metal. What do these terms mean?

The “blend factors” are the amount that the specified component is multiplied by while blending. “Destination” is the existing content that’s being blended “on top of”, while “source” is the new content that’s being blended “onto” it. When the blend operation is set to “add”, the final pixel value is calculated like this:
value = source * sourceBlendFactor + destination * destinationBlendFactor

…so a blend factor of “one” means the value is used unchanged (multiplied by 1), “[x]Alpha” means the value is multiplied by the alpha value of x (source or destination), and “oneMinus[x]Alpha” means the value is multiplied by (1 - the alpha value of x).
For additive blending, you most likely want the following:
Objective-C:
descriptor.colorAttachments[0].blendingEnabled = YES;
descriptor.colorAttachments[0].rgbBlendOperation = MTLBlendOperationAdd;
descriptor.colorAttachments[0].alphaBlendOperation = MTLBlendOperationAdd;
descriptor.colorAttachments[0].sourceRGBBlendFactor = MTLBlendFactorSourceAlpha;
descriptor.colorAttachments[0].sourceAlphaBlendFactor = MTLBlendFactorOne;
descriptor.colorAttachments[0].destinationRGBBlendFactor = MTLBlendFactorOne;
descriptor.colorAttachments[0].destinationAlphaBlendFactor = MTLBlendFactorOne;

Swift 5:
descriptor.colorAttachments.colorAttachments[0].isBlendingEnabled = true
descriptor.colorAttachments.colorAttachments[0].rgbBlendOperation = .add
descriptor.colorAttachments.colorAttachments[0].alphaBlendOperation = .add
descriptor.colorAttachments.colorAttachments[0].sourceRGBBlendFactor = .sourceAlpha
descriptor.colorAttachments.colorAttachments[0].sourceAlphaBlendFactor = .one
descriptor.colorAttachments.colorAttachments[0].destinationRGBBlendFactor = .one
descriptor.colorAttachments.colorAttachments[0].destinationAlphaBlendFactor = .one

There’s a really helpful tool for visualizing the different options here—it’s written for OpenGL, but the Metal stuff works the same way.


How can I remove aliasing in an image without introducing blur?

Depending on the specific content of your input images, you could either use the Gaussian filter with kernel size varying with distance from the camera ( Link ) or use a slightly more complex bilateral filter, which is widely used to eliminate noise while keeping edges mostly intact.


In games and other 3D scenes, generally when the user clicks the mouse, a ray is cast into the scene in the direction the camera is facing, and a check is done to see what geometry in the scene it intersects. If it intersects nothing, it is ignored. If it intersects a single object, then the application processes a click on that object (at the location of the intersection). If more than one object is intersected, then usually the front-most object is assumed to be the on the user wanted to click on, so that is processed.

In general, geometry is stored using x,y,z coordinates rather than spherical coordinates, as the GPU hardware works in cartesian coordinates.


Can (and how so) shaders paint screen pixels beyond those that occupied by the shaded mesh?

When you use linewidth or line antialiasing or pointwidth or pointsprites, OpenGL creates for you a small rectangle instead of the line or point, with texture coordinates. Nowadays you can even program this yourself using geometry shaders ot even the tesselator.

A totally different approach is to use defered shading, using one geometric pass just to store information in the RGBAZ buffer, then a second pass than you run on all the pixels of the screen to do some process. (to act on all pixels, simply draw one full-screen rectangle).
Nowadays you can even do the first pass as one or several "render to texture", then MIPmap these textures, so that the final pass can easily access to less local values.


Ladder to DNA using Transformation Matrix

You can't perform this transformation by applying constant transformation matrix to the ladder model since it's not linear transformation like joojaa said in the comments. What you would have to do instead is to define the transformation as a function of height as you have done in both of your proposals.

So both of your proposals are kind of correct, except for the typo in second one where you should use $R$ instead of $T$ that you raise to power of y. However, you might have difficulties raising a matrix to non-integer power (requires expansion of Taylor series, AFAIK), so your first proposal is much more practical.


How to load a model in .json in three.js

The error not well-formed        male.json doesn't appear to have anything to do with the code you've posted. I don't see any references to a JSON file called male.json.

If this file is in your code somewhere, the error is telling you the problem. It is malformed. Paste the contents of the file into an online validation tool such as JSONLint. It will tell you what is making it malformed.

Also, you're passing JSON into ObjectLoader.load but you don't do anything with the results. Instead try this:

var loader = new THREE.ObjectLoader();
loader.load('somemodel.json', function (object) {
    scene.add(object);
});

Check out the documentation for more information.


Final transformation matrix to transform world coordinate into vrc

Thanks for notification me, In my previous answer, I just combine the translation matrix and the rotation matrix, actually, It's the result translation matrix(T) multiply rotation matrix(R).

The rotation is a 3x3 matrix, we can extend the matrix to a 4x4 matrix with adding a dimension, the 4x4 rotation matrix is:

And the translation matrix is:

So, now we can calculate the transform matrix by translation matrix multiply(T) rotation matrix(R):

Cause the matrix multiply formula is too big for the 4x4 matrix, we just look at the 1st row, In the transform matrix(M), the M11 element equals to T11*R11+T12*R21+T13*R31+T14*R41, cause T12, T13, R41 is 0, so, the M11 equals to R11 right? And in the same way, M14 equals to T11*R14+T12*R24+T13*R34+T14*R44, only R44 equals to 1, so, the M14 equals to T14. so, I just simply put them together in my last answer. For the reason why we put the translation matrix on the left, you can check on the book that I gave you in my previous answer on page 91(translating first and then rotating).


Why should you do homogenization after clipping?

One good reason to do clipping in homogeneous space is that the perspective division loses the distinction between regions behind the camera and in front of the camera. Points in front of the camera have $w > 0$, and behind the camera $w < 0$. After dividing by $w$, points behind the camera get mapped (with a sign flip) to the same $x, y, z$ NDC space as those in front of the camera. So it is no longer possible to tell the difference; the original sign of $w$ is lost.

This implies that at least the $w$ clipping must be done in homogeneous space. For instance, triangles that have one vertex behind the camera must get clipped by the $w = 0$ plane to form a quad. Otherwise, you end up with a bizarre object called an "external triangle"—a triangle that appears in two pieces, both extending to infinity, originating from the in-front-of-camera and behind-camera regions of the original homogeneous triangle.

And I suppose that once you have to apply one of the clipping planes in homogeneous space, you might as well apply all of them there.

For further reading on this, here are a couple of good articles:

In-depth: Software rasterizer and triangle clipping by Simon Yeung
A trip through the Graphics Pipeline 2011, part 5 by Fabian Giesen


If I'm guessing right, you are drawing a plane which is facing the camera, so your normals need to be 0,0,-1.

change

double[] N=normalize((new double[]{pos[0],pos[1],1}));

to

double[] N=normalize((new double[]{0,0,-1}));


What exactly is it about reality that is so difficult to recreate in a 3D simulation?

As @AnnoyinC already mentioned in his comment, we are simply lacking computational power and storage requirements (RAM).
For the visual effects, you need to calculate every ray of light that somehow ends up in the observer's eye after an arbitrary number of reflections on different surfaces. The reflections depend on the surface's microstructure. This alone is so extremely demanding that is far from possible to calculate it in real-time with current consumer hardware. The first steps into this direction are already taken with ray tracing support of modern graphic cards but it is still very limited.
And that's just the "easy part".
It gets way more complicated if you want to have objects and fluids that behave physically correct. While each ray of light is more or less independent of other rays, physical objects influence each other. Also, simulating the correct reaction of an arbitrary object to an external force can already take days and weeks for a single timestep/frame since you need to solve extremely large equation systems that are often non-linear. If you want to know why I would suggest searching for a "Finite-Element-Method" or "Computational Fluid Dynamics" tutorial. These techniques are commonly used by engineers and separate a structure into a mesh of smaller interacting substructures. The results get closer to the behavior of the real structure/fluid the denser the mesh. However, doubling the mesh density in each of the 3 spatial directions comes at the price of needing 8 times more computational power and RAM. Now think of a tree waving in the wind with thousands of leaves where each leaf alone is a complex physical object... I think you get the picture.
To get around all these problems, a lot of approximations are used. While most of them are quite good illusions they will give wrong results at some point. However, that doesn't necessarily mean that we realize it.


Per Vertex Computation in OpenGL Tesselation

Based on the comment of "ratchet freak" I researched Transform Feedback Buffers and solved my problem that way.

I now generate the simulated points on the CPU and put them into a VertexBufferObject.
I generate a second VBO for the points (along with some others for velocity).
The connectivity of the cloth is given as an vertex-attribute in ivec4.

Using the transform feedback buffers and a Double-buffering trick using the two VBOs i can always read from the last step (using the connectivity info) and write to another buffer. This is in order to solve problems with concurrency.

The calculations are done in the vertex-shader as GL_POINTS.
Binding the output of the first shader into another regular shader using additional index-buffer to generate triangles I can without trouble render the cloth any way I want.

This idea follows the transform feedback buffer example made in the book "OpenGL Superbible" http://www.openglsuperbible.com/


Three quaternions interpolation with barycentric coordinates

Defining the problem
The weighted arithmetic mean $\mu$, of a set of vectors, minimizes squared L2 norm of the error ($v_i$) to all elements of the set.
$$
\begin{equation}
\begin{split}
v_i &= \mu - x_i \\
\mu &= \underset{\mu \in \mathbb{R^n}}{\text{argmin}} \; \frac{1}{2}\sum\limits_{v_i} w_i ||v_i||^2_2 \\
\end{split}
\end{equation}
$$
This empowers us to define special means, where we use different notions of distance.
Specializing for rotations
A reasonable notion of distance for SO(3) is the angle separating two rotations. That's $||\log(\mu \ q^{-1})||$ Your geometry library probably implements this $log$, and expressions for it can be found here. It is not the log one is accustomed to on the real numbers.
Intuition: Recall that every sequence of rotations corresponds to a single rotation about a single axis, so the difference between a pair of rotations can always be summarized by a single angle.
$$
\begin{equation}
\begin{split}
v_i &= log(\mu \ x_i^{-1}) \\
\mu &= \underset{\mu \in \mathbb{SO3}}{\text{argmin}} \sum\limits_{v_i} w_i || v_i||^2_2 \\
\end{split}
\end{equation}
$$
There's no analytical form for this minimum for more than two rotations. We can, however, treat this as a least squares problem and compute the result by Gauss-Newton.
To apply Gauss-Newton in general, for some error, we need to be able to find $\mathbf{J_i} = \frac{d}{d\mu}v_i(\mu)$, which requires differentiating $v_i$. Then we can take a step as
$$
\begin{equation}
\begin{split}
\Delta \mu &= \sum\limits_{i}(w_i \mathbf{J_i}^T \mathbf{J_i})^{-1} \sum\limits_{i} w_i \mathbf{J_i}^T v_i
\end{split}
\end{equation}
$$
Our error function cannot readily be differentiated, since its argument is a rotation $\mu$ and not a vector. We invent a parameter $\delta \in \mathbb{R^3}$ which we will $Exp$ into a rotation. This, we can differentiate
$$
\begin{equation}
\begin{split}
v_i &= log(Exp(\delta)\; \mu \ x_i^{-1}) \\
\mathbf{J_i} &= \frac{d}{d\delta} v_i = \mathbf{I} \\
\end{split}
\end{equation}
$$
So the update to $\mu$ is
$$
\begin{equation}
\begin{split}
\delta &= \sum\limits_{i}(w_i \mathbf{J_i}^T \mathbf{J_i})^{-1} \sum\limits_{i} w_i \mathbf{J_i}^T v_i\\
\delta &= \sum\limits_{i} w_i \sum\limits_{i} w_i v_i \\
\delta &= \sum\limits_{i} w_i v_i
\end{split}
\end{equation}
$$
Conclusion
$$
\begin{equation}
\begin{split}
\delta = \sum\limits_{i} w_i v_i &= \sum\limits_{i}w_i log(\mu_{k}^{\mathstrut} x_{i}^{-1}) \\
\mu_{k + 1} &= Exp(\delta) \mu_{k}\\
\end{split}
\end{equation}
$$
This will converge quadratically to a mean.
Caveat emptor: for rotations, there may be many "means". By analogy: What is the average point, on Earth's surface, of the north and south poles? Every point on the equator has equal right to be called the average. This is the case for all equispaced rotations.


How to use multithreading in 3D software renderer to speed up fetching texture values

It could be that you have to overcome a different bottleneck first.

Have you ever read Jim Blinn's "The Truth About Texture Mapping"? (I had a quick search to see if I could find a non-paywalled version but you may have better luck than me. Alternatively you might find a dead tree version of "Jim Blinn's Corner" in a library). Though this article is old and describes paging of texture data, it is still very relevant today.

Essentially, if your textures are large (i.e. too large to fit in the cache), in scan order, and they have been rotated when displayed on the polygons, you are very likely to be thrashing your cache and, as memory is an order or two of magnitude slower than the CPU, this will hurt performance.

To avoid the cache thrashing, textures are often stored in twiddled-order (that's what we called it in the early 90s but, more correctly, it'll be some variant of Morton order) or in a block order, which is what Blinn describes. This then makes texel/memory accesses far more coherent and the cache more effective.


Fading an object in add mode with correct color math

Turns this was not an issue with the scale of the values passed in, just that the log function in glsl is base e.  The solution was merely to do log(stuff)/log(10) and that got the result of a base 10 log.


LWJGL 3 render to texture offscreen without display

So the way I went by not showing the window seems the way to go, and the error about GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENTapparently wasn't related to this at all and is gone.


Problem

XMMATRIX and XMVECTOR (which use __m128 under the hood) require 16 byte alignment. The C++ compiler can automatically ensure this for XMMATRIX and XMVECTOR data allocated on the stack, but can't ensure this for XMMATRIX and XMVECTOR data allocated on the heap.

In your specific situation (referring to the RasterTek tutorial), three member variables of your D3DClass class are declared to be of type XMMATRIX and you construct one instance of this D3DClass class on the heap by using the default operator new (due to the absence of a custom one). Using the default operator new does not guarantee this 16 byte alignment. Even if your code would have run fine using the default operator new, you could still face this issue after adding some extra member variables in the future.

Solution 1

The most easy fix in this specific case, is to allocate your D3DClass class instances on the stack instead of on the heap. This, however, does not prevent you or future users of your code to allocate an instance on the stack instead.

Solution2

The programmers of the DirectXMath library advise the general programmer audience to use XMMATRIX and XMVECTOR for calculations only, and to not use them as containers for storing persistent data (i.e. class member variables). For the latter, they advise to use XMFLOAT4, XMFLOAT4x4, etc. The latter structs do not impose the 16 byte alignment restriction and their instances can simply be loaded in SIMD registers by using XMLoadFloat4 for XMVECTOR, XMLoadFloat4x4 for XMMATRIX, etc.

  However, often it is easier and more compact to avoid using XMVECTOR
  or XMMATRIX directly in a class or structure. Instead, make use of the
  XMFLOAT3, XMFLOAT4, XMFLOAT4X3, XMFLOAT4X4, and so on, as members of
  your structure. Further, you can use the Vector Loading and Vector
  Storage functions to move the data efficiently into XMVECTOR or
  XMMATRIX local variables, perform computations, and store the results.
  There are also streaming functions (XMVector3TransformStream,
  XMVector4TransformStream, and so on) that efficiently operate directly
  on arrays of these data types.

Solution 3

Ensure the 16 byte alignment by overwriting operator new/operator delete, operator new[]/operator delete[]. We declare a template class which does allow heap allocation with arbitrary alignment restrictions:

#include <malloc.h>

inline void *AllocAligned(size_t size, size_t alignment = 16) noexcept {
    return _aligned_malloc(size, alignment);
}

inline void FreeAligned(void *ptr) noexcept {
    if (!ptr) {
        return;
    }

    _aligned_free(ptr);
}

template< typename DataT >
struct AlignedData {

public:

    static void *operator new(size_t size) {
        const size_t alignment = __alignof(DataT);

        // __declspec(align) on DataT is required
        static_assert(alignment > 8,
            "AlignedData is only useful for types with > 8 byte alignment.");

        void * const ptr = AllocAligned(size, alignment);
        if (!ptr) {
            throw std::bad_alloc();
        }

        return ptr;
    }

    static void operator delete(void *ptr) noexcept {
        FreeAligned(ptr);
    }

    static void *operator new[](size_t size) {
        return operator new(size);
    }

    static void operator delete[](void *ptr) noexcept {
        operator delete(ptr);
    }
};

This code snippet is based on the DirectXTK and is also included in my own codebase (including documentation).

This data structure can now be used as follows:

__declspec(align(16)) struct Transform final : public AlignedData< Transform > {
    XMMATRIX m_transform;
}

A Transform structure can now be allocated on the heap with 16 byte alignment, since our custom operator new will now be invoked:

Transform *transform = new Transform();

If you use instances of the Transform struct as member variables in other structs of classes, the alignment restrictions remain of course. So instead of having only alignment restrictions for XMVECTOR and XMMATRIX, you will have now alignment restrictions for XMVECTOR, XMMATRIX and Transform.

Smart Pointer Pitfalls

Be careful if you use std::shared_ptr and especially std::make_shared. The latter allocates the data block together with the control block without using customly defined allocators such as our custom operator new.

This means that this method will not work:

template< typename T, typename... ConstructorArgsT >
inline std::shared_ptr< T > MakeShared(ConstructorArgsT&&... args) {
    return std::make_shared< T >(std::forward< ConstructorArgsT >(args)...);
}

but you should rather use an explicit invocation of operator new (which is also the case for std::make_unique):

template< typename T, typename... ConstructorArgsT >
inline std::shared_ptr< T > MakeAllocatedShared(ConstructorArgsT&&... args) {
    return std::shared_ptr< T >(new T(std::forward< ConstructorArgsT >(args)...));
}

std::unique_ptr and especially std::make_unique will work as intended since the latter explicitly calls operator new (due to the absence of a control block).

RasterTek

RasterTek's second series of D3D11 tutorials is a subset of RasterTek's first series of D3D11 tutorials with the minor difference that the latter uses the obsolete D3DXMath. The transition is from D3DXMath to DirectXMath is actually pretty straightforward. Therefore, it can be interesting to take a look at the first series as well (which I ported to Visual Studio 2017, run fine in both Debug and Release configurations, and can be founded as well in this repository).

If you want to stick to DirectXMath, you can also take a look at the D3D11 tutorials of Microsoft itself (explanation + code). (I also created a repository with some code refactors of these tutorials).


Objects in depth buffer are always black (0.0)

That's how I used to initialize viewport (CD3DX12_VIEWPORT). But I didn't realize that there are two additional fields minDepth and maxDepth. Therefore, I have min/max depth set to 0.0 and objects rendered to depth buffer are always black:
m_viewport.TopLeftY = 0.0f;
m_viewport.TopLeftX = 0.0f;
m_viewport.Width = static_cast<float>(m_windowSize.x);
m_viewport.Height = static_cast<float>(m_windowSize.y);

Fixed that by simply using constructor which default minDepth to 0.0 and maxDepth to 1.0:
m_viewport = CD3DX12_VIEWPORT{ 0.0f, 0.0f, static_cast<float>(m_windowSize.x), static_cast<float>(m_windowSize.y) };


Averaging RGB should be correct assuming you do it in linear RGB. If your images are in sRGB, you can remove the gamma correction by doing something like the following for each of the R, G, and B components:

float sRGBToLinear(UInt8 component)
{
    float tempComponent = (float)component / 255.0;
    if (tempComponent <= 0.04045)
    {
        tempComponent = tempComponent / 12.92;
    }
    else
    {
        tempComponent = pow((tempComponent + 0.055) / (1.055), 2.4);
    }
    return tempComponent;
}

You can then average together all of the red values in the image, all of the green values in the image, and all of the blue values in the image after they've been run through the above conversion. You can then do the opposite conversion to get back to sRGB:

UInt8 linearRGBTosRGB(float component)
{
    float tempComponent =  0.0;
    if (component <= 0.00318308)
    {
        tempComponent = 12.92 * component;
    }
    else
    {
        tempComponent = 1.055 * pow(component, 1.0 / 2.4) - 0.055;
    }
    return (UInt8)(tempComponent * 255.0);
}

(My apologies if I've messed up the syntax. I'm a C/C++ programmer and don't know the intricacies of Java.)

Note that alpha makes things slightly more complicated. If you're using premultiplied alpha, you can simply apply the above calculations for the average. If you're using straight alpha, you'll want to multiply each of the R, G, and B components by the alpha before doing the averaging.


Roughly speaking, what the Supersampling Anti-Aliasing method does is:

For each pixel in the image to render

"Split" the pixel of a target image, usually in 2, 4, 8 or 16.
Actually this step is more: pick the 2, 4, 8 or 16 locations ("samples") inside the pixel
Compute the color that the image will have at each of those locations (sample or "split part").
So instead of only computing the color that the image will have in that single pixel, we compute 2x, 4x, 8x or 16x times a similar value
Compute the average of all the 2, 4, 8 or 16 sample colors to decide the final color or the initial pixel

You can see that Super-sampling cost is proportional to the cost of computing one pixel's color in the image. Therefore it depends on the cost of shading.
It also depends on the number of pixels, i.e the resolution of the image.

In the context of Ray-Tracing applications, this means that 2, 4, 8 or 16 times more rays will have to be computed.

In the context of GPU-accelerated graphics computation, this means that 2, 4, 8 or 16 times more pixel shaders ("fragment programs") will be executed.

Now roughly speaking, what the Multisampling Anti-Aliasing method does is:

For each pixel in the image to render

Pick 2, 4, 8 or 16 locations ("samples") inside the pixel

For each primitive (triangle) in the scene (actually, only for those that overlap the pixel):

Determine which of these samples are covered by the current triangle
If there is at least one such sample, compute a single color of the contribution of that triangle to the pixel, and store it for each sample covered
The worse case happens if each sample is overlapped by a different triangle ;  in that case the color must be computed for each triangle

Compute the average of all the gathered sample colors to decide the final color or the initial pixel

You can see that evaluating the performance of Multisampling Anti-Aliasing is more complicated than that of Supersampling, since it also depends on the topology of the scene (the size and shape of the triangles).


What are some methods to render transparency in OpenGL

A set of techniques to avoid explicit ordering go under the name of Order Independent Transparency (OIT for short).

There are lots of OIT techniques.

Historically one is Depth Peeling. In this approach you first render the front-most fragments/pixels, then you find the closest to the one found in the previous step and so forth, going on with as many "layer" as you need.
It is called depth peeling because at each pass you "peel" one layer of depth. All your layer can be then normally recombined from back to front.
To implement this algorithm you need to have a copy of the depth buffer.

Another set of techniques are the blendend OIT ones. One the most recent and interesting one is the Weighted Blended OIT proposed by McGuire and Bavoil. It basically apply a weighted sum for all the surfaces that occupies a given a fragment.  The weighting scheme they propose is based on camera-space Z (as an approximation to occlusion) and opacity.
The idea is that if you can reduce the problem to a weighted sum, you don't really care about ordering.

Other than the original paper, a great resource for implementation details and problems of Weighted Blended OIT is in Matt Pettineo's blog.  As you can read from his post this technique is not a silver bullet. The main problem is that the weighting scheme is central and it needs to be tuned according to your scene/content.
From his experiments, whilst the technique seems to work fine for relatively low and medium opacity, it is failing when opacity approaches 1 and so could not be used from materials where big part of the surface is opaque (he makes the example of foliage).

Again, all come down to how you tune your depth-weights and finding the ones that fit perfectly your use-cases is not necessarily trivial.

As for what is needed to for the Weighted Blended OIT, nothing more than two extra render targets. One that you fill with the premultiplied alpha color ( color * alpha) and alpha, both weighted accordingly. The other one for the weights only.