API or Tools to query state of graphics during gameplay?

What I mean by “state of graphics” is, how polygons, textures, lighting and any other scene elements are stored and computed for any particular frame.

The closest to what you are asking and that I know of is RenderDoc. I have not tested it myself so far, but some features they list in their documentation that are related to your question are:

Full graphics pipeline state display
Shader source display
Flexible resource inspector
Mesh buffer inspection and visualisation
‘Raw’ buffer inspection for buffers
Texture/render target viewer

  I’m looking to somehow programmatically query the state of graphics during a PC gameplay session.

  ...

  Is there any need to be game engine specific in this matter, or will all games be drawn to the screen in the same way?

I am not sure here if you are asking to dissect the graphics state of a commercial game or software that is not written by yourself. If so, there is no guarantee that RenderDoc can capture data but it is possible. See for example this issue on their GitHub page, where the author says:

  However I want to be clear that I cannot and do not support capturing 3rd party programs that you do not control. It generally works but for obvious reasons I don't provide any help with problems...

As stated in this section of their documentation, RenderDoc uses a "man-in-the-middle" approach to get the data. So it intercepts any calls to the graphics API (OpenGL, DirectX, Vulkan) of the executed program, gathers the data it needs, and passes it along to the API afterward.


Why inverse of an allignment matrix is the same as its transpose?

To align vectors, you are applying a rotation.

If you consider a simple rotation matrix, e.g. rotation of angle $\theta$ around Z axis, then you will trivially see that its inverse matrix, a rotation of $-\theta$, is the transpose,  since $sin(-\theta)=-sin(\theta)$ and $cos(-\theta)=cos(\theta)$.

An arbitrary rotation matrix, R, can be constructed by multiplication of other rotations, e.g $R = A \cdot B$.  If the inverse of A and B exist, then $R^{-1} = B^{-1} \cdot A^{-1}$.
Similarly we know  $(A \cdot B)^T = (B^T \cdot A^T)$.

Put these together and you have your answer.


How to perform an image normalization after for example adding a constant to it?

I believe that Zrep is the output range. In 8-bit it would be 255. Let's say fmin was 25, fmax was 132. You want to scale that range to 0-255. The part of the equation in brackets: [(f-fmin) / (fmax-fmin)] will give you a percentage - a value between 0 and 1. If f is 25, you'd get 0%. If f is 89, you'd get ~60%. If it's 132, you'd get 100%

So once you have the percentage, you then need to multiply that by the output range, Zrep. That converts from a percentage to whatever range you need it in, and ensures that the largest value in the input is mapped to the largest output value, and the smallest input is mapped to the smallest output value.


OpenGL Framebuffer with multiple Depthbuffers inside

FBOs can only use a single depth image, but that image can be layered. Broadly speaking, VR multi-eye rendering ought to be done through layered FBO images for both the color and depth buffers.
The problem with your code is that you can't use glFramebufferTexture3D to attach a layered image. You must use glFramebufferTexture.
Also, you need to use layers for the color image as well.


Amount of light reaching the sensor of pinhole camera model

This effect is called vignetting. The cases you talk about are specifically natural and pixel vignetting as explained on the wiki page. Digital cameras compensate these effects, but optical vignetting has to be done in image software for DSLR's with interchangeable lenses. For example if you import an image to Lightroom, there's "lens correction" option, which removes vignetting, distortion and chromatic aberration caused by the lens based on the lens profile.


How to get the GLFW_CONTEXT_VERSION_MAJOR value?

According to the GLFW docs, the context version hint acts as a minimum version, i.e. the context you actually get may be a higher version than what you request. However, the context should be backward-compatible, so even if you get a GL 3.x context you're OK to use GL 2.x code.


How do graphic engine developers debug their 3D graphics code?

I'm programming mostly in OpenGL. In this case, the very first thing I do is setting up KHR_debug extension. It provides a way to subscribe to messages generated by GL drivers - from errors to warnings to detailed information about what the driver is doing. Especially open source Linux drivers (Mesa) generate lots of useful messages.

When it comes to tools, my first choice is usually ApiTrace. It allows you record a trace of all rendering API calls and then replay it and inspect state at any point in time (this includes contents of buffers, render targets, etc.). It also allows you to share a trace with other people, which is useful when working in a team. Then, there are other tools such as RenderDoc, NVIDIA Tegra Graphics Debugger, Xcode's graphics debugger and some others. Whichever you chose, it will improve your work a lot.

Another thing is testing. One thing is building a collection of simple scenes excercising single feature of the engine (such as different material settings, different animation techniques, etc.).
When a bug is introduced into the codebase, this helps debugging it (since you have simple to analyse test cases readily available) and verifying that a fix doesn't accidentally break something else. I guess it's quite common thing to do; Unity3D developers do it, for example.

You could extend this and automate it by creating test harness, which would build and run all test on regular basis, and verify that no regressions happened. If you have lots of test scenarios, this might be feasible, although obviously requires additional investments into infrastructure.

Then there are more "standard" unit tests. One area I've applied them is node-based material system, which can target different rendering techniques (forward, deferred, whatever) and APIs. I approached it by writing unit tests which generate shader code from material and other settings, compiler it and issues a simple draw calls with given inputs (pixel shader inputs). Outputs are written to a render target and fetched for comparison with expected values. IMHO this is a nice  thing to do when you have complex material system, esp. with shader code generation.

Finally, more "traditional" approaches can be applied to areas which can be separated from rendering. Things like spatial data structures, asset loading and similar can be unit/integration tested, as applicable in specific case.

Edit: Here's a post from Unity3D developer describing how they test graphics code.


I don't know any IDE-based tools that do this but a quick Python script should do the trick. I usually use Thijs Vogels' PyEXR to read the multi-channel EXR. Assuming your depth channel is called distance (which needs to be specified in your Mitsuba XML file under your hdrfilm), you can visualize it with the following code:

import pyexr

fp = pyexr.open('multichannel.exr')
img = fp.get('distance')
pyexr.write('distance.exr', img)


Where do the coefficients in the Catmull-Clark subdivision algorithm come from?

The derivation is presented in the original paper that introduced CC subdivisions as a generalisation of B-Spline patches: https://people.eecs.berkeley.edu/~sequin/CS284/PAPERS/CatmullClark_SDSurf.pdf


Where is the best place for Tangent-bitangent calculation, in shader or in C/CPP code?

Doing it in the CPU side during initialization is what I'd go for, this is assuming you are initialising that data once and passing it to the GPU.

On the GPU side, doing the calculations per fragment would be too costly and should be out of the question, unless it's really needed for a special effect then you shouldn't add unneeded ALU work to the fragment shader, this will only cause your application to slow down. On the vertex shader is more acceptable but again if you have highly tessellated geometry, it'll be expensive.

Whenever there is data that you can compute just once, then go for that instead of doing it every frame. Even if you were planning on doing it just once on the GPU and using transform feedback or an FBO to store the data, I think this approach would be more costly. As always though, profile your application, careful tradeoffs are specific to architectures.


Is colorized subpixel rendering possible?

Here's an example of a downsampling filter that takes pixel geometry into account: Increasing image resolution on portable displays by subpixel rendering.

Image (a) below is an image downsampled using pixel-based downsampling. Image (b) is downsampled using direct subpixel-based downsampling, which (as far as I can tell) effectively downsamples the R, G, and B planes of the image independently. Image (c) is downsampled using diagonal direct subpixel-based downsampling, which uses a diagonal pattern to improve apparent resolution in both horizontal and vertical directions.

You can see that the last two images look sharper, but also have the characteristic color fringing.

Here's the downsampling pattern used for the last image:


Luminance and Radiance in computer screen

1, 1, 1, the output value is white. If this represents radiance than the output should be green, because when the signal radiance is the same, the pixel should be perceived as green. Am I correct?

This is not correct. Your frame buffer most likely contains sRGB values. Of which 1,1,1 is used to represent white. Standard displays do not use spectral radiance values in the frame buffer. sRGB does not map directly or make any guarantees about the spectral power distribution (SPD) of the display. So "G" of 1 does not guarantee some wave with amplitude using W⋅sr−1⋅m−2 centered on the 550nm. Part of the problem is that different type of displays have different SPDs that map to the same perceived color.

[Evaluation of an organic light-emitting diode display for precise visual stimulation, Ito et. al]
(https://jov.arvojournals.org/article.aspx?articleid=2121657)

Now if you want to physically accurate rendering, you could store all of your lighting values as Radiance with SPDs. As L.C. mentioned, Physically Based Rendering describes that this is done with piece-wise linear functions with like 16+ channels per color. However even if you can get this to be performant, good luck converting all your RGB data into SPDs (there's an infinite # of SPDs for a given RGB value).

Instead what most renderers try to do is to treat the RGB as pseudo-radiance values for the purposes of lighting calculation. This isn't accurate, hence your lighting result changes if you are using Rec.709 or Rec.2020 primaries for instance (if you notice they have different values for mapping to luminance), but it's good enough for real time and even offline in most cases. At this point you can convert RGB Rec709 and Rec2020 values to a single scalar luminance using the Y values of it's primaries.

The Y values of the RGB primaries for Rec709 are:

    R       G       B
Y   0.2126  0.7152  0.0722

https://en.wikipedia.org/wiki/SRGB

Thus Rec709 Linear has the following nits conversion:

nits = dot(linear709Rgb, float3(0.2126, 0.7152,0.0722));

Probably one of the results of doing our calculations in RGB is that G and B add at equal weights, so it's likely to get some color shifting. There was a presentation at SIGGRAPH 2015 from Weta digital that had some examples, but it doesn't seem like the slides are available. As far as I know they are the only ones using a spectral renderer.


Flexible Vertex Format Constants in Direct3D 11

Vertex formats are specified in D3D11 using the input layout. An example input layout description for your vertex might look like:

D3D11_INPUT_ELEMENT_DESC aInputDescs[] =
{
    { "POSITION", 0, DXGI_FORMAT_R32G32B32_FLOAT, 0, UINT(offsetof(Vertex, p)), D3D11_INPUT_PER_VERTEX_DATA, 0 },
    { "NORMAL", 0, DXGI_FORMAT_R32G32B32_FLOAT, 0, UINT(offsetof(Vertex, n)), D3D11_INPUT_PER_VERTEX_DATA, 0 },
    { "UV", 0, DXGI_FORMAT_R32G32_FLOAT, 0, UINT(offsetof(Vertex, tu)), D3D11_INPUT_PER_VERTEX_DATA, 0 },
};


Why do we call it TRS and MVP, instead of SRT and MVP or TRS and PVM?

Because whether you use $MVP$ or $PVM$ actually depends on how you perform the multiplication: $vMVP$ versus $PVMv$, note that you have:
$(PVMv)^T = v^TM^TV^TP^T=vM^TV^TP^T$, where the last identity does not hold in math, but holds in glsl since vectors can be reinterpreted as either row or column vectors for convenience.

The same holds for $TRS$ and $SRT$. In general CG code and applications have an issue with sticking to standard established conventions, so you get all kinds of funny conventions.


What should I do when a new sampled direction is below a normal oriented hemisphere?

Disclaimer: I am assuming that you are implementing a classical Monte Carlo estimator.
The Problem
Discarding samples will change PDF of your sampling technique. You are cutting off part of the sampled domain where PDF is non-zero, which effectively leads to a trimmed version of the original PDF but implicitly re-normalized so the remaining part integrates to 1. If you don't adjust the directly evaluated PDF accordingly, it will lead to a biased estimator.
Practically speaking, implicit re-normalization increases the actual sampling PDF $p^{new}$, and if you use $p^{old}<p^{new}$ in your computations instead, the resulting Monte Carlo estimator will yield brighter values than it should:
$$
\frac{f(x)}{p^{old}(x)} > \frac{f(x)}{p^{new}(x)}
$$
A solution
Since adjusting the PDF isn't an easy thing to do, you will most likely need to treat the under-surface samples as valid but with zero contribution. Whether you handle this within your BRDF/BSDF or elsewhere in the renderer is your design decision.
Zero-contribution samples will, obviously, introduce some inefficiency in your renderer.
A better solution
You can improve the efficiency of your estimator by using a better sampling technique which tries to avoid creating samples under the surface. In case of the GGX normal distribution, there have been proposed some solutions by Eric Heitz | Eugene d’Eon in paper Importance Sampling Microfacet-Based BSDFs using the Distribution of Visible Normals. I believe there was one improved version of this technique (other that the one mentioned in the "Related Work" section) but I cannot recall the name of it. Maybe someone else can add it here...


Am I right if I understand

GLfloat vertices[] = {
    -0.5f, -0.5f, -0.5f, 0.0f, 0.0f,
    0.5f, -0.5f, -0.5f, 1.0f, 0.0f,
    0.5f, 0.5f, -0.5f, 1.0f, 1.0f,
    0.5f, 0.5f, -0.5f, 1.0f, 1.0f,
    -0.5f, 0.5f, -0.5f, 0.0f, 1.0f,
    -0.5f, -0.5f, -0.5f, 0.0f, 0.0f,
};

as being:

  x, y, z, texX, texY

for 6 vertices in total?

If so, then it seems like you do not use indexing yet (vertices 3 and 4 are the same), but nevermind.

Not being sure what order your coordinate system is in, the plane you are constructing here is perpendicular to an axis, but does not go through your origin.

An object that gets rotated by a rotation matrix rotates around its origin. The origin of your model matrix is (0, 0, 0), but this is not the center of your object.

In a right handed coordinate system, the following vertices give you a plane that is standing on the x-z-plane (perpendicular to it) and goes through your origin. Rotating that around y should rotate it like you wish to do:

 GLfloat vertices[] = {
     -0.5f, 0.0f, 0.0f, 0.0f, 0.0f,
     0.5f, 0.0f, 0.0f, 1.0f, 0.0f,
     0.5f, 1.0f, 0.0f, 1.0f, 1.0f,
     0.5f, 1.0f, 0.0f, 1.0f, 1.0f,
     -0.5f, 1.0f, 0.0f, 0.0f, 1.0f,
     -0.5f, 0.0f, 0.0f, 0.0f, 0.0f,
 };

Notice the third value for each vertex.


Fresnel reflectance of gold: red channel greater than 1?

RGB color is a bit more complicated a subject than readily seems apparent. The reflectance wavelength diagram shows the reason quite well actually.

RGB color model has several central problems:

What the colors represent: They represent 3 spikes in a continuous spectrum. The R, G and B aren't energetically equivalent let alone evenly spaced.
What their range is: The colors do not actually mean anything without information of what space they span. In the assumed sRGB color space the space does not span the entire sensable range. So energetically equal but more vivid colors exist.
Human sensory apparatus isn't actually reading 3 color spikes but the sensors nonlinearly overlap.

As a result one can not draw the conclusion that a reflectance color channel of greater than 1 automatically means that energy is inserted into the system. That is simply one of the possible interpretations.

Another interpretation is that the color is more intense than your color space allows. As a result your color vector component could be over 1.

Human eyes may also bleed color from one sensor to another due to overlap of the sensors. Such things happen with the sky which seems light blue, but is actually far darker blue but so intense that we see it as light blue. But in 50% reflections it would look wrong if we wouldn't account for this.

In the end, it can also mean energy is inserted into the system. Either the energy comes from elsewhere or is generated by the surface.

Rendering is often not a scientific measure of things. No energy principle needs to be broken to achieve this.

Summa summarum (tl;dr)

Color is often a compound attribute at the same time as it measures energy levels it also measures something other. Namely the location in the color space. Thus you can not differentiate the two signals (energy and color intensity) easily.

In this case it is a more intense color because the source says so: Outside of sRGB gamut = more intense color than the color space can make.


This depends on your calculations really, since you can do it either way. However, probably when you reach tone mapping you will be in linear space (i.e. no gamma correction has been done yet and you don't assume non-linear space for your calculations).
Assuming this, gamma correction should be applied after tone mapping, otherwise you have "linear" tone mapping on non-linear color values.

To quote from Real-Time Rendering 3rd by Möller, Haines and Hoffman, p. 145:

  It is important to apply the conversion at the final stage of rendering (when the values are written to the display buffer for the last time), and not before. If post-processing is applied after gamma correction, post-processing effects will be computed in nonlinear space, which is incorrect and will often cause visible artifacts. So operations such as tone mapping [...] can be applied to images to adjust luminance balance, but gamma corection should always be done last.


What graphics techniques can be used to simulate a color nebula effect?

I have no knowledge of SceneKit whatsoever, but the effect appears to be a simple post-process Gaussian blur of the app icon grid, with the blur radius increasing over time (i.e. increasing the blur amount) during the transition, with a touch of colour correction tacked on (which causes the darkening). It should be easy enough to implement in any graphics API that allows rendering to textures and performing image filtering on them (for instance, in OpenGL you could use framebuffer objects to achieve this).

You can have a look at an example WebGL implementation here. It's worth noting that it is a separable filter; meaning, it's a rectangle-shaped filter, but you can avoid sampling the same pixels (overlapping neighbouring samples) multiple times by splitting it into two passes – a horizontal and a vertical one – to the same effect.


Does normal mapping make sense for a view of earth from space?

In my opinion, yes, it does. I've heard it claimed that in reality, the ratio of the tallest mountain on Earth to the Earth's diameter is smaller than the variations in height of an apple's skin to its diameter. So I can't say that it would be realistic, but it certainly looks better than not doing it. At the least it gives you some shading between high and low spots, even with a simple directional light.


What is the correct order of transformations scale, rotate and translate and why?

Usually you scale first, then rotate and finally translate. The reason is because usually you want the scaling to happen along the axis of the object and rotation about the center of the object.

In your case you don't really need to worry about this generic solution though, but you only need to map range [0, 800] $\rightarrow$ [-2, 2] for x-coordinate and [0, 600] $\rightarrow$ [-1.5, 1.5] for y-coordinate, in order to map screen coordinates to real/imaginary components for Mandelbrot calculation. So this is simply done by:
$$real=4*(x+0.5)/800-2$$
$$imag=3*(y+0.5)/600-1.5$$

Note that you need to calculate Mandelbrot coordinates from screen coordinates and not the other way around. This is to ensure that you evaluate the Manderbrot equation for each pixel exactly once and are not left with holes in the image or do double evaluation per pixel.


Intuitively understanding conservation of energy for the Lambertian BRDF

A perfect Lambert reflector actually reflects light in a cosine distribution - that is, the amount of light per unit area reflected in any given direction $R$ is proportional to $N.R$. The reason the radiance appears constant for all angles is that as the view direction moves away from the normal, the reflected light per unit area decreases, but the surface area per projected beam area (and hence per pixel) increases by the same amount - the cosine factors cancel out.

Given this distribution, to be energy conserving, the probability density for the reflected light in all possible directions cannot sum to more than 1. If you integrate the cosine function over the whole hemisphere around the normal, you get $\pi$, so this is the normalization factor you need.


Why does the G1 term have a 2 in the numerator in Walter's formula, but not in Heitz's generalized formula?

Actually I just had to solve the following equation to find a solution, consistent with the presence of the "2" in the numerator:

$\frac{1}{1+\Lambda (m)}=\frac{2}{1+\sqrt{1+\alpha ^{2}tan_{2}\theta_{m} }}$

$1+\Lambda (m)=\frac{1+\sqrt{1+\alpha ^{2}tan_{2}\theta_{m} }}{2}$

$\Lambda (m)=\frac{-1+\sqrt{1+\alpha ^{2}tan_{2}\theta_{m} }}{2}$


There can be much dispute on which work is considered best or state-of-the-art. It will depend on your application. However, I'd like to recommend to give a look at this paper by Keenan Crane et al. It is impressive in terms of computational speed. Also, It is simple to implement.
Crane et al. Geodesics in heat: A new approach to computing distance based on heat flow. ACM TOG. 2013. (https://dl.acm.org/doi/10.1145/2516971.2516977)


What is a "light source" in global illumination?

Like @lightxbulb said, a light source (i.e. an emissive surface whose L_e term in the rendering equation is greater than zero) can also have a BRDF.
People usually do not model light sources really accurately down to the Tungsten filament geometry inside of a glass-enclosed gas chamber for incandescent light bulbs. Such light sources usually are modelled as simple geometries, like spheres, capsules, lines, rectangles or more generic polygons, etc.
So, most often, light sources are usually not exactly physically accurate anyways, because it would be too computationally expensive (imagine computing the intersection of a ray with the Tungsten filament in addition to account for a very little bit of volumetric scattering through the gas).
Therefore, in the simplest cases, such an incandescent light source is down to a simple sphere anyways. In that case, it would make most sense to model this light source with a translucent BTDF (Bidirectional Transmittance Distribution Function), because most of the rays will probably not hit the filament but would go straight through the glass.
What we would have to consider, however, is Fresnel and retro-reflection on the glass surface of the light bulb.
But it all depends on how accurately you want to model your light sources.
If, on the other hand, your light source is a milk glass light bulb, then you would probably need a BRDF + BTDF (or more generally a BSDF - bidirectional scattering distribution function) to account for the light that is reflected off the glass surface and the amount of light that is trasmitting through the glass.

In any case, stopping a ray at the light source (as you pointed out) would only be actually physically correct, if the surface of the light source is a black body not reflecting any light but only emitting light due to its temperature.


Is the laplacian operator for meshes just the sum of the differences of neighbouring vertices?

You can see why the sum of differences is giving a second derivative if you look at the derivative of a one-dimensional function $f$. The derivative of $f$ at $x_0$ is
$$ \frac{df}{dx}(x_0) \approx \frac{f(x_0 + dx) - f(x_0)}{dx} $$
The second derivative is then
\begin{eqnarray*}
\frac{d^2f}{dx^2}(x_0) &\approx&
  \frac{df/dx(x_0) - df/dx(x_0-dx)}{dx} \\
  &\approx& \frac{\left(f(x_0+dx) - f(x_0)\right) -
                  \left(f(x_0) - f(x_0-dx)\right)}{dx^2} \\
  &=& \frac{\left(f(x_0+dx) - f(x_0)\right) +
                  \left(f(x_0-dx) - f(x_0)\right)}{dx^2},
\end{eqnarray*}
i.e. the sum of the local differences, scaled by a distance factor $dx^2$. In a graph Laplacian, you don't have any distances, just graph connections, so it's just the sum of the local differences.
When working on meshes, though, we typically want to take the geometry of the mesh into account. There are many ways to do this, but the most popular seems to be the cotangent Laplacian, where the local difference for each edge is weighted by the sum of the cotangents of the angles opposite that edge.


What does the z-buffer look like in memory?

The Z buffer used to be specialized memory set aside for a single purpose, some web sites still explain it like that, but no longer.
Now the Z buffer is just a chunk of memory you allocate, or an API like OpenGL allocates on your behalf.
The size of that chunk of memory will depend on the type of the Z buffer values, in the example you gave it is a 32bit [floating point] values, but 24 bits is also very common. Always choose the smallest size the program needs as it can have a large effect on the performance of the application. It is indeed multiplied by the size of framebuffer so 8mb is correct for the example you gave.
The values that get stored in it are the depth values for any geometry drawn to the associated framebuffer. It is important to realize that these values are NOT the linear values of the MVP matrix computed in the vertex shader so the Z buffer can not be used for things like shadow maps.
Fragments resulting from each draw call have their depth values tested against existing values in the Z buffer and if the test passes the GPU will write that fragment and update the Z buffer with the new value, if not the fragment gets discarded and the Z buffer is left untouched.
A few other little details:
The Z buffer is generally cleared at the beginning of each frame with a clear value that can be set (or must be set) via the API. This becomes the default value that writes to the Z buffer are tested against.
GPU's have specialized hardware for writing the Z buffer, this hardware can speed up writing to memory by a factor of 2 or more and can be leveraged when creating things like shadow maps, so it is not limited for use with just the Z buffer.
Depth testing can be turned off/on for each draw call, which can be useful.


How is that possible that matrices can be thought as coordinate systems?

If you have a 3x3 matrix representing some transformation, you will actually have the X,Y,Z vectors of that transformation in the rows or columns (depending on if it's a row major or column major matrix).

In other words, if you have a 3x3, you can look at it and immediately get the up, right(*), forward vectors (asterisk due to handedness, it could be the left vector).

3x3 Details:

let's say you have a 3x3 matrix $M$:

$
M_{ij} =
\begin{bmatrix}
M_{1,1} & M_{1,2} & M_{1,3} \\
M_{2,1} & M_{2,2} & M_{2,3} \\
M_{3,1} & M_{3,2} & M_{3,3} \\
\end{bmatrix}
$

Let's see what happens when you multiply that by a vector $\begin{bmatrix}1,0,0 \end{bmatrix}$:

$
\begin{bmatrix}
M_{1,1} & M_{1,2} & M_{1,3} \\
M_{2,1} & M_{2,2} & M_{2,3} \\
M_{3,1} & M_{3,2} & M_{3,3} \\
\end{bmatrix}
*
\begin{bmatrix}
1 \\
0 \\
0 \\
\end{bmatrix}
=
\begin{bmatrix}
M_{1,1} \\
M_{2,1} \\
M_{3,1} \\
\end{bmatrix}
$

Now let's try vector $\begin{bmatrix}0,1,0\end{bmatrix}$:

$
\begin{bmatrix}
M_{1,1} & M_{1,2} & M_{1,3} \\
M_{2,1} & M_{2,2} & M_{2,3} \\
M_{3,1} & M_{3,2} & M_{3,3} \\
\end{bmatrix}
*
\begin{bmatrix}
0 \\
1 \\
0 \\
\end{bmatrix}
=
\begin{bmatrix}
M_{1,2} \\
M_{2,2} \\
M_{3,2} \\
\end{bmatrix}
$

And lastly, vector $\begin{bmatrix}0,0,1\end{bmatrix}$:

$
\begin{bmatrix}
M_{1,1} & M_{1,2} & M_{1,3} \\
M_{2,1} & M_{2,2} & M_{2,3} \\
M_{3,1} & M_{3,2} & M_{3,3} \\
\end{bmatrix}
*
\begin{bmatrix}
0 \\
0 \\
1 \\
\end{bmatrix}
=
\begin{bmatrix}
M_{1,3} \\
M_{2,3} \\
M_{3,3} \\
\end{bmatrix}
$

This shows that whatever matrix $M$ is, the first column is the X axis vector of the coordinate space defined by the matrix.  The second column is the Y axis vector, and the third column is the Z axis vector.

These vectors are in global space.

$
M =
\begin{bmatrix}
X_x & Y_x & Z_x \\
X_y & Y_y & Z_y \\
X_z & Y_z & Z_z \\
\end{bmatrix}
$

You can even see that this is true in an identity matrix, where the X,Y and Z vectors are just what you'd expect them to be:

$
M =
\begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1 \\
\end{bmatrix}
$

A 3x3 matrix is much like a vector in that it describes directions but has no position.

If you have a 4x4 matrix that represents a 3d transform and uses homogeneous coordinates for a translation, you can then use that to get the X,Y,Z orientation vectors, but also can get the translation from the origin.

Hopefully this is helpful, and not misleading for less common matrix types.  I saw an answer deleted which said similar, so fingers crossed :P


Found the solution, it turns out the lightVec is not the vector of light from the tube but rather the direction the tube will point. Therefore i will need to pass it a light rotation value to be used there.

Results:


Is it Possible to send data directly to the GPU via a custom hardware interface?

On a desktop (or laptop) the GPU is connected to the PCIe bus. This gives it direct access to the physical RAM.

If you map the buffer you want to fill and then read a file into that mapped memory. The OS will tell the disk drive to dump the file into the physical ram (the disk also has direct memory access), then when you go to use the mapped buffer the driver will tell the gpu to copy it into the vram. All the while the CPU never touched that bit of ram.


Explanation of a the transparency of an .EXR file

In an EXR, the convention is to store associated alpha, also known as premultiplied. The latter term however doesn't do justice to the nuances of the alpha format as compared to its evil distant cousin known as unassociated alpha, aka straight or key alpha.

In an associated alpha image, two facets are represented via the RGB triplet and its associated alpha channel:

Occlusion.
Emission.

In an associated alpha image, the alpha channel could be considered to represent a distillation of the geometry of occlusion covering a given pixel coordinate. 0.5 would represent a baked down 50% occlusion, while the RGB values would represent the degree of emission.

This also means that an RGBA triplet such as 0.0,12.4,10282.9,0.0 is entirely valid, and represents a pixel that is emitting and unoccluding. Naturally occuring things such as candles represent such a combination. Consider the following image:

The above image could not be created using unassociated alpha, and is symptomatic as to why ray tracing renderers cannot generate any alpha model other than associated alpha.

The fundamental difference between the two alpha formats is defined by the two different over operations, as described by Porter Duff.

Associated alpha over:

FG.RGB + ((1.0-FG.Alpha) * BG.RGB

Unassociated alpha over:

(FG.Alpha * FG.RGB) + ((1.0-FG.Alpha) * BG.RGB

The nuances of the difference mean that care should be taken when dealing with alpha and the RGB channels.


When Sean and I wrote stb_image_resize we chose Mitchell for downsizing. Mitchell is similar to Cubic, you can read about the cubic class of sampling filters in Mitchell Netravali 1988. They are all pretty similar and will get you very similar results.

I can't find any written record between Sean and I of why we decided to go with Mitchell, but if memory serves we just resampled a bunch of images and used the algorithm that we thought looked best. I wouldn't say that there is one authoritative or best filter, you should use the one that looks best on your data.

Edit: Like joojaa says, a windowed sinc filter is also good, if not quite as cheap. You can find some implementations here.


Label Placing for thousands of objects with OpenGL

Since you are new to computer graphics, you may be better off avoiding the complications of SIMD and sticking with the traditional 'create a CPU thread + collect results' approach, or even run in the main thread if the task is lightweight enough.

But if that approach fails to be performant and/or you are willing to dip into OpenCL / GLSL compute shaders, read on...

This is a per-object calculation, not a per-vertex or fragment calculation, so the traditional graphics pipeline is not the best choice. That said, you could lead a vertex buffer with vertex positions at each object world position, then transform the vertices to screen space (see below) in the vertex shader, but considering you are dealing with OpenGL 3.3 you will probably be limited to writing the output label positions to textures via render to texture or transform feedback buffers which may be slow depending on the hardware. If you can use the shader image load store extension, it could help, but bending the graphics pipeline to GPGPU is all probably more work than it is worth.

You have not detailed your overlap requirements (i.e., whether it must be perfect or approximate, overlap probability, whether you can guarantee sufficient screen room to achieve perfect non-overlap, whether you need all labels to be on visible screen at all times, etc.), but generally pairwise problems which can give non-convergent iterations when multiple particles are adjusted simultaneously are best done on a single thread on the CPU. It could be a dedicated CPU thread.

To understand what I mean by non-convergent iterations, imagine a grid of labels whose lower left and upper right corners touch. If they all are adjusted simultaneously in opposite directions of one neighbor, they may end up in a situation where they simply overlap their other neighbor. This cycle may repeat indefinitely since the corner overlaps are opposite. Whether this actually occurs depends on the specifics of the algorithm and the object positions / dimensions, but it is possible unless you can make some object spacing guarantees or some other assumption I am unaware of.

If you have access to GLSL computer shaders or (even better) OpenCL, and you  do not demand perfect overlap, or predict that overlap will be relatively seldom, the following simple SIMD approach may outperform a single-threaded CPU approach.

SIMD algorithm: At the beginning of each frame (or whatever label update freq. makes sense), pass the object world positions, label dimensions, and camera projection + view transform to a per-object + label pair compute shader / kernel to calculate the screen space positions:

vec4 objScreenSpacePosition = camera.projectionViewTransform * object.worldPosition;

Now place the corresponding label somewhere nearby.

Then place a sync fence in the shader / kernel to ensure all screen space positions have been calculated before proceeding. Finally, perform a pair-wise loop with all other labels and adjust the label position / depth to correct for overlap. You may need to repeat the loop multiple times until all labels have no overlap if you demand absolute assurance of zero overlap, or you could cutoff overlap adjustment after a certain number of iterations to ensure it doesn't take too long or run indefinitely (see above about non-convergence).

Note that the adjustment loop is only adjusting the label its compute shader was assigned. To avoid position read/write race conditions, it is best to use a temporary position and fence every time you decide whether to continue the loop. Here is an example:

vec4 objectScreenPos = camera.projViewTrans * object[kernelIndex].worldPos;

//...transform objectScreenPos to desired coordinate
// system if needed; either [0,1] or [-1,1] or whatever
// depending on your projViewTrans and needs of your
// overlap correction algorithm...

label[kernelIndex].pos = objectScreenPos.xyz + someOffset;

fence();

bool overlap = true;
uint maxIters = 5;
vec3 tempPos = label[kernelIndex].pos;
for (uint numIters = 0; overlap && numIters < maxIters; numIters++)
{
    for (each labelIndex)
    {
        // Of course a label overlaps itself!
        if (labelIndex == kernelIndex)
            continue;
        // ...check overlap and adjust position of tempPos...
    }
    label[kernelIndex].pos = tempPos;
    fence();
}

Of course avoiding fencing as much as possible and avoiding cross-kernel talk of any kind is ideal, but for this problem you can't completely avoid it. Still, I'd try turning off the fence in the loop and see what happens to perf and output. =)

There are more clever overlap correction algorithms, most of which are approximate, but this one can in principle perfectly correct overlap and is simple to implement and understand.

A GPGPU or even threaded CPU approach could actually be slower than performing a single-threaded calculation on the CPU, depending on the number of actual objects. This is why OpenCL is ideal for these sorts of problems: it gives you the option of running single or multi-threaded on CPU, or moving to the GPU. You could even switch between CPU and GPU on the fly if the number of objects radically changes over time and you can predict the change ahead of time.

I would probably write this in OpenCL since if all else fails, I could run it in a single thread on the CPU and come as close as possible to guaranteeing perfect non-overlap. Then I could experiment with multiple threads / GPU calculations and profile, profile, profile to my heart's content.

Update - OpenGL 3.3 graphics pipeline approach:

While OpenCL or the OpenGL compute pipeline would be ideal, you should be able modify the above SIMD alg to use the OpenGL 3.3 core graphics pipeline with a bit of extra effort and likely perf loss. Example: During a point draw, pass object positions and IDs as vertex data to a vertex shader that calculates the object screen position. Set glPosition to the object ID and pass the resulting screen position to the fragment shader using the flat modifier to prevent interpolation. Render the result to a 1D framebuffer with length equal to number of labels so that there is a 1:1 correspondence between 'pixel' (i.e. label) and vertex (i.e. object). You now have a 1D texture filled with label positions. You could then read this back to the CPU to perform overlap elimination (with possible guarantee of no overlap), then upload the updated label positions to OpenGL in a uniform buffer array, texture, or whatever works best in your situation. Alternatively you could launch a second vertex shader with label IDs as vertex attributes and perform a non-guaranteed overlap elimination which does not depend on the updated positions of the other labels, then follow a similar pattern from before to render the updated positions to a second 1D framebuffer, and finally use the related framebuffer texture as a label ID indexed lookup table during label rendering to determine the final positions of each of the four billboard corner vertices. The label ID could either be passed in using a uniform or vertex attribute.

Hopefully you are beginning to see how much the GL 3.3 core graphics pipeline limits your options for GPGPU and how much more complicated it makes things. What was before a single compute shader or kernel gets broken into multiple shaders (and possibly multiple draws) where you must abuse the concepts of 'vertex' and 'pixel' to achieve your ends. Achieving good perf with this approach often requires a decent understanding of the graphics pipeline and hardware you are working with. Depending on the hardware, driver, and number of objects, the above graphics pipeline algorithm may not be able to beat simply calculating screen space coordinates and eliminating overlap on the CPU then uploading results to GPU. Hence the reason I suggest you either opt for OpenCL or OpenGL compute, or else stick with traditional CPU threading.


It's easy to work out if you consider not that case but the angle at v3 (if the "cube" were continued past v3). By the time you get to v3, the angle is simply the desired bend angle. (That's not quite right, though: because you've got alpha on the decreasing side, it's 90 degrees minus the desired angle.) You have to split that angle equally among all the segments going along the bend direction, so the angle alpha is simply 90 degrees minus (the desired angle divided by the number of vertices). So for A, where the requested angle is 90 degrees, each vertex gets a third of that angle: alpha is (90 - 30) degrees.

For cube B, if the total bend is 35 degrees, each vertex gets just over 11 degrees, so alpha is just under 79 degrees.

This doesn't generalise to an arbitrary axis, though. It only works for bends like those in your diagram. For a more general (and more useful!) tool, it makes more sense to simply apply a 2D rotation (about the axis) to each vertex independently. Exactly what rotation to apply depends on some design decisions about how the bend should behave when the axis goes through the mesh, and how strong a bend you want to allow. The crucial point is that it's not the same rotation for every vertex: the rotation is proportional to the perpendicular distance from the vertex to the neutral line of the transformation. In your example, the neutral line is the bottom edge of each cube.


Change of speed but no change of size after OpenGL Perspective Projection

When you draw a point with GL_POINTS, it's not a 3D object with a size in world-space: its size is a fixed number of pixels. Because of that, the size of the point is independent of the distance from the camera.

If you want to draw dots whose size depends on distance, you either need to compute the perspective yourself and change the point size for each point (not recommended, because state changes like that are expensive); or use another method to draw them. For instance, you might draw a quad and use the fragment shader to make it a circle (e.g. by discarding the fragment if the world co-ordinate was more than a particular distance from the centre of the quad, or by using a transparent texture with a circle in it).


How do you calculate cos(theta) for diffuse in flat shading?

With flat shading, the normal is the same across the whole triangle (or quad or whatever). There are some assumptions that were made here and two approaches. The latter approach will probably be best for most basic problems (although more work). We don't have the winding order, so we need to calculate the normal based on the vertices.

Average normal approach

The assumption is that each vertex contributes the same amount of weight towards the triangle normal.

We first calculate the average of each:
$$normal = normalize(0,0,1) + normalize(1,1,0) + normalize(-1,-1,1) \approx \frac{(0,0,1)+(0.7,0.7,0)+(-0.58,-0.58,0.58)}{3} \approx (0.04,0.04,0.53)$$
$$normalize(0.04,0.04,0.53) \approx (0.076,0.076,0.994)$$

For Lambert shading:
$$cos(\theta) = normalize(0,3,4) \cdot (0.76,0.76, 0.994) \approx 0.84 $$

Geometric approach (recommended for this problem)

Another assumption Nathan Reed suggested was that each vertex normal holds equal weight, which would make sense in more common use cases. For this, we will want to find the face normals in both winding directions to compare this value to:

Winding order 1

$$normal1 = normalize((B-A) \times (O-A)) \approx (0.29,0.43,-0.85)$$

Winding order 2

$$normal2 = normalize((O-A) \times (B-A)) \approx (-0.29,-0.43,0.85)$$

Now, we can take the dot product between these two vectors and the one we computed above. Whichever one is greater or equal to 0:
$$geometric\ normal1 = normal \cdot normal1 \approx -0.79$$
$$geometric\ normal2 = normal \cdot normal2 \approx 0.79$$

So then,
$$normal = geometric\ normal2$$

For Lambert shading:
$$cos(\theta) = normalize(0,3,4) \cdot (-0.29,-0.43,0.85) \approx 0.42 $$


How could I check the correctness of my result of fluid simulation?

Compare it with someone else's software. Run some standardized test and find out if you get roughly the same answer as others. If you get the same answer, than the probability of having your code right is quite high.

Some tests:

Flow past cylinder. In 2d take rectangular domain, cylinder in the middle, inflow on the left, outflow on the fight and calculate the force on the cylinder. Here is benchmark comparing handful of codes.
Buoyancy flow. Closed box, hot plate on bottom, cold plate on top, hot fluid starts to rise due to the buoyancy force. Here is benchmark.
Rising bubble, benchmark.

But unfortunately it might be quite difficult to compare your code to scientific codes in those benchmarks. I guess you implemented something as SPH or stable fluids which are not made for accuracy but for stability.

Take for example the flow past a cylinder. I would start the test with very small Reynolds number and then measure the force on cylinder as you increase accuracy of your simulation(lower time step, increase subdivision or increase number of particles). Does the force converges to some number? If no, than you have a problem, if yes, than have a look at the benchmark paper and compare your result with others.

This method is very similar technique to one I use for testing my raytracer. I just render test scene with someone else's renderer and compare it with my result. Do they converge to the same result? If yes than I have it right, if no, than I have it wrong.


Why can I see the diffuse lighting effect on a perfect mirror?

Dielectric materials (which is what you get when metalness is 0) don't exhibit a mirror-like effect. Think of a sheet of smooth, non-transparent plastic. Real-life mirrors are panes of glass or transparent plastic covered with a thin layer of metal. Try a white base colour (1,1,1) and full metalness (1) instead.

As Hubble pointed out in his comment, Unity won't do full ray tracing for you out of the box, but it should be able to do some approximation with screen-space reflections and local reflection probes.


Sort half-edges around common vertex in 3d

If projecting them onto the plane of the vertex normal doesn't do the job, I'm not sure what would. There isn't any unambiguous way to do this in 3D. Imagine the new triangle that you are on there joining is perpendicular to the existing two triangles? What would you want the answer to be in that case?
I suppose you could try "voting" by dotting the new triangle's normal with each of the existing triangle's normals. A positive dot is a vote to keep the current orientation, a negative dot is a vote to flip it. Maybe weight the votes by the triangles' areas, or their angles at that vertex. Of course you'd have to decide how to break ties, and this might just give you similar problems to the plane of the common vertex normal.
This would make more sense if you were constructing a new triangle that shared an edge with one of the existing triangles. Then you could copy the orientation from the triangle that it was joined to. I'm not sure what kind of 3D modeling operation you're doing that produces this situation, but if there's any way you can accomplish it by building things in terms of common edges rather than common vertices, you'll probably have a much better time.


How to calculate spatial frequency of an image just in one direction?

You can take a 1D Fourier transform of a row of pixels from the image; it will give you the horizontal frequencies present in that row.
You could sample one row out of the image, or else average all the rows together to get an overall picture of the horizontal frequencies.


Maximum number of vertices after clipping a triangle against an AABB

Funnily enough, I asked this exact question on Math.SE a couple years ago: Maximum number of vertices in intersection of triangle with box.

The answer is 9 vertices, because each of the 6 planes of the box can cut off one corner of the polygon, replacing one vertex with two. So 3 vertices + 6 added vertices due to clipping = 9 total.


A good way to improve sprite rendering efficiency is geometry instancing. This lets you define a mesh using one vertex/index buffer pair, then render many instances of that mesh in a single draw call using a second vertex buffer whose elements define the instances. This is more widely supported than geometry shaders; in particular, it's is supported in WebGL via the ANGLE_instanced_arrays extension.

To apply this to sprites, one vertex buffer could define the shape of a sprite (i.e. a quad made of two triangles), then another vertex buffer could contain per-sprite information such as position, rotation, size, and so on.

If there are 4 vertices per sprite and N sprites, the vertex shader will be called a total of 4N times, onces for each vertex in each sprite. You'd use the vertex shader to put the information from the two buffers together, transforming each vertex appropriately to the sprite's properties. This allows the GPU to do the work of generating all the vertices for the sprites, and saves on data uploaded from the CPU.

Coincidentally, I published a small OpenGL instancing demo recently.

Using instancing this way does require that all the sprites are drawn with the same state, i.e. the same shader, textures, blending mode and so on. If different states are needed for some sprites, a separate batch of instances will be needed for each state combination.


Realistic rendering: which processes of the human eye and brain do I need to consider?

you might want to take it further: render an image as the human eye would capture it or even as the human being would perceive it.

There are two ways to interpret this.  I'll do both.

Interpretation 1: Render an image that looks perceptually realistic.

At the end of the day, your image still needs to be displayed somewhere.  Here's the key: you want to render your image in such a way that when you *display* that image on a particular display device, it will produce the same sensation the original radiometric image would have produced.

Here's how to unpack that idea.

In the real world, radiometric spectra (i.e., real distributions of light) enter your eye and stimulate approximately1 four light receptors.  The stimulations of the receptors produce the sensations of color we associate with images.

In rendering, we don't have arbitrary control over the spectra we produce.  Fortunately, since we (usually) have only three cones, each of which produces only a scalar value, color vision can be reproduced by using exactly three primaries.  The bottom line is you can produce any color sensation by using a linear combination of three wavelengths only (up to a few colors that might have to be negative, in which case, you just use different primaries).

You don't have a choice of primaries.  Almost all color display devices use the sRGB standard, which provides three primaries (which actually usually don't have a single wavelength).  That's fine because it turns out it's all abstracted and you don't have to care.

To clarify the mess that is perceptually accurate rendering, here's the algorithm:

Render your image using correct radiometric calculations.  You trace individual wavelengths of light or buckets of wavelengths.  Whatever.  In the end, you have an image that has a representation of the spectrum received at every point.
At each pixel, you take the spectrum you rendered, and convert it to the CIE XYZ color space.  This works out to be integrating the product of the spectrum with the standard observer functions (see CIE XYZ definition).
This produces three scalar values, which are the CIE XYZ colors.
Use a matrix transform to convert this to linear RGB, and then from there use a linear/power transform to convert linear RGB to sRGB.
Convert from floating point to uint8 and save, clamping values out of range (your monitor can't represent them).
Send the uint8 pixels to the framebuffer.
The display takes the sRGB colors, does the inverse transform to produce three primaries of particular intensities.  Each scales the output of whatever picture element it is responsible for.  The picture elements light up, producing a spectrum.  This spectrum will be (hopefully) a metamer for the original spectrum you rendered.
You perceive the spectrum as you would have perceived the rendered spectrum.

Interpretation 2: Attempt to simulate the end data the human eye might receive for visualization purposes or compensation for LDR displays.

This one has a less useful meaning, I think.  Essentially, you're trying to produce an image that tweaks the way the brain perceives it for fun/profit.

For example, there was a paper at SIGGRAPH this year where they simulated afterimages and color reduction to make images appear perceptually different.  Of course, the only reason they do this at all is because the displays we're working with are all low-dynamic range (LDR).  The point is to simulate the effects someone might see if exposed to a real high-dynamic range (HDR) display as actual image data.

In practice, this turns out to not work very well.  For afterimages, for example, we see afterimages because of a very bright stimulus exhausting color cells.  If you instead try to stimulate the effect with a fake afterimage, it might look kindof similar--but since it's a completely different mechanism, it's not very convincing.

This sort of graphics is actually underexplored in the literature if you want to make a go at it.  The mentioned paper is an example of more-or-less the most state-of-the-art approaches we have.  I think the current consensus, though, is that it is not really worth trying to simulate (at least at this time), since at best you'd only be approximating real vision effects by substituting different ones, and that this doesn't really work.

1Rod+3*cones, the usual case.  Approximate because humans may have as few as zero functional light receptors up to a conjectured maximum of seven (with the highest ever observed being five).


It does look like raytracing to me as well.  It could also possibly be ray marching, which would be easier to make the tubes with since ray marching is all based on distances from things (center of tube in this case).

Refraction is what's used to bend the light that goes through the tubes.

You are then just left with how to model the tubes. One way could be to use 3d bezier curves (:

This may not have been made in real time, but may have been rendered frame by frame and then assembled into the smooth "movie" you are showing.


This is the same problem as the rotation about an axis. What you need is a matrix (any matrix) that satisfies the following corner constraint: one axis must point toward $(-2, 2, 0)$ in the case of a rotation this matrix must be a rotation matrix, in the case of a planar mirror it just has to span the 3D space. But since we know how to make an arbitrary rotation matrix we use the same approach.

Pick any vector (u) that's not parallel to the normal (n)
Take cross product of u and n for b
Take cross product of n and b for a

You now have an arbitrary rotation matrix with one axis defined using vectors n, b and a. Otherwise same as the last question.


What is a common approach to smooth structures in textures?

I don't have enough rep to add a comment so...

You could use a post process based anti-aliasing technique like FXAA or SMAA. Also, if performance is not a priority, you could go for SSAA. Another option could be to write to a multi sampled texture (I'm not 100% sure if that is possible in your case).


When implementing transmission into a path tracer you have to keep in mind that neither BRDF's nor your previous sampling method are gonna cut it. You will need to think about what happens to light on both sides of your surface, which is where BTDF's and, more generally, BSDF's come in (the T and the S stand for Transmittance and Scattering, respectively).

It can be somewhat non-trivial to implement transmission to a path tracer that currently only supports reflection.

While i can't offer help with implementation details i can give you a few starting points and things you might want to read while you embarc on implementing this capabilty on your path tracer.

On refraction.

Pure and/or glossy refraction is a very common way of transmission, it is the type of transmission that happens on glass and probably what you are looking to implement.

Snell's Law.

Light directions in refraction follow a simple mathematical law.

$$n_1\sin(\theta_1)=n_2\sin(\theta_2)$$

this reads as: the sine of the transmitted angle $\theta_2$ equals the sine of the incoming direction $\theta_1$ times the ratio of the "outside" index of refraction and the "inside" index of refraction $\frac{n_1}{n_2}$

The GGX paper.

There is a wonderful paper titled "Microfacet Models for Refraction through Rough Surfaces" that goes over a good piece of the theory behind microfacet models for refraction and presents a really quite good model for it.

Fresnel and Monte-Carlo.

when rendering a material that has both reflective and transmissive components there are a few different approaches that you can take to mix the two.

two main ways come to mind

1. The Monte-Carlo way

Do a "coin flip" with probabilities weighted according to the calculated fresnel value for each rendered point. Depending on the result you do transmission or reflection

2. The recursive branching way

Generate a reflected ray and path trace.
Generate a transmitted ray and path trace.
Mix the two found color values according to Fresnel value for every rendered point.

I really hope this information is of use in solving your issue, and good luck! Great to see new users on the site.


How to simulate elastic material with restoring shape

A good introduction to finite element methods for deformable body animation is Sifakis and Barbič's course FEM Simulation of 3D Deformable Solids from SIGGRAPH 2012.

However, since you say

  I tried with a spring-mass system but the system become quickly instable, moreover it's computational expansive.

you should know that a finite element model is usually more expensive than a mass-spring system. If you're having problems with instability, you should try keeping your mass-spring system but (i) reducing the time step, or (ii) doing implicit integration; see the "Implicit Methods" chapter of the Witkin and Baraff notes.

You can also consider position-based dynamics or projective dynamics, which are fast, stable, and easier to implement than FEM. They may be good enough for your needs, unless you require high accuracy or nonlinear material models.


Hmm, that SDF doesn't look right. It should be much smoother, like this image for instance:

(image from this blog).

One possible issue I noticed in your description:

  My program first generates a mono bitmap at font size 64 (using FreeType), then generates an SDF from the bitmap.

The initial text needs to be rendered at a much higher resolution than the resulting SDF, so you can get subpixel precision in the distances stored in the SDF. For example, in the original Valve paper, they render the initial mono bitmap at 4096×4096 in order to generate a 64×64 SDF from it.


How does directional Gaussian blurring work?

If I understand your question, you are asking how to actually perform said directional blur in code? A Gaussian blur is typically done by sampling your image in all directions around your current point (or if in 2 passes, one vertical and one horizontal which equates to the same thing), with a specific set of weights for the falloff.

For a directional blur you just need to change the shape of your kernel to "not be round", for lack of better words.

Say you wanted to smear the image to the left only, you could just accumulate samples (with whatever weights you want) with

currentPixelUV.xy + float2( -1.f * someOffsetScale, 0.f) * invScreenRes.xy * currentIteration;

Bilinear texture filtering will be required for this to work when your resulting uvs are not hitting pixel centers.

The magnitude of the blur is controlled by how far you sample from your original pixel.


OpenGL Render front and back faces with different shading for 3d modeller

A fragment shader receives the gl_FrontFacing system-defined input value. It will be false if the fragment was generated from a back-facing triangle, and true otherwise. So you can use that in your shader to determine how to generate a color value.


Advice on how to create GLSL 2D soft smoke/cloud shader

Noise functions are definitely your friend here—FBM would be one good candidate. You’re right that it can look too uniform on its own, but if you blend multiple layers of it together, using different speeds / directions for each and maybe distorting their domains a bit, you should be able to get very close to the look of that footage. There’s lots of clever uses of noise functions on Shadertoy; for an example of something like the approach described above, check out this one.


GDDR5 vs. DDR3 for learning GPU programming on a limited budget?

If your goal is to learn GPU programming, it doesn't matter at all whether you have DDR3 or GDDR5 memory. The way you program it isn't going to change based on how fast the memory is. It will affect performance, but if that's not a primary consideration for you, then you don't need to worry about it.

Do make sure that you get a GPU that supports the latest graphics APIs (DX12 and OpenGL 4.5). I think that's essentially all GPUs made in the past 5 years or so.


How does the Modified Phong Lighting Model from the Phong Lighting Model?

The Phong lighting model computes the specular response as the dot product between the mirror reflection direction and the viewing direction, raised to a power.

For example, if $\vec{V}$ is the viewing direction, $\vec{L}$ the incoming light direction and $\vec{R}$ the perfect specular reflection direction for $\vec{L}$, then the specular response is $\text{max}(\vec{V}.\vec{R},0)^p$, where $p$ is the exponent that controls the fall-off of the specular response.

The Blinn-Phong model uses a half vector $\vec{H}$, which is computed as $\vec{H} = \frac{\vec{L}+\vec{V}}{|\vec{L}+\vec{V}|}$, which is then used to compute the specular response as $\text{max}(\vec{H}.\vec{N},0)^p$, where $\vec{N}$ is the surface normal. The idea is that when the eye vector $\vec{V}$ is aligned perfectly with the perfect mirror direction $\vec{R}$, the half vector $\vec{H}$ would be exactly aligned with the surface normal $\vec{N}$. Hence the Blinn-Phong can get away with computing an expensive mirror direction $\vec{R}$ everytime by using the half vector which can model the specular response.


Promoting my comment into an answer since it identified the problem...

The issue is confusion between degrees and radians for the angle parameter. GLM, up to version 0.9.5, was very inconsistent in how it dealt with angles, accepting degrees for some functions and radians for others. This behaviour could be overridden with #define GLM_FORCE_RADIANS before including the header.

In version 0.9.6, radians everywhere became the default. Unfortunately, if you Google 'glm rotate' at the time of writing, the top result is the 0.9.4 doc page, which still describes the parameters as taking degrees, so you need to be aware of which version you are using and its default behaviour in this regard. If you still want to use degrees for your angle variables, you can convert them with glm::radians(angle) before passing them as arguments.


What is the point of having integer-based draw routines in software rasterizer?

The following is really an assorted set of comments:

Why integer? Floating-point hardware is (in general) far more complex than integer and so on many (old) CPUs it (a) might not have been available (because of silicon budget) and/or (b) took considerably more clock cycles to execute than integer operations.

Re "Bresenham's" algorithm: IIRC there is a branch/decision per pixel. On old CPUs (with relatively little or no pipelining) this would have not been a penalty, but with modern, highly pipelined/superscalar  CPUs, each mispredicted branch will be detrimental.
Presumably (as I can't see your code) using floating-point removes that decision from the loop but, then again, you could use fixed-point (i.e. integer) to get the same effect.

Having said this, hardware rasterisation is probably using fixed-point (i.e. integer) maths when filling polygons as this (a) makes conforming to the OpenGL fill rules easier and (b) prevents some very nasty problems occuring with long/thin triangles.


How to project a 3D point onto a plane along another (axis) vector?

You can determine x by calculating line-plane intersection. Your line starts at P and has direction D=[1, 0, 0], and plane T=[V, N]. This can be done as follows:
$$x=\frac{(V-P)\cdot N}{N\cdot D}$$
x is the distance from P along D to the intersection point with T (assuming both D and N are unit vectors)


Do animation studios like Pixar, Dreamworks, and Disney use super-sampling?

Super-sampling for ray-tracing doesn't just mean rendering an area at higher resolution and then downscaling the finished image: in fact, this simplest super-sampling technique is really not effective enough for high-quality production rendering.

In a production ray-tracer, super-sampling means sending more than one primary ray per pixel (16 primary rays per pixel is a good average) and then using signal reconstruction techniques (i.e. not just averaging) to compute a colour for each pixel of the image. Using lots of primary rays doesn't just help to avoid aliasing (jaggies) from when polygons partially cover pixels; it also helps to smooth over noise added in shading, such as when sampling global illumination or fuzzy reflections.

In addition, super-sampling by sending more rays lets you jitter the sample positions to avoid the artefacts you get from regular sampling. It also lets you use the results of previously traced primary rays to decide where to put more samples.

For example, the simplest adaptive sampling algorithm is contrast based: within each pixel, you measure the contrast among all of the primary rays. Pick the pixel(s) with the highest contrast, and shoot more primary rays in those pixels. You can also use the contrast between samples to choose where in the pixel to shoot new primary rays, but then you have to use a more complex reconstruction algorithm to avoid bias in your result.

If you can find a frame-grab of the particular details you noticed, it would be possible to go discuss those in more detail. It's entirely possible that some artefacts are caused by a particular technique, and aren't related to super-sampling at all.


translate coordinates relative to line segment

I got a good answer on StackOverflow: https://stackoverflow.com/questions/41701713/translate-coordinates-relative-to-line-segment/41702475#41702475 from https://stackoverflow.com/users/1579780/giladd:

function perpendicular_coords(x1, y1, x2, y2, xp, yp) {
  var dx = x2 - x1,
      dy = y2 - y1;

  //find intersection point
  var k = (dy * (xp-x1) - dx * (yp-y1)) / (dy*dy + dx*dx);
  var x4 = xp - k * dy;
  var y4 = yp + k * dx;

  var ypt = Math.sqrt((y4-y1)*(y4-y1)+(x4-x1)*(x4-x1));
  var xpt = distance(x1, y1, x2, y2, xp, yp);
  return [xpt, ypt];
}

// Distance of point from line
function distance(x1, y1, x2, y2, xp, yp) {
  var dx = x2 - x1;
  var dy = y2 - y1;

  return Math.abs(dy*xp - dx*yp + x2*y1 - y2*x1) / Math.sqrt(dy*dy + dx*dx);
}


setting up integer texture and binding as image pt. 2

I use sampler objects to set texture filtering not glTexParameteri functions.

No, you do not. Image Load/Store doesn't use samplers, so you cannot be using sampler objects to set the texture's filtering, when you're using the texture with Image Load/Store.

Sure, Image Load/Store doesn't use the built-in sampler data stored in a texture object either. However, those parameters do have an effect. They can determine if a texture is complete.

And whether you're using those parameters or not, if a texture is not complete, you can't access the texture from the GPU. Among the texture completeness rules is that the texture's sampling parameters must be valid, with respect to the texture's image format.

The default filtering parameters for textures are some form of LINEAR filtering; OpenGL does not change these defaults based on the image format. Therefore, a fresh texture with an integer image format will be incomplete unless you explicitly set the mag/min filters to GL_NEAREST (or NEAREST_MIPMAP_NEAREST for min).

You can get away with what you have when using the texture with sampler objects because their parameters override the completeness check. But as previously mentioned, Image Load/Store doesn't care about the bound sampler objects. Therefore, the completeness check must look at the texture's sampling parameters.

This is regardless of the fact that Image Load/Store itself doesn't actually use those parameters. So it's pointless, but you've gotta do it.


Paint pixels on the image around the cursor for painting application

I don't know, why you consider your first approach hacky, but an alternative would be to analyze the shape of your brush and how it interacts with your underlying data structure. For example for your rectangle:
Let's assume the brush's sides are parallel to the sides of the image. Then you can calculate how man pixels are covered by your offset d. Let's call be the number of pixels between two corners n. Then you can determine the index of the upper left pixel of your rectangle and fill the next n indices since they represent the left border of your rectangle. Then add h to the index of the upper left point to get to the pixel right of it. Fill the next n pixels. Repeat this n times and you have finished filling your rectangle. Here is some untested code that should demonstrate the approach:
float left_x = cursorNorm.x - d;
float upper_y = cursorNorm.y - d;
float lower_y = cursorNorm.y + d;

int index_upper_left = getPixelIndex(left_x, upper_y, w, h);
int index_lower_left = getPixelIndex(left_x, lower_y, w, h);

int n = index_lower_left - index_upper_left;

for (int i=0; i < n; ++i)
    for (int j=0; j < n; ++j)
        int index = index_upper_left + j*h + i
        pixels[index] = {255, 0, 0, 255};

Since this is all symmetric, you can vary this to start at the center point, run i and j just up to n/2 and do something like that:
for (int i=0; i < n/2; ++i)
    for (int j=0; j < n/2; ++j)
        int index_left = index_center - j*h
        int index_right = index_center + j*h

        pixels[index_left + i] = {255, 0, 0, 255};
        pixels[index_left - i] = {255, 0, 0, 255};
        pixels[index_right + i] = {255, 0, 0, 255};
        pixels[index_right - i] = {255, 0, 0, 255};

More complex shapes like circles get a little bit more complicated but you can calculate for each affected column of pixels the index of the upper and lower affected pixel and fill between them. Remember that you can abuse symmetry here. Meaning if you found the upper pixel of a column, the lower pixel has the same y distance to the y-value of the center point. Additionally, the bounds to the left and to the right of the center are also identical. A helpful algorithm to find the bounds might be Bresenham’s circle drawing algorithm.


How to shift color values of a single RGB channel

I don't have any direct experience doing this so I might be missing an obvious solution or tool. That said, what you describe is in programming terms comparatively easy to achieve. The basic structure of such a custom processing would be:

Open the image file and get access to the array of pixels it contains.
Iterate over all the pixels and inspect/transform however you want.
Write out the resulting modified image in the desired format.

Both Gimp (1, 2) and Photoshop (tutorial) provide scripting or plugins support which will allow you to provide custom logic for step 2 and handle the rest for you. I don't have any experience with either, but from a very quick overview, I recommend that you use Javascript, Python or another scripting language which will be much more accessible as a new programmer than a language like C.

Another option I found outside of an image editor that looks good is the Python Imaging Library. Here is a very relevant other question on SO: https://stackoverflow.com/questions/36468530/changing-pixel-color-value-in-pil

From it, I was able to write the following Python script that does what you asked:

from PIL import Image

img = Image.open("input.png")
pixels = img.load() # create the pixel map

# configuration
channel = 0 # 0 = red, 1 = green, 2 = blue
change_amount = 2 # can be -2 for decrease
limit = 178

for i in range(img.size[0]): # for every pixel (in 2D):
    for j in range(img.size[1]):
        if pixels[i,j][channel] < limit:
            # Only modify pixels below the set limit.
            # The following "strange" code is used because Python tuples can't be modified
            # directly. The tuple is transformed into a (mutable) list and moved back again
            # to a tuple afterwards. You can safely ignore this and concentrate on the
            # second line that actually modifies a channel's value.
            color = list(pixels[i,j])
            # Increase (or decrease for a negative change_amount value) the chosen channel.
            color[channel] += change_amount
            pixels[i,j] = tuple(color)

img.save("out.png")

This is untested code, but it shows how to translate what you want into code. You already described what you wanted in a very programmer-friendly manner so the translation is almost direct. The sky is the limit then as to what you can do with pixels once you load them in a custom program to manipulate them.


Raytracer help - Background color writing over some colors?

In Lambertion::scatter there is a comment about attenuating light based on distance.
For a raytracer this is not required as the distribution of rays themselves cause attenuation over distance.


Why is precalculating a distortion different from calculating it in the shader?

I finally found the problem.

My precalculation was done with pixel coordinates from 0 to res[0] - 1 (i.e. the resolution width):

for (int i = 0; i < res[0]; i++)
{
    for (int j = 0; j < res[1]; j++)
    {
        reinterpret_cast<float*>(img->data(i, j))[0] = i / res[0];
        reinterpret_cast<float*>(img->data(i, j))[1] = j / res[1];
    }
}

However, pixels in OpenGL are accessed via their center, as can be read in another SO answer or in Real-Time Rendering (3rd and 4th Edition, p.21). Meaning, that if I have a pixel with 0 as its left border and 0.1 as its right border, then 0.05 is its center.
That means in my case, I have to offset the precalculated texture coordinate by half a pixel to read out of my prerendered image:

for (int i = 0; i < res[0]; i++)
{
    for (int j = 0; j < res[1]; j++)
    {
        reinterpret_cast<float*>(img->data(i, j))[0] = i / res[0] + 0.5 * pixelWidth;
        reinterpret_cast<float*>(img->data(i, j))[1] = j / res[1] + 0.5 * pixelHeight;
    }
}


How does the driver know what to update/transfer and when?

It doesn't. There is no transfer or update in that fashion.

All memory allocated through a Vulkan device represents memory that some device operations can read directly. If a Vulkan device advertises that it can use visible/coherent memory as source vertex data for rendering operations, that does not mean that some kind of upload is going on to make this happen. The device is reading that data from the memory you write to. The vkFlushMappedRange function merely expunges writes from CPU cache lines, so that they actually reach the memory.

Memory that is host-coherent and host-visible would represent memory that either:

Is not cached by the CPU, and therefore the GPU sees the memory as it is. This also means that CPU writes and reads tend to be slow.
The GPU can directly access the CPU's caches, to one degree or another.

Usually, #2 happens when the GPU is on-die with the CPU. Which would also mean that this would be device-local memory. Since you've said that it isn't device-local, odds are good that it is simply not cached. It may use write-combining (which improves the efficiency of sequential writes), but CPU writes otherwise are delivered as they are. Given that the memory does not seem to include "cached" in its description, I'd say #1 is the most likely.

Note that just because you don't need to explicitly flush memory does not change the fact that you still need a dependency between the CPU operation writing that data and the GPU command(s) that read it. This would generally be some from of event with a memory dependency.


Do occlusion queries respect the stencil test at all?

The specification has been clarified on this point, as has the Wiki article. So let's explain that.

The new wording in the spec says:

  the samples-passed count is incremented for each fragment still being processed after the depth test

This is better wording, as it makes clear that the order in which things are processed matters. OpenGL defines the depth test as happening after the stencil test, scissor test, etc. So this wording clearly states that the depth test, and everything before it, matter.

So it does respect the stencil test. And fragment shader discard, since the FS is specified to come before all of these tests.

Well, unless you force these tests to happen before the fragment shader. If you do that, then discarding the fragment in the FS will not prevent that fragment from being counted by the occlusion query. So if you enforce early tests, you lose the right to use discard to affect the occlusion query count.


Spherical Harmonics Diffuse CubeMap. How to get coefficients?

So at start you have samples from your cube map. Each sample has color and normal (dir) at which you sampled that color.
This is how I do it. I use coeffs from this paper (the same you linked), there are values for the first 9 of them.

So for constructing for each sample you:

Compute your SH basis using normal from sample

float Y00     = 0.282095;
float Y11     = 0.488603 * normal.x;
float Y10     = 0.488603 * normal.z;
float Y1_1    = 0.488603 * normal.y;
float Y21     = 1.092548 * normal.x*normal.z;
float Y2_1    = 1.092548 * normal.y*normal.z;
float Y2_2    = 1.092548 * normal.y*normal.x;
float Y20     = 0.946176 * normal.z * normal.z - 0.315392;
float Y22     = 0.546274 * (normal.x*normal.x - normal.y*normal.y);

Then multiply these coeffs by your color (L)

vec3 L00   = L * Y00;
vec3 L11   = L * Y11;
vec3 L10   = L * Y10;
vec3 L1_1  = L * Y1_1;
vec3 L21   = L * Y21;
vec3 L2_1  = L * Y2_1;
vec3 L2_2  = L * Y2_2;
vec3 L20   = L * Y20;
vec3 L22   = L * Y22;

And thats it, you have your coeffs. Now, you can store them in some data structure. You accumulate each of them like:

someArray[0] += L00;
someArray[1] += L11;
etc...

And you divide these accumulated coeffs by a number of samples;

Then when reconstructing :

compute your SH basis one more time for normal at point of the scene.
for approximating diffuse irradiance at given point of scene,
convolution with cosine lobe is performed using normal from that point.
Convolution means, for each SH order, multiplication by $
   \hat{A}=[\pi, \frac{2}{3}\pi, \frac{\pi}{4}]$

$\hat{A_0} = 3.141593$,
$\hat{A_1} = 2.094395$,
$\hat{A_2} = 0.785398$

L coeffs are sampled from data structure.

vec3 color = A0*Y00*L00
             + A1*Y1_1*L1_1 + A1*Y10*L10 + A1*Y11*L11
             + A2*Y2_2*L2_2 + A2*Y2_1*L2_1 + A2*Y20*L20 + A2*Y21*L21 + A2*Y22*L22;

Disclaimer: This basically works (for me) but I am not an expert and something might be missing or not perfect because this IS confusing topic...


How much precision (half, float, double, etc) is enough for a Color class?

Colors shown on your display or saved to standard image file formats use 8 bits per component. So to store these colors it suffices to use four bytes (unsigned char). These colors are usually using sRGB color space, which includes a non-linear "gamma" transformation that redistributes precision to make it somewhat more perceptually uniform, so it's actually pretty close to the limit of how finely humans can perceive color differences already. (However, the sRGB gamut is only a subset of all the colors that are physically perceivable. Wider gamuts need more bits.)

However, if you're generating or processing colors in graphics software (and not just loading/storing them), there are various reasons you might want to use more precision.

Performing operations on colors, such as adjusting brightness and contrast, alpha blending, gamma correction, etc all tend to lose precision. You might want to store colors as 16-bit or more internally to give yourself more precision headroom. Otherwise, repeated rounding off to 8 bits after each operation can cause a progressive loss of quality.
Similarly, when working with linear colors (not the sRGB gamma-encoded colors) for lighting math, extra precision is needed to ensure you have sufficient precision when you eventually convert back to sRGB.
It may be faster and more convenient to work with floating-point colors instead of integer ones. This is especially true on GPUs, where integer math instructions have only a fraction of the throughput of float instructions. But even on CPUs it's certainly easier and probably faster to convert colors to float, do a bunch of operations, and then convert back to integer, rather than trying to do everything with integer fixed-point math.
If you do HDR rendering at all then you'll need more than 8 bits of precision to handle the larger color gamut and intensity range. The new HDR displays just starting to come out accept images with 10 or 12 bits per component.

Using double for colors is massive overkill, but using float for internal representation of colors is common for all the reasons above. When working with GPUs, half (16-bit float) is also common, as they support that format in hardware.


(Ray tracing) Trouble converting to screen coordinates, objects being stretched

It looks like the parameters to your camera class are not correct
 const int img_ratio = 2;
 ...
 Camera<float> cam1(20.0f, img_ratio, Point3<float>(0.0f, 0.0f, 0.0f), Point3<float>(0.0f, 0.0f, -1.0f), Vec3<float>(0.0f, 1.0f, 0.0f));

In your code you set img_ratio to 2, I would expect it should be something like img_width/img_height (or the reverse img_height/img_width). img_ratio should also be also be a floating point to match the  type the camera class is being instantiated with.


Floating Point Operations in Line Drawing Algorithms

is it important with modern GPU architectures to avoid floating point operations in favor of integer operations?

Almost certainly not. CPUs have a long history where for many years they only handled integers natively and floating point operations were all done with slow math libraries. That changed about 20-25 years ago, but the thinking has remained entrenched for many years. (And certain applications like finance may require it for other reasons.)

Modern GPUs, though, have always been about doing lots of floating point calculations in parallel. If you think about how most 3D rasterizers (like OpenGL, Metal, DirectX, etc.) work, they take all the vertices, transform them into normalized device coordinates, and then interpolate between them to draw the actual geometry. While you certainly could do that all with fixed point math, it probably wouldn't gain you much. But you also aren't doing those calculations yourself, unless you've written your own rasterizer as a compute kernel.

Now depending on what type of work you're doing, the precision of the floating point operations may be reduced. In a language like glsl (in OpenGL ES, anyway) you can specify the precision of the uniforms. On mobile hardware, it might be worth doing the calculations as floating point, but with only 16 or 24 bits instead of 32 because it saves power,   for example. (And even some desktop GPUs have used lower-precision floating point in the past.) But the desktop hardware of today is optimized for doing floating point work.

In short, I don't see a compelling reason to stick to integer math when working on the GPU.


When to disable perspective correct interpolation ( noperspective )

Use cases are only limited by your imagination! noperspective means that the attribute is interpolated across the triangle as though the triangle was completely flat on the surface of the screen. You can do antialiased wireframe rendering with this: output a screen-space distance to the nearest edge as a noperspective varying and use that as coverage in the pixel shader.

Or if you're doing non-photorealistic rendering and want a pattern in screen-space like halftoning, you can enable noperspective on your UVs used for texturing.

Does it make a performance difference? Probably, but you probably won't notice (with the potential exception of less powerful graphics hardware). Most GPUs are composed of a series of pipeline stages that execute in parallel, and in some sense you only pay the cost for the most expensive stage. If rasterization is the most limiting part for you, then you may see a difference from the divisions that you're skipping per-pixel. I would guess that is most likely when rendering a shadow map or a depth prepass, but those also have the fewest attributes to interpolate.


Albedo image map to color's Light Reflectance Value (LRV)

I am not aware of what LRV is and what is a usage of it. However, I can recommend using luma instead of simply averaging RGB values with equal weight:

https://en.wikipedia.org/wiki/Luma_(video)
https://en.wikipedia.org/wiki/Relative_luminance
Example of using luma - http://blog.simonrodriguez.fr/articles/30-07-2016_implementing_fxaa.html

Human eyes are more sensitive to green colors [1], that's why 16 bit (aka high color [2]) palettes, might prefer to use R5G6B5 format.
[1] https://en.wikipedia.org/wiki/Color_vision
[2] https://en.wikipedia.org/wiki/High_color


I don’t think there’s a formally established name for it, but it’s generally known as distortion, warping, or refraction. It works by taking an already rendered version of the current frame, sampling it with warped/stretched texture coordinates, and, usually, compositing the result back in with the rest of the scene. One of the first games I know of that used it was Half-Life 2, for an effect when the Strider enemies fired their cannon:

For an example of what this looks like in a modern game engine, check out Unreal’s documentation.


Solution: Check for the minimum x,y,z value for all points. Add these values multiplied by (-1) to all points in order to guarantee that all points are not negative. Then take the largest point and subtract with the minimum point (max and minimum x y z) in order to get a cube-like bound for the object.

Then divide this cube by an arbitrary size constant representing the radius of each particle. Now you should have a 2D array for z-values, such that for any x, y position you can quickly store z-values that you find intersecting lines.

For each triangle in the mesh, check all possible that have a large or small enough x-y value for intersections with a line going through the whole cube. If there is an intersection, save the z-value in the x-y array for z-values.

Lastly loop through each possible z-values in the imaginary cube and check if it has an odd number of z-value hits before the current z-value. If that is the case, then we know that there is a particle here. Don't forget to add the minimum x,y,z values that we added in the beginning in order to make the particles have the correct positions again.


How can I get a spherical visibility mask map for a point on a concave surface?

This can be done analytically at least for polygonal meshes. You can convert points into polar coordinates and project on a sphere. Edges form planes that pass through projection circles center forming circles on the sphere. These lines are all great circles, because they must pass sphere center. Great circles form linear interpolations in polar coordinates.

Image 1: Projecting a geometry on a sphere.

Converting these great circles into either Cartesian coordinates is quite well known math. Projecting them onto 2d representation by Mercator projection is well known math on account fo this being central to map making. Since overlap is not a  problem you can just slap these triangles on top of each other and merge the vector results. Or use the polar coordinates for pixel graphics and even use z buffered overlap just like a normal camera.

The only problematic point is the one where your sphere is generated (if its on a open face edge). You can simply project the point on center into its inverse normal direction on the projection sphere. Or if for example Mercator projection is desired, you can align the sphere with surface normal. then just project them to a point on the south pole.

You can even interpolate the results of individual polygons positions to get any point on the triangle. And a analytic surface coverage for the triangle.

I haven't implemented this, but Ive done it manually a few times* so i know its possible. Should be pretty easy to do as the maths involved aren't all that complicated only problem is solving which side of the problem to keep.

Image 2: Horizon map of a simple all local geometry.

Comparison with raytracing

Both sampling triangles analytically is O(N) algorithm, if vector data is sufficient. Conditioning this data may push you over to O(N^2). Raytracing is O(N log(N)). Raytracing algorithms are easier to find of the shelf so implementing analytic polygon to sphere rendering is harder. In terms of speed analytical rendering is faster as long as theres a limited number of triangles, much like how scanline rendering is still sometimes faster than tracing despite being algorithmically more complex. This similarity is not just a coincidence it can be shown that this method can be turned into a scanline renderer.

Resources for drawing great circles in Mercator projection:

Wolfram demonstration on subject.
The Projection of Circles on a Mercator Map

* About 20-25 times. I also did it for this image (yes in 3d), its as accurate as a single span Bezier curve can represent a circle. Ive also done this in map making software a few times.


How to check which API's are available on a given machine?

Basic Vulkan availability can be checked by the presence of the loader dynamic library. This will reside in a standard place where you can load it with dlopen or LoadLibrary.

If it fails to load then vulkan is not installed. If it does load then you can get the vkGetInstanceProcAddr function pointer from it with dlsymor GetProcAddress. After that you can query the devices as normal and decide whether or not it's sufficient to support your app.


Fonts rendered through textures look thinned

From looking at your example images this looks like a case of Alpha blending being applied to the RGB channel of the font-atlas texture and then again applied via the A channel when rendering.

The tell-tale sign of this is usually a feint dark halo surrounding your glyphs which is caused by the unwanted mid-greys present in the texture being multiplied by alpha.

For font rendering it is typical to have anti-aliasing in the A channel and have your RGB channels a constant colour.


A prism is a polyhedron with a polygonal base, another parallel base of the same shape, and joining faces between corresponding edges of the two base faces.

The word "prism" is commonly used to describe a right prism with base faces perpendicular to the joining faces. In this case the joining faces are also rectangles, and so a rectangular right prism is simply a cuboid.

A cuboid can be described directly, but a simple way to generate one is to start with a cube and use directional scaling to give the edges the required lengths.

If the base faces are not perpendicular to the joining faces, this oblique prism can be achieved by applying a shear to a cuboid with the required edge lengths.

So all you need is a parametric description of a cube, and you can go from there to any rectangular prism required.

As joojaa points out, there isn't an easy way to describe a prism (including a cube) in just a single parametrisation. You will need to describe each face with a separate set of parametric equations. For example, using parameters u and v, you could describe one of the faces as follows.

$$  x=u  \\  y=v  \\  z=1  \\  0 \leq u \leq 1  \\  0 \leq v \leq 1  $$


A lot goes in to benching performance. I think most would agree that measuring the frame time of your application is the best metric you can gather about how well you're doing at run-time. An easy way to "bench" your shaders in your engine/environment is to simply record frame times for a fixed duration with a fixed view and fixed scene. Then save those frame times out to a CSV and plot them in something like Excel. This should give you a good idea of how much your frame time is fluctuating and if you're actually hitting your frame time targets most of the time.

However if you want to see if your shaders are properly utilizing the GPU you may want to dig into your application with something like AMD's GPU Perf Studio or Nvidia's NSight tool. I'm more familiar with Perf Studio but it can give you a static shader analysis to see if you're actually taking full advantage of the GPU or if you're overloading certain registers.


When does it make sense to use drawRangeElements

The problem glDrawRangeElements was invented to fix was effectively removed by the advent of vertex buffers.

What glDrawRangeElements allows the implementation to do is to know exactly what range of values will be used as indices for the vertex arrays. Given that knowledge, they can read that range of values into a separate buffer, and then render from that buffer. Without knowing the range of indices, that wouldn't be possible, since an index list could pick from any index.

But that's only useful if you're reading from client memory. Copying that range out of client memory allows you to render asynchronously. You copy out the values you know you need, issue a rendering command with that, and then return to the caller.

If you're reading from buffer objects, this isn't a useful thing to do. And since core OpenGL doesn't even allow you the option of using client memory, the Range commands are no longer useful.


Let's say you have your two points that define the line segment: $A$ and $B$, and a point $P$ that you are testing to see if it is on the line segment.

Firstly, you get a normalized vector from $A$ to $B$, as well as the distance from $A$ to $B$.

direction = normalize(B - A);
length = length(B-A);

Next, you use dot product to project the point $P$ onto the line segment defined by $A$ and $B$, by first getting the vector from $A$ to $P$ and then dot producting that against the direction vector.

PRelative = P-A;
projection = dot(PRelative, direction);

This projection value represents how far down the line segment from $A$ to $B$ that the closest point to $P$ is.  It may be a negative distance, or it might be farther from $A$ than $B$ is, so you next clamp this value to the line segment.

projection = clamp(projection, 0.0, length);

Note that in your case, instead of clamping, you could just detect when the projection was out of bounds and return false at this point as well.

Continuing on, you now have the distance from $A$ to $B$ that the closest point to $P$ is, as the value in $projection$.  You calculate that actual point:

closestPoint = A + projection * direction;

If $closestPoint$ == $P$, you know that $P$ lies on the line segment defined by $A$ and $B$ because the closest point on that line segment is the point $P$ itself.

However, in practice, you might run into some numerical precision problems due to having limited accuracy in computer calculations.

To get around this, instead of checking if $closestPoint$ and $P$ are the same, you should probably instead consider them the same if the distance between them is smaller than some threshold value.  This way, if precision issues come up, you'll still get the answers you want in the cases you want them.

Too large of a threshold value, and you will get points being considered on the line that you really wouldn't consider on the line (if this happens, make your threshold smaller).

Too small of a threshold value, and some points that you would consider being on the line will not register as being on the line (if this happens, make your threshold larger).


In a shader, how to store fragment coordinates in a spatial partitioning from one pass to the next?

Though I have never used them myself OpenGL in modern versions gives you something called "Shader Storage Buffer Object" These are buffers that you can fill with your data. They are guaranteed to be able to hold up to 16 MB of data and most implementations seem to have no problem with them taking up the whole GRAM.

This feature is core since OpenGL version 4.3.
The SSBOs are basically just a buffer that you can put any datatype in (a struct for example) and a huge amount of them.

I cannot really tell you how to use them, but I think with the name you will find some resource in the web.