The stencil buffer definition by Wikipedia is indeed not great, it focuses too much on the details of modern implementations (OpenGL). I find the disambiguated version easier to understand:

  A stencil is a template used to draw or paint identical letters, symbols, shapes, or patterns every time it is used. The design produced by such a template is also called a stencil.

That's what stencil meant before Computer Graphics. If you type stencil on Google Images, this is one of the first results:

As you can see, it is simply a mask or pattern that can be used to "paint" the negative of the pattern onto something.

The stencil buffer works in the exact same way. One can fill the stencil buffer with a selected pattern by doing a stencil render pass, then set the appropriate stencil function which will define how the pattern is to be interpreted on subsequent drawings, then render the final scene. Pixels that fall into the rejected areas of the stencil mask, according to the compare function, are not drawn.

When it comes to implementing the stencil buffer, sometimes it is indeed coupled with the depth buffer. Most graphics hardware uses a 1 byte (8 bits) stencil buffer, which is enough for most applications. Depth buffers are usually implemented using 3 bytes (24 bits), which again is normally enough for most kinds of 3D rendering. So it is only logical to pack the 8 bits of the stencil buffer with the other 24 of the depth buffer, making it possible to store each depth + stencil pixel into a 32 bit integer. That's what Wikipedia meant by:

  The depth buffer and stencil buffer often share the same area in the RAM of the graphics hardware.

One application in which the stencil buffer used to be king was for shadow rendering, in a technique called shadow volumes, or sometimes also appropriately called stencil shadows. This was a very clever use of the buffer, but nowadays most of the rendering field seems to have shifted towards depth-based shadow maps.


Workflow of the Naive Surface Nets algorithm

I think the basic idea, as explained in the article to which you linked, is to associate a vertex (shown in light blue) to each square that intersects the surface, with the vertex positioned at the average of the adjacent edge crossings (shown in gray). Then connect up with dual edges (shown also in light blue).


What is the difference between a point transformation and a vector transformation?

Here's the simple answer.

In 4D, to be able to multiply them by a 4x4 matrix, vectors are represented as (x,y,z,0) and points are represented as (x,y,z,1).

Since the 4th row of a 4x4 matrix represents the translation of the matrix, the above representations make it so points are affected by translation, but vectors are not.

Both vectors and points are affected by the rotation, scaling, etc though.

Caveat:

There is deeper discussion to be had if you expect the vectors to have certain properties.  For instance, if you transform a triangle's normal by the same matrix you transform the triangle's vertices, it likely won't actually be the normal vector of that triangle anymore. This is because normal vectors have a sort of inverse relationship to the vertices that they are calculated from.


Can we use anisotropic filtering on 3D textures?

I am not aware of hardware support for 3D anisotropic filtering.  I could be wrong about its existence though.  I believe it has been tried.

The motivation for 2D anisotropic filtering is to prefilter the texture function over the screen area it falls under in a more accurate way than doing a box filter in texture space (usually, anisotropic filtering does several box filters in texture space).

This definition is key.  2D anisotropic filtering tries to:

  Integrate over the part of a flat surface that you can see(within a single pixel on the screen).

What does 3D anisotropic filtering mean?

  Integrate over the part of  a . . . volume? . . .  that you can . . . see?(within a single . . . voxel? . . . on the screen)

The best analogue I can think of would be doing some sort of 3D texture lookup anisotropically along the viewing ray.  This isn't quite a direct analogy.  Anyway, the problem with this is that when you're doing volume rendering, you actually care about how this integration is done a lot more than in the 2D case.

Do you want absorption?  Emission?  (You need to do some kind of exponential to integrate.)  Do you want scattering?  (You need to do some kind of recursion or approximating blur to integrate).  And this doesn't even get into what kind of data the 3D texture stores.  Does this texture mean opacity, or does it mean density?  Is it diffuse or is it emission?  All of these need to be integrated differently.

New developments in volume rendering are happening all the time, and in any case there are so many variously suboptimal ways you could try to generalize 2D anisotropic filtering, the doubt expressed by the specification makes sense to me:

  Does the implementation . . . for 2D anisotropic texture filtering readily extend to 3D anisotropic texture filtering?

No.


Image saved as DDS with 16f format in photoshop has strange FOURCC text

The tool seems to be generating an unofficial extended version of DDS in which the FOURCC code is replaced by a value from the D3DFORMAT enum. The code 0x0000006F translates to decimal 111, which translates to D3DFMT_R16F.

The Microsoft DDS documentation notes that this is seen sometimes, although not recommended: DDS Variants

  There are some common variants in use where the pixel format is set to a DDPF_FOURCC code where dwFourCC is set to a D3DFORMAT or DXGI_FORMAT enumeration value. There is no way to tell if an enumeration value is a D3DFORMAT or a DXGI_FORMAT, so it is highly recommended that the "DX10" extension and DDS_HEADER_DXT10 header is used instead to store the dxgiFormat when the basic DDS_PIXELFORMAT cannot express the format.


Do stencil tests use any acceleration techniques?

I don't have a definite authoritive answer for you but I'm very confident that the answer is "yes". Here's why:
When using framebuffer objects, stencil attachments are usually combined with the depth attachment in a single texture (GL_DEPTH24_STENCIL8 or GL_DEPTH32F_STENCIL8). In fact, until recently, a combined depth and stencil was the only way to get a stencil buffer. It is now possible to create a stencil only attachment for an FBO.
This OpenGL wiki article on FBOs states (emphasis mine):

These color formats can be combined with a depth attachment with any of the required depth formats. Stencil attachments can also be used, again with the required stencil formats, as well as the combined depth/stencil formats. However, implementations are only required to support both a depth and stencil attachment simultaneously if both attachments refer to the same image.
This means that if you want stencil with no depth, you can use one of the required stencil formats. If you want depth with no stencil, you can use one of the required depth formats. But if you want depth and stencil, you must use a depth/stencil format and the same image in that texture must be attached to both the depth and stencil attachments.

All this strongly hints towards an accelerated storage format for stencils similar to the one used by depth buffers.
Also, empirically, over the years many rendering algorithms (e.g. stencil shadows) using the stencil buffer would have been prohibitively slow if stencil buffers did not use some form of accelerated structure.
The NV_packed_depth_stencil extension (2001) spec also provides interesting insight:

Many OpenGL implementations have chosen to interleave the depth and stencil buffers into one buffer, often with 24 bits of depth precision and 8 bits of stencil data.

and (emphasis mine)

Should the depth/stencil format support other standard types, like GL_FLOAT or GL_UNSIGNED_INT?
RESOLVED: No, this extension is designed to be minimalist. Supporting more types gains little because the new types will just require data format conversions.  Our goal is to match the native format of the buffer, not add broad new classes of functionality.


Calculating the angle between two polygons

I suggest computing the normal vector for each polygon and compute the angle between the normal vector of one triangle and the normal vector of a plane perpendicular to the other surface that passes through the common edge.

See the following illustration, where the red arrows are the normal vectors of the red polygons. The blue vector is the plane perpendicular to the left polygon that passes through the edge common to both polygons.

If your polygons are planar, then you can compute the normal for any polygon ABC by computing the cross product AB x AC.

To obtain the blue vector, take the cross product of the normal of the left polygon and the vector that is the common edge.

The angle you're looking for is formed between this blue vector and the normal to the other polygon (the red vector on the right in the diagram).

You then compute the cosine of the angle between two vectors u and v by taking their dot product, then dividing by their magnitudes.

Image taken from here.


Volumetric raycasting with transparent rasterized geometry

This is called "deep shadow mapping" and deep compositing.

Sadly invented long before you had this ideas yourself);

Now you are talking about implementing this idea specifically on a given architecture (GPU). It's up to you to make it work for this given architecture, and if you have technical difficulties with that maybe you can ask a question on this forum again.


Is there a standard specification for 3D graphic representation?

When talking about file formats, we are talking about persisting some data related to a 3D model/geometry. There is no universal standard on file formats for persisting 3D geometry. There are only a few formats more dominant than others.

Just as it is with image file formats, the PNGs and JPEGs are the most common formats out there today, but there is no universal agreement between applications on using one or the other. Each app uses the best fit for its purposes.

The same happens with file formats storing 3D data. Each 3D modeling software will generally have one or a set of preferred formats. Most actually define custom formats that only work with the specific version of the tool. This can happen for many reasons, from simplifying the inner workings of the application, or making loading of files faster, to binding the user to a given tool on purpose.

This .dat format you describe is a custom format created by the authors of the book, which was probably designed with simplicity in mind. It seems to be a text file similar to the Wavefront OBJ format, which in turn is a very popular format for storing static geometry, though a bit outdated by now. One could say that the .OBJ format is the .BMP of 3D model formats.

Other popular 3D model formats include:

The Collada format.
The 3DS format.
The PLY format.

And many others. More general explanation here.

Also note that I said at the beginning that these are formats for storing/persisting a 3D model or geometry on an offline storage. It doesn't mean (and it usually isn't the case) that the applications that use them will store the data internally in memory using the same layout as the file. Normally, this kind of data will undergo a lot of processing after being loaded from a file until it is, for instance, displayed on the screen.


In Ray-Sphere intersection is $b=2*(O-C) \cdot dirv$?

TL;DR $b = 2(f \cdot d)$

I'll try to derive it and then we can see what $b$ should be. If the sphere is centered at point $a$ with radius $r$, its equation is $|p - a| = r$, where $p$ is a point on the sphere. $|p - a|$ is the distance from a point $p$ to the centre of the sphere, and if that is equal to the radius, then it's a point on the sphere.

We'll assume the centre of the sphere is at the origin, so $a = 0$ and we have $|\vec{p}| = r$. Squaring both sides gives $|\vec{p}|^2 = r^2$ which I'm going to write as $$\vec{p}^2 = r^2 \tag{1}$$

Ray has equation $$x + td\tag{2}$$ where $x$ is the endpoint, $d$ is the direction, and $t$ is the parameter.

If the ray doesn't intersect the sphere, we will not be able to find a $t$ satisfying the ray equation. In the quadratic equation the discriminant (the bit under the square root) will be $< 0$. If it grazes the sphere (i.e. tangent) then we'll have a single point of intersection and a single $t$ and the discriminant will be $0$. If it goes through the sphere, we'll get a $t$ for the sphere entry, and a distinct $t$ for the sphere exit and the discriminant will be $> 0$. We need to plug $(2)$ into $(1)$ and solve for $t$:

$$
\begin{alignat}{2}
  &&(x + td)^2  &= r^2\\
  &\Rightarrow\quad
  &(x + td)(x + td)  &= r^2\\
  &\Rightarrow\quad
  &x \cdot x + 2t(x \cdot d) + t^2(d \cdot d)  &= r^2\\
  &\Rightarrow\quad
  &(d \cdot d)t^2 + 2(x \cdot d)t + (x \cdot x - r^2) &= 0 \tag{3}\\
\end{alignat}$$

Equation $3$ is a quadratic expression in $t$ (i.e. $at^2 + bt + c = 0$) where $$a = (d \cdot d)$$ $$b = 2(x \cdot d)$$ $$c = (x \cdot x - r^2)$$

This matches the code you posted, but the code is allowing spheres to be centered at an arbitrary point C. The code then has this line:

// f = E - C ; // Vector from center sphere to ray start

which effectively translates the endpoint of the ray E to a new endpoint f so that we can now consider the sphere to be centered at the origin. The ray direction is unchanged.


Real time image based lighting for arbitrary BRDFs coordinate alignment

The surface reflectance of a BRDF function doesn't depend on orientation or position of the surface in the world space (except for the view vector) but is defined relatively to the surface normal (or tangent space in case of anisotropic BRDF) so it should be defined in local space. Think of it as a hemispherical 2D function fixed to a point on the surface of an object that moves and rotates with the object.

Environment map on the other hand defines luminance from given direction in the world space so it is defined in that space.

To perform the convolution of these two functions you then have to either transform BRDF to world space or environment map to local space.

It's common to transform surface normal/tangent space to the world space and perform lighting calculations in that space. However, if you plan to do define BRDF with spherical harmonics, you might want to consider hemispherical harmonics for better frequency response and transform lighting from world space to the local space. Overall with SH/HSH you can only reproduce fairly low-frequency lighting (i.e. rough surfaces), so they may not be able to render as versatile surfaces as you wish.


searching for acceptable quality shadow mapping techniques

Well nevermind. I made some research and actually gonna:

sample ESM using Poisson sampling in PCF fashion (thanks Bart Wronski for great Poisson Sample generator app) and extend this to PCSS technique (variable penumbra). Render to texture.
optionally depth aware blur to soften - EDIT not so fast this option doesnt work as expected, too bad. Instead rotating PCF kernel seem to work fine.

And too bad not much of discussion here.


Why is the image from the CUDA version not as sharp as the CPU version?

If you look closely at the edge of the sphere, in the first image it is sharp, and in the second image it's blurry, blending into the background color. This suggests to me that it's not the texture, but the ray distribution on the image plane that's the issue. The second image might be distributing rays over a larger area per pixel, for instance by using a wider AA filter, or a wider lens aperture (defocus blur).


Why are reflections in a spherical surface sharp, regardless of depth of field?

There's no visible blur for the same reason you often don't see blur in very wide angle lenses: the circle of confusion is much smaller than a pixel. But it is still there. An extreme example of this is fisheye lenses.

If you had a camera with infinite resolution, such that you could zoom into the reflection on the ball, you would eventually see the blur. And it would be exactly the same blur as if you had aimed a flat mirror at the part of the scene you're looking at through the ball.

Now, we don't have such a camera, but we have something even better: software! So I built this simple setup with two perfectly sharp checkered planes, a sphere and a camera with some depth of field:

Rendering it with some depth of field at 512x512, you'd think the reflection in the sphere is perfectly sharp:

But here is a central crop of the exact same scene rendered at 8192x8192:

And we can see the exact same blur on the checker pattern. It's as simple as that: you don't see the blur because it's too small.


What is the simplest way to compute principal curvature for a mesh triangle?

When I needed an estimate of mesh curvature for a skin shader, the algorithm I ended up settling on was this:

First, I computed a scalar curvature for each edge in the mesh. If the edge has positions $p_1, p_2$ and normals $n_1, n_2$, then I estimated its curvature as:

$$\text{curvature} = \frac{(n_2 - n_1) \cdot (p_2 - p_1)}{|p_2 - p_1|^2}$$

This calculates the difference in normals, projected along the edge, as a fraction of the length of the edge. (See below for how I came up with this formula.)

Then, for each vertex I looked at the curvatures of all the edges touching it. In my case, I just wanted a scalar estimate of "average curvature", so I ended up taking the geometric mean of the absolute values of all the edge curvatures at each vertex. For your case, you might find the minimum and maximum curvatures, and take those edges to be the principal curvature directions (maybe orthonormalizing them with the vertex normal). That's a bit rough, but it might give you a good enough result for what you want to do.

The motivation for this formula is looking at what happens in 2D when applied to a circle:

Suppose you have a circle of radius $r$ (so its curvature is $1/r$), and you have two points on the circle, with their normals $n_1, n_2$. The positions of the points, relative to the circle's center, are going to be $p_1 = rn_1$ and $p_2 = rn_2$, due to the property that a circle or sphere's normals always point directly out from its center.

Therefore you can recover the radius as $r = |p_1| / |n_1|$ or $|p_2| / |n_2|$. But in general, the vertex positions won't be relative to the circle's center. We can work around this by subtracting the two:
$$\begin{aligned}
p_2 - p_1 &= rn_2 - rn_1 \\
&= r(n_2 - n_1) \\
r &= \frac{|p_2 - p_1|}{|n_2 - n_1|} \\
\text{curvature} = \frac{1}{r} &= \frac{|n_2 - n_1|}{|p_2 - p_1|}
\end{aligned}$$

The result is exact only for circles and spheres. However, we can extend it to make it a bit more "tolerant", and use it on arbitrary 3D meshes, and it seems to work reasonably well. We can make the formula more "tolerant" by first projecting the vector $n_2 - n_1$ onto the direction of the edge, $p_2 - p_1$. This allows for these two vectors not being exactly parallel (as they are in the circle case); we'll just project away any component that's not parallel. We can do this by dotting with the normalized edge vector:
$$\begin{aligned}
\text{curvature} &= \frac{(n_2 - n_1) \cdot \text{normalize}(p_2 - p_1)}{|p_2 - p_1|} \\
&= \frac{(n_2 - n_1) \cdot (p_2 - p_1)/|p_2 - p_1|}{|p_2 - p_1|} \\
&= \frac{(n_2 - n_1) \cdot (p_2 - p_1)}{|p_2 - p_1|^2}
\end{aligned}$$

Et voilà, there's the formula that appeared at the top of this answer. By the way, a nice side benefit of using the signed projection (the dot product) is that the formula then gives a signed curvature: positive for convex, and negative for concave surfaces.

Another approach I can imagine using, but haven't tried, would be to estimate the second fundamental form of the surface at each vertex. This could be done by setting up a tangent basis at the vertex, then converting all neighboring vertices into that tangent space, and using least-squares to find the best-fit 2FF matrix. Then the principal curvature directions would be the eigenvectors of that matrix. This seems interesting as it could let you find curvature directions "implied" by the neighboring vertices without any edges explicitly pointing in those directions, but on the other hand is a lot more code, more computation, and perhaps less numerically robust.

A paper that takes this approach is Rusinkiewicz, "Estimating Curvatures and Their Derivatives on Triangle Meshes". It works by estimating the best-fit 2FF matrix per triangle, then averaging the matrices per-vertex (similar to how smooth normals are calculated).


A few options:

Make the 0-1 discontinuity explicit in the mesh. That is create a 2 sets of vertices that lie exactly on the line where the value would be 1 or 0 (one set gets 1 and one gets 0) then connect the vertices up like they would makes sense.
Switch to a different texture mapping projection, I prefer a cubemap because it minimizes distortion at the poles and you don't need to have seams in the mesh.


What they're saying is you don't have to render the game at the same resolution that it's displayed. For example you could be in a display mode of 1920×1080, but have the game rendering at only 1280×720, to run faster. The OS and display hardware can automatically upscale the frames to the display resolution under the hood.

Usually, the OS runs a window compositor, a background process that draws the desktop, all app windows, menus, etc and presents them to the display. In windowed mode, any upscaling needed would be done by the compositor. However if the game's window is covering the whole display ('borderless fullscreen') and nothing else is on top, then the OS can switch off the compositor and let the game present to the display directly.

But then if the game is at a lower resolution than the display, it still needs to be upscaled. The GPU can do this in hardware (scanout) as long as the scaling factor isn't too big. For instance, there may be a limit of 4x scaling. So, if your rendering resolution is more than 4x smaller than the display, the OS can't use the hardware upscaling and would need to keep using the window compositor instead. All of this stuff happens automatically in Windows 10, without the app needing to know about it or do anything special.

Given all this, they point out that there's no real reason for the rendering resolution of the game to be limited to the classic display modes like 800×600, 1024×768, 1280×720, etc. You might as well have a continuous resolution slider instead of a fixed set of modes. There's no problem with choosing a "weird" resolution like, say, 1000×562 or something. The upscaler/compositor will just take care of it.

This what they mean by "consider providing users with the ability to choose a rendering resolution for performance scaling, independent from an output resolution". In other words, in your game's options screen, you should give the user a slider to control rendering resolution, so they can tune it for performance vs visual quality. But it need not be tied to the actual display resolution.


First, graphics is full of linear algebra, so you'll need that whatever you do. If you want to do any research into shaders, or to write a ray-tracer, you'll also need to understand integration and probably some statistics (to understand Monte Carlo integration).

  I want to know if I can teach myself the math necessary for CG

We can't answer that question for you: only you know how good you are at teaching yourself. If you "don't have a strong foundation in mathematics" because you struggled with it at school, you'll probably struggle to learn harder maths with less help than you had before. If it's not like that, and you're generally good at self-study, then now's the time. You'll never have more access to the means of learning than when you're at university. Hang out with your local maths students and pay attention when they talk about work. Find out when they have lectures relevant to you that don't clash with your own lectures, and go to as many as you can. Find out what books they use for their course, and read the same books.

It'll also help to approach any graphics projects you start with the mindset that when you encounter some maths you don't understand, you might need to put the project to one side while you learn what you need to know. You'll probably do this a lot as soon as you get past the "draw the first triangle on the screen" stage.

A counterpoint to this is that there are a lot of tools today that allow designers and artists to make really impressive graphics content without much maths knowledge, such as Unity and Processing. Maybe you can achieve what you want without studying computer graphics as an academic field at all! Try making a Unity game, play around with shaders a bit, and see if it tickles you.


MTLTexture getBytes returning blank image

maybe you forget to call MTLBlitCommandEncoder.synchronizeTexture to sync VRAM to system memory.

please check:
https://forums.developer.apple.com/thread/30488

and document:
https://developer.apple.com/documentation/metal/mtlblitcommandencoder/1400757-synchronize


AssImp library may be a good choice for you, assuming you are using C++ ?. It supports a decent number of model formats.


Please help me understand what happens as an image is faded to black in order to time-step particle fading

The basic equation for alpha blending is as follows:

$$ c_\text{final} = c_\text{source} \cdot \alpha + c_\text{dest} \cdot (1 - \alpha) $$

Here, $c_\text{source}$ is the color of the thing being blended, $c_\text{dest}$ is the background onto which you're blending it, and $\alpha$ is between 0 and 1.

In your case, $c_\text{source} = 0$ (black fading texture), so blending black $n$ times over a background reduces to

$$ c_\text{final} = c_\text{dest} \cdot (1 - \alpha)^n $$

which is indeed an exponential falloff as you guessed. You can plug this formula in and get numbers pretty similar to those you posted (although when I did it, my results were off by one or two—maybe due to rounding differences; I'm not sure).

Regarding your problem of making the trails appear the same regardless of framerate, a simple approach is to run the accumulation buffer passes multiple times per frame if necessary, so that there's effectively a fixed timestep for the trail regardless of framerate. That will probably give better-looking results than attempting to fiddle with the alpha based on the framerate. It might be too slow, though.

A potentially more efficient approach would be to have the particles spawn a new "trail particle" every so often, where each trail particle stays fixed in position, with its alpha going from 100% to 0 over some length of time (and then it gets recycled). You have to re-render all the particles every frame, but it could end up being faster overall if the overhead of the full-screen blending passes for the accumulation buffer get to be too high.

I should point out though that all of these approaches actually perform time-based fading, not distance-based as you asked for (100-pixel trails). If all the particles move at the same speed, you can just tweak the fade time to make the trails the desired length, but if different particles move at different speeds, then you would need to account for that when setting the fade times for their trails. There's also no trivial way to do that with the accumulation buffer approach, since it implicitly uses the same fade time for everything on the screen.


Does light particle loses energy when it hits something?

Both of your two proposed solutions are valid, with different properties. Which you choose is purely a design issue. The difference is that your probabilistic rays/particles (1) don't need to store their energy: they all have the same amount of energy, so you can just count particles to sum the energy. Your partial particles (2) need to store an energy so you can split them up.

The advantage of 1 is that each ray is smaller. Also, it's easy to keep track of where the energy ended up, because you can use the intersections where the ray was dropped.

Because you're only reflecting or discarding rays, the number of in-flight rays will decrease with each bounce, so you might find it harder to keep batches of rays together for better SIMD or parallel performance.

Also, because you're probabilistically discarding rays, you'll get a lot more noise in your results for the same number of rays.

The second solution is closer to a traditional ray-tracer, so you may find it easier to use existing algorithms and code. You can use fewer primary rays, and the number of rays in each batch is constant at first (but gets worse when the brightnesses start to reach zero).

It's also a lot more efficient to track multiple frequencies this way. If you give each ray a spectrum or RGB energies, the material can have a different effect on different colours without you having to fire multiple batches of rays (duplicating the ray intersection work).

On the other hand, you'll need a second data structure to count up where energy was absorbed, if you want to track that.

In your place, I'd probably use the second solution, but it depends on what other software you're trying to integrate with e.g. an existing ray-tracing library or photon representation, and what you need to do with the output/results.


When you have a curve you can adjust the knots so that they lie on top of each other. This is essentially a bit like having several control points on top of each other, except there's only one point. This is sometimes known as multiplicity or duplicity.

When you have as many knots on top of each other as you have degrees in the curve smoothness, you end up with a cusp, also known as as sharp corner. Once you have a sharp corner you can just go and delete the points on the opposite side as they no longer affect points no the other side of the corner.

Image 1: Curve and control cage (Top) and basis functions (bottom). In this case the curve acts as 2 adjoining bezier curves, The image is using the knot vector [0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 2].

If we look at a knot vector then the multiplicity looks like a seqence of n knots that have the same number. Note if you do not add ecactly as many knots on top as there is smoothness then you get a partially sharp corner.

Image 2: Animation with interpolation of knot form [0, 0, 0, 0, 1, 1, 1, 2, 2, 2, 2] to  [0, 0, 0, 0, 0.34, 1, 1.66, 2, 2, 2, 2] and back. If you do proper point insertion then the curve does not change. Just talking about what multiplicity is.

Alternatively to changing parametrization you can just add points on top of each other. This is equivalent although slight misuse of resources, but useful in surface modeling. Having many control points on top could also be useful for the uniform nature of the knot distribution.


Swap buffers twice does not show the first scene

If you swap buffers, you have a triangle in your screen and a empty color buffer in the other side.

No, you do not. Buffer swapping does not necessarily mean that the current contents of the front buffer are preserved. When you do a swap, the contents of the back buffer become the front buffer, but the state of the back buffer is undefined after the swap.

Some implementations use a true swap, where the two buffers really are switched. Other implementations copy the data from the back buffer on a swap, leaving the back buffer as whatever was there before. Still others could do something else.


Why are width and height divided by 2 in the perspective projection matrix?

Because $x_{proj}$ doesn't vary from $0 \to width$, it varies from $-\tfrac{width}{2} \to \tfrac{width}{2}$.

What's important is not the width, but the minimum and maximum values of $x_{proj}$. Because $(0, 0)$ is in the centre of the viewport, not the corner, the min and max values are plus and minus half of the width.

BTW, the Red Book is not a great introduction to graphics in general. It's a technical standard, not a tutorial, so the explanation of concepts is just enough to be unambiguous - it's not trying to teach the material. It's also about a scanline rendering library in particular, so there's a lot of important concepts it doesn't describe at all, such as what BRDFs are important, how they relate to real-world surface properties, what an image really is, and many higher-level techniques.

And finally, even in the area it's about - how to use a GPU to accelerate rendering - it's hideously out of date. The GL architecture is based on GPUs from 20 years ago. Although lots of features have been added to keep up with changes to hardware, the fundamental abstractions are not a good way to understand current GPU architectures.

If you want to learn about the theory of computer graphics, start with a proper textbook such as Physically Based Rendering or Computer Graphics: Principles and Practice, or an online course. If you want to learn how to make a 3D application, start with a tutorial about the kind of application you're interested in (e.g. if it's a game, try a Unity tutorial).


Are there common materials that aren't represented well by RGB?

There are various different types of limitation to take into consideration.

Effects for which the path of a ray is dependent on its wavelength

These are a class of effects for which spectral rendering is required, and a number of interesting examples have already been given in Benedikt Bitterli's answer. A simple example is a prism splitting white light into a spectrum, giving rainbow colours. Rays of different wavelength are refracted by different angles as they pass through the prism, resulting in the light striking the wall behind the prism being split into its constituent colours.

This means that in real life, shining monochromatic yellow light through a prism will result in yellow light coming out, but shining a mixture of red and green light that approximates yellow will result in separate red and green light emerging. When rendering using only 3 primary colours, white light will split into only those three colours, giving rainbow effects that look discontinuous, and monochromatic light that should not split at all will split into its approximating primary colour components. The splitting of white light can be improved by using a larger number of primary colours, but this will still give discontinuities close up, and the results for monochromatic light will still be split, albeit more narrowly. For accurate results a continuous spectrum must be sampled, with wavelengths distributed according to the light source being simulated.

Surface effects that cannot be captured in a single still image

Iridescence, for example, shows a different colour to each eye so that a still image will not look the same as the original object. There are many everyday examples that you might not notice at first. Many common birds have iridescent feathers even though they appear black or grey from a distance. Close up they are surprisingly colourful.

A renderer using only 3 primary colours will not be able to produce the spreading of light based on wavelength required for this effect. A spectral renderer can simulate the spreading correctly, but the full effect still cannot be captured in a single image. Even a 2d photograph cannot capture this correctly, whereas a 3d photograph of an iridescent object will give that shimmering effect as the photographs corresponding to left and right eyes will be coloured differently. This is a limitation of 2d images rather than the RGB colour space itself. However, even in a 3d image there will be colours in the iridescent object that are not displayed correctly, due to the inability of RGB to display monochromatic colours as described below.

Colours that the human eye can detect that cannot be displayed in RGB

RGB was historically device dependent and therefore unreliable between platforms. There are device independent perceptually uniform improvements such as the colour space Lab, but these are still trichromatic (having 3 components). It is not immediately obvious why three components is insufficient to display all the colours that can be perceived by a trichromatic eye, but this paper explains it well, and accessibly. From page 7:

  For example, using a modern laser-display system with monochromatic primaries at 635 nm (red), 532 nm (green), and 447 nm (blue), lets see if we can simulate the perception of a monochromatic light at 580 nm (an orange color). Since the monochromatic orange stimulus excites the greenish and reddish cones, a contribution is required by both the green and red primaries, while no contribution is required from the blue primary. The problem is that the green primary also excites the bluish cones, making it impossible to exactly replicate the orange stimulus

The diagram of human eye cones sensitivities (also on page 7) shows how wide the overlap is and helps to visualise this explanation. I've included a similar graph from Wikipedia here: (click on the graph for the Wikipedia location)

In short, the overlap between the range of colours that can be picked up by each of the three different cones (colour sensors) of the human eye means that a monochromatic colour can be distinguished from an approximating mixture of primary colours, and therefore mixing primary colours can never accurately display all monochromatic colours.

This difference is not usually noticeable in everyday life as most of our surroundings emit or reflect light across a wide range of frequencies rather than single monochromatic colours. However, a notable exception is sodium lamps. If you live in a part of the world that uses these yellow-orange street lights, the light emitted is monochromatic and will look subtly different from a printed photograph or an image on a screen. The wavelength of sodium light happens to be the 580 nm from the example quoted above. If you don't live somewhere that has sodium street lights, you can see the same single wavelength light by sprinkling finely crushed table salt (sodium chloride) over a flame. The scintillating yellow points of light cannot be accurately captured on film or displayed on a screen. Whatever three primary colours you choose, there will always be a range of monochromatic colours that cannot be displayed.

Note that this limitation applies equally to mixing 3 primary colours of paint, using 3 photoreactive chemicals on a camera film, or taking a photograph with a digital camera with 3 different colour sensors, or a single sensor with 3 different primary colour filters. It isn't just a digital problem, and isn't just restricted to the RGB colour space. Even the improvements introduced by the Lab colour space and its variants cannot recover the missing colours.

Miscellaneous effects

Multiple diffuse reflections (colour bleeding)

If a brightly coloured matt surface is near a white matt surface, the white surface will show some of the colour of the other surface. This can be modeled reasonably well using purely red, green and blue components. The same combination of red, green and blue that gave the colour of the coloured surface can reflect off the white surface and show some of that colour again. However, this only works if the second surface is white. If the second surface is also coloured, then the colour bleeding will be inaccurate, in some cases drastically.

Imagine two surfaces that look a similar colour. One reflects a narrow range of wavelengths around yellow. The other reflects a wide range of wavelengths between red and green, and as a result also looks yellow. In real life, the light showing on one surface due to the other will not be symmetrical. Most of the light reaching the wide wavelength range surface from the other will be reflected again, as the narrow range of incoming wavelengths are all within the wider range. However, most of the light reaching the narrow wavelength range surface from the other will be outside the narrow range, and will not be reflected. In an RGB renderer, both surfaces will be modelled as a mixture of monochromatic red and monochromatic green, giving no difference in reflected light.

This is an extreme example where the difference will be instantly noticeable to the eye, but there will be at least a subtle difference in most images that include colour bleeding.

Materials that absorb one wavelength and emit another

joojaa's answer describes the absorption of ultraviolet light by snow, to be re-emitted as visible light. I hadn't heard of this happening with snow before (and frustratingly I've been unable to find any evidence to support it - although it would explain why snow is "whiter than white"). However, there is plenty of evidence of it happening with a wide range of other materials, some of which are added to clothes washing detergents and paper, to give extra bright whites. This allows the total visible light outgoing from a surface to be more than the total visible light received by that surface, which again is not modelled well using only RGB. If you want to read more about it, the term to search for is Fluorescence.

Eyes with more than 3 primary colours

There are animals that have more than 3 types of cones in their eyes, allowing them to perceive more than 3 primary colours. For example, many birds, insects and fish are tetrachromats, perceiving four primary colours. Some are even pentachromats, perceiving five. The range of colours that such creatures can see dwarfs the range displayable using only RGB. Far beyond them is the mantis shrimp, which is a dodecachromat, seeing colours based on 12 different cones. None of these animals would be satisfied by an RGB display.

But more seriously, even for images intended for human eyes, there are believed to be human tetrachromats who see in 4 primary colours, and possibly some who see as many as 5 or 6. At present, such people don't seem to be present in sufficient numbers to make displays with more than 3 primary colours commercially viable, but if in future it becomes easier to identify how many primary colours a person can see, this may become an attractive trait leading to it spreading throughout the population in future generations. So if you want your great grandchildren to appreciate your work you may need to make it compatible with a hexachromatic monitor...

Not really relevant to this question, but related: If you want to see colours that are not available in either the real world or RGB images, have a look at Chimerical Colours...


How to adapt diffuse/GI light mapper to directional light maps?

Take a look at Valve's paper on combining Radiosity and Normal Mapping. It had a lot of useful insights.

If I'm not mistaken, it looks to me like you're trying to compute normal mapping using the energy value at the texel that is a result of Radiosity - e.g. it came from all directions through almost-endless bumping around the room across thousands of surfaces.

But your point light, for the normal mapping, has a very specific position, direction, distance and intensity. Meaning, you're combining apples and goats.

What I believe you should do is:
- compute the normal mapping separately for those surfaces
- blend the GI solution with normal mapping
- experiment with the blending factor till it looks 'good enough' for you


Although you may be able to detect what hardware is available as a first approximation of a computer's capability, depending on this might not be useful since the same hardware may behave differently depending on the machine it is hosted in, and any other tasks that may be competing for it.

Use the frame rate

It looks like you are already measuring the frame rate, so you could simply use this value to decide which features to include. If you settle on a target frame rate, then you can order your features from most essential to most optional, and gradually add in less essential features for as long as the target frame rate is exceeded.

You could also remove features in reverse order if the frame rate drops (in order to avoid poor frame rate if the available resources change). If you decide to do this, I recommend setting the target frame rate to a slightly different value for adding and removing features. Otherwise some machines near the border between two feature levels may end up skipping back and for between the two levels, adding and removing that feature endlessly. So set the trigger for removing a feature at a  slightly lower frame rate than the trigger for adding it.

You'll need to find a compromise between adding new features too quickly so that the frame rate suddenly drops further than you intended, and adding features too slowly so that there are several seconds of low quality animation before it settles on the appropriate level of features. You may be able to estimate that some features can be added 2 or 3 at a time if the frame rate is already very far above target. This will depend on the specific features involved and may take some trial and error on a variety of machines with different resources.

If you prefer, you can show a blank background until the frame rate has settled, so the user does not see the feature level being adjusted. Alternatively you may be able to adjust the features sufficiently gradually that there will be no perceptible change in any given moment.

Consider the window size

On my laptop I get 4 frames per second consistently regardless of the window size - whether full screen or a thumbnail sized browser window. Since it appears that the particles move only away from the centre, and do not contribute later to the area they have left, it might be worth considering abandoning particles sooner when the window size is reduced. If this allows a higher frame rate when the window size is reduced, then the extra features will be automatically activated even if they were not possible at full screen.


Finding the angle of any side of a polygon

Deducing the angle and rotating by that angle works quite well in 2D (describe in TLousky's post). This strategy, does not extend very well into three-dimensional realm. I will provide an alternative solution that shows a general strategy that works in a larger set of cases. As a bonus this without needing to think of trigonometry as it can be encoded away.

First identify the vector that you want to be transformed to orientation in your reference space. Let us call this vector $\vec{a}$, or a along vector.
Next you will need to identify a vector that is perpendicular to this vector. Let us call this vector $\vec{b}$, it is sometimes also called a up vector. In three-dimensions you need 3 vectors but once you identify 2 you can compute the third.

Finding out the $\vec{b}$ in 2-D realm is especially easy you could just rotate the vector by $90^{\circ}$ or simply swap the coordinates and make the x coordinate negative (see 1). Alternatively you can use the same procedure as in 3D.

$$\vec{b} = rotate(90^{\circ})\cdot\vec{a} =
      \begin{bmatrix}
         \cos(\pi/2) & -\sin(\pi/2) \\
         \sin(\pi/2) & \cos(\pi/2)
      \end{bmatrix}
      \begin{bmatrix}
         x \\
         y
      \end{bmatrix}
         =
      \begin{bmatrix}
         -y \\
         x
      \end{bmatrix}
     \tag{1}$$

In 3-D this usually requires a trick. Since each triangle is flat, one can use this to ones advantage. This gives us a perpendicular direction, now we still need a third one and we can get it by cross product of the 2 known vectors. You now have 3 vector.

Image 1: Vector $\vec{a}$ and derived $\vec{b}$
Normalize the vectors
Use these vectors to produce a a rotation matrix (or a affine matrix wich allows you to specify point to be at zero, your choice). This matrix represents the local coordinate of the rotated object.
All you now need is to invert this matrix and point matrix multiply your polygon corner vectors for a aligned version of the polygon.

What is especially nifty is the sidestepping of trigonometrical operations. Now i see no point i defining basic matrix operations but i see value in code. So i will use numpy.

import numpy as np, pylab

pts = np.array([
        [0.,0.],[10.,1.],[12.,12.],[-2.,11.],[0.,0.]
      ]).transpose()

f = pylab.figure(facecolor="1.")
pylab.axis('off')
pylab.axis('equal')
pylab.plot(pts[0], pts[1])

# 1. Get the vector
vec_a = np.copy(pts[:,2]) - np.copy(pts[:,1])

# 2. up vector
vec_b = np.copy(vec_a[::-1])
vec_b[0] *= -1

# 3. normalize
vec_a /= np.linalg.norm(vec_a)
vec_b /= np.linalg.norm(vec_b)

# 4. build rotation matrix
mtx_R = np.concatenate(([vec_a], [vec_b])).transpose()
mtx_R = np.linalg.inv(mtx_R)

# 5. transform, or set hierarchy transform
pts2 = np.copy(pts)
for i in range(len(pts2[0])):
    pts2[:,i] =  np.dot(mtx_R, pts2[:,i])

pylab.plot(pts2[0],pts2[1])
pylab.show()

I admit it cold be have slightly shorter by refactoring the code and eliminate redundant manipulations but its a demo only so...

Image 2: Polygon, in blue axis fixed in green.

Same method works well for straightening, skew and other problems such as perspective correction. You can extend this to affine matrices that way you can encode the translation into the calculation with just 1 more lines. In essence you specify what point is at origin and with 2 more operations one could define around what to rotate. Now it just just rotates about the origin.


What framework shoud I use to implement the RAISR upscaling algorithm on the GPU?

Are there algorithms that cannot be implemented this way and have to be implemented in OpenCL, and if so, how do I know if my algorithm is one of them?

Yes. Generally scatter-gather algorithms work poorly in OpenGL as the data locality is poor. When you have to grab bits from all over the input image for each output pixel, it's going to be slow or nearly impossible to implement in a typical GL shader. Likewise, when a calculation on an input pixel results in data that needs to be spread out to several output pixels, it's a poor fit.

It's hard to tell just from the abstract, but it looks to me like this wouldn't be a great fit for a typical OpenGL shader. I think you'll end up needing to sample an area of the input, classify it, then sample from one or more of a large set of small exemplars.

I mainly work on iOS and macOS which both come with OpenGL and OpenCL. Not sure how Windows and Linux handle that.


welcome to the world of padding. Doubles are 8 byte aligned so when a 4 byte float gets inserted between 2 doubles some padding gets inserted to keep the next double aligned to a 8 byte boundary.

You can make sure the offset is always correct (even when alignment changes in future versions) by using the Marshal.OffsetOf(Type, String) method.


Can the uniform id be changed from run to run on the same machine?

The uniform locations are set when the program is linked and are tied to the program.

This means that when you create a new program then the uniform locations can change.


Because all matrices are column-major,
the translation matrix $\mathbf{T}$ should be

$$
\mathbf{T}=\begin{bmatrix}
1 & 0 & 0 & e_x \\
0 & 1 & 0 & e_y \\
0 & 0 & 1 & e_z \\
0 & 0 & 0 & 1
\end{bmatrix}
$$

I thought you mistakenly treat the eye as a row vector just because they were written as a single line in the source code

mat4 translation = {
    vec4(   1,      0,      0,   0 ),
    vec4(   0,      1,      0,   0 ),
    vec4(   0,      0,      1,   0 ),
    vec4(-eye.x, -eye.y, -eye.z, 1 )
};

Another issue in that article is that the variable names are misleading:
the orientation denotes $\mathbf{R}^{-1}$, and the translation denotes $\mathbf{T}^{-1}$(I think it is not a good practice even they commented on these variables).

The final view matrix is:

$$
\begin{eqnarray*}
\mathbf{M} & = & (\mathbf{T}\mathbf{R})^{-1}\\
& = & \mathbf{R}^{-1}\mathbf{T}^{-1}\\
& = & \mathbf{R}^{T}\mathbf{T}^{-1}\\
& = &
\begin{bmatrix}
r_x & r_y & r_z & -<\vec{r}, \vec{e}> \\
u_x & u_y & u_z & -<\vec{u}, \vec{e}> \\
f_x & f_y & f_z & -<\vec{f}, \vec{e}> \\
0 & 0 & 0 & 1
\end{bmatrix}
\end{eqnarray*}
$$

and the implementation in that article is correct.


Can I use the same vertices to render multiple things?

If your intention is to render some textured or colored triangles and points colored diferently at the same time at vertex postions with one draw call:

glDrawElements(GL_TRIANGLES, draw_count, GL_UNSIGNED_INT, 0);

In order to access vertex data in vertex shader, you can create vertex array object which references to vertex buffer object with data you provided (vertices) for example at index 0.

then in vertex shader you have:

layout (location = 0) in vec3 vPosition; //positions for each vertex

You might then do something like that (vertex shader):

flat out vec3 fpos; //not interpolated vertices
out vec3 pos; //interpolated vertices (for default)
void main(void)
{
  pos = vPosition;
  fpos = vPosition;
}

And then in fragment shader:

if(length(fpos-pos)<threshold) //small threshold
 fragColor = pointColor;
else
 fragColor = triangleColor;


Does film filter introduce bias in path tracing?

This is an answer to your comment rather than the title of your question. It can be considered as a partial answer.
The bias in path tracers are introduced by the estimators.
For example, think of the following two equations:
$$ I = \int_{\Omega} f(\omega) d\omega $$
$$ F = \frac{1}{N} \sum_{i}^{N} \frac{f(\omega)}{p(\omega)}$$ where $p: \Omega \to R$.
It should be evident that as the number of samples N increases, the function $F$, ie the estimator, outputs values that are closer to $I$.
The unbiased estimator satisfies the following $E[F_N] \approx I$ or $\lim_{N\to\infty}F_N=I$, that is the expected value of the estimator matches the desired outcome in $I$ as the number of samples goes to infinity.
The biased estimator does not offer such a guarantee.
You can read more from D. Van Antwerpen's master thesis, I am giving the full citation in case the link goes away:
Van Antwerpen, Dietger. “Unbiased Physically Based Rendering on the GPU.” Master of Science in Computer Science, Delft University of Technology, 2010.


What is the optimal radius of gaussian distribution for determining pixel colour?

I'm not sure that there's a truly optimal radius—it's going to be a subjective matter based on what the image looks like.  As you say, too large a radius results in blurring and too small a radius results in aliasing.

I like to set sigma = 0.5 px, so that the overall radius is about 1.5 px (since the Gaussian has the majority of its weight within ±3 sigma of its mean).  In my experience that gives a good trade-off between blurring and aliasing, but that's just my taste, not based on any objective considerations.

By the way, as part of a blog post on antialiasing I wrote last year (which was based on an answer I posted on the previous incarnation of this site!), I tested a variety of antialiasing kernels against a synthetic test image and came out with 0.5 px Gaussian as my subjective favorite.


What mechanisms are being used by Directx12 and Vulkan APIs in order to communicate with graphic card drivers internally?

It's very different between the Khronos standards (including Vulkan) and DirectX.

In DirectX, Microsoft implements the API, but they publish to GPU vendors a HAL API. There's actually two HALs: one that runs in kernel-mode, to communicate with the card directly; and one that runs in user-space, to do other tasks (like manage memory, set up data structures, and compile shaders).

In Vulkan, Khronos only publishes the specification and a test suite. It's up to each GPU vendor to ship an implementation of the library. The application's interface to the driver is a normal C API, like any other library. You'll get confused if you try to think of an "API part" communicating with a "graphics card driver": the driver is the API implementation.

This is why it's very hard for third-parties to make open-source OpenGL (or Vulkan) implementations: you need to know all the details of the GPU hardware to do it. These graphics APIs are defined by industry groups consisting of the same people who design the hardware (at a high level), and deep understanding of the GPU architecture is what you need to design an efficient graphics API.


Why is Russian Roulette used in ray tracing?

The most usual way of employing Russian roulette in path tracing is to cut paths with high attenuation (low throughput) without adding bias into the resulting estimator, which would occur if you just use constant maximum path length.

Although Russian roulette incorporates additional variance into the resulting estimator, it helps throwing away long paths consuming a lot of processing power while contributing little into the resulting image, possibly leading to increased performance of a renderer.


This is based on the formula for a sphere that expresses its surface $z$ as a function of $x, y$:
$$z(x, y) = \pm\sqrt{r^2 - x^2 - y^2}$$

So, if $x, y$ is the vector from the sphere's center to your sample point, the total length of the line segment (both occluded and unoccluded parts) is the distance between the top and bottom surfaces, which is $L = 2\sqrt{r^2 - x^2 - y^2}$. (This formula appears on slide 51 of that paper, BTW, although they've apparently scaled so that the radius is 1.)

Then, if you know the depth of the sample point relative to the sphere's center (call it $\Delta z$), you can calculate the fraction of the line that's occluded as
$$\frac{\tfrac{1}{2}L - \Delta z}{L}$$
or equivalently
$$\frac{1}{2} - \frac{\Delta z}{L}$$


Why are they deprecated?

Those functions are deprecated because the OpenGL API moved in favor of a programmable pipeline in contrast with the old fixed pipeline. The programmable pipeline allows the flexibility necessary to enable a wide variety of effects and solutions that before were difficult or not possible at all to implement. You can still access those functions, as they are necessary to support older systems, but besides that I don't see any reason not to use the programmable pipeline.

How to pass parameters

If we are talking about parameters of lights (such as their position or intensity), you can pass those using uniforms. Particularly, you should look at Uniform Buffer Objects.

It would look like this in a shader:

struct Light {
   int type;
   vec3 diffuse;
   vec3 position;
   bool on;
   ...
};

#define MAX_NUM_LIGHTS 8

uniform lightBlock {
    Light lights[MAX_NUM_LIGHTS];
};

Note that this already gives you a way of passing the lights that you want. Make your shaders expect an uniform buffer with 8 lights, copy those into an array in your client code and send them using the API. For more info on how to do this using the API functions, check the link above about UBOs and this.

Is it too complicated to call every frame?

Depends on your scene and the criteria for how to choose the lights that you want to pass to the shader. If you are worried about sending the data of 8 lights through uniforms each frame, then don't, that is the least of your worries. The problem lies in the selection process of the lights and the complexity of your scene/shading model, as the cost increases with the number of lights.

Forward Renderer: If your scene and shading models are simple, maybe 8 lights are okay. If you implement modern shading models, 8 lights might be a bit too much. The point is that you will have to test it with the scene and shading models and find a compromise between all of that. Maybe pass 4 instead of 8 lights, maybe you should simplify shading models, etc.
Deferred Renderer: A deferred renderer does help with multiple light rendering due to the way it is designed. However, deferred rendering may introduce other difficulties as it is, in my opinion, a more complex system. Also, transparency and other aspects may be a pain to deal with.
Other considerations: Keep in mind that there are possible optimizations. It may be possible to optimize specific lights. Maybe not all of them need to be dynamic: think of light maps and other image based lighting. You should consider if your scene can apply any of this.

In summary, choosing between deferred or forward is a big decision. Carefully think about your scene requirements first and devise a strategy for light selection. Take into account the complexity of your shading models. For optimization, you can combine several techniques, such as simplifying some lights that are far away from the camera and only pass the relevant ones. When you have a framework for testing, find a compromise between all of this.


How are digital painting canvas rendered?

Digital painting and raster graphics software like Krita and Photoshop have a canvas where you can paint with brushes. From a programming perspective, how is this canvas rendered?

I don't have access to the code of the programs you mentioned, so I can't tell how exactly they did this since there are usually multiple ways to solve a programming problem. However, I guess everybody with some graphics programming experience will immediately think of the solution @lightxbulb already mentioned in his comment because it is the simplest one.

Textures are just an array of color values and a graphics API like OpenGL expects you to upload them as such. So a possible solution would be to represent your modifiable canvas in memory in this form too. Every time a modification is made, you find the affected pixels in your array, adjust their color values, and upload the updated canvas as texture (or update the already existing one). Then you just draw a fullscreen (or any other size you like) quad and attach the texture to it. Simple as that.

  Can it be included on a graphics pipeline like OpenGL?

As you might already guess: Yes, you can do that. If you use the texture approach, you can do anything you can do with an ordinary texture in the graphics pipeline. If your intention is to include a canvas in a 3d game, then keep in mind that finding the pixel that should be modified gets a little bit more complicated due to the perspective projection. However, the rendering process stays the same.


Do I have to have a buffer bound before making a draw call?

There is nothing in the OpenGL specification that requires you to have enabled attribute arrays in the bound VAO when you render. All of the user-defined vertex attributes will have undefined values, but since you're not reading any of them, that won't be a problem.

What may be a problem is driver bugs. Rendering with an empty VAO is not exactly the most common thing in the world, so it's possible that some drivers won't do the right thing. But the implementation should allow this.

  However, if I comment out the call to glVertexArrayVertexBuffer my program crashes.

Of course it does. You have to not enable the attribute array as well. If you have a vertex attribute enabled, it has to come from somewhere. So there must be a buffer behind it. If you don't want to put a buffer there, then you can't have an attribute array source its data from that buffer.

So while you are not required to have enabled attribute arrays, if you have enabled an array, you are required to provide a buffer for it. If you don't want to use vertex attributes, you have to not use vertex attributes.


Role of PDF of Uniform Random Sampling in a path tracer

Firstly, as @trichoplax correctly pointed out, your randomPoint function calculates a point in a cube, then uses rejection sampling to return all points that are inside a unit sphere. In order to return points on a sphere, you would need to change the greater than to an equals. That said, rejection sampling is very inefficient.

A better way to sample a sphere, is to sample in spherical space, then transform to cartesian. ie:

float phi = PI * randf();
float theta = 2 * PI * randf();

float x = radius * std::sinf(phi) * std::sinf(theta);
float y = radius * std::cosf(phi);
float z = radius * std::sinf(phi) * std::cosf(theta);

However, this is not uniform, and will cause samples to clump at the poles. To prevent this, we transform phi:

float phi = std::acosf(2.0f * randf() - 1.0f);
float theta = 2 * PI * randf();

We can use some trig identities to make it a bit more efficient to calculate with the computer:
$$\begin{align*}
\sin(\cos^{-1}(x)) \equiv& \ \sqrt{1 - x^2}\\
\cos(\cos^{-1}(x)) \equiv& \ x\\
\\
\phi =& \ \cos^{-1}(2\xi - 1)\\
\sin(\phi) =& \ sin(\cos^{-1}(2 \xi - 1))\\
=& \ \sqrt{1 - (2 \xi - 1)^2}\\
\cos(\phi) =& \ 2 \xi - 1
\end{align*}$$

So the full change is:

float cosPhi = 2.0f * randf() - 1.0f;
float sinPhi = std::sqrt(1.0f - cosPhi * cosPhi);
float theta = 2 * PI * randf();

float x = radius * sinPhi * std::sinf(theta);
float y = radius * cosPhi;
float z = radius * sinPhi * std::cosf(theta);

However, in the case of next event estimation, uniform sampling the whole sphere is inefficient, because a ray can only 'see' half of the sphere at a time. So if we generate a point on the 'back' of the sphere, the sphere will occlude the point, and your calculation will be wasted.

Instead, you generate samples in a cone, which covers the great circle of the sphere, as viewed from the starting point of the ray.

// Sample sphere uniformly inside subtended cone
float rand1 = randf();
float rand2 = randf();

// Compute theta and phi values for sample in cone
float distanceSquared = DistanceSquared(rayOrigin, sphereCenter);

// We use geometry to calculate the angle of the cone (aka, the maximum phi can be when we sample)
// It's easier / cheaper to use geometry to calculate sin/cos phi directly, rather than generating phi and using sin/cos
float sinPhiMaxSquared = radius * radius / distanceSquared;
float cosPhiMax = std::sqrt(1.0f - sinPhiMaxSquared);
float cosPhi = (1.0f - rand1) + rand1 * cosPhiMax;
float sinPhi = std::sqrt(1.0f - cosPhi * cosPhi);

// Phi can be anything in 2 PI
float theta = 2 * PI * rand2;

float x = radius * sinPhi * std::sinf(theta);
float y = radius * cosPhi;
float z = radius * sinPhi * std::cosf(theta);

NOTE: x, y, z are in LOCAL coordinate space.

So we need to transform to world coordinates:

float3 zAxis = normalize(sphereCenter - rayOrigin);
float3 xAxis;
if (std::abs(zAxis.x) > std::abs(zAxis.y))
    xAxis = float3(-zAxis.z, 0.0f, zAxis.x) / std::sqrt(zAxis.x * zAxis.x + zAxis.z * zAxis.z);
else
    xAxis = float3(0.0f, zAxis.z, -zAxis.y) / std::sqrt(zAxis.y * zAxis.y + zAxis.z * zAxis.z);
yAxis = cross(zAxis, xAxis);

float3 samplePoint = x * xAxis + y * yAxis + z * zAxis;

And the pdf is calculated as follows:

float pdf = 1 / (2 * PI * (1 - cosPhiMax));

The code here is heavily influenced / copied from PBRT v3. They have a series of classes and functions for sampling from shapes.

Finally, the pdf. In monte-carlo integration, we need to combine the pdf's of each integration we do. In path tracing, we can integrate over many many things. For example, the general rendering equation integrates the incoming light over the hemisphere, depth of field can be treated as an integration over a focal distance, etc.

For next event estimation, you explicitly split the rendering equation into two integrands, direct lighting, and indirect lighting.

Standard rendering equation:
$$ L_{\text{o}}(p, \omega_{\text{o}}) =  L_{e}(p, \omega_{\text{o}}) \ + \  \int_{\Omega} f(p, \omega_{\text{o}}, \omega_{\text{i}}) L_{\text{i}}(p, \omega_{\text{i}}) \left | \cos \theta_{\text{i}} \right | d\omega_{\text{i}} $$

Next Event Estimation:
$$ L_{\text{o}}(p, \omega_{\text{o}}) =  L_{e}(p, \omega_{\text{o}}) \ + \ \int_{\Omega} f(p, \omega_{\text{o}}, \omega_{\text{i}}) L_{\text{i, direct}}(p, \omega_{\text{i}}) \left | \cos \theta_{\text{i}} \right | d\omega_{\text{i}}\ \ + \ \int_{\Omega} f(p, \omega_{\text{o}}, \omega_{\text{i}}) L_{\text{i, indirect}}(p, \omega_{\text{i}}) \left | \cos \theta_{\text{i}} \right | d\omega_{\text{i}} $$

In simple naive forward path tracing, everything is is treated as indirect light. In next event estimation, we directly calculate the direct lighting and add it to the indirect lighting. And if we hit a light, we ignore the contribution, since we're calculating the direct lighting.

Since we have two integrations, each will have its own pdf. Aka:
$$L_{\text{o}}(p, \omega_{\text{o}}) =  L_{e}(p, \omega_{\text{o}}) \ \ + \ \sum_{k=0}^{\infty } \frac{f(p, \omega_{\text{o}}, \omega_{\text{i}}) L_{\text{i, direct}}(p, \omega_{\text{i}}) \left | \cos \theta_{\text{i}} \right | }{pdf_{direct}}\ \ + \ \sum_{k=0}^{\infty } \frac{f(p, \omega_{\text{o}}, \omega_{\text{i}}) L_{\text{i, indirect}}(p, \omega_{\text{i}}) \left | \cos \theta_{\text{i}} \right | }{pdf_{indirect}}$$

If you want to see how this is implemented, you can check out my implementation here. Note: my light sampling is a bit more complicated, since it does multiple importance sampling. But it can be as simple as:

float3 Renderer::EstimateDirect(Light *light, UniformSampler *sampler, float3a &surfacePos, float3a &surfaceNormal, float3a &wo, Material *material) const {
    float pdf;
    float3 wi;

    // Sample a point on the light and get the pdf
    float3 Li = light->SampleLi(sampler, m_scene, surfacePos, surfaceNormal, &wi, &pdf);

    // Calculate the brdf value
    float f = material->Eval(wi, wo, surfaceNormal);
    return f * Li * / pdf;
}


How is a light probe different than an environmental cube map?

There are two different common meanings of "light probe" that I'm aware of. Both of them represent the light around a single point in a scene, i.e. what you would see around you in all directions if you were shrunk down to a tiny size and stood at that point.

One meaning is a spherical harmonic representation of the light around a point.  Spherical harmonics are a collection of functions defined over a spherical domain, which are analogous to sine waves that oscillate a certain number of times around the equator and from pole to pole on the sphere.

Spherical harmonics can be used to create a smooth, low-res approximation of any spherical function, by scaling and adding together some number of spherical harmonics—usually 4 (known as linear, first-degree, or one-band SH) or 9 (called quadratic, second-degree, or two-band SH). This is very compact because you only have to store the scaling factors. For instance, for quadratic SH with RGB data, you only need 9*3 = 27 values per probe. So SH makes a very compact, but also necessarily very soft and blurry, representation of the light around a point. This is suitable for diffuse lighting, and perhaps specular with a high roughness.

This screenshot from Simon's Tech Blog shows an array of SH light probes spaced throughout a scene, each one showing the indirect lighting received at that point:

The other currently common meaning of "light probe" is an environment cube-map whose mip levels have been pre-blurred to different extends so it can be used for specular lighting with varying levels of roughness. This image from Seb Lagarde's blog shows the basic idea:

The higher-resolution mips (toward the left) are used for highly polished surfaces where you need a detailed reflected image. Toward the right, the lower-res mip levels are increasingly blurred, and are used for reflections from rougher surfaces. In a shader, when sampling this cubemap, you can calculate your requested mip level based on the material roughness, and take advantage of the trilinear filtering hardware.

Both of these types of light probes are used in real-time graphics to approximate indirect lighting. While direct lighting can be calculated in real-time (or at least approximated well for area lights), indirect lighting is usually still baked in an offline preprocess due to its complexity and computational overhead.

Traditionally, the result of the baking process would be lightmaps, but lightmaps only work for diffuse lighting on static geometry, and they take up a lot of memory besides. Baking a bunch of SH light probes (you can afford a lot of them because they're very compact), plus a sparser sprinkling of cubemap light probes, allows you to get decent diffuse and specular indirect lighting on both static and dynamic objects. They're a popular option in games today.


Minimum requirements to uniquely represent a 3D object in space

A rigid body has 6 degrees of freedom, in 3D- space. So that means you need 6 values to represent the object. The common way to do this is to store a position vector for position and 3 rotations. But for obvious reasons any 6 variables that are independent of each other would do this.

The problem with vectors is that they aren't the most efficient way to store the data. If you have a vector for origin and a normal you still need one value for the rotation around the normal resulting in seven variables (this is called an axis angle rotation). Two reference points on object has same problem since you dont know the rotation around the axis of point 1 and point 2 leading to 7 variables.

Image 1: Storing vectors of 2 points has a freedom to rotate along the axis of those the vectors

Now storing extra values can be beneficial for other purposes than position and orientation. So it is quite common to store 7 values for solid bodies. This way it is easier to interpolate so that you dont need to convert between representations all the time.


3D object to slices like in medical scan

Some terminology: Intersection of a polyhedron with a plane.

Wolfram Demo.


float3 in Metal Shading Language has the same size/alignment as float4, i.e. 16 bytes. I think your original code would have worked using packed_float3 (12 bytes) in the vertex shader.


How does a path tracer with next-event estimation work?

I noticed three potential problems. First, this bit of code looks suspicious:

Vector r = new Vector(PRNG.nextDouble(), PRNG.nextDouble(), PRNG.nextDouble()).normalize();
Vector v = r.cross(w);
Vector u = v.cross(w);
Vector wi = u.scale(sample.x).add(v.scale(sample.y)).add(w.scale(sample.z));

The vectors v and u are not normalized, so the construction of wi will be distorted. (Also, using a random vector r to construct the basis is really more work than necessary; you might take a look at this paper for a faster way of doing that.)

Second, you have a getAmbientReflection() in your direct lighting code—that should not be there for path tracing, where "ambient" light is properly calculated by the recursive trace.

Finally, I think the result of your direct illumination path ought to be scaled by $1/π$. This is because the Lambert BRDF is equal to $c_d/π$ where $c_d$ is the diffuse color. The direct illumination path needs to evaluate the full BRDF, including that $1/π$ factor, using the irradiance sample coming in from the light source.

However, the cosine-weighted recursive trace does not need a $1/π$, as it is cancelled out by the cosine PDF. That one should just use $c_d$, like you have it.

You can check that the $1/π$ factors are correct by checking that the average brightness is equal with and without next-event estimation (keeping the path length the same).

BTW, note that the image getting somewhat brighter as you increase the path length is to be expected, as longer paths = more bounces = more illumination. (Russian-roulette path termination fixes this.) Longer paths will also give stronger global illumination / "color-bleeding" effects. That seems to be what's happening in your images—look at the color of the shadow under the sphere; it gets progressively more influenced by the color of the walls and sphere as you increase the path length.

If this is undesirable, try reducing the diffuse reflection factors of all the materials in the scene; they might be unrealistically high, and cutting them down will reduce the bounce light and color-bleeding.


But with ray tracing we normally cast cylinder-like rays instead of cones.

Rays aren't really cylinders: they're infinitely thin. This infinitely thin ray can be an approximation to the cone just as much as it can be an approximation to the cylinder.

  If we assume that the intensities of the light sources are the same, so that one of the pixels would "see" white average color, the other must necessarily have a color tint ... Am I right in this?

Sort of. Your left-hand diagram doesn't have a colour tint. Although it covers more of the red wall than the blue wall, both of them subtend the same solid angle from the camera's point of view. The same $\tfrac{1}{r^2}$ ratio between the amount of light included in the cone is exactly countered by the $\tfrac{1}{r^2}$ falloff of the light. If the world worked the way you describe, you could make anything brighter by moving it further away from your eye. The right-hand diagram would have a blue colour tint, because the blue light is closer, but this diagram is not how ray-tracing works.

There has been research in a few variants of beam tracing, where instead of tracing an infinitely thin ray, you trace a cylinder, or cone tracing, where you trace a cone or frustum as a simple approximation to the irregular solid that contributes to the pixel. The maths (and therefore the algorithm) are somewhat more complex (and therefore slower) than simple ray tracing, and then you have to deal with what happens when the cone partially intersects an object, and you have to use the same shape to approximate the continued path of the cone. These approaches never caught on.

Instead, supersampling with rays offers the same advantages as cone tracing but with the performance advantage of doing the same simple algorithm many times. You don't supersample by moving the origin of the rays, which would make a cylinder shape, but by varying the directions, which makes a cone shape. Instead of approximating that cone by a single ray along its axis, you're approximating it by (say) 32 rays spread throughout its volume.


Is there any reason to prefer Direct3D over OpenGL?

About the fact that there are more games for Windows, some reasons are

Windows has the majority of the market and in the past to develop cross platform games was more complicated than it is today.
DirectX comes with way better tools for developing (e.g. debugging)
Big innovations are generally first created/implemented in DirectX, and then ported to/implemented in OpenGL.
As for Windows vs Linux, you have to consider that when there is an actual standard due to marketing and historical reasons (for that see Why do game developers prefer Windows? | Software Engineering, as I said in the comments), it has its inertia.

The inertia thing is very important. If your team develop for DirectX, targeting 90% of the market (well... if you play games on pc you probaly have windows, so... 99% of the market?), why would you want to invest in OpenGL? If you already develop in OpenGL, again targeting 99% of the market, you will stick to it as long as you can. For exampe Id Tech by Id Software is an excellent game engine (powering the DOOM series) that uses OpenGL.

About the topic of your discussion, a comment.

As of today, there are many many APIs, and a common practice is to use a Game Engine that abstracts over them. For example, consider that

On most mobile platform you have to use OpenGl ES.
On PC you can use both DirectX and OpenGL
I think on XBOX you have to use DirectX.
I think that on PS you use their own API.
With old hardware you use DirectX9 or OpenGL 3, or OpenGL ES 2.
With more recent hardware you can (and want to) use DirectX 11, OpenGL 4, OpenGL ES 3.

Recently, with the advent of the new low overhead APIs - whch is a major turning point for graphical programming - there are DirectX 12 for Windows and XBOX, Metal for iOS and Vulkan (the new OpenGL) for Windows and Linux (including Android and Tizen).

There still are games that only target Windows and XBOX, but IMHO today that can only be marketing choice.


Where does the cosine factor comes from in the GGX PDF?

Normal distribution functions are defined a bit differently than you might expect. They're not strictly a probability distribution over solid angle; they have to do with the density of microfacets with respect to macro-surface area. The upshot is that they're normalized with an extra cosine factor:
$$ \int_\Omega D(m) \cos(\theta_m)\, d\omega_m = 1 $$
This cosine factor accounts for the projected area of microfacets onto the macrosurface. When importance-sampling the NDF this cosine factor must be accounted for as well.

This is actually hinted at in the text of the paper, as it says

  The equations for sampling $D(m) | m \cdot n |$ are:

(just above equations 35–36). The $|m \cdot n|$ factor there is equal to $|cos(\theta_m)|$. So the quantity being sampled is not just the NDF itself, but includes this extra factor. Then, as you note, the sine shows up as part of the area element for spherical coordinates.

I wrote a blog post a few years ago that goes into more detail about this.


Sharing calculation result between shader programs in OpenGL

Yes you can, this is how deferred rendering for example would work. For that you would render to a texture in one pass, in openGL this would be via an FBO, then feed that texture as the input to the next pass/shader.


For shader math, why should linear RGB keep the gamut of sRGB?

Talking about Linear RGB must be avoided because it does not tell you anything about the RGB colourspace intrinsics, i.e., Primaries, Whitepoint and Colour Component Transfer Functions. A few years ago, assuming it was sRGB was middling but nowadays with DCI-P3 and BT.2020 being very common, it must be ruled out.

The ideal gamut for rendering is the one that will minimize errors in respect to a real-world reference or more conveniently a ground truth spectral render. The first takeaway from this sentence is that the various RGB colourspaces are not equivalent and will not produce similar results.

One might think that performing two renders with the same base colours but one where they are encoded with sRGB/BT.709 and the other one where they are encoded with DCI-P3 and then converting the two resulting images to for example ACES2065-1 will yield the same images but it is not the case. Some mathematical operations because of the nature of linear algebra and matrices are dependent on the given RGB colourspace primaries, i.e., on the colourspaces basis. The same operations performed in different RGB colourspace will yield different tristimulus values once converted back to CIE XYZ color space. For example multiplication, division and power operations are RGB colourspace primaries dependent while addition and subtraction are not.

This image illustrates the effect of multiplying various colours by themselves into different RGB colourspaces: the resulting colours are different. The various samples are generated as follows: 3 random sRGB colourspace values are picked and converted to the three studied RGB colourspaces, they are exponentiated, converted back to sRGB colourspace, plotted in the CIE 1931 Chromaticity Diagram on the left and displayed as swatches on the right.

Tests and research conducted by Ward and Eydelberg-Vileshin (2002), Langlands and Mansencal (2014) and Mansencal (2014) showed that gamuts with primaries closest to the spectral locus, i.e. spectrally sharp primaries, tend to minimize the errors compared to spectral ground truth renders.

Here is an image I recently rendered with Mitsuba to re-validate our findings with Anders:

Those are renders of the same scene using BT.709 primaries (first row), 47 spectral bins (second row), BT.2020 primaries (third row), spectral minus BT.709 primaries renders residuals (fourth row), spectral minus BT.2020 primaries renders residuals (fifth row). The last row showcases composite images assembled with three vertical stripes of respectively the BT.709 primaries, spectral and, BT.2020 primaries renders. Direct illumination tends to match between the renders. Areas that show the effect of multiple light bounces, i.e., the ceiling, in the BT.709 and BT.2020 primaries renders tend to exhibit increased saturation, especially in the BT.709 primaries render or slight loss of energy, especially in the BT.2020 render. Excluding outliers, e.g., the visible light source, the RMSE with the spectral render are 0.0083 and 0.0116 for respectively the BT.2020 primaries and BT.709 primaries renders.

Now it does not mean they will always perform better, and one might be able to produce examples that will exhibit a bias toward BT.709/sRGB. The main takeaway is that RGB renders cannot match spectral renders and sharp wide gamuts tend to perform better. As for choosing a rendering colourspace, I would pick one with a wide gamut that encompasses Pointer's Gamut and DCI-P3, BT.2020 or ACEScg are excellent candidates for that.


How to implement Constructive Solid Geometry in ray tracing with implicit surfaces (spheres)?

I'm not sure what exactly you are asking but, IIRC, to do CSG with ray tracing, you maintain not just the closest intersection with an object, but a list of ordered pairs of [Inpoint, OutPoint] 'interval' distances. With a single sphere test (or any solid convex primitive for that matter), this list would have at most one [Inpoint, OutPoint] entry or is empty if there is a miss.

To implement CSG you then apply the binary operator to a pair of lists, which in turn means applying them to the entries in the lists in a manner akin to merge-sorting two lists.

For example, for intersection, if you had  [a,b] in one list and [c,d] in the other, compute  e=Max(a,c) f=Min(b,d).  If e > f there's no intersection else return the intersection, [e,f].

Does that help?


How different is the SPIR-V accepted by OpenGL (ARB_gl_spirv) from SPIR-V accepted by Vulkan?

Vulkan and OpenGL use different models for their resource bindings. Vulkan uses descriptor sets + binding, while OpenGL uses binding indices. As such, the SPIR-V they consume use different decorations for their resources.

There are also a number of differences in terms of built-ins that the two APIs provide. For example, Vulkan accepts the InstanceIndex built-in, while OpenGL does not. OpenGL uses InstanceId, which has a different meaning from InstanceIndex.

The ARB_gl_spirv extension has a pretty exhaustive list of the differences between it and KHR_vulkan_glsl. Many of those are differences that the output SPIR-V will have to expose.


Given a matrix $D \in \mathbb{R}^{3N\times k}$ where each column is normalized $D_i$, You can get $W = D^TS_i$, and later it can be used to reconstruct the shape with $S_i = DW$ (With some error if you truncated some basis). The acquisition of weights is equivalent to calculating dot products between each PCA basis and $S_i$. What is done here is you project $S_i$ onto each orthogonal basis $D_i$, and calculate the length of the projected vector, so you are basically calculating the required contribution of each basis to construct the given $S_i$.

If you are using unnormalized PCA basis, you can decompose the unnormalized PCA basis matrix into a diagonal matrix an an orthonormal matrix (Say, $\Lambda D$), then use a similar approach to obtain the contribution of each basis. In this case, you'll have to use $\Lambda^{-1}D^T$ instead of $D^T$.


Can I accelerate rendering an image from a ray cloud using a GPU?

I'm not quite sure I fully understand the problem, this is what I think you're asking:

You start with a set of rays pointing a all sorts of directions
You want to know which of these rays intersect with the image plane, taking into account occlusion.
I'm going to assume these rays carry some kind of color information along with them. IE. when they hit the image plane, they will accumulate color.

With that goal in mind, I would do the following:

Create 2 image buffers

For accumulated color
To keep track of the number of hits

Clear the accumulation buffer to black, and the hitCount buffer to all zeros
Create a kernel per ray

Use classic ray tracing math to find the nearest intersection

Here you can do anything from a simple 'iterate through all the object in the scene' to something like BVHs, or kd-trees.

If the closest intersection is the image plane, use the WorldView matrix for the image plane to transform the intersection point from world coordinates to view space coordinates.
Use the x, y coordinates of the image space intersection point to get the pixel coordinates of the image buffer
Add the color of the ray to the corresponding pixel in the accumulation buffer
Increment the value of the corresponding pixel in the hitCount buffer

For 4 and 5 you could also use a filter in order to apply the color to multiple pixels, rather than just one. In that case, you would add a value less than 1 to the hitCount buffer

On the CPU, or in another pass per pixel:

Divide the accumulated color buffer by the hitCount buffer

I don't see any problem with doing these steps on the GPU. Any of the compute languages should handle it fine. You might get some divergence in the actual ray tracing code, but it should be ok. The biggest potential problem is that you have to store the entire scene in VRAM.


Correctness of logical steps for create the matrix of viewing transformation

I'm assuming you mean $A$ is object$\rightarrow$world matrix and $B$ is your camera$\rightarrow$world matrix? If that's the case, to calculate object object$\rightarrow$camera matrix you calculate:

$$C=B^{-1}*A$$

In general your math is correct, i.e. the order of matrix multiplication is reversed for the inverse calculation. It's just that normally these transformations are combined to a single 4x4 transformation matrix which does rotation, scaling and translation, and then you just work with that single matrix instead of it being decomposed to separate rotation & translation matrices. Anyway, if you want to decompose the matrix multiplication, based on the above you would do:

$$A=T_A*Ry_A*Rx_A$$
$$B=T_B*Ry_B*Rx_B$$
$$C=Rx_B^{-1}*Ry_B^{-1}*T_B^{-1}*T_A*Ry_A*Rx_A$$


How does this small fractal shader work?

Here is a line by line breakdown of my shader:

#define mainImage(o,p)\
    for( vec2 r = iResolution.xy, q = 2.*(p+p-r)/r.y; o.a++ < 30.; )\
        o += length( q = abs(q)/dot(q,q) - iMouse.xy/r) / 2e2

First, it is important to know that the “\” at the end of the first two lines do not count towards the character count and merely serve to make the code more readable by splitting it onto multiple lines.

#define mainImage(o,p)

Begins the definition of the function which would normally be written
void mainImage(out fragColor, in fragCoord){...}
This saves a few characters.

The first part of the loop:

vec2 r = iResolution.xy, q = 2.*(p+p-r)/r.y;

Establishes two variables of type Vector2:

r is the resolution which allows me to refer to iResolution several times later in fewer characters.

q is the coordinate in the fractal which is calculated from the fragCoord p and is then iterated in each loop.

Instead of fragCoord starting at (0,0) in the bottom left corner and extending to the (r.x,r.y) in the top right, (p+p-r) transforms each fragCoord so that the center of the screen is (0,0). This doubles the fragCoord value and then subtracts the resolution of the window.
The division by r.y fits the coordinates such that the top has a y value of 1 and the bottom has a y value of -1. With this, the x is scaled accordingly without stretching.
Finally, the multiplication by 2 just zooms out by a factor of 2.

Then for the iterator, I use o.a (the alpha channel of o).
o is the output color which is a vec4. As defined by Shadertoy, o begins with the value vec4(0,0,0,1). I use this so that I do not need to use characters declaring a new float as an iterator. o.a++<30 combines the test expression and the iterator in the for loop.

The content of the loop is one line where a number gets added to each channel of the output color. This line:

o += length( q = abs(q)/dot(q,q) - iMouse.xy/r) / 2e2

Can be broken down into a few parts.
Each iteration of the coordinates in the fractal occurs here:

q = abs(q)/dot(q,q) - iMouse.xy/r

Where iMouse.xy/r is a way to transform mouse coordinates from pixel to the range (0,0) to (1,1)

Each iteration of this function changes the value of q to be used.

What produces this fractal is the idea that some initial values for q will escape to infinity and others will converge or cycle around a few small values. Each iteration I increase the lightness of the pixel based how far q is from (0,0). This length of q is then divided by the constant 2e2 (200) which is to prevent o.rgb from increasing too rapidly.

The contents of the loop could be understood as two steps:

q = abs(q)/dot(q,q) - iMouse.xy/r;
o += length(q) / 2e2;

In the end, even if o contains a value in one of its channels that is greater than 1, it gets clamped by Shadertoy automatically, and a trailing semi-colon is not needed on the last line.


It's not quite correct, today, to think of compute shaders as being "in the shader pipeline" in the same sense that your vertex and fragment shaders are literally hooked up into a pipeline. Compute shaders are not "hooked up" to anything currently, cannot drive rasterization, or directly consume the outputs of rasterization.

What it allows you to do, though, is consume and produce memory resources also used by draw calls in a relatively efficient way. OpenCL makes this rather difficult (see for example https://software.intel.com/en-us/articles/opencl-and-opengl-interoperability-tutorial). This allows you to efficiently integrate compute passes within your renderer, but it is not integrated into the rendering pipeline


What determines the color gamut of a display?

The physical imaging elements of your monitor define your gamut. So things like how bright the elements can be as well as the color spectra of the color elements define the gamut. But, thats quite the simplification, its also affected by such things as the screen surface material and background lighting conditions.

Can the gamut be changed? Sure, if you want a smaller gamut then you can restrict your colors to a subrange of your gamut.


How to do a color separation with more than 3 primary colors

The mathematical approach to this is to represent the other colors/light sources in terms of your standard primaries in the color space. Represent your target color and available light sources as vectors with, say, the RGB color space as the basis. Figuring out how to display your target color using your available sources then becomes a change of basis, a standard linear algebra operation, where the target basis consists of your LEDs/nonstandard primaries.

If the dimensions of the source and destination bases are different, the change of basis matrix won't be square and thus won't be invertible. In that case, you need to solve a system of linear equations, expressed in matrix form:

$$c_{RGB} = [r, y, b, v] \cdot c_{RYBV}$$

Where $c_{RGB}$ and $c_{RYBV}$ are the coordinate vectors of your desired color expressed in terms of RGB and custom primaries, respectively, and $r$, $y$, $b$ and $v$ are your custom primaries (basis vectors) in RGB coordinates.

Solving this system is another common linear algebra operation, and there are a variety of numerical and symbolical algorithms to do so. The system is over-constrained, so solutions will likely have a free variable, however since we can't physically produce negative light, nor do the primaries have infinite dynamic range, for most practical purposes each coefficient will have to be limited to be within the $[0, 1]$ range. This will render many colors unrepresentable under certain sets of primaries.


How do I create a Win32 window with a Vulkan context?

There's no such thing as a Vulkan context, the way there is with OpenGL.  This is because Vulkan is a lower level API and an OpenGL context ties together a number of concepts that are distinct in the Vulkan API.
For example, the OpenGL context ties together the concept of extension management, memory management, draw commands and surface presentation.  In Vulkan those are all managed through independent interfaces.
There are a number of things in Vulkan that kind of correspond to an OpenGL context in different ways, depending on what you're trying to do.
VkInstance encapsulates access to the Vulkan libraries on the system, including enumerating the global extensions.
VkPhysicalDevice lets you query the hardware capabilities and hardware specific extensions.
VkDevice lets you actually interact with the hardware to do things like allocate memory.
VkCommandBuffer is the interface for setting up and executing draw calls.
However, since you're focused on windows, what you're probably interested in is VkSurfaceKHR.  This encapsulates interaction with a windowing system, including initializing a surface from the OS specific window handles and doing presentation calls.
On Windows, to interact with a window, you need to have an existing VkInstance, and you need to call vkCreateWin32SurfaceKHR and pass it a structure containing an HWND and an HINSTANCE.  If it succeeds then you have a surface to which you can then present images.


How to make the lines of a (binary) image dotted or dashed?

A kernel can (probably) not help you, because they are very localized. However, you need to know, where a line is, and that may be out of the reach of a kernel (e.g. a 3x3 Pixel kernel would only be able to detect a line 3 pixels wide. Is that meeting your demands of being a line?)

Rather you need to define, when a line is a line with the information you have. For simplicity you could say "Having at least 5 pixels in either X or Y direction black next to each other, but not having any pixels black in any other direction makes a line". Then you can scan the image for those lines and store them in some data structure.

Now that you have your lines figured out, you need to define what to do with them, e.g. "Every line longer than 20 pixels needs to be dashed, every other line dotted".
Also "A dashed line has a white pixel at every third and fourth pixel, except the first and last pixel may not be white" and "A dotted line has a white pixel at every other pixel, except the first and last pixel may not be white". Since you know the lines, you either know or can check the lines, and therefore you can edit them one by one to be dotted or dashed.

This is of course a pretty high level description, but you don't give much to go on with your question. Perhaps though, this suffices for you to get started.

The hard part of course is to identify lines. Try to think what makes a line for you (e.g. direction, thickness/width vs height), what edge cases you may have (a single pixel or square block of pixels) and then you can go about finding them in your image. Note that in a rasterized image, non horizontal and non vertical lines will have "strange" patterns of how they fill the image and you will have to decide how to treat those on your own.
Also, make sure to not have any lines stored twice (or more) when scanning, because your dotting / dashing process could possibly lead to an unwanted outcome.


Why is it easier to inverse transform every object in a scene than to transform a camera?

That's actually incorrect. You can transform every ray of your camera if you wish (and numerous implementations do so). There are some advantages and disadvantages to each method (e.g. if your rays are more than your vertices you end up doing more transformations, however you don't incur cache misses by running over all vertices).