Is a constant condition more costly than switching shaders?

On modern hardware if all invocations in a group follow the same path then the unused path doesn't get evaluated.

in pseudo code:

if(cond){
   res = ...
}else{
   res = ...
}

becomes

if(anyInvocationARB(cond)){
    res1 = ...
}
if(anyInvocationARB(!cond)){
    res2 = ...
}
res = cond?res1:res2;

Where anyInvocationARB will be true if any invocation of the shader will have true as cond (from the opengl extension ARB_shader_group_vote).

If cond is derivable from uniforms alone then the driver may optimize and evaluate the condition before starting the render and replace the if with a goto to the correct branch. OpenGL has a feature called uniform subroutines that makes it explicit.


Advantages of spherical harmonics over baked vertex lighting?

Spherical harmonics are generally used for dynamic objects in your scene, while fully-baked lighting is used for static objects.

A typical game engine will use both. During the GI pass, all surfaces marked as static will have their global lighting fully baked - generally into a lightmap texture rather than per-vertex. At the same time, a bunch of light probes distributed throughout your scene will receive omnidirectional lighting, and this gets converted into SH. Since SH coefficients interpolate nicely, a mesh moving through your scene can pick up the nearest few light probes (generally 4), and use the weighted average to decode a good approximation of the ambient lighting at its current location.

Another use for SH is to render less-important dynamic lights in a scene with many lights in it. The most important light or two can be rendered fully dynamically per-pixel, then for each renderer, the SH coefficients of the less-important lights are calculated and added to the light probe data. This allows the lights to change in real-time, and be rendered effectively for free per-vertex in the shader, with a small CPU overhead.


row and column majored rotation matrix pre- or post- multiplied

Okay let us start by pointing out that a colmun major matrix is the same as a transposed row major matrix. So:

$$
{M_{cm}}^{T} = M_{rm} \tag{1}
$$

Then notice that matrixes have following properties. Matrix multiplication is associative (2a) and that the distribution of transpose reverses computation order (2b).

$$
A(BC)  =  (AB)C \tag{2a}
$$
$$
(AB)^T  =  B^TA^T \tag{2b}
$$

From these follow that the chain of multiplication events can be as long as one likes and all transpose does is inverse the computation. So given that we can conclude

$$
{T_1}\ {T_2}\ \ldots\ {T_n}\ \vec{v} = \vec v\,^T\ {T_n}^T\ \ldots\ {T_2}^T\ {T_1}^T, \tag{3}
$$

so calculating in column major order is the same calculation as calculating in row major orientation. The difference is that the order of computation is reversed. An interpretation of this is that the difference is the same as looking at the object from the outside or looking at the situation from the object out.

Formula 3 should provide you the insight. Just one more thing a column vector is the same as a transpose of a row vector.

Note: You are allowed to mix and match the computations as much as you like. All you have do is remember is that the other approach is just transposed.


Please, help me to understand why my raytracing camera rotates in wrong directions?

The problem is that instead of a camera-to-world matrix, a world-to-camera matrix is being made by glm::lookAtRH.

This is because GLM is a math library made for OpenGL. In OpenGL, you do not move the 'camera' as there is no camera in OpenGL. You move the whole world the other way around, so that it lines up in the frustum. The lookAtRH method returns a matrix that moves the whole world in a way that it seems like you have the camera there.

Inversing the matrix is a fine way to get the proper camera-to-world matrix.

Scrathapixel does have an article regarding making your own lookAt function. One thing to note about this article is that they use a row-major notation. It might be that you need to switch it up and use a column-major notation (just swap the x index with the y index).

Placing a Camera: the LookAt Function | Scratchapixel


Am I calculating perspective projection correctly?

In the video you linked to, $d$ is just the distance from the eye to the image plane.  To simplify things though, you can just set $d$ to 1, which is the usual case and causes it to disappear from the formulas.

Your formula for projection is a bit wrong but don't worry, projection is actually pretty simple.

$x_{Screen} = x_{World} / z_{World}\\
y_{Screen} = y_{World} / z_{World}$

If you want to move the center of projection, you can do so with a screen space coordinate, by modifying the formula to be this:

$x_{Screen} = x_{World} / z_{World} - x_{ScreenCenter}\\
y_{Screen} = y_{World} / z_{World} - y_{ScreenCenter}$

If you want to re-introduce the $d$ value to control the distance from the eye to the projection plane, you can use the formula below.  Adjusting $d$ will make the camera look like it is zooming in or out.  Again, you can just set $d$ to 1 though and not deal with it.

$x_{Screen} = d*x_{World} / z_{World} - x_{ScreenCenter}\\
y_{Screen} = d*y_{World} / z_{World} - y_{ScreenCenter}$


Loading compressed image to GPU and reading back decompressed on CPU

Desktop OpenGL requires implementations to convert pixel data between the internal format of the image and the format you specify in the pixel transfer command.

OpenGL ES does not. Indeed, it is so serious about not allowing this that it actually changes the meaning of the parameters to functions like glTexImage*d. In ES, you cannot use sized internal formats (well, outside of texture storage calls). Instead, the internal format is kinda there. What really defines the internal format is the pixel format and type parameters.

That is, when you tell OpenGL ES that the pixel data you're providing looks like X, it requires the implementation to create a texture whose real internal format exactly matches that. And if you're using glTexSubImage*d or glGetTexImage, you must provide pixel transfer parameters that match those you provided to the glTexImage*d call.

So OpenGL ES doesn't allow any of that pixel conversion stuff.


How to scale signed distance field fonts properly?

After some trials I've realized my problem was a programming error and not an actual issue with SDF fonts.
Originally I was rendering the numbers on the texture of the canvas, which had a fixed scale. I thought that it will redraw the quads during the zoom but it wasn't the case. After I've separated the character drawing from the canvas and scaled them separately they finally look sharp.
The linked comments about the small text rendering was slightly misleading. They do have some issues with smaller scale, but that's only the case when the view is zoomed out.


The Opengl deals this problem automatically?

How could it? At no point do you inform OpenGL that you only want 900 invocations. You have exactly 2 mechanisms to control the number of invocations: the work group size in the shader, and the work group count in the dispatch call. That determines how many invocations there are.

If you want a variable number of invocations with a fixed work group size, then your shader has to be given the actual maximum number of operations, and you have to explicitly not do anything for invocations beyond that size.


Separating photos from other images algorithmically?

Difficulty depends on what you really want to classify, i.e. what exactly your data base of images is made ok.
For simple hand-drawn or computer art, I would think of just looking the color histogram (but for complex art with gradients; but still these gradients are likely to induce a very smooth colormap).

At the other extreme, distinguishing realistic computer graphics image from reality is probably impossible.


A recent approach for subsurface scattering

As mentioned in the comments, I would highly suggest starting with Full Volumetric Scattering. This is two fold:

Since you are doing path tracing, adding volumetrics isn't super difficult.
Fully understanding how full volumetric scattering works will be a great basis for understanding the estimations. In addition, it can provide great "references" to see if your estimations are performing well / correctly.

With that in mind, below is a basic introduction to how to implement Full Volumetric Scattering in a backwards path tracer.

To start, let's look at the code for a backwards path tracer with only reflection, no transmission / refraction:

void RenderPixel(uint x, uint y, UniformSampler *sampler) {
    Ray ray = m_scene->Camera->CalculateRayFromPixel(x, y, sampler);

    float3 color(0.0f);
    float3 throughput(1.0f);
    SurfaceInteraction interaction;

    // Bounce the ray around the scene
    const uint maxBounces = 15;
    for (uint bounces = 0; bounces < maxBounces; ++bounces) {
        m_scene->Intersect(ray);

        // The ray missed. Return the background color
        if (ray.GeomID == INVALID_GEOMETRY_ID) {
            color += throughput * m_scene->BackgroundColor;
            break;
        }

        // Fetch the material
        Material *material = m_scene->GetMaterial(ray.GeomID);
        // The object might be emissive. If so, it will have a corresponding light
        // Otherwise, GetLight will return nullptr
        Light *light = m_scene->GetLight(ray.GeomID);

        // If this is the first bounce or if we just had a specular bounce,
        // we need to add the emmisive light
        if ((bounces == 0 || (interaction.SampledLobe & BSDFLobe::Specular) != 0) && light != nullptr) {
            color += throughput * light->Le();
        }

        interaction.Position = ray.Origin + ray.Direction * ray.TFar;
        interaction.Normal = normalize(m_scene->InterpolateNormal(ray.GeomID, ray.PrimID, ray.U, ray.V));
        interaction.OutputDirection = normalize(-ray.Direction);

        // Calculate the direct lighting
        color += throughput * SampleLights(sampler, interaction, material->bsdf, light);

        // Get the new ray direction
        // Choose the direction based on the bsdf
        material->bsdf->Sample(interaction, sampler);
        float pdf = material->bsdf->Pdf(interaction);

        // Accumulate the weight
        throughput = throughput * material->bsdf->Eval(interaction) / pdf;

        // Shoot a new ray

        // Set the origin at the intersection point
        ray.Origin = interaction.Position;

        // Reset the other ray properties
        ray.Direction = interaction.InputDirection;
        ray.TNear = 0.001f;
        ray.TFar = infinity;

        // Russian Roulette
        if (bounces > 3) {
            float p = std::max(throughput.x, std::max(throughput.y, throughput.z));
            if (sampler->NextFloat() > p) {
                break;
            }

            throughput *= 1 / p;
        }
    }

    m_scene->Camera->FrameBufferData.SplatPixel(x, y, color);
}

In English:

Shoot a ray through the scene.
Check if we hit anything. If not we return the skybox color and break.
If this is the first ray, or we just bounced off a specular surface, check if we hit a light. If so, we add the light emission to our color accumulation.
Sample the direct lighting.
Choose a new direction for the next ray. We can do this uniformly, or importance sample based on the BRDF.
Evaluate the BRDF and accumulate it.
Create a new ray based on our chosen direction and where we just came from.
[Optional] Use Russian Roulette to choose if we should terminate the ray.
Goto 1.

For more details on Direct Light Sampling, see this answer.

BSDFs

To add transmission, we first need to upgrade some of our BRDF materials to full BSDFs, with a BTDF function.

For example, a BSDF for an ideal specular dielectric (ie. glass) is:

class IdealSpecularDielectric : public BSDF {
public:
    IdealSpecularDielectric(float3 albedo, float ior)
        : BSDF(BSDFLobe::Specular, albedo),
          m_ior(ior) {
    }

private:
    float m_ior;

public:
    float3 Eval(SurfaceInteraction &interaction) const override {
        return m_albedo;
    }

    void Sample(SurfaceInteraction &interaction, UniformSampler *sampler) const override {
        float VdotN = dot(interaction.OutputDirection, interaction.Normal);
        float IORo = m_ior;
        if (VdotN < 0.0f) {
            IORo = 1.0f;
            interaction.Normal = -interaction.Normal;
            VdotN = -VdotN;
        }

        float eta = interaction.IORi / IORo;
        float sinSquaredThetaT = SinSquaredThetaT(VdotN, eta);
        float fresnel = Fresnel(interaction.IORi, IORo, VdotN, sinSquaredThetaT);

        float rand = sampler->NextFloat();
        if (rand <= fresnel) {
            // Reflect
            interaction.InputDirection = reflect(interaction.OutputDirection, interaction.Normal);
            interaction.SampledLobe = BSDFLobe::SpecularReflection;
            interaction.IORo = interaction.IORi;
        } else {
            // Refract
            interaction.InputDirection = refract(interaction.OutputDirection, interaction.Normal, VdotN, eta, sinSquaredThetaT);
            interaction.SampledLobe = BSDFLobe::SpecularTransmission;
            interaction.IORo = IORo;
        }

        if (AnyNan(interaction.InputDirection)) {
            printf("nan");
        }
    }

    float Pdf(SurfaceInteraction &interaction) const override {
        return 1.0f;
    }
};

The interesting part of the code is Sample(). This is where a ray will choose whether to reflect or refract. How we do this is really up to us, as shown in this answer. However, an obvious choice would be to sample based on the Fresnel equations.

In the case of an ideal Specular Dielectric, there are only two potential output directions: perfect refraction or perfect reflection. So we evaluate the Fresnel and randomly choose to refract / reflect with proportion equal to the Fresnel.

Media

Next, we need to talk about media, and how they affect light bouncing around. As explained by Nathan Reed in this answer:

  The way I like to think about volume scattering is that a photon traveling through a medium has a certain probability per unit length of interacting (getting scattered or absorbed). As long as it doesn't interact, it just goes in a straight line unimpeded and without losing energy. The greater the distance, the greater the probability that it interacts somewhere in that distance. The interaction probability per unit length is the coefficient $\sigma$ that you see in the equations. We usually have separate coefficients for scattering and absorption probabilities, so $\sigma = \sigma_s + \sigma_a$

  This probability per unit length is exactly the origin of the Beer-Lambert law. Slice a ray segment into infinitesimal intervals, treat each interval as an independent possible place to interact, then integrate along the ray; you get an exponential distribution (with rate parameter σσ) for the probability of interaction as a function of distance.

  You can technically choose the distance between events however you want, as long as you correctly weight the path for the probability that a photon can make it between two adjacent events without interacting with the medium. In other words, each path segment within the medium contributes a weight factor of $e^{-\sigma x}$, where $x$ is the length of the segment.

  Given this, a usually good choice for the distance is to importance-sample it from the exponential distribution. In other words, you set $x = -(\ln \xi)/\sigma$

In summary:

A ray travelling through a medium has some probability to:

Be absorbed
Be scattered

Since this is Monte Carlo Integration, we are free to choose the interaction distance however we want. However, a good choice is to importance sample from an exponential distribution similar to Beer-Lambert.

I chose to implement this with a scattering coefficient, and letting Beer-Lambert take care of the absorption coefficient.

class Medium {
public:
    Medium(float3 absorptionColor, float absorptionAtDistance)
        : m_absorptionCoefficient(-log(absorptionColor) / absorptionAtDistance) {
        // This method for calculating the absorption coefficient is borrowed from Burley's 2015 Siggraph Course Notes "Extending the Disney BRDF to a BSDF with Integrated Subsurface Scattering"
        // It's much more intutive to specify a color and a distance, then back-calculate the coefficient
    }
    virtual ~Medium() {
    }

protected:
    const float3a m_absorptionCoefficient;

public:
    virtual float SampleDistance(UniformSampler *sampler, float tFar, float *weight, float *pdf) const = 0;

    virtual float3a SampleScatterDirection(UniformSampler *sampler, float3a &wo, float *pdf) const = 0;
    virtual float ScatterDirectionPdf(float3a &wi, float3a &wo) const = 0;

    virtual float3 Transmission(float distance) const = 0;
};

A Non-scattering medium is one where a photon travels straight through, but it is attenuated along the way according to Beer-Lambert:

class NonScatteringMedium : public Medium {
public:
    NonScatteringMedium(float3 color, float atDistance)
        : Medium(color, atDistance) {
    }

public:
    float SampleDistance(UniformSampler *sampler, float tFar, float *weight, float *pdf) const override {
        *pdf = 1.0f;
        return tFar;
    }

    float3a SampleScatterDirection(UniformSampler *sampler, float3a &wo, float *pdf) const override {
        return wo;
    }
    float ScatterDirectionPdf(float3a &wi, float3a &wo) const override {
        return 1.0f;
    }

    float3 Transmission(float distance) const override {
        return exp(-m_absorptionCoefficient * distance);
    }
};

An Isotropic Scattering Medium allows the ray to have scattering (reflection) events while the ray is traveling through it. The ray can be reflected in any direction in the sphere around the interaction point (hence Isotropic).

The distance between reflections is random, based on an exponential "scatter" probability.

class IsotropicScatteringMedium : public Medium {
public:
    IsotropicScatteringMedium(float3 absorptionColor, float absorptionAtDistance, float scatteringDistance)
        : Medium(absorptionColor, absorptionAtDistance),
          m_scatteringCoefficient(1 / scatteringDistance) {
    }

private:
    float m_scatteringCoefficient;

public:
    float SampleDistance(UniformSampler *sampler, float tFar, float *weight, float *pdf) const override {
        float distance = -logf(sampler->NextFloat()) / m_scatteringCoefficient;
        // If we sample a distance farther than the next intersecting surface, clamp to the surface distance
        if (distance >= tFar) {
            *pdf = 1.0f;
            return tFar;
        }

        *pdf = std::exp(-m_scatteringCoefficient * distance);
        return distance;
    }
    float3a SampleScatterDirection(UniformSampler *sampler, float3a &wo, float *pdf) const override {
        *pdf = 0.25f * M_1_PI; // 1 / (4 * PI)
        return UniformSampleSphere(sampler);
    }

    float ScatterDirectionPdf(float3a &wi, float3a &wo) const override {
        return 0.25f * M_1_PI; // 1 / (4 * PI)
    }
    float3 Transmission(float distance) const override {
        return exp(-m_absorptionCoefficient * distance);
    }
};

At high scattering distances, the material behaves almost like a NonScatteringMedium, because it has a very small probability of scattering before it travels through the medium.

At low scattering distances, the material behaves like wax or jade.

As the scattering distance gets lower and lower, we have a higher and higher chance to scatter while traveling through the medium. This presents a few issues:

The number of bounces per ray is going to explode.
If Russian Roulette kills the ray, or if you go over "maxBounces", you will "lose" any color contribution from inside the medium.

Thus, you need to set maxBounces to some high number, and rely on Russian Roulette to terminate rays in the general case. In addition, IsotropicScatteringMedium is extremely inefficient for representing mostly opaque materials. To get better performance, we should use a non Isotropic phase function, like Henyey-Greenstein or Schlick. These bias the scattering in a certain direction. So we could set them to have high back-scattering, thereby lessening the issue of "lost" rays.

Putting it all together

So, with this new information, how do we modify the integrator to understand BSDFs and media?

First, we need to start tracking what medium we are currently in, and its Index of Refraction (IOR).

SurfaceInteraction interaction;
interaction.IORi = 1.0f; // Vacuum
Medium *medium = nullptr; // nullptr == vacuum

Then, we break the integrator up into two parts: transmission and material interaction.

void RenderPixel(uint x, uint y, UniformSampler *sampler) const {
    Ray ray = m_scene->Camera->CalculateRayFromPixel(x, y, sampler);

    float3 color(0.0f);
    float3 throughput(1.0f);
    SurfaceInteraction interaction;
    interaction.IORi = 1.0f; // Air
    Medium *medium = nullptr;
    bool hitSurface = false;

    // Bounce the ray around the scene
    uint bounces = 0;
    const uint maxBounces = 1500;
    for (; bounces < maxBounces; ++bounces) {
        m_scene->Intersect(ray);

        // The ray missed. Return the background color
        if (ray.GeomID == INVALID_GEOMETRY_ID) {
            color += throughput * m_scene->BackgroundColor;
            break;
        }

        // We hit an object
        hitSurface = true;

        // Calculate any transmission
        if (medium != nullptr) {
            float weight = 1.0f;
            float pdf = 1.0f;
            float distance = medium->SampleDistance(sampler, ray.TFar, &weight, &pdf);
            float3 transmission = medium->Transmission(distance);
            throughput = throughput * weight * transmission;

            if (distance < ray.TFar) {
                // Create a scatter event
                hitSurface = false;

                ray.Origin = ray.Origin + ray.Direction * distance;

                // Reset the other ray properties
                float directionPdf;
                float3a wo = normalize(ray.Direction);
                ray.Direction = medium->SampleScatterDirection(sampler, wo, &directionPdf);
                ray.TNear = 0.001f;
                ray.TFar = infinity;
                ray.GeomID = INVALID_GEOMETRY_ID;
                ray.PrimID = INVALID_PRIMATIVE_ID;
                ray.InstID = INVALID_INSTANCE_ID;
                ray.Mask = 0xFFFFFFFF;
                ray.Time = 0.0f;
            }
        }

        if (hitSurface) {
            // Fetch the material
            Material *material = m_scene->GetMaterial(ray.GeomID);
            // The object might be emissive. If so, it will have a corresponding light
            // Otherwise, GetLight will return nullptr
            Light *light = m_scene->GetLight(ray.GeomID);

            // If this is the first bounce or if we just had a specular bounce,
            // we need to add the emmisive light
            if ((bounces == 0 || (interaction.SampledLobe & BSDFLobe::Specular) != 0) && light != nullptr) {
                color += throughput * light->Le();
            }

            interaction.Position = ray.Origin + ray.Direction * ray.TFar;
            interaction.Normal = normalize(m_scene->InterpolateNormal(ray.GeomID, ray.PrimID, ray.U, ray.V));
            interaction.OutputDirection = normalize(-ray.Direction);
            interaction.IORo = 0.0f;

            // Calculate the direct lighting
            color += throughput * SampleOneLight(sampler, interaction, material->bsdf, light);

            // Get the new ray direction
            // Choose the direction based on the bsdf
            material->bsdf->Sample(interaction, sampler);
            float pdf = material->bsdf->Pdf(interaction);

            // Accumulate the weight
            throughput = throughput * material->bsdf->Eval(interaction) / pdf;

            // Update the current IOR and medium if we refracted
            if (interaction.SampledLobe == BSDFLobe::SpecularTransmission) {
                interaction.IORi = interaction.IORo;
                medium = material->medium;
            }

            // Shoot a new ray

            // Set the origin at the intersection point
            ray.Origin = interaction.Position;

            // Reset the other ray properties
            ray.Direction = interaction.InputDirection;
            if (AnyNan(ray.Direction))
                printf("bad");
            ray.TNear = 0.001f;
            ray.TFar = infinity;
            ray.GeomID = INVALID_GEOMETRY_ID;
            ray.PrimID = INVALID_PRIMATIVE_ID;
            ray.InstID = INVALID_INSTANCE_ID;
            ray.Mask = 0xFFFFFFFF;
            ray.Time = 0.0f;
        }

        // Russian Roulette
        if (bounces > 3) {
            float p = std::max(throughput.x, std::max(throughput.y, throughput.z));
            if (sampler->NextFloat() > p) {
                break;
            }

            throughput *= 1 / p;
        }
    }

    if (bounces == maxBounces) {
        printf("Over max bounces");
    }

    m_scene->Camera->FrameBufferData.SplatPixel(x, y, color);
}

In English:

Shoot a ray through the scene.
Check if we hit anything. If not we return the skybox color and break.
Check if we are currently in a medium (not in a vacuum).

Sample the medium for a distance.
Evaluate the medium's transmission and accumulate the throughput.
If the scatter distance is less than the distance from the ray origin to the next surface, make a scatter event.
Sample the medium for a scatter direction.

Check if we hit a surface (ie, we didn't have a scatter event).

If this is the first ray, or we just bounced off a specular surface, check if we hit a light. If so, we add the light emission to our color accumulation.
Sample the direct lighting.
Choose a new direction for the next ray. We can do this uniformly, or importance sample based on the BRDF.
Evaluate the BRDF and accumulate it.
If we refracted into the material, update the IOR and medium for the next bounce.

Create a new ray based on our chosen direction and where we just came from.
[Optional] Use Russian Roulette to choose if we should terminate the ray.
Goto 1.


Formula for the ambient light intensity of a scene?

Using an 'ambient light' term in your lighting calculations is an approximation of the indirect light reflected around the scene. Usually it's up to the artist to choose an appropriate value. There isn't really a formula, because the indirect light is dependent on many factors:

Light intensity
Light shape
The BRDFs of the surfaces the light will bounce off of
Visibility to the light source (Think the back of a deep cave)
etc.

You could use some kind of formula to get an approximate value, based on these factors, but, in my opinion, it's simpler just to do it manually per scene. Or, you can remove it entirely, and use more sophisticated methods to calculate indirect lighting.


In the rendering pipeline, in what coordinate space are vertices when they arrive at the primitive assembly stage?

As pointed out by Simon F, the clipping stage needs to clip against edges (and thus triangles/primitives). Thus the primitive assembly stage cannot happen after clipping, even though the Pipeline is described such that primitive assembly happens after clipping.

Slide 16 of Ohio university's Han-Wei Shen's 2010 course The OpenGL Rendering Pipeline suggests that vertices are transformed to Clip Space, then the primitive assembly happens and only afterwards is clipping actually done.

According to Khronos' wiki the primitive assembly happens after clipping, which includes the following:

  The vertex positions are transformed from clip-space to window space via the Perspective Divide and the Viewport Transform.

Window Space is described as the space after the viewport transform and that means after NDC space:

  The viewport transform defines the transformation of vertex positions from NDC space to window space. These are the coordinates that are rasterized to the output image.

Since the viewport transform includes a scaling for the width and height of the viewport, as well as a translation with the viewports position on screen, you should end up with pixel coordiantes of your screen.


Dealing with blown-out highlights in PBR

Using physically based BRDFs only makes sense if your entire pipeline is built for physical units - the extreme range of values can't be displayed properly without some form of tone mapping. You didn't include that part of the code but from the looks of it I'd say you're doing a simple clamp() followed by linear->sRGB conversion, which causes the bad highlights.

Since you're not ready to run your entire frame in HDR, you could apply something simple like a sigmoid operator to just the specular component. This may look a bit off next to other BRDFs but should make those highlights much more pleasant.


What graphic languages are simpler than SVG for database diagrams?

Graphviz is a popular tool for making network diagrams. It takes a text format ("DOT language") describing the nodes and edges as input, and can render the resulting diagram to SVG or other image formats.


Why is eye-based ray tracing preferred over light-based ray tracing?

As you point out there is problems about which pixel on the camera to connect to. You probably wouldn't notice where you took the sample on an area light because generally the light is pretty uniform throughout, and even if it is textured you can just take a few samples and it will average to look ~correct. However if you connect to a random pixel, you would very easily notice that some pixels are black or have aliasing problems. By starting at the camera it's easier to ensure that all pixels are being sampled enough.
You are also only considered the light from one bounce. If you also want to get indirect lighting then your statement about weights comes into play again, and you are more likely to get a good path by starting at the camera.
Finally there are many techniques which start from the light! And techniques which start at both ends and try and connect. Try searching for photon mapping and bi-directional path tracing. It just turns out that forward path tracers generally work the best in most situations. Renderman has an integrator which combines all these techniques  which you can read about here https://rmanwiki.pixar.com/display/REN/PxrVCM .


as paulHK said:

One fundamental issue here is you're rotating in destination space rather than source space (e.g. apply the shear on source coordinates and output a rectangle). This way you can guarantee every pixel in the output is drawn once and only once and there are no gaps in between.

and the line that's causing the 180 problem is this line:
tangent=math.tan(angle/2)
If the angle is 180, the tan of 90 will be basically infinity.
This is a graph of 1 / (cos(rotation) ^ 2), one of the shearing calculations the narrator at datagenetics.com uses:

That means his method stalls at certain angles.
In all the code i wrote, I never needed to call tan(x).
Use cos(x) and sin(x) instead, also use as few divisions as possible that could create infinity. Most rendering engines tackle this problem by limiting the angle of the camera to 89 degrees.
To get unaliased rotated images, i suggest a better drawing method, which computer graphics use today:
Use a 3x3 rotation matrix and a translation matrix , and loop over the pixels image that you want to draw on:
cos(angle) -sin(angle) + offset x
sin(angle) cos(angle)  + offset y
0          0           1
to get this matrix, multiply first:
1 0 -[rotation centre x]
0 1 -[rotation centre y]
0 0 1
then:
cos(angle) -sin(angle) 0
sin(angle) cos(angle)  0
0          0           1
and then:
1 0 [rotation centre x]
0 1 [rotation centre y]
0 0 1
and then(if needed to translate):
1 0 [translation x]
0 1 [translation y]
0 0 1
Multiply the position on the target image by the INVERSE of the combined matrix to get the position on the source image.
If the position is in the bounds of the source image,
fill the target pixel with the source pixel at that location.


PDF when sampling from Rectangular Area Lights

The correct PDF for an area light is indeed $1/A$. The reason why you see the area term at the top is because you are importance sampling the light, meaning you need to divide by its PDF. This is a Monte Carlo estimator with one sample. Shirley simplifies this by simply writing $1/(1/A) = A$.

I am not sure how he got the $1 / \frac{\|V_1 \times V_2\|}{2}$.


Copying the bytes of a JPEG file to the framebuffer device doesn't display the image represented by the JPEG file. The file needs to be decoded by an application that understands the JPEG format, to get the actual image. Your system is working normally. You can use a command like display to decode the file and draw the image to the screen.


How to choose which reflection probe to use?

Are you trying to make your own rendering engine?

Here is a solution that prioritizes visual quality ; it is slightly more complex than the object-based solution you are referring to.
For better visual quality your shading should be on a per-pixel basis.
(If you highly prioritize performance, or do not require detailed reflections, you should probably stick to the object-based solution though).

If you are doing deferred rendering, instead of selecting only a single cubemap for each pixel, you can use a formula that combines all the cubemaps that are close enough to a pixel.

A trick sometimes used to achieve this is to add a pass that computes the number of cubemaps that affect a single pixel (so you can do averages in the final shading pass).
If you are doing forward rendering for transparent objects for example, I think it might be a bit trickier to compute the average of cubemaps.
That's because you need to have pixel positions already available before shading ; you might think it could be possible to do it if you had a kind of depth pre-pass buffer for that purpose, however for transparent objects you usually do not want to write their depth in that buffer, because this leads to artifacts when transparent objects are overlapping.
You would therefore need to have a rather complex logic to handle those cases...

However even with forward rendering you can still select the closest cubemap on a per-pixel basis, although without the "averaging" formula the transitions between the boundaries of two cubemap influence zones will be very visible and reflections will not look realistic.


How to use Shaders with vertex & index buffers?

Apparently there was just a problem with the loading of the shader. A bug that crept into the code from an old project...

EDIT: The GLuint of the compiled shader was never saved in the class and only local to the load function. Just a stupid mistake really.


How to deform some mesh so that it fits along a spline curve?

I have no knowledge of the literature on the topic, but I did something very similar to what you're asking some time ago: I wanted to generate lathe meshes and bend them according to a spline. I think the same technique could be adapted to your case quite easily.

First you would need to define what your default axis is: if the input mesh corresponds to the case when the spline curve is a straight line, where is that line in 3D? It could be defined arbitrarily as the Y axis, the PCA of the mesh, or some manually defined axis.

The algorithm is then:

for each vertex v
{
    matrix4 M = ComputeTransformFromAxisToSpline(v);
    v = ApplyTransform(v, M);
}

The tricky part is obviously to define a transform $M$. You could do something like:

matrix4 ComputeTransformFromAxisToSpline(vector3 v)
{
    float h = GetPositionOnAxis(v, originalAxis);
    vector3 p1 = GetPointOnSpline(h);
    vector3 p2 = GetPointOnSpline(h + dh); // dh is a small delta
    vector3 u = normalize(p2 - p1);

    // Define an orthogonal basis
    vector3 v = v0;
    vector3 w = normalize(cross(u, v));
    vector3 v = normalize(cross(w, u));

    return MakeTransform(u, v, w, p1) * MakeTranslation(originalAxis, -h);
}

$v_0$ is going to be the problem. I have tried two approaches:

Use an absolute value, like an axis orthogonal to the default axis. This will work only if the spline never gets parallel to that axis. It may also lead to undesired shapes.
Start with an absolute value for the points at $h=0$, then reuse the previous $v$ for bigger $h$ values. This means you have to treat vertices ordered by $h$. In my case this was trivial because I was generating the vertices, but in your case that would probably mean to sort them. The result is a mesh that will tend to twist in curves, which may or may not be desirable.

Another possibility would be to define the orientation by having more information in the curve, but I haven't tried that approach as I was already satisfied with the result.


Algorithm for constructing doubly connected edge list in 3d

The problem statement in the linked post says that only the vertices and edges are input to the algorithm, but I don't think it's possible to do this unambiguously in 3D without additional input about the faces of the mesh. In the 2D case, since the input is specified to be a planar graph, the faces are unambiguous: any region in the plane contained within an edge loop, and empty of edges in its interior, is a face. However, in 3D, you don't know which edge loops should be faces and which shouldn't.
Consider a cube, represented as just vertices and edges: you would want the "usual" 6 sides of the cube to be treated as faces, but you wouldn't want the algorithm to create additional faces that stretch across the diagonal of the cube, internally. But there is no way for the algorithm to know that. Moreover, the vertex/edge mesh might not even be possible to assign faces to in a sensible way; it might have non-planar faces, might be non-orientable (e.g. Möbius strip) or non-manifold.
Typically in 3D applications we would have the mesh faces already defined, and we can assume it to be a manifold, orientable mesh. With the face data as input to the algorithm, such as by lists of edges or vertices in counterclockwise order around each face, then it becomes straightforward (if tedious) to figure out the half-edge relationships.


Object rotating around origin instead of object center?

Your triangle's coordinates:
(-0.1f,0.7f)
(0.0f,0.8f)
(0.1f,0.7f)
are defined with the origin at the center of the screen.

Multiplying a rotation matrix by a vertex position will rotate around the point (0,0,0). In your case, that is the center of the screen.

Possible solution:

Define the vertices as: (-0.1f,-0.5f) (0.0f,0.5f,) and (0.1f,-0.5f)
Rotate the vertices with your rotation matrix (Like you did):

Vector2D v = Matrix2D.MatrixTimesVector2D(m, p.getPosition());

Vector2D v2 = Matrix2D.MatrixTimesVector2D(m, p2.getPosition());

Vector2D v3 = Matrix2D.MatrixTimesVector2D(m, p3.getPosition());
If you want your triangle at the top of you screen, use a translation matrix:

I'm not sure what language/library you are using, but create a translation matrix by yourself or using a built in fuction. Above dx, dy and dz are the translations in x,y and z.

Multiply your positions v, v2 and v3 by the translation matrix.

Google tranlsation and rotation matrix and you will understand better! Good luck!


This is a very broad question. The basic way to do this is to generate 2 frames - one for each eye. You can do this using any modern 3D API from OpenGL to DirectX to Vulkan, Metal, or DX12.

You'll need to understand things like interocular distance, depth cues, and 3D compositing.

You'll also need some way to allow the user to move the curser in and out, so you'll want to read up on various user interface, visual interface, and user experience (UI, VI, and UX, respectively) techniques.


How do you design the code of a graphics engine to target multiple frameworks

You do it by abstracting away the underlying technology. You know, for example, at a very high level that you'll need to work with images, geometry, and I'm guessing you'll have to write shaders to do your rendering. The textures and geometry can be in the same format for both systems.

So from that, you can design an interface for rendering. You'll probably have a scene of some sort that has the geometry in it. The geometry will likely have the textures applied to it. When rendering the geometry you'll need to use a shader. Those might be good places to start with your abstraction. You might need a scene object that holds all the different objects you want to render. Each object is made up of geometry and the images that will be applied to the geometry. The scene object, the geometry objects and the images can be the same between OpenGL and Metal because they aren't specific to either. (I'm assuming here that the geometry is just a buffer of floating point coordinates, normals, etc. and the image starts out as a buffer of RGBA pixels on the CPU.)

Next you need to come up with a way to render those things on the hardware you're currently running on with the framework you wish to use. So you create an abstract interface. Perhaps you have a Renderer object. It would have methods that allow you to add a piece of geometry with an image to use as the texture. It would allow you to associate a shader with the geometry. And it would have a method to render the geometry to either a texture or the screen.

You then either subclass the Renderer class (assuming you're using something like C++) or write a new class that implements the Renderer protocol (if you're using something like Objective-C or Swift). You have 2 different subclasses or implementations. One that does things using OpenGL and one that does things using Metal.

So you might have an OGLRenderer and a MTLRenderer. The OGLRenderer class will implement the addGeometry() method by creating a Vertex Buffer Object and eventually calling glBufferData() to actually upload the geometry to the GPU. In your MTLRenderer you'll create a MTLBuffer via -newBufferWithLength:options:, and then copying the data into the buffer with memcpy(). And so on for the other functions.

So when writing the main code for your application, you'll code to the Renderer interface without knowing whether the underlying renderer that you're actually going to use is an OpenGL renderer or a Metal renderer. (In fact, you could even make it so they can be switched at run time if that would help in some way.) You can have a factory that either decides at run time which to use based on the OS you're on, or you can decide at compile time by only including the appropriate subclass.


Do straight lines always remain straight when projected with a perspective camera?

"A not so simple approach". I may have messed a little bit too much with grouping the terms, do forgive my elementary math skills, it's a side effect of using tools like wolfram and mathematica too much.

Without loss of generality we can assume that the near plane is given as $z=1$ (that is having normal $(0,0,1)$ and a point on it $(0,0,1)$) and the camera center (the reflected pinhole camera aperture point) is at $(0,0,0)$. Let $p,q \in \mathbb{R}^3$ be two points on a line (non-degenerate, meaning $p\ne q$, otherwise it's just a point and the projection is also a point - unless it's at $(0,0,0)$ then it's undefined). If the line defined by $p,q$ passes through $(0,0,0)$ the projection is a point and the requirement is trivially satisfied.
Now let this not be the case. A perspective projection onto the plane through $(0,0,0)$ is given by $p' = \frac{p}{p_z}$, and $q' = \frac{q}{q_z}$. The line defined by $p,q$ is given as $r(\lambda) = p + \lambda(q-p), \lambda \in  \mathbb{R}$. Its projection is given by $r'(\lambda) = \frac{r(\lambda)}{r_z(\lambda)}$. Since $z=1$ for all projected points, then we need only study the $x,y$ coordinates. We want to show that any projected point $r'(\lambda)$ can be written as an affine combination of $p',q'$ (this defines a line). Taking that into account:
$$p'+\mu(q'-p') = \frac{r(\lambda)}{r_z(\lambda)}$$
$$(1-\mu)\frac{p}{p_z} + \mu\frac{q}{q_z} = \frac{p + \lambda(q-p)}{p_z + \lambda(q_z-p_z)}$$
$$(1-\mu)q_z(p_z+\lambda(q_z-p_z))p + \mu p_z(p_z+\lambda(q_z-p_z))q = q_zp_z(p + \lambda(q-p))$$
$$q_zp_z(p - \lambda p) + q^2_z\lambda p - \mu q_z(p_z+\lambda(q_z-p_z))p + \mu p_z(p_z+\lambda(q_z-p_z))q - q_zp_z(p - \lambda p) - q_zp_z\lambda q = 0$$
$$(1-\mu)\lambda q^2_z p -\mu(1-\lambda)q_zp_z p + \mu(1-\lambda)p^2_zq - (1-\mu)\lambda p_zq_z q = 0$$
$$(1-\mu)\lambda q_z(q_zp - p_zq) - \mu(1-\lambda)p_z(q_zp-p_zq) = 0$$
$$((1-\mu)\lambda q_z - \mu(1-\lambda)p_z)(q_zp - p_zq) = 0$$

For this to be satisfied one of the terms must be $0$. The first case concerns the second term: $p' = q'$, which we already considered as the trivial case. Then we must show that we can always pick a $\mu$ such that $(1-\mu)\lambda q_z - \mu(1-\lambda)p_z = 0$. Solve for $\mu$ and you get: $\mu = \frac{\lambda q_z}{(1-\lambda)p_z + \lambda q_z}$. Thus I have not only found that it is a straight line, but also the relationship between the parameterizations.

In general a projection onto a different 2-manifold (not a plane) will not result in the lines remaining straight (think of a sphere or a cylinder).


What effects are used to light this scene?

The rendering engine that rendered this image is using a technique called deferred rendering, which first writes scene information (as seen from the camera) to seperate buffers (such as position, normal, and albedo buffers) and then calculates lighting for the image based off this information. It is important that we have position and normal buffers on top of the albedo buffer so that lighting can be calculated at each pixel (these buffers are not visible in your example, but they are there under the hood). As an example, take a look at the image below that shows the different buffers of an engine I worked on - it may help to better understand what is going on.

Once a buffer has been made with all of the lighting calculations, you can then achieve post processing effects by processing the buffer with various algorithms to produce the final image. Some post processing effects that are visible in your image are bloom (which adds "glow" to bright spots), motion blur and an implementation of screen space ambient occlusion (SSAO).
A full list of the rendering features can actually be found at the 1:02 mark of the video, I have attached the image here to make it easy to see. It may be worth googling what each individual post-processing effect does so that you can better understand the effect they have on the overall image.

Hope this was of help :)


DLT algorithm (4+ points) not working, what's missing?

Just realized that this is an equation involving homogeneous vectors; thus the 3-vectors should not equal, they have the same direction but may differ in magnitude by a nonzero scale factor.


Converting color to equivalent with opacity

This depends on what function you're blending with. Since you don't say where you're doing this, I'll just take OpenGL as an example.
As you can see in the API, there are several ways of blending two colours (or rather how to interpret these two colours for a given blend function).
You speak of a colour and a background colour, so let's say your result is $c_r$, your object in the foreground has colour $c_s$ (for Source, the new colour value being "added" someway) and your background has colour $c_d$ (for Destination, the old colour value that has been in the buffer already).
Let $c_s = \left(\begin{array}{c} R_s\\G_s\\B_s\\A_s \end{array}\right)$ (and analogously for $c_r$ and $c_d$).

To determine the blended RGBA values of a pixel, the system uses the following equations:
$R_d = min(k_{R'}, R_s s_R + R_d d_R) \ G_d = min(k_{G'}, G_s S_G + G_d d_G) \ B_d = min(k_{B'}, B_s s_B + B_d d_B) A_d = min(k_{A'}, A_s s_A + A_d d_A)$

A bit confusing with the doubled $R_d$ and so forth, so I'll rewrite it to the denomination I defined earlier:
$R_r = min(k_{R'}, R_s s_R + R_d d_R) \ G_r = min(k_{G'}, G_s S_G + G_d d_G) \ B_r = min(k_{B'}, B_s s_B + B_d d_B) A_r = min(k_{A'}, A_s s_A + A_d d_A)$
The min function is there for not going above a certain constant (and $k_{R'},\ k_{G'},\ k_{B'},\ k_{A'}$ are simply constants). For our purposes, let's just assume that we don't surpass this threshold, so we end up with
$R_r = R_s s_R + R_d d_R$ (and of course the same for the other channels).
The rest of the components $s_R$ and $d_R$ which are some scaling factors for combining the two colours.

Transparency is best implemented using blend function (GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) with primitives sorted from farthest to nearest.

This helps, because now we know what the factors $s_R$ and $d_R$ actually are: the alpha values of the Source.
This leaves us with
$R_r = R_s A_s + (1 - R_d A_d)$
Therefore, to come back to your example, we have (for the green channel, since that is not $255$):
$G_r = 230$, an undefined $\alpha$ value for the source, the background green $G_d = 255$ and probably a solid background, meaning $A_d = 255$. We need the values to be in $\left[0, 1\right]$, therefore we get $G_r = \frac{230}{255} = 0.90196078431$, $G_d = 1$ and $A_d = 1$.
Filling this in we get:
$0.90196078431 = G_s A_s + (1 - 1 \cdot A_s)$
Your question lets me believe, that you know $A_s$, thus only $G_s$ (and of course the same for the other colour channels) is missing. This can then be calculated by isolating the respective colour component.


map 16-bit integers to RGB colors with monotonic relative luminosity

Since you want monotonically increasing luminance, it's probably best to work in a color space that separates luma from chroma, such as a Y'UV space. You can assign values in Y'UV and then convert it to RGB using the matrix at that link.

Start by setting $Y' = N / (2^{16} - 1)$, where N is your input 16-bit integer. Now you just need to make up some function to use for the U and V components.

It wasn't clear to me if you want the chroma part of the color to vary continuously with the input N, or be pseudo-random. For a continuous curve, you could just have it be a circle in UV space:
$$
U = 0.5 \cos(2\pi N/2^{16}) \\
V = 0.5 \sin(2\pi N/2^{16})
$$
or whatever other curve you feel like coming up with. For a pseudo-random chroma, you could use a hash function, or a Halton sequence or another low-discrepancy sequence.


In line 15 use the half-open interval, i.e.,

if u>=t[0][i] and u<t[0][i+1]:

Otherwise, at knot values, you evaluate two basis functions at the k=1 basis when you only want one. This causes the wrong evaluation of the basis function at the knot values and therefore the spikes.


Fast and exact Geodesics on meshes, Backtracking confusion

The solution I found to this problem is to ALWAYS intersect the edge. In other words the distance to the geodesic source $v$ from a point $p$ is $||v-p'|| + ||p - p'||$ where $p'$ is the intersection of the line $\overline{vp}$ with the current edge. In cases where $p'$ is "behind" $p$ with respect to $v$ we set the distance to the edge to be infinity.
The above seems to work in the simple tests I have generated at minimum.


How to reduce the number of draw calls when rendering one object multiple times?

There are many, many ways to draw things in OpenGL, so this is naturally confusing sometimes. The first method you describe, setting the shader parameters and issuing one draw call per object is usually the most inefficient, due to the high API overhead. The second one, using instanced drawing is a much smarter approach for objects with the same parameters.

When it comes to particles, specifically, there are two other methods which I'm aware of and have tested:

The first one, more traditional and easier to implement, is to generate a unique quadrilateral for each particle in the application code. Then use one of the several optimized buffer streaming paths of OpenGL to upload this data and issue a single draw call. This is the most straightforward method and provides good results. It will involve very few API calls if you can map the vertex/index buffers (glMapBuffer/MapBufferRange).
Another method is to move the whole thing to a shader program, using Transform Feedback. This method is a little more complicated to get up and running, but you can find a lot of references on the subject, such as this tutorial. This one should be the optimal path, since it moves the whole simulation to the GPU.

Those are some of the optimized ways of rendering particle effects. But OpenGL provides several other rendering paths that are better for different cases, one of such is indirect draw, which isn't available on ES at the moment, but is probably one of the fastest drawing paths available on modern PC OpenGL. Transform Feedback also requires a geometry shader, so it is not available for current OpenGL-ES.

For more on the optimized rendering paths of OpenGL, I recommend watching this very good presentation by NVidia: Approaching Zero Driver Overhead in OpenGL. At the end of the talk, they show a very interesting benchmark of several methods and how they compare with the Direct3D equivalents.


This never writes anything to the output file because you haven't included any code to write to the output file. The only time you touch the file after opening it is in the line

png->MakeFromFILE(pFile);

which reads from the file into the SkData instance. It won't even do that in this case, because the file handle has to be open for reading only.

I'm no expert on Skia, but you probably want to use SkFILEWStream to write the file. Or just call png.bytes() and write the bytes to the file using the standard library.


How can virtual texturing actually be efficient?

Overview

The main reason for Virtual Texturing (VT), or Sparse Virtual Textures, as it is sometimes called, is as a memory optimization. The gist of the thing is to only move into video memory the actual texels (generalized as pages/tiles) that you might need for a rendered frame. So it will allow you to have much more texture data in offline or slow storage (HDD, Optical-Disk, Cloud) than it would otherwise fit on video memory or even main memory. If you understand the concept of Virtual Memory used by modern Operating Systems, it is the same thing in its essence (the name is not given by accident).

VT does not require recomputing UVs in the sense that you'd do that each frame before rendering a mesh, then resubmit vertex data, but it does require some substantial work in the Vertex and Fragment shaders to perform the indirect lookup from the incoming UVs. In a good implementation however, it should be completely transparent for the application if it is using a virtual texture or a traditional one. Actually, most of the time an application will mix both types of texturing, virtual and traditional.

Batching can in theory work very well, though I have never looked into the details of this. Since the usual criteria for grouping geometry are the textures, and with VT, every polygon in the scene can share the same "infinitely large" texture, theoretically, you could achieve full scene drawing with 1 draw call. But in reality, other factors come into play making this impractical.

Issues with VT

Zooming in/out and abrupt camera movement are the hardest things to handle in a VT setup. It can look very appealing for a static scene, but once things start moving, more texture pages/tiles will be requested than you can stream for external storage. Async file IO and threading can help, but if it is a real-time system, like in a game, you'll just have to render for a few frames with lower resolution tiles until the hi-res ones arrive, every now and then, resulting in a blurry texture. There's no silver bullet here and that's the biggest issue with the technique, IMO.

Virtual Texturing also doesn't handle transparency in an easy way, so transparent polygons need a separate traditional rendering path for them.

All in all, VT is interesting, but I wouldn't recommend it for everyone. It can work well, but it is hard to implement and optimize, plus there's just too many corner cases and case-specific tweaks needed for my taste. But for large open-world games or data visualization apps, it might be the only possible approach to fit all the content into the available hardware. With a lot of work, it can be made to run fairly efficiently even on limited hardware, like we can see in the PS3 and XBOX360 versions of id's Rage.

Implementation

I have managed to get VT working on iOS with OpenGL-ES, to a certain degree. My implementation is not "shippable", but I could conceivably make it so if I wanted and had the resources. You can view the source code here, it might help getting a better idea of how the pieces fit together. Here's a video of a demo running on the iOS Sim. It looks very laggy because the simulator is terrible at emulating shaders, but it runs smoothly on a device.

The following diagram outlines the main components of the system in my implementation. It differs quite a bit from Sean's SVT demo (link down bellow), but it is closer in architecture to the one presented by the paper Accelerating Virtual Texturing Using CUDA, found in the first GPU Pro book (link bellow).

Page Files are the virtual textures, already cut into tiles (AKA pages) as a preprocessing step, so they are ready to be moved from disk into video memory whenever needed. A page file also contains the whole set of mipmaps, also called virtual mipmaps.
Page Cache Manager keeps an application-side representation of the Page Table and Page Indirection textures. Since moving a page from offline storage to memory is expensive, we need a cache to avoid reloading what is already available. This cache is a very simple Least Recently Used (LRU) cache. The cache is also the component responsible for keeping the physical textures up-to-date with its own local representation of the data.
The Page Provider is an async job queue that will fetch the pages needed for a given view of the scene and send them to the cache.
The Page Indirection texture is a texture with one pixel for each page/tile in the virtual texture, that will map the incoming UVs to the Page Table cache texture that has the actual texel data. This texture can get quite large, so it must use some compact format, like RGBA 8:8:8:8 or RGB 5:6:5.

But we are still missing a key piece here, and that's how to determine which pages must be loaded from storage into the cache and consequently into the Page Table. That's where the Feedback Pass and the Page Resolver enter.

The Feedback Pass is a pre-render of the view, with a custom shader and at a much lower resolution, that will write the ids of the required pages to the color framebuffer. That colorful patchwork of the cube and sphere above are actual page indexes encoded as an RGBA color. This pre-pass rendering is then read into main memory and processed by the Page Resolver to decode the page indexes and fire the new requests with the Page Provider.

After the Feedback pre-pass, the scene can be rendered normally with the VT lookup shaders. But note that we don't wait for new page request to finish, that would be terrible, because we'd simply block on synchronous file IO. The requests are asynchronous and might or might not be ready by the time the final view is rendered. If they are ready, sweet, but if not, we always keep a locked page of a low-res mipmap in the cache as a fallback, so we have some texture data in there to use, but it is going to be blurry.

Others resources worth checking out

The first book in the GPU Pro series. It has two very good articles on VT.
Mandatory paper by MrElusive: Software Virtual Textures.
The Crytek paper by Martin Mittring: Advanced Virtual Texture Topics.
And Sean's presentation on youtube, which I see you've already found.

VT is still a somewhat hot topic on Computer Graphics, so there's tons of good material available, you should be able to find a lot more. If there's anything else I can add to this answer, please feel free to ask. I'm a bit rusty on the topic, haven't read much about it for the past year, but it is alway good for the memory to revisit stuff :)


Here's a hint to get you started: Parallel lines include the line through the camera.

So really all you need is the direction from the camera to the vanishing point on the view plane. Then create lines parallel to that line.


Is it possible to bind dynamic texture samplers to a shader in DX12?

Yes it is possible by putting Samplers in descriptor tables to dynamically assign and index into them. Source: https://youtu.be/Wbnw87tYqVg?t=754

Here's an example that uses the HLSL root signature syntax to do this here:
https://youtu.be/Wbnw87tYqVg?t=5277


What kind of anti-aliasing is used in font rasterization?

Sean Barrett, the author of the stb libraries, wrote an article on how stb_truetype's rasterization works. It's quite readable.

The article describes two methods of antialiasing, both based on scanline rasterization. In scanline rasterization, you iterate through the lines (rows) of pixels and keep track of which edges from the glyph outline are intersecting the scanline (considered as a mathematical zero-thickness line); then you can identify which pixel-centers are inside the outline by tracking enter/exit intersection points along that line.

You can do horizontal antialiasing by calculating partial pixel coverage, using the position of the intersection points relative to the pixel grid. For example, if you have an entering point at $x = 4.6$ on the current scanline, then pixel index 4 would be set to 60% empty and 40% covered. This sounds somewhat like what you tried to do. However, this doesn't help with vertical antialiasing since the scanline is still considered as a mathematical line with zero thickness. So, the article recommends to just supersample along the vertical axis: that is, process multiple "scanlines" spaced out vertically within each pixel row, and average the results together.

The second method performs both horizontal and vertical antialiasing at once, by evaluating the area of intersection of the glyph outline with each pixel square. It's still based on scanline rasterization, but now treats the scanline as a 1-pixel-tall rectangle and tracks all edges that touch the rectangle. To evaluate pixel coverage, the trapezoidal rule is used to calculate a signed area for each glyph edge—positive or negative depending on the edge's orientation—and these are summed up to give the total coverage area within the pixel square. (There are a lot more implementation details covered in the article, plus you can read the source.) This is somewhat faster and more accurate than the supersampling method, though more difficult to implement.

Finally, it's important to consider whether you want gamma correction when rendering text (as with rendering anything else). Theoretically, the antialiased coverage values shouldn't be used directly as pixel values; they should be treated as an alpha, and used for linear-color-space blending between the background color and text color, followed by conversion to output gamma. However, in practice almost no one seems to do this; we're accustomed to seeing non-gamma-correct text, and screen fonts are designed to look good rendered this way. (Print fonts are another story.) On the other hand, if the text is animated, or rotated or not aligned with the pixel grid for any reason, gamma correction will help it look more consistent regardless of its placement relative to the pixel grid.

So, the choice of whether to apply gamma correction comes down to personal preference and the needs of the application. Either way, you may see artifacts such as the font appearing to get heavier or lighter depending on color and font size.


Fundamentally, how are 2D bitmaps rendered?

You have to distinguish the text and graphical modes of the graphics board of your machine.

In the old days, mostly the text mode was supported. In this mode the board took care of storing the bitmap definition of the characters and displaying them at the current cursor position. All you had to do was to provide the ASCII code of the character (one byte per character) in a small text buffer.

Nowadays, a high-resolution raster buffer is provided, which is pixel accessible and to which you write color information in some supported format (in the "highest" mode, 3 bytes (RGB) per pixel, for a megapixel or more).

Originally, simple (packed) binary bitmaps of different sizes were used and "blittted" to the raster memory via a request to the device driver, with possible format translation.

Nowadays, characters are mostly defined as vector drawings, which are a resolution-independent description of the outlines, and need to undergo a complex rendering process that includes antialiasing for smooth results.

The rendered output can be cached for fast display, again by blitting.

The overall process is complex, can be hardware accelerated, and is handled by the operating system (GUI management) in a transparent way, together with other graphical primitive drawing operations.


What is the basis of rendering light caustics?

For offline rendering, caustics and dispersion effects (different colors of light separating, as in a prism) are usually calculated using some form of photon mapping. That's a fairly expensive technique, even by the standards of offline rendering, and it falls more under the "physical simulation" side of things.

If you're looking for more approximate techniques suitable for real-time rendering, Chris Wyman's papers are a good place to look. He wrote several papers on caustics between 2006 and 2009. The one to start with is Interactive Image-Space Techniques for Approximating Caustics (I3D 2006), #25 on that page. This paper re-uses a GPU refraction rendering technique he developed earlier, #28 on that page, so you'll want to look at that as well for background. Later papers build on these two.

The basic idea of these papers is to use GPU rasterization to perform a limited, approximate analogue of photon mapping. The scene is rendered from the light's point of view, similar to a shadow map, and each pixel is treated as a photon; its refraction through transparent objects is calculated approximately (usually considering only 1 or 2 layers of refraction), and its final location is stored in a texture. Then, one can render each photon as a small splat into a "caustic map", which finally gets projected onto the scene, similar to a shadow map.


In Normal Mapping, you want to go to "Tangent Space", that is the space where the center of the coordinate system is your Vertex (/Fragment). Just like you go from Model Space to World Space and from World Space to Eye Space, you can go to Tangent Space with a matrix multiplication.

In Tangent Space, the vector $n_T=\left(\begin{array}{c}0\\0\\1\end{array}\right)$ is always the Normal - since you read the normal $n_N$ out of the Normal Map in your Fragment Shader, you will obviously not use $n_T$ but $n_N$ for lighting calculations and this choice of $n_N$ rather than $n_T$ brings about the nice effect normal mapping has.

What's a bit special is that every vertex has its own Tangent Space, which is why you need the tangent, bitangent$^1$ and (geometry) normal of every vertex to construct your TBN matrix.

So much for theory - since you didn't really ask about that specifically, I assume you understand this (otherwise just edit your question).

  by the way, I'm not sure I visualize why the LearnOpenGL tutorial doesn't use vec4(modelSpaceVec, 1); I'm not sure what either value would represent or how it would change the results. I assume this just a property of normals that we always represent them with a w of 0, but I don't get why it applies to the tangent and bitangent

Vectors in general are represented with a $0$ in the $w$ coordinate. If you look into a translation matrix, you will notice that the last row is the one that moves points around. For vectors, this is not a wanted behaviour - a vector is the same, no matter where, since it is a direction. So to avoid "translating" a vector, you set the $w$ coordinate to $0$. Since your tangent and bitangent vectors are vectors, the $w$ coordinate is $0$ here as well.

  But more importantly, I don't understand the modelMatrix * ... part of the equation.

Using the the model matrix $M$ at this point is correct in most cases - but not all of them. The normal matrix $N$ is (as the answer you linked correctly said) the inverse transpose of the model matrix. But, since normals are vectors, you don't need a 4x4 matrix, a 3x3 matrix is just fine. And if you leave out that translation part, and you scale the same amount in all three coordinates, then the transpose of your model matrix is the same as the inverse of your model matrix. Therefore you get:

$
N = (M^{-1})^T = (M^T)^T = M
$

And that is why in most cases (i.e. everytime you use the same scaling in every coordinate) the normal matrix is the same as the model matrix.

  vec3 worldSpaceVec = inverse(transpose(modelMatrix)) * modelSpaceVec;

This is correct, however

  vec3 worldSpaceVec = inverse(transpose(mvpMatrix)) * modelSpaceVec;

this is not correct. Once you have transformed your normals into world space, you need to use the same view and projection matrix as before. Your view matrix brings your viewer into the origin (or rather it moves everything such that your viewer is the origin), inverse transposing this doesn't make sense (and similarly it doesn't make sense for the projection matrix. plus, the order in matrix multiplications is important, which is the next problem... don't do it ;-) )

Now, since you know that the model matrix may be the same as the normal matrix, you can (mathematically correctly) just go with the inverse transpose of the model matrix. Obviously this costs quite a lot of performance, especially inverting a matrix is costly. It is therefore more efficient to just go with the model matrix.

  In other words, should the tangent and bitangent vectors of a vertex be converted from one space to another using the same transformation matrix as the vertex's normal vector, and if so, why?

Unlike normals, tangent and bitangent vectors should be transformed with the model matrix, independently of the whether the normal matrix equals the modal matrix. A normal is always supposed to be perpendicular to a surface, and that's why simply taking the model matrix may be incorrect - in some cases the normal matrix would not be perpendicular to the surface anymore. Tangents however are always "lying" on the surface, so if you rotate a surface, the tangent must be rotated the same way to still "lie" on the surface.

$^1$ Sometimes this is called binormal, however Eric Lengyel explains why this is an incorrect term in Mathematics for 3D Game Programming and Computer Graphics.


How can I implement GIMP's "Colorize" function?

It appears to me that in the image, the saturation of the original image is 0 since it is grayscale. (Unless I've misunderstood the question.) When the saturation is 0, the hue is undefined, but it appears that the GIMP is setting the hue to 0. It is then using the sliders from the dialog to modify the HSL. It adds 180° to the hue (so from 0 to 180 which gets you from red to cyan). The saturation is increased by 75 units, and the lightness is kept the same. It may have special protections for black and white, given that the white part of the letter "A" remains white in the final image (although even there, it appears to have an RGB value of (0.99, 1.0, 1.0), whereas on the right, it's fully white (1.0, 1.0, 1.0)).

You can always look at the source code since it's an open source package.


There shouldn't be any performance penalty inherent in using a 3D group over a 2D one, the dimensions are just an aid for programmers to map invocations on to memory locations (since you can read or write wherever you like).

The group size affects which invocations can access the same block of groupshared memory, so it forces all warps from the same group to run on the same SMP. If you have a shader that needs to move stuff around a lot in its local memory block (e.g. a bitonic sorter), a large group size is preferable, but in your case it shouldn't make much difference.

I suspect your problem is with the idle threads within your warps. If I understand your question properly, you're using a thread per input texel or sample to do the texture reads, but then only a subset of these threads to do the processing? If so, you're sacrificing a lot of compute power within each warp. Remember, texture reads are done in minimum 2x2 blocks, and the results get cached in a way optimized for locality for the dimension of the texture (see this question). So although your general purpose shader might seem slow with all the texture reads, a lot of them will be effectively almost free. Also, the GPU does latency-hiding on slow memory reads, swapping in other warps until the result of the read is available. So your general purpose shader actually seems like the smart option here, as it maximizes compute power.


Vertices, Vertex Normals and Face Normals

I'm not familiar with the exact nuances of the STL format, but here are some answers on general computer graphics principles:

In the source file, the Vertex Normals are specified and in the destination, the face normal of the triangles are present which are different than the normals one would get by (a-b) x (a-c), where a, b, c are the vertex coordinates and x is the cross product. How one would compute the face normal in the presence of vertex normals?

There is no general method to convert vertex normals to face normals. Vertex normals (that are not all equal) can express the normals of a curved surface, which face normals cannot. Thus, converting to a format with only a normal vector per face, rather than per triangle vertex, is a lossy process — some information was discarded. The converter must make a compromise.
That said, from some quick research, it seems that the intent of the STL file format is that the normal is the normal perpendicular to the triangle (that can be computed as you specified), not a chosen vector, even though that makes it redundant. So perhaps your converter is wrong (though I can't say much as you didn't say how it was different).

Also, I am given to understand that STL format has a positive octant rule which requires all vertex coordinates to be positive. When the OBJ file has negative vertex coordinates, what should be done in the conversion process?

The only option is to choose some offset by which to translate all of the vertices so that they are positive. This also loses some information — what the origin of the original model was. That's easier to recover if needed, though, since it's just a 3D vector that you could translate back by when loading the model.


Compute shaders: one-time-only versus persistent buffers

Yes, it's totally possible, and in fact it's preferable to keep buffers in memory if you will be accessing them over the life of your program, rather than deallocating and reallocating them again.

I can't speak for Unity's way of allocating persistent buffers - I work with Unity and do a lot of shader writing in Cg, but most of our stuff is on mobile so compute shaders aren't an option. However, from a recent project I did using OpenGL, a buffer will generally hang around until you tell it not to, and even then, it is not in any way guaranteed to release the memory in a timely manner.

Allocating a buffer in modern OpenGL is done by calling glCreateBuffers and passing it a pointer to a GLuint array which will store your buffer 'names', then calling glBufferData or glBufferStorage to actually allocate and populate the VRAM for that buffer. Calling glBufferData a second time on the same buffer name will allocate new storage for that buffer, and the original storage will be 'orphaned'. It's kind of like reallocating an object in Java, the variable name will no longer point to that object, but other parts of the system may still have references to it (e.g. if it's still being used for a texture read somewhere), so it will not be eligible for cleanup until all references to it are gone.

Calling glDeleteBuffers with the buffer name has a similar effect but without allocating new storage. In both cases, the storage will not be deleted until absolutely all references to it within the system are gone. Even then, it is more like Java's garbage collection than C++'s delete, the system may well hold on to the memory for quite some time in order to reuse it for any future allocations you may make.

The upshot of all this is it's often better to allocate buffers once when you launch your app and cycle different data through them as needed. In OpenGL, a buffer will persist for the lifetime of the application unless you explicitly delete or reallocate it. Unity's rendering system has its own quirks as I'm sure you're aware, but in principle it should behave similarly.


Rigorous and simple explanation for satisfying boundary conditions in Stam and Bridson papers

First of all, I would like to state that handling the boundary conditions correctly is challenging, both from mathematical perspective and from an implementation perspective. Additionally, I think most of the papers don't explain the handling of boundary conditions sufficiently.

Furthermore, it is worth noticing that you have to apply boundary conditions to both the velocity field and the pressure field (and they are linked to each other). Additionally, you have to distinguish between is the pressure solve (or projection step) and the diffusion step, which must be treated separately.

In the very beginning you need to decide what kind of physical behavior you want to have at the boundaries. The standard example is a square surrounded by solid walls where no fluid can enter or leave. In this case you constrain the normal component of the velocity field to be zero ( a dirichlet boundary condition for the diffusion step). As we don't want changes for the normal component of the velocity in the projection step,  we also need to constrain the pressure. If you check the pressure update routine (which makes the velocity field divergency free) you can see that the derivative of the pressure field in direction of the normal must be zero, I.e. a Neumann boundary condition for the pressure field.

Let's assume you are talking about the boundary conditions in the pressure solve/ projection step. Stam states that for pressure there is a Neumann boundary condition. As written above this condition implies that the velocity won't be modified at the last stage of the projection step (the pressure update). In order to achieve the dirichlet boundaries for the velocity as bridson states it, you require to have homogenous Neumann conditions. So for pressure, both authors have equivalent boundary conditions.

With these considerations you need to set up your system of linear equations. For this it is worth to note the equations for some cells (I.e. one row of the system of linear equations) and check how the matrix and the right hand side must be modified to account for the boundary conditions. In a 2d scenario we have a 5 point Laplace stencil that results in a 4 on the diagonal and -1 for each of the adjacent cells (times some constants). In cells that are adjacent to the boundary there is at least one value that is outside of the simulation domain and care has to be taken what we do with this value. For homogenous dirichlet conditions this value is simply zero and no changes are required for the matrix and the right hand side. For Neumann boundaries the constraint is that there is no change across the boundary,  I.e. the value should be identical to the one of the cell. This reduces the value on the diagonal of the matrix by -1, I.e. 3 for a cell with one Neumann boundary. These considerations result in the linear system that you have to solve.

The set bnd method (that doesn't appear in the paper by the way) is a trick to make the computations without the necessity to modify the matrix. This trick only works with gauss seidel or Jacobi iterations (As commented on your other question). The set bnd method is part of the code that stam published. Personally I find it difficult to read and understand because of the variable names and the "b" that changes the behavior significantly and that seems to confuse you as well (see your other question).

Edit:
At first glance it looks like stam and bridson apply different boundary conditions, but they are actually the same. If we assume a solid body that is not moving and the velocity is zero before the pressure correction then the difference between them is zero. The term in parentheses in (4.23) will then be evaluated to -p_{i,j} which is exactly a  homogeneous Neumann boundary condition for the pressure. I've got the second edition of bridsons book,  where  this is explained in equation 5.3 and at the bottom of the page in more detail.

Edit2: up to know we discussed the Neumann boundary conditions for the pressure. No slip/slip boundary conditions are applied to the velocity and for both (with non-moving walls) you need homogenized Neumann boundary conditions for the pressure. With stam's approach of discretization, I.e. storing all velocity components at the cell center it is easier to realize no slip boundary conditions as the boundary for all velocity components are at cell faces. So to realize no slip boundary conditions in the diffusion step (mathematically you can't realize no slip conditions with the Euler equations ( no diffusion)) you need to use homogeneous dirichlet boundary conditions for all velocity components.

Edit3: It is not straight forward to use the set_bnd method on a staggered grid for no slip boundary conditions. The storage for the tangential (to the boundary) components is not ideal.

Edit 4: A Neumann boundary condition for the pressure implies that there are no changes for the (normal) velocity at this boundary. If you subtract the pressure gradient at such a boundary, the pressure gradient is simply zero due to the boundary condition.

Edit 5: Slip / no slip boundary conditions (for the velocity) are not(!) realized in the pressure correction step. For non-moving solid walls there is always a homogeneous Neumann boundary condition for the pressure no matter if you impose a slip or no-slip boundary condition.

Edit 6: I don't "have" facebook ;)

There are more subtleties like singular systems for Neumann- only boundary conditions, central differences in staggered or standard grids, distance to the boundary (at cell border or at cell center outside three domain) etc., that I could address if you detail your question.


Determining the 'outer' subpath in a compound SVG path

If you can assume the subpaths are non-intersecting (and non-self-intersecting), and that the fill rule is always the nonzero rule, the easiest way is probably to pick the first point on the first subpath, and do a standard point-in-polygon test with the polygons defined by each other subpath. If the point is not inside any other subpath, then this subpath is the outermost one. If the point is inside one other subpath, that one must be the outermost one. If the point is inside more than one other subpath, take the set of subpaths it is inside and repeat the test.

Obviously doing it this way requires that there is only one outermost subpath. It's entirely possible in SVG to have multiple disjoint subpaths in one path, and you didn't say whether this is possible in your source data. If it is possible, then you'll have to test every subpath to find out which other subpath(s) it is inside. Each time you run the point-in-polygon test, you find the subpath you're considering is either an outermost one, or you'll find out which subpath it is inside. If it's inside another path, you don't need to consider it in any further tests. At the end, you have a list of outermost subpaths, each of which has a list of all the subpaths it encloses. You can turn each outermost subpath into a separate GeoJSON polygon.


Assuming you mean a camera that rotates based on mouse movement:

One way to implement it is to keep track of the camera position and its rotation in space. Spherical coordinates happen to be convenient for this, since you can represent the angles directly.

float m_theta;
float m_phi;
float m_radius;

float3 m_target;

The camera is located at P which is defined by m_theta, m_phi, and m_radius. We can rotate and move freely wherever we want by changing those three values. However, we always look at, and rotate around, m_target. m_target is the local origin of the sphere. However, we are free to move this origin wherever we want in world space.

There are three main camera functions:

void Rotate(float dTheta, float dPhi);
void Zoom(float distance);
void Pan(float dx, float dy);

In their simplest forms, Rotate() and Zoom() are trivial. The just modify m_theta, m_phi, and m_radius respectively:

void Camera::Rotate(float dTheta, float dPhi) {
    m_theta += dTheta;
    m_phi += dPhi;
}

void Camera::Zoom(float distance) {
    m_radius -= distance;
}

Panning is a bit more complicated. A camera pan is defined as moving the camera to the left/right and/or up/down respective to the current camera view. The easiest way we can accomplish this is to convert our current camera view from spherical coordinates to cartesian coordinates. This will give us an up and right vectors.

void Camera::Pan(float dx, float dy) {
    float3 look = normalize(ToCartesian());
    float3 worldUp = float3(0.0f, 1.0f, 0.0f, 0.0f);

    float3 right = cross(look, worldUp);
    float3 up = cross(look, right);

    m_target = m_target + (right * dx) + (up * dy);
}

inline float3 ToCartesian() {
    float x = m_radius * sinf(m_phi) * sinf(m_theta);
    float y = m_radius * cosf(m_phi);
    float z = m_radius * sinf(m_phi) * cosf(m_theta);
    float w = 1.0f;

    return float3(x, y, z, w);
}

So, first, we convert our spherical coordinate system to cartesian to get our look vector. Next, we do the vector cross product with the world up vector, in order to get a right vector. This is a vector that points directly right of the camera view. Lastly, we do another vector cross product to get the camera up vector.

To finish the pan, we move m_target along the up and right vectors.

One question you might be asking is: Why convert between cartesian and spherical all the time (you will also have to convert in order to create the View matrix).

Good question. I too had this question and tried to exclusively use cartesian. You end up with problems with rotations. Since floating point operations are not exactly precise, multiple rotations end up accumulating errors, which corresponded to the camera slowly, and unintentionally rolling.

So, in the end, I stuck with spherical coordinates. In order to counter the extra calculations, I ended up caching the view matrix, and only calculate it when the camera moves.

The last step is to use this Camera class. Just call the appropriate member function inside your app's MouseDown/Up/Scroll functions:

void MouseDown(WPARAM buttonState, int x, int y) {
    m_mouseLastPos.x = x;
    m_mouseLastPos.y = y;

    SetCapture(m_hwnd);
}

void MouseUp(WPARAM buttonState, int x, int y) {
    ReleaseCapture();
}

void MouseMove(WPARAM buttonState, int x, int y) {
    if ((buttonState & MK_LBUTTON) != 0) {
        if (GetKeyState(VK_MENU) & 0x8000) {
            // Calculate the new phi and theta based on mouse position relative to where the user clicked
            float dPhi = ((float)(m_mouseLastPos.y - y) / 300);
            float dTheta = ((float)(m_mouseLastPos.x - x) / 300);

            m_camera.Rotate(-dTheta, dPhi);
        }
    } else if ((buttonState & MK_MBUTTON) != 0) {
        if (GetKeyState(VK_MENU) & 0x8000) {
            float dx = ((float)(m_mouseLastPos.x - x));
            float dy = ((float)(m_mouseLastPos.y - y));

            m_camera.Pan(-dx * m_cameraPanFactor, dy * m_cameraPanFactor);
        }
    }

    m_mouseLastPos.x = x;
    m_mouseLastPos.y = y;
}

void MouseWheel(int zDelta) {
    // Make each wheel dedent correspond to a size based on the scene
    m_camera.Zoom((float)zDelta * m_cameraScrollFactor);
}

The m_camera*Factor variables are just scale factors that change how quickly your camera rotates/pans/scrolls

The code I have above is a simplified pseudo-code version of the camera system I made for a side project: camera.h and camera.cpp. The camera tries to imitate the Maya camera system. The code is free and open source, so feel free to use it in your own project.


why doesn't a ray hitting a sphere at only 1 point count as a hit in sphere/ray intersection?

It is mainly personal preference, but it does have some logic behind it which would make it a better option. It should not have any big impact on your final image, though.

When the discriminant is 0, then that means that there is only one solution. For a sphere, when there is only one solution, that means that the ray is tangent to the sphere. When a line or ray is tangent to a sphere, that means that there is a 90° angle between the line/ray and the normal of the sphere at the intersection. When comparing that situation to ray-triangle and ray-plane intersections, they always ignore the hit when it has an angle of 90° with the normal. This is because the ray would then not really be able to hit it, and in real life you cannot see an infinitely thin plane or triangle that is perfectly parallel to your viewing direction.

Discarding hits when the angle is 90° could also help with shading. Depending on the shader, a 90° angle might always return black or start giving funky results, while when the angle is less than 90° it is just perfectly fine. Also as @Rahul has said, if you would calculate a reflected ray, it would end up in the same direction as your original ray and with most shaders that reflection ray would contribute to 100% of the ray's colour and not be coloured, so it would give the exact same colour as if you just discarded the ray.

The thing that makes this basically personal preference, is the fact that it would practically not happen that a ray would hit the sphere perfectly tangent, in an average render engine (as Rahul has also said). When looking at a sphere from a certain point, there would be a single circle with an infinitely small thickness that could create the occurrence of the discriminant being 0.

In short, it basically does not matter a single bit if you discard those hits or not.


I'm guessing this has to do with providing a specific line/point width? Else they are too thin for intersection.

Exactly. True points and lines have no spatial extent: a point is infinitely small, a line is infinitely thin. You can't intersect those with a ray. But a point with a size is just a sphere, and a line with a size is a cylinder, and those are classic ray-casting primitives, so of course you can intersect them.

The only difficulty is if your path-tracer currently only has one geometry primitive (the polygon or polygon mesh): adding new primitives might be hard for you, if you've structured it for just one. But even in that case, instead of intersecting the points and lines directly, you can just turn them both into polygon meshes inside the renderer, before casting any rays. If you know how to polygonise a sphere and a cylinder you already know what to do.


Why are inverse transformations applied to rays rather than forward transformations to objects?

Most raytracers do both!

Have you ever seen a ray-triangle intersection test where the triangle is transformed so that one vertex is at the origin and the triangle is flat along one of the axes? That would simplify the test, but storing the inverse transformation to apply it to the ray takes more space than the triangle itself. So you could say that almost all ray-triangle tests do apply the forward transformation to the object.

And yet most of the time you are rendering a mesh of many triangles, and "most raytracers" don't transform every vertex in the mesh by the object's transform matrix, they inverse transform the ray into object space. There isn't necessarily a good reason for this—it can definitely be more performant in some cases to preprocess the mesh and put every vertex in world space.

But then say you're doing animation studio-scale path tracing. Now your scene geometry might not all fit in RAM at once. It's no longer a performance win to pre-transform all your vertices/control points, because (a) you need one copy of the object per instance, and (b) you need to transform it each time you read it into RAM. When data bound, the cost of two matrix multiplies per ray is insignificant anyway.

Floating point precision is a potential reason as well: when you have an object a long way from the origin, you get more error from floating point positions, and if you pre-transform the object you apply that error in different directions to each vertex, whereas if you inverse transform the ray you have one fixed error amount from the imprecise position of the ray, but your vertices have less error relative to each other.

Although I suspect the real answer to "why do most raytracers inverse transform the ray" is that most raytracers with public source code implement a wide range of geometric primitives, either for educational purposes or as a proof-of-concept, and it's easier to implement one cheap ray inverse transform than N shape forward transforms. (Plus, as has been mentioned, some intersection algorithms are much simpler if the object is in its canonical space.)


These aren't macros pointing to memory addresses. They're just integer preprocessor constants. Back in the days when OpenGL originated, using preprocessor constants for your numeric constants was common practice in C.

Nowadays it's considered better practice to define new types (single-element structs) for your constants, but it's too late to change OpenGL.


Are there specific techniques for generating False Radiosity in surfaces?

Gobal illumination, radiosity, etc is not a mutually exclusive set with using a fill light to adjust the feeling. Sometimes the artist can do wonders with a few well placed lights.

So yes they are used, each game and situation is different. Perhaps they are just going for a unrealistic mood or the artist wants to tweak the result a bit.


Retrieve real distance from depth buffer for shadow ray (vulkan hybrid rendering)

Thanks for your help, I fixed it by changing the way i am computing the depth :)

const double z_b = texture(depthBuffer, texcoord).x;
const vec4 clipSpacePosition = vec4(texcoord * 2.0 - 1.0, z_b, 1.0);
vec4 viewSpacePosition = cam.projInverse * clipSpacePosition;
viewSpacePosition /= viewSpacePosition.w;
float depth = length(viewSpacePosition.xyz);


Finding the maximum number of disconnected fragments

A convex polygon has the property:

A line drawn through a convex polygon will intersect the polygon exactly twice.

From this follows that any line trough splits the convex polygon in 2 pieces.

Image 1: convex polygon split by a line.

Using this, and assuming the inner piece is discarded we can craw the conclusion that there are five cases:

When the lines of the square do not exit the polygon. You now have a piece with a hole or the entire polygon is contained and theres no hole and no piece.
When the lines exit 2 times you have one fragment*
when the lines exit 4 times you have two fragments*
when the exits are 6 times you have three fragments*
when the exits are 8 times you have four fragments *

Image 2: the five cases, clipping rectangle and remaining shards in black. Even a squares has same cases if the polygons vary, just easier to draw this way.

The last case is the maximum so answer is four or five if you count the clipped part as one piece.

* Since we haven't tackled the cases very rigorously we know that this is the highest possible answer, there could be less for each case. There shouldn't be but I so far haven't given evidence for that. But thats okay it wasn't asked.


Because you are working on screen space, the view ray points towards the eye (being Z the camera space depth). In order to create an orthonormal base, the vectors must be perpendicular to each other. To do this, you set the coordinate z to 0 for the tangents and bitangets, which ensures these vectors will live in the plane perpendicular to the view ray.

Lastly, you need to make the tangent and bitangent perpendicular to each other, which is accomplished with this

vec3 tangent = normalize(rvec - normal * dot(rvec, normal));
vec3 bitangent = cross(normal, tangent);