How to apply wire texture for realistic rendering of embroidery?

Picking up from step 2, where you generated line segments to represent individual stitches, I would first suggest that you apply rounded ends to each segment, rather than square ends.

I would then convert each segment to an outline, (i.e. each rounded stitch becomes an separate poly-line representing the outer boundary.)

For example, suppose you start with point A & point B, representing a single stitch in line segment form. We can derive the rounded outline of this segment with the following Javascript code using THREE.js:

    function outlineSegment (Ax, Ay, Bx, By, radius){
        var radius = radius || 0.5; //millimeters?
        var A = new THREE.Vector2(Ax, Ay);
        var B = new THREE.Vector2(Bx, By);
        var C = B.clone().sub(A).normalize().multiplyScalar(radius);
        var angle = C.angle();
        var outline = new THREE.Shape();
        outline.moveTo( B.x + C.y, B.y - C.x );
        outline.absarc(B.x, B.y, radius, angle-Math.PI/2, angle+Math.PI/2);
        outline.lineTo(A.x - C.y, A.y + C.x);
        outline.absarc(A.x, A.y, radius, angle+Math.PI/2, angle+Math.PI*3/2);
        outline.autoClose = true;
        return new THREE.ShapeGeometry(outline);
    }

The above function finishes by triangulating the polyline into faces, (See the orange area in the included picture.)

Next, send these faces to be rendered by the GPU, applying a vertex shader which sets a vec2 varying variable (sounds redundant, but it's not) indicating the vertex position relative to the original line segment (which we have included as vertex attributes)

Now, we move on to the fragment shader, where we use the varying variable created in the vertex shader to derive a normal.

Let's say that "linePos" is the position along the line segment, (i.e. the x-value, if the line is horizontal. y-value if it is vertical, etc.)

Furthermore, let's say "lineDist" is the distance from the central line segment, e.g. -2 if it's 2 units to the left of the line (purple arrows) or +5 if it's on the right (blue arrows).

We can use these two values, derived from the varying variable, to create a normal for the point on the thread.

A bit of this code might be:

abs(sin(linePos+lineDist))

but there's a fair bit more than that to write.
I can't go through and write the entire program for you here & now, but hopefully these ideas & diagram will shine some light in the right direction.

Best of luck.


Do we ever consider the size of the pixel to calculate the color in Ray Tracing?

Yes, and there are several ways this can apply. First off though, we need to fix a misconception in your diagram. Even if pixels really were square, the volume contributing to the pixel gets wider as it gets further away from the image plane, because of perspective. For a square pixel, the volume would be a square pyramid, whose apex is at the camera. It's not quite a regular square pyramid, because the apex is not in the centre of the square: the outer pixels are very skewed. But pixels aren't square anyway, they're cone-shaped.

First off, some people have experimented with cone tracing, where instead of a single ray, you trace a cone with a half-angle determined from the area of the image covered. It's not very popular, because when the cone intersects the edge of a polygon, you need special handling for the remainder of the cone - the part that wasn't blocked by the polygon. For a square pyramid, it's a little easier, because you can change the shape of the base as parts of the pyramid are blocked, but it's still tricky and slow.

You also don't gain much from this technique compared to just supersampling. Even if you trace rays, you don't just trace a single ray through the centre of each pixel. You trace many rays across the area of the pixel, and use a reconstruction filter to take the shape of the pixel into account. This technique has many possibilities that cone tracing doesn't offer: it works better with volumes, you can jitter your samples to avoid structured artefacts, you can use full path-tracing techniques, etc.

Another area where the pixel size matters is in texture sampling. If you want to get nice texturing, you don't just take a point sample of the texture, you set some filter width according to the area of the texture the sample relates to. This doesn't just make your sampling more efficient by using the right mipmap level: it is required for correct sampling to avoid sampling artefacts. In a GPU fragment shader, you might use the dFdx and dFdy functions to do this. In a ray-tracer, you might have use a cone-tracing style of carrying the filter width with the ray, initially based on the level of supersampling at the camera, but updated whenever a child ray is spawned.


Is it possible to directly generate mipmap levels of perlin noise?

I would say yes with a small asterisk.

When generating a perlin noise texture, using multiple of octaves of noise like you are talking about, the point of adding higher octaves (higher frequency lower amplitude) is to add high frequency details to the noise.

When making mipmaps of a texture, the point is to remove high frequency content that would cause aliasing at the resolution of that mip map level.

So, when making a specific mip level of a multiple octave perlin noise texture, you could find out which frequency would cause aliasing, and just use all the octaves up to (but not including) that point.

The result would be that you would have reasonable mips.

But, there are some reasons for that small asterisk:

The result you get would not be the same as if you mipped the image that has the highest frequency content.  This might not necessarily be a bad thing, because the usual way to make mips causes some bluring (a blur is a low pass filter) so you wouldn't have the bluring that comes with mips.  However, it's possible that a higher octave happens to add some lower frequency content than you'd expect.  Your lower mips, which could handle this frequency, would have that data missing unfortunately.
It might be that your octaves don't perfectly line up with the frequencies that your various mip levels can handle.  For instance, if an octave is just barely higher than nyquist frequency, you are going to miss out on some of the frequency components that the octave gave you.

Because of these two points, it might be more correct to process all octaves for each mip level, but to low pass filter the data for the octaves that are above nyquist.

That would be a lot more expensive than just omiting octaves at a cutoff point though (:


Earth curvature and horizon in modeling large scenes

Having the horizon fall off is simply dropping the ground in the distance down somewhat.

A point $x$ km away if you follow the curve of the planet will be $r-r\cdot \cos \frac{x}{r}$ down and $r\cdot \sin \frac{x}{r}$ out horizontally where $r$ is the radius of the planet you are modeling (~6.3k km for earth). In the vertex shader you can account for that before you apply the view and projection matrix.


Zoom in orthographic vs perspective projection

Perspective projection changes the size of an object as it's distance changes, while orthographic projection does not.  That is part of the definition of those projection types.

To simplify things a bit, a simple perspective projection of a 3d point to a 2d point can be calculated like this:

$x_{2d} = x/z\\
y_{2d} = y/z$

As you can see, the 2d version of the x and y change depending on the value of z.

Orthographic projection on the other hand can be calculated like this:

$x_{2d} = x\\
y_{2d} = y$

In that example, you can see that the x and y values do not depend at all on z, so you can move closer to or farther away from an object in orthographic projection and it's size will not change.

As a fun bit of related trivia, the sun is so far away that when the light gets here, it's all basically going in the same direction.

This is a reason why we have directional lighting, where light beams are always moving in the same way, because that is how sun light works.

Anyways, since the light beams are parallel it means that your shadows in sun light are orthographic projections of your silhouette.  In other words, as you move your hand closer to or farther from the sun here on earth, the shadow of your hand doesn't change size.

However, if you have a more local light source - light a lamp - it casts a perspective projection type of shadow, where the closer to the light source your hand is, the larger the shadow it casts.

This is tied to the definition of an orthographic projection, which is that the focal length is infinitely long.  The sun is far enough away that we can consider it infinity in lighting, for as much accuracy as we care about :P


How can i make my ray tracing algorithm fast in terms of intersection testing?

You have several techniques.

Use acceleration structures. The idea is that you store your geometry (triangles) into generally some kind of volumes. The general idea behind acceleration structure is to discard quickly large parts of the scene, which we know (using these structure) don't have any object that your ray will intersect. An introduction to the topic can be found here.
You can also take advantage of multi-threading and parallelism (threads and SSE instructions or similar - this is harder than 1. and the benefits are not always as high but you optimise for your hardware).
You can ray-trace in the GPU but ray-tracing is a memory-bound problem so you will have memory problems quicker with the GPU than with CPU.

These are just some ideas.


Rendering a particle system as a smooth blob

For culling faraway particles: as lightxbulb said, the keyword to search for is "acceleration structure". There are a variety of options: BVH, octree, kd-tree, uniform grid, or some combination of these. Which one will be most suitable is going to depend on the details of your situation—how large/dense is the simulation, how much you care about performance vs memory vs implementation complexity, and so on.
They all work by subdividing space into volumes in some fashion and keeping track of which particles are in each volume. Then, when you're marching you only need to keep track of which volume you're in, and that will tell you which particles can potentially contribute to the density at that point. For instance if it's a uniform grid, you only need to look at the current cell and possibly some of its immediate neighbors if they're within the maximum particle radius of your current point.
Getting a smoothed mesh with surface normals etc. is usually done by defining a density field for the particle system. This is a function on 3D space that consists of summing up a falloff function for each particle over a defined radius:
$$
D(\vec r) = \sum_{i \in \text{particles}} f \left(\frac{|\vec r - \vec r_i|}{R_i} \right)
$$
where $f(u)$ is some function that falls off smoothly from $f(0) = 1$ to $f(1) = 0$ (for example, a simple cubic falloff: $f(u) = 1 - 3u^2 + 2u^3$), and $\vec r_i, R_i$ are the position and radius of the $i$th particle. Then you set some threshold density value and let the surface be defined by all the points where $D(\vec r) = \text{threshold}$.
For getting the surface normal: the surface normal is just the (normalized) gradient vector of the density field. This can be calculated by finite differences between nearby points in the field:
$$
\begin{aligned}
\nabla D(\vec r) &= \left( \frac{\partial D}{\partial x}, \frac{\partial D}{\partial y}, \frac{\partial D}{\partial z} \right) \\
&\approx \frac{1}{\epsilon} \left( D(\vec r + \epsilon \vec x) - D(\vec r), D(\vec r + \epsilon \vec y) - D(\vec r), D(\vec r + \epsilon \vec z) - D(\vec r) \right)
\end{aligned}
$$
or analytically by adding up gradient contributions from each particle.
$$
\begin{aligned}
\nabla D(\vec r) &= \sum_i \nabla \left( f \left(\frac{|\vec r - \vec r_i|}{R_i} \right) \right) \\
&= \sum_i \text{normalize}(\vec r - \vec r_i) \frac{1}{R_i} \frac{df}{du}
\end{aligned}
$$
For getting the surface in the form of a mesh: the keyword to search for is "isosurface extraction". Marching cubes is one example of such an algorithm, as lightxbulb mentioned. It evaluates the density field at evenly spaced grid points, uses those values to estimate where the intersection of the surface with the cubes' edges will be, and constructs from that a tiny mesh within each cube. Stitching all those meshes together yields a representation of the full surface. Naturally, the accuracy of it depends on the size of the grid used.
There is also a variant that uses tetahedra instead of cubes, and another similar algorithm called "dual contouring". Here's a nice article showing how these work interactively (in 2D). There's also a great deal of literature on how to improve accuracy, performance, etc. with these kinds of methods.


This is not really an answer to the latter parts of your question but more to address

As we know, tiles can make it quicker to update large portions of the display at once, as compared with a bitmap. To put, say a letter, on the screen, the Commodore 64 typically does two writes (the screencode and the attribute)

What you are effectively describing is a compressed framebuffer where you are directly writing/storing compressed data and letting the display pipeline decompress on the fly.
I would say that such things "still exist" in today's games/hardware, but not perhaps in form you are expecting.

Compressed frame buffers do still exist. Mobile devices certainly do (for example Imagination/PowerVR have PVRIC, and I believe ARM devices have their own scheme)
These compression schemes are generally "transparent" to the programmer, but it does mean the GPU and display pipeline does less reading and writing of external memory.

A display decompression system like PCG  (Programmable Character Generator) is effectively a precursor to (and effectively a subset of) some texture compression schemes, e.g. such as S3TC/DXT1. In that sense, therefore it still exists, but users don't typically write directly to them, but certainly take advantage of the bandwidth savings.


Canonical Space to Screen Space: Why do we have the strange -0.5?

I can't really answer this without the context so post some links where this is done.

What I can tell you is that the general formula for viewport transformation as done by OpenGL is given as (taken from wiki)

Here x,y are the what your picture says xmin, ymin and width, height is xmax, ymax. Now there is no specific reason I can think of why you would wanna do a shift of -0.5 but the viewport allows us to draw to a specific portion of the screen.

For example, in video games you would want to draw the minimap on the lower left corner of the screen, hence you could provide the values

x = 0, y = 0, width = 200, height = 200

This would render a 200x200 minimap in the lower left corner. Your second picture does this but instead it chooses a value of

x = -0.5, y = -0.5, width = nx, height = ny

If you substitute it in the above matrix you'll get

$ \frac{nx}{2} * x_{ndc} - \frac{1}{2} + \frac{nx}{2}$

for the $X$ value. This equals

$ \frac{nx}{2} * x_{ndc} + \frac{nx - 1}{2}$

Which is the same as what is done in the picture you posted. Hope it helped somewhat.

EDIT:- 0.5 offset is commonly used to get to the pixel center. See this image as an example.

Here the pixel 3,5 has the center 3.5, 5.5. This is the 0.5 offset you are seeing. I was confused at first when you were talking about an offset of negative 0.5.


Exponential shadow maps sampling with PCF for sampler2DShadow instead of sampler2D

PCF does N shadow tests, where each test returns 1 (in light) or 0 (in shadow), and returns the average of the results. sampler2DShadow is a special sampler where PCF is applied to the nearest texels to a sample, based on the depth coordinate provided. So you can't use it with ESM.


Ray vs AABB algorithm that also gives which side was hit?

Given an intersection algorithm like the one you linked which first generates a $t$ value for every plane of the AABB, then computes $t_{min}$ and $t_{max}$ from those, you simply need to figure out which of the six initial values correspond to your final $t_{min}$. There will be two matches if your ray hits an edge, three if it hits a corner. In that case, this may include a back face (see the fix below).

This will not require a tolerance as $t_{min}$ is always one of the six values. In fact, it is always one of three intermediate values and you can use that to make your check simpler, faster and better. Given code, from the link above, such as:

float tmin = max(max(min(t1, t2), min(t3, t4)), min(t5, t6));

you could rewrite as:

float tminx = min(t1, t2);
float tminy = min(t3, t4);
float tmin = max(max(tminx, tminy), min(t5, t6));
int axis = 2;
if (tmin == tminx) axis = 0;
if (tmin == tminy) axis = 1;

As an added bonus, unlike testing all six values, this will not match the back face in the event that a far edge is hit.


You probably want to set sigma to be a fraction of the radius, rather than a fixed value. Sigma controls the actual shape of the filter, while radius just controls how far out the filter gets cut off (as Gaussians truly have infinite radius, so we have to cut them off somewhere to use them in practice). With sigma set to 0.5, you're making a filter that's very sharply peaked within a radius of 0.5 pixels, which is consistent with the image you're getting.
Instead, try setting sigma = 0.25 or 0.3 times the radius. Or alternatively, make sigma your main parameter and then set the radius to 3 or 4 times sigma.


How do you scale an image across planes while retaining perfect tiling?

One solution would be to use repeating texture wrapping, and then use a texture transformation matrix. So let's say you normally set the texture coordinates of your GL_TEXTURE_2D to be 0-1 in u and v. Instead, ensure that your GL_TEXTURE_WRAP_S and GL_TEXTURE_WRAP_T are set to GL_REPEAT or GL_MIRRORED_REPEAT and set the texture coordinates to be the tile number to tile number + 1. So the first one would still be 0 to 1. The second one would be 1 to 2, the third would be 2 to 3, etc.

Next, your vertex shader would have a mat4 uniform called textureMatrix or something like that. The texture matrix you pass in could have scaling in it (and probably other transformations, too). You just multiply the texture coordinates for the polygons by the texture matrix in the vertex shader, and it should apply appropriately.


Why do polygons have to be "simple" and "convex"?

Polygon rasterization (the conversion of the analog polygon data into a raster image) is a key operation in rasterization-based rendering. As such, performing this operation fast, and with predictable performance, is key to the performance of graphics rendering.

It is possible to rasterize any arbitrary polygon. But rasterizing triangles is extremely simple to code and offers numerous efficiency improvements over general polygon rasterization.

Triangle rasterization is done by something called "scan conversion": you break the triangle up into a series of horizontal scan-lines (one pixel tall) of a length determined by the sides of the triangle. Building these scan-lines and getting their lengths is something you can do through basic integer math involving only additions and multiplications. Furthermore, each scan-line can be constructed independently of the others, so highly threaded hardware can be employed to perform this process efficiently.

None of this is true for arbitrary polygons. Concave polygons in particular require much more complex math. A concave polygon can have multiple lines in the same horizontal row.

Furthermore, interpolation of values across the surface of a triangle can be done by relatively simple math. For each scanline, you can compute a gradient that represents how much each value changes from one end of the line to the other. So given a starting value at one line, you can easily compute the interpolated position at any point on the scanline simply by multiplication and addition. If you're computing them in sequence, you don't even need the multiplication; successive addition is adequate.

Interpolation of values across an arbitrary polygonal surface is much more complicated.

Given all of these advantages, and the fact that you can use a series of triangles to build any arbitrary polygon, there's really no reason to rasterize any more complex shape.


Detect collision for non axis-aligned bounding boxes

Write two functions: (1) point in 3D box, and (2) segment intersect rectangle in 3D. Point in box is a conjunction of six left-of-plane checks. For (2), solve for the point of intersection between the line containing the segment and the plane containing the rectangle. Then check if that point is on the segment, and if so, in the rectangle.
Then box $B_1$ intersects box $B_2$ if any corner of $B_1$ is inside $B_2$, or if any of the $12$ edges of $B_1$ intersects any of the $6$ faces of $B_2$.
This can be sped up a bit by ordering the tests (for example, if no edge of $B_1$ intersects a face of $B_2$, then you need only check one corner of $B_1$ inside/outside $B_2$).


I think you may be overcomplicating things with your idea of using the stencil. In 2D graphics layers, we usually do not need to use the depth buffer to resolve depth order between 2D elements (their depth order is determined by draw order instead). Therefore, we can reserve the nearest depth value for all the foreground elements, and the farthest depth value for all background elements.
This would work as follows, assuming depth 0.0 is the far plane and depth 1.0 is the near plane.

Render opaque foreground elements in front-to-back order, using depth test VK_COMPARE_OP_GREATER, and arranging in the vertex shader for all elements to be placed at depth 1.0.
Render opaque scene objects roughly front-to-back.
Render opaque background elements in front-to-back order, placing them at depth 0.0.
Render transparent background elements back-to-front, now using depth test VK_COMPARE_OP_GREATER_OR_EQUAL, disabling depth writes, and placing them at depth 0.0.
Render transparent scene objects back-to-front.
Render transparent foreground elements back-to-front, placing them at depth 1.0.

The switch from GREATER to GREATER_OR_EQUAL is important here. Since the opaque elements are going front-to-back, you want to exclude the "equal" case, since you want later elements to be occluded by earlier elements. Then, when coming back-to-front for the transparents, you want to include the "equal" so that later elements will correctly render on top of earlier ones in the foreground/background layers.
Edit: Now that I'm thinking about this more, this may not work since transparent elements in foreground/background won't be able to be occluded by opaque elements in the same layer. This could be addressed by pre-clipping the transparent element geometry, or if this is important to support on the GPU, it could be addressed by rendering opaque elements depth-only (no fragment shader) in the first passes and then rendering all elements, both opaque and transparent, back-to-front in the later passes.


Is this the correct way to implement Beer's Law?

Your image definitely does not look correct, and it appears that you are not correctly computing the internal path of light rays as they travel through your mesh. From the looks of it, I would say that you are computing the distance between the point where the view ray first enters the cube and where it first hits the interior wall, and using that as your absorption distance. This essentially assumes that light will always exit the glass the first time it hits a wall, which is a poor assumption.

In reality, When light enters glass from air, it often does not immediately exit the glass. This is because when the light strikes the glass/air interface, a phenomenon known as total internal reflection (TIR) can occur. TIR occurs when light travels from a medium with a higher index of refraction (IOR) to one with a lower IOR, which is precisely what happens in the case of light hitting the interior wall of a glass object. This image from Wikipedia is a good visual demonstration of what it looks like when it occurs:

In basic terms, what it means is that if the light hits at a shallow angle, the light will completely reflect off of the interior of the medium. To account for this, you need to evaluate the Fresnel equations every time that your light ray hits a glass/air interface (AKA the interior surface of your mesh). The Fresnel equations will tell you the ratio of reflected light to the amount of refracted light, while will be 1 in the case of TIR. You can then compute the appropriate reflected and refracted light directions, and continue to trace the light's path either through the medium or outside of it. If you assume a simple convex mesh with a uniform scattering coefficient, then the the distance to use for Beer's law will be the sum of all internal path lengths before exiting the medium. Here is what a cube looks like with your scattering coefficients and IOR of 1.526 (soda lime glass), rendered using my own path tracer that accounts for both internal and external reflections and refractions:

Ultimately the internal reflections and refractions are a major part of what makes glass look like glass. Simple approximations really don't cut it, as you've already found out. It gets even worse if you add multiple meshes and/or non-convex meshes, as you not not only have to account for internal reflections but must also account for rays that leave the medium and enter it at a different point.


Vector has a few different semantics from static arrays. For one it's a struct containing a pointer a capacity and a length (at the very least). That means that sizeof will not reflect how much data is actually stored in there (that only works on static arrays).

If you need to get the size in bytes of the data then you need

const UINT vertexBufferSize = sizeof(vertices::value_type) * vertices.size();


Finding a color perceptually similar to an image

You could try some form of color quantization algorithm, which generally extract the most dominant N colors. The one I've seen referenced most is modified median cut quantization [.pdf], which is based on median cut quantization [.doc]. The benefit to this kind of algorithm is that instead of simply averaging every color in the image, it extracts and discards other highly prominent colors instead of allowing them to pollute the average.

The concept is that color space (RGB space) is partitioned into 3D axis-aligned rectangular regions (the paper calls them vboxes) and iteratively subdivided by splitting vboxes, attempting to leave half of the pixels on each side of the split. The result is color clusters that should correspond to color clusters in the image. The largest color has a strong likelihood of being "perceptually similar" to the image.

There's a JavaScript implementation and demo of this algorithm called Color Thief.


Normally if you paint a background image you simply draw a textured full-screen square while depth testing and writing is disabled before drawing the actual scene (where you re-enable depth)

In the code you posted you draw the quad but then clear the screen again. Don't do that. Move the drawing of the quad from main to right after the glClear call.


How linear interpolation works between mipmaps?

It's called trilinear interpolation. You first do a bilinear interpolation of the higher-res texture, then do a bilinear interpolation on the lower-res texture, then interpolate between the 2 results. The weight of the final interpolation is based on where between the 2 textures your Z-coordinate falls. If 0 is fully the low-res texture and 1 is fully the high-res texture, then you can use the standard glsl mix() function to combine the two:

result = (highres texture color * weight) + (lowres texture color * (1.0 - weight));


Instead of using cubic Bezier spline, you should use cubic Hermite spline. For cubic Hermite curve you specify the segment end points and tangents, which is what you then control in the tool. Hermite spline is almost the same as Bezier spline but with different $B$ matrix, so if you already understand Bezier, then understanding Hermite is a trivial step.

So in the Adobe Illustrator pen tool (which based on the video I take is the same as Adobe Photoshop pen tool that I have used) you keep adding more Hermite cubic spline segments to your spline. By default the Hermite start tangent matches (with opposite sign) the end tangent of the previous segment and in GUI they are linked together (changing one changes the other). So in that configuration the curve appears smooth. However, you can unlink the tangents to cause C1 discontinuity in the spline and control them separately. This is purely GUI work though and how you control the Hermite spline tangents & positions.


Perturbed image texture implementation from renderman language

As you've surmised, the transform() function transforms points from one co-ordinate space to another. (There are also vtransform() and ntransform() for transforming direction vectors and normal vectors, respectively.) The string argument names the co-ordinate space to transform into.

The Renderman Shading Guidelines have this to say about it:

  At the start of shader execution, all point, vector, normal, and matrix variables are expressed in the "current" coordinate system. Exactly which coordinate system is "current" is implementation-dependent. It just so happens to be that "current" is "camera" for PRMan*, but you should never count on this behavior - it is entirely possible that other RenderMan compliant renderers (including future renderers from Pixar) may use some other space (like "world") as "current" space.

It goes on to give a case like this as an example. Most lighting calculations should be done in camera space, but evaluating a noise function should be in the object's co-ordinate system, because you want the noise to stay the same as the object moves through world space.

In your C# implementation, you'll also need to transform the point being shaded from camera space to the object's co-ordinate system. Maybe you've done this already before computing the texture co-ordinates. If not, you'll need to multiply by the object's transformation matrix. Remember that the only use of this transformed point is as an input (like a seed) to the Perlin noise generator. It sets the domain that the noise varies over: world space co-ordinates.

In RSL, the noise() function can return any type you like: a float, a color, a point, or a vector. As you're adding it to another float (u or v), you'll get a float in this code. Really, the two noise() calls, added to s and t, are acting to generate a single 2D noise vector. In your own code, if you are using a 2D vector to store your texture co-ordinates, you can use a single noise function that returns a 2D vector, to get the same effect in one line of code.

If you're interested in making a nice noise generator, Shadertoy has a lot of noise shaders featuring different variants of Perlin noise with different properties (isotropic or not, configurable smoothness and bandwidth) and is worth looking at for inspiration as well as implementation hints.


Physically based rendering and ray tracing

"Physically based" is not a very well defined term, so it's difficult to answer this question exactly.

In general, "physically based" refers to the fact that the algorithm in question is derived from physically based principles. It's not physically correct (because we can't afford that) and some approximations usually have to be made, but it's well known what those approximations are and ideally there is some intuition as to what kind of error it introduces.

This is in contrast to ad hoc models, which are usually made by an artist who observes an effect in real life and tries to write a shader or similar that somehow mimics the look of it. Usually ad hoc models are simpler and cheaper and tend to be the first solutions to turn up for a given problem, but they don't offer any particular insight into what's actually happening. It's also practically impossible to say how accurately such a model is able to reproduce the effect that it attempts to simulate.

In the context of rendering, a "physically based renderer" would therefore be simply something that renders an image using physically based principles, which is a very vague classification. A ray tracer is not inherently physically based, and most early ray tracers in fact used ad hoc models for lighting and similar. From my personal experience, "physically based rendering" used to usually refer to solving the rendering equation. However, it seems that in recent years, many game engines have claimed this term as well to mean "we do energy conservation" or "we no longer use phong".

So really, there's no hard classification of what "physically based rendering" means, and using ray tracing by itself does not make a renderer physically based. In offline rendering, this term is still mostly used to refer to renderers that solve the rendering equation, whereas in real-time rendering, it more likely refers to the use of microfacet models or similar. But it's possible that the meaning of this term will change over the years.


BVH uses aabb for a sphere in Ray Tracing: The Next Week

It does indeed do exactly what you described. Have a close look at how the bvh is constructed, it is a passed a list of concrete shapes that implement hittable, e.g. xy_rect, sphere, etc. The base cases for the recursive bvh construction create leaf nodes that directly reference the concrete shapes that make up the scene.
bvh_node::bvh_node(
    const std::vector<shared_ptr<hittable>>& src_objects,
    size_t start, size_t end, double time0, double time1
) {
    auto objects = src_objects; // Create a modifiable array of the source scene objects
    ...
    size_t object_span = end - start;

    // **create leaf nodes, concrete shapes referenced here**
    if (object_span == 1) {
        left = right = objects[start];
    } else if (object_span == 2) {
        if (comparator(objects[start], objects[start+1])) {
            left = objects[start];
            right = objects[start+1];
        } else {
            left = objects[start+1];
            right = objects[start];
        }
    } else {
       // create internal bvh_node
       ...
}

When bvh_node::hit is called and the ray/aabb test passes, the left and right child nodes have their hit functions called recursively. Eventually when this recursion bottoms out the concrete shapes referenced by the leaf nodes will have their hit functions called as you would expect.
bool bvh_node::hit(const ray& r, double t_min, double t_max, hit_record& rec) const {
    if (!box.hit(r, t_min, t_max))
        return false;

    bool hit_left = left->hit(r, t_min, t_max, rec);
    bool hit_right = right->hit(r, t_min, hit_left ? rec.t : t_max, rec);

    return hit_left || hit_right;
}


How are we supposed to fix brightness with square roots?

What the video is talking about is called gamma correction and it's a very familiar topic for graphics programmers. The first 30 minutes of John Hable's talk on Uncharted 2 rendering is my favorite introduction to "why graphics programmers should care about gamma", as well as being a good introduction to HDR rendering. If you don't want to watch the talk (though I highly recommend it), The Importance of Being Linear is also a good article, and Charles Poynton has a Gamma FAQ that goes into more technical detail and explains the terminology used.
By the way, square/squareroot is only an approximation; the sRGB gamma curve is what's most commonly used in practice (see the formulas there mentioning $C_\text{srgb}$ and $C_\text{linear}$). See also this related question about gamma and color spaces. It's a bit more complicated of a topic than the video makes it sound. :)
To answer your specific questions:

Yes, most image data is stored in gamma format, so the pixel values do not represent linear luminance. I wouldn't describe it as "incorrect", since gamma encoding has the useful property of putting more values in the darks, which is better for perceptual precision (as mentioned in the video). Gamma encoding is absolutely necessary to make good-looking images with only 8 bits per RGB component; if you stored linear values you would need more like 12 bits per component to get enough precision in the darks.

To be more physically correct, when blurring an image, one should convert it to linear luminance, then do the blur (Gaussian or otherwise), then convert it back to the original color space.

The monitor accepts image data in gamma format. So the pixel values do not represent linear luminance, and the output brightness is not linearly proportional to the pixel values.
If it's a good-quality, calibrated monitor, then it accepts data in a specific color space such as sRGB or Adobe RGB and reproduces the colors accurately based on the definition of that color space. If it's a crap uncalibrated monitor, then who knows what you'll get, but it'll be something kinda like sRGB.

GPUs can be configured to do various things here. Modern GPUs have hardware support to automatically convert values between gamma and linear when sampling a texture, or storing a value out to the framebuffer, but the programmer has to explicitly enable that using an API such as Direct3D or OpenGL.

Computer graphics people certainly do know about this topic, and it's critical to get it right when building a realistic renderer for film, AAA games, and so on. However, a lot of more "everyday" apps like web browsers and image editors (even Photoshop, as the video mentions) don't really handle gamma correctly.
It's not an issue of processing power; maybe that was a problem 20 years ago, but today even mobile phones have ample processing power to convert images between gamma and linear. The reason why web browsers and image editors still don't get this right is some combination of (a) ignorance, (b) not believing that gamma is important enough to be worth getting right, and (c) simply being stuck with bad behavior because it's been that way for a long time and users have come to expect it, so they freak out if something changes.

As mentioned, we have to think a lot about gamma when building a renderer. Most computations (e.g. lighting) should be done in linear space, but the input textures and output image should be in gamma space. Also, maybe some parts of rendering such as UI compositing should happen in gamma space because it's more familiar to artists (it works more like Photoshop layer compositing and such). Also, we have to decide if internal render targets are going to be stored in linear or gamma space, as it will affect precision. And we have to get all the conversions right, so that data gets converted between linear and gamma at the right spots in the rendering process.


OK, when you call the regular SubImage functions, you are passing pixel data. You are provoking a pixel unpack operation, where the implementation will read the pixels from your data in the format specified and convert them to the internal format of the image.

However, if already have compressed data, you don't want that conversion. You just want to pass the compressed data to the GL, and have it use it directly. That's what the Compressed functions are for.

If you want to compress uncompressed data, you use the non-Compressed functions. But you really, really, shouldn't. Indeed, with ASTC, you can't compress the data online. And even for other formats, the quality of the result will be rather sketchy.

Do you really want to rely on the quality of a compression algorithm written to be fast rather than good?

The format and type transfer parameters describe a single pixel's worth of data, and compressed formats don't generally store data in units of "pixels". Compressed formats usually use blocks of some size. Regular pixel transfers expect a 1:1 correspondence between a pixel in your data and a pixel in the GL-owned image. That simply isn't the case when dealing with compressed data.

So you need a new function, which just takes a block of bits of a known size, whose format is defined by the image's format, and jams it into the image.

The reason why you can't use a generic compressed format with the Compressed transfer functions should be obvious by now. These functions are for providing compressed data in a specific format; by definition, the generic formats are implementation defined. So there's no way you could possibly know what data to provide for generic compressed transfers.


Avoid having an opaque and transparent variant of each pixel shader

You can use a threshold set through a constant buffer to clip pixels (ie an alpha test):

float threshold; // in constant buffer

float4 color = myTexture.Sample(...);
if (color.a < threshold)
    discard;

Set threshold to 1.0 to discard all transparent fragments; set it to 0.0 to discard nothing. There will be some minor inefficency in the latter case compared to having a separate pixel shader with the alpha test removed entirely, but it shouldn't be a big deal on most GPUs nowadays.


Rounding the edges in a mitered line segment inside of a fragment shader

First, adressing the concerns of speed and performance, you will always have to make some trade-offs between quality and performance, as you already do. If the first version looks fine enough for your use-case and all other versions are much slower, you have to make the decision yourself. You will, however, not get around actually measuring any time difference. But since various ways to do this are not too hard to integrate into an existing wide lines renderer, just testing the various approaches in action and getting some time measurements shouldn't be too difficult, especially with OpenGL timer queries (or whatever the Direct3D equivalent).

So let's get to some possible improvements to your situation. It seems from your link and your explanations that you're using a classic geometry shader based approach for wide line rendereing, converting each line into a screen-aligned quad (or two triangles), computing the vertex positions based on the line width you want to achieve. I recently built something similar, too and there's a few ways you can improve the quality.

As you said yourself, since you're working on rather arbitrary rounded shapes it's probably best to let the fragment shader figure out the exact pixel-shape rather than trying to approximate the circular middle region in the geometry shader. Though, that idea is not completely off-the table and might give for a nice adjustable quality-speed trade-off by adjusting the number of vertices of that circle-part. But it also, together with your current approach, has the problem that it actually needs to know the line's neighbouring lines in order to compute the middle section, which makes it difficult to just put into an existing shader-agnostic rendering system that just draws all the lines as vertex pairs (not to speak of line strips/loops), since you need to add the adjacency information for each line.
If you just want to let the fragmet shader do the job using simple alpha testing, you generate a screen-aligned rectangle that's a little larger than the line, i.e. it spans half the width across the endpoints and thus covers the circular end caps, too. In the fragment shader you just check each fragment's screen-space distance to the nearest point on the actual line (which you transfer from the geometry shader to the fragment shader as a varying). And whenever that distance is larger than the line width (or half the line width, to be precise), you discard the fragment. This way you will basically render a rounded box for each line.

To be precise you render your line as three quad-sections (i.e. 6 triangles in a single strip). The middle section between the ednpoints as one rectangle and the two end-caps as rectangles spanning from the endpoint half the width in the direction of the line. You do this separation to give the rightmost 4 vertices their respective endpoint as varying value and the leftmost one their respective endpoint. This way you get smooth interpolation of the line point across the middle section and clamping in the cap sections.

This distance computation might also be used to antialias the lines, i.e. by using alpha blending (or multisampling with alpha to coverage) and letting alpha go from 1 to 0 along the outermost ~1-pixel thick border of the line.

But this approach has the disadvantage of drawing the uneccessary grey area from your above picture, twice even. This not only causes possibly slowing overdraw (and the explicit discarding inhibits early z-testing), it will also interfere with any possible alpha-based antialiasing as described above.
A solution to mitigate this could be to actually just draw the completely uncapped lines as rectangles from one endpoint to the other (i.e. like your very first picture). And then drawing the connection points (i.e. the line's vertices) as additional circular large points (i.e. using a similar technique of transforming points into screen-aligned quads with the geometry shader and testing for the distance to the actual point center in the fragment shader to get pixel-perfect circles).

This has the advantage that you can actually use stencil testing to only draw the connective sections in the regions where you haven't drawn other parts of the line set before and thus only the violet area, which will tremendously reduce overdraw (at the cost of more complicated rendering and per-fragment operations). This gives very nice results with an alpha-based antialiasing approach as described above (there are however still some minor areas where there's some tiny artefacts).

This stencil testing approach might as well be used on the earlier approach where you draw the line section together with its end-caps when you don't perform any kind of antialiasing or blending, but I think to remember that it caused more artefacts than the separate rendering if you do. I haven't spend too many thoughts on this right now, though, and it might be worth some more investigation. However the seperate solution also has the advantage of requiring the expensive fragment test and possible discard only in the actual connective regions, the line middle sections can be rendered with a straight-forward fragment shader.

Those are just a few thoughts that I happened to ponder on just recently and deemed worth sharing. As said, if they are appropriate for your use-case has to be evaluated, possibly with some timing and quality measurements.


To solve the problem, we will flow three steps, first, calculate translation matrix, second, calculate rotation matrix, third, get transformation matrix.

Because translation is a affine transformation, we need to use a 4x4 matrix to represent translation. I find you use column vector post-multiply matrix. the result of the translation matrix is same as yours.

Translation matrix =

Then, same as your idea, we use basis vectors(same as unit vectors of the coordinate axes) to calculate rotation matrix.

basis z vector: we can get rotated basis z vector by using a vector from VRP to point A, and normalize it.

z = b(VRP) - a = [5,-2,1] - [1,1,1] = [4,-3,0] ,then, normalize to [0.8,-0.6,0]
basis x vector: we know basis vector is a vector perpendicular to z-up plane, we get the x vector by cross product up and z

x = up x z = [1,1,1] x [0.8 ,-0.6 ,0] = [0.6, 0.8, 0]
basis y vector, y is prependicular to z-x plane.

y = z x x = [0.8 ,-0.6, 0] x [0.6, 0.8, 0] = [0, 0, 1]

finally, the rotation matrix is:

Rotation Matrix =

The 4x4 matrix is:

Transformation matrix =

The transformation matrix is a matrix from Object Space to World Space, we should invert this matrix to get World Space to Camera Space matrix.(Thanks for @JarkkoL correcting me).

Refer to 3D Math Primer for Graphics and Game Developmen.

Following your request for clarification: I just combine the translation matrix and the rotation matrix, actually, It's the result translation matrix(T) multiply rotation matrix(R).

The rotation is a 3x3 matrix, we can extend the matrix to a 4x4 matrix with adding a dimension, the 4x4 rotation matrix is:

And the translation matrix is:

So, now we can calculate the transform matrix by translation matrix multiply(T) rotation matrix(R):

Cause the matrix multiply formula is too big for the 4x4 matrix, we just look at the 1st row, In the transform matrix(M), the M11 element equals to T11*R11+T12*R21+T13*R31+T14*R41, cause T12, T13, R41 is 0, so, the M11 equals to R11 right? And in the same way, M14 equals to T11*R14+T12*R24+T13*R34+T14*R44, only R44 equals to 1, so, the M14 equals to T14. so, I just simply put them together in my last answer. For the reason why we put the translation matrix on the left, you can check on the book that I gave you in my previous answer on page 91(translating first and then rotating).


I assume that your texture is deemed to belong to the plane of the trajectory.

Looking at the unprojected ellipse, you can map the circle to it by an anisotropic scaling. If necessary, rotate the texture first if there is any orientation requirement.

To achieve this first mapping, you will compute the four corners of a rectangle, presumably related to the frame formed by the ellipse axis.

Then transform those vertices to the global scene coordinates, the same way to transform the ellipse.


Is there a DSA technique for transform feedback?

There isn't a DSA way to bind buffers to the feedback buffer bindings. But that's because transform feedbacks are objects, and there is a DSA way to attach buffers to transform feedback objects:

GLuint buffer;
glCreateBuffers(1, &buffer);
//Fill in buffer's data.

GLuint feedback;
glCreateTransformFeedbacks(1, &feedback);
glTransformFeedbackBufferRange(feedback, 0, buffer, offset, size);

You can then bind feedback when you're ready to actually do your feedback operations into those buffers.


In order to replace a color with another color, you need some sort of distance metric between the colors and a function for calculating the blending based on that distance.

Finding a distance between 2 colors can be tricky. RGB is a bad color space to measure the distance between colors for perceptual uses. HSV or YCbCr are better.

Once you have the colors in a color space you like, you need to decide how to calculate the distance. I generally calculate the hue distance for starters. If the hue of the color I'm considering replacing is within +/- some threshold (possibly set by the user), then I check to see if the saturation and value are within a certain range, too.

Once I've found a color that's within the ranges specified, I calculate a blending amount. Usually this will be 100% if the color in the image exactly matches the color I want to replace, and will fall off to 0 at the thresholds of hue, saturation and brightness.

You then need to calculate the color you're going to use as a replacement color. It's useful to offset the hue, saturation and value (or Y, Cb, and Cr components) by the same amount as the color in the image differs from the replacement color.

So now you have the original color in the image, the color to replace it with, and a blending amount. You still need to figure out which blend mode you want to use. I would start with what Photoshop calls "Normal" or "Over". That would be a linear mix of the two. If you have colors A and B with a mix amount of 25%, it would be calculated like this:

newColor = A * .25 + B * (1.0 - .25);

Here are some relevant answers I gave to similar questions on Stack Overflow:

How to Change the Hue of a Texture with GLSL

Better Image Coloring Logic/Algorithm


Non least squares formulation to fit catmull rom spline

Disclaimer: I've done some work in the past with wavelet decomposition of image data but it occurred to me that it may be applicable to your problem.  Admittedly, I don't actually know if the following will work well in practice but, given how simple the approach is, and, as no other answer has been put forward, it seems to warrant investigation

Let's assume you have a sequence of 2D (or nD) vertices, $\small[P_0,P_1,P_2,...P_{N-1}]$, and you want to reduce those to a sequence of just $M$ which form your Catmull-Rom control points. For the moment let's assume $\small N=2^kM$, for some small value of $\small k$. We will apply k wavelet decompositions to reduce N to M points.

For our purposes, a single wavelet decomposition step takes a sequence of 2R points, $\small P=[P_0,P_1,...P_{2R-1}]$, and replaces it with two sequences of R points each: a low frequency set, $\small L=[L_0,...L_{R-1}]$, and a high frequency set, $\small H=[H_0,...H_{R-1}]$. I suspect in your use case you might only be interested in the low frequency sequence(s).

There are multiple different wavelets that can be used - the simplest being the Haar wavelet - but I think a linear (see Sweldens & Schröder for an easy introduction)  or perhaps a 'Catmull-Rom-based' cubic wavelet might be suitable (though I don't know if this appears in any literature).

For each wavelet decomposition step we perform two passes; a 'lifting' operation to create all the high frequency terms, i.e. either...

  Linear:  $$H_i := P_{2i+1} - \frac{(P_{2i} +P_{2i+2})}{2}$$
  CR-cubic:$$H_i := P_{2i+1} - \frac{9(P_{2i} +P_{2i+2})-(P_{2i-2} +P_{2i+4})}{16}$$

...followed by an 'update'  which filters some of the high frequency terms back into the 'even'/low frequency terms as a correction. (We'll just assume, perhaps sub-optimally, this is the same for both linear and cubic)

  $$L_i := P_{2i} + \frac{(H_{i-1} + H_{i})}{4}$$

Notes: Actually, in practice, with clever indexing, these calculations can be done 'in place' on the original array.
This has also glossed over what happens when exceeding the bounds of the arrays, e.g. at $P_{2N}$ and $L_{-1}$, but one approach is to simply repeat values and/or pad with zeros.

To do the next wavelet decomposition, you repeat the process with the low frequency data, e.g. "P:=L" and repeat.

I imagine that the final set of low frequency points, L, will form a reasonable set of control points for a piecewise Catmull-Rom curve.

More notes: In this I've assumed that all the original points were 'equally important' and probably 'reasonably evenly spread'. It could be that you may want to look at the magnitudes of some of the {H_i} terms and if some are relatively large (compared to their neighbours) perhaps take action, e.g. say, re-insert them into the lower frequency set?

As requested in the comments, I've put together a little bit of C code to do the "cubic" decomposition. For simplicity, this just uses separate arrays for each of the lower frequency results rather than, say, doing the calculation "in place" in the original array.

#include <stdio.h>
#include <stdlib.h>
#include <assert.h>

#define IS_EVEN(X) (((X)&1)==0)
#define MAX(X,Y) ((X)>=(Y)?(X):(Y))
#define MIN(X,Y) ((X)<=(Y)?(X):(Y))

typedef unsigned int UINT;

#define NUM_DIMS (2)

typedef struct
{
    float pos[NUM_DIMS];
}vec_t;

/*
// Lift
//
// Applies a wavelet "lift" step, i.e. computes high frequency terms, by
// predicting using Catmull-Rom spline through even points, and computing
// differences from odd
*/
static void Lift(const vec_t  PointsIn[],
                 const UINT NumPtsIn,
                 vec_t HFOut[])
{
    //this *example* assumes an even number of input values
    assert(IS_EVEN(NumPtsIn) &&  NumPtsIn>=2);

    /*
    // step through odd values doing a "lifting" step, i.e. predicting
    // a position from neighbourhood of even points and computing the difference
    */
    for(UINT i = 1; i < NumPtsIn; i+= 2)
    {
        /*
        // get 2 neighbours on either side of this odd point. IF we step out of range, then
        // clamp to nearest valid point
        //
        // If you want more efficient code, move these decisions out of the loop
        */
        const vec_t * pv_m3 = &PointsIn[MAX(i, 3) - 3];
        const vec_t * pv_m1 = &PointsIn[i-1];
        const vec_t * pv_p1 = &PointsIn[MIN(i+1, NumPtsIn - 1)];
        const vec_t * pv_p3 = &PointsIn[MIN(i+3, NumPtsIn - 1)];
        const vec_t * pv    = &PointsIn[i];

        vec_t *pResult = &HFOut[i/2];

        for(UINT j = 0; j < NUM_DIMS; j++)
        {
            //Predict using Catmull-Rom interpolation -
            //i.e. mid point of CR spline through the even points {p[i-3], p[i-1], p[i+1], p[i+3]}
            const float PredictedPos = (9.0/16.0) * (pv_m1->pos[j] + pv_p1->pos[j]) - (1.0/16.0) * (pv_m3->pos[j] + pv_p3->pos[j]);

            //Compute the difference from actual pos
            pResult->pos[j] =pv->pos[j] - PredictedPos;
        }
    }
}

/*
// Update
//
// Applies a wavelet "Update" step, i.e. computes the low frequency terms,
// by filtering in the 'errors' in the prediction (i.e the HF terms) into the
// 'even' vals
*/
static void Update(const vec_t  PointsIn[],
                   const UINT NumPtsIn,
                   const vec_t HFIn[],
                   vec_t LFOut[])
{

    static const vec_t TheZeroVec = {0};

    //this *example* assumes an even number of input values
    assert(IS_EVEN(NumPtsIn));

    //init as if for the "i = -1" iteration
    const vec_t * pHFToRight = &TheZeroVec;

    /*
    // step through even values doing an "update" step,
    */
    for(UINT i = 0; i < NumPtsIn; i+= 2)
    {
        /*
        // move one spot over in HF data
        */
        const vec_t *pHFToLeft = pHFToRight;
        pHFToRight = &HFIn[i/2];

        const vec_t * pv    = &PointsIn[i];

        vec_t *pResult = &LFOut[i/2];

        for(UINT j = 0; j < NUM_DIMS; j++)
        {
            //Add in 1/2 of the average of the adjacent HF/Odd terms
            pResult->pos[j] = pv->pos[j] + (pHFToLeft->pos[j] + pHFToRight->pos[j]) * 0.25;
        }
    }
}

//Some test data ...
#define NUM_ORIG_POINTS (512u)
static const vec_t PointsIn[NUM_ORIG_POINTS]=
{
#include "fjord.h"
};

#define NUM_ADDITIONAL_PASSES (4)
extern int main(int argc, const char *argv[])
{

    vec_t HighFreq[NUM_ORIG_POINTS/2];
    vec_t LowFreqs[NUM_ADDITIONAL_PASSES+1][NUM_ORIG_POINTS/2];

    /*
    // Do initial wavelet pass
    */
    Lift(PointsIn, NUM_ORIG_POINTS, HighFreq);
    Update(PointsIn, NUM_ORIG_POINTS, HighFreq, LowFreqs[0]);

    /*
    // Do a few more wavelet passes
    */
    for(UINT Pass=0; Pass < NUM_ADDITIONAL_PASSES; Pass++)
    {
        const UINT CurrentNumSrcPoints = NUM_ORIG_POINTS >> (Pass+1);

        Lift(LowFreqs[Pass],   CurrentNumSrcPoints, HighFreq);
        Update(LowFreqs[Pass], CurrentNumSrcPoints, HighFreq, LowFreqs[Pass+1]);
    }
}

The above has been run on the outline on a section of a map (a section of a fjord apparently) which I've plotted using Excel's scatter plot tools (which I hope will give a 'reasonable' approximation of Catmull-Rom splines without me having to write (or find) my own rendering code)

This shows an original set of 512 points:

..the set of 256 low-frequency points obtained from one wavelet step...

...through to the 128 points obtained by two steps:


Is it possible to create minimal glTF files with vertex and index buffers?

While creating a set of glTF models for a tutorial, I also intended to create THE minimal glTF file.

Update: The following referred to glTF 1.0/1.1. See below for an update of this example to glTF 2.0.

As already mentioned in the answer by 5chdn, one issue may be the material. According to the Appendix A: Default Material of the specification, an asset that does not contain a material should be rendered with a default material that simply shows the object in a uniform, 50% gray color.

Some viewers might not support this, and even if they do, there have been some minor subtleties in the spec regarding this point: In glTF 1.0, it was still necessary to define an "empty" material, because the mesh.primitive.material property was not yet optional. But this should be fixed in glTF 1.1.

So the following should indeed be THE minimal possible glTF asset (version 1.1) :

{
  "scenes" : {
    "scene0" : {
      "nodes" : [ "node0" ]
    }
  },
  "nodes" : {
    "node0" : {
      "meshes" : [ "mesh0" ]
    }
  },

  "meshes" : {
    "mesh0" : {
      "primitives" : [ {
        "attributes" : {
          "POSITION" : "positionsAccessor"
        }
      } ]
    }
  },

  "buffers" : {
    "buffer0" : {
      "uri" : "data:application/octet-stream;base64,AAAAAAAAAAAAAAAAAACAPwAAAAAAAAAAAAAAAAAAgD8AAAAA",
      "byteLength" : 36
    }
  },
  "bufferViews" : {
    "positionsBufferView" : {
      "buffer" : "buffer0",
      "byteOffset" : 0,
      "byteLength" : 36,
      "target" : 34962
    }
  },
  "accessors" : {
    "positionsAccessor" : {
      "bufferView" : "positionsBufferView",
      "byteOffset" : 0,
      "componentType" : 5126,
      "count" : 3,
      "type" : "VEC3",
      "max" : [ 1.0, 1.0, 0.0 ],
      "min" : [ 0.0, 0.0, 0.0 ]
    }
  },

  "asset" : {
    "version" : "1.1"
  }
}

It also shows a single triangle:

The version without embedded buffers is available at https://github.com/javagl/gltfTutorialModels , as well as a triangle with indices, at https://github.com/javagl/gltfTutorialModels/tree/master/Triangle

Update for glTF 2.0:

The concepts and top-level elements of glTF 2.0 are basically the same as for glTF 1.0. But there still have been some changes in the format. Most importantly (for this small example), the top-level dictionaries are now arrays. The following is the same "Triangle" model as above, updated accordingly:

{
   "scenes":[
      {
         "nodes":[ 0 ]
      }
   ],
   "nodes":[
      {
         "mesh":0
      }
   ],
   "meshes":[
      {
         "primitives":[
            {
               "attributes":{
                  "POSITION":1
               },
               "indices":0
            }
         ]
      }
   ],
   "buffers":[
      {
         "uri":"data:application/octet-stream;base64,AAABAAIAAAAAAAAAAAAAAAAAAAAAAIA/AAAAAAAAAAAAAAAAAACAPwAAAAA=",
         "byteLength":44
      }
   ],
   "bufferViews":[
      {
         "buffer":0,
         "byteOffset":0,
         "byteLength":6,
         "target":34963
      },
      {
         "buffer":0,
         "byteOffset":8,
         "byteLength":36,
         "target":34962
      }
   ],
   "accessors":[
      {
         "bufferView":0,
         "byteOffset":0,
         "componentType":5123,
         "count":3,
         "type":"SCALAR",
         "max":[
            2
         ],
         "min":[
            0
         ]
      },
      {
         "bufferView":1,
         "byteOffset":0,
         "componentType":5126,
         "count":3,
         "type":"VEC3",
         "max":[
            1.0,
            1.0,
            0.0
         ],
         "min":[
            0.0,
            0.0,
            0.0
         ]
      }
   ],
   "asset":{
      "version":"2.0"
   }
}

This and other glTF sample models can be found in the glTF sample models repository.


Equivalent of Fresnel law for determining transmitted/diffuse split

Physically, the origin of diffuse light is subsurface scattering, which happens continuously as light travels through a material. So, the proportion of transmitted light depends on the thickness of the object.

There's no precise equivalent to the Fresnel law, but maybe the closest thing is the Beer–Lambert law. It states that the transmitted light falls off exponentially with distance, proportional to $e^{-\sigma x}$, where $x$ is the distance the light has traveled through the material and $\sigma$ is the extinction coefficient (with units of inverse length). The non-transmitted portion of the light may be partly absorbed, and partly scattered through the material.

You get the appearance of "diffuse" when typical length scale of scattering is smaller than you can see ($1/\sigma \ll  1\text{ pixel}$). Then the light gets scattered so many times in such a short distance that by the time it comes back out, its distribution looks isotropic (Lambertian). The apparent diffuse color is controlled by the wavelength-dependence of the scattering and attenuation coefficients.

When the scattering length scale is larger, you get a soft appearance to the material (as in wax, marble, human skin, etc), or partial translucency.


PNG format is lossless format where for compression the image is first "filtered" and this filtered image is then passed to DEFLATE lossless compression algorithm. The purpose of filtering stage is to make the image more compressible by DEFLATE and current method uses delta-compression from previously decoded pixels.

So if your plan is to pre-process the image in some lossy way, then if you can reduce the deltas so that the following entropy encoding can benefit from it, then you get better compression ratio. For example "blockifying" the image as you suggested would only require more storage for the top-left pixel in the block while the deltas for the remaining pixels would be zero, which DEFLATE can compress very efficiently. This same would apply to any continuous constant color regions.


Why is this conditional in my fragment shader so slow?

Pixels get grouped into little squares (how big depends on the hardware) and computed together in a single SIMD pipeline. (struct of arrays type of SIMD)

This pipeline (which has several different names depending on the vendor: warps, wavefronts) will execute operations for each pixel/fragment in lockstep. This means that if 1 pixel needs a computation done then all pixels will compute it and the ones that don't need the result will throw it away.

If all fragments follow the same path through a shader then the other branches won't get executed.

This means that your first method of computing every 16th pixel will be worst case branching.

If you want to still down size your image then just render to a smaller texture and then upscale it.


Strange artifacts while rendering a dielectric material

I solved my problem by adding a small value to t value when the ray is getting refracted.
But I still doubt the result is true. Below is the new generated photo:

Why is the area light reflected twice on top of the sphere? Is there still a bug in my path tracer? The glass sphere is closer to the camera than the area light.


How to keep an object constant in screen space?

Scale the object proportional to its depth (z in camera space) and it will retain the same size on screen regardless of its position in world space.

Additionally, you might also wish to scale the object proportional to the field of view so that it retains the same screen size regardless of the camera zoom. (Specifically, scale it by tan(fov/2)).

Finally, you might also wish to scale it inversely proportional to the pixel size of the window if you want the widget to retain a constant size in pixels regardless of how large the window is. Otherwise it will be constant as a fraction of the window size.

I wouldn't recommend trying to do this in post-projective space (by manipulating w, etc) because it will scale the object's distance from the camera at the same time as scaling its size. Effectively you're scaling it in camera space. What you really want to do is scale it in its own local space before applying world-view-projection transformations.


What does "makeContextCurrent" do exactly?

OpenGL is a state machine. An OpenGL context holds that state. The state contains information such as which textures are bound to which texture units, which attachments the current FBO has, and things like that.

When you set the current context, you are switching all the state from the old context to the new context. Here's an example:

glfwMakeContextCurrent(window1);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, texture1);

glfwMakeContextCurrent(window2);

What this does is makes the context of window1 current. It then enables the 2D texture unit and binds texture1 to it.

Next it makes the context of window2 current. If you were to check the enabled state of GL_TEXTURE_2D it would be false (unless you had previously enabled it on the window2 context). Likewise, if you check which texture is bound to the 2D texture unit, you would likely find no texture was bound, or some texture other than texture1 was bound.

Basically all drawing happens in the current context. If you make a different context current, then all drawing will now happen in that context.

It is common to have a single context per thread in a multi-threaded OpenGL application. That way each thread is drawing into its own context and doesn't mess up the state of contexts on other threads.


Is there any free software to visualize a texture mapped to a sphere?

Although this is borderline "off-topic" for the site, there is perhaps a simple answer:

Why not use a webgl approach such as this one?
http://learningwebgl.com/lessons/lesson11/index.html
(parent page here: http://learningwebgl.com/blog/?p=1253)

Just rotate the image of the moon around until you have one of the poles visible and you'll see the typical artefacts you get with mapping a rectangle to a sphere.
Though I've not tried, you could probably modify the code (use, say, "view page source" to read it, or save a local copy and edit) to load whatever texture your heart desired.


Help me grasp Anisotropic Filtering (AF)

To understand the nature of anisotropic filtering, you need to have a firm understanding of what texture mapping really means.

The term "texture mapping" means to assign positions on an object to locations in a texture. This permits the rasterizer/shader to, for each position on the object, fetch the corresponding data from the texture. The traditional method for doing this is to assign each vertex on an object a texture coordinate, which directly maps that position to a location in the texture. The rasterizer will interpolate this texture coordinate across the faces of the various triangles to produce the texture coordinate used to fetch the color from the texture.

Now, let's think about the process of rasterization. How does that work? It takes a triangle and breaks it up into pixel-sized blocks which we will call "fragments". Now, these pixel-sized blocks are pixel-sized relative to the screen.

But these fragments are not pixel-sized relative to the texture. Imagine if our rasterizer generated a texture coordinate for each corner of the fragment. Now imagine drawing those 4 corners, not in screen space, but in texture space. What shape would this be?

Well, that depends on the texture coordinates. That is, it depends on how the texture is mapped to the polygon. For any particular fragment, it might be an axis-aligned square. It might be a non-axis-aligned square. It might be a rectangle. It might be a trapezoid. It might be pretty much any four-sided figure (or at least, convex ones).

If you were doing texture accessing correctly, the way to get the texture color for a fragment would be to figure out what this rectangle is. Then fetch every texel from the texture within that rectangle (using coverage to scale colors that are on the border). Then average them all together. That would be perfect texture mapping.

It would also be exceedingly slow.

In the interest of performance, we instead try to approximate the real answer. We base things on one texture coordinate, rather than the 4 that cover the entire fragment's area in texel space.

Mipmap-based filtering uses lower-resolution images. These images are basically a shortcut for the perfect method, by pre-computing what large blocks of colors would look like when blended together. So when it selects a lower mipmap, it is using pre-computed values where each texel represents an area of the texture.

Anisotropic filtering works by approximating the perfect method (which can, and should, be coupled with mipmapping) through taking up to a fixed number of additional samples. But how does it figure out the area in texel space to fetch from, since it's still only given one texture coordinate?

Basically, it cheats. Because fragment shaders are executed in 2x2 neighboring blocks, it is possible to compute the derivative of any value in the fragment shader in screen-space X and Y. It then uses these derivatives, coupled with the actual texture coordinate, to compute an approximation of what the true fragment's texture footprint would be. And then it performs a number of samples within this area.

Here's a diagram to help explain it:

The black-and-white squares represent our texture. It's just a checkerboard of 2x2 white and black texels.

The orange dot is the texture coordinate for the fragment in question. The red outline is the fragment's footprint, which is centered on the texture coordinate.

The green boxes represent the texels that an anisotropic filtering implementation might access (the details of anisotropic filtering algorithms are platform specific, so I can only explain the general idea).

This particular diagram suggests that an implementation might access 4 texels. Oh yes, the green boxes cover 7 of them, but the green box in the center could fetch from a smaller mipmap, thus fetching the equivalent of 4 texels in one fetch. The implementation would of course weight the average for that fetch by 4 relative to the single texel ones.

If the anisotropic filtering limit was 2 rather than 4 (or higher), then the implementation would pick 2 of those samples to represent the fragment's footprint.


the canny algorithm is a great start.
it takes a sobel input like so.

Computes its gradient. Then depending on the gradient orientation it compares all neighbouring pixels aligned with it. If its a local maximum the pixel remains black otherwise it is set to white.
this article will probably explain it a lot better.
https://towardsdatascience.com/canny-edge-detection-step-by-step-in-python-computer-vision-b49c3a2d8123
Ive tried to implement my own version using the algorithm provided however I'm struggling to port it to glsl as I think the interpolation might be causing an issue.
here is a working one in shatertoy though, just a beginner though so a bit complex for me.
https://www.shadertoy.com/view/wl2cW3


Best way to group 3d points into planes, if any

A reasonable approach is a continuous sample consensus method. The hough transform can be though of as a discrete sample consensus method, and so becomes intractable for some problems. Sample consensus is nice, because we can impose arbitrary constraints, and handle even the most horrific noise.

Getting Planes: Sample Consensus

The idea is determine the minimum number of observations needed to form a hypothesis, and then check how many observations agree with that hypothesis. We'll call those observations "inliers".

Three point determine a plane. Select three points (deterministically, at random, whatever) from your dataset, fit a plane to them.
Check how many points in your point cloud "support" the plane. Meaning that the plane existing in the world could explain what the sensor saw; the observations "support" the conclusion that the plane exists. For your case, a reasonable metric for "support" would be perpendicular distance to the plane.
In postprocessing, you might cull planes that don't have sufficient width.

Extracting Parallel Planes: Normal Binning

If you don't have a prior on the structure of your data other than "has parallel planes", then you're likely trapped trying to discover as many planes as possible and then filtering for parallel ones.

One might consider a "binning" approach

Start with the first plane, and say its normal belongs to $bin_0$
For each discovered plane, check the angle it makes with every existing bin
If the angle is less than $\epsilon$, add that plane to $bin_n$
If the angle is greater than (or equal, for completeness) to $\epsilon$, create a new bin, $bin_{n+1}$

Presumably the largest bin is the thing you want if you have culled small planes.

Other Notes

Sample consensus will generally perform better if, after fitting to data and finding points that agree, you run a quick optimization to fit a plane to all of the inlier points. Then recompute inliers, and re-run the optimization. After all, it's quite unlikely that the three points you happened to choose were exactly the true plane. Much more likely is that varying plane parameters to minimize point-to-plane error over all of the inlier observations will yield a nice plane, with the noise averaged out.


Is a Lambertian reflector illuminated by a smaller fraction of the incident radiation when it's tilted?

I see some problems in the quote you posted.

  In effect, a point rotated around its normal vector will not change
  the way it reflects light.

This is true, because a Lambertian reflector will never change the way it reflects light. The underlying principle stays the same. Also, Lambertian surfaces are isotropic, so the amount of reflected light won't change either (which is probably what this sentence is aiming for).

  However, the point will change the way it reflects light if it is tilted away from its initial normal vector since the area is illuminated by a smaller fraction of the incident radiation.

Again not true, because the principle doesn't change. The amount may change, except for the special case that the cosine is <= 0 before and after the tilting. The amount does not necessarily grow, except if we define that the cosine equals 1 before, i.e. that the normal points directly towards the light source.

This whole paragraph should probably be rewritten to be less ambiguous. Including isotropy could make it more complete.


A question about derivation in oblique view frustum

check this link

quotations ：

“The points (+/-1, +/-1, 1, 1) are the four corners of the view frustum on the far plane in clip space.”

this should be the point Q' which equals (sgn(Cx), sgn(Cy), 1, 1)


Calculating normals for vertices which aren't a vertex of all neighbouring triangles

For the shading to match, the normals (and any other vertex attributes) of the small triangles' verts would have to match the linear interpolation along the large triangle edge that they adjoin to.

For example: all the vertices in the green box below would need to be set by interpolating between the two blue vertices, based on their position along that edge.

As Nicol alluded to however, you may not get perfectly watertight rendering—or a perfect match in the shading—when constructing a mesh with T-junctions like this.

]


It depends on your GPU. "Tiled renderers" do this for you so the benefit of doing it yourself is minimal (unless you have some more efficient way of knowing the order, e.g. sorting a fixed-viewpoint scene in advance). In this case, your best way of keeping efficiency is by ensuring they are allowed to draw the scene all in one go by avoiding glReadPixels and similar sync points between draw calls.

Even for a truly immediate-mode renderer, instancing can be hugely powerful and reducing overdraw is unlikely to be worth it, but the trade-off comes down to whether you're bottlenecked on the fragment shader or by the memory bandwidth used feeding vertices to the vertex shader. In turn that will depend on how big your instanced geometry is (and how many instances), and how long your fragment shader takes to execute. These are all specific to your application and to the GPU you're running on, so you can only really know by profiling.

As an outright guess I'd say it's unlikely that the change you're suggesting will improve performance, unless you have few instances and a big fragment shader on a non-tiled GPU architecture.


Gravitational calculation of a particle via GLSL / WebGL

Gravity is a form of acceleration. So what I've done in the past is continually add it into the velocity. On each frame, for each particle, I'd do something like this:

// Gravitational acceleration, adjust as necessary
const vec2 gravity = vec2(0.0, -9.8);

// Calculate the new velocity for the particle
velocity = (particleVelocity + gravity) * timeDelta;

// Calculate the new position given the new velocity
newParticlePosition = particlePosition + velocity;

// Update our parameters for the next time around
particleVelocity = velocity / timeDelta;
particlePosition = newParticlePosition;

In this case timeDelta is the fraction of a second between the previous frame and the current frame. So if you're running your animation at 60 fps, it would be 1.0 / 60.0 = 0.01666…

I'm not sure that using a texture for the gravity would help you here as gravity is usually a constant when you're close to the surface of the planet, as it sounds like your particles are from the description given.


Need a fast ray-box intersection that handles if a ray is parallel and in line with a plane of the box

Here is how I found this can be solved.

(The below also happens at the start, just fyi)

  float tmin = std::numeric_limits<float>::lowest();
  float tmax = std::numeric_limits<float>::max();

Add an additional check:

if(rayDir.x == 0.0f)
{
  if (rayOrigin.x() < box.min.x() || rayOrigin.x() > box.max.x())
  {
    return false;
  }
}
else
{
  tx1 = (box.min.x - rayOrigin.x) * invRay.x;
  tx2 = (box.max.x - rayOrigin.x) * invRay.x;

  tmin = std::max(tmin, std::min(tx1, tx2));
  tmax = std::min(tmax, std::max(tx1, tx2));
}

This way the tx1/tx2 values never get calculated if the ray's direction vector is zero in that direction, aka if it's parallel to that plane.

It's not particularly compact or pretty, but it does neatly solve the issue.


I think getting the real data with high precision is not easy without some API.
The first idea would be just to manually use some color based selection tool that you can tell to select all areas in the screenshots according to the lookup table.
But that would not result in some 3D data, but only segmented images where you can annotate the values.

Another method would be to make a UV-map of the mesh, and then by making screenshots from various regions try to extract the color-coded texture.
Then you could read the interpolated color-values of the texture and map them back to the actual values.

The last but maybe strangest idea would be to make a huge number of screenshots and then use some multiview object reconstruction algorithm (I don't know if there is software or if you have to write it yourself.). There are algorithms that not only reconstruct the geometry, but also the texture.
This would then give you a model with the color-texture that again you could map back to the values.

But ideally there would be some export function in the software you used. Maybe this is a question you should ask the developer of the software. Maybe they have this function somehow but it is a bit hidden.


Why can't I utilize multiple CPU cores with OpenGL like Vulkan?

Because the fundamental foundation of OpenGL makes multi-CPU-core submission impossible.

OpenGL, at its core, is a synchronous API. Every OpenGL command behaves "as if" it were executed and completed immediately. Sure, glFlush and glFinish are provided, so that implementations can have some asynchronous execution of commands. But these are essentially fig leaves over the definition of OpenGL as synchronous.

You can attach a texture to an FBO, bind that FBO, render to it, unbind that FBO, bind the texture as a source, and read from it. You can do all of this without any thought as to the fact that you've issued two sets of commands where there can be zero overlap between them. This is because OpenGL implementations are required to issue whatever synchronization mechanisms are needed to make this work. You as the user can be completely oblivious to this.1

Because of that, consider what would happen if you could issue that render-to-texture in one thread and the read-from-texture in another. How exactly would that even work? You would need some way to say "don't execute the read until the write is done". But since the two sets of commands are on different threads, how do you say that?

OpenGL has no mechanism for doing so; the closest it gets would be some form of fence sync. And even that would require that the render-to-texture thread has already finished sending those commands, since glFenceSync issues the fence actual fence (you cannot create a fence that has not yet been submitted). Which means that the render-to-texture thread may as well send the read-from-texture commands too.

To make OpenGL work asynchronously means making lots of adjustments to its memory model. You'll have to add lots of commands for making things visible to other threads. For allowing thread A to not have its commands execute until thread B is finished. You'd need to make command buffers and queues first-class constructs of the API. And so forth.

And you have to do it in a way that's backwards compatible with the existing memory model. Unless you want to go through another core/compatibility split again.

NV_command_list shows the challenges of trying to build this sort of thing into OpenGL. They force you to use ARB_bindless_texture. They had to invent bindless buffers, so that you can provide vertex arrays, UBOs, and SSBOs.

And even then, it's not something that AMD could even implement. Why? Because of image layouts. NVIDIA's hardware has fewer layouts and handles transitions automatically. AMD's hardware requires explicit control of layouts. Once you break the synchronous memory model, that implementation detail of what layout a texture is in becomes really important. You have to have control over it. You have to explicitly state it in various API, and you have to know to change it when you use a texture in a different way.

This is also almost certainly why AMD doesn't implement D3D11 deferred contexts.

Or you can do what Khronos did: make an API that naturally handles all of this stuff. A clean break is ultimately best for everyone. It allows us to jettison other garbage that OpenGL had, so that we can have a nice, clean, lower-level API.

1: While you can be oblivious, obviously you shouldn't be oblivious if you care about things like performance. You should put some work between the render-to-texture and the read from that texture, if at all possible. That's one of the biggest problems with OpenGL's synchronous-by-default API: it makes it easy to do things the slow way, since the API itself doesn't force you to issue that synchronization. In Vulkan, you cannot be ignorant of that synchronization. Or at least, not without your code invoking UB.


What is the name of the RGB thing I want to be able to do, and what free program(s) can do it automatically?

Swapping color channels is called channel swap or swizzling. (Searching for "RGB channel swap" brings up lots of relevant results.) Looking at your image, the squares in the right half appear to be flipped across the diagonal.

Most editing apps have a channel mixer tool that allows you to calculate each output color channel as some combination of the input color channels. In Photoshop it's called "Channel Mixer". I've also seen it referred to as "Color Operations" or just "Color Ops".

The flipping can usually be done with a horizontal flip followed by a vertical flip.


I did only quick google search and found this PhD thesis - "Inverse rendering for computer graphics" by [Marschner1998] - http://graphics.stanford.edu/~srm/thesis/thesis.pdf
Also two published papers - Ramamoorthi et al "A Signal-Processing Framework for Inverse Rendering" http://cseweb.ucsd.edu/~ravir/papers/invrend/invrend.pdf
And Yu, Smith - "InverseRenderNet: Learning Single image inverse -
https://openaccess.thecvf.com/content_CVPR_2019/papers/Yu_InverseRenderNet_Learning_Single_Image_Inverse_Rendering_CVPR_2019_paper.pdf

Based on my experience, just follow prior works in this area and you'll get to the first or first worth mentioning paper in this area. With every paper in the area of raytracing, you'll get reference to [Whitted1980]. Every BRDF will get you back to
[CT1967] and every Spherical Harmonics work to [Green2003].
Based on what you wrote, you must be familiar with reading papers. Therefore, I would suggest to read "Introduction" and "Prior work" to find references to past works in this area of computer graphics.


Reduce kd-tree's nodes' bounding box sizes to their shapes' extents?

No disadvantage that I can see but what you have is not a kd-tree if you have axis-aligned bounding boxes in every node.

A kd-tree only has a splitting plane in every node, so there is no AABB to shrink. The way to deal with empty space in them is to split the node further into an empty node and a smaller node with the same objects as its parent, when justified by whatever heuristic is used. It is just one more possible split to consider when picking how to split a node.


Inconsistent behavior involving VkPhysicalDeviceFeatures.fillModeNonSolid

It turned out that the fillModeNonSolid feature needed to be explicitly enabled during logical device creation.


With vsync enabled where does the wait happen in OpenGL?

It's a bit complicated because the actual wait for vsync does not happen on the CPU, it happens on the GPU. The GPU wait then creates back-pressure that propagates back to the queue of submitted command lists on the CPU, through the GL driver, and eventually to your app. The exact behavior will depend on the details of how the GPU vendor chose to implement their GL driver, and also probably depends on the behavior of the windowing system as it controls frame presentation. Unfortunately there are no real guarantees about this in GL as far as I'm aware, it's really up to the vendors.

In this case, it seems likely that you're seeing the wait become visible to your app at the first GL command that causes the driver to try to begin recording into a new command list (which may entail waiting for a previous command list to finish executing on the GPU, so that the driver can reuse it for the new frame). Clears, draw calls, dispatches, buffer mapping or copying, etc would probably qualify, while (for example) state-setting calls or memory allocation calls might not issue any actual commands to the GPU.


How can I use multiple texturing with shadow mapping?

If multi-texturing is supported, texture coordinate generation will affect the active texture unit (quote from the documentation, emphasis is mine):

  When the ARB_multitexture extension is supported, glTexGen sets the texture generation parameters for the currently active texture unit, selected with glActiveTexture.

So you simply need to make sure texture coordinates generation is disabled for the color texture, but enabled for the shadow texture. As you guessed, this is done with glActiveTexture(), by writing for example:

glActiveTexture(GL_TEXTURE0); // Assuming the color texture to be #0
glDisable(GL_TEXTURE_GEN_S);
glDisable(GL_TEXTURE_GEN_T);
glDisable(GL_TEXTURE_GEN_R);
glDisable(GL_TEXTURE_GEN_Q);

glActiveTexture(GL_TEXTURE1); // Assuming the shadow texture to be #1
glTexGeni(GL_S, GL_TEXTURE_GEN_MODE, GL_EYE_LINEAR);
glTexGenfv(GL_S, GL_EYE_PLANE, textureMatrix.GetRow(0));
glEnable(GL_TEXTURE_GEN_S);
// [...]

This being said, you're adding to yourself a lot of trouble by sticking with fixed pipeline instead of using GLSL.


Operations between two vectors do not make sense unless both vectors are in the same space. So you must either transform the camera position into object space or transform the object position into camera space. I would suggest the latter, since you already likely have a matrix lying around for doing that for other reasons (lighting, etc).


It's happening because they just happened to define the rotation matrix with counterclockwise rotation direction, which is the common convention for polar$\rightarrow$cartesian coordinate system transformation. If you google for the transformation, you'll see that the angle is universally shown to rotate to the counterclockwise direction like below


Renderbuffer needs to be bound before glFramebufferRenderbuffer?

You must fully create a renderbuffer before you can attach it to an FBO.

The pre-DSA OpenGL Object creation functions do not create objects; they create object names. It's like creating an uninitialized pointer variable; it doesn't point to anything yet.

The actual creation of the object happens the first time the name is bound to the context. At that moment, the name is given a real object.

Under normal circumstances, it was pretty much impossible to use an object without binding it to the context, so this distinction couldn't be noticed. However, once the ability to attach objects to one another came to be, this distinction became noticeable.

Pre-DSA, the only way to fully create a renderbuffer object was to bind it to the context.

However, with ARB_DSA/GL 4.5, the glCreate* functions actually create objects (imagine that!). So if you use glCreateRenderbuffers, you can attach the renderbuffer to an FBO without binding it first.


Passing Colour from program to vertex shader

I don't know what language that is, but I'm going to hazard a guess that the arrays points and gl.Ptr([]float32{0.5,0.5,0.0}) are not the same size. If you intend to provide per-vertex colors, then you must provide one color for every vertex. If you want to provide a single color that all vertices share, then that would be a uniform, not a vertex shader input.

Also, commenting out your layout declarations isn't helping. Though if you are going to declare a location for the VS output variable, then the matching FS input needs to have the same location declared.


How to get Ray Tracer working in Visual Studio

No MakeFile or Visual Studio Solution file is provided. So you will need to do the setup yourself in the Visual Studio IDE. Note that you could just drag and drop the files (.h and .cc are known extensions to Visual Studio) in a new console application if you are not/less familiar with Visual Studio.

The code itself is just plain C++ code. No special stuff. So it does not matter which platform or IDE was used during development. In fact the code is pretty self containing except for

#include <math.h>
#include <stdlib.h>
#include <iostream>
#include "float.h" -> <float.h>

But if your Visual Studio is installed and configured correctly, every console application can just include these headers without any problems. (Just try a basic hello world console application, since the problem has in my opinion nothing to do with the code you want to use.) Note that you need a Win32 Console application and NOT a Win32 Project.

It could be possible that the pre-compiled header file is enabled (StdAfx.h or pch.h) in which case you could disable this or include the header as the first included header in every .cpp file.


How to perform vertex selection using unproject (Qt/openGL)

You're basically mixing up your spaces.

If you look at the documentation for QVector3D::unproject, you see that it actually expects coordinates in window space. And this makes sense, since you explicitly query the viewport (i.e. basically the window's pixel dimensions) and give that to the function.

But with your ratio computation above you already transformed your coordinates from that space into normalized device space, i.e. the $[-1,1]$ square. So this whole computation isn't necessary at all and is already done for you by unproject. Likewise are the near and far distances from world/eye space not the same in window space, which uses the normalized $[0,1]$ depth range. If we assume that your viewport is your whole window, then all you need to do is put those pixel coordinates you clicked on into unproject (still accounting for the different Y-convention, though):

dir = QVector3D(event.x, height()-event.y, 0.);    //in window space 0 = near, 1 = far
dir = dir.unproject(modelViewMatrix, projectionMatrix, vp);

A problem is that you don't easily know the window space coordinates of the camera position, since it is in front of the near plane, but the transformation from $[0,1]$ to $[near,far]$ is non-linear. One way to overcome this would be to just unproject a point on the near plane (depth = 0) as origin and the difference to the unproject of that same point on the far plane (depth = 1) as the direction.

QVector3D near(QVector3D(event.x, height()-event.y, 0.);
QVector3D far( QVector3D(event.x, height()-event.y, 1.);
origin = near.unproject(modelViewMatrix, projectionMatrix, vp);
dir = far.unproject(modelViewMatrix, projectionMatrix, vp) - origin;

Or you compute dir just the same and don't unproject the origin at all, rather than just transforming the eye-space origin (which is conveniently $\mathbf{0}$) back into model space (assuming your modelview matrix isn't projective, which it really shouldn't be, otherwise don't forget the W-division):

origin = QVector3D(modelViewMatrix.inverted() * QVector4D(0., 0., 0., 1.));
//origin = QVector3D(modelViewMatrix.inverted().column(3));    //would work the same

If your viewport is not the same as your window dimensions however, you need to fiddle with the mouse coordinates a little more and transform them from the entire window into your viewport rect before unprojecting them. But the point is that

unproject expects window space (i.e. pixel) coordinates.


Why map Hammersley 2D set's (u,v) to sphere's (θ, φ) coordinates (and not to (φ, θ) )?

You can of course, as you suggested, map (u, v) to (φ, θ). Unfortunately, it does not solve the problem for 5 points:

I've changed Holger Dammertz' code a bit (switched u and v), and you see that the problem still persists. For a higher number of points, it really doesn't seem to make a (visual) difference.

The Hammersley point set is a way to very quickly determine points on a sphere that have good, but don't need to have a perfect distribution. In the problematic case of very few points, Hammersley might just not be the tool for the job. Pre-calculating and storing points sounds like a better option to me.


If complex metric is acceptable I would suggest to look at perceptual based approach described here. The metric is designed to pick perceptual difference of two images. There are two main tests for that : luminance based and color based. First one allows to answer the question how important luminance change is by estimating a non uniform threshold factor based on sensitivity to contrast changes depending on spatial frequencies of the image. The second one is based on euclidean distance in CIE LAB color space, but slightly modified to make color difference less important when luminance is in mesopic and scotopic ranges. A list of papers related to that metric can be found here.