How do I efficiently calculate the distance to the edge of a shape?

A texture that stores distance from the edge of the shape, like you described, is called a "distance field" (you'll find lots of results if you google that phrase). Distance fields can be calculated efficiently on the GPU using the jump flooding algorithm (JFA).
This is a multi-pass algorithm that works by sampling pixels at a distance of first ½ the image size, then ¼, then ⅛, and so on down to single pixels. At the end it comes out with a texture with the shortest distance from each pixel to any of the initial "seed" pixels you put in (which would be along the edge of your shape). For more details see the paper, or also check out this blog post by Alan Wolfe which also has some shadertoys with example implementations.
Another article which may interest you is The Quest For Very Wide Outlines by Ben Golus, which addresses this exact issue, discusses a wide variety of GPU implementations (and settles on JFA), and addresses some of the implementation details such as using JFA on an antialiased edge.


Encode two R11G11B10F buffers into one using Y'CbCr?

You might want to look at The Compact YCoCg Framebuffer. It uses a 2-channel buffer to store luminance for every pixel and the two chroma components in half the pixels each, forming a checkerboard. It also uses an edge-aware upsampling filter at the end of the frame to reconstruct the missing chroma components and convert back to RGB.

You could extend this to a single 3-channel buffer to store the diffuse luminance, specular luminance, and checkerboarded chroma (assuming based on the diagram in your question that it suffices to store just one pair of chroma values).

You might also consider using R10G10B10A2 format instead, unless you really need the range of the floating-point format. It's the same amount of bandwidth but will give you a good deal more precision (R11G11B10F only gives you 5–6 bits of mantissa per component). Plus, you can use the 2 bits of alpha to store a couple of flags, such as your diffuse/specular flag.


one popular Real time surface reconstruction method is TSDF (Truncated Signed Distance Function) used by Microsoft for the Kinect. It is based on the VRIP method but it is faster. It is based on depth maps from different (known) camera positions.
you can read more:
https://cs.nyu.edu/courses/fall12/CSCI-GA.2945-001/dl/jiakai-slides.pdf
file:///C:/Users/chaim.dryzun/Downloads/kinectfusion.pdf
From the methods in the reference you gave the fastest is Wavelet based surface reconstruction. If the overall point cloud is good and regular (uniform point density) and without holes and the normals are correct than it gives a very good mesh in a short time.
A more robust and popular method is the Poisson based surface reconstruction. You will need the normals. It gives very good mesh although it takes more time.
Another good method is the PowerCrust surface reconstruction method. It gives very good and accurate mesh, and it does not need the normals. This method produces a polygonal mesh and not triangular mesh, but it is very easy to transform it to triangular. The original code is very slow, but as it is based on calculating Voronoi cells - you can easily re-write it so it will run much faster.
You can read more:
http://web.cs.ucdavis.edu/~amenta/powercrust.html
Regarding the time, working with million of points will take much more than 40 ms, no matter which method you choose (except maybe TSDF). You will have to downsample the point cloud before meshing.


Deleting an OpenGL object is merely a suggestion. Deleting an object will unbind the object from any binding point it is currently bound to. However, buffer objects are not "bound" to VAOs. They are attached to VAOs.

When you call glVertexAttrib*Pointer, you attach a buffer object to the VAO. The buffer object to be attached is the one bound to GL_ARRAY_BUFFER. Unbinding that buffer, or even flat-out deleting it, will not undo the attachment.

And an OpenGL object will continue to exist so long as it is attached to a container object.


Convert RGB, Hex or any other color format to "Standard Color" Programmatically

Problem is that people do not really agree on a 1:1 mapping of color names. But yes such tools exists, this is a simple nearest neighborhood test. You could use any spatial query module to do this. Here is a simple (naive) python example:

from __future__ import print_function
import webcolors
from scipy.spatial import KDTree

# lets populate some names into spatial name database
hexnames = webcolors.css3_hex_to_names
names = []
positions = []

for hex, name in hexnames.iteritems():
    names.append(name)
    positions.append(webcolors.hex_to_rgb(hex))

spacedb = KDTree(positions)

# query nearest point
querycolor = (10,88, 200)
dist, index = spacedb.query(querycolor)

print('The color %r is closest to %s.'%(querycolor, names[index]))

Which results in: The color (10, 88, 200) is closest to royalblue.

I'm using here the 140 css3 defined colors. Bigger and more comprehensive lists can be found online, if you want a smallerlist that would also work. Theres all kinds of possible things you could do, you could for example linearize the sRGB space before calculating distances to name. You could ask for 3 closest names and give double names if the names are halfway.


Your understanding of the behavior of texture units and state is confused. That's to be expected: OpenGL's pre-DSA API is not making it easy to understand this.

As far as core profile OpenGL is concerned, the only texture unit state is what textures are bound to which targets. That's all. Therefore, all glTexParameter calls affect the state of the texture object bound to the currently active texture unit, not of the texture unit itself.

So there is no "precedence". Texture objects have state, which are modified by glTexParameter calls. This is made more clear in direct state access APIs, because glTextureParameter directly takes a texture object, rather than a bound texture target.

If you call glTexParameter when texture object 0 is bound to the given texture target, then you're setting the parameter of the default texture object for that unit/target. And you shouldn't do that.


understanding glm::perspective vs glm::ortho

I don't understand why, if I make the first argument to glm::lookAt be glm::vec3(0.0f, 0.0f, 2.0f), I see nothing.

It's because of the up vector. If you think about the look-at operation, you specify where the camera is and what point it's looking towards. These two things alone leave one degree of freedom: the rotation around that axis. Which way is up on your screen? The third argument is for specifying that: it's a direction in world space which will correspond to up on the screen. If your first argument is that and the rest of the code is as your example, you're calling lookAt((0,0,2), (0,0,0), (0,0,1)). Your up vector points along the $z$-axis, which is the same direction the camera is looking (well, the opposite direction. This can't define which way is up on the screen.

You probably want to pass a vector (0,1,0) as the third argument, which will make $y$ point up on your screen.

If you're making a plan view of a 3D scene, then perspective projection is probably the right answer, but if all your content is going to be in the $z=0$ plane, then you probably want to use an orthographic rather than a perspective projection. Look at What's the difference between orthographic and perspective projection? to learn more about that.

Last of all, remember that if you're projecting an $800m \times 800m$ scene onto $800 \times 800$ pixels, then a $1m$ radius circle will only be one pixel large. You probably need to scale up the size of your circle.


How to blend a segment of an image with another image

Move the child down slightly so the right foot is behind the shoulder.
Reduce their size almost 50%.
Blend only the outline of the child with the background, at most a few pixels.
The men appear to be standing straight while the child appears to be leaned forward slightly - rotate the top of the inserted image backwards ~5°.
Reduce the brightness of the child slightly.

Doing the above should result in a realistic result.


What is the effect of the BRDF denominator?

Firstly, I highly suggest reading Eric Heitz's paper "Understanding the Masking-Shadowing Function in Microfacet-Based BRDFs", which covers the full derivation of microfacet-based BRDFs.

The $\frac{1}{4(N \cdot V)(N \cdot L)}$ term is a side effect of the derivation of the BRDF for specular microfacets. Specifically, it comes from the Jacobian of the reflection transformation. See the paper and/or Walter et. al's 2007 paper for more details.

As for your concern for a divide by zero, the definition of the rendering equation prevents it. Let me explain:

For $(N \cdot V)$ to equal zero, they must be orthogonal. In this case, the visibility term of the rendering equation will cull the cases at / below the horizon.


How to implement physically based unit in pbr renderer

First we do gamma correction

Make sure you actually need to do gamma correction on the albedo texture (or rather "uncorrection", i.e. making it linear). Not all textures are stored in sRGB color space. Also, you're doing it the wrong way around: Going from sRGB to linear color space you need to do the pow(1/2.2) part here and the pow(2.2) part in the end.

  Should I transform the luminance to radiance first before doing the tonemap or maybe I should convert it even before that when doing BRDF calculation?

Yes you should convert before doing the BRDF calculation. Tonemapping is not a feature that's part of the actual physical simulation of light, therefore it doesn't "need" any physical units. The BRDF however is, and here you should make sure that your units "add up". E.g. you wouldn't calculate the surface area of a rectangle by using meters as the unit for one side and inches as the unit for another side.

  Do I need to convert the albedo with photometry curve after doing gamma correction?

I'm not entirely sure here, but I think you don't need to do that. The albedo color basically tells you, how much of a certain light will be reflected. Since you change your light to photometric units (if you do it as suggested in the Frostbite notes), than I would say this already taken care of. You could also look at it like this: Since the color is a ratio and thus unit-less, there is no need to convert it from unit to another.


How does Processing language relate to the industry standard?

It does not relate to low level libraries. Yes, usually each frame is drawn separately, nothing can change if i dont change anything. However the draw loop is often handled by somebody else in for example game engines, etc. So in fact processing is often lower level stuff than what many people would do day to day.

Pushing matrix state is pretty common, it exists even in graphics languages from 1970's.


Animated noise can be created by using time as an extra dimension. So instead of 2D noise, you'd use 3D noise with time as the z-axis position, like ofNoise(x, y, time).

To control the level of detail, you'd use octaves of noise: multiple noise layers with different scales and amplitudes, mixed together. The basic Perlin routines just generate a single octave, so you can re-invoke it with scaled coordinates to add more detail, something like:

float noise = ofNoise(x, y, time);          // first octave
noise += 0.5 * ofNoise(2*x, 2*y, 2*time);   // second octave
noise += 0.25 * ofNoise(4*x, 4*y, 4*time);  // third octave

and so on. The more octaves you put in, the more detailed the noise will be. The article you linked describes this in more detail if you scroll down to Working with Octaves.


Most normal distribution functions (NDFs) are parametrized by some variable (tipically $m$ or $\alpha$) that determines the "roughness" or "spikiness" of the NDF (this is often meant to be the rms slope of the surface).

Here is an example of how a roughness parameter effects an NDF: https://www.desmos.com/calculator/kcevxp3wm0

we can think of the NDF as a function of the direction as well as of the roughness value (as in $D(\theta,\phi,m)$. Here, $\phi$ and $\theta$ were used to represent spherical coordinates for the microsurface normal direction).

A roughness map can also be thought of as a function: one that returns a value for $m$ for every point $p$ on a surface. Hence, $m(p)$.

Since $m$ is an argument of the NDF we can write the previous expression like this: $D(\theta,\phi,m(p))$

And, because $m(p)$ may be different for every $p$ but stays fixed when we only vary $\phi$ and $\theta$, we effectively get a whole different NDF on every point on the surface.


Is it possible to turn a 3d rotation matrix (4x4) into its component parts (rotation, scale, etc.)?

You can decompose the matrix $\mathbf{M} = \mathbf{TRS}$ into basic transformations: translation, scaling, and rotation. Given this matrix:

$$\mathbf{M} = \begin{bmatrix}
a_{00} & a_{01} & a_{02} & a_{03}\\
a_{10} & a_{11} & a_{12} & a_{13}\\
a_{20} & a_{21} & a_{22} & a_{23}\\
0 & 0 & 0 & 1
\end{bmatrix}$$

You can decompose the translation by inspection using the last column $\mathbf{t} = (a_{03}, a_{13}, a_{23})$.

For scaling, we know that the first three columns of the matrix corresponds to the bases (axes). We can get the scale by the length/norm of these vectors, i.e. how much the bases were scaled. So the scale is $\mathbf{s} = (s_0, s_1, s_2)$ where:

$$\begin{matrix}
s_0 = \left \|(a_{00}, a_{10}, a_{20}) \right \|\\
s_1 = \left \|(a_{01}, a_{11}, a_{21}) \right \|\\
s_2 = \left \|(a_{02}, a_{12}, a_{22}) \right \|\\
\end{matrix}$$

Now you have the scale, you can get rid of it using the $3\times 3$ sub-matrix that corresponds to $\mathbf{RS}$ by multiplying the matrix with the inverse of the scale $\mathbf{S}^{-1}$ to get $\mathbf{R}$

$$\begin{align}
\mathbf{(RS)S}^{-1} &=
\begin{bmatrix}
a_{00} & a_{01} & a_{02}\\
a_{10} & a_{11} & a_{12}\\
a_{20} & a_{21} & a_{22}\\
\end{bmatrix}
\begin{bmatrix}
s_0 & 0 & 0\\
0 & s_1 & 0\\
0 & 0 & s_2
\end{bmatrix}^{-1} \\
&=
\begin{bmatrix}
a_{00} & a_{01} & a_{02}\\
a_{10} & a_{11} & a_{12}\\
a_{20} & a_{21} & a_{22}\\
\end{bmatrix}
\begin{bmatrix}
1/s_0 & 0 & 0\\
0 & 1/s_1 & 0\\
0 & 0 & 1/s_2
\end{bmatrix}
\end{align}$$

Thus ($\mathbf{(RS)S}^{-1} = \mathbf{RI} = \mathbf{R}$):

$$\mathbf{R} =
\begin{bmatrix}
a_{00}/s_0 & a_{01}/s_1 & a_{02}/s_2\\
a_{10}/s_0 & a_{11}/s_1 & a_{12}/s_2\\
a_{20}/s_0 & a_{21}/s_1 & a_{22}/s_2\\
\end{bmatrix}$$

This is the final rotation matrix. You can further decompose it using many ways. It is quit lengthy but you can search for decomposing a rotation matrix.

This method only gives an equivalent values in the form of translation, scaling and rotation (the original matrix maybe the result of other types of transformations). It may has problems with floating point precision with the rotation angles if you further used the decomposed angles, rounding errors may accumulate in the computations. You should not use it unless you did not construct the matrix yourself.

If you are the one who constructed the matrix and wanted the decomposition in order to be able to edit and display the translation, scale and rotation individually and independently, probabbly the cleanest why is to store the components of $\mathbf{t}$, $\mathbf{s}$ and $\mathbf{r}$ in a transform class individually as vectors (maybe quaternion for the rotation). Only when you need the transform matrix, construct a $\mathbf{TRS}$ matrix from these components (You can cache the matrix until some component is changed).


Weird patch on reflective spheres on planes in raytracer

I have to disagree with @jack11111, this is definitely not a clipping issue. I have two possible explanations which you can easily test and verify.
Clipping theory

The patch looks pretty constant in color, clipping would be sort of a gradient (modulo 255) of full intensity in at least one channel (R/G/B).
There is no direct light source to cause a strong specular highlight. According to the ground shading and the Lambert-shaded green balls a point light source is placed above, little bit in front of the camera. It seems to have normal intensity that should not cause any trouble. In fact I see in your code that you apply only basic Lambert n . l shading, so no specular highlights possible.
There is no indirect source to cause such a peak. The rays reflect to towards ground that is rather dark. The bright spot on the ground is located else where. That is well visible on the red sphere where the error occurs closer than the next green sphere, but the highlight on the ground is even further away.
It is a strange coincidence that the secondary color of the patch is exactly the same for both spheres.

Normals theory

Normals seem to be computed correctly elsewhere, all your shading is consistent with nice gradients.
You reflections are also correct elsewhere, that will be on problem.

It's a trap! (almost)
Looking closely at your image I believe I can explain what is happening there. The rays hitting the suspicious regions at the sphere will reflect towards ground in a direction very similar to the ground normal.
You ground does not seem to be reflective, but if it were: The rays would reflect back in almost the exact opposite direction hitting back the ball and then heading towards the camera. So they would hit the ball twice which could cause its color to gain dominance. You can test this theory by comparing the rendering results with maxDepth = 1. If the discontinuity disappears then that is the problem.
Upside down
My second explanation is much more likely to be the case. It is also based on the observation that the rays reflected from the suspicious patches hit the ground in a direction pointing almost exactly down. I noticed a bit strange setup of your plane in your code. It has its normal pointing down!?
self.normal = Vec3(0,-1,0)

I am not sure how the Lambert shading of the ground works then (should be then all just ambient in fact), but assuming Vec(0,1,0) is your up then the intersection will in fact yield None
denom = ray.dot(self.normal) #something like 1e-6 in the suspicious area
if abs(denom) > 0.0001: #evaluates to false
    #... true branch ignored
return None #false branch activated, returns no intersection

Having no intersection means that the hit point is shaded only with a constant color

if closest_object == None:
   return Vec3(173/255, 216/255, 230/255)

So to test this explanation, just change the returned color for this case to black and see if the patch darkens.
if closest_object == None:
    return Vec3(0, 0, 0)

I hope that helps. Please let me know the results.
Update
After the author confirmed my second explanation I found the reason in the code. The source of the problem is in the ray-plane intersection method.
Most important of all the epsilon in the comparison should be omitted, since that is causing the patch to be discontinuous. As mentioned earlier, the incoming ray direction is almost Vec3(0,-1,0). You have set your plane normal to exactly Vec3(0,-1,0) so the dot product yields something very close to 0.
So the quick fix would be:
denom = ray.dot(self.normal)
if abs(denom) >= 0: #omit the epsilon
    #...

Your plane normal is inconsistent with how a normal is usually defined. I believe your plane is not supposed to face down. It would be much better and correct to have the plane normal pointing up and then testing for a negative dot product.
A front-face hit occurs when the face normal and the ray are pointing in opposite directions, i.e. their angle θ is smaller than -π/2 or larger than π/2, so cos(θ) < 0. Note that denom is in fact cos(θ). If |θ| < π/2, i.e. cos(θ) > 0, then it is a back-face hit. You ignore such hits for now but once you add transparency to your raycaster you will need them as well.
The correct way would be:
def __init__(self, color, y=0, specular=-1, reflective=0):
    # ...
    self.normal = Vec3(0,1,0) #pointing UP

    def intersect_at_point(self, origin, ray):
        denom = ray.dot(self.normal) # denom == cos(θ)

        if abs(denom) <= 0: #is negative <=> |θ| > π/2; omit the epsilon
            diff  = self.center - ray
            t = diff.dot(self.normal) / denom

            if t > 0: #I suggest to omit the epsilon here as well
                return t
        return None


How to determine if SceneKit can produce sufficiently 3D graphics like Minecraft?

What is missing?

Now that we can see images from your app, the difference that stands out most to me is the lack of shadows and ambient occlusion. This gives a very flat look as the colour of a surface does not vary according to its surroundings. This makes a big difference to how much an image appears to have depth.

Another effect which may be less important is that the surfaces are crystal clear all the way up to the horizon. Adding a mist/fog effect so that objects become gradually more obscured the further they are from the camera helps to give the impression of distance.

Both these effects can be seen most obviously in the first example image in the question. The nearest building shows ambient occlusion on top of the stepped red roof, and also the grass is in partial shade near to the buildings and animals. The trees in the distance look more obviously far away due to the distance fog making them only partially visible.

Can SceneKit provide these?

I haven't used SceneKit myself, but a search online reveals that both ambient occlusion and distance fog effects are both possible using SceneKit.

Another effect not used in the example images, but that you might like to look into, is depth of field. The example images all have everything in focus at the same time, whereas photographs often have a focal distance at which everything is in clear focus, with anything nearer or farther being out of focus. It seems SceneKit can also handle this. Since your competitors don't appear to be using this, you could either view this as unimportant, or as a feature that could give you an advantage.

I'd expect narrow depth of field to make a building app awkward, as parts of the object you are building would be out of focus. You may be interested in a wide depth of field that only creates a subtle blur for objects too distant to be part of what is currently being built, if that suits your purposes.

As you guessed, asking for off site resources is off topic. However, you can ask further questions about computer graphics programming here on this site, if you run into any problems implementing these effects.


Vulkan: Uniform Buffers versus Push Constants for static data

UBOs can be updated at any time with vkCmdUpdateBuffer

From the specification: "vkCmdUpdateBuffer is only allowed outside of a render pass." So "at any time" is not the case.

Even if it were allowed inside of a render pass, it's still a transfer operation. Which means you need to synchronize the memory transfer with the commands that use it. Which slows down performance.

For the general Push Constant vs. Uniform thing, use your judgment. By "judgment", I mean just look at how they work. Push Constants allow you to change their data at any time without doing heavy-weight processes like memory operations, synchronization, or altering the descriptor state. Clearly, they are for frequently changing data. How frequently is "frequently"? Well, that's a judgment call.

Failing that, profile the performance difference.


How is Iridescence handled in a Path Tracer?

Iridesence can be seen in several variations: diffraction as seen in CDs and DVDs (see Stam's SIGGRAPH 1999 paper on this), phase shifting and modulation in multi-layered surfaces as you see in oilspills on water, or structural coloration as you see in butterfly wings. (Also there is the related pearlescence). However, it's not trivial in any way, probably worth a paper or two.


I wouldn't recommend trying to write this code yourself. It's not a question of using into OpenGL - which won't get you another process' framebuffer - but of asking the OS. This is not at all easy to do, and it's work you don't need to do yourself.

Depending on what the streaming protocol is, the easiest way might be just to write or use a client for that protocol directly instead of hooking another application. Failing that, a screen capture tool such as OBS can already do what you want. It can capture another application's frames or your desktop with relative efficiency. You can set up overlays or write a plugin to manipulate the video the way you want. If you access the stream through a web browser, you might not even need to capture a window: it has the built-in feature of being able to render into your video a web browser component it runs itself (you tell it what URL to point to) without having to run a separate web browser and capture it. This feature is how most streamers get things like a Twitch chat view, score board, or timings onto their video.

OBS has a C API for writing fast, native plugins as well as Python and Lua interfaces. And it's open-source, so in the unlikely event you can't write a plugin to do what you want, you could hack it into a private fork directly.


How to convert from XYZ or sRGB to ACEScg/AP1

The discrepancy comes from chromatic adaptation, which is included in the matrix from shadertoy. If you combine XYZ->AP1 with D65->D60 from the aces page you end up with the same matrix that I got by chaining XYZ->sRGB and sRGB->AP1 (with chromatic adaptation) together.


Ray tracing the next week: understanding Instances

There are two ways to 'translate' your object. The first is by moving each point of your object by the desired translation. The second is by translating the origin of your coordinate system. In this case it's the latter. Basically it turns out to be the same, whether you translate your object by a vector $\vec{t}$ or whether you translate your origin by $-\vec{t}$.

For 2) note that you only care about the intersected points (since that is what you use to compute the pixel color), so you compute the offset only when you need it. So yes, you do translate the origin of the ray at each intersect, it's the same as translating the object though (at each ray intersect), since $\vec{p}+\vec{t} - \vec{o} = \vec{p} - (\vec{o}-\vec{t})$.


Frustum aligned volume integration - aliasing issue - temporal reprojection

You can look into alias-free volumetric sampling algorithm by Huw Bowles for potential solution to the ray marching aliasing issues. The basic idea is to snap your samples to planes based on the ray direction that's best explained with this Shadertoy demo.


Confused on reflection coeffcient of Fresnel equations


Algorithms for scan converting b-spline and nurbs

Ignoring Non-uniform B-splines (rational or not), I have had some experience with rasterisation of Beziers and, since there is a trivial mapping from Uniform B-Splines to Beziers, those too.

I have used two different techniques: The first was a scan-line renderer that used Newton-Rhapson to compute the intersection of the current scanline with the curve. This requires the first derivative but, as that can be trivially derived, again as a Bezier representation, from the control points of the parent curve, it's easy to obtain. Further, the bounding box of the control points of the 1st derivative can be useful to detect if the region might have more than 1 intersection with the scan line.
The current scanline can use the previous scanline's solution as a starting point - usually that gives a highly accurate solution with one iteration.

Alternatively, a second approach is to simply apply binary subdivision of the curve and render it as series of straight line segments with your favourite polygonal renderer. If I recall correctly, at least with cubic Beziers, with each subdivision the error between the line segment joining the end points of the (sub)Bezier and the true curve decreases by a factor of 4. In most cases it therefore doesn't take many subdivisions before the difference between the polygonal approximation and the true curve is insignificant.

Finally, if you need info on derivatives of Beziers (or almost everything else related to them), this astounding web-page appeared in my twitter feed recently.


If I understand you correctly then you are probably looking for a way in which lights in the scene interact with models in the scene. Also, you want each light to be visually debuggable in scene, maybe by drawing a cube. Here's one of the simplest way to do it. There could be more better & optimized ways of doing this.

You can have a class for Each light type eg> PointLight, DirectionalLight etc each with their own properties. You need to have a class which then keeps track of all the lights in the scene. This class will have list of light types in the scene. This class will also have functions for adding active lights from the scene to it's list of lights. Usual get-set functions for accessing individual lights will be required too.

class LightsManager
{
// maybe create singleton object here..
public:
   void GatherPointLights(PointLight* light);
   PointLight GetPointLight(int id);

private:
   int numPointLights;
   std::vector<PointLight*> vecPointLights;
}

void LightsManager::GatherPointLights(PointLight* light)
{
    vecPointLights.push_back(light);
    numPointLights = vecPointLights.size();
}

In your class where you create objects in the scene, you can then add light objects from the scene to this LightManager using the function you created earlier.

Assuming scene class where all the objects including Lights are created...

void Scene::Init()
{
    // load mesh
    ...
    ...
    // add lights
    PointLight light1;
    light1.color = Color::Red;
    light1.position = vec3(0,0,0);
    LightManager::getInstance()->GatherPointLights(&light1);
}

If LightManager creates singleton object then you can then access all the lights in the scene by their type during actual rendering to extract their properties in a class where you render actual scene geometry by passing light color, position etc to a shader which draws scene geometry.

Now drawing the actual debug geometry for any light object, say cube for point light, I would make sure my PointLight type object has a way of rendering cube at a Point light position, using it's color property to draw cube of that same color with a flat color shader.

As per making sure lights are not lighting actual light debug geometry, if your cube drawing shader is a flat color shader representing color of a light, then it will not be affected by any light in the scene.

Hope this is what you asked for.


Do I need to use glMemoryBarrier with atomic counters?

The post October 22, 2019 version of the OpenGL 4.6 standard gives atomic counters the same incoherency restrictions as image load/store and SSBO operations. You have to use explicit barriers to synchronize access to them and so forth. This is a textual change from older versions of the standard, but it isn't a behavioral change from implementations. The older wording is considered a bug and thus it should be assumed that atomic counters have always worked incoherently.

Note: the following predates the October 22, 2019 update to OpenGL 4.6.

From the OpenGL 4.6 specification, section 7.12: Shader Memory Access:

  As described in the OpenGL Shading Language Specification, shaders may perform random-access reads and writes to buffer object memory by reading from, assigning to, or performing atomic memory operation on shader buffer variables, or to texture or buffer object memory by using built-in image load, store, and atomic functions operating on shader image variables. The ability to perform such random-access reads and writes in systems that may be highly pipelined results in ordering and synchronization issues discussed in the sections below.

Notice something missing from that list? It mentions "shader buffer variables", and "shader image variables". But atomic counter variables are not mentioned. And atomic counter variables are a different kind of thing from either of those.

Therefore, that entire section (which is what explains the behavior of glMemoryBarrier) does not apply to atomic counters. Well, except where it specifically says otherwise, but that's only where it defines that helper FS invocations don't have side-effects. Note that even this explicitly calls out atomic counters as being something different from regular buffer or image operations: "stores, atomics, and atomic counter updates".

To lend greater credence to this view, the ARB_atomic_counter_buffer_object extension does not mention barriers at all. It doesn't require ARB_shader_image_load_store as a companion extension either, which is what defines glMemoryBarrier. Indeed, it has no interactions whatsoever with image load/store, while image load/store does have specified interactions with atomic counters.

Given the weight of the evidence in the specifications, I would have to say that the Wiki is right. In the interest of full disclosure, I did write that Wiki article. Though I also did all this research before I wrote it.

Basically, what the API seems to be saying is that buffer updates from atomic counters are treated like buffer updates from feedback operations. The implementation is required to track when you attempt to use that buffer for a read operation, then issue any synchronization needed to make sure that you can read the value. Or if you write to the value in the buffer (like clearing it to a value), then the implementation must automatically synchronize this.

The likely reason the specification permits this for atomic counters but not for image load/store is that the range of data for atomic counters is fixed. SSBOs and image load/store through buffer textures can write to arbitrary parts of the bound range of buffer data. By contrast, atomic counters write to 4 bytes times the number of atomic counter variables used by the shader.

Note however that atomic counters are affected by the in-shader memoryBarrier function. That is, atomic counters are subject to the requirements of, as the Wiki calls it, "Internal Visibility". Granted, that's not very useful most of the time, as the primary use of atomic counters is for when multiple invocations are all updating the same one.

On the other hand:

  the glMemoryBarrier() function supports a bit specifically for synchronizing access to atomic counters with other parts of the OpenGL pipeline

The specification describes this particular barrier as:

  Memory accesses using shader atomic counter built-in functions issued after the barrier will reflect data written by shaders prior to the barrier. Additionally, atomic counter function invocations after the barrier will not execute until all memory accesses (e.g., loads, stores, texture fetches, vertex fetches) initiated prior to the barrier complete.

What's curious about this is the "all memory accesses" part. This specific wording is only used in two other bits: shader storage and shader image. Other barriers for coherent operations like query buffers and transform feedbacks tend to use the term "all shader writes".

So... I would say that there is confusion at the specification level.


There is a difference between being able to determine value at compilation time and at runtime.

false && bla
Gets eliminated because of C/C++ lazy operand evaluation rule which is standardized with left to right order.
The compiler generates no code so the machine has no overhead.

step(1.0, x) * texture2D(pic1, uv);

That will depend if x is coming from a uniform value, in which case the drivers will compile an adjustment of the shader (and put it in a combinator cache). At some point before starting to render.
(this can cause hiccups in framerate when new uniform values are encountered that cause massive shader rebuild)

Now, if x is dependant, which means it comes from a previous calculation that comes from varying values or system values, the compiler will cut your expression in 2 parts, the lerp calculus and the sample call.
It will move the sampling of the texture in the first instructions of the shader. (provided it's unconditional in the shader flow).
So you're basically screwed when it's 0, you'll pay the sampling anyway.

In your case (0-2 wrt frag pos), I would recommend using if (inrect(0,0,1,1)) because this is a low frequency "step" function, which means very good branch elimination in 3 quarters of the screen.


When do they actually discard points that have a z component smaller than the near clipping plane?

For actual rendering hardware, they don't. Clipping happens in clip space, before the NDC transform. "Points" outside of the clipping volume are never seen by the rasterizer (in theory, at least. In practice, guard-band clipping is often used to minimize the need for clipping against the sizes).

The renderer presented at that site doesn't appear to do clipping at all. It has an approximation of what would be clipped, but it doesn't actually do the clipping.


What are sample gaps during scan conversion?

Sample gaps will occur if you don't do things "properly".

In OpenGL or D3D, assuming a consistent winding order, if you have triangles  ABC and CBD, then when a sample point - that is a test during scan conversion to determine if a point (read pixel) is inside a triangle - lies exactly on the shared edge BC, then that sample will belong to exactly one of those two triangles.

This avoids both gaps and double filling. The latter would be problematic with, say, translucency or stencil operations.


Identifying your axes in both figures and adding the camera position to your first figure would help you understand what's going on.

You could also have a single variables for all your points, generating a 2D matrix with the rows as each point and columns as the components $x$, $y$ and $z$. That way, you could handle the projection using a simple matrix multiplication instead of handling each row separately.

In the problem statement, would be interesting to know the direction of your rotation, and more importantly, the original camera direction and its up vector. I guess you camera is rotated by 160° counter-clockwise, the original camera direction is $[0, 0, 1]$ and its up vector is $[0, 1, 0]$. If any of those assumptions is wrong, the rest of the answer will be wrong.

There is no sign of mm to m conversion in your code. Either your focal should be $0.016$ and $S_x = S_y = 0.0001$ or your point coordinates should be multiplied by $0.00001$.

Let's reason about where should the points end up in your image. For example, your camera center, without rotation, is pointing at the $[-1,1,x]$ line. Since the points are all on the $z=0.5$ plane, we can do the following analysis: if we focus on the $x$-axis, we can see that $tan(160°) \cdot (5 - 0.5) = 1.64...$, so the camera center will end up slightly to the left of the points (as the camera is at $x=-1$), so the center will end up at $\approx 0.64$, meaning that the points will appear on the right section of the image. Furthermore, the camera has the same $y$ coordinates as two point, and since the $y$ coordinates are not changed by the rotation, they should still end up on the same coordinates after the transform, meaning on the central row of the image.

A good way to check your answer is by using an existing 3D modeler like Blender:

Be cautious with Blender' coordinates system, for example the default camera vector is [0, 0, -1].
Here's the render:

Focal was set to another value to make the sphere more visible. So we see that the bottom two points are on the middle row of the image and the points are slightly to the right of the image.

I implemented your homework in Python:

import numpy as np

from matplotlib import pyplot as plt
from mpl_toolkits.mplot3d import axes3d, Axes3D

# Parameters
f_mm = 0.016
f_px = f_mm / 0.00001
t_cam = np.array([[-1., 1., 5.]]).T
t_cam_homogeneous = np.vstack((t_cam, np.array([[0]])))
theta = 160. * np.pi / 180.
ox = 320
oy = 240
# Rotation and points are in homogeneous coordinates
rot_cam = np.array([[np.cos(theta), 0, np.sin(theta)],
                    [0, 1, 0],
                    [-np.sin(theta), 0, np.cos(theta)]])
points = np.array([[1, 1, 0.5, 1],
                   [1, 1.5, 0.5, 1],
                   [1.5, 1.5, 0.5, 1],
                   [1.5, 1, 0.5, 1]]).T

# Compute projection matrix using intrinsics and extrinsics
intrinsics = np.array([[f_px, 0, ox],
                       [0, f_px, oy],
                       [0, 0, 1]])
extrinsics = np.hstack((rot_cam, rot_cam.dot(-t_cam)))

rot_cam2 = np.identity(4); rot_cam2[:3,:3] = rot_cam
camera_coordinates = rot_cam2.dot(points - t_cam_homogeneous)
camera_coordinates = camera_coordinates[:3,:] / camera_coordinates[3,:]

# Perform the projection
projected_points = intrinsics.dot(camera_coordinates)
projected_points = projected_points[:2,:] / projected_points[2,:]
projected_points[0,:] = -projected_points[0,:] # Inverted x-axis because camera is pointing toward [0, 0, 1]

fig = plt.figure()
ax = Axes3D(fig)
ax.scatter(points[0,:], points[1,:], points[2,:], label="Points")
ax.scatter(t_cam[0], t_cam[1], t_cam[2], c="red", label="Camera")
ax.set_xlabel("X axis"); ax.set_ylabel("Y axis"); ax.set_zlabel("Z axis")
plt.title("World coordinates")
plt.legend()
plt.savefig('world_coordinates.png', dpi=300, bbox_inches="tight")

fig = plt.figure()
ax = Axes3D(fig)
ax.scatter(camera_coordinates[0,:], camera_coordinates[1,:], camera_coordinates[2,:], label="Points")
ax.scatter(0, 0, 0, c="red", label="Camera")
ax.set_xlabel("X axis"); ax.set_ylabel("Y axis"); ax.set_zlabel("Z axis")
plt.title("Camera coordinates")
plt.legend()
plt.savefig('camera_coordinates.png', dpi=300, bbox_inches="tight")

plt.figure()
plt.scatter(projected_points[0,:], projected_points[1,:])
plt.xlabel("X axis"); plt.ylabel("Y axis")
plt.title("Image coordinates")
plt.savefig('image_coordinates.png', dpi=300, bbox_inches="tight")

plt.show()

This yield me those figures:

Respectively: World coordinates, Camera coordinates, Camera coordinates rotated to fit slightly the camera orientation (note that here the camera vector goes toward the figure viewpoint, it does not "enter" the figure) and image coordinates.

So we see that the vertical coordinates for the bottom points are correctly on the middle row (240) and the points are on the right side of the image (horizontal value > 320).

I believe one bug you had is that you found negative X values, so you negated the focals (-f/Sxy) in the intrinsics matrix to compensate. The issue here is that we've assumed that the camera initially was pointing toward $[0, 0, 1]$ (otherwise the 160° rotation would not point toward the points). If you look at it that way, the $x$-axis increases when going toward the left, the inverse of this axis should be taken.

Both our results seems similar to me, just that you supposed a $[0, -1, 0]$ up vector for the camera (in fact both axes were mirrored because you negated both focals), and did the computation in mm instead of meters.


Why is it twice as expensive to make a noise function that can be tiled?

It's unfortunate that people commonly recommend this.  Blending between two (or four, etc.) translated copies of a noise function in that way is a pretty bad idea. Not only is it expensive, it doesn't even produce correct-looking results!

On the left is some Perlin noise. On the right is two instances of Perlin noise, stacked and blended left-to-right.

The difference is kind of subtle, but you can see that the second image has lower contrast in a vertical column running down the middle.  That's where it's a 50% blend between two different instances of the noise function. Such a blend doesn't look like the original noise function: it just looks like a muddy mess.

OK, so it's not quite that bad just looking at the raw noise...but if you then do any nonlinear transformations on the image, the nonuniform contrast can cause issues. For instance, here are those images thresholded at 60%. (Think of generating islands in an ocean, for instance.)

Now you can plainly see how the image on the right has fewer, smaller white areas in the middle.

Like you mentioned, for grid-based noise like Perlin, a better way is to tile the pseudorandom gradients at the grid points. That's easy and cheap to do, and then you can apply the interpolation algorithm to the gradients as usual (much like bilinear interpolation of a tiling texture). This produces tiling noise without any weird artifacts, because it works with the underlying noise algorithm rather than over the top of it. You can use a similar strategy with Worley noise (cellular noise) by tiling the random feature points it uses as a basis.

With multiple octaves of noise it's not always so easy, though. If the relative scale between the octaves (aka "lacunarity") isn't an integer or simple rational number, then you may not be able to find a convenient tiling point where all the octaves' grids match up. You could tile each octave independently, but the overall noise would still not be tilable in that case.


How to include OpenGL extension loader in my project

GLAD is designed around you simply including it directly in your project. That's the point of it; it is easy to use because you don't have to deal with any external dependencies. No messy libraries or DLLs to generate. It's just another source/header file in your build process.

Furthermore, your code will (presumably) be written to expect GLAD generation with a specific OpenGL version and set of extensions. If you require the user to use GLAD to generate the file, you also have to tell them what version+extensions to use, or else your code won't compile. If you just ship the GLAD-generated files with your program, there's no chance of breakage (or at least, you could detect the breakage because your own build doesn't work anymore).


How to Sample 3D Points Outside and Inside the Mesh Surface

What you want is something that can perform the inside-outside test for meshes. The simplest solution would be to use Trimesh's implementation which can be ran with Embree to accelerate ray queries.
I recommend you check out recent literature on neural implicit representations if you want more alternatives. Your problem arises when one wants to learn 3D shapes using small neural networks; it is then necessary to sample points and their groundtruth attributes with respect to the mesh (signed distances or occupancies of points). Here's one example for SDFs written in CUDA. NVIDIA's Kaolin also has a CUDA kernel for this IIRC.
For more troublesome meshes, you can also use the generalized winding number, which has an implementation in libigl but I think their code is CPU-only.


I did not go into much detail with your code but the first thing that comes to mind is that you are rendering with respect to the resolution passed into the shader.

vec2 uv = fragCoord.xy/iResolution.xy;

hope this helps.

Maybe calculate the aspect ratio?

float aspect = float(iResolution.x) / float(iResolution.y ? iResolution.y: 1);


Does this type of memory-mapping for a display exist?

In your 31 bit solution, the thing most likely to happen there is that you'd give an extra bit to green and make a 32 bit solution.  In 16 bit color for instance, r and b get 5 bits while g gets 6 bits, because your eye can distinguish more greens.

Or maybe you'd use that extra bit to have 1 bit alpha? Not sure if that'd be very useful though...

As far as the rest, whether you have a gpu or not, writing to single pixels at a time is bad form.

Instead, you ideally want to do block operations that either fill or copy large strips of memory.

For some things, like alpha blending, that isn't able to be done as a block operation very easily so is a more expensive operation in general.

As a solution to this, modern computers have a processor that is closer to the video memory, so there isn't too much latency when pushing pixels back and forth across the main bus to do things like calculate alpha blended values involving the source and destination pixel.

Also, instead of fewer more powerful cores, there are many less powerful cores, which helps address the situation of needing "one off" pixel operations.

That processor I'm referring to is the gpu and the programs you write for it are called shaders (;

What you are asking about doesn't exist to my knowledge and seems like a bad usage pattern, so seems unlikely to exist IMO.


Do I need to rebind uniforms or attributes when changing shader programs?

You don't need to rebind the attributes, so long as you ensure that their location stays the same in both shaders. (Usually using the layout(location = X) syntax in GLSL, but can also be done with glBindAttribLocation if former is not available.)

Uniforms, however, are part of the Shader Object state, and so will need to be set at least once for every shader. One way to minimize the cost of this is to use a Uniform Buffer Object, which will contain all your uniform values and can then be bound to a shader with a single call.


How to improve on bilateral upsampling in real time scenarios

Nearest-depth filtering is an alternative to bilaterial filtering that was specifically developed for upsampling low-resolution transparent rendering. It's a bit simpler than bilateral sampling, in that it only requires one sample from your low-resolution texture. However it can still have issues, particularly with high-frequency geometry that isn't well-represented in your low-resolution depth buffer.

At SIGGRAPH 2013, Bungie presented a new approach for handling very-low-resolution particles using what they called a "variance depth map". You can find the slides here in PDF and PPTX format.

One interesting approach that hasn't been used yet (at least, to my knowledge) would be to leverage MSAA. The idea is that you take your full-resolution depth buffer, and then alias it (or convert it, if the API doesn't allow you to do so) as a 4xMSAA depth buffer with half the width and height. You could then render your low-resolution transparents to a half-resolution render target with 4xMSAA, using your "4xMSAA" depth buffer for depth testing. Then to upscale, for each high-resolution pixel you would access the corresponding subsample from your low-resolution MSAA render target, and blend it on top. This would give you full-resolution depth testing, but half-resolution shading (with framebuffer bandwidth somewhere in between full-res and half-res due to compression). The catch here is that you need programmable MSAA sample points to do this, since you'll want to use an ordered grid sampling pattern for your low-resolution rendering instead of the typical rotated grid pattern. Newer GPU's (Maxwell from Nvidia, GCN from AMD) support doing this through vendor extensions in D3D11 and OpenGL, and it can be done natively on consoles.


I don't know about other contexts, but for DirectX resolving a texture means blending multi-sampled texture into a non-multisampled one. For simple scenarios this is usually done automatically by the output merger, but is often needed to be done explicitly (e.g. ResolveSubresource) when using multi-sampled render target as an input for a next render pass (e.g. post-processing). Reasons being both performance and lower complexity of the following pass shaders.


Maps with z = 1.0 everywhere are likely partial derivative maps. This is a variant on normal maps that's become fairly widespread. There are a few reasons to prefer it, but probably the most important is it lets you store the map as just two channels (red and green), which allows using better quality BC5 compression, while being cheap to decode in the shader. The normals in such a map are of necessity not all unit length, and as you pointed out that will affect interpolation and blending slightly, but this is generally deemed an acceptable trade-off.

However, if it's purporting to be a traditional old-school normal map and it has pixels that are significantly off from unit length, that sounds odd and probably unintended. I'm not sure why that would have happened in the particular Wikimedia .png example you linked. However, non-unit-length normals could generally be introduced by doing filtering/blending/compositing operations on the image without renormalizing after (for example, if a normal map was rendered at a higher resolution and then downsampled using a standard image resampling algorithm). They could also be introduced by lossy compression.


Volume of Intersection of Cube and Sphere

There are many ways, but the fastest I think is to create an octtree for your cube and choose a small subdivision level. Then do simple collision detection of the cube to the sphere. If the sphere collides with the cube, subdivide it into 8 smaller cubes. Test each of those cubes and repeat. When you are finished, you will know how many cubes have intersected the sphere, just multiply their dimensions by the number of intersections and you have a close approximation of total volume.

You may want to say that a cube collides with a sphere if its center lies within one radius of the spheres center.

I recently did something similar for a CNC simulator I wrote.

Good luck!

As far as the second part of your question, there would have to be constraints. If the cube is substantially smaller than the sphere, it may not be possible to satisfy your volume requirements. Given that constraints are in place, this method is fast enough that you could just drive your sphere into the cube until your required volume is met.


What is the state-of-the-art on using computers to "clean-up" images?

The closest thing I know of to the "computer: enhance!" trope in real life is the family of Single Image Super-Resolution techniques. That page shows a number of examples of the results on various images. You can see that while it improves the visual quality of the enlarged images, it's a long way from what you see on TV where they can read the text on a letter that's reflected in a wine glass, or recognize the face of someone standing in the shadows, etc.

The technique basically works by observing that a single image often contains recurring instances of the same patterns or structures. When you have multiple copies of the same pattern, they often occur at different sub-pixel offsets, i.e. aligned differently to the pixel grid, which means that each copy contains slightly different information about the underlying pattern, and by putting them together you can recover (really, guess at) a higher-resolution version of that pattern. This can then be used to "fill in the missing detail" wherever that pattern occurs. This is a very handwavy explanation, but you can see the paper (linked from the above site) for more details. Most of it is automated, but I'd guess it still requires a fair amount of human parameter-tuning to get good results.

The paper is from 2009, and it looks like there have been a few follow-ups since then, but only incremental improvements.

Also note that if you have a specific use case such as reading vehicle license plates, I'd guess it's possible to use machine-learning techniques to get much better results than you can in general. In that case, you have a concrete, known set of possible shapes (the characters that can appear on a license plate, in the particular font used for license plates), and rather than "enhancing" an image of literally anything, you're just trying to find the set of characters that best matches the image you're looking at. That's not my area of expertise, though.


Stretch the cube mesh but repeat the texture?

You need to set your texture coordinates appropriately. Let's look at one side of your cube. Let's say it's 100 x 100. If the user stretches the cube to be 100 x 150, you need to set the texture coordinates of the corner vertices to be:

lower left = <0.0, 0.0>
lower right = <1.5, 0.0>
upper right = <1.5, 1.0>
upper left = <0.0, 1.0>

That's the easiest way to handle it. Another way to handle it would be to pass either as scale factor or scale matrix into your fragment shader and apply that to your texture coordinates before doing any sampling.

To elaborate on the texture matrix idea, it's fairly simple. In your fragment shader, you'd have a uniform, say textureMatrix that's a mat4. It can contain any sort of transform you need - scale, rotation, translation, shear, etc. Before you sample any texture, you run the uv coordinates through the texture matrix, and then use the resulting values for your lookup. Something like this:

vec4 uv = gl_TexCoord[ 0 ]; // or whatever you called it coming from your vert. shader
vec4 newUV = textureMatrix * uv;
vec4 pixel = texture2D(texture, newUV.xy);

To do your non-uniform scaling, your matrix would be a scale matrix. For example if you wanted to scale the Y coordinate, you could use a matrix like this:

| 1 0 0 0 |
| 0 2 0 0 |
| 0 0 0 0 |
| 0 0 0 1 |


This seems, however, completely unrelated since no gamma correction is involved yet at this stage in the code.

The values are indeed not gamma corrected. Earlier in the presentation (On page 32) you have the toInt function which does the gamma correction. This function is called when the image is written to a file.

Then the clamping itself.
If we have a list of 5 samples.

0.1; 0.4; 0.2; 0.8; 0.6

It averages to 0.42. However, if one samples becomes very bright.

0.1; 0.4; 8.0; 0.8; 0.6

The average would become 1.98. That is a lot brighter. Because we take random samples, it could be that these two averages are neighboring pixels. This would cause a few pixels in the entire image to be much brighter than their neighbors. This is called a firefly. These fireflies are often not wanted, as they stick out in the image and make it look a lot more noisy. Therefore we want to try and remove these fireflies.

An example image of this can be found in blender's cycles documentation here. The bright pixels are fireflies and are unwanted.

There are three ways to remove a firefly.

Add more samples. By adding more samples, you get a truer image and that one bright sample would end up with less influence. This does however take longer to render.
Some sort of denoising algorithm. The algorithm could find these fireflies and remove them. However, it will almost always change other parts of the image which is often unwanted, and there is also no room for it in smallpt.
Discarding the bright sample or similar. This bright sample is the culprit of the problem and we could always decide to just ignore it. However, that bright sample is there for a reason and removing it would most of the times not be the greatest idea (it also makes things more complicated). Clamping however would still allow us to somewhat discard the bright sample without actually discarding it and is very simple and easy to implement. We just make it not have such a big influence on the image.

Clamping is often the choice for most render engines. If you would look at any commercial render engine, or even hobby path tracers, they allow you to clamp the samples. Normally you clamp the individual samples instead of the sub-pixels and more specifically you clamp indirect samples with its own clamping value, but they have about the same effect.

This clamping does change the image as it makes some areas or all darker since it removes energy (by clamping the color values) from the scene. This would become unbiased, yet it is often done (and almost always available in unbiased render engines) simply because it produces visually nicer images, because there are no fireflies and fireflies are generally worse than noise. Often you would not notice any artifacts when clamping, other than the removed fireflies.

In the case of smallpt, clamping it between 0.0 and 1.0 seems to work well for that one scene. However, I do recommend to just tinker with the code and render two images out for yourself, one with clamping and one without.


How to do CSG on 3D triangulated objects, originally stored as a .obj file, using openGL or other libraries?

Describing a full CSG algorithm in a forum post is something really hard, and probably you will not find a complete answer to what you are asking here, but I can give you some suggestions.

First of all, CSG on polygon meshes is not usually employed on real-time graphics, because the algorithms are still considered to be too much resource-intensive. If you are looking for these kinds of algorithms, I would suggest reading books on computational geometry, of which Mark De Berg's is commonly considered a good introduction to the topic. If you are planning to pre-compute everything, then classic CSG algorithms are the way to go.

However, for just adding craters on a field, CSG seems to be too much overkill to me, and I would suggest some completely different approaches. If you want just a visual result (i.e., your player won't be able to go into the crater), you could use techniques such as z-buffer carving or parallax occlusion mapping, combined with some sort of decal system (preferably, using some deferred-like rendering system). Otherwise, if you need proper collision detection, you will actually need to change your mesh. You can easily apply some vertex modification operators around the center of each crater, such as pushing them down according to a radial-basis function (i.e., spheres, gaussians, etc). To accomplish this in realtime graphics, I would suggest using compute shaders or vertex feedback transforms, instead of computing everything in the CPU. These techniques will even allow you to add craters dynamically. Think of some really nice meteor strikes taking place.

I'm sorry for not being too descriptive, but I hope that some of the keywords that I have put here may help you out on finding a solution to your problem.


Is there a material that contributes to GI, but isn't directly visible?

You can use single sided triangles for the ceiling so that they are pointing towards the room. This way the ceiling influences the GI in the room but you can see through it when observing from outside


You could phrase it in terms of elevation away from various planes. So 30° above the x-z plane, and 0° from the y-z plane (or whatever it actually is). It's sort of like spherical coordinates where you'd have theta and phi.


Is it viable to create computer graphics via terrain scanning?

Yes that is possible using photogrammetry like @SimonF mentioned in his comment. There is plenty of commercial and free software. Peter Falkingham has a great blog with many reviews mainly for the free tools.

Here is a video of scanning a mountain using my favourite open-source solution. Specialists do much more extreme scanning using their tools, the resulting Matterhorn point-cloud dataset is used by some of my research colleagues as well.

Requirements

To cover a mountain, you will need a lot of photos. Depending on the peak size and required precision you may end up with thousands to hundreds of thousands of photos. To process all of them, it needs a lot of RAM and powerful GPU(s). I did a reconstruction for 10k photos on a Ryzen 2700X with 64 GB Ram and a 1080 Ti inside... it took 10 days to complete.

There are two critical phases:

Alignment to create the sparse point cloud is partly CPU-based and requires a lot of RAM. Once the system starts to swap to the HDD, you've lost.
Patch-match for the dense point cloud heavily uses the GPU and takes most of the reconstruction time.

Scanning procedure

In ideal case all photos would be taken in exact the same moment. When you fly around it takes time and the sun moves. Shadows change. That is often a problem at reconstruction so cloudy days are generally better. Try to get all footage on one day and as fast as possible. The atmospheric conditions may change the next day(s). I hope there is no snow and no water, those are the most difficult to reconstruct due to reflections :)


Alpha blending between two overlapping semi-transparent shapes

I used Stencil Buffer to fixing your problem , you need a way for checking overlapping two or more shapes

 Shader "Custom/SemiTransparent"
{
    Properties
    {
        _Color("Color",Color) = (0,0,1,0.1)
    }
    SubShader
    {
Tags {"Queue"="Transparent" "IgnoreProjector"="true" "RenderType"="Transparent"}
ZWrite Off Blend SrcAlpha OneMinusSrcAlpha Cull Off

        LOD 100

        Pass
        {
            Stencil {
                Ref 0
                Comp Equal
                Pass IncrSat
                Fail IncrSat
            }

            CGPROGRAM
            #pragma vertex vert
            #pragma fragment frag

            #include "UnityCG.cginc"

            fixed4 _Color;

            struct appdata
            {
                float4 vertex : POSITION;
            };

            struct v2f
            {
                float4 vertex : SV_POSITION;
            };

            v2f vert (appdata v)
            {
                v2f o;
                o.vertex = UnityObjectToClipPos(v.vertex);
                return o;
            }

            fixed4 frag (v2f i) : SV_Target
            {
                fixed4 col = _Color;
                return col;
            }
            ENDCG
        }
    }
}


What happens with the framebuffer after the fragment shader is done?

It entirely depends what hardware and OS you're on. It's up to the integration between the windowing system and GL. In the simple case where one application is rendering full-screen directly, the framebuffer is in memory that's accessible by both the GPU and the display controller. After the GPU is finished, and when the display controller gets to a vsync (the time to start drawing a new frame to the screen), the display controller just reads out the framebuffer.

In a compositing system like modern MS Windows or Android, your app's framebuffer is not used by the display controller directly. After you finish drawing your frame, the compositor runs. On the GPU, it reads in your framebuffer, puts it in the right place on the desktop, and writes the composited desktop into the real framebuffer. The display controller then reads out the desktop framebuffer. There's also an optimization possible using overlays, which we have another question about.

Copying completed frames into CPU memory is to be avoided. If you had to do this every frame, your application probably couldn't run at a reasonable frame-rate. Even reading back a single pixel (with glReadPixels) is a big slow-down, because the CPU has to wait for the GPU to finish, and they have to invalidate the cache of any shared memory. Reading back a million pixels is extra slow on desktop because the CPU and GPU have separate memory and the path between them is relatively slow. It's very rare that it should be necessary: mainly it's needed for screenshots, or for debugging.


Reliable test for intersection of two Bezier curves

An alternative way to formulate the problem is to define a function that gives the distance between points on the two curves, as a function of the curves' parameters. Then attempt to find the global minimum of this function. If the curves intersect, the minimum will be zero; otherwise the minimum will be some positive distance.

To be explicit, given a pair of 2D curves defined by $c_1, c_2 : [0, 1] \to \mathbb{R}^2$, define the distance-squared as

$$ f(u,v) : [0,1]^2 \to \mathbb{R}_{\geq 0} \equiv \bigl|c_2(v) - c_1(u)\bigr|^2 $$

For cubic curves, the function $f$ is then a sixth-degree polynomial in two variables. You can then apply numerical optimization techniques such as the simplex method or conjugate gradient descent. Unfortunately the function can have several local minima (it's not convex), so optimization isn't easy. There may well be more specialized optimization methods available for polynomials, but this isn't an area of expertise for me.


Defocus blur: Computing the pixel plane distance

You don't need that equation.

You know your focus point (point on y-axis with focus_distance as z component) and the ray origin. All that needs to be done then is to compute the ray direction from the origin through that focus point.

For what it's worth, I think a camera space formulation of a thin lens makes it easier to understand the concept:

float fov = 2.0f * atan((sensorWidth / (2.0f * focal_length))); // in radians
float tan_fov = tanf(fov / 2.0f);

// create point on lens, screen space samples are in [-1, 1] domain
vec3 p(screen_space_sample_x * tan_fov, screen_space_sample_y * tan_fov, 1.0);

// calculate direction vector from origin to point on lens
dir = normalize(p - origin);

// get uniformly distributed points on the unit disk
vec2 lens(0.0, 0.0);
concentricDiskSample(random1, random2, &lens);

// scale points in [-1, 1] domain to actual aperture radius
lens *= apertureRadius;

// new origin is these points on the lens
origin.x = lens.x;
origin.y = lens.y;
origin.z = 0.0;

// Compute point on plane of focus, intersection on z axis
float intersection = std::abs(focus_distance / dir.z);
vec3 focusPoint = dir * intersection;
dir = normalize(focusPoint - origin);

// now looking down -Z (optional depending on your coordinate system)
dir.z *= -1.0;


Are there any gltf example datasets available?

Also check out the glTF models used in Cesium:

https://github.com/AnalyticalGraphicsInc/cesium/tree/master/Apps/SampleData/models
https://github.com/AnalyticalGraphicsInc/cesium/tree/master/Specs/Data/Models

Patrick


Instead of "stacking" or overwriting these transformations on one matrix You may try doing that using matrix multiplication:

Matrix4f translate = translate(translation,  new Matrix4F() /*identity*/);
Matrix4f rotateX = rotate((float) Math.toRadians(rx), new Vector3f(1,0,0), new Matrix4F());
Matrix4f scale = scale(new Vector3f(scale,scale,scale), new Matrix4F());

Matrix4f transformation = translate*rotateX*scale;


Multiple swapchains in Vulkan app with ImGui

The Vulkan specification is very clear on this:

  A native window cannot be associated with more than one non-retired swapchain at a time.

You can create a new swapchain for a surface, but only by passing the old swapchain as the oldSwapchain parameter, which will cause the old swapchain to enter the retired state.

So yes, both you and imGUI should get on the same page about who owns the swapchain.


I'll explain the "geometric way" of solving the problem since that's the solution you seem to be after, instead of using rasterization that Dan suggested in the comments.

You can get the list of potential silhouette edges without 2D projection by first iterating through all edges and taking dot product of the normal of the attached two triangles against the eye->edge vector (any point on the edge will do, e.g. one of the end-points). If the dot products for the two triangles on the edge have different signs, then it's a potential silhouette edge. If the object is convex, then you are done and got the list of silhouette edges.

For concave objects it gets more complicated. For concave objects it could be that an edge is only partially a silhouette edge and can be potentially split to N fragments that all belong to the silhouette. So you need to process the list of potential edges and clip them with the geometry. The way you can do the clipping is to extrude every triangle in the geometry away from eye and use this volume to clip the list of potential silhouette edges. The remaining edge fragments that are not clipped away are your silhouette.


Points of a line given in global coordinates to calculate the pixels that are set using the midpoint algorithm without anti-aliasing

When drawing objects that exist in 3D space to a 2D plane (like your monitor or an image), there are a number of spaces that are useful to work in:

Global Coordinates - Somewhere in your 3D world there is an origin, and objects will be placed relative to that origin. The camera will also be placed somewhere relative to that origin and will point in some direction in the world.
Object Coordinates - If you have 3D objects in your scene, they are generally created outside of the world and then transformed in some way to be placed in the world. You might create, say, a chair, with the origin of the chair where the front right leg touches the ground. In Object space, that's (0,0,0). But you'll transform that chair to where it needs to go in the scene and put it at some location in world (global) coordinates.
View Coordinates - Your virtual camera, as mentioned above, is placed somewhere and is pointing at something. Whatever your camera is pointing at will generate a space relative to the camera. You can think of the camera as always being at the origin of camera space.
Clip Space - eventually during rendering, you will use a projection matrix of some sort (perspective, orthographic, etc.) to render the final image. When running your geometry through this matrix, any coordinates that end up outside of [-1..1] will be clipped out. (In some cases [0..1] is used instead, but this detail doesn't change the point.)
Screen Space - finally, after you've clipped everything and gotten it into normalized coordinates, you then have to flatten it to 2D and display it to the user. This is screen space, and is often the same as the pixels (or some portion of them) on the user's physical display (or in an image if your rendering to an image).

So in the original problem, you've been given the requirement of having a field of view for your camera of 135° (3/4π radians), and a window that's 15x15 pixels large. So you need to construct a projection matrix with a 135° field of view and an aspect ratio of 1:1.

The view parameters aren't specified in your problem, so I guess the camera is at the world's origin, so camera/view space and world space are the same for now.

Normally, you'd take the world coordinates, (p1 and p2) and multiply them by the view matrix (identity here, so no work to do), then multiply that result by the projection matrix to get to clip space. Once you have coordinates in the normalized range, you can multiply them by the window's size (15x15 in this case) to get them into screen space and determine which pixels to set.


Due the way floating point values work, you probably don't want to do an exact equality check. You probably want something more like this in your loop:

uint32 sums [ 11 ] = { 0 };
for (int j = 0; j < 1920 * 1080 * 4; j += 4) {
    size_t bucket = static_cast<size_t>(framebuffer [ j ] * 10.0);
    sums [ bucket ]++;
}

Note that there are 11 buckets because the range includes both 0 and 1. If you want only 10 buckets and want to put any pixels that have a value of 1 in them into the top bucket, you could do:

uint32 sums [ 10 ] = { 0 };

and change the bucket index calculation to:

size_t bucket = static_cast<size_t>(framebuffer [ j ] * 10.0);
bucket = std::min(bucket, 9);


Ok, I've found the solution myself.

It goes like this:

// create shape
shape = new THREE.Shape();

// move it to first point of the chord
shape.moveTo(0+(RADIUS-2)*Math.cos(-1.4639537206164), 0+(RADIUS-2)*Math.sin(-1.4639537206164));

// create all the parts afterward
shape.absarc(0, 0, (RADIUS-2), -1.4639537206164, -1.3041396376011, true);
shape.quadraticCurveTo(0, 0, 0+(RADIUS-2)*Math.cos(4.5262534414805), 0+(RADIUS-2)*Math.sin(4.5262534414805));
shape.absarc(0, 0, (RADIUS-2), 4.5262534414805, 4.6513392175639, true);
shape.quadraticCurveTo(0, 0, 0+(RADIUS-2)*Math.cos(-1.4639537206164), 0+(RADIUS-2)*Math.sin(-1.4639537206164));

// create geometry
geometry = new THREE.ShapeGeometry( shape );
// create mesh
mesh = new THREE.Mesh( geometry, new THREE.MeshBasicMaterial( { color: 0x000000, side: THREE.DoubleSide } ) ) ;
// rotate
mesh.rotation.x = 90 * Math.PI/180;
// add to scene
scene.add( mesh );

1) create Shape object
2) move offset to the first point on the arc
3) create arc with the same center and smaller radius as circle (start angle of the first point, end angle second)
4) use quadratic curve with control point (circle center) and the third point
5) use arc the same way as before
6) complete shape with quadratic curve and the last point
7) set geometry, material, add it to scene


How to set equivalent PDFs for cosine-weighted and uniform-sampled hemispheres?

So, for Uniform sampling the PDF is $1/2π$, For Cos-weighted its $cos(θ)/π$. The Lambertian BRDF has a $\pi$ term as well in the denominator for energy conservation.

When not optimizing things you should be dividing by $\pi$ during the BRDF calculation, then dividing by the proper PDFs mentioned above.

Considering all the factors into account, for uniform sampling you'd have

$L_i*brdf*\cos(\theta) * \displaystyle{\frac{1}{pdf}}$

$L_i * \frac{color}{\pi} * \cos(\theta) \displaystyle{\frac{1}{\frac{1}{2\pi}}}$

$L_i * {color} * \cos(\theta) * 2$

Similarly for Cosine-weighted hemispherical sampling you'd have

$L_i*brdf*\cos(\theta) * \displaystyle{\frac{1}{pdf}}$

$L_i * \frac{color}{\pi} * \cos(\theta) \displaystyle{\frac{1}{\frac{\cos(\theta)}{pi}}}$

$L_i * {color} $

Since you weren't dividing by $\pi$ in the BRDF, the division by the PDF should have been $1/2$ and $\cos(\theta)$ as you suggested but couldn't understand since the actual pdfs have a factor of $\pi$ in them.


I just found that if I use pdf = 1.0f/(4.0f * Pi) in unit sphere sampling or pdf = 1.0f/(2.0f * Pi) in unit hemisphere sampling,the result are almost the same as the importance sampling one(I also get the same result when set the baseColor to vec3(0.4f,0.4f,0.4f),the half of vec3(0.8f,0.8f,0.8f).
And the surface area of a unit sphere is just 4.0f * Pi(I forget why I use 2.0f * Pi before).I HAVE to come to two conclusions:
1: the importance sampling one is correct!
2: I'm too stupid!!!!!!


Fastest way to put a point on the circumference of a circle

Before focusing on optimising the individual process of moving a point, consider how many points are being processed this way. This is likely to give a much more significant speed up if it hasn't yet been addressed.

For example, if every point on the screen is being tested against every circle, then most of the points are being checked needlessly.

If instead you divide the screen into a grid, with each grid cell keeping a list of all points currently within it, then each circle only needs to be checked against the points that appear in the grid cells which that circle overlaps. This allows checking a few tens of points, instead of hundreds or thousands.

This is a much bigger speed up than you can hope to achieve by optimising the circumference calculation, and after this you may find that further optimisation is not needed.


Upscaling 720p to 1080p vs upscaling 768p to 1080p

With a good-quality upscaling filter (bicubic, for example), there's no particular importance to the 3:2 ratio. Starting from a higher-resolution source will produce a better-looking final image.

It's true that if you're programming the upscaler from scratch, making one for a fixed ratio like 3:2 is much simpler than making one that handles arbitrary ratios. But image-processing software, monitors, etc probably have arbitrary-ratio upscaling built in anyway.


Does UV-mapping generate for some points of the 3D model several points in the 2D projection?

The way it is usually defined, every vertex in a mesh corresponds to a UV-coordinate, and there must be exactly one such coordinate per vertex, because that is how a renderer would fetch texels. So it is impossible to have multiple UV coordinates for a single vertex unless you have multiple parameterizations for your mesh.

However, meshes often get split up during parameterizations and the same vertices can end up in multiple locations, just like in your example. Most often, those vertices are simply replicated while loading/rendering so the vertex-to-uv mapping remains 1-to-1 or many-to-1. So a renderer will likely interpret your example as the following (v5 is split into 4):


What is the performance tradeoff between forward and deferred rendering?

It is possible to avoid overdraw from opaque objects even with forward rendering by doing a depth pre-pass and using that information to reject any pixel that is not actually visible. However, depending on the vertex cost of your scene, a depth pre-pass may add an unacceptable amount of performance overhead. Additionally, rendering using the pixel shading pipeline of the GPU means that you don't pay a cost per pixel that is rendered, you pay a cost per 2x2 pixel quad that is rendered. So even doing a depth pre-pass still causes triangle edges to waste work shading pixels that will be discarded.

GPU scheduling is a complex topic, and the tradeoff between forward and deferred does not boil down simply to "runs faster but uses more bandwidth." If you have two equally cheap operations that run in sequence and each use the same number of resources, there's no reason to split them into separate shaders: two small wavefronts that each use X resources don't fundamentally work better than a single longer wavefront that also uses X resources. If you have a cheap operation and an expensive operation to run in sequence, though, it may benefit from splitting into separate shaders: the shader in general will reserve the maximum amount of resources that it might use at any point. It's conceivable that forward rendering may not be able to use all the bandwidth of your GPU because there are so few wavefronts in flight that it cannot issue enough operations to saturate the bandwidth. But if you are bandwidth limited, there may be no advantage to deferred rendering (since it will probably use more bandwidth).

An additional performance concern is that forward rendering supports different material types (different BRDFs, say) simply by using a different shader for that object. A straightforward deferred renderer needs to handle different material types in a different way (potentially a branch in the shader), since work is no longer grouped into warps/wavefronts coherently depending on the object being rendered. This can be mitigated with a tiled renderer—if only specific areas of the screen use an alternate material type (say, for hair), then you can use the shader variation with a material type branch only for tiles that contain any pixels with that material.


Gpu derivatives. How it's done across 2x2 boundary?

In Vulkan the shader only looks at each 2x2 and won't attempt to look beyond the neighbourhood: http://vulkan-spec-chunked.ahcox.com/ch15s05.html

$$dPdx_{0,0}=dPdx_{1, 0} = P_{1,0}−P_{0,0}\\
dPdx_{2,0}=dPdx_{3, 0} = P_{3,0}−P_{2,0}$$

For times when a pixel would fall off the geometry the shader is invoked for the values it would have if the triangle extended a pixel further.


How does a 2D Fourier Transform of an image work?

A 2D Fourier transform is performed by first doing a 1D Fourier transform on each row of the image, then taking the result and doing a 1D Fourier transform on each column.  Or vice versa; it doesn't matter.

Just as a 1D Fourier transform allows you to decompose a function into a sum of (1D) sine waves at various frequencies, a 2D Fourier transform decomposes a function as a sum of 2D sine waves.  These waves can have different frequencies along the x and y axes.  They generically have the form:

$$ \exp \bigl(i \cdot (k_x x + k_y y) \bigr) $$

where $k_x$ and $k_y$ are the frequencies along the $x$ and $y$ axes. These two values form a vector called the wave-vector.  In the spatial domain, the wave is oriented along the $(k_x, k_y)$ vector with a frequency along its axis of $\sqrt{k_x^2 + k_y^2}$.

Just as for the 1D Fourier transform, there exist both discrete and continuous versions.  The result of a discrete 2D Fourier transform is a matrix of complex amplitudes for a set of discrete $(k_x, k_y)$ values.  This is commonly visualized (like in the paper you linked to) as an image where the pixel at coordinates $(k_x, k_y)$ represents the amplitude of that wave-vector.

So, an annular shape in a 2D Fourier transform indicates rotational invariance of the distribution of frequencies (i.e. just as much amplitude for waves in every direction), with a narrow range of magnitudes (from the inside of the annulus to the outside).  In other words, the paper is using the Fourier transform to demonstrate that their noise is reasonably isotropic and band-limited.


understanding SDF[Signed Distance Functions] for a torus

This turned into a rabbit hole for me hopefully it helps answer your question, I recommend trying out the various functions on the ShaderToy website. Just drop them in and change the call to "sdCircle". I used the values R=0.75 and r=0.35 to test them out.
Part 1: Defining what an SDF means
Signed Distance Field or SDF, in this context, is a function that returns the signed distance from a point to a primitive. The returned value is always a float that can take on 3 context specific meanings.

zero: This is on the surface, while it is rare to hit exactly on the surface of the primitive we can approximate a hit with a simple inequality like abs(d)<sigma where d is the signed distance and sigma is some small value like 0.001.
negative: In this context negative values represent hits inside the primitive but choosing the meaning of a negative return value is context specific. Smaller (more negative) values represent values that are deeper in the interior of the primitive.
positive: Again this is context specific, but in this case this is a hit outside the primitive and larger values mean it is farther away from the primitive.

Stop for a second and take another look at the first case, "zero". This may seem unimportant at first glance but it turns out that by defining a generic base primitive and then changing the value of sigma we can easily compute a primitive of any size. In the case of a torus we could have a very thin torus or one that is so big it envelopes itself. Then by adding one more variable representing the large Radius of the torus, the torus can now be changed to fit our needs as desired just by controlling two variables.
Part 2: Defining the primitive
Take a look at IQ's 2D function for a circle.
float sdCircle( vec2 p, float r ){ return length(p) - r; }

All this function is doing is computing the "distance" using the length function of our point "p" from our circle (which is sitting at the origin) then subtracting our radius "r". And easy as that any point greater then radius is positive, any point less then radius is negative.
This is almost a torus already, we are just missing the signed distance for the inside diameter of the 2D torus. But we already know that is really just some sigma value from our original circle, so all we have to do is add a second radius and compute the sigma.
So:
float sdTorus2D( vec2 p, float R, float r )
{
  float q = length(p)-R;
  return abs(q)-r;
}

No need to recalculate the distance, so just figure out sigma to our second circle with abs(q)-r.
Part 3: Going 3D
If you stop and think about it for a second (ok...maybe more then a second) then it is easy to realize that our 2D torus is just a slice of a 3D torus. When you think about it like that then we can take another slice of our torus in 2D on its edge just by rotating our torus 90 degrees. This should result in two SDF circles side by side.
But turning the torus on its side is really just a matter of computing our signed distances in the opposite order...lets try:
float sdTorus2DSide( vec2 p, float R, float r )
{
  vec2 q = vec2(abs(p.x)-R,p.y);
  return length(q)-r;
}

The two different views essentially compute the distance in two different ways, so by combining them we should be able to view our torus in 3D from any angle just by adding an extra component "z" so that both computations can be independent of each other. Then combine our two radii into a vec2 and we end up with a function that looks just like IQ's.
float sdTorus( vec3 p, vec2 t )
{
  vec2 q = vec2(length(p.xz)-t.x,p.y);
  return length(q)-t.y;
}

This would still look two dimensional in the "circle" shadertoy, but now we can vary z to look at the torus slices. Or rotate our view point and see it emerge from the 2D world.  Neat stuff!
BTW you can go through this same process with many primitives and get many different shapes as IQ has demonstrated on his website.


SVG opengl implementation - strokes overlap on transparency

You can use the depth buffer to ensure that the second time you get on a point the pixels aren't written.

You start with the maximum depth value and each new path (that you want to add to the existing drawn image) you decrement the depth.


It's pretty easy to create a single frustum that encloses both of the individual eye frusta, as shown in this diagram created by Cass Everitt at Oculus. You just have to place the vertex of this frustum between the eyes and behind them a little bit. You can also increase the near plane distance to match the original near plane of the eye frusta.

Such a frustum includes a little space that wasn't in the union of the original two frusta. There's a small volume horizontally between the two eyes; also, the diagram doesn't show the vertical dimension, but pulling the frustum vertex back makes it include some space above and below the original frusta. However, for practical culling purposes it probably doesn't matter. Frustum culling is usually conservative anyway, and the extra space is only a few centimeters in thickness.

It's also possible to simply cull against the original frustum top and bottom planes (which are shared between eyes) together with the outer side planes. These planes don't form a frustum, technically, but if using "the usual" culling algorithm that tests against one plane at a time, the algorithm will happily work on any set of planes you care to give it.

Sharing frustum culling, LOD selection, etc. between eyes, rather than re-doing it twice every frame, is a great way to optimize the CPU cost of stereo rendering. This is also a prerequisite if you want to use a rendering method that draws both eyes in a single pass, such as instanced stereo rendering.


Ringlike artifacts produced by rays bouncing off a light source

Fixed this today.
The artifacts were produced by shadow acne and worsened by dodgy averages (assuming every ray would complete the maximum number of bounces before returning or exiting the scene) + an undersized PRNG state buffer; maybe I was just really tired, but I still don't know why bouncing rays off my light sources by default seemed like a good idea. The point of backward path tracing is to trace paths from the camera back to the light source(s) in a scene, and there's no reason to continue following a path (=> evaluate bounces) if rays immediately intersect with a source after leaving the camera.


How to compute volume of intersection of non-axis-aligned cuboids in 3D?

You can use CGAL's half_space_intersection_3 or half_space_intersection_with_construction_3 functions, where the half-spaces are defined by the faces of the cuboids.


How to access several textures from huge amount of textures in a shader?

Well, it's unlikely that 10,000 cubemaps could fit into memory at all (at 128x128x4-bytes-per-pixel, 10,000 cubemaps would require upwards of 4GB of RAM). But as far as the mechanism to use a large number of such textures, that's pretty easy. You have two options.

The option that's widely available is to use cubemap array textures. You allocate the storage for the array cubemap with the number of cubemaps you want * 6. Each set of 6 array layer-faces is a single cubemap. In your shader, you just provide an array index for the cubemap layer (not layer-face) that you want.

The other option is to employ bindless textures. You create however many cubemaps you want, then provide them to the shader as an SSBO array of 64-bit integer handles. You index this array as needed and convert the handle into a sampler value before fetching.


How I call glVertexAttribPointer for the GL_ELEMENT_ARRAY_BUFFER buffer?

You don't. The element array buffer is used to store vertex indices, not vertex attribute data and therefore it doesn't have attribute pointers. The count, size, and offset for index data is specified in the draw call, e.g. glDrawElements.


Benefits of Clustered Shading vs Tiled Shading?

Both Unity's "Deferred shading rendering path" and "Legacy Deferred Lighting Rendering Path" work only for opaque surfaces. They both rely on a very similar set of passes:

Render the opaque objects' lighting parameters to a number of render targets. This is referred to as the "G-Buffer pass" or "base pass".
Lighting is then computed in screen-space using the buffers of the first pass to compute lighting. The main benefit is that the cost of lighting is independent of scene complexity. This allows for much more complicated lighting equations with a greater number of lights.
Full-screen pass to resolve all these buffers to a final color.

The reason this doesn't work for transparent objects is that they must be rendered back to front completely shaded (lighting and all). If transparent object A is in front of transparent object B, B must be completely shaded when A blends its color with B. We need that final shaded color because the lighting parameters accumulated in G-Buffers cannot be blended.

So, if you can use it (there are downsides), deferred shading has some very nice properties that produce very good results with good performance. That is why opaque objects will "likely" use deferred shading. Quoting from Unity doc:

  When using deferred shading, there is no limit on the number of lights that can affect a GameObject. All lights are evaluated per-pixel, which means that they all interact correctly with normal maps, etc. Additionally, all lights can have cookies and shadows.

  Deferred shading has the advantage that the processing overhead of lighting is proportional to the number of pixels the light shines on. This is determined by the size of the light volume in the Scene regardless of how many GameObjects it illuminates. Therefore, performance can be improved by keeping lights small. Deferred shading also has highly consistent and predictable behaviour. The effect of each light is computed per-pixel, so there are no lighting computations that break down on large triangles.

I wasn't able to find much information about Unity's clustered rendering. From what I gather it's an optimization technique for forward rendering, but it's still fundamentally the same process. Here's a very nice write-up of how it works in Doom (go to the "Clustered-Forward-Rendering of Opaque Objects" section).


Finding the normals of a tileable 2D surface extracted from 4D space

Since the question was somewhat clarified I will formalize both the question and the answer for future readers.
Having a differentiable scalar field $f : \mathbb{R}^4 \rightarrow \mathbb{R}$ we want to find the gradient of the field with respect to $\theta, \phi$ on the 2-manifold defined parametrically by:
$$(x(\theta,\phi), y(\theta,\phi) z(\theta,\phi), w(\theta,\phi)) = (r_0\cos\theta, r_0\sin\theta, r_1\cos\phi,r_1\sin\phi)$$
Where $r_0,r_1$ are constants. We can compute the partial derivatives as follows:
$$\frac{\partial f}{\partial \theta} = \frac{\partial f}{\partial x}\frac{\partial x}{\partial \theta} + \frac{\partial f}{\partial y}\frac{\partial y}{\partial \theta} + \frac{\partial f}{\partial z}\frac{\partial z}{\partial \theta} + \frac{\partial f}{\partial w}\frac{\partial w}{\partial \theta} \\
= -r_0\frac{\partial f}{\partial x}\sin\theta + r_0\frac{\partial f}{\partial y}\cos\theta$$
$$\frac{\partial f}{\partial \phi} =
\frac{\partial f}{\partial x}\frac{\partial x}{\partial \phi} + \frac{\partial f}{\partial y}\frac{\partial y}{\partial \phi} + \frac{\partial f}{\partial z}\frac{\partial z}{\partial \phi} + \frac{\partial f}{\partial w}\frac{\partial w}{\partial \phi} \\ = -r_1\frac{\partial f}{\partial x}\sin\phi + r_1\frac{\partial f}{\partial y}\cos\phi$$

Assuming that the gradient $\nabla f = (\frac{\partial f}{\partial x},\frac{\partial f}{\partial y},\frac{\partial f}{\partial z},\frac{\partial f}{\partial w})$ is known, it is straightforward to compute the 2d gradient.

Note that another possibility is to compute the gradient numerically (in case you don't have the partial derivatives of $f$):
$$ \frac{\partial f}{\partial \theta} \approx \frac{f(r_0\cos(\theta+h_{\theta}), r_0\sin(\theta+h_{\theta}), r_1\cos\phi,r_1\sin\phi) - f(r_0\cos\theta, r_0\sin\theta, r_1\cos\phi,r_1\sin\phi)}{h_{\theta}}$$
$$ \frac{\partial f}{\partial \phi} \approx \frac{f(r_0\cos\theta, r_0\sin\theta, r_1\cos(\phi+h_{\phi}),r_1\sin(\phi+h_{\phi})) - f(r_0\cos\theta, r_0\sin\theta, r_1\cos\phi,r_1\sin\phi)}{h_{\phi}}$$
For $h_{\theta}, h_{\phi}$ chosen adequately (note that the choice of these parameters is a nontrivial problem).


Define position and orientation in a local coordinate system

Assuming that your 'models' can be represented by collections of points, then these points' coordinates are defined with respect to some coordinate system. You said that $m_1$ is defined wrt the coordinate system $W$. And you have some other coordinate system $L$ and want to find the positions of the points of $m_1$ in it. To do this you need to multiply each of your points with the matrix used to go from $W$ to $L$. To build this matrix you need to know the coordinates of the basis vectors of $W$ wrt $L$. Then you can make a 3x3 matrix where each column is a basis vector with which you can go from $W$ to $L$. Note that a translation will also be necessary if the origins do not agree.
The solution for $m_2$ is similar.


OpenCL isn't really a drawing API, it's designed for doing compute workloads across different devices, not necessarily GPUs. You can certainly write a kernel that renders to a texture in OpenCL, and have this texture be available to Open GL or D3D. AFAIK, though, you wouldn't have access to the fixed-function hardware like rasterization, attribute interpolation, and fragment blending, so you'd have to implement it yourself (unless you're writing a ray tracer). You'd also likely have to use one of the drawing APIs to present your texture to the screen, by rasterizing a full screen quad and sampling your texture at each pixel. This carries a synchronization penalty - although OpenGL and D3D can interoperate with OpenCL, they can't access the same memory at the same time, so you need to wait for control to transfer from one API to the other.

Using compute on an already rendered buffer for screenspace effects is a good idea, since you save the fillrate of rasterizing a full screen quad every time you apply a new effect compared to the fragment shader approach. These days though, OpenGL and D3D have compute shaders built-in to handle this kind of use case. Why do you want to use OpenCL exactly? Seems to me you'd be better off using one of the rendering APIs.