How are textures projected onto 3d models in texture painting applications

I don't know if there is a smarter/more efficient way to do it, but a possible approach is the following:
Step 1 - Projection volume
First, you calculate the "projection volume" where its near plane corners are identical to the texture corners on your screen. You can do this by transforming the screen coordinates of the textures' corners into the range [-1, 1]. Then you use those coordinates to create a quad in NDC by combining the transformed x-y coordinates with z=-1 (near plane) and z=1 (far plane).
Step 2 - Collision test
Now you need to have both, your model and your "projection volume", in the same space. You can either transform the "projection volume" into your model space or your model into NDC. I think the second choice makes the next step a little bit easier.
Now you have to find all triangles that are fully or partially inside the "projection volume". This is basically a collision detection check (triangle vs. quad or frustum) and I won't go into the details here. For all triangles that are fully inside the volume, you now have a connection between each vertex and its position inside the texture you want to project. Simply transform the vertex position into screen space and from there to the textures pixel coordinates. The second transformation depends on where you positioned your texture on the screen and how you have scaled it.
For all partially covered triangles, the basic procedure is the same, but you have to introduce extra "helper vertices" and to transform them too to separate the covered from the uncovered areas. Doesn't mean that you have to really split your triangles into multiple smaller ones, but at least you need the extra vertices as intermediate data for the final step.
Step 3 - Adjust model data
In the last step, you have to bake this information somehow into your model. This totally depends on your model's data structure and layout. You can for example just split partially covered triangles into new ones and then set the texCoord of every vertex that is inside the "projection volume" accordingly.
If you have some kind of model-specific  UV-mapped textures, you would now use the information of the previous step to interpolate and copy the data of your projected texture to your model texture. For each vertex, you know where it is located inside the projected texture and inside its models UV texture. Now you basically cut out each triangle from your projected texture, and transform it into the corresponding triangle in your UV texture. Notice that this is not trivial since the triangle usually gets distorted and scaled. However, you can use the graphics pipeline for that by rendering directly to your UV-texture.
There are probably a lot of other data representations that might require a different approach for this final step. However, the key here is the second step which establishes the connection between your vertices and the texture. Once you have this, it shouldn't be too hard to come up with a method to add the projected texture to your model.
Further information
During the second step, you might also need to perform some kind of depth testing, so that only the closest area gets affected and not all sides that are inside the projection volume.


How to calculate curvature of isoline of image

Gradient magnitude and angle can be found using a Sobel operator. It's more sophisticated than the [1, 0, -1]. This is commonly used either directly for edge detection, or as part of a more complex algorithm.

Or, if I've misunderstood, and you want only the curvature of the 2D isoline, this answer on Math Overflow might be of use.


Colorblending shader recompilation scenarios

In GL's model of the GPU, the colour blending step is performed by special-purpose hardware. This idea dates back to the fixed-function origins of GL, before programmable shaders were even a thing.

But not all GPUs today implement all colour blend options with special-purpose hardware. The hardware vendor can reduce gate count and power use by getting rid of some of the less-used blend modes (or even by getting rid of the blender hardware completely and doing everything in shaders). Instead, the GL driver has to add some code to the shader to do the blending. This code is added transparently to the driver, and isn't visible to the application, because GL doesn't have a way to make this visible or to control it from the application.

Now, the shader object (the compiled shader) doesn't just depend on the shader source code, but also on the blend mode in effect when it was compiled. If the application subsequently changes the blend mode, the driver needs to work out which shader objects are affected, retrieve the stored source of those shader objects, add the code for the new blend mode, and recompile the shader. Again, it has to do this behind the application's back, because GL doesn't have a way to control it: GL assumes the shader object is independent of the blend mode.

Normally, the application would compile shaders during a loading screen, or in a thread, but because the shader compilation isn't triggered by the application, it can't control when it happens. It just happens by magic when the application changed something that should be unrelated (the blend mode), which might be in the middle of a frame, so it causes hitching. And of course because it's hidden, the app can't simply ask to go back to the old state: changing the blend mode again might trigger another shader compilation (though it might come from a cache in the shader compiler).

Blend mode isn't the only parameter that causes this. Some unusual texture modes are also supported by adding code to the shader. (YUV textures commonly work this way, on implementations that support them.) Exactly which parameters cause a problem depends on the implementation: it's different for different hardware architectures. I think it's typically a bigger problem on mobile, because the GPU vendors are more sensitive to gate count and power use and more keen to replace special-purpose hardware with shader code.

The reason Vulkan makes this less of a problem is that it creates a new kind of state, which depends on the blend mode, the shaders, and other parameters. Changing any of these requires the app to perform an explicit "prepare" operation, which might recompile shaders. This gives the app control over when it happens, including the ability to prepare all the combinations of parameters up-front in a loading screen or in a background thread. It means that shader compilation never happens without the app asking for it explicitly. And it means the app can just go back to the old state by switching to it, instead of changing the blend mode and waiting for another shader recompilation.


That looks like multiplying the front image by the luminance of the shirt behind it.

There are multiple definitions of luminance but for a start, I think you could assume your data is sRGB and follow the transformation to CIE to get the CIE Y component.


Getting the color palette from an SVG folder

You can convert SVGs to PNGs using Inkscape
inkscape in.svg --export-type=png --export-filename=out.png
You can batch convert using
inkscape *.svg --export-type=png


Why isn't PNG preferred over JPG for being lossless art quality yet?

I don't know if this is the real reason, but here's one possibility: because it's unnecessary.

It's easy to say "JPEG = lossy = bad", but JPEG doesn't just destroy image quality willy-nilly. What the format gives you is a slider with "small, low-quality" at one end and "big, high-quality" at the other. You can easily reduce the byte size of a PNG image by encoding it as JPEG, and if you use the high-quality end of the slider, you needn't lose any visual appeal. Although normal JPEG is lossy in the information-theoretic sense, a high-quality JPEG appears identical by the naked eye.

Many image hosting sites will replace images by lower-quality JPEG files not because of the storage cost but because of the bandwidth cost. If you're serving millions of views of these files every day, the cost of sending those extra bytes is pretty big, even though the cost of storing them is tiny. And those bytes don't just cost money: they also cost load time for your users on mobile networks. These sites will have targets for how long their pages take to load on slow connections, and reducing the quality slider ever so slightly makes a big total difference.

In summary, JPEG encoding can be lossless or lossy, and it can be as high- or low-quality as you want. Using PNG everywhere instead of JPEG would be costly for image sites and would make you wait longer for your art files to download. Choosing a high-quality JPEG file can give you something that looks just the same as the PNG without being as "heavy".


Hiding Boundaries with the Eye close to a set of adjacent textured quads

I figured out the mistake that I was making and confirmed the hypothesis in my comment above. I was rendering each quad with its distinct set of vertex normals calculated from the local parametrisation. With lighting enabled, the vertices were therefore shaded differently for adjacent quads owing to the different dot products between the local normal and the light direction vector.

What I did was average out the normals to create a unique normal at each vertex. Now when I fly into the clouds, I do not see the quad boundaries.

As Dan suggested, I could avoid the problem altogether by disabling lighting, but it would eliminate the ability to simulate different times of the day and in any case, the fix above worked and only involved about 30 lines of code!

Problem solved.

In case anyone else finds this useful, here's the code (I wrote it in VBA) to calculate averaged vertex normals given a set of vertices on a surface that is topologically rectangular.

Sub Calculate_Averaged_Vertex_Normals(Vertx#(), M%, N%, VnormX#())
' INPUT:
'L,M: Array Dimensions
' Vertx - 3D array of vertex coordinates
' OUTPUT: VnormX: 3D array of Averaged Vertex Normals

 Dim i%, j%, k%, p%, q%, w%, NormX#(3), Ncount%(), Vx#(), Fn#(3),Gn#(3),Hn#(3),A#
 ReDim Ncount%(0 To L, NF + 1), Vx(0 To L, NF + 1, 4, 3)
 For k = 0 To M - 1
  For i = 0 To N - 1
   Call Quad_Normal_Vector_Parametrisation(Vertx,k, i,Fn,Gn,Hn) ' Local Normals at each Vertex

     p = k: q = i: Ncount(p, q) = Ncount(p, q) + 1: j = Ncount(p, q)
     Call Quad_Vertex_Normal(0, 0, NormX,Fn,Gn,Hn)
     For w = 1 To 3: Vx(p, q, j, w) = NormX(w): Next w

     p = k: q = i + 1: Ncount(p, q) = Ncount(p, q) + 1: j = Ncount(p, q)
     Call Quad_Vertex_Normal(1, 0, NormX,Fn,Gn,Hn)
     For w = 1 To 3: Vx(p, q, j, w) = NormX(w): Next w

     p = k + 1: q = i + 1: Ncount(p, q) = Ncount(p, q) + 1: j = Ncount(p, q)
     Call Quad_Vertex_Normal(1, 1, NormX,Fn,Gn,Hn)
     For w = 1 To 3: Vx(p, q, j, w) = NormX(w): Next w

     p = k + 1: q = i: Ncount(p, q) = Ncount(p, q) + 1: j = Ncount(p, q)
     Call Quad_Vertex_Normal(0, 1, NormX,Fn,Gn,Hn)
     For w = 1 To 3: Vx(p, q, j, w) = NormX(w): Next w
  Next i
 Next k

 For k = 0 To M
  For i = 0 To N
   For w = 1 To 3
    A = 0#
    For j = 1 To Ncount(k, i)
     A = A + Vx(k, i, j, w)
    Next j
    VnormX(k, i, w) = A / Ncount(k, i)          ' Averaged Vertex Normal
   Next w
  Next i
 Next k
End Sub

Sub Quad_Normal_Vector_Parametrisation(Vertx#(),k%, i%,Fn#(),Gn#(),Hn#())
 Dim j%, Bv#(3), CV#(3), Dv#(3)
 For j = 1 To 3
  Bv(j) = Vertx(k, i + 1, j) - Vertx(k, i, j)           ' Tangent Vectors from Bilinear Parametrisation
  CV(j) = Vertx(k + 1, i, j) - Vertx(k, i, j)
  Dv(j) = Vertx(k + 1, i + 1, j) - Vertx(k, i + 1, j) - Vertx(k + 1, i, j) + Vertx(k, i, j)
 Next j
 Call CrossProd(Bv, CV, Fn)             ' Normal Vector from Cross Product of Tangent Vectors.
 Call CrossProd(Bv, Dv, Gn)
 Call CrossProd(Dv, CV, Hn)
End Sub

Sub Quad_Vertex_Normal(x#, y#, NormX#(),Fn#(),Gn#(),Hn#())
 Dim j%, Norm#
 Norm = 0#
 For j = 1 To 3
  NormX(j) = Fn(j) + Gn(j) * x + Hn(j) * y
  Norm = Norm + NormX(j) * NormX(j)
 Next j
 Norm = Sqr(Norm)
 For j = 1 To 3
  NormX(j) = NormX(j) / Norm
 Next j
End Sub

Sub CrossProd(x#(), y#(), Z#())
 Z(1) = x(2) * y(3) - y(2) * x(3)
 Z(2) = y(1) * x(3) - x(1) * y(3)
 Z(3) = x(1) * y(2) - y(1) * x(2)
End Sub


Making low discrepancy sequence noise textures, not LDS sample positions

In theory, it is possible to stuff every sample distribution into a texture to "pre-bake" it for fast access. The question is whether any of the results might be useful. For blue noise, this makes a lot of sense, as blue noise distributions have global influence and are hard to evaluate at runtime. Uniform random sampling, on the other hand, is so simple precisely because it is not global, but has no influence whatsoever from neighboring samples. So the best you can achieve, as you noticed yourself, is writing uncorrelated white noise to the texture, which gains you very little.

I think the key property of the blue noise textures that you failed to mention is that they are progressive. If you take, for example, Blue Noise Masks, Forced Random Dithering, or Relaxation Dart Throwing to generate a matrix/texture, you will not get grayscale values, but unique, ascending integer values corresponding to the order of insertion of the samples/texels. This means that you can also re-arrange such textures to a 1D list of width * height elements, where each element is a 2D position within the matrix. I.e., the first element of this list will be the (x, y) of the first sample inserted into the texture upon creation. The second element will be the (x2, y2) of the second sample, which will be quite far away from the first one, because the blue noise texture algorithms try to maximize distances between similar values.

The list representation is nothing but a sequence accomplishing what 2D low-discrepancy sequences do: giving you a 2D sample position (albeit discrete and in a grid) for a scalar index. You do not lose any information while converting. So applying the reverse conversion to low-discrepancy sequences, it should be possible to write their element indices into a texture for fast lookup. What I don't know is how fast this will degenerate, because sooner or later higher values will end up at the same discretized locations as previously written ones. Low-discrepancy sequences guarantee spatial uniformity on average, but if you want to fill an n² texture and create exactly n² samples, I bet that you will get some of these overlaps while also having some holes. These conflicts can, by design, not occur in blue noise textures, but here you would have to find a strategy. Just discarding the higher values? I don't know how this influences the distribution properties. I assume that results vary between different low-discrepancy sequences. I have only ever worked with Halton and Hammersley, which don't seem like the best fit in this case. Maybe something based on a grid/lattice would help you to reduce/eliminate the overlaps. Not my field of expertise, unfortunately.

This is the key problem, that low-discrepancy sequences (at least the ones I know) and in general sampling distributions used in computer graphics output nD values in the continuous domain rather than in a discrete grid. The reason blue noise textures work so well is because the discretization is already part of the generation. What you want is a permutation of n² integer values such that the result has special properties. Maximizing distances between similar values would be blue noise. Having no correlation between values would be white noise. And some other magic property could be pink noise, green noise, or a perfect interleaved gradient noise as in your provided picture. Would this be a reasonable approach, to understand your problem as the problem of permuting given values? Instead of continuous sampling? Maybe it simplifies the solution.

Another point to consider for dithering and related effects is that blue noise textures can usually be tiled seemlessly because the sample domain is taken to be toroidal. You will lose this feature unless you find a way to make your low-discrepancy sequence toroidal, which I think is pretty tough, but would be a great scientific contribution!


Is there any way to generate primitives in a geometry shader without any input geometry?

No, there is not really a way to do that.

A geometry shader invocation requires an input primitive and generates 0 or more output primitives. Without an input primitive, there is not really a way to actually invoke the geometry shader. Of course you can stretch the limits of the geometry shader's maximum number of output primitives for each input primitive (don't know the practical limits right now, should be on the order of thousands maybe). So you maybe could generate 1024 triangles for each point, but you always have to have some input primitives.

What you don't need however is an actual notion of geometry. You don't really have to render 3D points at any reasonable location, they could as well have just some abstract index or texture coordinates or whatever as attributes and not necessarily a meaningful 3D position. Noone dictates what attributes your vertices have. And you can even render vertices without any attributes. But you have to render some primitives to invoke the geometry shader on them, even if those primitives don't have any actual attributes (how you then compute your output geometry in the geometry shader is a different question, though).

But what you actually did, rendering a point for each grid cell and generating the marching cubes triangles for that cell from it, is exactly the straight-forward approach. Of course what attributes this cell contains is up to you, it could be a 3D position, a texcoord into a 3D texture, whatever, but those are the grid cells you render. Purely semantically spoken, you don't actually "discard" those points and then "replace" them by triangles, you "convert" each point into a set of triangles. That's exactly what the geometry shader is for and there's nothing "hacky" or "improper" about it. Noone says the geometry shader has to generate the same output primitive type as was input to be "proper".

What you can do to achieve a largely input-less way of rendering your voxel grid (and this might be what you were actually asking for) would be to just draw a set of attributeless points. This means you don't need any attribute arrays at all and just disable them all and invoke a simple glDrawArrays with the number of cells you need. Then in the vertex shader or the geometry shader you can generate the necessary 3D grid cell index with a little index magic from the input vertex ID (i.e. gl_VertexID, which is the only information you have) and then compute your marching cubes geometry from a lookup into the 3D volume texture (or whatever datastructure).

So, in retrospect I should relativate my statement from the beginning: You can't generate primitives without any input primitives, but you can generate them without any input geometry.


Writing to a compressed texture using a compute shader, with no extra copies

After looking into this for a while, I found out a couple of things:

You cannot avoid a memcpy: You cannot write directly into texture storage allocated for a compressed texture using only OpenGL API calls. This means that you cannot avoid the call to glCompressedTexImage2D with a bound PBO. That being said, you may be able to use a 16-bit RGBA texture and a GLSL image type in your compute shader.
You do need to synchronize memory: In order to make sure that your compute shader finishes writing to your storage buffer, you must make sure that all reads and writes to it finish. This is done by calling glMemoryBarrier with GL_SHADER_STORAGE_BARRIER_BIT.

The full code for something that writes into a buffer to be used as a compressed texture looks like this:

GLuint buffer;
glGenBuffers(1, &buffer);
glBindBuffer(GL_SHADER_STORAGE_BUFFER, buffer);
glBufferStorage(GL_SHADER_STORAGE_BUFFER, tex_size_in_bytes, 0, 0);

glUseProgram(compute_shader_prog);
glBindBufferBase(GL_SHADER_STORAGE_BUFFER, compute_shader_bind_point, buffer);
glDispatchCompute(wg_x, wg_y, wg_z);
glMemoryBarrier(GL_SHADER_STORAGE_BUFFER_BIT);

glBindBuffer(GL_PIXEL_UNPACK_BUFFER, buffer);
glCompressedTexImage2D(GL_TEXTURE_2D, 0, fmt, w, h, 0, tex_size_in_bytes, 0);


How are volumetric effects handled in raytracing?

Overview

The appearance of volumes (also called participating media) in nature is caused by tiny particles, such as dust, water droplets or plankton, that are suspended in the surrounding fluid, such as air or water. These particles are solid objects, and light refracts or reflects off of these objects as it would on a normal surface. In theory, participating media could therefore be handled by a traditional ray tracer with only surface intersections.

Statistical Model

Of course, the sheer number of these particles makes it infeasible to actually raytrace them individually. Instead, they are approximated with a statistical model: Because the particles are very small, and the distance between the particles is much larger than the particle size, individual interactions of light with the particles can be modeled as statistically independent. Therefore, it is a reasonable approximation to replace the individual particles with continuous quantities that describe the "average" light-particle interaction at that certain region in space.

For physically based volumetric light transport, we replace the inconceivably many particles with a continuous participating medium that has two properties: The absorption coefficient, and the scattering coefficient. These coefficients are very convenient for ray tracing, as they allow us to compute the probability of a ray interacting with the medium - that is, the probability of striking one of the particles - as a function of distance.

The absorption coefficient is denoted $\sigma_a$. Say a ray of light wants to travel $t$ meters inside the participating medium; the probability of making it through unabsorbed -- i.e. not hitting one of the particles and being absorbed by it -- is $e^{-t\cdot\sigma_a}$`. As t increases, we can see that this probability goes to zero, that is, the longer we travel through the medium, the more likely it is to hit something and be absorbed. Very similar things hold for the scattering coefficient $\sigma_s$: The probability of the ray not hitting a particle and being scattered is $e^{-t\cdot\sigma_s}$; that is, the longer we travel through a medium, the more likely it is that we hit a particle and are scattered into a different direction.

Usually, these two quantities are folded into a single extinction coefficient, $\sigma_t = \sigma_a + \sigma_s$. The probability of traveling $t$ meters through a medium without interacting with it (neither being absorbed or scattered) is then $e^{-t\cdot\sigma_t}$. On the other hand, the probability of interacting with a medium after $t$ meters is $1 - e^{-t\cdot\sigma_t}$.

Rendering with Participating Media

The way this is used in physically based renderers is as follows: When a ray enters a medium, we probabilistically stop it inside the medium and have it interact with a particle. Importance sampling the interaction probability$1 - e^{-t\cdot\sigma_t}$ yields a distance $t$; this tells us that the ray traveled $t$ meters in the medium before striking a particle, and now one of two things happens: Either the ray gets absorbed by the particle (with probability $\frac{\sigma_a}{\sigma_t}$), or it gets scattered (with probability $\frac{\sigma_s}{\sigma_t}$).

How the ray is scattered is described by the phase function and depends on the nature of the particles; the Rayleigh phase function describes scattering from spherical particles smaller than the wavelength of light (e.g. our atmosphere); the Mie phase function describes scattering from spherical particles of similar size than the wavelength (e.g. water droplets); in graphics, usually the Henyey-Greenstein phase function is used, originally applied to scattering from interstellar dust.

Now, in graphics, we don't normally render pictures of an infinite medium, but render media inside a scene consisting of hard surfaces as well. In that case, we first fully trace the ray until it hits the next surface, completely ignoring the participating medium; this gives us the distance to the next surface, $t_{Max}$. We then sample an interaction distance $t$ in the medium as described before. If $t \lt t_{Max}$, the ray hit a particle on the way to the next surface and we either absorb it or scatter it. If $t \geq t_{Max}$, the ray made it through unscathed and interacts with the surface as usual.

Outlook

This post was only a small introduction to rendering with participating media; among other things, I completely ignored spatially varying coefficients (which you need for clouds, smoke, etc.). Steve Marschner's notes are a good resource, if you're interested. In general, participating media are very difficult to render efficiently, and you can go a lot more sophisticated than what I described here; there's Volumetric Photon Mapping, Photon Beams, Diffusion approximations, Joint Importance Sampling and more. There's also interesting work on granular media that describes what to do when the statistical model breaks down, i.e. particle interactions are no longer statistically independent.


Vector graphics output from shader program

In OpenGL, there are 6 different shader types:

Vertex Shader
Tesselation Control Shader
Tesselation Evaluation Shader
Geometry Shader
Fragment Shader (Pixel)
Compute Shader

The first 5 are part of the render pipeline and fulfill a very specific purpose in that pipeline. They are executed in ascending order. The shader types 2.-4. are optional and can be skipped. Between those stages, some operations are performed, that are very specific to rendering. Have a look at this link for further information.

The problem here is: Since the rendering pipeline has a very specific purpose (generating an image of pixels), it is very hard to use it and its shaders for something else than rendering. Won't say it is impossible, but it gets certainly harder and harder the less your goal has something to do with pixel generation.

For general-purpose computations, you can use compute shaders. You can basically compute anything there, so this is probably the shader type you are looking for. However, the way you ask your questions gives me the feeling, that you have some misunderstandings on how rendering works in general and you should probably read some literature about how the render pipeline creates an image from 3d data.

  But let’s say you had a shader that created a silhouette (outline) around some geometry. Is it possible for a shader program to generate vector graphics (ie. Curves) that follow that exact silhouette (instead of pixel data)?

As written above, generally yes. It is possible to write a shader that does that. But probably not with the same technique as the outline is created in the "pixel case". It is certainly much more complicated.

  Or in a different example, this image was created by using a pixel shader. Would it be possible for a shader to generate the lines on a teapot as actual curve geometry rather than just a bitmap that is mapped to the 3D geometry?

Again, yes it is possible for a shader to fulfill this task, but you wouldn't use a pixel (fragment) shader for that. You would also need to approach the whole task very differently.

  Are shader programs fundamentally different and as such could not generate lines, curves, etc. as a form of output?

The shaders of the rendering pipeline are tied to a very specific task and have very specific boundaries on their inputs and outputs. But if you know what you are doing, you can probably hack around those limitations. However, I think it wouldn't make much sense to do so, except you want to prove that you can.
Instead, you can use a compute shader for any computation, that has nothing to do with rendering.

To sum it up

You can use compute shaders to get other output types than just pixels and images made of pixels. However, calculating basic 2d shapes/curves/vector graphics from 3d data will most likly be fundamentally different than generating a corresponding bunch of pixels from that same data. In conclusion, you can't just turn a switch (change 2-3 lines of code) to select which output you like to have.


How to achieve gimbal lock with Euler angles?

Rotations in 3D are normally done with matrices. The xyz Euler angles can be converted to matrices so that it can be used in the rotation.
That is where something called rotation order comes in.
Basically it says in what order you rotate the object.
First you rotate the object around the x axis, then the y axis and lastly the z axis for example.

This means that when you rotate the object around the x axis first, you change the way the y axis and z axis rotate.
Of course this changes depending on the rotation order.

This GIF shows how changing one axis value can change the others, but it doesn't work that way for everything.

Here you can also see at the end that I add in that gimbal lock.
This is where I lose one axis of rotation.

This could be fixed by changing the rotation order to an order, where it doesn't break the way it should work.
It could also be fixed by not using Euler angles but Euler angles are handy and simple in 3D animation.

In 3D animation it could lead to weird rotations since the values of the rotations have to be changed a lot to get a certain rotation that you want.
If you would look at the xyz Euler angle values in a graph, you would see spikes because of that.
This could lead to weird interpolation of the keyframes and a weird end result.

If you would like to read more about the math behind it, I would recommend reading about rotation matrices in 3D.
https://en.wikipedia.org/wiki/Rotation_matrix#Basic_rotations

A little extra explanation for the animation:
Rotation in this case is around axis (shown as circles) and by rotating on one axis, you rotate the other axis. The consequence is that with a normal rotation sphere (the sphere in the animation, but then without the axis being rotated) in the animation, you don't see these axis being rotated. If you get a gimbal lock, one axis is the same as another one, but with a normal rotation sphere it still tries to rotate as if that axis wasn't the same as another one, this means that it uses weird values to get to that rotation and this can make an animation look weird.

I hope this helps!


Replacing Vertex Attributes (glBufferData vs glBufferSubData)

There is a difference between "can" and "should".

You "should" not use glBufferData to change the size of an existing non-immutable buffer. You can still do so, but don't expect this to be advantageous.

You "can" not change the size of an immutable buffer with any function.

  How do I replace the whole VBO with my new data, but if my new data is smaller than the old data then how do I reallocated the VBO efficiently?

Why would you want to? If the size of your data got smaller, then it is entirely possible that a few frames from now it will be bigger again. Perhaps bigger than it was before. That extra storage isn't waste; it is extra storage you're keeping around because you intend to use it at some point in the future.

The cost of allocating memory is high; you should avoid it whenever possible. If you need to increase the size of a buffer, then the implementation must allocate new storage for it. You want to avoid that whenever possible.

What you ought to be doing is figuring out what a reasonable maximum size would be for this data, and then allocate a buffer of that size.

  Or is there something equivalent to a NULL terminating character where the graphics card can know not to read any further data?

That's up to you. You tell OpenGL exactly how many vertices to read from a vertex buffer. This is determined based on:

The vertex format for each attribute.
The offset within the buffer for each attribute.
The number of vertices you send in the glDraw* command.

OpenGL doesn't arbitrarily process data; it reads the data you tell it to read. If you want it to not read past a certain point, stop telling it to read past that point ;)


Dynamically allocating morph targets in shaders

With modern hardware, morph targets probably should not be VS inputs. They should be SSBOs or the D3D equivalent. The vertex shader should fetch from these arrays of positions and act on them appropriately. You can pass the number of targets as a uniform or something.


WebGL Can't render anything, wrong projection?

Try This... I guess you got the matrix wrong.

matrix.n11 = -(2 * near) / (r - l);
matrix.n12 = 0;
matrix.n13 = 0;
matrix.n14 = -near*(r+l)/(r-l);
matrix.n21 = 0;
matrix.n22 = (2 * near) / (t - b);
matrix.n23 = 0;
matrix.n24 = -near*(t+b)/(t-b);
matrix.n31 = 0;
matrix.n32 = 0;
matrix.n33 = (near + far) / (near - far);
matrix.n34 = (2 * near * far) / (near - far);
matrix.n41 = 0;
matrix.n42 = 0;
matrix.n43 = -1;
matrix.n44 = 0;

I referred the following site after I googled =>Link


Determining the function of a radial gradient fill with midpoint

It seems to be just a scalar Bezier function to me, where the second coefficient is determined by $a$
$$(1-x)^2 + a 2 (1-x) x,$$
here $a \in [0, 1]$ is a normalized percentage. This gets you pretty close to the function that is graphed, but the function will not be as flat for $a = 0$ and $a = 1$. I suspect the function is of a higher degree:
$$(1-x)^3 + a 3(1-x)^2 x + a 3(1-x) x^2$$
or even
$$(1-x)^4 + a 4(1-x)^3 x + a 6(1-x)^2 x^2 + a 4 (1-x) x^3$$
With increasing degree comes increased flatness (dragging out the colour of the midpoint). However, these higher order function are not exactly straight at $a = 0.5$. Using the coefficients $\frac{2}{3}$ $\frac{1}{3}$ in the cubic equation will get you a straight line for the cubic version
$$(1-x)^3 + \frac{2}{3} 3(1-x)^2 x + \frac{1}{3} 3(1-x) x^2.$$
For the cubic version you can create two additional functions
$$q(a) = 0.975 \cdot 2 (1-x) x + x^2$$
$$r(a) = 0.025 \cdot 2 (1-x) x + x^2$$
and then replace the second and fourth coefficents in the quartic function with these
$$(1-x)^4 + q(a) 4(1-x)^3 x + a 6(1-x)^2 x^2 + r(a) 4 (1-x) x^3$$
this will preserve flatness at extremes and (approximate) linearity at $a=0.5$. You can view it here: https://www.desmos.com/calculator/hwj7jyozv9


Are there LOD Algorithms that work through removing objects?

Here's how I'd do it:

Assign an sequential id to each tree. This can be an implicit id like an index in an array if you have this structure. The important part is that you can create N stable groups of trees that will be removed based on distance.

Then decide at which distance you want to start removing each group of trees. Each tree group will have a specific distance at which it gets removed from the rendering. You must choose the appropriate granularity based on your use-case.

Then, for each tree, compute its distance from the viewpoint and compare that with its removal distance to decide if it should be rendered or not. To get a nicer effect, use a fade-out distance over which the trees progressively disappear.

Essentially the process is to tag trees with some LOD parameters (removal distance, fade distance, etc.) and remove them based on distance from the viewpoint.


The most straightforward way would be to put x into the red channel, y into the green channel, and z into the blue channel of each pixel.

You could also make a 1 channel image where intensity is used to describe height. This is called a height map. Usually the x and y of the mesh are the same as the x and y of the image, so no need to store them in any way.


What's the justification of multiplying colors together?

When you multiply colors together the surface color acts as a filter. Remember that colors are scaled from 0 to 1.

That way with white incoming light a green object will remain looking green.

This does bring a few issues because physical light isn't as orthogonal (where the 3 primary colors are actually overlapping frequency response functions) but for the vast majority of cases you can tweak the values enough for it to stay looking realistic.


Do we have OpenGL shaders write out the intermediate *.o-esque file on first compile so that we can skip compiling them in run-time every single time?

OpenGL 4.1's glGetProgramBinary and glProgramBinary exist for this purpose. The first one retrieves a binary that represents the compiled program in its entirety (though none of the state stored within it. It, and its implementation-defined format, can be stored and reloaded via glProgramBinary.

Of course, there are limitations. Implementations are not required to be able to load the format they gave you. That is, a driver change could cause implementations to switch to a different versions, leaving all of your binaries obsolete and un-loadable. So you have to be ready to recompile from the source.

Alternatively, GL 4.6 allows you to employ SPIR-V, an intermediate shader language. The idea here is that you compile your code to SPIR-V, then load that SPIR-V into a shader, where you can move forward from there. There are no version changes that could break your code.

Of course, SPIR-V is an intermediate language, so it still has to be compiled to some degree, along with all of the error checking and the like needed to make the final GPU code. It won't be faster than program binaries, but it's a bit more reliable (for implementations that support it).


Is it possible to send Texture Buffer to shader dynamically?

These:

glTexBuffer(GL_TEXTURE_BUFFER, GL_R32I, this->index_tbo);

uniform samplerBuffer index_tex;

Do not match. If the texture is a signed integer texture, then the sampler type needs to match. It should be isamplerBuffer, and you won't need to do the cast.


Proper way to handle diffuse + refraction + reflection rays in path tracing?

Let us take one step back.
When you do path tracing, you are doing a Monte Carlo integration.
What does this mean? You try to solve $\int f(x)\mathrm{d}x$ by sampling.
The Monte Carlo integration says for a sufficiently big $n$: $\frac{1}{n}\sum_{i=0}^n f(x_i) \rightarrow \int f(x)p(x) \mathrm{d}x$

If we instead sum $\frac{f(x_i)}{p(x_i)}$ we get $\frac{1}{n}\sum_{i=0}^n \frac{f(x_i)}{p(x_i)} \rightarrow \int f(x) \mathrm{d}x$.
Now your $f(x)$ is the rendering equation and you got yourself a way to approximate the rendering equation.

Now that we are on the same page about the basics, let's consider what you do.
You generate a path of light, that will become one of the samples $\frac{f(x_i)}{p(x_i)}$. In order to do this you model the interaction of the light with the surface at each step along the path. And you always update your current path probability $p(x_i)$.

What is now the correct way to handle a material that has multiple different ways that a ray could travel? There is no one correct way. The only important part is, that your values $n$, $f(x_i)$ and $p(x_i)$ stay correct.

When you diverge the path and spawn two rays, you have to increase the number $n$ of total samples accordingly because now you have done two samples. Then handle both pathes as if they were separate from the beginning.

If you want to only continue one path, you can do it with the russian roulette way.
Define three intervals (one common way is to chose the intervals to be a partition of the numbers between 0 and 1). The relative sizes of these intervals have to be proportional to the material. The tricky part is how do you design this. In my class we defined a number reflectivity $\rho$.
We draw a random number between 0 and 1, if it is lower than $\rho$ the path ends. Else we reflect. The BRDF will take care to weight the specular vs diffuse part if you do enough samples.

Of course you can improve this via importance sampling, and that is probably where you have problems. For a very specular surface design a random distribution that just draws more directions from the specular reflection direction. Rest will be diffuse. I don't have enough experience with actually bulding path tracers to help you how to design one, but your sources will probably have something here.

Important here is again: Design it so that you know how probable a specific direction is, and then keep your $p(x_i)$ up to it, so you can in the end divide by this and get a correct Monte Carlo estimate.

TL;DR There is no correct way. It is ONLY important to keep the probabilities of samples in mind.

I hope I haven't missed the point of your question entirely.

Addition:
Monte Carlo samling, as long as you correct weighting by probability, will always with infinite samples converge against the correct value. This means even if you use a complete stupid way to generate samples, if you do it long enough, that will work. Maybe not in your lifetime or that of your great grandchildren, but it will.


Are some 3D objects "solid"? Do they have internal density? If so, when, and in which file formats?

Assuming what you’re referring to is this Slicer and the models are the ones produced by its Model Maker module: it looks like it’s creating hollow, surface-only models.

Specifically, judging by the crunchiness of the mesh shown in the top right of this screenshot, it looks like it’s using marching cubes to create a mesh from the isosurfaces at given density thresholds in the volume.
Most 3D model formats—especially the ones used for real-time rendering—are surface-only; as graphics hardware gets more powerful, though, there’s been wider use of volume rendering, often via raymarching, to render volume data directly rather than having to go through the intermediate meshing step. One common format for volume data is OpenVDB; it’s also possible to embed such data in Alembic files.


Clipping circle and polygon and generate a CAD drawing

I don't know how the clipping library you are using returns the clipped objects, but if I understand your question, you want a way to represent your circles that does not use much memory? If you are only using perfect circles (and not ellipses) you could represent each arc segment as simply an origin point, a radius, and a start/stop angle.

This is the part where it would be helpful to know what the clipping library returns. If it returns a chain of points as a separate object for each clipped arc segment, and you already know the radius and center of the circle (from before clipping), you could take the first and last point in the returned chain and compute the angle for each.

atan2(x - origin.x, y - origin.y)

Then when you would like to draw the thing, you can write and algorithm to trace the arc going from the start to stop angle. If you want to do further clipping, you could recreate the arc as a chain of points.

Be careful about whether the returned segment was clockwise or counterclockwise.


How did the hard, accurate shadows, work in early 3d (2000 - 2004) games?

Having not worked on these games, I can only speculate... but from your description, it sounds like a crude shadow map with orthogonal projection and no depth information. It probably has (or had) a specific name but the way I suspect they did it is:

Render the character from above to a binary texture (eg. white there there's nothing, black where the character is), using an orthographic camera.
When rendering the scene, perform a lookup in that texture and multiply by that color. You would of course exclude your characters from that.

The calculations involved for the texture lookup would be trivial as everything is 2D, in a plane of the world coordinates. It also does not need programmable shaders which IIRC were not around yet at that time. What you would need as texture coordinates is simply the (x,z) location of the vertex (assuming y is toward the sky here), along with an offset equal to the (x,z) location of your character. In OpenGL, I believe that could be achieved by glTexGen along with a texture transformation matrix.

The reason for the artifacts is that this really does make a "character-silhouette shaped tube", as you put it. Looking from above, the shadow is painted on every surface regardless of it being in front of or behind the character (above or below, from the camera's view). It's like you're dropping black ink from the sky inside the outline of the character and it goes through everything.


Is there ever a time when I should *not* use the GL* types?

Yes, there's a time not to use GL types when interacting with OpenGL: GLvoid. There is absolutely no reason to use this type.

With everything else (assuming you're using a "standard" OpenGL loader), you should use OpenGL's types when talking to OpenGL. That is, the values you pass to OpenGL functions should be in OpenGL's types. This may mean that a function that talks to OpenGL may need an OpenGL data type in its function parameter list. This should be avoided where possible, to prevent you from creating dependencies between OpenGL code and non-OpenGL code.

However, if you don't mind having large portions of your application have dependencies on OpenGL, you may use OpenGL's data types throughout your code.

OpenGL's data types are specified to have a particular bitdepth. As such, they are (conceptually, if not strictly C++-legally) pointer-interconvertible to certain types defined in <stdint.h>/<cstdint>. Indeed, many OpenGL headers will check for the existence of this header and use it wherever it is available to define OpenGL's data types. As such, if you want a fixed bitdepth integer, you have a choice besides using OpenGL's data types.

  I am wondering if there is any case where using the GL types could cause issues when it it is not explictly clear that the values are coming from OpenGL.

OpenGL is C, not magic. A function in C cannot have any control over the return value of the function that calls it. Even a C++ function cannot; at best, it can throw an exception, but this still doesn't have anything to do with its caller's return value.


BDPT - How to handle deterministic ray PDF?

Sorry about leaving it for so long but I was so busy implementing the goddamn algorithm, I forgot about this :)

So after reading up on Veach's thesis pretty closely, I understood what was going on. First of all Veach's thesis describes about BDPT using MIS. However I implemented without MIS using the techniques described in the original Lafortune's paper.

One thing to note is, that we have to calculate the radiance by the light path inversely as opposed to the normal flow in the eye path. As lightxbulb said for point 1, there is no pdf since the step is deterministic. However the deterministic step needs to be multiplied by the Geometric factor, the BRDF evaluation of light vertex to eye vertex and the BRDF evaluation of eye vertex to light vertex.

For the second point, you don't have to worry about NEE since at each eye path vertex there is a path that connects to the light vertex directly, according to PBRT and the paper we can discard the already generated light source and use explicit light sampling there. Calculating the light path radiance separately (as done in light tracing) handles all the respective PDFs for generating the vertices there.

Will post another question regarding the result comparison soon.


How to change sampler/pipeline states at runtime in Vulkan?

You're sort of asking two different questions here but that's okay.

Changing Sampler states is pretty straightforward. When you create a descriptor set, you write your VkSampler object that you've created into a descriptor set. If you want to update a sampler in a descriptor set, create a VkSampler with the sampler settings you want and then call vkUpdateDescriptorSets with the new VkSampler writes that you want to perform. When you bind that descriptor set again it should contain your new sampler. This is a bit roundabout but you have to architect for it. You probably don't want to update samplers every frame so only do this when you need to. This is not an extremely expensive operation if you do it right. Vulkan expects descriptor set data to change; so long as that data lines up with the descriptor set layout it's going to go pretty fast.

Changing Pipelines is EXTREMELY expensive. Vulkan doesn't like to perform context switches like this so you can't just update an existing pipeline that's in use. You do have to completely rebuild your rendering path to use the new PSO. If you want to change between a few different Pipelines it's best to build them all upfront, build all the different command buffers that use the different pipelines and then execute the command buffer that has the pipeline you want to run. If you want to change your pipeline, execute the other command buffer that uses the pipeline you want.

Vulkan wants you to create, cache and reuse data as much as possible. That's how you get a lot of savings. You don't need to update descriptor sets, command buffers etc. unless they actually change.


UV unwrapping is a difficult topic. They can be both combinatorial algorithms or variational methods but in general they're optimization based, i.e. you setup an optimization problem and you solve it using some numerical optimization solver. I'm just going to give you some names and some libraries you can use eventually. I'll give you few names (both classic and more recent) and you can have a look yourself. ABF (Angle Based Flattening), LSCM (Least Square Conformal Mapping), ARAP (As rigid as possible), BFF (Boundary first flattening), Variational Surface Cutting, OptCuts.

For ARAP and LSCM you can use libigl. Boundary first flattening was developed by Keenan Crane few years ago. Variational surface cutting is an improvement over BFF. And I recently discovered OptCuts. The code is available and you can run them yourself.

You need to read through the papers to understand how they works if you wanted to implement them from scratch (not going to be easy, parametrization is a difficult subject). But if you want to just to run them just clone the repos and try them yourself.
(libigl comes with tutorials, and two of these are about ARAP and LSCM). BFF, Variational Surface Cutting and OptCuts have a GUI based app so you can see the unwrapping and you can easily isolate the code if you need them in your own code.


Roughness Value of (Close to) Zero in Physically Based Rendering

Yes, I think it's expected that setting roughness = 0, combined with using point lights for illumination, leads to no visible specular highlight. The size of the highlight is infinitesimally small, so the sample points (e.g. pixel centers) almost surely miss it. The math breaks down as well, as the reflectance would become infinite on the zero-sized highlight (a delta function).

Clamping roughness to a minimum seems like a physically reasonable thing to do. 0.04 feels like too large a minimum, though; I might pick 0.001 or 0.0001 or something like that.

If you use area lights, then a perfect mirror surface (roughness = 0) could still work and would show the sharp reflected image of the area light. This would probably still require special-casing the perfect-mirror case in shading code, to avoid the problems with infinities.


Shader that transforms a mercator projection to equirectangular?

This question and its answer, actually answer my question. I had to read its answer carefully and find my mistake.

Now my fragment shader look like this:

const GLchar* fragmentSource = R"glsl(
    #version 330 core

    in vec2 TexCoord;

    uniform sampler2D ourTexture;

    #define PI       3.141592653589793238462f
    #define TWO_PI   6.283185307179586476924f
    #define PI4      0.7853981633974483096155f

    void main(){
        float lat = (TexCoord.y - 0.5f) * PI;
        if(lat >= -1.48442222974871 && lat <= 1.48442222974871){
            float y = log(tan(PI4 + (lat / 2.0)));
            y = (y + PI) / TWO_PI;
            gl_FragColor = texture(ourTexture, vec2(TexCoord.x, y));
            return;
        }
        gl_FragColor = vec4(1, 1, 1, 0);
    }
)glsl";

And it's works perfectly.


Yes ( with 50% of the FMA 32b throughput on nVIDIA Maxwell).

See
https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#arithmetic-instructions


Graphics engines during the sixth console generation

Renderware was common on the PS2.

List of PS2 Renderware titles

I would guess because these platforms were proprietary and none-standard from a graphics and resources POV that it would be a bigger effort to port Unreal along with all their existing content creation tools.


Direct illumination, specular surfaces, and MIS

Specular surfaces which use MIS are not perfectly specular like a mirror. They have a small amount of blur, otherwise there is indeed no point in sampling the light as all the samples will evaluate the BRDF as 0. In fact, you would only need to trace a single reflection ray.

A small amount of blur means that a given camera ray will see a small area of the lightsource (assuming it hits where the specular highlight is visible). Thus, a small fraction of the light samples will get a BRDF which is not 0 so it will eventually converge. The less blur, the more samples it will take.

You are correct that such a case will converge faster if you sample only the BRDF. Likewise, a very diffuse BRDF will usually converge faster if you sample only the light. But this is not true if your light is very large and your surface very close to it1. Where MIS shines is that it handles those cases as well as the BRDFs which are neither very diffuse nor very specular.

If you choose to ignore it, what you have to do is a simple integral. There are a few ways to go about it but if you start from MIS equations, just use 0 as the PDF of the BRDF since you're not generating those samples. And be prepared for some cases which won't converge well.

The assumption is that the light samples all have a similar contribution to the final result. Even with very good sampling of the light, this must assume that the BRDF will be similar for all light samples. At least I don't know of any direct light sampling formula which takes the cosine term of the diffuse surface into account. Pushing this to the extreme, you can build an example where most of the light is behind the surface, so most of the light samples will contribute nothing at all to the final result. BRDF samples are obviously sent only on the front side of the surface so they might converge faster, even if only a small % of them hit the light.


How to draw surface normals from surface normal maps?

Typically the normal vector, which has XYZ components in [−1, 1] is linearly mapped to the RGB range [0, 255]. You can retrieve the normal vector by reversing this mapping.
For completeness, the encoding is:
$$
r = \text{round}((0.5x + 0.5) \cdot 255) \\
g = \text{round}((0.5y + 0.5) \cdot 255) \\
b = \text{round}((0.5z + 0.5) \cdot 255)
$$
Decoding is:
$$
x = (r/255) \cdot 2 - 1 \\
y = (g/255) \cdot 2 - 1 \\
z = (b/255) \cdot 2 - 1
$$


Short answer, set the precision of the image to a higher value.

Long answer,

When looking at a gamma correction curve, you can see that the lower values get changed much more, this means that the difference between lower values will get greater and that causes this effect. You have a limited amount of values for a color channel and this means that when it rounds the number down to the 8 bit value, it makes some neighboring pixels the same color. You don't notice this in a normal image, but after gamma correcting it this effect comes out, because you lose that bit of change in the color that later on can become a much bigger difference in color.

To fix this, you just give the gamma correction more precision to also get that small color change it would've else rounded away.

In the actual shader, it has floating point color information and that means that it has more precision and thus not the weird artifact.

So when making the frame buffer, you can use for example GL_RGBA16F or GL_RGBA32F. This would increase the precision and remove the artifact.

I hope that this can help you fix the problem, good luck and have a great day!


OpenGL Compute Shader generating triangle indices: How to get correct element count for glDrawElementsIndirect?

If you have access to glsl 4.3+ (or glsl ES 3.1) you can use atomicAdd

The next option is to use a barrier() after all vertices are generated in the compute and then multiplying the value in the counter:

main(){

    // generate vertices

    barrier();
    if(gl_localInvocationID == vec3(0)){
        indirectCount = atomicCounter(indirectIndexCount)*3;
    }
}

This emulates running another 1x1x1 compute shader to multiply the index.

Otherwise you can use a second atomic to hold the vertexcount:

void addTriangle (uint i0, uint i1, uint i2) {
    uint ic = atomicCounterIncrement(indirectIndex);
    meshIndices[ic*3+0] = i0;
    meshIndices[ic*3+1] = i1;
    meshIndices[ic*3+2] = i2;
    atomicCounterIncrement(indirectIndexCount);
    atomicCounterIncrement(indirectIndexCount);
    atomicCounterIncrement(indirectIndexCount);
}

Both indirectIndex and indirectIndexCount are initialized to 0 and indirectIndexCount is what is passed through to  glDrawElementsIndirect.


perspective projection transformation matrix

Given your equations (which i just assume are correct):
$$
\begin{align}
x_p &= x \frac{z_{vp}-z_{prp}}{z-z_{prp}}+x_{prp} \frac{z-z_{vp}}{z-z_{prp}}\\
y_p &= y \frac{z_{vp}-z_{prp}}{z-z_{prp}}+y_{prp} \frac{z-z_{vp}}{z-z_{prp}}
\end{align},
$$

if we take $(x_{prp}, y_{prp}, z_{prp}) = (0, 0, 0)$, then we have

$$
\begin{align}
x_p &= x\frac{z_{vp}}{z}\\
y_p &= y\frac{z_{vp}}{z}
\end{align}
$$

Assuming the projection plane is at distance $d$ from the origin, and, by your drawing, that the projection plane is on the negative side of the Z axis, we have $z_{vp} = -d$, which leads to

$$
\begin{align}
x_p &= - x\frac{d}{z}\\
y_p &= - y\frac{d}{z}
\end{align}
$$

Now we can follow the same reasoning (homogeneous coordinates) as in my answer to your other question (using column notation) to write

$$
\begin{align}
\begin{bmatrix}
x\\y\\z\\ \frac{-z}{d}
\end{bmatrix}
=
\begin{bmatrix}
1&0&0&0\\
0&1&0&0\\
0&0&1&0\\
0&0&\frac{-1}{d}&0
\end{bmatrix}
\begin{bmatrix}
x\\y\\z\\1
\end{bmatrix}
\end{align}
$$

which after the division by the last coordinate will give the projected point. Notice that because we are writing it in column notation, the perspective matrix is the transpose of the matrix in your other question. Also, the negative sign is because the camera reference frame is a right-handed one.

As for the second textbook that you are reading, I cannot understand why you would have a negative sign only in the x axis without knowing more about the camera reference frame. By the diagram you posted, there should not be a negative sign.


Transformations and change of basis in CG

You're correct, we don't usually worry about pulling the inner product through a change of basis in graphics. As a practical matter, calculations that depend on angles and distances, such as lighting and shading, are always done in world space or in a space orthonormally related to it (e.g. camera space).
The place where non-orthonormal transformations show up is usually in local-to-world transforms of meshes—most frequently with skeletal animation, where things get deformed, or where bones are set up with non-orthonormal coordinates for rigging/animation reasons. However in those instances we are always bringing the vertices, normals, etc into world space or camera space to process. We neither push back the world inner product into the local spaces, nor do we want to pull a locally defined inner product into world space, as neither of those is really useful for the typical kinds of things we need to do.
(And for the normals, we do know how to use the inverse transpose matrix to transform those properly, thank goodness!)
The other big non-orthonormal transformation in graphics is the projection matrix, but again, we don't really ever want to use the inner product in post-projective space, so the issue doesn't really arise. We just keep all our inner product-ing in world space.


How to create static alpha masks of an image

You can either use a threshold, or, since you know the location and size of the 2 circles, simply generate one programatically. In this case, you have a circle that's centered at (279, 283.5) and has a radius of 265. (I haven't calculated the center and radius of the smaller circle, but you should be able to do that fairly easily by looking at the image in any image editor that shows you mouse coordinates.)

Once you have the smaller circle radius, for each pixel in the mask find the distance to the large circle center and if it's less than the large radius, color it white. Otherwise take the distance to the smaller circle and if it's less than the smaller radius, color it white. Otherwise, color it black.


Return x; y; coordinates of vertex C of triangle

You can get the coordinates of C by using circle equations for one circle C_A with point A as center and AC as its radius and one circle C_B with point B as its center and BC as its radius.

Now you can calculate their intersection points, which leaves you with 2 points (iff C is not on a line between A and B).

I am not entirely sure how you want to determine which of the two candidates C_1 and C_2 is the correct one. If I understand correctly, you always label the triangle counterclockwise. There is a neat way to check this, so you should have everything to find your point.


Blend Shape, Pose Blend Shape, and Shape Blend Shape

Blend shapes, also known as morph targets, are a widely used character animation technique that helps get around the limitations of linear-blend skinning (skeletal animation). A blend shape is defined as a set of per-vertex offset vectors, usually in a localized region of a character mesh, such as a neck or upper arm. The idea is that during animation, the basic pose produced by skinning is supplemented by adding these offset vectors, scaled by an animated weight factor, prior to applying the skinning transformations. This allows character artists to model such motions as the bulging/thinning of muscles when a limb is extended or contracted, or the shifting of soft tissue around the underlying rigid skeleton, which are not captured by skinning.
Blend shapes are also commonly used for facial animation, with different blend shapes representing different motions of the facial muscles, such as raising/lowering eyebrows, squinting the eyes, flaring the nostrils, moving the lips in various ways, etc. These are very difficult to model realistically using linear-blend skinning alone. By manipulating the weight factors of all these blend shapes, the animator can produce various facial expressions. A captured performance of an actor can be "solved" into animation curves for a predefined set of facial blend shapes, as a means of compression, or for transferring performance to a different face.
In the most basic version of blend shapes, their weight factors are animated by hand. But their weights can also be driven programmatically by some other set of parameters—for example, by the pose of the skeleton. Essentially, you can say something like "whenever the elbow is bent, apply the bicep-bulging blend shape", so that will happen automatically based on whatever the skeletal animation is doing, without having to animate it manually. This is broadly called "pose space deformation", since the blend shapes (deformations) are associated with regions in "pose space", the high-dimensional space of all the joint rotations/translations. The slide deck you posted refers to this as "pose blend shapes", however the more standard term is "pose space deformation" (and there's plenty of literature on it).
The "shape blend shapes" in that slide deck appear to be something specific to the pipeline they're describing, for creating a parameterized human model that can output many different body shapes and proportions. The "shape" parameters in the model presumably index the range of individual variation in body shape—different heights, limb lengths, hip to waist ratios, etc. They attempt to do this by starting with a single generic human mesh and applying various blend shapes (as well as moving joints around) to parameterize this space of body shapes.


There is no single method of layering materials. Some implementations, as you have observed, simply interpolate shader input parameters. This helps especially in contexts where code divergence comes with performance penalties (GPUs for example), since all material layers use the same code path and only have different data. Other blend between the output of the individual shader layers.

More complex approaches take the interface between material layers into account, as well as layer thickness. A recent publication describing such a method is "A Comprehensive Framework for Rendering Layered Materials" by Jakob et al.


Do GPU performance improve when rendering at native or half resolution, compared to fixed scaling?

GPUs can efficiently scale an image by an arbitrary amount (within limits - display options fall within those limits by design) either using a 3D rendering operation or as the signal is sent from GPU to the display. Both of these paths have fully dedicated hardware for the arbitrary resizing and are not likely to be optimized for doubling or halving. Both support a technique called "bilinear filtering" though advances in display hardware may provide higher quality.

If a game is run at non native resolution on a laptop (TVs and desktop displays have their own scalers, but laptop displays generally rely on the GPU), one of these two methods will be used. Display scaling is essentially "free", so it is preferred and the lower the resolution the game is originally rendered the faster the frame rate (or the lower power consumption) - with no special performance benefit for, say, doubling. If a 3D operation is required (say, due to a limitation or issue in the display scaler), the scale operation is not free, but is going to go up with the source and destination sizes smoothly, with no special casing at doubling/halving.

The game may look far better at certain scaling factors, but that's up to the user to decide.

Edit: The intuition behind the question is correct: if GPUs didn't have dedicated scaling hardware, doubling/halving could implemented more efficiently than arbitrary scaling, either in shader code or via a slightly simpler hardware design - though the 3D rendering operation is a requirement of graphics APIs and display scaling is a requirement from systems vendors.


Difference between rendering in OpenGL and 3D animation software

The main difference would be that with OpenGL in let's say a video game you will have a process called rasterization which basically takes care of determining what part of the scene you see.

It needs to be fast so we can experience it as realtime.

Therefore the algorithm does a few simple steps.

check if a certain part of the scene is in my view frustum

check if something is in front of it which may needs to be rendered later using a depth buffer

order the objects we found to draw
draw them by projecting them on the screen
shade them based on textures / shaders / lights / ...

On the other hand a rendering software (Blender/Max/Maya/...) most likely uses some kind of raytracing

This involves alot more math to achieve a higher degree of realism.
It basically works in the same way:

create a camera and an image plane infront of it
shoot one ray (or multiple sample rays) through each pixel
check if the ray hits anything in the scene
closest hit is the one to be drawn in the pixel finally (like depth buffer)
calculate the light for the given point

....

I'll stopped listing here since this is the point where raytracing takes off.

Instead of only checking if a point is hit, most raytracer now begins to calculate:

the amount of light a surface penetrates
how much light gets reflected
cast new rays from the hitpoint into the scene until it may hit a light source

There are a ton of techniques with different degrees of realism which can be used to calculate the light of a certain point in the scene.

TL;DR
The gist would be that a raytracer mostly tries to be physically accurate when it comes the lighting and therefore has a lot of more calculations to do per pixel (sometimes shoot thousands of rays) and on the other hand games get their speed by drawing bigger chunks of the screen with simpler light calculations and a lot of shader tricks which let it look realistic.


GL_SAMPLE_ALPHA_TO_COVERAGE and depth testing

(Promoting comment to answer)
Do the faces have the same alpha value? That might generate the same coverage mask for the surfaces and, hence, only the closest one would be seen.


I've done this 2 different ways in the past:

Apply a pre-blur to the image before running the Sobel operator on it. This will have the side effect of getting rid of any noise that's smaller than the blur kernel, but it will also thicken the lines you end up drawing, too.
Apply a post-process where you thicken the lines by either stamping a shape at every point that's an edge, or using some sort of dilate process.


Why does the red color channel lose the most details during jpeg compression

One issue is "chroma subsampling". The image data of a JPEG is not stored in packed RGB but YCbCr as 3 seperate planes. The Cr and Cb channels contain colour information and the Y channel is luminance. In JPEG it is common to use a lower resolution Cr/Cb channel, typically at half the resolution of the Y channel, this is know as 422 Chroma subsampling (another variant is 420 where only the horizontal resolution is halved).

What you can do is select 444 Chroma subsampling, if possible, which will mean no down-scaling of the Cr/Cb channels.

In Gimp you can select the subsampling type in "advanced options" when exporting JPEG files. This is independent of the quality setting slider.

There is a great image from wikipedia which compares the various subsampling modes side-by-side with a visualisation of one of the chroma channels, you can see the red decal in the centre is most affected by downscaling as that channel has high contrast edges

Wikipedia article for Chroma Subsampling


Your if condition makes me suspicious. You should include the diffuse and specular shading if the intersection test didn't hit an object; that is, if intersections > ldist. So, your code should look as follows:

if (intersections > ldist)
{
    colour += diffuse(...);
    colour += specular(...);
}

Your comparison with 0.0000001f suggests that you've also been having trouble with self-intersections (that is, with the ray intersecting the object being shaded). In a one-hit tracer, you can't fix that problem by simply discarding the intersection, because you still don't know whether the shadow ray would have intersected another object further along. To avoid self-intersections, you need to push the start point of the ray along its direction by epsilon:

glm::vec3 direction = normalize(ldir);
glm::vec3 start = surfacePoint + epsilon * direction;
float intersections = myObjGroup->testIntersections(start, direction);

One more thing: don't forget to test that the light is in front of the surface that you're illuminating. You can do this with the dot product of ldir and the surface normal: if the dot product is positive, the light is on the correct side; if it's negative, the light is behind the surface and thus shadowed before you even cast a ray.


How many GPUs can you simultaneously output from using a single PC?

I don't think there are any specific limits. Each GPU can probably drive three or four monitors, and with multiple GPUs in a system, I don't think there's any barrier to hooking up the maximum number of monitors to each GPU. Here's a video, for instance, detailing a 16-monitor machine with 4 GPUs, on Windows.

Googling around, I did find some people stating that Windows has a 16-monitor or 64-monitor overall limit, but I couldn't verify this from any official documentation. I also saw some claims that Windows has a limit on the overall desktop size that the monitors are virtually placed within, but again could not verify that. In Linux any such limits (if they exist) would naturally be different, and presumably would depend on which window system you're using.

The other potential limiting factor would be CPU performance and PCIe bus bandwidth for keeping all those GPUs fed with data.


First of all, a good reference for Monte Carlo path tracing in participating media is these course notes from Steve Marschner.

The way I like to think about volume scattering is that a photon traveling through a medium has a certain probability per unit length of interacting (getting scattered or absorbed). As long as it doesn't interact, it just goes in a straight line unimpeded and without losing energy. The greater the distance, the greater the probability that it interacts somewhere in that distance. The interaction probability per unit length is the coefficient $\sigma$ that you see in the equations. We usually have separate coefficients for scattering and absorption probabilities, so $\sigma = \sigma_s + \sigma_a$.

This probability per unit length is exactly the origin of the Beer-Lambert law. Slice a ray segment into infinitesimal intervals, treat each interval as an independent possible place to interact, then integrate along the ray; you get an exponential distribution (with rate parameter $\sigma$) for the probability of interaction as a function of distance.

So, to answer your questions directly:

You can technically choose the distance between events however you want, as long as you correctly weight the path for the probability that a photon can make it between two adjacent events without interacting with the medium. In other words, each path segment within the medium contributes a weight factor of $e^{-\sigma x}$, where $x$ is the length of the segment. (This is assuming a homogeneous medium, but see section 4.2 in the Marschner notes linked above for what to do if it's inhomogeneous.)

Given this, a usually good choice for the distance is to importance-sample it from the exponential distribution. In other words, you set $x = -(\ln \xi)/\sigma$ and then leave out the $e^{-\sigma x}$ factor from the path weight.

Then, to account for absorption, you can use Russian roulette to kill off a fraction $\sigma_a/\sigma$ of the paths at each event. This is particularly necessary for very large or infinite media (think atmospheric scattering) where the path could bounce around for an arbitrarily long time if it's not killed. If you're only dealing with small and not-too-dense media, then it might be better to just factor in a weight of $1 - \sigma_a/\sigma$ per event, rather than using Russian roulette.
No, if you follow the importance-sampling procedure just described, Beer-Lambert is already incorporated implicitly in the sampling, so you don't want to apply it to the path weights.
The volumetric equivalent to a BSDF is the combination of the scattering and absorption coefficients $\sigma_s, \sigma_a$ and the phase function. By convention, the coefficients control the overall balance of transmission, scattering, and absorption, while the phase function is always normalized.

You could do something like this for BSDFs too; you could factor out the overall albedo, and have the directional dependence always be normalized. It's mostly a matter of convention AFAICT.
Try "participating media" (that is, a volumetric "medium"—plural "media"—which "participates" in light transport), and "volumetric path tracing".


How can I check if a polygon can completely contain a circle of a certain radius?

This is likely more complicated than you would prefer, but: Compute the medial axis, which immediately yields the largest disks that fit inside the polygon:
their centers are vertices (degree $\ge 3$) of the axis (see the
figure below).

  Chin, Francis, Jack Snoeyink, and Cao An Wang. "Finding the medial axis of a simple polygon in linear time." Discrete & Computational Geometry 21.3 (1999): 405-420.

(Image from Alexander Tsvyashchenko.)

See also, Largest circle inside a non-convex polygon for an ad hoc approach
(whose correctness I did not attempt to verify).


Ray tracing: transforming ray to object's local space for intersection test

I figured out the problem. It was not related to the ray / point transformation itself, but to a problem with the camera update after moving it (Up vector wasn't being computed properly).

I leave the answer in case someone have similar issues.


OpenGL Polygon Offset Fill implementation example code

I found a workaround suggested at the bottom of this page : to deal with coplanar lines and polygons, one can apply a different glDepthRange to polygon face and polygon wireframe.


Yes, there's a better approach than using a screen recorder. You can capture frames directly inside Unity, using ScreenCapture.CaptureScreenshot. There's a choice of formats to save as, and you can use PNG so that, when you later assemble the frames into a video (using ffmpeg or another tool), you have full control over how much compression to use.

This function is very slow, because it reads back the framebuffer, encodes the image, and writes it to disk, all in the main thread. You may want to use it in combination with setting the Time.captureFramerate value. This setting lets you slow down Unity by completely disconnecting the in-game frame rate from real time. That is, if you set it to 30, the game will run at 30 fps as measured by Time.deltaTime and the calls to Update on your components, regardless of whether a frame takes a millisecond or a minute as measured by wall-clock time.


Ray casting in fan configuration produces point cloud with curvature, how to eliminate curvature?

The direction vectors start at +15 degrees on the YZ plane, and continue in increments of -2 degrees down to -15.

There's your problem—by generating the rays that way, you're doing something more like a spherical projection or fisheye projection, not a linear perspective projection. The nonlinear projection is the origin of the curvature you see.

To get a perspective projection instead, conceptually the way to think about it is that there's an image rectangle floating in front of the camera, divided into grid squares (pixels), and you fire a ray from the camera position toward the center of each pixel. In code, this looks something like:

p = tan(horizontalFOV / 2)
q = tan(verticalFOV / 2)
for y = 0 to imageHeight - 1
    for x = 0 to imageWidth - 1
        rayDirection = Vec3f(1.0,
                             p * ((x + 0.5) / imageWidth * 2.0 - 1.0)
                             q * ((y + 0.5) / imageHeight * 2.0 - 1.0)).normalize()

Note that the directions generated this way are not equally spaced in angle (they're closer together in angle toward the edges of the image than at the center), but they are equally spaced in distance on the image rectangle.


Can I use my phone's gpu as an external gpu for my laptop?

No. Your phone's GPU is in one chip with the CPU, some of the memory, and its display controller, and they're all tightly integrated together. Also, there'd be no way to connect it efficiently to the laptop. Only the latest generation of USB and Thunderbolt have enough bandwidth to be even worth considering for external GPUs, and even then, they're only used with dedicated hardware at the GPU end of the link - not a general-purpose processor. Apart from the fundamental problems, there's the very practical issue that your PC doesn't have any drivers for the GPU on the phone.

I know mobile GPUs are pretty effective, but if your phone's GPU seems faster than the Nvidia chip in your laptop, there's something wrong with your laptop. If you can give more information on the problem to the people at Ask Ubuntu, they can probably help with that.


Is there a similarly fast way to draw antialiased lines?

No, because by definition an anti-aliased line touches more pixels.  Such algorithms will be slower.

In a software rasterizer, the ubiquitous way to draw anti-aliased lines is Xiaolin Wu's line algorithm.  It's not hard to implement, and anyway there's unusually high-quality pseudocode at that link.

In a hardware raster pipe, the line primitive is expanded to a screen-space quad by the default (or user-provided) geometry shader, and then drawn as two triangles, which can then be anti-aliased in the usual ways.

In a raytracer, there are a variety of options.  It's worth thinking about how you actually want to draw a 1D object.  Maybe as a cylinder (woo shadows!).  Note that this introduces issues of perspective/foreshortening which may (or may not) be what you want.  There isn't a clear generalization.  Then, obviously, whatever you do, you just supersample it.


Vulkan best way of updating pipeline viewport

Both suggested methods are fine, but they should be preferred in certain situations.

Dynamic state should be preferred in situations where you have many, perhaps thousands of, pipeline objects or you need to change the resolution often (such as in some sort of windowed tool. The downside to this is that it may reduce runtime performance since the driver may not be able apply optimizations for this.

Having the viewport only defined when building the pipeline objects might be more appropriate for games or simulators where you don't change the resolution often. Of course, when the viewport does change, it hits performance hard during resizing. This also has the drawback of needing to generate new pipeline objects, which could be especially problematic if you cache pipelines in a hash table of something similar.


What is wrong with my glDrawElement code

glDrawElements specifies the second argument to be the number of elements to be taken from the buffer of indices, not the number of vertices. The correct number would be the number of triangles times three.


The simplest solution would be just a multiplication, which would give black edges:

combinedColor = sceneColor * edgeRGB.r;

For more control I would suggest something very similar to what @PaulHK proposed in the comments:

float opacity = maxEdgeOpacity * (1. - edgeRGB.r);
combinedColor = mix(sceneColor, edgeColor, opacity);

edgeColor is the desired edge color and maxEdgeOpacity is meant to be a parameter between 0 (no edges at all) and 1 (opaque edges), that you both choose to your liking.


Through help from several people, & referencing and re-referencing the commented urls, I managed to get a sampling scheme with matching PDF that handles everything from perfect mirror surfaces to roughness = 1. Here's the eventual result:

// Cook-Torrance microfacet model
type Microfacet struct {
    F0        rgb.Energy
    Roughness float64
}

// https://schuttejoe.github.io/post/ggximportancesamplingpart1/
// https://agraphicsguy.wordpress.com/2015/11/01/sampling-microfacet-brdf/
func (m Microfacet) Sample(wo geom.Direction, rnd *rand.Rand) geom.Direction {
    r0 := rnd.Float64()
    r1 := rnd.Float64()
    a := m.Roughness * m.Roughness
    a2 := a * a
    theta := math.Acos(math.Sqrt((1 - r0) / ((a2-1)*r0 + 1)))
    phi := 2 * math.Pi * r1
    x := math.Sin(theta) * math.Cos(phi)
    y := math.Cos(theta)
    z := math.Sin(theta) * math.Sin(phi)
    wm := geom.Vector3{x, y, z}.Unit()
    wi := wo.Reflect2(wm)
    return wi
}

// https://schuttejoe.github.io/post/ggximportancesamplingpart1/
// https://agraphicsguy.wordpress.com/2015/11/01/sampling-microfacet-brdf/
// https://en.wikipedia.org/wiki/List_of_common_coordinate_transformations#From_Cartesian_coordinates_2
func (m Microfacet) PDF(wi, wo geom.Direction) float64 {
    wg := geom.Up
    wm := wo.Half(wi)
    a := m.Roughness * m.Roughness
    a2 := a * a
    cosTheta := wg.Dot(wm)
    exp := (a2-1)*cosTheta*cosTheta + 1
    D := a2 / (math.Pi * exp * exp)
    return (D * wm.Dot(wg)) / (4 * wo.Dot(wm))
}

// http://graphicrants.blogspot.com/2013/08/specular-brdf-reference.html
func (m Microfacet) Eval(wi, wo geom.Direction) rgb.Energy {
    wg := geom.Up
    wm := wo.Half(wi)
    if wi.Y <= 0 || wi.Dot(wm) <= 0 {
        return rgb.Energy{0, 0, 0}
    }
    F := fresnelSchlick(wi, wg, m.F0.Mean()) // The Fresnel function
    D := ggx(wi, wo, wg, m.Roughness)        // The NDF (Normal Distribution Function)
    G := smithGGX(wo, wg, m.Roughness)       // The Geometric Shadowing function
    r := (F * D * G) / (4 * wg.Dot(wi) * wg.Dot(wo))
    return m.F0.Scaled(r)
}


You need to be careful about spaces - position and normal need to be in view space.

To transform depth into view space position I used this method.
So:

vec3 viewPositionFromDepth(vec2 vTexCoord)
{
    float z = texture2D(depthTexture, vTexCoord).r;
    // Get x/w and y/w from the viewport position
    float x = vTexCoord.x * 2.0 - 1.0;
    float y = vTexCoord.y * 2.0 - 1.0;
    vec4 vProjectedPos = vec4(x, y, z, 1.0f);
    // Transform by the inverse projection matrix
    vec4 vPositionVS = invProjection*vProjectedPos;
    // Divide by w to get the view-space position
    return vPositionVS.xyz / vPositionVS.w;
}

To transform normal into view space you need to multiply it by transpose of inverse of view matrix (should be done on CPU):

vec4 n = transpose(inverse(ViewMatrix)) * normalize(texture2D(normalTexture, texCoord)*2.0 - 1.0 );

And in loop while sampling depth with offset to compare with kernel position, this depth should be in same space as position so I did it the same way:

float sampleDepth = viewPositionFromDepth(offset.xy).z;

This is not the most optimal way though, but it worked for me.


The problem is that if you have 8 light sources (as your test image appears to have), then anything that is under only one shadow will be 7/8ths of full brightness, which is a fairly small change. Anything other than that is unphysical, so it'll be hard to convince your raytracer to produce a different result by changing the lighting.
Instead, after rendering the image, you can apply some kind of nonlinear processing of the image colors so that that 1/8th difference stands out more. In your second image, this seems to have been done by directly replacing the light gray fully-lit areas with white, but a more proportionate, yet still very visible, result could be achieved with a simple "levels" color edit applied to the whole image to increase contrast, as I've done for this version of your first image:

By pushing light gray to white, and darker grays more to black, the contrast in the image is greatly increased. The cost is that the building shapes are dark and hard to see.
To make the buildings look better and stand out, you could render your buildings a second time without any ground (so the resulting image has a transparent background), then layer that on top of the color-edited image, so that they are unaffected by the contrast adjustment.

It might be a good idea to use color to help distinguish the buildings from the shadows. In particular, I'd suggest avoiding making buildings gray (or, more generally, colored the same as the sunlight). Here I've used a "color temperature" edit on the above to make the buildings layer bluer and the shadowed ground layer yellower (probably too much):

Of course, this is all rough examples based on quickly editing your sample image; I think you can produce very legible and elegant results by taking these ideas and doing them better (by being able to adjust your raytracer lighting as well as postprocessing the image, and rendering a separate buildings-without-ground image as I suggested).


The math for the projection matrix is (with fov as $\alpha$):

$q \leftarrow \frac{1}{tan(\frac{\alpha}{2})}$

$a \leftarrow \frac{q}{aspect}$

$b \leftarrow \frac{(far + near)}{(near - far)}$

$c \leftarrow \frac{(2 * far * near)}{(near - far)}$

Notice that there're some things you're doing that are differently, such as the order of your subtractions between near and far, how you organize the matrix values, and your multiplication between your r * t.

Using the variables above, the column-major matrix below would be the resulting perspective projection matrix:

\begin{bmatrix}
  a & 0 &  0 & 0 \\
  0 & q &  0 & 0 \\
  0 & 0 &  b & c \\
  0 & 0 & -1 & 0
\end{bmatrix}

From the above, we get:

def perspective_projection_matrix(fov, aspect, near, far):
    q = 1 / tan(radians(fov * 0.5))
    a = q / aspect
    b = (far + near) / (near - far)
    c = (2*near*far) / (near - far)

    # construct column-major matrix here...

NOTE: I left the last part out because I'm not familiar enough with numpy to know whether it expects row-major or column-major order.

Also, you should validate all your arguments (e.g. both near > 0 and far > 0, far > near, etc.) if you want to avoid future headaches.


What is name for far lowcontrast blueish landspace?

In computer graphics the effect is called "atmospheric scattering", that's sun light scattering from atoms and other particles in the atmosphere. In modern computer games atmospheric scattering is commonly modelled with Bruneton sky model by Eric Bruneton and is most prevalent in the clear sky due to long view distances, but also visible for distant objects on ground as you have observed.

Atmospheric scattering in Bruneton model is composed of Mie scattering and Rayleigh scattering. Mie scattering is mostly wavelength independent strongly anisotropic scattering from particles that are quite a bit larger than the wavelength of light (water droplets & pollution in the atmosphere). Mie scattering results in effects like halo around the sun.

Rayleigh scattering on the other hand is wavelength dependent scattering from atoms in the atmosphere whose size is close to the visible light wavelength range. Rayleigh scattering is responsible of effects such as the blueish sky at noon and blueish tint of objects in the distance.


It could be generated as a Lindenmayer System (L-system) with some rules for branch widths (plus some fractal noise) and branch bifurcation. They are parameters you can play with.

You can find lots of resources in the website of the Biological Modeling and Visualization research group in the Department of Computer Science at the University of Calgary. Professor Przemyslaw Prusinkiewicz is the researcher who popularized these systems, and he has made his main book The algorithmic beauty of plants available for free.


You have to use the appropriate register type for the resource: b registers for constant buffers, t for textures, and u for UAVs. AFAIK, it is not possible to bind a UAV to a texture slot, or otherwise mismatch registers and resources.

However, a resource can have multiple views, so it is possible to have a resource bound as a UAV in one shader and as a regular texture (SRV) in another, which enables all the usual GPU texture sampling features to be used in the latter case.

The details of how the DirectX API concepts of slots, registers, and resources map to the actual hardware are vendor-specific. On one GPU they may literally represent registers of some kind, while on another they may represent a small chunk of metadata in memory somewhere, which describes the resource to the hardware.


Inconsistent shading in pathtraced image

I have managed to fix the error. It turns out the error was never situated in the normal calulation it was the shading algorith. The floating point precision error caused the new ray to be slightly below the surface, which meant it would intersect itself and become darker. This is a problem common in biased raytracers, the phenomenon known as "Shadow acne".

To fix it I added the following code:

bool Lambertian::scatter(Ray &ray, HitInfo &hit, float3 &attenuation, Ray &new_ray) const
{
    float3 offset = hit.normal*(1e-5);
    new_ray = Ray(ray.getHit(hit.t) + offset, cosineSampleHemi(drand48(), drand48()));
    attenuation = m_colour;
    return true;
}

the offset variable moves the ray origin to slightly above the surface by a small amount and that seems to fix the issue.


I don't know the specifics of what looks to be blender. But I would grab the view matrix of the camera. The rows of which should be orthonormal and give you forward, up, and right. Or columns depending on row or col major (technically its the inverse of the view, but that's just the transpose). Then just do some axes angle rotations around that. Something like this:
Vec3 dx = scale * (currPosX - prevPosX)
Vec3 dy = scale * (currPosY - prevPosY)
Matrix4x4 m1 = axesAngleRot(dx, camera up)
Matrix4x4 m2 = axesAngleRot(dy, camera right)

Matrix4x4 deltaRot = m1 * m2

Where scale is defined by you, and dx and dy give the delta coords of the mouse.
Then apply that to the object. You may have to tack on an extra transform to move back to origin (however you have that transform defined) before doing rotations. The cube you showed is at origin though.
Matrix4x4 finalTransform = inverseFinalTransform * deltaRot * finalTransform


Improved Laplacian smoothing of noisy surface meshes

This is a quite common property of smoothing in 3D. When you have some data and you smooth it out you get some kind of average of the local variation. And this works fine, and stably, in one dimensional graphs.

But when your entire data set is being smoothed over its coordinates the neighboring coordinates drag your data with them. Those again are dragged by their neighbors. The net effect is that the local variation of the surface as a whole is dragging itself towards an average that is stable in the smoothing operation itself. This will cause shrinkage because a closed object as a whole has a local average that is towards the common center of all the points in operation. Even in cases where you go locally outwards you are eventually caught up with the edges that drag you inwards.

In other words there is no structure to keep the surface locked to its place. The tendency to drag towards the center eventually wins because there is nothing to counteract this. There just simply are more opportunities to go inward than outward.


Volumetric Modeling usually refers to using a regular 3D grid of voxels, analogous to 2D pixels, to define properties of an object at points or small regions of space.

While more complex data might be stored at each voxel, often each contains a simple vector of values which can be interpreted directly as density or material properties for analysis and rendering. There is a whole literature on such techniques (search for Volume Rendering), often in the context of medical imaging where e.g. MRI scanners produce such data directly.

It is natural to consider extending techniques for generating images composed of 2D pixels to 3D voxels and modeling solids.


I strongly suspect that you are simply running out of floating point precision (at least in "U" dimension). With this sort of mathematics, the LSBs will soon become noisy resulting in the jagged appearance when you are (effectively) doing yes/no comparisons against known constants.

May I suggest instead that, rather than doing point sampling of your texture, try doing (at the very least) bilinear filtering which will hide the precision artefacts and also IMHO look better than having little squares.

UPDATE It just occurred to me that one of the possible causes of inaccuracy is that you are doing increments for each pixel X step, e.g. these two

w  += wStep;
texCoords += texCoordsStep;

These will soon cause errors to accumulate. Remove them and replace with multiplies by the X value and I expect things should improve.

UPDATE2 It will also be affected by how far away from the origin of the texture you are when you sample it. If, for example, your source texture is defined over [0..1]x[0..1] and then repeats outside of that range, you might expect sampling  at uv = (1000000.125, 100000.125) to be the same as (0.125,0.125) but, because the precision of FP decreases with increasing magnitude (i.e. the steps between floats also increases), you might end up with something quite different.


AI_MATKEY_COLOR_TRANSPARENT is used to define the transparent colour and AI_MATKEY_REFRACTI is the refractive index.


Apologies in advance for the poor quality of this answer, but this sounds a little like what an ex-colleague was doing as part of his PhD. The "Free-Viewpoint video" papers listed at the bottom of https://www.researchgate.net/profile/James_Imber might be useful, or at least a starting point to find related work.


What are the most efficient ways to convert between f32 and smaller float formats on the CPU?

You definitely want to check out meticulous half to float and float to half implementations by Fabian Giesen. I don't think there is anything faster than those:
https://gist.github.com/rygorous/2144712
https://gist.github.com/rygorous/2156668

For other formats you will find an implementation in DirectXMath (part of WindowsSDK), Mesa and similar libraries. For example, R11G11B10_Float conversions are implemented by DirectXMath in XMFLOAT3PK and by Mesa in u_format_other.c.


Yes, it is guaranteed by the GLSL language specifications (click me). Section "4.1.10. Implicit Conversions" says

  In some situations, an expression and its type will be implicitly converted to a different type.

The spec shows all possible implicit conversions inside a table, also int -> float. It further says:

  When an implicit conversion is done, it is the same conversion that would be done under explicit
  conversion, using a constructor.

e.g.

float a = 1;

is equivalent to writing

float a = float(1);

There is one sentence at the end of the section, that you should keep in mind:

  The conversions in the table above are done only as indicated by other sections of this specification.

If read it correctly, it says that you can not rely on implicit conversions in any kind of instruction you might come up with. However, regarding vec3 var = vec3(1, 0, 0) section "5.4.2. Vector and Matrix Constructors" says:

  If the basic type (bool, int, float, or double) of a parameter to a constructor does not match the
  basic type of the object being constructed, the scalar construction rules (above) are used to convert
  the parameters

So it calls the constructor float(int) before consuming each value.

Even though both cases are explicitly backed by the language specifications, my personal advice is to never rely on implicit conversions. They can lead to subtle, hard to track bugs. Not sure how it is in GLSL, since I didn't test it, but in C++ this one:

float a = 1 / 2; # <--- this evaluates to 0.f!

, deeply hidden somewhere in larger calculations, has cost me several hours of debugging more than once. It gets worse if one or both values of the fraction are taken from a variable. However, in C++ I can step through the code with a nice debugger, something that is not so easily done in GLSL. So if the only reason for using implicit conversions is laziness or some sort of taste-related decision, force yourself to be explicit. It will safe you a lot of pain.