Why don't Direct3D/Metal support texture swizzles?

Why is this feature present in OpenGL/Vulkan and not Direct3D/Metal?

It's present in OpenGL because someone thought it would be a good idea, wrote an OpenGL extension specification for it, released and supported the extension in their drivers, and the extension was eventually adopted into core profile. While it's usually pretty obvious why feature X is present in some API (because it's a thing that makes sense to have), it's generally hard to say why feature X is not present in some other API. There's a wide range of possible reasons. You would have to talk to whoever were the people responsible for the design of the API in question at the time to find out…

Concerning hardware support: In case of doubt, you can simply convince yourself by comparing rendering times with and without using a texture swizzle. You would notice if you were hitting a software path there. The fact that the extension was introduced by NVIDIA would lead me to the conclusion that there's at least one hardware vendor that is able to support this on their hardware; otherwise, why would they introduce it!? If you think about it: Texture sampling hardware already has to support lots of different formats (floating point, signed integer, unsigned integer, normalized signed integer, normalized unsigned integer, all for different numbers of components, component sizes, sRGB versions for some, compressed formats, …) including some where the order of components is different on the output side from what it is on the input side (e.g., BGRA). Adding a few bits to the texture descriptor to control which components go where in the output is probably not a big deal. I'm not a hardware engineer, but it would seem to me that it probably even makes the hardware simpler because it reduces the number of special cases. So I'd say chances are that hardware was already using a swizzle mask internally long before it was exposed on the API side…

Apart from that, while I don't know about Metal, Direct3D 12 does allow you to specify a shader component mapping when creating a resource view, which is effectively exactly the same thing as the OpenGL texture swizzle mask…


Yep, you've got that right. In Perlin's reference implementation of "improved noise", the noise will be periodic, repeating after 256 units along each axis. It's usually not very noticeable even if you have a large extent of noise visible, since there's no large-scale features for the eye to track.

But there's no particular reason it needs to tile after 256 units; that's pretty much just an arbitrary choice made by Perlin. He uses a permutation table as a hash function to map grid coordinates to gradient vectors, and the hard-coded table is of size 256, so has to be tiled after that.

However, any other hash function you like could be used as well. It just has to pseudorandomly map the integer parts of the x, y, z coordinates to a gradient vector.


If you are going for physics. You can do physically based rendering. Physically Based Rendering(PBR for short) is a filed of computer graphics where we take consider of physics(energy conservation, spectrum, etc) in to account and try to not do approximations.

The are two main sub fields of PBR.
(quick intro, layman's term below)

Production Rendering.
This is where accuracy is the focus. We do as little or no approximations as possible. There are algorithms in this field where if the wave-like property of light is ignored. Then the result generated by the algorithm is physically accurate. But each frame might take more than a few minutes or even hours to render. There are a lot of different topics actively being researched in this field(fast ray-scene intersection, better Monticello integration, fast/robust acceleration struct building)
Realtime Rendering.
In this field, speed is the key concern. Where accuracy is achieved by doing good approximations. But will never be physically accurate. Most research on this field is focusing on doing good and fast approximations.

These are my random ideas for fluids.

Fast Ray-Liquid intersection testing
Fast triangle mesh generation from particles.
Fast acceleration structure building from particles.
Simplification of liquid surface to generate simpler mesh

And some random ideas for general GC.

Improve sampling efficiency for sky maps in ray based rendering algorithm
Robust method of dealing with floating point accuracy problem in shadow maps

I don't know if the above topics are possible or not. Those are just ideas.


Why does DirectX use COM pointers and interfaces?

C++ has no defined ABI, so C++ programs can't link to libraries that were compiled with a different compiler. In addition, MS's Visual Studio C++ compiler is not ABI-stable, so you don't just have to use the same compiler: you have to use the same version that the library was compiled with.

This would be a huge problem for a widely-used library like DirectX.

COM doesn't just enable interoperability between different languages, but also between different C++ compilers. In fact, I'd say this is its main use, and language compatibility is less used. By being a COM library, DirectX can avoid application developers needing to care what compiler it was compiled with. Application developers are free to use a Visual Studio version of their choice, or a completely different C++ compiler.


A very novice mistake, I was compiling with the old HLSL compiler. But why would that be the default? Very strange.

For anybody wondering, just right click the HLSL file in the solution explorer and go to properties -> HLSL Compiler -> General, and switch Shader Model to the one you want which was newest one for me (5.0)


What is better to use for real-time computing Mesh - Plane intersection points, GPU or CPU?

doing the calculation to decide whether a point is on one side of a plane or the other is very simple (a single dot product). Doing that 3 times and having a special case when they don't match to split the triangle is pretty fast. It's also a parallel problem. The hardest part is reserving the space for the output.

Preparing the data to start computing this on the gpu is already enough overhead that it's not worth doing it on the gpu unless the mesh is very big or you are splitting it multiple times (due to the plane moving for example). Or the data is already in good enough condition to be sent over to the gpu directly through DMA without the cpu needing to touch it at all.


Pixel screen-space derivatives do drastically impact performance, but they impact performance whether you use them or not, so from a certain point of view they're free!

Every GPU in recent history packs a quad of four pixels together and puts them in the same warp/wavefront, which essentially means that they're running right next to each other on the GPU, so accessing values from them is very cheap. Because warps/wavefronts are run in lockstep, the other pixels will also be at exactly the same place in the shader as you are, so the value of p for those pixels will just be sitting in a register waiting for you. These other three pixels will always be executed, even if their results will be thrown away. So a triangle that covers a single pixel will always shade four pixels and throw away the results of three of them, just so that these derivative features work!

This is considered an acceptable cost (for current hardware) because it isn't just functions like fwidth that use these derivatives: every single texture sample does as well, in order to pick what mipmap of your texture to read from. Consider: if you are very close to a surface, the UV coordinate you are using to sample the texture will have a very small derivative in screen space, meaning you need to use a larger mipmap, and if you are farther the UV coordinate will have a larger derivative in screen space, meaning you need to use a smaller mipmap.

As far as what it means in less mathematical terms: fwidth is equivalent to abs(dFdx(p)) + abs(dFdy(p)). dFdx(p) is simply the difference between the value of p at pixel x+1 and the value of p at pixel x, and similarly for dFdy(p).


How to pass only the needed portion of the data calculated in a fragment shader to another fragment shader?

I'm assuming that you don't know how many objects there will be or have any way to index them until the image processing operations are completed, i.e. you have some final pass that determines for each pixel "do I want to put an object here, or no".
OpenGL ES 2.0 is pretty limiting. There may be no better way to do this than what you described: output from the fragment shader to a texture, pull it back to the CPU to extract the points, then put those into a uniform buffer for the second pass.
One thing you could attempt, though I'm unsure if it could be fully made to work in ES 2.0, is to do a parallel prefix sum over the output texture from the first pass. This would involve recursive fragment shader reduction passes (a little bit like generating mipmaps) that would first count up the number of objects within each region of the image, starting with 2×2 regions and doubling at each step until you've covered the full image, then go back down the mip chain and calculate the prefix sums, which are the number of objects "before" each region (using some defined ordering). At the end of that, you would have the exact index where to store each object into an output buffer.
FYI, in GL ES 3.1 and other more fully-featured APIs, there are shader atomic counters, which allow implementing an "append buffer"; your first fragment shader pass could output directly to this buffer, then turn around and use it for an input buffer, or for indirect draw parameters, on the next pass.


How does Next Event Estimation work with transformed lights?

Sampling is typically done in local coordinates first and then mapped to world space. You don't have to change the formula for this, but $\mathbf{p}'$ needs to be in world space when you compute the solid-angle-to-area Jacobian determinant. For a sphere for instance, you would have a warping function that maps two canonical uniform random numbers to a 3D point on the unit sphere. Only then would you apply the local-to-world matrix on this sample. If this matrix is a rigid motion (e.g. a combination of translations and/or rotations) with a possible scaling factor, then your samples will remain uniformly distributed in world space. However, if you apply a mapping that distorts areas (e.g. stretching), this will not be the case. It technically still "works" for sampling but some regions of your sphere can be overrepresented. You would have to implement an area-preserving sampling mechanism taking into account this stretching factor.

In practice, it's better to stick with rigid transformations and avoid non area-preserving mappings for analytical shapes. If you truly wish to sample a stretched sphere uniformly, your best bet would be to use an ellipsoid mesh. Then you could sample by area by first choosing a triangle proportional to its area and then picking a point on it by barycentric interpolation. In fact, this is what people have to do to sample area lights of arbitrary shapes.

There is a small discussion at the end pbrt that addresses the possibility of removing analytical shapes from a rendering engine. Analytical geometry primitives are fun because they can teach you stuff like subtended solid angle sampling when you learn GI and importance sampling, but they are limited and your question highlights one such limitation.


Your code is non-functional, but only because (in the core profile) you cannot bind a buffer to GL_ELEMENT_ARRAY_BUFFER unless a VAO is also bound. So the part that creates the buffer would need to use a different binding point, which is totally fine:

const GLuint indices[6] = {
    0, 1, 2,
    0, 3, 2
};
glGenBuffers (1, &EBO);
glBindBuffer (GL_ARRAY_BUFFER, EBO);
glBufferData (GL_ARRAY_BUFFER, sizeof(indices), indices, GL_STATIC_DRAW);

Everything else is perfectly valid.


Sampling the light vs. sampling the BRDF: difference in intensity

Shoot me... After the fixes I posted above, I started debugging Mitsuba to see whether it does anything differently on a large scale. Of course it didn't; my code is completely correct, the dimness is due to this macro def:

#define EPSILON   1e-5

This is used in divisions to avoid dividing with zero. Unfortunately, it accumulates to a large value during the calculations. Changing it to 1e-9 fixes everything:

For future generations I leave the source code up (just remember to change this value).

UPDATE:

I found another mistake in my calculations, but it isn't apparent in this scene. Microsurface-based BRDFs are usually sampled according to the microfacet normal distribution function (D(h, a), in this case GGX).

Don't forget however, that you also have a shadowing-masking function (G(l, v, a)) which has to be evaluated according to the chosen microfacet distribution. This means that I can't just mix-and-match f.e. GGX sampling and the Smith-Schlick evaluation; I have to use the Smith-GGX solution:

float G_Smith_GGX(float ndotl, float ndotv, float roughness)
{
    float a = roughness * roughness;
    float a2 = a * a;

    float lambda_l = sqrt(a2 + (1.0 - a2) * ndotl * ndotl);
    float lambda_v = sqrt(a2 + (1.0 - a2) * ndotv * ndotv);

    return (2.0 * ndotl * ndotv) / (ndotv * lambda_l + ndotl * lambda_v);
}

Which explains the UE4 approach, as:

vec3 F = F_Schlick(F0, ldoth);
float G = G_Smith_GGX(ndotl, ndotv, roughness);

// PDF = (D(h) * dot(n, h)) / (4 * dot(v, h))
return (F * G * vdoth) / (ndotv * ndoth + EPSILON);

See this article for the derivation.


How do I include all extensions my system supports using GLAD?

In this github issue the GLAD author offers the following way to correctly check if there is an extension is available at runtime:

  The defines just allow you to check at compile time which extensions
  (and opengl versions) are potentially available. After
  loading/initilaizing glad you can check with GLAD_
  if the extension is available at runtime (= supported by the driver
  and loaded).

// [initialize glad] // ...
if (!GLAD_GL_EXT_memory_object) {
    die("GL_EXT_memory_object not supported!");
}


How to get the 3D position for the point with (0,0) UV coordinates?

I realize that the solution I'm giving further below is basically the one you mentioned in your question. So let's begin by addressing this:

  The solution in question assumes that M° is linear.. if that was the case you can have:

  M°(P) = alpha*M°(A) + beta * M°(B) + gamma*M°(C)

  But that is not the case (correct me if I'm wrong).

The solution in question works directly in UV space. I don't see where it would make any assumptions concerning the nature or even existence of a "mapping from 3D to UV space"!? All it does assume is that UV coordinates vary linearly across each triangle (note: each triangle, not the entire mesh)…

$$
\def\mvec#1{\begin{pmatrix}#1\end{pmatrix}}
\def\ivec#1{\left(\begin{smallmatrix}#1\end{smallmatrix}\right)}
\def\mmat#1{\begin{bmatrix}#1\end{bmatrix}}
\def\vec#1{\mathrm{\mathbf{#1}}}
\def\mat#1{\mathrm{\mathbf{#1}}}
$$

In general, the $u$ and $v$ coordinates at a point on a triangle's plane given by the affine coordinates $\lambda_1$ and $\lambda_2$ are
\begin{align}
  u &= u_1 + \lambda_1 (u_2 - u_1) + \lambda_2 (u_3 - u_1) \\
  v &= v_1 + \lambda_1 (v_2 - v_1) + \lambda_2 (v_3 - v_1)
\end{align}
where $u_i$, $v_i$ are the $u$, $v$ coordinates at vertex $i$. We can rewrite this as
\begin{equation}
  \mmat{
    u_2 - u_1 & u_3 - u_1 \\
    v_2 - v_1 & v_3 - v_1
  } \cdot \mvec{
    \lambda_1 \\
    \lambda_2
  } = \mvec{
    u - u_1 \\
    v - v_1
  }.
\end{equation}
Knowing $u_i$ and $v_i$ at each vertex of a given triangle, we can compute
\begin{equation}
  \mat M = \mmat{
    u_2 - u_1 & u_3 - u_1 \\
    v_2 - v_1 & v_3 - v_1
  }.
\end{equation}
We can then find the affine coordinates corresponding to the point with whatever $u$, $v$ coordinates we want:
\begin{equation}
  \mvec{
    \lambda_1 \\
    \lambda_2
  } =
  \mat M^{-1} \cdot \mvec{
    u - u_1 \\
    v - v_1
  }.
\end{equation}
Note that these affine coordinates always exist as long as $\mat M^{-1}$ exists (which makes sense; there will always be exactly one point somewhere in the plane of the triangle where $u$ and $v$ reach whatever value you may be looking for as long as $u$ and $v$ vary independently). However, they will not necessarily fall inside the triangle.

So one way to find the point you're looking for would be to iterate over all triangles of the mesh and, for each triangle, compute $\mat M^{-1}$, and $\lambda_1$ and $\lambda_2$ for $u = 0$ and $v = 0$, and check whether
\begin{align}
  \lambda_1 &\geq 0 & &\land & \lambda_2 &\geq 0 & &\land & \lambda_1 + \lambda_2 &\leq 1.
\end{align}
If this is the case, you have found a triangle that contains a point at which $u = 0$ and $v = 0$. You can then compute the coordinates of that point as
\begin{equation}
  \vec p = (1 - \lambda_1 - \lambda_2) \vec v_1 + \lambda_1 \vec v_2 + \lambda_2 \vec v_3
\end{equation}
where $\vec v_1$, $\vec v_2$, and $\vec v_3$ are the coordinates of the respective triangle vertices.

It may make sense to, for example, use an initial check against a given triangle's bounding rectangle in UV space and/or other methods to quickly rule out triangles that cannot contain the coordinates of interest to speed up the search…


OpenGL back and front buffer - counting and adjusting the number of red pixels before displaying to screen

I have tried doing glCear(GL_BACK_LEFT) between each iteration but it is not clearing, I'm wondering if I'm missing a line of code before the glClear, or if I'm calling it right. I think the problem is that the back buffer isn't clearing

This comment of yours sheds some light on the matter. This function call doesn't do what you think it does and is actually wrong usage. The glClear function doesn't actually take the buffer identificator (that you give to glDraw/ReadBuffer) to clear rather than a mask of the general parts of the framebuffer to clear.

The framebuffer usually consists of one or more color buffers, a possible depth buffer and a possible stencil buffer. Therefore you give glClear an ORed combination of the respective flags GL_COLOR_BUFFER_BIT, GL_DEPTH_BUFFER_BIT and GL_STENCIL_BUFFER_BIT and those buffers (of the currently specified draw buffer for the color buffer) are then cleared.

So after setting the draw buffer with glDrawBuffer you call glClear to clear that. As said, the flags depend on what you want cleared (usually at least GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT, special algorithms notwithstanding).


You tile idea seems to be a good one to cut down on local complexity. You can't reasonably evaluate every primitive everywhere. Not if you want good performance anyway. So you need a simplified version for each tile, where you remove elements which have insignificant contribution. This is an interesting problem of its own. A simple version of this would be to:

Assign each primitive "usefulness bounds" beyond which it is considered 0. What exactly those are will depend on the curves you're using to map distance to world height. To make your life easier, pick something which actually goes to 0 over a finite distance. Or alter your favorite curve so it does.
For each tile, trim out primitives for which the bounds don't intersect the tile. You can do this with a simple loop to begin with.
Further simplify the operation tree. This is a little vague because you did not provide details on the kind of operations which will be in your function. But essentially, you want to do the like of replacing $x \times 0$ by $0$ and $x + 0$ by $x$, until you can't anymore. The zeroes come from the previous step where you removed some primitives.

You may need spatial hierarchies if you want this to scale to a world with millions of primitives. This can quickly become a complex topic so I'll leave it at that for now. Ask again when you get there :-)

For your more immediate problem, I suggest using the CPU to rewrite your SDF using reverse polish notation. This will allow you to write your shader as a simple interpreter, with two large buffers as inputs:

An integer buffer with commands (union, add, multiply, point primitive, segment primitive, etc).
A float buffer with operands and transforms for the primitives.

The shader is then just a loop over commands which consumes operands as needed. You'll need different command/operand buffers for every tile, obviously. Depending on the typical nature of your SDF, you may find it more efficient to have your primitives always perform union and have an explicit push command for the rare exceptions. There are a lot of ways to go about this really.

You should take care to make sure the resulting SDF can be evaluated with a reasonably small stack as this will be fixed in your shader. This is another problem by itself, perhaps well suited for the computer science SE, but probably not too difficult to solve on your own either.

I'm sorry if I have not provided enough OpenGL specific details but GPUs are not my specialty.


algorithm that openGL uses to construct 3D surface from vertices

Most 3D models are not just collections of points; they’re also the information about how those points are connected. Those are polygon meshes. What you’re describing sounds like a point cloud, which is what processes like 3D scanning often produce. That article has a “conversion to 3D surfaces” section; if you search for “point cloud meshing” elsewhere you should also find a bunch of information about ways to do this. One new-ish technique worth looking at, which is more robust than a lot of older approaches to the problem: Fast Winding Numbers for Soups and Clouds.
For OBJ files in particular, polygons are specified with a list of vertices using the f command as described here.


Screen coordinates, barycentric coordinates and global coordinateas

It sounds like you're rasterizing a line segment between two endpoints. The points on the line are obtained by linearly interpolating between the endpoints (whether in world space, screen space, or any other coordinate system). The barycentric coordinates identify points on the line segment.
It's defined this way to make range of $\lambda$ be [0, 1]. (When $\lambda = 0$ you obtain $(x_1, y_1)$ and when $\lambda = 1$ you get $(x_2, y_2)$.) This is typical for barycentric coordinates for either lines or for triangles—they always range from 0 to 1, and sum to 1. These coordinates therefore also act as weights for interpolating other quantities, such as color or z-value, across the line or triangle.
Probably the $p_i$ are the original 3D points before projection.
This is about perspective-correct interpolation. It's stating the transformation between $\lambda$ (interpolation in screen space) and $\mu$ (interpolation in world space). The two parameters are not the same, due to perspective foreshortening—in screen space, perspective will compress parts of the line that are farther away from the camera. So, for example, the midpoint of the line in world space ($\mu = \tfrac{1}{2}$) will generally project to some place that is not at the midpoint in screen space ($\lambda \neq \tfrac{1}{2}$). The equation given shows how to convert between $\lambda$ and $\mu$ for a given point on the line. Note that if $p_1^z = p_2^z$, the equation reduces to $\mu = \lambda$, showing that both kinds of interpolation match if and only if the line is parallel to the screen, so the perspective effect is uniform along the whole line.
$p_z$ is going to be the depth of the interpolated point. It's the $z$-component of the vector $p$, so $p_z$ itself is a single value, not a vector. Presumably the reason you want to know this is so you can store it in the z-buffer during rasterization. In other words, while rasterizing a line, you can easily calculate $\lambda$ for each pixel based on how far you are between the endpoints in screen space; then this equation tells you the $z$ for each pixel to put into the z-buffer.


Using imageLoad/imageStore to perform atomic add

If you could use imageLoad/Store to do atomic operations, there would be no need for atomic operations as a distinct class of operations. So no, you can't. There is no correct set of barriers or other properties that will allow this to work generally the way atomic operations do.


How can I control memory for command allocators and command queues in D3D12?

You don't.

These objects typically work with CPU memory, not device memory. And to the extent that they involve device memory, such allocations tend to be rare and/or fixed in size (a queue may have a small spot of device memory that the hardware queue reads commands from or something, but even that is implementation-dependent).

Vulkan allows you to give objects which may allocate non-device memory allocation parameters to allow you to allocate said memory for them. But even then, there is no guarantee that they will always allocate memory through your functions. Such systems can bypass your allocation routines, and only tell you allocator that it is allocating X bytes of memory (this is typically done for memory pages that have to be allocated in a special way). It doesn't let you actually provide it.

That documentation does not seem to be talking about controlling how those objects allocate memory, but when and how you create those objects. Command allocators and queues are critical, so you create X allocators and Y queues at the start and keep them around forever.


If the OS uses a frame-buffer with potentially different textures being displayed each frame why dont we?

The compositor in the OS wants to run at a consistent 60 FPS whatever rate your application runs at. If there were a single framebuffer for your application, then you might end up writing to it at the same time the compositor or overlay hardware wants to read from it to put it into the final frame. This would cause tearing. (Tearing could be avoided with a lock, but then the compositor might get stuck waiting for your frame to finish, which would also stop the user using the system GUI to get out of your slow/hanging app.)

This is pretty much the same reason double buffering is used when you're drawing to a monitor directly, just earlier in the pipeline.

This doesn't usually apply for processing within your application, because generally everything has to run at the same rate, and you can't run multiple stages in parallel. Say you draw some 3D stuff and then use a full-screen fragment shader to add some film grain and vignetting. It doesn't help if your full-screen shader can run faster than the 3D work, because you'll just be processing the same intermediate frames again.

I can think of two situations really where it might help. The first is if it's not a post-process but something different, such as if you have a GUI or HUD that you want to run at full frame-rate even if you get frame drops in the main content. (This is the case the OS's window compositor falls into.)

The second is if you can run the multiple stages of your processing pipeline in parallel, e.g. on multiple GPUs. That way, you can start rendering the next 3D frame while the full-screen shader is vignetting your previous frame. You'd have to know your hardware pretty well to split the work appropriately. On something like a gaming PC or graphics workstation that's using SLI to use multiple GPUs, you might find your parallel processing is slower because the workload is unbalanced and one GPU is idling a lot.


Why smoothed meshes in 3D studio end up with the same number of vertices/triangles? How then can they be smoothed with the same geometry?

Smooth in this case just makes the surface normals at vertices point the same way, when interpolated it looks smooth. Meshsmooth would add vertices.

1) how is the smoothing possible without increasing the detailing of the mesh geometry?

Human eyes cant actually see curvature except on the edges of objects. All they can do is approximate the smoothness and process the gradient slope. So having a continuous field does give a air of smoothness. The eye however is extremely sensitive to abrupt changes in color, and interprets that as a hard crease.

By interpolating the vertex normals your surface will get the appearance of smooth flowing. Since this normal is used to calculate the final reflected color you get a smooth color field.

Image 1: a flat shaded normal versus the normals of a smooth interplation. The black normal's lie on a vertex. The colored ones are interpolated.

There is nothing that says we need to do a linear interpolation. In fact by perturbing the normals we can cause the flat surface to change appearance. This is how bump mapping and normal mapping works. The effect can be convincing unless the surfaces edge plays a too big part in which place the illusion breaks.

Image 2: a flat shaded surface (back), Smooth shaded (middle) and a mapped smooth normal. The illusion of a wavy surface breaks because the edge plays so prominent part in the image, you could instead increase the normals

2) does the smoothing at least increase the memory allocated

Hard to say definitive things about the underlying graphics engine. The normals need to be emitted to the graphics card anyway, most likely this data is cached, but could be calculated on the fly (in both cases).

Since Max uses smoothing groups it seems to me that memory usage is constant regardless. Hard to say, even if its not cached then it wouldn't make a big difference. It makes the shader tiny a bit more complicated, but only just most likely this complexity is present use it or not.


Trick is, to move the entire object so that the point about which you want to rotate is at the center. Then rotate and after that counter move it so that the point is were it was.

In fact this is not so much of a trick, as such, nearly all graphics engines work this way. It is just abstracted away in many cases. Most often you will see it done in matrix notation like this:

$$ T^{-1} R T $$

Where T is to transformation from world to pivot and R is the rotation matrix. Since it can be easily packaged this way you end up with a function called rotate about the pivot point. This is usually the reason why drawing API's allow for a transformation matrix.

Image 1: Rotate about a pivot point, image source available for investigation. Sub images in RTL reading order: Original, inverse transform, inverse and rotate, inverse rotate and move back.


Hard shadows are simple that only needs a point light.

How it's done is by rendering the scene from the point of view of the light and only keep only the depth information. This is the shadow map.

Then when doing the actual rendering you calculate the point on the triangle in world space and find where it would be on the shadow map. Then you sample the depth from the shadowmap at that point and compare it to the distance to the light.

Soft shadows are harder. They require figuring out how close you are to the edge of what would be a hard shadow and adjusting the light value based on that. There are several techniques for that. One of which is simply sampling multiple times.


Can someone explain this formula for parse RGB to HSL?

First it converts the red, green, and blue values from the 8-bit unsigned integer range to floating point values between 0 and 1. It then figures out which component is the maximum, which is the minimum and the distance between the minimum and maximum.

If the min and max are equal, then the saturation is 0, and it makes no sense to calculate a hue. Here they are setting the hue to 0, but in reality, it's simply undefined. Assuming that no hue equals 0 hue can cause problem if you later manipulate the colors, so be careful with that.

If the min and max are not equal, then we can calculate a hue. The shape of the HSL color space is a double hexagonal cone. What the next part does is attempt to figure out which segment of the hexagonal cone the hue is in. If the maximum of the red, green, and blue, was the red channel, then it's in the area bounded by -60° to +60°. If green is the max, then the hue is between 60° and 180°, and if blue is the max, then the hue is between 180° and 300° (aka -60°).

The division in the hue calculation generates a value between -1 and 1. If red is the maximum component, then green - blue is either the middle value minus the lower value, or it's the lower value minus the middle value. And since the middle and lower value are never greater than the maximum, dividing by the delta will always give a value between -1 and 1. Once you have a value between -1 and 1, you multiply it by 60 to get the degrees. For the other 2 cases, you add either 2 or 4 before the multiply in order to rotate it 120° or 240° for the green and blue.

Next it calculates the lightness by simply averaging the min and max components. (This is a very poor way to represent the perceived brightness of a color, by the way.)

Finally, it calculates the saturation. If the delta is 0, then saturation is 0. Otherwise it divides the delta by 1 - abs(2 * L - 1). Essentially, it's saying that saturation is proportional to the distance between the minimum and the maximum values and inversely proportional to the lightness's distance from 50% lightness. In other words, as you move away from the vertical center of the double hex cone, towards the pointy ends, the distance between the min and max components has a bigger effect on saturation. A difference in the delta of, say, .1 at the center will not show much difference in saturation, but when the lightness is very close to either 0 or 1, it will make a large difference in saturation.


How do I calculate average hue, since the numbers loop around?
I will get silly answers sometimes. For example, in HSB/HSL, the average of hue=10 (slightly orange red) and hue=350 (slightly blue red) would arithmetically be hue=(10+350)/2=180 (cyan).

How to calculate mean and standard deviation for hue values from 0 to 360?
 and Averaging circular values (particularly Hues in HSL color scheme) which provides C++ source code.

What if the values were 0° and 180°, then there are two answers: 90° and 270°; the same thing applies whenever the two inputs are opposite, there's two answers and no clear choice.

The program "Image Color Summarizer" (Linux version) has a file called "peravg.pm" which says: "compute the average and median of a periodic data set, like a set of angles or hues". The main page offers a Windows version and a webpage image upload along with an API.

The author explains:

The colour summarizer will produce descriptive colour statistics for an image. Reported will be the average, median, minimum and maximum of each component of RGB, HSV, LCH and Lab. Average hues are calculated using mean of circular quantities.
Some of the questions the summarizer will answer are:

what is the average color hue, saturation and value in my image?
what are the colours that are most representative of the image?
what is the image's human readable colour description (e.g. dark pure blue)?
what do the RGB, HSV, LCH and Lab histograms look like?
what are the RGB, HSV, LCH and Lab pixel values?

The paper: "Perceiving the average hue of color arrays" (May 1 2014), by Jacquelyn Webster, Paul Kay, and Michael A. Webster examines the subject from the viewpoint of an observer's perception of color.


Yes, it's a meaningful stat: GPUs have dedicated triangle setup HW and the rate is measured in triangles/GPU clock. According to white papers available on NV's website, the 680 (Kepler) could issue one triangle per SM every other clock - with 8 SMs, this yielded 4 triangles/clock. The Maxwell white paper doesn't indicate a change in this rate per SM - the 980 has 16 SMs so, if there is really is no rate change per SM, it can produce 8 triangles/clock. While the 980 has 2048 CUDA cores, the 950 has 768, implying 6 SMs and 3 triangles/clock. The chip runs around 1 Ghz, so the 950 is probably limited to 3 billion triangles per second.


GIF animation compressors normally do this automatically, using transparency in a given frame to avoid storing what has not changed from the previous frame.

The reason it's not working in your case is that your input is bad. The flat part of the parchment is not really static. It is moving slowly upwards, at a rate of about 1 pixel every 20 frames. Fix your animation so the non moving parts are perfectly identical between frames and you should get a much smaller file.

You would also get a smaller file if you avoided the noise in the unfurling part (most visible in the shadow). Such noise barely compresses at all. This looks like it came out of some 3D software so crank up the quality settings until the input is noise free.

If dithering, which is really added noise, is used when producing the images, make sure it does not change from frame to frame. Or just turn it off entirely. You might get a little banding but the file will be smaller. You'll need to decide which you prefer.


In a traditional camera, the photons from the scene travel through the lens of the camera, then hit the sensor at the focal length. A consequence of the lens is that the image is upside down and backwards.

In ray tracing, we can optimize this by imagining the focal plane is in front of the camera. You can think that the photons from the scene travel towards the camera hole, and resolve on the image plane.

Another optimization in ray tracing to to trace rays from the camera into the scene, rather than tracing them from light sources and hoping they hit the camera. Therefore, the first rays we shoot out are the ones that go from the eye through each pixel on the virtual screen.

In order to create the ray, we need to know the distances a and b in world units. Therefore, we need to convert pixel units into world units. To do this we need a define a camera. Let's define an example camera as follows:
$$origin = \begin{bmatrix} 0 & 0 & 0 \end{bmatrix}$$
$$coordinateSystem = \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1 \end{bmatrix}$$
$$fov_{x} = 90^{\circ}$$
$$fov_{y} = \frac{fov_{x}}{aspectRatio}$$
$$focalDist = 1$$

The field of view, or fov is an indirect way of specifying the ratio of pixel units to view units. Specifically, it is the viewing angle that is seen by the camera.

The higher the angle, the more of the scene is seen. But remember, changing the fov does not change the size of the screen, it merely squishes more or less of the scene into the same number of pixels.

Let's look at the triangle formed by fovx and the x-axis:

We can use the definition of tangent to calculate the screenWidth in view units
$$\tan \left (\theta  \right) = \frac{opposite}{adjacent}$$
$$screenWidth_{view}= 2 \: \cdot \: focalDist \: \cdot \: \tan \left (\frac{fov_{x}}{2}  \right)$$

Using that, we can calculate the view units of the pixel.
$$x_{homogenous}= 2 \: \cdot \: \frac{x}{width} \: - \: 1$$
$$x_{view} = focalDist \: \cdot \: x_{homogenous} \: \cdot \: \tan \left (\frac{fov_{x}}{2}  \right)$$

The last thing to do to get the ray is to transform from view space to world space. This boils down to a simple matrix transform. We negate yview because pixel coordinates go from the top left of the screen to the bottom right, but homogeneous coordinates go from (-1, -1) at the bottom left to (1, 1) at the top right.
$$ray_{world}= \begin{bmatrix} x_{view} & -y_{view} & d\end{bmatrix}\begin{bmatrix} &  & \\ & cameraCoordinateSystem & \\ &  & \end{bmatrix}$$

You can see an implementation of this here (In the code, I assume the focalDist is 1, so it cancels out)


Does cosine weighted hemisphere sampling still require NdotL when calculating contribution for indirect light?

You always need to multiply by the cosine term indeed (that's part of the rendering equation). Though when you do indirect diffuse using ray-tracing and thus monte-carol integration (which is the most common technique in this case), you have to divide the contribution of each sample by your PDF. This is well exampled here.

Note also that in the mentioned reference, if the PDF has terms that you also find in the rendering equations then you can optimise the code if you wish by cancelling out these terms.

  Don't forget that the BRDF of a diffuse surface is ρ/π where ρ stands for the surface albedo. So we need to divide the result by π. Though in the case of the indirect diffuse component, don't forget that we should have divided the result of castRay by the PDF of the random variable, which as we showed earlier in this chapter is 1/(2π). Dividing indirectDiffuseby 1/(2π) mis the same as multiplying this value by 2π. And since the albedo is also divided by π we can simplify the code...

You have a similar situation. If you look at the PDF for the cosine sampling, then you will realise that terms can be cancelled out. Which doesn't mean they are 'not' strictly necessary. They are, they just cancel each other out which allows to optimize the code slightly (and avoid a few division, multiplication, etc.). You are more in the micro-optimisation here... which can be confusing if you try to learn the theory by just looking at optimised code (which is often not properly commented).

$ \dfrac{(cos(\theta) ... )}{PDF} = \dfrac{(cos(\theta) ... )}{\dfrac{cos(\theta)}{\pi}} = ... $


Iridescence/Thin-film interference integration as a layer

Unfortunately, the iridescence model is not made to be applied to a diffuse term. Pascal and I made it for microfacet models only (that is the specular term).

One way to understand how to include it to a game engine might be to look at Unity's HDRP implementation.

In the Lit.hlsl to see how to incorporate the iridescence Fresnel into a specular + diffuse BRDF:

https://github.com/Unity-Technologies/ScriptableRenderPipeline/blob/master/com.unity.render-pipelines.high-definition/Runtime/Material/Lit/Lit.hlsl

PreLightData GetPreLightData(float3 V, PositionInputs posInput, inout BSDFData bsdfData) {

    [...]

    if (bsdfData.iridescenceMask > 0.0)
    {
        bsdfData.fresnel0 = lerp(bsdfData.fresnel0, EvalIridescence(topIor, viewAngle, bsdfData.iridescenceThickness, bsdfData.fresnel0), bsdfData.iridescenceMask);
    }
}

See the implementation of the EvalIridescence in BSDF.hlsl:

https://github.com/Unity-Technologies/ScriptableRenderPipeline/blob/30f1b16c8094ca133c165e56380ebb7ab9e7c0d9/com.unity.render-pipelines.core/ShaderLibrary/BSDF.hlsl

What they are doing is to blend the standard Fresnel term with the iridescence one based on a iridescence mask value.

I hope this is helpfull.


Transfer the texture between two different sets of uv

Are the two sets well correlated? This would boil down to having a good set of vertex attributes for each set of uv's. If each set of uv's in mapping 1 has a know set of mappings in set 2 then it should be relatively straight forward to read from the first set of uv's and write to the second set.
If this is as simple as vertex 1 has a mapping in uv set 1 and a different mapping for uv set 2 then read from set one then resample the texture for set 2.
On the other hand if the two sets are completely uncorrelated then it gets more complex:
A fully generic brute force solution would be to read 1 of the vertex attributes of the known mapping. Then search the entire set of uv's for set 2 filtering out those vertex positions that overlap with the current uv under consideration. Then use that list to read, resample and write the new texture.
The first case is easier to implement on the gpu. The second case is more difficult and may be better to implement on the cpu for debugging then move to the gpu as performance requirements dictate.
Edit for the first case:
This is the easy case, Use the uv coords from set 1 on the gpu just like you would normally to read the texture. Then use the second set as actual vertex positions adding a z value that is some constant. Don't use a MVP matrix just go straight to clip space. The gpu will worry about the resampling. In the end you will have a fullscreen quad with the final mappings. Then just read the image back from the gpu.
More detail:
The vertex attributes for this would be:
Vertex Position = vec2{u_set2, v_set2}
uv coords = vec2{u_set1, v_set1}

The vertex shader:
vec2 remapping = vertex_position * 2.0 - 1.0; // remap x,y to -1 to 1
gl_position = vec4( remapping, 0.5, 1 );
out texcoord = uv_coords_in;

The fragment shader:
gl_color = texture( image_sampler, tex_coords);


What you are talking about is transform feedback. With transform feedback you can capture primitives (vertices, triangles, lines) in a buffer such they can be used in subsequent draw calls. You don't have to grab the data from the GPU to the CPU and back again as you can directly use the GPU buffer you have just rendered to.
I'm not exactly sure what your use-case exactly is, but you mention simulation and timesteps. Imagine a particle system where each particle has a position and a velocity at each time step of the simulation. We have two buffers A and B. A will be rendered from on even numbered frames and will be written to on odd-numbered frames and B that we write to on even numbered frames and render from on odd-numbered frames. We also have a simple vertex shader that updates the positions of vertices using the aforementioned position and velocity attributes.
Then using some initial position and velocity for each particle we can start rendering from buffer A into buffer B. After having finished this process all particles will have an update position and we can render from buffer B into buffer A.
With this you also have the option to not render to the at all screen by using the GL_RASTERIZER_DISCARD functionality.
Take a look at the examples listed on this page under 'Transform Feedback'. Or at this example.


ping pong between same framebuffer two color attachments

Actually, GL_COLOR_BUFFER_BIT also need to be cleared

After creation of the framebuffer, draw buffer need to be set to:

glDrawBuffer(GL_COLOR_ATTACHMENT0);

and then after each swap:

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

instead of only

glClear(GL_DEPTH_BUFFER_BIT);

To clear current attachment 0.
I don't really understand these clearing bits extensively but this seems to fix blinking "chess board" artifact.

Borders seems to be connected with motion blur shader.


How to calculate vertex normals on a mesh with non-planar polygons

If you're interested in vertex normals specifically, there's an easy answer even for non-planar polygons that avoids the question of defining what the exact surface is: for each vertex, calculate the normal of the plane formed by the two edges entering and leaving that vertex.
More formally, given vertices $\mathbf{v}_1, \mathbf{v}_2, \ldots \mathbf{v}_n$ with counterclockwise winding, define the normal at the $i$th vertex as:
$$
\mathbf{n}_i = (\mathbf{v}_i - \mathbf{v}_{i-1}) \times (\mathbf{v}_{i+1} - \mathbf{v}_i)
$$
(where the indices wrap around).
You can then proceed to accumulate normals calculated this way from all the faces that share a given vertex, as usual for smooth shading.
As has been discussed, there are multiple ways to define a smooth surface corresponding to a non-planar polygon, but any reasonable way of defining such a surface must converge to the normals as defined here near the vertices, or else the surface can not both be smooth (locally flat) and meet the straight-line edges between the vertices.
A caveat to this approach, though, is that it won't define a normal for collinear vertices (where the entering and exiting edges are parallel), since the cross product goes to zero there. If this is a problem, it might work to patch up such vertices by interpolating normals to them from the surrounding non-collinear vertices.


How to calculate interpolation between multiple skeleton poses?

I will suggest something for the rotational part but I do not know how fast it is. Assume you have a 3D coordinate system centered at the pivoting point of the joint and fixed with respect to one of the bars that meet at the joint. The other bar moves in this coordinate system always attached the origin of the system. So its motion is a curve on a sphere centered at the joint's pivoting point and radius the bar's length $l$. Assume you have a sequence of positions $\vec{x}_0, \, \vec{x}_1, \, \vec{x}_2, \, ..., \vec{x}_m$ of the moving bar with respect to this system. Recall the bar has length $l$. Then calculate the cross products:
$$\vec{n}_j = \vec{x}_0 \times \vec{x}_j$$ and the cosine of the angle $$c_j = \cos(\theta_{j}) =  \frac{1}{l^2} \, (\vec{x}_0 \cdot \vec{x}_j) $$ as well as the magnitude $\|\vec{n}_j\| = \sqrt{ (\vec{n}_{j} \cdot \vec{n}_j) }$. Normalize
$$\vec{N}_j = \frac{\vec{n}_j}{\|\vec{n}_j\|}$$
So now you have the list of pairs
$$\big(\vec{N}_1, \, c_1\big), \, \big(\vec{N}_2, \, c_2\big), \, ... ,\, \big(\vec{N}_m, \, c_m\big)$$
Apply a cubic or some other appropriate spline interpolation to the list of data. The result is the pair of smooth families
$$\vec{N}(t), \, c(t)$$
Use  Rodrigues' rotation formula to obtain the smooth family of rotation matrices
$$U(t) = \text{Id} \, \pm \, \Big(\,\sqrt{1 - c(t)^2}\, \Big)\, \hat{N}(t) \, + \, \big(1 - c(t)\big)\, \hat{N}(t)^2$$ where $\hat{N}(t) = \big( \vec{N}(t) \times \cdot \big) $ is the three by three skew symmetric matrix that acts on a vector $\vec{x}$ by
$\hat{N}(t)\, \vec{x} = \vec{N}(t) \times \vec{x}$. Notice the presence of the $\pm$ sign, which is negative when the angle between the vectors $\vec{x}_0$ and $\vec{x}_j$ starts to exceed $180^{\circ}$. This really depends on the range of motion of the joint. Anyway, then the joint moves smoothly by
$$\vec{x}(t) = U(t)\, \vec{x}_0$$ and at each moment of time $t_j$ for which
$c(t_j) = c_j$ and $\vec{N}(t_j) = \vec{N}_j$ due to the interpolation scheme we get
$$\vec{x}_j = U(t_j)\, \vec{x}_0$$
All of this can be done with quaternions, the idea is analogous. It is up to you.

The case of framed joint: If you have one coordinate system attached firmly to the first bar of the joint and a second coordinate system attached firmly to the second bar of the joint (both have origin at the pivoting point of the joint, i.e. I am ignoring translations), then the position of the second bar with respect to the coordinate system of the first one is described by the position of the second coordinate system with respect to the first. Then, if $\vec{u}_1, \, \vec{u}_2, \vec{u}_3$ are the unit orthogonal column-vectors of the second coordinate system written with respect to the first coordinate system, the orthogonal matrix that maps the first system to the second is $U = [\vec{u}_1 \,\, \vec{u}_2 \,\, \vec{u}_3]^T$ (the vectors are 3 x 1 columns of coordinates of the basis vectors of the second coordinate system with respect to the first coordinate system). Consequently, in this case (of framed joint motion) you may actually have a sequence of orthogonal matrices, describing the positions of the second system relative to the first at different times:
$U_1, \, U_2, \, ..., \, U_m$. In this case, calculate the skew symmetric matrices $$\hat{n}_j = \frac{1}{2} \big( U_j - U_j^T \big)$$ and calculate the traces
$$\|\hat{n}_j\| = \sqrt{ - \frac{1}{2} \, \text{trace}\big(\hat{n}_j^2\big) }$$ Then
$$\hat{N}_j = \left(\frac{1}{\|\hat{n}_j\|}\right) \, \hat{n}_j$$ is a skew symmetric matrix with the property that there exists a unique unit vector $\vec{N}_j$ such that for any vector $\vec{x}$
$$\hat{N}_j \, \vec{x} = \vec{N}_j \times \vec{x}$$
and
$$c_j = \cos(\theta_j) = \frac{\text{trace}(U_j) - 1}{2}$$
As before, you have the list of pairs
$$\big(\vec{N}_1, \, c_1\big), \, \big(\vec{N}_2, \, c_2\big), \, ... ,\, \big(\vec{N}_m, \, c_m\big)$$ or equvialently the list of pairs (whichever works better for you, although they are equivalent)
$$\big(\hat{N}_1, \, c_1\big), \, \big(\hat{N}_2, \, c_2\big), \, ... ,\, \big(\hat{N}_m, \, c_m\big)$$
Apply again a cubic or some other spline interpolation to either of the lists of data. The result is the pair of smooth families
$$\hat{N}(t), \, c(t)$$
Just as before,  by the properties of rotational matrices, you obtain the smooth family of rotation matrices
$$U(t) = \text{Id} \, \pm \, \Big(\,\sqrt{1 - c(t)^2}\, \Big)\, \hat{N}(t) \, + \, \big(1 - c(t)\big)\, \hat{N}(t)^2$$ This $U(t_j) = U_j$ for all $j = 1, .., m$.


Why does my image has a square pattern in cuda path tracer?

I managed to fix it. It was indeed related to repeating random sequences.
The problem was the following. curandState* randState is an array of curandStates, and most calls to curand_* require a pointer of a curandState.
I was sending the randState, the array, to curand_* functions and not for example, a pointer of one of its members.
Now most of my images are like this:


Is there any open code to get information about COLLADA or OBJ files?

For OBJ Files

Free tools like MeshLab, Blender, and GLC_Player are able to load OBJ files and give you basic information about their content.

Apparently, GLC_Player can also load COLLADA.

In code you can use the following standard tools:

Assimp - Library for loading many common formats including OBJ
TinyObjLoader - .h/.cc only, no external dependencies


Is gamma still important to take into account?

Yes, while many screens and OS operations are using a gamma of 2.2 your hardware and computation result still need to be corrected. There are also special mediums such as broadcast TV's that have a different gamma. Sensor equipment like cameras are mostly linear so they need to be adjusted accordingly. Besides this the user can set their system to whatever gamma they like.

In computer graphics primary reason to account for gamma is that your computation is most likely done in linear space* or summing light contributions becomes unnecessarily complicated. In any case gamma is really a simplification of things, you'd be much much better of doing profile to profile conversions if its possible to invest the computational time to do so.

Mathematical explanation

This may be a simpler to understand. Your monitor has a display error of $g(x)$ which is no problem if your data is conforming to $g(x)$, however if you want to calculate:

$$ a+b+c+d $$

and $a$, $b$, $c$, $d$, have the error of $g(x)$ you would actually need to calculate.

$$ g(g^{-1}(a)+g^{-1}(b)+g^{-1}(c)+g^{-1}(d)) $$

You would need to do this for each sub element over and over again. Therefore instead of running the transform each time you transform all once to linear and then once back.

And finally a artistic reason

It may just look better with a different gamma. Many games have a gamma adjustment for this reason.

* Worst case is that you think your computations as linear but do not compensate for the monitors nonlinear output charachteristics.


How is motion blur implemented in modern realtime renderers?

There are several techniques used.

A simple, but limited,  post-process approach that is not really used any more consists in reconstructing the world space position of a pixel using both the view projection matrix from current and previous frame. Using these two values you can compute the velocity at a pixel and blur accordingly, sampling along the velocity vector with how many samples you want ( the more sample the blurrier, but the more expensive as well).

This method unfortunately takes into account just the camera movement and therefore is not accurate enough if you have a dynamic scene with fast moving objects.

As you mentioned, other more modern techniques use a velocity buffer.

This velocity buffer can be created during the base pass in the deferred pipeline transforming each vertex using the world-view-projection matrix of the previous frame and then computing in the pixel shader the velocity vector using the screen space position of both frames.
With this velocity buffer you can, in a post-process step, sample across the per-pixel velocity direction like in the approach I described above.  The motion blur that you will get is obviously not camera-only; however you get an overhead in terms of memory (you need to keep an extra buffer) and time (you need to render additional info per pixel).

One technique based on velocity buffer that I particularly like is McGuire et al. that IIRC, is used with some variations in Unreal Engine 4 and many other games.

From [McGuire et al. 2012]

They divide the scene in tiles, assuming that each of these have a dominant velocity. Then, starting from the full res velocity buffer, they locate the velocity vector with higher magnitude for each tile, producing a downsampled version of the velocity buffer where 1 pixel = 1 tile.
Using this buffer, in another pass, they can generate a buffer that stores for each pixel/tile the dominant velocity among the neighbourhood pixels.
Note that the tile size is based on the max blur radius.

Using these buffers they then proceed with the gathering step sampling in the neighbourhood dominant velocity direction as this is the most plausible direction of the blur for each sample (as I said, the tile size is the max-blur radius). The weighting used for the blur takes into account the full res velocity and depth informations. Note that the paper propose three different weighting for three different scenarios (blurry over sharp, sharp over blurry and blurry over blurry).

Another important advantage of the method is that you can early-out if the dominant velocity in the neighbourhood is smaller than a certain threshold.


inside and outside region identification from 2D contour

In general, I recommend the book Real-Time Collision Detection.

For your particular case, my first choice would be to use the 3D test for point-inside-mesh. I assume that the contour is a sequence of 2D segments (a sequence of vertices, essentially) and that the model mesh is "sealed", i.e. there are no holes in its surfaces that would make distinguishing the "inside" and "outside" of it impossible.

Find a bounding box for your mesh.
Cast rays from an edge of the bounding box to the tested point. In your case, the ray would be coplanar with the slicing plane, and the point would be one of the vertices of the contour.
Count how many times the ray crosses mesh triangles.
Odd number of crossings means that the point is on the inside; even number means it's on the outside.
Repeat points 2-4 for all contour vertices.
Classify contour regions by whether their vertices are on the inside or outside.


Problem sampling uniformly envmap with GGX

How are you randomizing the samples you take? It looks to me like maybe the random values are the same across all pixels in the image, which would explain why you see 10 coherent bright spots when you have 10 samples.
The samples should be re-randomized for each pixel. This should produce a noisy result, with a single blurred-out bright spot instead of 10 (or however many) coherent bright spots.


What's the difference between Sample and tex2D

They represent the same operation, but Sample is what it's called in D3D10 and newer versions of HLSL, while tex2D is what it's called in D3D9 HLSL, and NVIDIA's (defunct) Cg language. By the way, the operation is also called texture in GLSL.


How much should I rely on Geometry shaders in WebGL?

WebGL doesn't even currently support geometry shaders, so to directly answer the question: 100% of all users.


What are the current open problems in Computer Graphics?

There is a vast amount of open problems in real-time graphics: shadows, aliasing, reflections, global illumination, transparencies (blending order and lighting) etc.

SIGGRAPH annually hosts a course called "Open Problems in Real-Time Rendering", which describes current issues in real-time graphics (mostly game dev industry standpoint). You can find last year's materials here: http://openproblems.realtimerendering.com


Does PBR incur a performance penalty by design?

As mentioned in this answer, Physically-Based Rendering isn't a set number of things. It's a concept. It's akin to saying something is 'Environmentally Friendly'. There are many different techniques to be environmentally friendly and someone can implement those techniques to varying degrees. The same is for PBR.

In the end, Physically Based Rendering is just that, based on Physics in some way, shape, or form. One of the most basic 'physical' rendering techniques I can think of is using Lambert shading. While you may say, "Well that doesn't take into account specular light, or etc, etc. etc.", I agree. Lambert shading is a crude representation of the physics of light. But, pedantically, it is PBR, because it is based off the physical observation that light intensity decreases with viewing angle.

A renderer can implement as many, or as few PBR techniques as they choose. Some can be very computationally expensive, others, quite cheap. They can also choose to enable or disable certain techniques for different targets. For example, UE4 has a different set of techniques for mobile games, than for high end PC games.

Now, that all said, when today's modern renderers talk about being "Physically-based", they are generally referring to using a modern BRDF, generally some form of Cook-Torrance, and physically plausible inputs. Inputs being: materials with physically-plausible color values and lights with physically based intensity values.

Using a modern BRDF can be very computationally expensive, but, there are many approximations that can make it more friendly.

Using physically-plausible inputs is generally not that difficult to implement on the renderer side, rather it's more difficult for artists. IE. teaching them what the different material values mean, and what are physically plausible values. This can be a huge task, depending on your artists, as seen in the presentations here, here, and here, for example. In my opinion, this is the hardest task in switching a studio.

Edit:

One last issue, as brought up by @NathanReed in the comments:

Implementing a modern BRDF in the shader doesn't necessarily immediately make the rendering better. In many cases, it can make other parts of the pipeline a glaring issue. Below are some additional things that are important in getting the most out of a modern BRDF:

HDR lights and doing all rendering calculations in HDR
Gamma correct rendering
Indirect lighting of some kind (skybox lighting, static environment probes, screen space reflections, etc)

This is huge for metals. Metals have no diffuse reflectance, so most of the surface of the metal will be black if you don't have indirect reflections
This is also one of the more difficult things to get real-time, especially for dynamic scenes.


In HLSL, you can use the built-in fmod function for this.

I'm not sure what's going wrong with your implementation, though. It looks mathematically correct. It's conceivable you're encountering a precision issue with the subtraction, depending on the inputs.

BTW, an alternative implementation which avoids the subtraction is:

float Mod(float x, float y)
{
    return frac(x / y) * y;
}


How to calculate attenuation and radiance in PBR? (point and directional lights)

How should I define the radiance, if I have a directional light, for example the sun?

The radiance in physics has different definitions in different books. I will use the definition with irradiance and radiant flux here.

Imagine photons having some energy $Q$. Your radiant flux $\Phi$ is the amount of energy $Q$ (and therefore the amount of photons) per unit time $t$. To get the energy for an instant (an infinitesimally short time), you use the change ($\Delta$) of energy over the change of time for $\lim_{t\rightarrow0}$ i.e.

$$\Phi = \lim_{t\rightarrow0} \frac{\Delta Q}{ \Delta t} = \frac{dQ}{dt}$$

Energy is measured in Joules and time in seconds, therefore radiant flux is measured in $\frac{J}{s}$.

This however doesn't say anything about the surface you're measuring the energy at, but if you want to light an object, that needs to be taken into account.
Therefore, you define irradiance $E$ as your radiant flux per area $A$ (note that irradiance usually means energy arriving at an area, whereas radiosity is energy leaving the area, but is otherwise defined exactly the same way). Since some integrations later on are easier to do with infinitesimally small area patches, we go that route again, but I will not rewrite the limes stuff all the time.

$$E = \frac{d\Phi}{dA}$$

Note that it is assumed, that the photons' traveling direction is assumed to be perpendicular to the surface.
Irradiance is thus measured in $\frac{J}{sm^2}$.

To understand radiance, you need to understand solid angles. I will just assume that you do, otherwise read up on it, this is really an important concept for physically based lighting.

The problem with irradiance is, that you cannot take into consideration any directionality of the light. The directionality can be expressed through solid angles $\omega$, where an infinitesimally small solid angle $d\omega$ can be seen as a vector.

Imagine that you have a sheet of paper parallel to another sheet of paper, one emitting and one receiving light, both equal in size. You can express all you need with irradiance. Now let the emitting sheet of paper transform into your light bulb centered at where the emitting sheet of paper was before. All of a sudden, it makes a difference, if you measure the light at the corners of the receiving sheet of paper or at its center.

Therefore, to take directionality into account, you define radiance $L$ as the irradiance per solid angle of the light. $\omega$ is the solid angle and we want to use the infinitessimally small one again, thus $d\omega$.
$$L = \frac{dE}{d\omega}$$
Thus your radiance is measured in $\frac{J}{sm^2sr}$ where sr is the solid angle unit steradians and really is dimensionles.

Looking at the definitions of Watts it's $W = \frac{J}{s}$ , therefore radiance measurement can be seen as $\frac{W}{m^2sr}$.

Edit 04th of August 2017:
The radiance in the end is exactly what you calculate as a pixel's color, it is what you see. In the real world, it is wave length dependend, but of course in computer graphics it is an RGB color vector. Therefore, you don't define radiance, you calculate it.

  What should the lightColor be, if the light source is a 60W lightbulb (780 lm) and in my scene one unit is one cm?

Light usually uses different wave lengths lambda which determine the color of the light. For example, imagine a photon of red light, i.e. $\lambda=700nm$ , then it has the energy of $\frac{hc}{\lambda}$ where $h$ is Planck's constant and $c$ is the speed of light, leading to about $2.838\times10^{-19}J$ for this red photon.

If your light bulb "uses" just one single wave length and is a $60W=60\frac{J}{s}$ light bulb, then you can calculate the energy, radiant flux, irradiance and finally radiance... If you know the exact spectral emission, you can also calculate this.
However, that is probably not really what you are looking for.

Edited 04th of August 2017:
Therefore, your (or rather the tutorials code) to calculate radiance is an oversimplified way of modeling the lightsource.  It does not take into account the power of the light source directly, it is just modelled by a color vector.

What the tutorial is asking you to do is to think of the color you want your light to be, and write down the RGB value for it. If your light is just white, use an RGB value of (1,1,1), try it in your scene and if you then think it's too bright, lower the values until you're satisfied (or increase them, if it's not bright enough).

Edited 04th of August 2017:
If you want to use the physically based light bulb, then you will need to take more into account and I'd really advice you to read up on this.
A good book for this is Physically Based Rendering (by Matt Pharr et al.).
As that is expensive and takes a long time to read, you can also try to get along with some of the Physically Based Shading course materials of previous years, to be found on the Selfshadow Blog. Especially, there is some explanation on Sebastien Lagarde's Moving Frostbite to Physically Based Rendering

On a side note, the sun is usually modeled as being so distant to the receiver of the light, that you don't take attenuation into account. Thus you would just multiply by the light color (or set the attenuation to 1).


What does "muted" mean in the context of color?

In terms of the HSV color space, "muted" colors are those with lower saturation and/or value.

"Deep" colors are saturated but not too high in value (e.g. deep red) while colors with both high saturation and value might be called "bright" (e.g. bright red).

"Colorful" is somewhat more vague; it might mean bright colors, but it also might mean having a range of different hues within an image or palette, such as including complementary colors or analogous colors rather than just variations on a single hue.

Note that these are general terms that would be understood by any graphic designer or artist, but the specific algorithms that Adobe Kuler attaches to these words are proprietary and only known to Adobe.


Ambient occlusion of a sole object in a scene

When you compute ambient occlusion (AO) for a point on an object's surface, you sample multiple directions and measure what fraction of them escape. That is, you're trying to approximate what fraction of the unit hemisphere is visible to that point on the surface. (Note that it's a unit hemisphere because half of the unit sphere is on the wrong side of the surface to start with.)

So in your pipe example, the outside of the pipe is not at all occluded. From every point on the outside, any ray fired in that hemisphere will escape without intersecting the object. All of the outside would be lit with 100% of the ambient light.

The inside is another matter. The inside of the end cap can only see a small circle of that hemisphere, through the other end of the pipe. The exact fraction depends on the radius and length of the pipe, but call it 10% for the example. 90% of rays fired from a point on the end cap will hit the inside of the cap, and the other 10% will escape along the length of the pipe and out of the other end. These points are 90% occluded, so they'll be lit with 10% of the ambient light.

Consider a point at the open end of the pipe, but on the inside. This point is 50% occluded. Half of the rays fired from it go into the pipe and intersect with it at some point. The other half go straight out and escape to the unit hemisphere. This point will be lit with 50% of the ambient light, and it's easy to see that the 50% will smoothly decrease to the 10% (or however much is the minimum) as you travel down the inside of the pipe towards the end cap, as less of that hole is visible.

The important point I think you misunderstood is that you measure or estimate what fraction of the unit hemisphere is visible from the point you're shading, not whether it's visible at all. Casting many rays and counting what fraction escape is one way of estimating this.


You can figure out which thread has the max value by modifying x: pack gl_LocalInvocationIndex in the least significant bits of x. The number of bits you need for the index depends on your group size.

If you need all 32 bits of x then I suggest using 2 compute shaders - first record max per group. Then compute x again and compare.


Rotation matrix for a 3D object in space

Skimming the math.  Starting with a quaternion $Q=w+\left(x,y,z\right)$ then we can rotate $\mathbf{v}$ by:

$$\mathbf{v}' = Q\mathbf{v}Q^{-1}$$

and if $Q$ is unit magnitude this reduces to:

$$\mathbf{v}' = Q\mathbf{v}Q^*$$

To create a matrix we need to apply the rotation to the basis set to form our three equations:

$$
\mathbf{x} = \left(1,0,0\right) \\
\mathbf{y} = \left(0,1,0\right) \\
\mathbf{z} = \left(0,0,1\right) \\
$$

which expanded and reduced gives:

$$
\mathbf{x}' = Q\mathbf{x}Q^{-1} = \left(1 - 2 \left(y^2+z^2\right), 2\left(xy+wz\right), 2\left(xz-wy\right) \right) \\
\mathbf{y}' = Q\mathbf{y}Q^{-1} = \left(2 \left(xy-wz\right), 1 - 2\left(x^2+z^2\right), 2\left(wx+yz\right) \right)  \\
\mathbf{z}' = Q\mathbf{z}Q^{-1} = \left(2 \left(wy+xz\right), 2\left(yz-wx\right), 1 - 2\left(x^2+y^2\right) \right)
$$

Sticking to the math convention of column vectors, then we shove the three equations into the first three columns and to add a translation by $\left(t_x,t_y,t_z\right)$ we shove that into the last column giving:

$$
{
\left( \begin{array}{ccc}
1 - 2 \left(y^2+z^2\right) & 2\left(xy-wz\right)       &  2\left(xz+wy\right)  & t_x\\
2 \left(xy+wz\right)       & 1 - 2\left(x^2+z^2\right) &  2\left(wx-yz\right)  & t_y \\
2 \left(wy-xz\right)       & 2\left(yz+wx\right)       & 1 - 2\left(x^2+y^2\right) & t_z \\
0 & 0 & 0 & 1
\end{array} \right)
}
$$


Getting texture value of cartesian coordinate from a Obj file

Your question is very broad because it is close again to "how to I render 3D objects". So some people might give you a more precise answer than mine but at least I will try to give you some pointers.

First look on the web. There are plenty of tutorials that explain to you exactly what you to do (which is to raytrace texture mapped objects). While it's great to ask questions, there is really no excuse for posting such a generic question here, considering the amount of information that is available on the topic on the Web.
Though I will try to give you some pointers nonetheless (because we are kind and not here to judge each other;0)

What you want to do is called texture mapping. The way it's done is simple. Each triangle of your geometry holds what we call vertex coordinates, basically it explains how a triangle is actually mapped onto an image (or texture). Imagine your triangle laying flat on the image. The part that it lays on is that part of the texture that should appear on the triangle once rendered. Note that texture coordinates are 2D coordinates and that the shape of the triangle in the texture space (when it lays flat on the texture) doesn't have to match the shape of the triangle in 3D space (in which case the texture will be stretched).

So your 3D object when you render it should have a set of 3D points (the vertex position) 3D normals (the normals of the object at the vertex position) and 2D coordinates (read from the OBJ file).

When the ray hits a triangle your compute the texture coordinates at the intersection point using the barycentric coordinates of the intersection point and the texture coordinates set at the 3 vertices of the hit triangle. This is very common technique you will find many tutorials about that.

Then once you have your interpolated texture coordinates you use it to do a lookup into the texture. The texture coordinates are in the range [0:1] (in general), where (0,0) might be the top-left corner if your texture and (1,1) the bottom right. You need to remap that texture coordinate to pixel space (multiply it by image width and height and cast to integers eventually).

Now just read the pixel value at that position (remapped to pixel coordinates), and that's the color of the mapped intersection point.

Easy, breezy!


finding the projection matrix for one point perspective

When reading on transformation matrices over a range of distinct source material, there are some main concepts that you must understand, which are as follows.

Column vs Row Notation

Some authors write vectors as a single row matrix (1x3), while others write them as a single column matrix (3x1). The latter seems to be the preferred representation in most modern books on computer graphics. The main difference between the two notations is that when using the former, matrices will be multiplied on the right side of the vector, and when using the latter they will be multiplied on the left. When dealing with linear transformations, such as rotation matrices, in a particular notation, they are represented as the transposed matrix in the other notation. For instance, the rotation of point $p=(1, 1)$ for an angle $\theta$ in the plane is shown below. As you can easily see, $Row(x)^T = Column(x)$, i.e., one notation is the transpose of the other.

$$
\begin{align}
Row(p) =&
\begin{bmatrix}
1 & 1
\end{bmatrix}
\begin{bmatrix}
\cos(\theta) & sin(\theta)\\
\text{-}sin(\theta) & \cos(\theta)
\end{bmatrix}\\
Column(p) =&
\begin{bmatrix}
\cos(\theta) & \text{-}\sin(\theta)\\
\sin(\theta) & \cos(\theta)
\end{bmatrix}
\begin{bmatrix}
1 \\ 1
\end{bmatrix}
\end{align}
$$

The website you linked uses row notation. This is important because if you read about the perspective matrix in other sources, you may see the result written as the transpose matrix if the author used column notation.

Basis Orientation

In the website, the camera basis vectors $X_1$, $Y_1$, and $Z_1$ form a left-handed coordinate system, because $X_1 \times Y_1 = - Z_1$. This is important as well, as you may notice some sign changes in the perspective matrix if the system was right-handed. You can read more about handedness in Wikipedia. As with row notation, this is another arbitrary decision that the author has taken.

Perspective Projection

Given the appropriate basis, and a projection plane at a distance d from the origin, perspective projection is a simple exercise in similar triangles, as shown below.

The point $\mathbf{P} = (x,y,z)$ when projected onto the plane gives another point $\mathbf{p} = (x_p, y_p, z_p)$ that can be written as (in row notation)
$$\mathbf{p} = \begin{bmatrix}
 d \displaystyle \frac{x}{z} & d \displaystyle \frac{y}{z} & d
\end{bmatrix}.$$

Now we just need to put this formula into matrix form (again, in row notation).

Homogeneous/Projective Coordinates

If we just use the raw 3D coordinates of $\bf P$ there is no way to represent the formula for $\bf p$ as a matrix multiplication. The use of homogeneous coordinates allows us exactly to do that. We start by augmenting $\bf P$ onto the fourth dimension, by defining it as $\mathbf{P} = (x, y, z, 1)$, and we use 4x4 homogeneous matrices instead of 3x3 matrices to represent transformations. I strongly urge you to read on projective spaces, as otherwise the following explanation will look like too much hand-waving as I am not going to dwell on mathematical formalities.

Now, in this new space every point with coordinates $(wx,wy,wz,w)$ can be mapped onto the point $(x,y,z,1)$ by a simple division by w. We can use this property in our advantage to fill-out a matrix in such a way that after the division by w is made, we get exactly the perspective projection formula for our point.

$$
\begin{bmatrix}
x & y & z & \displaystyle\frac{z}{d}
\end{bmatrix}
=
\begin{bmatrix}
x & y & z & 1
\end{bmatrix}
\begin{bmatrix}
1 & 0 & 0 & 0\\
0 & 1 & 0 & 0\\
0 & 0 & 1 & \displaystyle \frac{1}{d}\\
0 & 0 & 0 & 0
\end{bmatrix}
$$

Now if we take the point $(x, y, z, \frac{z}{d})$ and we divide it by its last coordinate, we get $(d\displaystyle\frac{x}{z}, d\displaystyle\frac{y}{z}, d, 1)$, which are exactly the coordinates of our projected point if we just ignore the augmentation onto fourth dimensional space that we did (i.e., discard the last coordinate).

As you can see, the "perspective matrix" does not actually do any perspective transformation, it just "prepares" the transformation by coding a specific value in the last coordinate, such that after all coordinates are divided by it, we get the true projection.


The magic is that the mesh is attached to the skeleton.  In it's simplest form, this is done by assigning each vertex to a bone.

When a vertex is assigned to a bone, that means that it will always keep the same position relative to that bone's position, and orientation (normal, tangent, bitangent aka the bone's local X,Y,Z axis) as the bone moves as directed by the animation data, physics impulses, or whatever else driving it.

As you might expect, some triangles will have vertices that belong to different bones.  When this happens, since the triangle is made up of those vertices, the triangle will stretch or squish as the bones move.

The link you mention explains a way to set up the vertices for this specific type of movement such that the vertices stretch and squish in realistic and well behaved ways.

In modern games, vertices are usually attached to multiple bones with a weight per bone so that each vertex is actually affected by the movement of multiple bones.  This gives a more natural and organic look to the mesh.

This answer only scratches the surface though, and this is a very deep topic which has many sub topics that are active areas of research!

If you want to read more about skeletal animation, this link is a good starting point:
http://blog.demofox.org/2012/09/21/anatomy-of-a-skeletal-animation-system-part-1/


How to find vertices on borders of geometry

I'm assuming you are rendering this object as a set of triangles. To do what you are trying to do, you will need to store all your triangles (and vertices) in some sort of buffer or list. Then, for each triangle, test if all three of its sides are matched with another triangle side. If so, it is enclosed and inside the object. Otherwise, it is outside the object. Some basic pseudo-code (I apologize for all the foreach loops :) )

foreach t in triangles:
    all_sides_touched = True
    foreach side_t in t:
        this_side_touched = False
        foreach u in triangles.exclude(t):    // run through all other triangles to test
            foreach side_u in u:
                if side_t.equals(side_u):  this_side_touched = True
        all_sides_touched = all_sides_touched and this_side_touched
    if not all_sides_touched:
        foreach side in t:
            foreach point in side:
                // do whatever you want to do with outside vertices


How do you add text dynamically to an image and export it?

To do this in a web app is trivial. Judging by the image, even the built-in HTML canvas should be able to do everything that you need.

Once you have obtained your image and text, to give you a vague idea of the ammount of code involved, drawing them with the canvas API looks roughly as follows:

ctx.drawImage(img, x, y, width, height);
ctx.font = '12px serif';
ctx.fillText(txt, x, y);

Saving the image is equally simple:

canvas.toDataURL("image/png")

It's actually the obtaining and sharing of images are the scary parts. Spotify surely have all the images on their cloud and similarly that is where they export them to before sharing to social media. There are so many sollutions that without more detail this is probably something you'll need to find yourself.

If you just need a tutorial on html canvas, mozzilla usually have great interactive tutorials.


Okay. So after 6 hours of tinkering with my code, trying different approaches and googling for solutions, I found this page:

http://www.glprogramming.com/red/chapter03.html

There is a short C listing which put me on the right track to solve this.

What I did wrong was that I should have set the frustum to something like this:

GLdouble x0 = -(1366/768);
GLdouble x1 = 1366/768;
glFrustum(x0, x1, -1, 1, 1, 100);
// the last two values aren't critical, as long as they are positive

But before that, I should have set the glViewport properly, before setting up the frustum:

glViewport(0, 0, 1366, 768);

I didn't do it and that's why my code produced such weird results.

Now I can create a unit cube with following vertices:

0.0, 1.0, 0.0
1.0, 1.0, 0.0
1.0, 0.0, 0.0
0.0, 0.0, 0.0
0.0, 1.0, 1.0
1.0, 1.0, 1.0
1.0, 0.0, 1.0
0.0, 0.0, 1.0

And it will display properly as a cube of size 1 in every direction.


Converting raster shape/blob into displacement map

One way I can think of is to make a "signed distance transform" of the image where there is information for each pixel about how far the pixel is to the closest surface of the shape.  Since it's signed, youll be able to know if the pixel is inside or outside he shape, and by how much.

Using this knowledge, you could easily make a new image, where the pixel is black if the distance is greater than -10, else it is white.  That would make the shape shrink by 10 pixels.

You can use smoothstep or other curves to transition between black and white over a number of pixels, to soften it anti alias the edges.

You can make the distance transform by brute force, but another method would be to make a voronoi diagram first and then turn that into a distance transform.

There is lots of info out there on both voronoi diagrams as well as (signed) distance transforms but these two links will probably be helpful to get you started:

http://blog.demofox.org/2016/02/29/fast-voronoi-diagrams-and-distance-dield-textures-on-the-gpu-with-the-jump-flooding-algorithm/

http://blog.demofox.org/2016/03/02/actually-making-signed-distance-field-textures-with-jfa/


If you have this data in a 3d array where a block is a 1 and no block is a 0 you can apply a 3d blur and then set all voxels that are left with a value higher than 0.5 to 1 and those that arent to 0. this will smooth out sharp details and leave flats mostly uneffected


How to obtain generalized barycentric coordinates for n-sided polygon?

Quadrilateral basis functions can be calculated using an outer product of the basis of two linear functions (1-r, r) and (1-s, s). The same thing does not apply for arbitrary $n$-sided domains. For this we use generalised barycentric coordinates. Wachspress coordinates are arguably the simplest generalised barycentric coordinates available, are fast to evaluate and are defined for arbitrary $n$.

For a point $p$ on a planar convex polygon defined by vertices $v_1, v_2, \ldots, v_n$, ordered counter-clockwise, you can find weights

$$w_i(p) = \frac{A(v_{i-1}, v_i, v_{i+1})}{A(v_{i-1}, v_i, p)A(v_{i}, v_{i+1}, p)}. $$

Here, $A(v_1, v_2, v_3)$ denotes the signed triangle area. The coordinate functions $\phi_i$ can then be found be normalising the weights $w_i(p)$

$$\phi_i(p) = \frac{w_i(p)}{\sum_{j=1}^n w_j(p)}.$$

Now, the coordinate functions correctly parametrise the point (linear reproduction property)

$$p = \sum_{i=1}^n v_i \phi_i(p).$$

At vertices the coordinates satisfy the Lagrange property $\phi_i(v_j) = \delta_{ij},$ where $\delta_{ij}$ is the Kronecker delta, meaning that the corresponding coordinate function evaluates to 1 at its vertex and linearly interpolate on the edges. It is also important to notice that the coordinate partition unity $\sum_{i=1}^n \phi_i(p) = 1.$

For, non-convex polygons you could use mean value coordinates.


Is there a general way how to specify the "image compression loss"?

First off, it's worth noting that not all artefacts are regions of the same colour. Artefacts just means structured errors introduced by the compression, as opposed to noise or blurring. JPEG artefacts include flattened colours as you mention, but also edges introduced at block boundaries, and "ringing" around sharp edges in the image.

There are two popular metrics for specifying the losses, but they're not independent of the image content, and they measure the whole loss, not just structured artefacts. PSNR is most commonly used to quantify the loss of image and video compression algorithms. It's not a very useful measure, because it measures the absolute reconstruction error: it doesn't take into account what kind of artefacts are introduced and how they appear. Structural similarity (SSIM) is better because it takes the structure of the image and the errors into account in the measure.

The measure can't be independent of the image, for several reasons. Different images are easier or harder to reconstruct. For example, a completely flat colour will be perfectly reconstructed even with some very lossy algorithms and parameters. At the opposite end, completely random white noise will suffer some loss in almost any algorithm. Some images just contain more information than others. In addition, different domains have different requirements, so algorithms can be tuned for different kinds of images. JPEG is tuned for photographs, and performs badly on hard-edged synthetic images like diagrams. A texture compression algorithm might be tuned specifically to avoid block artefacts for tileable textures. JBIG2 is tuned for black-and-white document scans. A single generic measure wouldn't help you to understand the performance of the algorithm on the kind of image you're interested in.

To get a broader measure, it's common for researchers to quote a mean PSNR and/or SSIM measured across a corpus of images. The Kodak PCD0992 corpus has been used for many years, so it's an easy way to compare different papers, but it is waning in popularity now.


Nice way to make noise seamless (without tampering with the noise generator)

If you can't change the textures, I see 2 possibilities:

Use mirrored repeat on the noise textures so they tile seamlessly.  (taken from here)
Do like Gimp does (or many other alternatives) and create tilable versions of the noise textures. The idea is to blend the left and right, and bottom and top edges together in pairs. Here's how to do it in an image editor: translate the image by 1/2 in both axis so the corners are in the middle. Blur, smudge, etc. to remove the seams. Revert the translation.

Keep in mind that the randomness and quality of the noise will of course be affected by either technique.


maximal convex patching in Computer graphics

Start from any triangle. Traverse it's edge's and check that the angle between the two triangles is less than 180deg. If it is add it to the current selection and continue expanding.

The check is actually really simple if you use vector geometry. Say A - B is the common edge with C on the selected side and D on the other. Then just check if dot((D-B), cross((A-B), (C-B)) < 0.

Single triangle patches should be ruled out, as for us to determine convexity, it has to span across several triangles.
We have to keep sampling until we've tested everything.  When a convex patch continues further, it's size keeps on increasing by including the neighbouring triangles that satisfy convexity, until the patch reaches the maximum size it can attain.


One of the downsides to shadowmapping is that you need to offset, or bias, the point used for shadowmap-lookups. This is done in a variety of ways - typically by a constant (shadowmap-depth) distance or based on the slope of the casting triangle. Both of these tend to be done during the shadowmap-rendering pass.

I think what you need to solve your sphere-problem, is to offset the point you are shading a bit along its surface normal, before doing the shadowmap-lookup. This happens during shading, when sampling the shadowmap, not during shadowmap-rendering. Incidentally it is also part of the Unity standard rendering pipeline, so you can steal it from the shader include-files.

ref http://www.dissidentlogic.com/old/images/NormalOffsetShadows/GDC_Poster_NormalOffset.png
ref http://www.digitalrune.com/Blog/Post/1765/Shadow-Acne


how to map rectangular coordinate system onto JavaFX GraphicsContext canvas

You can map [x, y] pixel coordinates in an image of size [width, height] to your given range as follows:

x'=minx+(maxx-minx)*(x+0.5)/width;
y'=miny+(maxy-miny)*(y+0.5)/height;


Name and influence of style used for futuristic line patterns?

That looks a lot like it’s based on the straight skeleton of some outline shapes drawn on the image. You can see similar structures in the middle one of these diagrams from a research page by Stefan Huber, one of the researchers who’s published a lot of work on straight skeletons:


A ray is just a semi-infinite line, and casting the ray finds intersections between the ray and an object in the scene. It doesn't just tell you the position in space that the ray intersected the object: it also tells you the texture co-ordinates of that point on the object, the surface normal and tangent, and (depending on the algorithm you use) filter widths.

So in a ray-tracer, there are different kinds of rays. Primary rays and reflection and refraction rays use their intersection to run a shader and compute a fragment colour. They also fire shadow rays at lights, which don't compute a colour but just tell you whether any object is shadowing the light.

So rays are about finding the objects which are in the scene, and shading those objects to make an image. They're also about finding the reflections and refractions on shiny objects. They are used to draw shadows. They're also used for more advanced effects like ambient occlusion and photon tracing.

This is one of the main conceptual advantages of ray-tracing over scanline rasterisation: instead of having to use different techniques for computing reflections, shadows, and all those other effects, the same core ray-cast algorithm can be used recursively to compute the whole image.


Path Tracer - sampling dimensions confusion

Only one path per sample.

If you had 64 bounces per first hit and 64 per second hit and so forth you'd never get an image.

Edit: And that's why you need to sample each pixel so many times (easily more than 1024 samples) in order to get it to converge, ie get rid of the noise.

As per 2) (from comment below) The 64 primary rays will not hit the same object in the same spot so you will have to calculate everything for each sample.


Rotation of a random unit vector at a point in 3D space by an angle

Rotation matrix | wikipedia.org in section that says "in three dimensions". It's the first search result when searching about rotating a vector about an axis. Let me know if you have any questions.

For your specific problem: bring the point P to the origin, then rotate the vector about the Z axis, then bring the point P back to its original location.


What is the practical difference between 3D Graphics Engine, 3D Game Engine, 3D ........?

At the base, rendering APIs like OpenGL or Direct3D provide a unified interface to communicate with graphics adapters at a low level. Differences in hardware are hidden behind the abstraction which simplifies development a lot.

The low-level APIs provide basic functionality, which gives you total control but leaves you without a lot of desired features. 3D Rendering Frameworks like OGRE or Irrlicht introduce the more understandable concept of 3D objects that can be moved in space (without having to worry about transformation matrices most of the time), introduce light sources and algorithms for shadow casting (e.g. shadow mapping), loading of common mesh formats, and the list goes on. I call this a framework, because you still need to develop the application yourself using actual code, while an engine provides you with an application to develop your software in. This is just my personal distinction and both terms are used differently in a variety of contexts.

In a 3D Authoring / Rendering Software like Blender or Autodesk Maya (offline) or 3DExcite DeltaGen (real-time) you can load, display or create meshes using the built-in functionality. These applications target the creation of 3D data, images or movies, and usually you can't create interactive applications easily with it. Be aware that some 3D Authoring have modules specifically targeting this, e.g. Blender and the integrated Blender Game Engine.

A 3D Game Engine finally provides you with all the tools to make it easy for you to develop a game. You will usually be provided with a level editor, a material editor and maybe a visible scripting language (like Unreal's Blueprint or earlier Kismet) to implement dynamic behavior and interactivity. You don't have to worry about rendering algorithms, because they are already implemented for you. Your engine of choice may for example feature SSAO and cascaded shadow maps for dynamic objects and a solution to bake global illumination for static geometry. Also, a game engine includes way more than only rendering, e.g. artificial intelligence, sound, loading of assets, exporting your content into a ready-to-install package etc.

As you can see, each software package generally puts another layer of abstraction on, ideally making your life simpler. Each has its own purpose and allows you to do different things. Note that said terms are very broad and people have different opinions and sometimes even debates about where one category ends and the next one begins. My advice is to not lose sleep over it.


The best algorithm depends on the condition like whether or not the square is axis aligned for example. I'm gonna discuss the more general case which can find find intersection for any arbitrary oriented square. The algorithm works by first checking the intersection of ray with the plane containing the square or rectangle etc. Then check if the ray is within bounds of the square.

For this you require the following information

Ray's in the form $R =e+td $ where $e$ is the origin or eye point and $d$ is the direction vector.

Secondly you need to define your square by using the following information

Poisition - This might be the vector to the center or any corner of the square. The algorithm I'm describing assumes it is one of the corners.
Normal - Self explanatory.
Width and Height not as scalars but as vectors. These can be called the two edge vectors. I will denote them as $e1$ and $e2$. Note that both edge vectors need to grow outward from the corner of the square chosen in point 1.

After you have the following information you first need to compute Ray - plane intersection. The plane being where your square lies. This is done as follows. We know the point normal form of a plane as,

$(p-p1).n= 0$

If we suppose $p$ is our ray here and $p1 = position$, then we will have,

$(e+td - p1).n=0$
$(e+td - p1).n=0$
$e.n + td.n - p1.n = 0$
$t = p1.n - e.n/d.n$
$t = (p1-e).n/d.n$

If $t \geq 0$ then we have an intersection. Now all we need to do is check whether the ray lies inside the square or anywhere else on the plane. This is easy to do.

$intersect\_point = e + td$
$v = intersect\_point - p1$

$width = ||e1||$
$height = ||e2||$

$proj1 = v.e1 /width $
$proj2 = v.e2 / height$

$if((proj1 < width \;\&\&\; proj1 > 0) \;\&\&\; (proj2  < height \;\&\&\; proj2 > 0) )$
$return \, true$

Note that the above assumes that the edge vectors grow outward from the corner, i.e. A square with the corners A,B,C,D in clockwise fashion starting from  the top left. If we choose corner $A$ then the edge vectors are $\vec{AB}$ and $\vec{AD}$.


Compression of user-drawn "whiteboard" image

Don't store the rasterized image instead store the drawn strokes as vector graphics. Those should compress pretty well.

Adding timestamps to each stroke should then be very simple.

You can caching the rasterized image during execution to avoid needing to rerender every stroke every frame.


Why isn't rasterization combined with raytracing more often?

Rasterization is sometimes used for primary rays. However, it limits greatly what you can do - depth of field, motion blur, participating media, refraction, only basic camera models, no adaptive or custom sampling, etc.