Physically Based Shading for Diffuse surfaces

So after reading up some more research papers and the concepts of most people working in this field here and there, I've reached an answer I'm satisfied with. If someone disagrees or thinks there is a better or more correct way to think the problem do share your ideas.

1) First to answer my point 1. I've noticed that most di-electric surfaces' Index of refraction remains fairly constant over the visible spectrum. This means the Fresnel Reflectances remain constant and are quite low over the visible spectrum for normal incidence. For angles near 90 degrees, even rough di-electric surfaces start reflecting the incoming light specularly. You can see this data here

However in the case of metals the IOR or more specifically the Fresnel Reflectances take on the exact same value as the color (in case of gold and copper). From this I conclude that,

For metals where Fresnel Reflectances (FR) vary greatly over the visible spectrum, FR is linked and is just a consequence of the surface of metals absorbing photons of certain frequencies. This means for Gold, the electrons at the surface absorb the blue-end of the spectrum, the remaining golden colored light either gets reflected or refracted. Thus the Fresnel Reflectance is golden in color which we can also call the "Specular Color".
Note that the absorption depends on whether there are electrons available. It's possible for some blue photons to refract or reflect if there aren't much electrons for absorbing to begin with.

In-case of di-electrics the FR don't change much over the visible spectrum. However we can see color. The possible explanation is that the color in case of di-electrics comes from within the surface. That is, there are no such electrons present at the "interface" that absorb certain frequencies. Else we would have seen different FR for different wavelengths. The electrons absorbing wavelengths of certain frequencies come from within the surface, like pigments lying inside a liquid. Hence for di-electrics FR is not really linked to the concept of electrons absorbing certain frequencies imo.

This makes sense to me as colored metals (gold, copper) tint the reflections with their color. However certain di-electrics don't such as a polished marble floor. The reason for that again is that in case of metals the cause of color i.e. absorption by electrons occurs directly on the surface. The reflected light thus has a different composition for R,G,B giving us the tint. However for di-electrics the cause of color lies beneath the surface. Hence the reflection is non-tinted.

Thus simply stating for metals FR is the same as the albedo/color of the surface. For non-metals or di-electrics FR is different from the albedo/color and just defines the amount of specular reflection.

2) The proper way would surely be to use a BRDF with both diffuse and specular components since diffuse surfaces reflect greatly at near-grazing angles.

3) Still don't know about this. It might be baked in or not. What I do know is that the diffuse component should surely get lesser and specular component larger as angle increases to 90.

Another interesting point to note is that the basic Fresnel Equations apply for transparent and Flat surfaces. For metals we have complex IOR making the equations a little more complex. Sebastian Lagarde did a comparison here. Fresnel equations don't apply to rough surfaces but for microfacet models where we assume each facet to be perfectly specular we can maybe. Either way they are a good way to approximate the behaviour for increasing reflection near 90 degrees.


Tessellation Controll Shader how to get maximum subdivisions limit?

The tessellation limit is GPU dependent..Any number above the GPU limit is technically an error but most gpu's just clamp to the upper limit. To get beyond the limit either use a Level Of Detail (LOD) approach where you have different models that are used based on distance, or save and recompute like you suggest.
To get the max just set all the edges to the limit AND set all the internals to the limit in the tessellation control shader:
#version 460 core

  layout(vertices = 4) out;

void main(void)
{
   gl_TessLevelOuter[0] = 64.0;
   gl_TessLevelOuter[1] = 64.0;
   gl_TessLevelOuter[2] = 64.0;
   gl_TessLevelOuter[3] = 64.0;

   gl_TessLevelInner[0] = 64.0;
   gl_TessLevelInner[1] = 64.0;

   gl_out[gl_InvocationID].gl_Position = gl_in[gl_InvocationID].gl_Position;
}

And how can I ask for the maximum division on CPU?

Yes use glGetInteger. At the bottom of this page is all the values you would be intrested in: Khoronos Tessellation limits, search for tessellation on the page


Bitpacking into buffers with webgl (shadertoy)

Assuming you are not going to filter the buffer using HW texture filtering: since you only need 4 values for the material index, pack it in the sign bits of uv x and y.


Setting up integer texture and binding as image

glTexImage3D(GL_TEXTURE_3D, 0, GL_RGBA, width, height, depth, 0, GL_RGBA, GL_INT, NULL);

This is not an integer image format. It's an unsigned, normalized, fixed-point image format. For those, you use floating-point image/sampler variables. So for a 3D texture, you would use image3D, not iimage3D.

If you want a true signed integer format, you would do this:

glTexImage3D(GL_TEXTURE_3D, 0, GL_RGBA32I, width, height, depth, 0, GL_RGBA_INTEGER, GL_INT, NULL);

Note two changes. First, the internal format is GL_RGBA32I, which specifies a 32-bit-per-channel signed integer format. The second is the use of GL_RGBA_INTEGER for the pixel transfer format. Even though you're not actually performing a pixel transfer (since you're passing NULL), you still must use an appropriate format and type. And the _INTEGER formats must be used if you're transferring to/from a texture with an integer format.

When you do this, you must of course use the iimage3D and isampler3D types in your shaders.


Efficiently drawing multiple instances of 3D objects

What you are looking for here is instanced rendering. Instanced rendering coupled with attribute divisors is a very powerful tool to render multiple copies of the same object using only one drawing call.

Using instanced rendering you only have to specify the geometry of an object once on the GPU. Then, you could pass along an additional buffer of matrices, one matrix for each instance of the object you are rendering. Using attribute divisors you can tell the GPU that you will only need one single matrix per a number of vertices. Attribute divisors tell the GPU when and how to advance an index in a attribute buffer. This allows you to only define a single matrix per instance instead of passing along a single matrix for every vertex in a single model.

Here, is an excellent tutorial on using instanced rendering and attribute divisors.


If we sidestep your typo (the last term has one absolute too much), both formulations are correct. They just express different things. The $k$ in Hooke's law is for a particular spring. $k_s$ is the siffness for a paricular material.

Now in the linear portion there is a direct relationship betwen these the material stffness is directly propotional to the spring factor and length of member. So for example when the rest length of a different spring with same material is 2 times as long its 1/2 as stiff.


Was Evans & Sutherland CT5 really created in 1981?

The CT5 was designed before 1981. I started working at E&S in May 1981. The CT5 was fully designed by then and 3 had been built. All 3 were still at E&S and hadn't shipped yet when I started. Two of them were on the test floor and the third was the engineering prototype that was later sold. The prototype was in the "snake room", so called because of all the black power cords hanging from the ceiling. I also remember seeing the prototype in the snake room when I made a recruiting visit to E&S in early January 1981. So, the prototype, at least, was up and running before 1981.

The first two CT5 systems were CH-46 helicopter simulators and the first one went to Cherry Point NAS. I don't think it shipped in 1981 as I remember it being there on the test floor for about a year after I joined the company. So, it probably shipped in 1982. As I recall, the two systems built after the CH-46 systems were AV8B Harrier jet simulators.


How is a projector handled in a Ray Tracing environment?

Just treat it as a special case of a point light + shadow ray. Whenever a ray hits the screen (or any surface onto which you'd like to project the slide), fire another "projector" ray back towards the projector. If it is not obscured, then use the X & Y angles (relative to the coordinate system of the projector) to look up a texture map of the slide.


Difference between Cropping, Scaling, Resizing & Changing Aspect Ration of an Image?

"Cropping an image" means some part of the image is removed in order for the image to fit a certain size. For example, removing the rightmost part of a rectangular image can result in a square.
"Resizing an image" usually means to stretch and squeeze it as needed so it fits a given size. Width and height are modified independently. If the ratio between them is not the same in the original size and in the final size, the image will appear distorted.
"Scaling an image" usually means resizing, but by the same amount horizontally and vertically. The ratio between the width and height is not modified, therefore the final image is not distorted.
The "aspect ratio" is the ratio between the width and the height. Changing the aspect ratio will perceptually stretch the image either horizontally or vertically.

Resizing might change the aspect ratio, and from the wording of your question I have assumed scaling meant not changing the aspect ratio.


First of all - a number must not occur twice, that is implied since we're talking about permutations. So filling the table with a simple random(255) function won't work.

Secondly, you need to ensure that there are no premature recurrence patterns:

Consider the values 1,2,3,4 - the permutation table 4,3,2,1 is not a very good one because of its short cyclic properties, i.e. 1 -> 4, 4 -> 1. Likewise with 4,2,3,1 or 1,2,3,4. The optimal tables take you all the way through all positions: 3,1,4,2 or 2,4,1,3.

This property becomes increasingly  important as you increase the number of dimensions and perform recursive lookups.

However this approach alone may create clusters of too similar values, which may or may not be wanted, which leads me to next point.

Thirdly, When you generate a table with the non cyclic properties, you need to step through remaining unassigned indices in a random manner. When possible constrain the random step distance here to a certain min and max range, e.g. 5..120 to avoid clustered groups of similar values. These numbers are worth experimenting with.


This is a pure optimisation problem. It depends on many factors like:

What you need to do with the tween
How many different tweens you need to calculate
How much signaling between systems you need
Whether you need to use that data in your CPU bound process or not.
How much time you have to write the code
What your tool set supports easiest
...

See if you only need one value to interpolate then your CPU is more than likely faster (unless you're running some weird embedded system). Even if you move a bunch of points you can still calculate the tween of that one position and do the needed cluster transformation in the GPU, like bone animation.

The GPU only starts to win against your processor if you need to calculate many truly separate animation curves in parallel. Or you're in an otherwise performance bound system like running JavaScript in a WebGL application. If you do use the GPU then it's most likely best to tween all vertices, at least unless your object is somehow massive. Since initiating a new draw call is costly and maintaining many states is more complex. Its just that those vertices that don't move would just move from starting point to starting point.


Understanding inequality to equality in the Hartmann & Gribbs frustum planes extraction method

The volume inside of the frustum is defined by a set of inequalities, and since we are trying to find the boundaries of the frustum, we set the inequalities to equalities. That's really all it is.
To take a 1D example, if we have a "frustum" defined by 0 < x < 5, then of course the bounding points of it are defined by x = 0 and x = 5. Replacing each inequality by an equality, in turn, gives you the most extreme point you can get to in that direction.
Slack variables, BTW, don't really convert inequalities to equalities. They just transform the inequality into a standardized form "x > 0" (or "x ≥ 0") for some slack variable x.


What do the OpenGL version strings from glxinfo actually mean?

When creating a OpenGL context you have to specify a profile mask, which is a way of telling the driver whether you want to use compatibility ("legacy") features or not.

Drivers are required to implement all core profile features, but compatibility profile is optional - creating a compatibility context can fail even if the driver supports the version you've requested.

The output from glxinfo you've pasted shows that you can create GL context with version up to 4.5 only when using core profile. If you want to use compatibility profile, the highest version you can request is 3.0.

One reason of the problem you are having is that you:

requested context version greater than 3.0, and
requested a compatibility profile (by setting GLX_CONTEXT_COMPATIBILITY_PROFILE_BIT_ARB in GLX_CONTEXT_PROFILE_MASK_ARB).

If removing compatibility bit doesn't help, you'll need to double check if other context attributes are correct.

What I described above is documented in a more detailed way here.


What is the modern equivalent of display lists?

Nvidia has an extension for creating command buffers in modern GL.

The reason for the lack of similar functionality is that there is a lot of state involved regarding how to render and the display list be affected by a lot of different state. For example changing the blend state requires patching the fragment shader on some hardware.

NVidia solved it by capturing all state and reseting to the state after a dispatch:

  1) What motivates the design?

  The primary goal is to be able to reuse pre-validated command buffers.
  Other APIs and proposals have addressed this with various incarnations
  of command  lists or state objects, but a recurring problem is that
  interactions between various stages of the pipeline prevent this
  prevalidation and reuse. These  interactions are often
  hardware-specific (and differ from vendor to vendor  or even
  generation to generation) and new interactions are introduced by  new
  features that were not imagined when the prevalidation scheme was
  proposed.

  We attempt to address this by having a monolithic state object that
  encompasses (almost) the entire state of the pipeline. This should
  provide enough information for all implementations to do any needed
  cross- validation. We try to create these in a way that minimizes the
  new API  footprint - since we want ALL state (including any added in
  the future), we just capture it from the current state of the context.

  [...]

  23) In what condition is the state left, that is modified by tokens,
  after the dispatch call?

  RESOLVED: state is reset.

(from the extension text linked above)

However the true successor is the command buffer functionality in DX12 and vulkan. Those will also capture all render state into a single object to use when creating and filling the command buffer. The NVidia extension is based on that architecture as a result of NVidia's involvement in the Vulkan design.


Topology: Polygonal Meshes, Euler's Formula

Let g be the number of vertices of valence 4. A closed sphere has G=0 and we have the equation V-E+F=2. Now V=x1+g+x2 and F=275.

As a first step let us count the edges. If we multiply each face by its valence and sum those up, we get twice the number of edges (as each edge is shared by two faces), so:

(1)   2*E = (274*4 + x3*1)

Immediately we can see that 2 divides x3.

Similarly if we multiply each vertex with its valence we get again twice the number of edges, as each edge is shared by two vertices, so:

(2)   2*E = 3*x1 + 4*g + 5*x2

And again we see that x1 and x2 must have the same parity.

Plugging the first of this equation (with the very first ones) into V-E+F=2 we get

(3)   2 = V-E+F = x1 + g + x2 - (1096+x3)/2 + 275 = x1 + x2 - x3/2 + g + 548

Equating the two equations we derived above we get

(4)   1096 + x3 = 3*x1 + 4*g + 5*x2

Now the system of equations (3)&(4) has all the information encoded (excluding the parity information), so it appears you don't already have everything you need to know to determine these four unknowns.

I made a small Haskell program that does some bruteforcing with these conditions: Try it online!

The first solution it does get is (0,732,2564,0) = (x1,x2,x3,g)


DirectX / OpenGL(Vulkan) concepts mapping chart

I haven't been able to find such a chart on the web, so I made one here. (Everyone, feel free to add, elaborate, or correct any mistakes. Some of these are just best guesses based on a partial understanding of the API and hardware internals.)

API Basics

D3D11                          OpenGL 4.x
-----                          ----------
device                         context
immediate context              (implicit; no specific name)
deferred context               (no cross-vendor equivalent, but see
                                GL_NV_command_list)
swap chain                     (implicit; no specific name)
(no cross-vendor equivalent)   extensions
debug layer; info queue        GL_KHR_debug extension

Shaders

D3D11              OpenGL 4.x
-----              ----------
pixel shader       fragment shader
hull shader        tessellation control shader
domain shader      tessellation evaluation shader
(vertex shader, geometry shader, and compute shader
 are called the same in both)

registers          binding points
semantics          interface layouts
SV_Foo semantics   gl_Foo builtin variables

class linkage      subroutines

(no equivalent)    program objects; program linking

(D3D11's normal
 behavior; no      separate shader objects
 specific name)

Geometry and Drawing

D3D11              OpenGL 4.x
-----              ----------
vertex buffer      vertex attribute array buffer; vertex buffer object
index buffer       element array buffer
input layout       vertex array object (sort of)

Draw               glDrawArrays
DrawIndexed        glDrawElements
(instancing and indirect draw are called similarly in both)
(no equivalent)    multi-draw, e.g. glMultiDrawElements

stream-out         transform feedback
DrawAuto           glDrawTransformFeedback

predication        conditional rendering
(no equivalent)    sync objects

Buffers and Textures

D3D11                    OpenGL 4.x
-----                    ----------
constant buffer          uniform buffer object
typed buffer             texture buffer
structured buffer        (no specific name; subset of SSBO features)
UAV buffer; RWBuffer     SSBO (shader storage buffer object)
UAV texture; RWTexture   image load/store

shader resource view     texture view
sampler state            sampler object

interlocked operations   atomic operations
append/consume buffer    SSBO + atomic counter

discard buffer/texture   invalidate buffer/texture
(no equivalent)          persistent mapping
(D3D11's normal
 behavior; no            immutable storage
 specific name)
(implicitly inserted     glMemoryBarrier; glTextureBarrier
 by the API)

Render Targets

D3D11                     OpenGL 4.x
-----                     ----------
(no equivalent)           framebuffer object
render target view        framebuffer color attachment
depth-stencil view        framebuffer depth-stencil attachment
multisample resolve       blit multisampled buffer to non-multisampled one

multiple render targets   multiple color attachments
render target array       layered image

(no equivalent)           renderbuffer

Queries

D3D11                       OpenGL 4.x
-----                       ----------
timestamp query             timer query
timestamp-disjoint query    (no equivalent)
(no equivalent)             time-elapsed query
occlusion query             samples-passed query
occlusion predicate query   any-samples-passed query
pipeline statistics query   (no equivalent in core, but see
                             GL_ARB_pipeline_statistics_query)
SO statistics query         primitives-generated/-written queries
(no equivalent)             query buffer object

Compute Shaders

D3D11                     OpenGL 4.x
-----                     ----------
thread                    invocation
thread group              work group
thread group size         local size
threadgroup variable      shared variable

group sync                "plain" barrier
group memory barrier      shared memory barrier
device memory barrier     atomic+buffer+image memory barriers
all memory barrier        group memory barrier

Other Resources

Porting from DirectX11 to OpenGL 4.2 – not exhaustive, but a quick discussion of common porting issues
MSDN: GLSL-to-HLSL porting guide – lists correspondences and differences between GLSL and HLSL shading languages


The mathematics of two dimensional interpolation on a quad

(It actually is easier to think (and compute) about this with triangles, but for the sake of the answer, let's first stick to your quad example.)

For this you just have to define the point you're interested in in terms of the quad's (or whatever primitive's) vertices. This is called a barycentric coordinate system. More practically-speaking, you basically give the point $n$ weights, each describing the influence the $i$th vertex of the quad has on the point. Those are the point's barycentric coordinates with respect to that quad.

So for example the center point of the quad (or "center of mass") would have the weights $\frac{1}{4}$, $\frac{1}{4}$, $\frac{1}{4}$, and $\frac{1}{4}$. A point in the middle of the left edge would have a weight of $\frac{1}{2}$ for the left two vertices each and a weight of $0$ for the right two vertices. (Notice how they always sum to $1$?)

These barycentric coordinates can then be used to interpolate whatever kind of per-vertex values you want, be that positions or texture coordinates. It all reduces to a simple weighted sum. And even more than that, they can also be quite easily computed by simple vector algebra from any existing per-vertex values if you have the point's values given (comes down to just a few little 2x2 determinants for a triangle, or similarly easy computation). This means if you know a pixel's screen space position and those of the vertices, you can compute its barycentric coordinates. These you can then use to interpolate the pixel's texture coordinates (or normal, or whatever) from the per-vertex values, or vice versa.

So much to the theory, there's a few practical considerations to keep in mind, especially when using them for interpolation in a rasterizer, like GPUs do.

For anything but triangles the barycentric coordinates don't necessarily have to be uniquely defined for any arbitrary point. That's why thinking about triangles is much easier, especially for the computer.
Due to the fact that they are negative if and only if the point doesn't actually lie inside the triangle they can also be used for checking if the point belongs to the triangle at all and thus are more or less a by-product of the pixel-in-triangle test that the rasterizer already performs to, well, rasterize the triangle.
When it comes to interpolation in projected space, i.e. when projecting a 3D world onto the plane using perspective distortion, it becomes a little more involved since you have to account for the fact that a line's length in screen space is not the same as its length in world space, or more precisely its screen space length doesn't increase linearly when going along a triangle edge in world space. The keyword here is perspective correct interpolation. But this still uses the point's barycentric coordinates.
Of course in an actual rasterizer you also have to take account for shared triangle edges and only rasterize a shared edge once, which requires special rules as to which triangle a shared edge belongs and might make the computation of barycentric coordinates a little more complicated, together with using fixed-point coordinates or whatever optimized computations.

(This is a more general overview of the matter and how it generally works. For specific mathematical as well as technical details, I'd advise for further reading, maybe starting with the above Wikipedia articles.)


Can you avoid querying for a texture location in OpenGL?

By using the exact same syntax as you did for SSBOs:

layout(binding = 0) uniform sampler2D texture_name;

This is a GL 4.2 feature, so if you can use SSBOs, you can use this.


Deferred decals are a common way to achieve this without re-rendering the same geometry. It also allows you to change the appearance, i.e. add some gap or smooth out the details.

The algorithm:

Draw a box around your height field, the size should be as tight as possible to avoid over draw.
sample the depth buffer to get the world-space position of the current pixel (the height field / other geo)
Project your box down in world space
Use the projected box coordinate to sample from your overlay texture.


Why do vulkan extensions need to be enabled?

Extensions can change the visible behavior of the implementation, not to mention the implementation of it.

Consider KHR_external_semaphore and its various sub-extensions. If you don't specify them, then the VkDevice's implementation knows that you're not going to need to be able to turn a VkSemaphore into an externally-accessible mutex and back. So the implementation may be able to use a more efficient VkSemaphore implementation.

If the extension is always on, the device must assume that you're going to use it. And thus, any possible efficiency is lost.

  wouldn't it be easier to just create one VkInstance and one VkDevice with everything enabled?

How? If you have two VkDevices, unless they are both attached to the same VkPhysicalDevice, there's no guarantee that "everything" is the same thing for both devices.

Cross-process Vulkan interop is not something external code is expected to define. There are extensions that provide various aspects of such interop.


Shader storage buffer indexing by 4 bytes instead of 3?

Does anyone ahve[sic] any idea of where to start looking

You fell victim to one of the classic blunders. Never use a vec3 in a UBO or SSBO.

While vec3s take up 12 bytes, they always have the alignment requirements of a vec4. Yes, even under std430 layout, they always have vec4 alignment. So if you have an array of vec3s in a U/SSBO, what you really have is an array of vec4s where one of the elements doesn't get used.


I don't think uniform arrays can be dynamically sized. In your case you should define the array as the maximum number of lights you will process and then use a uniform to control the number of iterations you do on this array. On the CPU side you can set a subset of the lights[] array according to the 'size' variable.

e.g.

#define MAX_LIGHTS 128

uniform int size;
uniform SceneLights lights[MAX_LIGHTS];

void main()
{
    for(int i = 0; i < size; i++) {
        /* Do Some Calculation */
    }
}

I think (speculate) the likely reason for this is that it would be impossible to determine the location of a uniform if there are variable sized arrays in your uniform list which depend on the value of another uniform.. e.g.

uniform int arraySize;               // offset 0
uniform int myArray[arraySize];      // offset 4
uniform int anotherVar;              // offset ????

GLSL would not know were to place anotherVar, because it would need arraySize to be set with a value before it can compute the offset of 'anotherVar'. I suppose the compiler could be clever and re-arrange the order of uniforms to get around this, but this will fail again if you have 2 or more variable sized arrays..


How to display edges in a 3D file on Facebook

3D graphics are not images they are instructions for images. Now the formats that export data beween systems work on lowest common denominator. Geometry is fine, but any effects on top of geometry would require you to agree on a set of instructions to implement. (no such reffort has been made for npr rendering lines).

The lines are not really part of the model that has been exported trough the gltf pipeline. Simply because the line rendering is a special feature of the sketchup system. There is no generic format that support those, since they need to be generated frame by frame. And there is
quite possibly a infinite number of such features that could be implemented, none of these are part of the generally accepted set of things to export.

Most likely your out of luck the engine in question simply has no way to do this. So there is no solution for your problem. STL is a even more rudimentary format it only has triangles nothing else.

PS: This is why it is called an export not save. Although many applications have gone so far as to call even partial implementations save. But you can not just expect all things to go from application to application, especially in 3D


Is `groupshared` memory stored in L2 cache of GPU?

There seems to be some confusion of terminology here. In Direct3D, you have threads and thread groups. "work item" and "work group" are generally encountered in OpenCL terminology, where a "work item" would be what a thread is in Direct3D and a "work group" corresponds to a Direct3D thread group. groupshared memory is memory accessible to all threads that are part of the same thread group.

It is unclear what exactly you mean by "manipulating L1 cache directly". A cache is a hardware component that is used to transparently optimize memory access. The word "transparent" is crucial here. While, at the hardware level, caches necessarily contain some form of memory, they are generally not explicitly "accessible" as a region of storage. You can't have a pointer that points into a cache. You can have a pointer that points into memory. Memory accesses will typically go through a cache, i.e., any memory request will be handed off to the next cache, which can then check whether it can serve the request directly or has to go further down the memory hierarchy. There's more to a cache than just the memory it uses to cache stuff. A cache also contains additional logic for managing the cache on top of just the storage itself. Caches are implicit by nature. As you manipulate memory, you will typically also cause some caches to manipulate their memory along the way. But you generally can't "manipulate" a cache directly in the sense of explicitly defining what is when to be stored where in the cache. As soon as you can explicitly point into it and read/write from/to it, it kinda stops being a cache and starts being just some form of storage.

That being said, groupshared memory is generally located in fast on-chip storage just like caches are. The whole point of groupshared memory is to take advantage of the knowledge that certain threads reside on the same part of the chip to allow them to communicate faster. And on some hardware (e.g., most of the recent NVIDIA architectures), groupshared memory and the L1 cache will actually use the same physical memory. However, that just means that one part of that memory is used as "normal" memory, accessed directly via addressing through some load-store-unit, while another part is used by the L1 cache to cache stuff…


Efficient rendering with many light sources

Yes, but you need a paradigm shift.

What you are accustomed is called forward rendering. You submit your geometry and then you proceed immediately with the shading pass. In the basic forward rendering you can either loop inside the shader for each light or perform one pass per light and blend the result together (with additive blending).

But things have evolved quite a lot. Enters: Deferred Rendering

Now that are so many variants that describe them all in details will take way more than acceptable for an answer here. So here I am just going to describe the gist of Deferred shading, there are plenty of other resources that you can easily find using google, hopefully after reading this you'll have the right keywords to find what you need.

The basic idea is to defer the shading to after down the pipeline. You have two main steps:

Render your geometry and all the information needed for shading into multiple render targets. This means that typically in a basic implementation you would have a depth buffer, a buffer containing the normals of your geometry and the albedo colour. You'll soon find that you need other information of the materials (e.g. roughness, "metallic" factor etc.).

This image from wikipedia shows three buffer (color, depth and normals)

Again, the amount, type and content of the buffers used varies quite a lot among different projects. You will find the set of buffers with the name of GBuffers.

After this is the time of applying the actual lighting. During the lighting pass for each light you want to draw a light volume that depends on the type of light:

For a directional light you render a full screen quad.
For a point light you render a sphere where the radius is based on the attenuation of your point light.
For a spot-light you render a cone which dimensions again depend on the characteristics of your light.

In the pixel shader of this pass you pass your GBuffers and perform your lighting and shading using the information in them. This way you process only the pixels affected by each of the lights having a sensible speed-up if compared to the classical forward rendering.

It has also various disadvantages, most notably the handling of transparent  objects and higher consumption of bandwidth and video memory. But also it is trickier to handle various models for materials.

You have other side-advantages (as having lots of info ready for post-processing) and is also pretty easy to implement. But this is not the coolest thing around for lots of lights anymore.

Newer techniques are for example Tiled rendering ones. The main idea of those is to subdivide the scene in screen space "tiles" and assign to each tile the lights affecting it. This exists both in a deferred and forward fashion.  These techniques lead to some problems when you have various depth discontinuities in a tile, but is generally faster than the classical deferred and it solves various problems of it. For example, among the advantages, with tiled deferred you read the GBuffers once per lit fragment and pixels in the same tile coherently process the same lights.

Further evolution on this side is the Clustered shading which is conceptually similar to the tiled based approaches, having instead of screen space tiles, clusters with a 3D extent. This method handles better the depth discontinuities problem and generally performs better than the tiled methods.

IMPORTANT NOTE: I have described the basics of the deferred shading. There are multiple variations, optimizations and improvements around so I urge you to experiment with a simple version and then do some researches on other techniques such the one I mentioned above.


How to test if one object A intersects and is therefore partly or wholly inside object B

Do you need exact triangle - triangle intersection test that is do you need to know exactly the points of intersection of each object? If you just need a simple yes / no they have intersected then you could get away with creating an AABB ( https://en.wikipedia.org/wiki/Bounding_volume ) that encompasses each object and do a AABB -> AABB ( good info here https://studiofreya.com/3d-math-and-physics/simple-aabb-vs-aabb-collision-detection/ ) intersection test? even if you need a more complex collision check Triangle -> Triangle it is always a good idea to do a simple collision check first to make sure its worth doing the complex Triangle -> Triangle intersection tests. Look here ( http://www.realtimerendering.com/intersections.html ) for all the normal object -> object intersection tests you could possibly need


How to take consistent screenshots across multiple machines?

Nicol Bolas explains why you can't expect pixel-accurate reproductions, but that doesn't mean you can't test with screenshots.

Don't compare the screenshots byte-for-byte. Instead, use a perceptual diff to quantify how different the renders are. You can then set a threshold in your test and have the test fail if the renders are too different. A harsh threshold will let you filter out the rounding differences between platforms, and the slight differences around object silhouettes, but still detect objects in the wrong place or inconsistent texturing. A more forgiving threshold will allow bigger differences in texturing (or even with different shader versions or extensions, if you have different rendering paths for different hardware), and only complain if you have gross differences such as missing lights or a completely different scene being rendered.

The pdiff tool is commonly used for this task, and I've set up test systems based on it before.


Calculate an equally bright grey to a linear RGB colour

The easiest way to get the perceived brightness of a color is to calculate the Luma. Finding the grey color with the same luma is easy - just set all of the RGB components to the desired luma value. That works because the coefficients for calculating the luma sum up to 1.

public Vector3d get_equally_bright_grey(Vector3d color) {
    double luma = color.x * 0.299 + color.y * 0.587 + color.z * 0.114;
    return new Vector3d(luma, luma, luma);
}


Old Hardware
Some older cards didn't used to jump instructions in warps, so this was indeed an issue. If you have a conditional with these cards, the inside logic of the conditional would still be evaluated even if the block wasn't entered by any of the pixels, vertices, etc. There's little you can do in this case because every instruction will be executed. The worst case is also the best case here.
Modern Hardware
However, if you are working with relatively modern hardware though, this isn't an issue and "if/else" blocks will be skipped when necessary. There really shouldn't be a downside to using "if/else" blocks compared to not using them. You are still branching with an "else"-less statement. In fact, some post-processing effects/deferred rendering specifically benefit from this.
The worst case scenario is that some threads go through the "if" block and the rest go through the "else" block. The best case scenario is that all go through the "if" block or the "else" block. There isn't a downside to using "if/else" conditionals as they are needed (either on old hardware or newer hardware), but there's a possible upside with newer hardware.


Why is the z direction away from the screen/viewer?

If you want to use the traditional x is towards right and y is up you are used to in math and physics classes. Now if you follow a right handed coordinate space, which is also commonly used. Then the direction of the z follows from the x and y direction.  In this sceme positive z is pointing out from the screen.

Now you can also choose a left handed coordinate system in which case the z is reversed. Its just a matter of choosing which convention you want to use.

Likewise if you choose y to be down, ala traditional windowing math then z is reversed even if you use right hand rule.

None of these conventions are set in stone you can do whatever you want. A bit like you can choose row major or column major matrixes. You just need to know what comvention you follow.


What remedies are there for percieved color artifacts due to pixel geometry?

In computer typography, the technique for adjusting strokes so that they align with the pixel grid is called hinting.

The color artifacts you are seeing are due to another technique called subpixel rendering.


How can I make a graphics demo without a frame buffer?

This is a case where looking at the workings of more retro hardware would be reasonable. Older hardware had strong limitations in both memory and processing power. Your 100+Mhz chip is far faster than most consumer-grade chips of the 80s and before. So while those CPUs needed dedicated graphics chips to be able to render, your much faster CPU can probably handle the task adequately.
I would suggest starting with something like the NES's tilemap-and-sprite architecture. These architectures are designed to be both memory efficient (getting a lot out of limited storage space) and computationally efficient in a line-by-line output, as they generated pixel data and sent it directly to the display device at the expected speed of a CRT.
The broad idea is this. A tilemap consists of two parts: a series of tiles and a 2D array of indices into the tile map that represents the image being created. The tiles are typically 8x8, and on the NES were 2-bits-per-pixel and the tiles used palettes. Or more specifically, the index into the tile map includes not just the index of a tile but also the index of the palette to use with that tile. So a tile is not inherently associated with a palette; the association is made at the point of use (technically, on the NES, the tile map and palette map were separate, as each 2x2 block of tiles all had to use the same palette).
To allow for scrolling, the tile map is bigger than the visible screen, and there is an offset that represents where the top-left corner of the visible screen is within the tilemap. Let this offset be at position Xoff and Yoff.
You can see how this makes row-by-row processing trivial. To generate a horizontal row for the horizontal position Ypos (in screen-space), you need to get the starting pixel within the tilemap. That requires transforming the XY position (0, Ypos) from screen-space to tilemap space. So you add to it the vector (Xoff, Yoff), hielding the vector ``Xoff, Yoff + Ypos)`.
Note that when doing any mapping from screen-space to tilemap space, that the tilemap space should wrap around in both the X and Y axes. So whenever you compute a new pixel in tilemap space, you have to wrap it around the size of the tilemap space.
Now, we need to break down this tilemap pixel into two 2D components: the tile index within the tilemap that gives us this this pixel, and the pixel within that tile that we need to fetch for this pixel. The tile index is just the tilemap-space pixel integer-divided by the tile size. The pixel coordinate is the tilemap-space pixel integer-modded by the tile size. Given an 8x8 tile size, you're doing this:
ivec2 tilemap_pixel = ...; //Compute the tilemap starting pixel as above.
ivec2 tilemap_tile = ivec2(tilemap_pixel.x & ~0x7, tilemap_pixel.y & ~0x7); //Mask off the lower 3 bits.
ivec2 pixel_in_tile = ivec2(tilemap_pixel.x & 0x7, tilemap_pixel.y & 0x7); //Mask off all but the lower 3 bits.

So given tilemap_tile, we now have the index of the tile we are currently working within. And pixel_in_tile gives us the pixel we're working in. So we can fetch that pixel, whose value can be mapped into a palette to produce the final color for that pixel. Or we can use it directly.
To get the next pixel is pretty trivial. We just increment pixel_in_tile.x by 1, modulo the tile size. If the increment exceeded the tile size, then we also increment tilemap_tile.x by 1, modulo the tilemap size. And you keep going until you've filled out the row of pixels.
There are many opportunities for performance optimization of such an algorithm.
The tiles probably represent the largest data. A 128 element tile set of 8x8 tiles, even at 2bpp, is 2K. But the thing is, those can be stored in ROM, since you're probably not changing the tiles themselves. The size of the tile map (assuming it must be in RAM) depends on the desired output resolution and the tile size. A tile map for 8x8 tiles that can cover a 320x240 screen is 1,200 bytes. Not exactly tiny. You'll need to take up more memory if you want room for smooth scrolling (and thus a larger tile map).
That being said, it's also important to note that the "output size" doesn't have to be the actual size of the display device. For example, if you intend to draw to a 1080p display device, you could still render to an internal resolution that's, for example, 8 times smaller, 240x135. It's easy to modify this algorithm to essentially generate the same pixel value 8 times in a row, as well as reusing the same line 8 times per 1080p line. Indeed, it wouldn't be too difficult to have this algorithm allow for subpixel scrolling (scrolling in 1080p space, not 135p space) and even adding some filtering between pixel values. A 240x135 "output size" tilemap would only require 510 bytes for the visible region, so you would have more room for a larger scrolling area.


The easiest way to deal with this would be to provide thickness for the edges in the continuous setting. That is, make your edges out of solid capsules/cylinders, then you would not have this issue. Technically, this is is neither a supersampling nor a filtering technique, but rather a reformulation of the problem in the continuous setting. Another equivalent reformulation is considering only the vertices and then your reconstruction function being thick edges between those (using a thick line algorithm).
This doesn't seem to be your setting however? Since you mentioned that you only have the raster (aliased) image already, in which case you have already introduced the error so it is hard to get rid off. If you can "access" the continuous model it is a lot easier to fix. If you cannot, then I could recommend using a different reconstruction filter than the dilation that you are using. For instance you can do a thickening based on distance to pixels (sdf) with a step function. This should remove the issues you are having, at least for the example you showed.


Is there any 3D software that supports rendering an artificial scene to a light field image?

I took a look at Myst and realMyst, and found this YouTube video: "LightField synthesis & realtime render in Unity - sneak preview (raw footage)" with accompanying background introductory information from Frooxius on Reddit. You can see the pipeline he used to convert lightfields, though copying it may be a challenge. He has obtained pretty good results.

So the answer is yes.

  I guess animations would be out almost of the question. It should at least in theory be easier to incorporate animations in cgi-light fields than in real photos.

It's an interesting experiment but the scene is baked, adding characters or new objects will kind of look like cutting out one photograph and pasting onto another - thus what you hoped to gain in speed is lost trying to smooth the two scenes together.

To walk into a scene and see a moving object you essentially need to create a dynamic 'lightfield movie' so the moving object can move normally and then you simply walk/look around the dynamic lightfield. That will be somewhat computationally intensive but most of all it will take a lot of memory.

Check out Frooxius' video and let me know if that's the look you were hoping for.

This webpage: "Light fields – the future of VR-AR-MR" by Mike Seymour (May 26, 2015) says:

  "The Octane renderer has plugins for a range of products from 3DsMax to Maya and even Nuke.".

There's mention of OTOY and on their website the latest announced is: "OTOY and Epic announce OctaneRender® 2019 for UnrealEngine 4", their DEMO version is free, the least expensive paid version is U$400.


splitting hue & saturating and leaving value behind

Removing hue and saturation ("desaturating") leads to a grayscale image with the same luminance as the original colors, for instance:

(source: my own photo)
It is not possible to remove hue alone without removing saturation, as any saturated color must be saturated in some direction (hue). Nor is it possible to "split" the image into HSV "components" in quite the way you can split into RGB components. Hue works like an angle of rotation around the "color wheel", while saturation is the radius from the center:

(image from here)


Synchronizing successive OpenGL Compute Shader invocations

No it is not guaranteed, since the OpenGL specification allows that two Compute Shader run concurrently or even in different order.

You need to call glMemoryBarrier(GL_SHADER_STORAGE_BARRIER_BIT) before the second glDispatchCompute to ensure visibility of the writes from program_one.

From the OpenGL.org wiki article on the memory model:

  [...] invocations between stages may be executed in any order. This includes invocations launched by different rendering commands. While it is perhaps unlikely that two vertex shaders from different rendering operations could be running at the same time, it is also possible, so OpenGL provides no guarantees.

From the Opengl.org wiki article on Shader Storage Buffer:

  SSBOs reads and writes use incoherent memory accesses, so they need the appropriate barriers, just as Image Load Store operations.


Do you need to use a lowpass filter before downsizing an image?

If the downsampling pass is properly designed, it will effectively perform low-pass filtering as part of the downsampling.  There is no need for a separate low-pass filter operation.

Essentially, when you downsample, you are performing a filter over the source (high-res) image pixels, but only evaluating it at the locations of the destination (low-res) pixels.  The footprint of this filter needs to be approximately the spacing between destination pixels, to avoid missing information by skipping over in-between source pixels.  But that means the filter footprint will be several source pixels wide, so it will effectively low-pass the source.

For example, let's suppose you downsample an image by exactly 10x on each axis.  With a box filter (for example's sake), you would set each destination pixel to the average of a 10x10 box of source pixels.  That would wipe out any features smaller than 10px, so it's effectively a low-pass filter.

You mention bicubic interpolation; we have to make a distinction between filtering and interpolation here.  Interpolation is appropriate for upsampling, not downsampling.  Bicubic interpolation works by fitting a bicubic spline patch to a 4x4 neighborhood of pixels, then evaluating the patch at interpolated points. While it may work well enough for downsampling images by a small factor (up to 2x or so), it will fail if you go much further than that.  For instance, if downsampling by 10x as in the previous example, you can see that bicubic will miss the majority of the source pixels, and the result may be quite aliased.

On the other hand, bicubic filtering is just standard filtering, using a kernel that's a bicubic function (as opposed to a box, triangle, Gaussian, Lanczos, etc. kernel).  The Mitchell-Netravali kernel is the classic example of this type.  If used for downsampling, the kernel should be sized appropriately for the destination pixel spacing as discussed earlier, and you would sum over all the pixels in the footprint, not just a 4x4 or other fixed-size neighborhood.


An interesting problem. I've done a bit of work in texture compression and this sounds something like a generalisation of Campbell et al's "Color Cell Compression".  It's also a little like a feature we were asked to include in the Dreamcast VQ compressor so that sub-palettes could be swapped to create different colour schemes on textures.

I was thus thinking that maybe you could adapt a standard VQ algorithm, applied in stages, to the problem.

You say that

  "the exact colors, the arrangement of colors into subpalettes, and the subpalette for each tile of the image are all unknowns"

but can we assume the X*Y tile size is predetermined?

Assuming that is the case, perhaps the following might be worth trying:

1) Guess an initial palette size for the entire image, perhaps,
 $$Psize=(M * SizeOfEachSubpalette)$$
where $1 < M < N$.

Use standard VQ quantisation to generate this palette and map the pixels to it.

2) Now consider each $X*Y$ tile to be a "super pixel" with $Psize$ channels. Set each channel value in the tile's "super pixel" to the number of pixels that mapped to the corresponding colour in our initial palette.

For example, if in our initial palette, entry $j$ is [0x11,0x21,0x31] and 5 pixels in the tile mapped to that colour, set the $j'th$ component in the tile's superpixel to 5.

Run a second VQ operation on the set of all "super pixels" using '$Psize$-dimensional space' to generate $N$ "palette" entries. (Actually, you don't need the 'palette' values - the VQ process is just meant as a means of partitioning the tiles into $N$ sets)

Hopefully, this will now identify which tiles should share the same sub-palette.

3) For each tile set, collect all the pixels in the tiles, and apply a 3rd VQ operation to identify the $SizeOfEachSubpalette$ colours needed for all tiles in that set.

I would think some sort of clever GLA/k-means process might be needed for step 2 to juggle the assignments of tiles to sub-palettes. The choice of M might also be interesting.


Simple Intersection Algorithm for Ray and 3D Bézier Curves of Varying Thickness

You say that you don't need the fastest possible algorithm, so instead of solving it analytically, you might try considering the numeric alternatives, which in this case could mean ray marching.

How you could do this is take steps down the ray and test if the point is inside or outside of this bezier tube.

You would iterate down the ray until you either find an intersection or leave the bounding box of this shape (or maybe when you take a maximum number of steps, or some other fail case).

The challenge then only becomes "how can I tell if a point is inside or outside of the shape?" which is a lot simpler.  If you had constant thickness you would just find the closest point on the bezier to your point and do a distance comparison.

Some details you might be interested in:

taking smaller steps, and more of them, will give you more accurate results, but will be more expensive computationally. That makes this algorithm tuneable for quality vs speed.
if the step size is too large or the tube is too thin, you may completely miss the tube when there is really a hit.
when/if you do find a hit, you are finding the first point inside the shape. The real answer is somewhere between the last sample and the current sample.  You could refine the hit location with a binary search or some other method if you wanted to.
you can likely increase the step size as you move farther from the camera. Smaller details are harder to see at greater distances.
you might be able to calculate or approximate the gradient of your point vs bezier tube function as you ray march. If so, this gradient can be used to give you a lower bound on the distance from your point to the bezier tube, which would let you take larger steps through empty space.

Regarding the "secondary ray" issue you mention like for shadow rays or reflection rays that comes up with both analytical and numerical solutions.  Getting a (probably) less precise answer than a numerical solution does make the problem worse, but there should be ways to get around it.

For instance, after you detect a hit, you could find a point on the surface near the hit even if it isn't where the hit was (like, push the hit point to the surface, in whatever direction is easiest to compute), add an epsilon (needed for analytical solutions anyways) and spawn secondary rays from there.

Another way could be to have every object have an ID and when ray marching / tracing, don't count intersections with the same ID as what you just hit.  This only really works for opaque convex objects though.


Bug with the Bresenham's line drawing algorithm?

Note that the approach you appear to be taking will only work for lines with a slope in a single octant. For the other seven octants the algorithm requires changing the approach, for example by changing the roles of x and y. Wikipedia gives a table of cases that can be used for this conversion.

For this answer I will assume you are simply trying to get the code working for the first octant.

int parameter = 2 * dy -  dx;

parameter starts off negative only if dx > 2 * dy.

In this case the total horizontal distance to be covered by the line is more* than twice the vertical distance, which means the first step should be purely horizontal. So if parameter is negative then y0 should not change.

Here is the source of the problem:

if(parameter < 0){
           y0 += 1;

y0 should only be incremented if parameter is not negative. This y0 += 1 is causing vertical movement even if the first step should be purely horizontal.

The same problem applies on later steps too. The code causes parameter to switch back and for between negative and non-negative in proportion to the ratio between 2 * dx and dy. y0 must only be incremented when parameter is non-negative. That additional occurrence of y0 += 1 prevents y0 from ever staying still, causing a slope of 1 regardless of the inputs. Simply remove that line and it should work correctly.

*If you are wondering why the algorithm only steps purely horizontally when the parameter is less than zero, rather than less than or equal to zero, bear in mind that the horizontal distance to be covered is one less than the number of pixel columns displayed. If x0 is 0 and x1 is 3, then dx is 3, but 4 columns of pixels will be displayed. So a line 4 pixels across and 2 pixels down actually has xd as 3 and yd as 1. In this case the number of pixels across is exactly twice the number of pixels down, but xd is more than twice yd, so this still results in the parameter being negative and the line moving purely horizontally on the first step.


Which method for detecting edges of white object on white background?

It's quite difficult. There is so little contrast, even the JPEG compression artifacts have more contrast than the object on the background.

You would require a highly specialized deblock filter to eliminate the compression artifacts first. With knowledge about the block size of the used compression algorithm and the number of coefficients used per block you may be able to predict some of the edges.

For edges you could predict in the previous step, you may try to filter these from the detected edges.

All around the shirt in the photo, there are over-swings due to an excessively  lossy compression. At the edge of each block, the over-swings form a hard edge which you also successfully detected.

The block size is 8x8 pixels for JPEG (and hence also in this image), so every vertical or horizontal edge which falls directly onto position 8*n or 8n+1 X or Y is most likely just a compression artifact and can be ignored.

This approach can only work though if the image hasn't been re-sampled after compression, respectively hasn't been re-compressed multiple times with potentially different block sizes. At that point, isolating the compression artifacts becomes nearly impossible.


Presumably you are using a lighting system already, let's say Lambert shading for simplicity..

So we have a texture which is going to be illuminated by some light source. The following psuedo/shader code will do lambert shading as normal..

diffuseIllum = lightColour * dot(normal, lightDir);
outputColour = textureColour.rgb * diffuseIllum;

But we want certain parts of our texture to emit colours instead of being illuminated by only our light source.

One method would be to use an extra channel in the input texture, say the Alpha channel, to control which pixels are emissive

diffuseIllum = lightColour * dot(normal, lightDir);
outputColor = mix(textureColour.rgb * diffuseIllum, textureColour.rgb, texture.a);

Unfortunately this won't work so well if you want say a dark red glow because the output pixel will either be the emissive colour (dark red) or the lit texture colour, which could be much brighter. In effect our emissive areas could end up darker than the surrounding illuminated areas.

We could add the emissive colour to the illuminated texture but that limits how much control over final colour we have.

e.g.

outputColour = textureColour.rgb * diffuseIllum + textureColour.rgb * textureColour.a;

A more correct method would involve using 2 textures, one for diffuse illumination and another for purely emissive. These would be added together in the pixel shader.

e.g.

 outputColour = textureColour.rgb * diffuseIllum + emissiveTextureColour.rgb;


Two suggestions:

If your data is from an image you are displaying on a standard monitor, the chances are it is (or you are implicitly assuming that it's) in sRGB format. This means that the colour components are not linear. Ideally, you should first map into a linear colour space, do your filtering (e.g. blurring) operations, and then map back.
*(If you just want to try a cheap and cheerful approach, squaring each of the R G B components is not a terrible approach to "linearise" the values

obviously, use sqrt() to map back)*.
The alpha channel can be assumed to be linear.

Once in linear colour space, the next thing you want to do before filtering is to convert the pixels to a pre-multiplied-alpha format (AKA "associated colour").
Basically, given 8bpc {A,R,G,B} pixel input, produce  {A, A*R/255, A*G/255, A*B/255}.  You then do your averaging on this premultiplied data as above on all four channels, and then map back by dividing through by the resulting alpha (if A=0 then, choose, RGB=0).

This should produce a better result.
For more info I suggest reading through Jim Blinn's "Compositing, Part1: Theory".
Particularly, note the line "Additionally, we must use associated colors for any filtering or interpolation operations"

Update: Seems I've posted a similar answer before


OpenGL compatibility, naming conventions and ARB vs EXT

Question 1 -- Usually the EXT version comes first as a collaboration between two or more vendors. ARB extensions require more discussion among voting members of Khronos and can have changes from the EXT version before getting approved. See the GL_ARB_direct_state_access extension which has many changes compared to GL_EXT_direct_state_access.

Question 2 --
The Issues section of the GL_ARB_framebuffer_object extension states why the functions do not have ARB suffixes:

  (8) Why don't the new tokens and entry points in this extension have suffixes like other ARB extensions?

   RESOLVED: Unlike most ARB extensions, this is a strict subset of
   functionality already approved in OpenGL 3.0. This extension
   exists only to support that functionality on older hardware that
   cannot implement a full OpenGL 3.0 driver. Since there are no
   possible behavior changes between the ARB extension and core
   features, source code compatibility is improved by not using
   suffixes on the extension.

Question 3 --
If you want to use framebuffer objects on a context where the GL version is less than 3.0 you need to look at the extension string:

If GL_ARB_framebuffer_object is supported, use the non ARB functions.
If only GL_EXT_framebuffer_object is supported, use the EXT functions.
If neither extension are supported, you need to fall back to OS level offscreen rendering like pbuffers.


Is there a way to script image creation?

ImageMagick is a set of command-line tools that can do the sort of things you describe. For example, this command line will overlay picture B with a centered copy of picture A, resized to 100 pixels wide (keeping aspect ratio):

convert pictureB.png ( pictureA.png -resize 100 ) -gravity center -composite output.png

See the ImageMagick docs for more info about its command lines are structured...it's a powerful piece of software, but it takes a bit of work to get the hang of how to use it.


How to triangulate from a Voronoï diagram?

The centre point in your diagram is a degenerate edge of the Voronoi diagram. If you generate a Voronoi diagram for an irregular point cloud, every vertex will have degree 3. A vertex with degree 4 (or more) can only happen when two (or more) vertices coincide. That means there is a zero-length edge between them. But that edge should still have a corresponding edge in the Delaunay triangulation. The issue is that it's arbitrary which of the two possible edges you pick, because the zero-length edge has no associated direction.

To visualise what I'm talking about, consider starting with four less regularly spaced points (such that we start off with only degree-3 vertices) and gradually translating them into their regular positions.

We can do this in two different ways, which both lead to the degenerate case in your diagram. You will see that you end up with two different Delaunay triangulations, which are both valid limits for the degenerate case:

I assume that your code is missing this degenerate case for one reason or another, but without actually seeing how you compute the Delaunay triangulation from the Voronoi diagram it's impossible to point you any further than this.

Also note that having even higher degeneracies (by more than four points distributed at equal angles around a circle) would probably require additional attention:

These animations also show that (even in a non-degenerate case), the corresponding Voronoi and Delaunay edges do not necessarily actually cross within their finite extent. That might make it harder to see that the 2 (or 3) edges that triangulate the regular polygon at the end actually correspond to several degenerate edges which are all at the centre. Also note that in total there are 5 different triangulations of the pentagon and 14 triangulations of the hexagon (although I don't know whether all 14 can be obtained by deforming a non-degenerate triangulation).

Edit (by OP)

Voronoi diagrams computed with Boost.polygon enable to walk through each Voronoi vertex, and each edge linked to those vertices (clockwise or counter-clockwise). In this way, it is possible to create one triangle for each pair of edges (two connected edges will link to 3 cells).


This has to do with gamma correction.

If one pixel has red component with value of 1 (where 255 is max), the next pixel has value for red that is 2, there no guarantee that exactly twice as much photons are exiting from the second pixel. Displays have different curves that predict what's the expected brightness. It also has to do with how our eyes work: they're more sensitive for darker shades than brighter shades.

To read more about these phenomena:

https://gamedevdaily.io/the-srgb-learning-curve-773b7f68cf7a#.di88vr58t
http://marcinignac.com/blog/pragmatic-pbr-setup-and-gamma/ (gamma section)
http://colour-science.org/posts/the-importance-of-terminology-and-srgb-uncertainty/
http://www.essentialmath.com/GDC2015/VanVerth_Jim_DoingMathwRGB.pdf

The three images are used as low value, medium grey and bright examples. Indeed, the lowest brightness is used so you are able do see the difference between the black background and "barely lit" object, and the other two images are just used to display other important values for the set gamma curve.


What happens when you display a high resolution image on a low resolution screen?

If you scale an image down to a screen that has a lower resolution than yout image, you have to descide what you do, there is no single way of doing this.

There are many different techniques that yield very different results.
This is very similar to displaying a low resolution image on a high resolution display where do the inverse thing by desciding how to fill the "more" pixels on the screen. There are also many techniques for doing this.

The image shows two upscaling-techniques and shows how different the results can look if different techniques are used. (A nearest-neighbor scaling on the left and a 2*Sal on the right.) Imagine that there are also such differences if you downscale an image.

You can find a nice overview on different scaling techniques on the wikipedia page for Image scaling.


How to remove background text from an image file

It is all about filtering, you can see there are 4 colors, i.e. C_Font, C_Paper, C_Watermark_Paper, and C_Watermark_Font, the filter is just f(C_Watermark_Paper) = C_Paper and f(C_Watermark_Font) = C_Font.
Of course you can use GIMP and easily Google references, i.e. http://howto.nicubunu.ro/gimp-remove-watermark/


Sorry folks for posting such a trivial issue! The issue is solved. I was using the wrong function. Here goes the correct one:

glm::vec2 testVec(6,-4);
float len  = glm::length(testVec);

The member function of the same name returns the number of components instead (i.e. vec2::length will always yield 2, vec3::length will always yield 3, etc.).


OpenGL set a vertex element stride of zero

Disable the vertex attrib array and use glVertexAttrib to set the data that should be used.


Are vertices reprocessed per-poly for indexed meshes?

The key expression you may be looking for is "Post Transform Cache".

This is usually effective because of the natural vertex reuse, but reordering your mesh, e.g. with Hoppe's method (note: the page also lists some more recent work) or Forsyth's can improve things further.


Can a scene be efficiently pre-rendered and displayed along a fixed camera line?

Would Diego Nehab et al's "Accelerating Real-Time Shading with Reverse Reprojection Caching" be the sort of thing you are looking for?

Paper Link and/or Slide presentation from Graphics Hardware 2007

IIRC, the idea is to reuse a previous frame's rendered image results in the current frame rather than running a, potentially expensive, shader for every pixel.

At the conference, I did ask Diego if he'd considered doing 'bidirectional' reuse (i.e. rendering frames out of order somewhat akin to h.264) but I can't recall his answer. I think in your proposed use case it might be of value.


In theory yes, but practically speaking, I doubt it would be very easy. First, the color may not be representable by the color space that your camera of choice produces images in. We've recently started seeing wide color gamut (P3) cameras and displays. There are colors these cameras record and displays produce that fall outside of the sRGB color space that most cameras and displays support. Likewise, there are visible colors that are outside of the P3 color space. So the first hurdle is to have a device which records and stores the colors you're dealing with.

Next you have to do the lighting correction once you get the image into the computer. This is actually not as hard as it sounds. There are products that allow you to do this using a system similar to what you describe. Here's a video for one that works with video rather than stills.

Once you've done all that, you need some way of translating that color from some RGB space into something that your local paint store can understand. I'm not sure how to do that. Most paint store employees are not trained in color science and have to use a limited toolset provided by the paint companies.

Is there any way you could get a small, inconspicuous piece of the actual object? Most paint stores have a machine that will take a physical object and create a paint that's a fairly close match to that object. Whether their matches are close enough for your needs is something you'd have to determine. I've used them before and found that while they get very close, I'm still able to see the difference between the original and the new paint, unless I can paint up to corners.


Video-games; Rendering textures on scope zoom-in. Texture compression problem?

Well there's no way to know for sure unless you look at the source code but my guess is that they do zooming by lowering the FOV (field of view) of the camera. This is easy to implement and actually sort of represents how a real scope works (in the way that light rays are bent through an optical lens). A zoom effect where you change the actual position of the camera might be more trouble than its worth. What if there is an object between the player and the desired camera position? So as you can see, the camera position has not changed at all and so there is no reason the textures would update when zooming.

Now you would think that hey maybe they could just scale the distance required to trigger the higher detail textures as the player zooms in but this may or may not be easy depending on how that is implemented in the engine. The game developers may not even be able to change LOD (level of detail) distances dynamically. Like you said, games like sniper elite which are based all around sniping have dealt with this but general purpose game engines might not have this functionality.

Also you have to consider possible popping effects. If you zoom in to an area and suddenly the textures and meshes update dramatically, this would look very bad. This gets even worse if the zoom effect is gradual or the sight is adjustable. I would even go as far as to say that it might be preferable to have lower quality textures than to have really bad popping.


You can use barycentric coordinates to calculate points inside the triangle with a determinate value for uv.

The process is easy =>

Let's say that the UV coordinates of each vertex of the triangle are:

vertex 0 => u0,v0

vertex 1 => u1,v1

vertex 2 => u2,v2

A point inside the triangle has UV coordinates value expressed in function of its barycentric coordinates:

u coord = p0 * u0 + p1 * u1 + p2 * u2

v coord = p0 * v0 + p1 * v1 + p2 * v2

where p0,p1,p2 are the barycentric coordinates of this point.

As barycentric coordinates has the property:

p0 + p1 + p2 = 1

you can calculate the barycentric coordinates of a point with a specific value for UV just by solving these equations. I mean, for example, you want to know which point inside the triangle has

UV = (1,2)

To calculate such a point just do:

1 = p0 * u0 + p1 * u1 + p2 * u2

2 = p0 * v0 + p1 * v1 + p2 * v2

p0 + p1 + p2 = 1

as you know the values for (u0,v0), (u1,v1) and (u2,v2),  you can easily calculate the barycentric coordinates of this point, (p0, p1, p2).

Once you have the barycentric coordinates, you can calculate the coordinates (px,py,pz) of the point:

(px,py,pz) = p0 * vertex0 coordinates + p1 * vertex 1 coordinates + p2 * vertex 2 coordinates


How does the OS + hardware support multiple screen resolutions and multi-display?

Any hardware in the graphics card is fixed.

No, the storage of a graphics card is fixed. How that storage gets used, particularly on any semi-modern graphics card, is flexible.
Essentially, what a graphics display device has is a mechanism for being told, "the bytes of video memory starting at address X is in format Y and size Z; send it at the next vblank period to display device W". Really old graphics cards had specific memory structures for holding framebuffers, which imposed strict limits on how things could work. But that hasn't been the case on PCs for a good decade at least.
There may be limitations on how many display devices a particular piece of hardware could hypothetically output to, or the sizes and formats which the hardware can use. Some hardware devices may only be able to use non-virtualized GPU addresses (which I'd guess is why NVIDIA has special memory types for allocating render targets) or somesuch, but even that is a matter of the user interface, not what the internal hardware is doing. As far as the display device's output is concerned, it could use more or less any GPU memory address.
So, what happens when you plug in a new display device? The OS allocates a framebuffer's (or two) worth of video memory for that display device's video output. If you change the resolution of a display device, the OS allocates a differently-sized slice of memory and frees the prior framebuffer allocation. And so forth.


What Are The Default Values of `glReadBuffer/glDrawBuffer(s)` for an FBO?

You can usually track down the answer for such questions in the OpenGL specification.  Assuming the value you care about is consistent across OpenGL versions (often the case for initial values, seldom the case for minimum values).

In this case, in the GL 4.5 specification, check out the following:

glReadBuffer(...) initial value: (default framebuffer: GL_FRONT iff single-buffered, GL_BACK iff double-buffered) (FBO: GL_COLOR_ATTACHMENT0) (see page 506)
glDrawBuffer(...) initial value: (see glDrawBuffers(...))
glDrawBuffers(...) initial value: (default framebuffer: fragment color 0 goes to GL_FRONT iff single-buffered, GL_BACK iff double-buffered) (FBO: fragment color 0 goes to GL_COLOR_ATTACHMENT0) (both: fragment colors other than 0 go to GL_NONE) (see page 497)


Total emitted power of diffuse area light

Radiance (in terms of flux) has the following definition:

$L_o = \frac{\mathrm{d}\Phi}{\mathrm{d}\omega^\perp\mathrm{d}A}
= \frac{\mathrm{d}\Phi}{\cos{\theta} \mathrm{d}\omega \mathrm{d}A}$.

Thus in order to get the total emitted power we need to integrate over the area of the light and we need to integrate over the projected solid angle of the hemisphere (in the direction of the surface normal). The radiance for the area lights in pbrt is constant. This gives us:

$\Phi = \int_A \int_{H^2(\mathbf{n})} L_o\mathrm{d}\omega^\perp \mathrm{d}A =
\int_A \int_{H^2(\mathbf{n})} L_o \cos\theta \mathrm{d}\omega \mathrm{d}A =
\int_A L_o \int_0^{2\pi} \int_0^{\pi/2} \sin\theta \cos\theta \mathrm{d}\theta \mathrm{d}\phi \mathrm{d}A =
\int_A L_o \pi \mathrm{d}A =
A \cdot L_o \cdot \pi$


Is it possible to set visible and color for all 12 edges(spines) of the figure (cube) in matplotlib?

matplotlib's 3d rendering settings are not yet complete (it's meant to be a 2d renderer program). I don't think at this stage it has this option without big hacks to the code locally (i.e. done by yourself). Mathematica offers this setting naturally.


If you just want to explicitly sample an area light, then here's the general procedure you should follow. Pick light $i$ out of $L$ lights with some probability $p_i$ (the other probabilities being $p_1,...,p_L$, a light may be picked through inverse transform sampling). Pick a point $\pmb{y}$ on the surface of the light with some probability $q_i(\pmb{y})$. Then your estimator is:

$$I_N = \frac{1}{N}\sum_{k=1}^{N}\frac{1}{p_{i_k}q_{i_k}(\pmb{y}_k)}f(\pmb{z} \leftarrow \pmb{x} \leftarrow \pmb{y}_k)L_e(\pmb{y}_k\rightarrow\pmb{x})\frac{\cos\theta_x\cos\theta_{y_k}}{\|\pmb{x}-\pmb{y}_k\|^2_2}V(\pmb{x},\pmb{y}_k)$$

Note that this is only for direct illumination at point $\pmb{x}$. I have generalized it to a secondary estimator using $N$ samples, so essentially you pick $N$ points $\pmb{y}_1,...,\pmb{y}_N$ on possibly different lights. This is an estimator that can directly be derived from the area formulation of the rendering equation:

$$L(\pmb{z} \leftarrow \pmb{x}) = L_e(\pmb{x} \leftarrow \pmb{z}) + \int_{\Omega}f(\pmb{z} \leftarrow \pmb{x} \leftarrow \pmb{y})L(\pmb{y}\rightarrow\pmb{x})\frac{\cos\theta_x\cos\theta_y}{\|\pmb{x}-\pmb{y}_k\|^2_2}V(\pmb{x},\pmb{y})\,dA(\pmb{y})$$

The area formulation can be derived from the solid angle one, by using the identity:

$$d\omega = \frac{\cos\theta_y}{r^2}dA$$,

as well as:

$$L_i(\pmb{x}, \omega) = L(r(\pmb{x},\omega) \rightarrow \pmb{x}) = L(\pmb{y} \rightarrow \pmb{x})V(\pmb{x},\pmb{y})$$


Why isn't the clear color visible on the screen between frames?

You're confusing the screen (the physical display) with the framebuffer (the memory which gets read out by the display controller). The framebuffer might get cleared at the start of each frame period, but that doesn't change the display. The order of events is as follows:

At the start of a new frame, glClear or equivalent sets the framebuffer to the clear colour.
glDraw or equivalent calls draw more stuff on top of the clear colour.
eglSwapBuffers or equivalent queues the finished frame to be drawn on the display.
When the display is ready, this process is followed. It reads out the contents of the framebuffer and shows them on the physical screen.

The framebuffer that's being drawn into is never shown on the screen; only when it's finished can it be shown. Double- or triple-buffering gives you multiple copies of the framebuffer, so the application can be using one while the display is using the other. It's nothing to do with your eyes or brain.

It's actually a lot more complicated than this, because most GL implementations today are not immediate-mode, so none of the real work happens when you call the functions named above. In fact, clearing the buffer might not even happen at all: if every pixel is drawn to anyway, there's no need to. Back when clearing the framebuffer took significant time, games would go faster by not calling glClear. Today it's more important to call glClear because it lets the implementation know when the start of the new frame is, but it will probably do something smarter than just writing to every pixel.


One way would be to break out the mip levels of the cube into a bunch of separate textures, then implement the mip selection and trilinear filtering yourself, by branching based on the LOD value.

That doesn't sound fast, though, given you're targeting older devices. You could speed it up by forgoing trilinear filtering and just using the nearest-neighbor mip level. Gloss values would be effectively quantized to the number of mip levels, but maybe it won't look too bad.


Rendering part of the scene inside bounding sphere

How can we render the scene around the light by rendering it onto a sphere?

Well, answer me this: how do you render a scene around a like by rendering a "screen quad"?
We're ultimately talking about deferred rendering, and what is being done is rendering the lighting pass. During the lighting pass, you read data from the gbuffers that tell you that pixel's position, normal, and any other information used by the lighting equation. The texel location you read this data from those gbuffers is just the fragment's location.
The point being made by the video is that the geometry you render to do a lighting pass does not matter. You are only using it to invoke the fragment shader over a region of the image. This is why rendering a full-screen quad works; you're executing the fragment shader over all of the pixels in the screen.
While that works, it does mean you're executing more fragment shader invocations than are strictly necessary for any particular light. Points that correspond to distances outside of the light's useful effect still have to have an FS executed on them.
A sphere is a useful way to pre-cull such fragments. The sphere's radius represents the useful range of effect for a light. Any fragments horizontally or vertically outside of this range don't get produced, since fragments are only produced by rasterizing the sphere. And because the sphere's vertices will be transformed by the same transformation matrix as the scene, and because you will turn on depth tests, you also can cull out cases that are outside of the light's useful range in depth (or at least, if they're closer to the camera than the light's effect).
But there are downsides of this approach. Spheres are complex objects, so there will typically be some overlap in terms of the number of helper fragment shader invocations being executed for a particular pixel. Of greater concern is using relatively high resolution spheres that appear very small on the screen.
Also, you can only execute a single light per FS invocation, instead of being able to bundle lots of lights together. This is especially important for deferred rendering on tile-based renderers.


Efficiently generating mesh for self-generated voxel grid

So I've been all around the houses on this looking for an answer. In the end I found a combination of techniques worked quite well:

To take advantage of all of the 'empty space', the concept of 'surface tracking' from the Octree Decimation paper useful - you can follow the surface from cell to cell and ignore all of the cells in the middle.

The main advantage of generating the voxels ourselves is what we know two vital pieces of information: firstly, the exact position of each vertex within the cells on the 'front' surface; secondly, the 'orientation' of the mesh in each region and which way that 'front' is to be found.

Putting all the pieces together I came up with this Dual Marching Cuboids algorithm, that has pretty good results in very reasonable time.


Is there a list of OpenGL extension support?

There is no official list, but a community driven one. It is largely up to date:
https://opengl.gpuinfo.org/listextensions.php

As can be seen here AMD doesn't support the ARB_shading_language_include extension at the time I wrote this answer.


Given a composited image, is it possible to find the color and opacity of an overlaid shape?

You can solve the colors by solving a system of linear equations if you make few assumptions:

The alpha of the color wheel is constant (i.e. same for all colors in the wheel)
The beige background extends half way through the color wheel, and is constant for top and bottom halves of the image
The background in the center is constant for all colors
The alpha blending equation is the standard $f=c*\alpha+b*(1-\alpha)$, where $f$ = final color, $c$ = color on the wheel, $b$ = background color, $\alpha$ = alpha of the color wheel

If you now pick two colors with the same beige background (e.g. green and yellow), and feed them into the above alpha blending equation you get 4 equations with 4 unknowns, which you can then solve. I.e.

$$f_1=c_1*\alpha+b_1*(1-\alpha)$$
$$f_2=c_1*\alpha+b_2*(1-\alpha)$$
$$f_3=c_2*\alpha+b_1*(1-\alpha)$$
$$f_4=c_2*\alpha+b_2*(1-\alpha)$$

where $f_1$ & $f_2$ = brighter & darker green colors respectively, $f_3$ & $f_4$ = brighter & darker yellow colors respectively, $c_1$ & $c_2$ = the unknown green & yellow colors respectively you try to solve, $\alpha$ = alpha you try to solve, $b_1$ = beige background color, $b_2$ = unknown background color in the center.

Then you can repeat this process for all the colors, or because you solved $\alpha$ you can use this information to solve the rest of the colors simply by:
$$c=\frac{f-(b_1*(1-\alpha))}{\alpha}$$


Subtle bugs in Fortune's algorithm implementation

This commit contains the fix of the bug.

In short, it is wrong to disable event for second edge, during event checking, when first edge may be truncated sooner then stated by its current associated event.


using default depth buffer in off-screen framebuffer

Is it possible to bind default framebuffer's depth buffer to another framebuffer?

First, you "attach" images to framebuffers. You "bind" objects to the context; you "attach" objects to other objects.

Second, no, you cannot attach images of the default framebuffer to anything.

  If not, how can I copy default depth to another depth buff, or what is best way to do this?

You render to your own depth buffer entirely. Just copy/render the image data out to the default framebuffer when its time to display it.


Is it possible to alpha blend multiple render targets using a specified alpha?

As far as I'm aware there's no way in either DX or GL to re-use RT 0's alpha for all the blending operations.  Unfortunately, it doesn't seem to be something that's supported by hardware.

You can configure different blend modes for each render target, or enable blending for some and disable for others; however, if blending is enabled for a render target, it always uses its own alpha.

There's also a mode called "dual-source blending" (see DX11 doc and OGL doc), which allows you to specify the alpha for blending with an entirely separate output from the pixel shader, not the render target's alpha channel.  However, this mode only works with one render target on current hardware.

So as far as I can tell, the only options for blending several render targets with the same alpha are:

Output the same alpha on all render targets (sacrificing the ability to store other values to the alpha channel as you would in deferred shading, for instance).
Repeat the rendering in a separate pass for each render target, using dual-source blending.
Use UAVs / image load-store to execute the blending in the pixel shader (only works if geometry is non-self-overlapping in screen space, because there's no protection against race conditions; also probably kinda slow).
On hardware that supports it, DX11.3/DX12 Rasterizer-Order Views, NV_fragment_shader_interlock, or INTEL_fragment_shader_ordering (the latter also exposed by AMD GPUs).  These are three names for the same thing: basically a "critical section" in the pixel shader that lets you read-modify-write a texture atomically with respect to other pixel shader invocations. It essentially enables arbitrary programmable blending, but it's probably fairly slow, and only available on recent hardware.


Can raytracing be used to imitate the behavior of light to a higher degree?

You are late by about 35 years. This was addressed in the historical paper "Whitted, T., An improved illumination model for shaded display. Communications of the ACM, Volume 23 Issue 6, June 1980, Pages 343-349", using  ray tracing.

Since then radiosity has been introduced to better deal with diffuse phenomena. Correct physical modeling of the image formation process was discussed six years later in the seminal "Kajiya, James T. (1986), "The rendering equation", Siggraph 1986: 143".


Ask for help on understand an algorithm which combines stratified sampling and importance sampling of Monte Carlo

It's difficult to parse, but here's my best understanding.

In this case the goal is determine number of $N_i$ out of $n$ samples to distribute to each strata with probability given by the discrete pdf in $\{p_0, ..., p_n\}$

The summation of $P$ += $p_i$ is computing the CDF for $i$.

I believe "sample the ith term $N_i$ times" means to take a jittered sample within that strata using new random variables.

Let's take a small example where we have the pdf with 4 elements.
$$
PDF = \{0.2, 0.1, 0.6, 0.1\}
$$

And it's corresponding cdf, each element corresponds to $P$ in the ith iteration in the pseudo code.
$$
CDF = \{0.2, 0.3, 0.9, 1.0\}
$$

Now given $\left \lfloor{P*n + u}\right \rfloor$ we can illustrate for a few example cases.

With $u = 0$
$$
\left \lfloor{P*n + u}\right \rfloor = \{0, 1, 3, 4\}
$$
$$
N_i = {0, 1, 2, 1}
$$

With $u = 0.5$
$$
\left \lfloor{P*n + u}\right \rfloor = \{1, 1, 4, 4\}
$$
$$
N_i = {1, 0, 3, 0}
$$

With $u = 0.99$
$$
\left \lfloor{P*n + u}\right \rfloor = \{1, 2, 4, 4\}
$$
$$
N_i = {1, 1, 2, 0}
$$

Notice how for example $N_3$ corresponds to a higher number of samples. When we now sample the 3rd element $N_3$ times we are weighting that strata in proportion to its pdf.

Also, I don't see any reason that the sample count has to be the same $n$ as the # of discrete elements in the pdf.


Changing sin frequency smoothly in shader

You could go the calculus way and have an explicit rate of change that is outside your shader

float frequency = getFrequency();
sineTime += frequency * deltaTime;

(sineTime could be a float that is scoped right outside of your render loop, so that it retains its value)

And if you pass sineTime into your shader you can just do

float s = sin(sineTime);


Identify different contour cycle after model slicing

Take a random vertex on your loop, shoot a ray trough all the other loops. For loops you crossed a odd number of times your inside for loops you crossed a even number of times your outside.

The loops that are outside of all others is in the root of your tree. Then find loops that are inside that, but not inside some other. Then find next level and next level and so on.

Image 1: Animation of arranging in tree by shooting rays

You can optimize this by using bounding boxes/ bsp trees for the test areas. But for a small number of areas that might be overkill. just sorting them left to right top to bottom should suffice. You can use the same algorithms as your scan line operators.

Image 2: And now you can offset the shape. But you need to recheck that there is no intersection left after this.

Now the topology is somewhat stable across levels of slices so you can use that to your benefit. Just follow the 3D surface.

Or you could do the offset in 3D on the polygons so you do not need this info. Also if you just need the winding for knowing which way to offset rule you can derive it from the 3d model itself, each vertex has a normal so project the normal for one triangle in each loop with your slicing algorithm per loop. That then gives you the direction of loops outside which is inside for nearly free. You wouldn't get the tree but then again you might not need it.


Sun light can be represented as a distant light. Light sources are radial like a bulb, though in the case of the sun, it is so far away from the Earth, that a reasonable approximation is to consider that all rays coming out of the sun are parallel at the Earth's surface. Using a distant light is an approximation. Because it's far away it is also assumed that there is no variation of light intensity as you go up or down (change height).
If you do physically based rendering, the sun is an area light. From the Earth it looks like a small disk so you can approximate this with a large disk that is far far away. Area light like all light sources obeys a light intensity square fall off. So you will need to put your area light far away so that the intensity on the ground or as you change height doesn't vary much. Representing the sun as an area light avoids having to do strange things to the surface shader such as specifying a specular size.


VAO can share VBO's because they do not store vertex data itself but references to Vertex Buffer Objects.

So you can first generate your buffers and upload your buffer data (vertex data and index data) to currently bound buffers.

glGenBuffers(1, &VBO_xzCoord_IDs[i]);
glGenBuffers(1, &VBO_yCoord_IDs[i]);
glGenBuffers(1, &VBO_indices_IDs[i]);
glGenBuffers(1, &VBO_normals_IDs[i]);
glGenBuffers(1, &VBO_colors_IDs[i]);

//xz coords
glBindBuffer(GL_ARRAY_BUFFER, VBO_xzCoord_IDs[i]);
glBufferData(GL_ARRAY_BUFFER, sizeof(GLfloat)*xzCoords.size(), &xzCoords[0], GL_STATIC_DRAW);

...

//indices
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, VBO_indices_IDs[i]);
glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(GLuint)*indices.size(), &indices[0], GL_STATIC_DRAW);

And then generate your VAO's and connect vertex array object data index you provide in glVertexAttribPointer call with currently bound GL_ARRAY_BUFFER.

The GL_ELEMENT_ARRAY_BUFFER can be a little confusing because it doesn't require glVertexAttribPointer call. So how it is connected with VAO? It seems that binding the buffer ties it to the currently bound VAO.

//VAO
glGenVertexArrays(1, &VAO_IDs[i]);
glBindVertexArray(VAO_IDs[i]);

//xz coords
glBindBuffer(GL_ARRAY_BUFFER, VBO_xzCoord_IDs[i]);
glEnableVertexAttribArray(0);
glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, 0, (void*)0);

...

//indices
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, VBO_indices_IDs[i]);

glBindVertexArray(0);

And in your rendering call you don't need to bind GL_ELEMENT_ARRAY_BUFFER because it was already connected with VAO.


reconstruction drawing sequence from video

This is more a Computer vision question I think.
I don't know how exactly you can achieve that but some ideas and keywords you may use to find additional resources:

Feature tracking of hand and pen can yield the movement of said things. Assuming you know that the camera is not moving I think that would give you good results. Related is maybe motion tracking. Any information you have about the hand and the camera can in some way be used to make more accurate estimates.

Removing the hand from the video could be much harder. I don't remember that part from my lecture. We copied parts of the image onto the part we want to remove and used image metrics to make sure it fits best possible... But since the hand covers a drawing we don't know much about this could be very hard to do in an convincing way.


THREEjs: updating an object's matrix doesnt change it's position and rotation params

I think you need to call setFromMatrixPosition(m) again after updating the matrix. After all, you need to update both the position and matrix, not just the matrix.
mesh.position.setFromMatrixPosition(mesh.matrix) or mesh.position = new THREE.Vector3().setFromMatrixPosition(mesh.matrix) right after mesh.updateMatrix()


What is exactly the third component in homogeneous coordinate system?

The relationship between standard coordinates $(x,y)$ and homogeneous coordinates $(X,Y,Z)$ is $x = X / Z, y = Y/Z$. Homogeneous coordinates are a type of projective coordinates. All points on the line $(Z x, Z y, Z)$ are equivalent to the point $(x,y)$ in 2D (the same thing can be applied for 3D). So technically 2D points are represented as 3D lines passing though the origin, and 2D lines are represented as 3D planes passing through the origin.
In computer graphics the above interpretation is largely irrelevant, however, since homogeneous coordinates are used mostly for convenience. A translation is of the form $x' = x+t_x, y' = y+t_y$. If one uses homogeneous coordinates, then this can be achieved through $X' = 1 \cdot X + 0 \cdot Y + t_x \cdot Z$, $Y' = 0 \cdot X + 1 \cdot Y + t_y \cdot Z$, and $Z' = 0 \cdot X + 0\cdot Y + 1 \cdot Z$.  If you set $Z=1$ then you get exactly the expression for translation, and you can use the matrix induced from the 3 equations above. For directions you can set $Z=0$ so that translation would not be applied to those.
In 3D the homogeneous coordinates $W$ also plays a role in the perspective projection, since the projection matrix modifies it to hold the relevant value by which the other coordinates must be divided, and the division is typically carried out by the hardware.


Consider the following scheme that mimicks the generation of a sample for phong shading. where V is the viewer direction, N is the normal, R is the specular reflection direction and $S_i$ is some sample. This space (commonly called tangent space) uses the normal as one of its basis vectors. you can think of it as the 'Z axis'

Now, when we generate samples for Phong shading wo do not generate them in this space.

Rather, we use a different reference system where the Specular reflection direction is the 'z axis', thus, our angle $\theta$ is the angle between the reflected direction and the sample direction, and the angle $\phi$ is the angle between some $S_{ix}$ that is the projection of our sample direction onto the plane perpendicular to $R$ and some arbitrary but constant vector perpendicular to our reflected direction

$\phi$ is a number that is sampled uniformly in the range $(0,2\pi)$ this means that rays will be generated around the reflected direction with no bias towards any particular direction.

Generated rays will, however, be biased to have a direction close to $R$ due to the way that $\theta$ is sampled.

When we generate rays in a backwards pathtracer we use the view direction to get the directions where the light is likely to come from, instead of throwing light around and seeing where it falls. The later technique is closer to photon mapping.