<blockquote>
  <p>is it correct having such a big light?</p>
</blockquote>

<p>I don't see any problem with having a big area light. That said, it also depends on the scale of your scene. If the light is large compared to it, shadows will tend to be more diffuse, like under an overcast sky.</p>

<blockquote>
  <p>shadows are missing.. [...] Can you help me understanding why?</p>
</blockquote>

<p>I haven't found any blatant mistake, but there are two elements that look suspicious to me.</p>

<pre><code>if (depth &gt; 1 &amp;&amp; checkRayLightIntersection(O, D, hitLightPoint)) {
    // ...
    return true; //light hit, path completed
}
</code></pre>

<p>Although this part seems to be correct as far as I can tell, I would try to make sure there is no bug introducing a bias. For every bounce, it checks if there is a ray hitting the light first, and if so it breaks out of the loop: in case the code is incorrect and it happens to hit the light more often than it should, the shadows will disappear.</p>

<p>Moreover, in the first if:</p>

<pre><code>return false; //nothing hit, path non valid
</code></pre>

<p>and the related loop:</p>

<pre><code>for (int y = 0; y &lt; SCRHEIGHT; y++) for (int x = 0; x &lt; SCRWIDTH; x++)
{     
  Path path;
   if(generatePath(x,y, path)){
      Sample(path);
   }
}
</code></pre>

<p>Make sure you still take the paths that hit nothing into account when averaging the color of the pixel, otherwise this will also introduce a bias.</p>

<p>As joojaa said in the comment, you should try using a simpler scene: a few spheres or cubes, no texture. You can also reduce the number of bounces to one or two to see only direct lighting. If necessary, maybe even use a constant sky dome first instead of a light area.</p>

<hr>

<p>Other remarks:</p>

<pre><code>struct Path {
   PathVert verts[MAX_DEPTH]; //maxDepth is 15 for now
   int vertCount;
   int x, y; //which pixel this path is referring to
};
</code></pre>

<p>Do you need to store x and y?</p>

<pre><code>float distFromObj = length(vert.p);
float distFromLight = length(hitLightPoint);
</code></pre>

<p>Don't you already have two <code>t</code> that you can compare?</p>

CC BY-SA 3.0
2016-04-07T19:52:56.443Z
First of all thanks for your answer :)

So, the method to check the intersection between the ray and the square light is right. I have already been asking about it cause of a small error I couldn't find. The method and the relative solution can both be found here:

http://stackoverflow.com/questions/36180741/intersection-of-ray-and-rectangle-in-c/36186088#36186088

I also edited the question adding a picture that shows the result in case I omit the **depth > 1** check. Maybe it can help understanding where the problem is.. thanks once more
Tarta
CC BY-SA 3.0
2016-04-07T16:55:14.700Z
2016-04-11T01:58:24.743Z
2016-04-11T01:58:24.743Z
Julien Guertault
<h2>The problem appears to be unintentionally transparent surfaces</h2>

<p>Although the image is grainy, it is sufficiently clear to estimate that all of the darker regions are due to surfaces facing away from the light, rather than due to shadows cast onto surfaces facing towards the light. So it does seem that there is a problem, and the lack of shadows is not just due to having soft shadows from a large area light that don't show up well, or excessively reflective surfaces that provide too much ambient light. Somehow light is getting through your surfaces.</p>

<h2>The probable source of this problem</h2>

<p>Since you check for intersection with the light and the objects separately, the crucial point where this could go wrong is where you check</p>

<pre><code>if (distFromLight &lt; distFromObj)
</code></pre>

<p>You correctly terminate the path and return the light colour if this check is true (that is, if the light is closer than the nearest object in that direction). So the problem is likely to be with one of the two variables you are comparing. If they were both correct, then you would see shadows.</p>

<p>These are calculated on the preceding lines</p>

<pre><code>float distFromObj = length(vert.p);
float distFromLight = length(hitLightPoint);
</code></pre>

<p>As pointed out in <a href="https://computergraphics.stackexchange.com/a/2281/231">Julien Guertault's answer</a>, these calculations may be redundant.</p>

<p><code>length(vert.p)</code> should still be available as <code>t</code>, used 11 lines previously in the calculation of <code>vert.p</code>. This not only avoids unnecessary calculation, but eliminates a potential source of error. Then this length is likely to be correct unless there is a problem with <code>checkIfRayIntersectSomething()</code>.</p>

<p>This leaves only <code>length(hitLightPoint)</code> as a source of error, unless there is a more subtle problem. It would be useful to be able to eliminate the possibility of problems with <code>length()</code> or <code>hitLightPoint</code>, which is calculated by <code>checkRayLightIntersection()</code>.</p>

<h2>We need to see the code to be able to help</h2>

<p><code>length()</code>, <code>checkIfRayIntersectSomething()</code> and <code>checkRayLightIntersection()</code> are not currently included with the code in your question, so we can't analyse these. If you are able to share these that would help narrow down the source of the problem.</p>

<h2>A simple way to check whether this is the real problem</h2>

<p>You could also verify whether this analysis of the problem is correct by rendering a scene which will eliminate the other possibilities (shadows too soft to distinguish, or shadows washed out by too much reflected light from the rest of the scene). If the light getting through the surfaces is the source of your problem, then it will apply even if there is no path from the light to a surface, provided that surface is facing the light. So render a simple scene that has a light with a plane beneath it, and another plane beneath that, separated by a gap big enough to easily be seen. The lower plane is completely occluded by the upper plane, so should receive no light (no soft shadows and no reflected light from elsewhere). If the lower plane is lit, then you know that this is indeed the problem and you can focus your efforts here.</p>

<p>If the lower plane is <em>not</em> lit, then either this is not the problem, or there is a problem with the scale and the occluding plane may need to be at a different distance from the light in order for the problem to show up. You can use your existing scene as a source of estimated distances for which this problem occurs, and you can use inclined planes in order to see the effect for a variety of ratios of object distance to light distance within a single image.</p>

<p>Sharing the images resulting from such simple scenes may help us identify the problem more precisely.</p>

<hr>

<p>I notice that in a comment on the question you mention that you do not have a straightforward way of generating scenes yourself. Perhaps there are much simpler scenes already publicly available in the same file format, that you can use to test this?</p>

<p>If not, it sounds like the only thing you have control over is the light. In that case, you could try putting the light underground. If surfaces are not transparent, the scene should be uniformly black. If surfaces are being lit despite an occluding surface, you may see evidence for it this way that would be hard to discern when there is lots of secondary light bouncing around.</p>

CC BY-SA 3.0
2016-04-10T17:15:00.353Z
your answer was very useful, thanks first of all ! You where right, **distFromLight** and  **distFromObj** where wrongly calculated. I fixed it. Something has been improved but the problem is not fixed yet. I posted one picture showing the new results. I thought maybe the error relies on too soft shadows due to a too big light, therefore I tried to lower the dimension of the light. No matter how increase the power of the light, but in this case results are bad (posted a second picture about it). Hope it helps you. In the meantime I am working on the .obj loader for a simpler scene.
Tarta
CC BY-SA 3.0
2016-04-10T22:45:39.950Z
A smaller light means more of the rays will never reach the light, so you will need to increase the number of rays to give a better quality image. This will also increase the time it takes to render. It might be worth instead making the light as large as possible (an infinite or very large plane). This way, if the problem is transparent surfaces, you should see even more evidence for it, with the added benefit that the largest possible light will reduce the graininess of the image.
trichoplax is on Codidact now
CC BY-SA 3.0
2016-04-10T22:49:17.140Z
I've added an additional final paragraph about putting the light underground as a test (see edited answer) that may help if you are unable to change the scene, but able to move the light.
trichoplax is on Codidact now
CC BY-SA 3.0
2016-04-11T13:06:12.773Z
Ok, I followed your suggestion. I tried to put the light beneath the scene and nothing is lit up. No light passes through surfaces and therefore surfaces are not transparent. At least we can exclude this possibility.. so it can be that shadows are too soft or washed out by too much light reflected from the rest of the scene. How do you suggest me to proceed?
Tarta
CC BY-SA 3.0
2016-04-11T14:03:53.410Z
Have you tried rendering just a small section of the image yet? For example, the bottom right sixteenth of the image contains are strong shadow in the target image. Rendering just that small image would allow you to use a much higher number of samples per pixel, to give a clearer image and more idea of what is going on.
trichoplax is on Codidact now
CC BY-SA 3.0
2016-04-11T14:38:21.090Z
just did it, shadows are not there.. you can find the piece of image above at EDIT 4
Tarta
CC BY-SA 3.0
2016-04-11T15:39:07.487Z
Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/38237/discussion-between-tarta-and-trichoplax).
Tarta
CC BY-SA 3.0
2016-04-17T10:22:27.487Z
EDIT 6 contains the final explanation
Tarta
CC BY-SA 3.0
2016-04-08T14:06:10.073Z
2016-04-10T22:48:51.737Z
2017-04-13T13:00:43.660Z
trichoplax is on Codidact now
<p>I am trying to implement for research purposes a gradient-domain path tracer. In order to achieve this step I first need a working Path tracer. I have been creating one so far but results are wrong and I will explain you why. Some concepts:</p>

<p>I am working on paths that are generated before sampling them. I mean that the first step of my algorithm consists in calculating a path for a certain pixel (x,y). This path will perform some bounces within the scene and, if terminates on the light, will be considered valid.</p>

<p>First of all: the following is the definition of a struct (will be declared and initialized later), containing some scene info needed to cast ray from the camera into the scene.</p>

<pre><code>struct RenderData
{
    vec3 E;
    vec3 p1;
    vec3 dx;
    vec3 dy;
};
</code></pre>

<p>It is initialized in the InitializeScene method:</p>

<pre><code>void InitializeScene(){ // setup virtual screen plane
  vec3 E( 2, 8, -26 ); //Eye position
  vec3 V( 0, 0, 1 ); //LookAt vector
  float d = 0.5f, ratio = SCRWIDTH / SCRHEIGHT, focal = 18.0f;
  vec3 p1(E + V * focal + vec3(-d * ratio * focal, d * focal, 0)); // top-left screen corner in SCREEN SPACE
  vec3 p2(E + V * focal + vec3(d * ratio * focal, d * focal, 0)); // top-right screen corner
  vec3 p3(E + V * focal + vec3(-d * ratio * focal, -d * focal, 0)); // bottom-left screen corner
  mat4 M = rotate( mat4( 1 ), r, vec3( 0, 1, 0 ) );
  p1 = vec3(M * vec4(p1, 1.0f)); //rotating the above points
  p2 = vec3(M * vec4(p2, 1.0f));
  p3 = vec3(M * vec4(p3, 1.0f));
  renderData.dx = (p2 - p1) / (float)SCRWIDTH;
  renderData.dy = (p3 - p1) / (float)SCRHEIGHT;
  renderData.E = vec3(M * vec4(E, 1.0f));
  renderData.p1 = p1;
}
</code></pre>

<p>The code above is to make you understand how I initialized the scene.
I also have structs to store informations about my paths:</p>

<pre><code>struct PathVert {
   vec3 p; vec3 n; //hit point and normal of the surface hit 
};

struct Path {
   PathVert verts[MAX_DEPTH]; //maxDepth is 15 for now
   int vertCount;
   int x, y; //which pixel this path is referring to
};
</code></pre>

<p>therefore I start consider pixel after pixel.</p>

<pre><code>for (int y = 0; y &lt; SCRHEIGHT; y++) for (int x = 0; x &lt; SCRWIDTH; x++)
{     
   Path path;
   if(generatePath(x,y, path)){
      Sample(path);
   }
}
</code></pre>

<p>The <strong>generatePath()</strong> method indeed tracks the path into the scene and checks all the vertices it hits. The <strong>checkIfRayIntersectSomething(t)</strong> method you will see used, it's just a pseudo method implemented in my framework and that I omit posting cause of its length. I use it to check if my ray hits something in the scene, if it does, it update the "t" with the distance to that object. NOTE: the light is not considered an object itself. Hence, I also have a <strong>checkRayLightIntersection(hitLightPoint)</strong> which checks the intersection with the light, if there is any, the hitLightPoint is updated with the point on the light I have been hitting.
The light is a 2D surface.</p>

<pre><code>Vec lightPos = Vec(5, 15, 2); //hard coded position of the light
</code></pre>

<p>And the light, as said, is a surface, but exactly a square surface, whose 4 angles are:</p>

<pre><code>Vec P1 = Vec(lightPos.x - 20, lightPos.y, lightPos.z + 20);
Vec P2 = Vec(lightPos.x + 20, lightPos.y, lightPos.z + 20);
Vec P3 = Vec(lightPos.x + 20, lightPos.y, lightPos.z - 20);
Vec P4 = Vec(lightPos.x - 20, lightPos.y, lightPos.z - 20);
</code></pre>

<p>Quite big, I know, so the first question relies on this aspect, is it correct having such a big light?</p>

<p>But let's go to the main methods. Hereby you can see the GeneratePath method:</p>

<pre><code>bool GeneratePath(int x, int y, Path &amp;path){
   path.x = x;
   path.y = y;
   path.vertCount = 0;

   vec3 P = renderData.p1 + renderData.dx * ((float)(x) + Rand(1)) +  renderData.dy * ((float)(y) + Rand(1));
   vec3 O = renderData.E + vec3(Rand(0.4f) - 0.2f, Rand(0.4f) - 0.2f, Rand(0.4f) - 0.2f);
   vec3 D = normalize(P - O); //direction of the first ray, the one from the camera towards the pixel we are considering

   for (int depth = 1; depth &lt;= MAXDEPTH; depth++){
    float t;
    Vec hitLightPoint;
    PathVert vert;
    if (!checkIfRayIntersectSomething(t)){
        //we didn't find any object.. but we still may have found the light which is an object non represented in the scene
        //the depth check avoids me rendering the light as a white plane
        if (depth &gt; 1 &amp;&amp; checkRayLightIntersection(O, D, hitLightPoint)){
            //update the vertex since we realized it's the light
            vert.p = hitLightPoint;
            vert.n = Vec(0, -1, 0);//cause the light is pointing down
            path.verts[depth - 1] = vert;
            path.vertCount++;
            return true; //light hit, path completed    
        }
        return false; //nothing hit, path non valid
    }
    //otherwise I got a hit into the scene
    vert.p = O + D * t; //reach the hitPoint
    vert.n = methodToFindTheNormal();
    vert.color = CalculateColor(vert.p); //according to the material properties (only diffuse objects so far)
    path.verts[depth - 1] = vert;
    path.vertCount++;

    //since I have the light, and a path terminates when it hits the light, I have to check out also if my ray hits this light,
    //and if does, I have to check whether it first hits the light or the object just calculated above
    //moreover with the "depth &gt; 1" check, I avoid again rendering the light which otherwise would be visible as a white plane

    if (depth &gt; 1 &amp;&amp; checkRayLightIntersection(O, D, hitLightPoint)){
        float distFromObj = length(vert.p);
        float distFromLight = length(hitLightPoint);
        if (distFromLight &lt; distFromObj){
            //update the vertex since we realized it's the light
            vert.p = hitLightPoint;
            vert.n = Vec(0, -1, 0);
            vert.color = Vec(1, 1, 1);// TODO light color? or light emission?

            path.verts[depth - 1] = vert;
            return true; //light hit, path completed
        }
    }
    if (depth == MAXDEPTH) return false;
       Vec newDir = BSDFDiffuseReflectionCosineWeighted(vert.n, D);//explained later
       D = newDir;
       O = vert.p;
   }
   return false;
}
</code></pre>

<p>The BSDFDiffuseReflectionCosineWeighted() just calculate the new directions, tested and working. What remains last is the Sample method which calculates the final color of the pixel.</p>

<pre><code>Vec Sampling(Path &amp;path){

   Vec color(1, 1, 1);

   for (int vert = 0; vert &lt; path.vertCount - 1; vert++) { //considers the last vertex as the light
      const PathVert &amp;currVert = path.verts[vert];
      const PathVert &amp;nextVert = path.verts[vert + 1];
      Vec wo = (nextVert.p - currVert.p).norm();
      double cosTheta = fabs(wo.dot(currVert.n));
      float PDF = cosTheta/PI;
      if (cosTheta &lt;= 1e-6) return Vec();
      //considering only DIFFUSE objects
      color = color.mult(currVert.color * (cosTheta / M_PI) / PDF);
   }
   return color.mult(Vec(10.0f, 10.0f, 10.0f)); //multiplication for the light emission?
}
</code></pre>

<p>Result with 16SPP is:</p>

<p><a href="https://i.sstatic.net/HNOah.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/HNOah.jpg" alt="enter image description here"></a></p>

<p>As you can see the results are not that bad but there is one main problem: shadows are missing.. tried many combinations but no improvements. There is some error in the algorithm itself. Can you help me understanding why? Thanks in advance.</p>

<p>EDIT: this is my target reference:</p>

<p><a href="https://i.sstatic.net/QHG2z.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/QHG2z.jpg" alt="enter image description here"></a></p>

<p>EDIT 2: this is what happens in case the <strong>depth > 1</strong> check is omitted:</p>

<p><a href="https://i.sstatic.net/hiqEY.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/hiqEY.jpg" alt="enter image description here"></a></p>

<p>EDIT 3: after fixing the <strong>distFromLight</strong> and <strong>distFromObj</strong> I got the following results:</p>

<p><a href="https://i.sstatic.net/JvGhf.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/JvGhf.jpg" alt="some clue of shadows"></a></p>

<p>One of the 2 problems as suggested by @trichoplax could be too soft shadows. So I tried to lower the size of the light, increasing his power to compensate. Results are not good and shown here:</p>

<p><a href="https://i.sstatic.net/8aYPH.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/8aYPH.png" alt="enter image description here"></a></p>

<p>Moreover as requested I can post some more code:</p>

<pre><code>bool checkRayLightIntersection(Vec O, Vec D, Vec &amp; hitLightPoint){
  //getting the 4 corners of my light
  Vec P1 = Vec(lightPos.x - 20, lightPos.y, lightPos.z + 10);
  Vec P2 = Vec(lightPos.x + 20, lightPos.y, lightPos.z + 10);
  Vec P3 = Vec(lightPos.x + 20, lightPos.y, lightPos.z - 10);
  Vec P4 = Vec(lightPos.x - 20, lightPos.y, lightPos.z - 10);

  //the majority of the methods first find out where the ray intersects the plane that the rectangle lies on Ax + By + Cz + D = 0
  //in our case the equation of that plane is easy -&gt; D = 20
  // answers.google.com/answers/threadview?id=18979

  float t = -(-O.y + lightPos.y) / (-D.y);
  if (t &gt; 0){
    Vec hitPoint = O + D * t;
    Vec V1 = (P2 - P1).norm();
    Vec V2 = (P3 - P2).norm();
    Vec V3 = (P4 - P3).norm();
    Vec V4 = (P1 - P4).norm();
    Vec V5 = (hitPoint - P1).norm();
    Vec V6 = (hitPoint - P2).norm();
    Vec V7 = (hitPoint - P3).norm();
    Vec V8 = (hitPoint - P4).norm();

    if (V1.dot(V5) &gt; 0.0 &amp;&amp; V2.dot(V6) &gt; 0.0 &amp;&amp; V3.dot(V7) &gt; 0.0 &amp;&amp; V4.dot(V8) &gt; 0.0){
        hitLightPoint = hitPoint;
        return true;
    }
  }
  return false;
}
</code></pre>

<p>Instead, the <strong>checkIfRayIntersectSomething</strong> method is part of the framework itself (it has just another name), it's not done by me but it's tested and working.</p>

<p>EDIT 4: tried to render a small part of the scene with high number of spp. Tried 512 spp for the bottom right corner where should appear shadows according to the target image. Results show that shadows are not there:</p>

<p><a href="https://i.sstatic.net/ohZJu.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ohZJu.png" alt="Bottom right corner at 512spp"></a></p>

<p>EDIT 5:</p>

<p>I hereby post the code used to render the target image, it does at the same time my <strong>Sampling()</strong> and <strong>GeneratePath()</strong> methods using a recursive structure.</p>

<pre><code>Vec Sample(vec3 O, vec3 D, int depth)
{
  vec3 color(0, 0, 0);
  float t;
  if (checkIfIntersectSomething(D, t))
  {

    vec3 I = O + t * D; //get to the intersection point on the object
    vec3 diffuse = getMaterial(I);

    vec3 L = vec3(-1 + Rand(2.0f), 20, 9 + Rand(2.0f)) - I; //(-1,20,9) is Hard-code of the light position, and I add Rand(2.0f) on X and Z axis
    //so that I have an area light instead of a point light
    L = normalize(L);
    float ndotl = dot(I.getNormal(), L); //the closer the dotProdutc is to 1.0, the more the light and surface face each other
    if (ndotl &gt; 0)
    {
        if (!checkRayLightIntersection(L)) {
            float dist = distFromLight(I);
            color += diffuse * ndotl * vec3(1000.0f, 1000.0f, 850.0f) * (1.0f / (dist * dist));
        }
    }

    // russian roulette
    float Psurvival = CLAMP((diffuse.r + diffuse.g + diffuse.b) * 0.33333f, 0.2f, 0.8f);

    // continue random walk 
    float rand = Rand(1.0f);
    if (depth &lt; 10 &amp;&amp; rand &lt; Psurvival)
    {
        //Besides russian roulette, I also do another weight, because rays that go towards the horizon will bring back very little energy
        //so I make a random distribution that favours those rays who are close to the normal of the hit point, this is DiffuseReflectionCosineWeighted(). It creates a Random bounce but proportional to N dot R
        vec3 R = DiffuseReflectionCosineWeighted(I.getNormal());//there is a weight

        float prob = 1.0;
        float cosTheta = fabs(dot(I.getNormal(), R));
        if (cosTheta &gt; 1e-6) prob = cosTheta / M_PI;

        color += diffuse * Sample(I + R * EPSILON, R, depth + 1) * (1.0f / Psurvival); //the cosTheta() of the attenuation of the rendering equation gets simplified with the cosTheta of the "prob"
        //the PI of the prob gets simplified with the BRDF where we are using the ideal BRDF = diffuse/PI
    }
}
return color;
</code></pre>

<p>}</p>

<p>EDIT 6: Trichoplax was right, I had a problem calculating the <strong>distanceFromObject</strong> and <strong>distanceFromLight</strong>. After solving this, we realized that some sort of shadows appeared (shown in one of the images above). He then suggested that such a broad light might have been washing out the shadows, and so it was. By reducing the light size and by using a high amount of SPP we realized that shadows are there and visibile: <a href="https://i.sstatic.net/Eez7y.jpg" rel="nofollow noreferrer">https://i.sstatic.net/Eez7y.jpg</a> . As a drawback tho, way less rays reach the light, giving back an image with high noise, but this is another kind of problem and I created another question for it. Thanks to @trichoplax and @julien for their help</p>

CC BY-SA 3.0
2016-04-07T13:10:39.740Z
I can see shadows. Therefore i do not concurr with your analysis. However secondary light might have too high reflectivity. Could you try using a simpler scene.
joojaa
CC BY-SA 3.0
2016-04-07T13:31:07.520Z
@joojaa edited the description with a target reference image
Tarta
CC BY-SA 3.0
2016-04-07T13:57:06.897Z
Using a simpler scene helps you debug the effect. To me it looks like the shadows are there your scene is just not color corrected and you bounce too nuch energy.
joojaa
CC BY-SA 3.0
2016-04-07T16:27:09.340Z
@joojaa I just posted some of the code because I am using a much more complex framework. That's why I was avoiding rendering more simple scenes, because it requires me some effort finding out how exactly the scene loader works.  Ok, may you suggest me code-wisely what I could change to color the scene correctly?
Tarta
CC BY-SA 3.0
2016-04-07T23:24:54.983Z
It might help to restrict your image to a very small region in which you expect to have a very obvious shadow. Then you can get a more detailed look at that region in a much shorter rendering time, to see if it really is zero shadow, or just a less obvious softened shadow.
trichoplax is on Codidact now
CC BY-SA 3.0
2016-04-11T01:55:11.087Z
Seeing your updated screenshot (3rd edit), is it still rendered with 16spp? With a only small area light, and no bidirectional path tracing or next event estimation, it's expected to have a lot of noise. Have you tried a much higher number, like 1000spp?
Julien Guertault
CC BY-SA 3.0
2016-04-11T12:59:48.033Z
@JulienGuertault yes it is still rendered with 16 spp. No I didn't try with 1000spp because it would take a huge amount of time and this project is about a research topic which aims to good quality images with low number of spp.
Tarta
CC BY-SA 3.0
2016-04-13T22:48:22.993Z
Comments are not for extended discussion; this conversation has been [moved to chat](http://chat.stackexchange.com/rooms/38357/discussion-on-question-by-tarta-path-tracer-not-rendering-shadows).
trichoplax is on Codidact now
CC BY-SA 3.0
2016-04-13T23:00:09.740Z
@SlippD.Thompson and Tarta, this is not an appropriate place to have a discussion about behaviour on an external site. You are both very welcome on this site, but complaints relating to other sites will need to be addressed there, not here. If you use the chat room created to continue discussing the original path tracing question, please do so respectfully. Heated conflict over differences tends to decrease understanding rather than increase it.
trichoplax is on Codidact now
<c++><lighting><shadow><pathtracing><global-illumination>
Path tracer not rendering shadows
Tarta
# Path tracer not rendering shadows

(Asked by: username_0 on 2016-04-07)

I am trying to implement for research purposes a gradient-domain path tracer. In order to achieve this step I first need a working Path tracer. I have been creating one so far but results are wrong and I will explain you why. Some concepts:

I am working on paths that are generated before sampling them. I mean that the first step of my algorithm consists in calculating a path for a certain pixel (x,y). This path will perform some bounces within the scene and, if terminates on the light, will be considered valid.

First of all: the following is the definition of a struct (will be declared and initialized later), containing some scene info needed to cast ray from the camera into the scene.

```
struct RenderData
{
    vec3 E;
    vec3 p1;
    vec3 dx;
    vec3 dy;
};

```

It is initialized in the InitializeScene method:

```
void InitializeScene(){ // setup virtual screen plane
  vec3 E( 2, 8, -26 ); //Eye position
  vec3 V( 0, 0, 1 ); //LookAt vector
  float d = 0.5f, ratio = SCRWIDTH / SCRHEIGHT, focal = 18.0f;
  vec3 p1(E + V * focal + vec3(-d * ratio * focal, d * focal, 0)); // top-left screen corner in SCREEN SPACE
  vec3 p2(E + V * focal + vec3(d * ratio * focal, d * focal, 0)); // top-right screen corner
  vec3 p3(E + V * focal + vec3(-d * ratio * focal, -d * focal, 0)); // bottom-left screen corner
  mat4 M = rotate( mat4( 1 ), r, vec3( 0, 1, 0 ) );
  p1 = vec3(M * vec4(p1, 1.0f)); //rotating the above points
  p2 = vec3(M * vec4(p2, 1.0f));
  p3 = vec3(M * vec4(p3, 1.0f));
  renderData.dx = (p2 - p1) / (float)SCRWIDTH;
  renderData.dy = (p3 - p1) / (float)SCRHEIGHT;
  renderData.E = vec3(M * vec4(E, 1.0f));
  renderData.p1 = p1;
}

```

The code above is to make you understand how I initialized the scene.
I also have structs to store informations about my paths:

```
struct PathVert {
   vec3 p; vec3 n; //hit point and normal of the surface hit 
};

struct Path {
   PathVert verts[MAX_DEPTH]; //maxDepth is 15 for now
   int vertCount;
   int x, y; //which pixel this path is referring to
};

```

therefore I start consider pixel after pixel.

```
for (int y = 0; y < SCRHEIGHT; y++) for (int x = 0; x < SCRWIDTH; x++)
{     
   Path path;
   if(generatePath(x,y, path)){
      Sample(path);
   }
}

```

The **generatePath()** method indeed tracks the path into the scene and checks all the vertices it hits. The **checkIfRayIntersectSomething(t)** method you will see used, it's just a pseudo method implemented in my framework and that I omit posting cause of its length. I use it to check if my ray hits something in the scene, if it does, it update the "t" with the distance to that object. NOTE: the light is not considered an object itself. Hence, I also have a **checkRayLightIntersection(hitLightPoint)** which checks the intersection with the light, if there is any, the hitLightPoint is updated with the point on the light I have been hitting.
The light is a 2D surface.

```
Vec lightPos = Vec(5, 15, 2); //hard coded position of the light

```

And the light, as said, is a surface, but exactly a square surface, whose 4 angles are:

```
Vec P1 = Vec(lightPos.x - 20, lightPos.y, lightPos.z + 20);
Vec P2 = Vec(lightPos.x + 20, lightPos.y, lightPos.z + 20);
Vec P3 = Vec(lightPos.x + 20, lightPos.y, lightPos.z - 20);
Vec P4 = Vec(lightPos.x - 20, lightPos.y, lightPos.z - 20);

```

Quite big, I know, so the first question relies on this aspect, is it correct having such a big light?

But let's go to the main methods. Hereby you can see the GeneratePath method:

```
bool GeneratePath(int x, int y, Path &path){
   path.x = x;
   path.y = y;
   path.vertCount = 0;

   vec3 P = renderData.p1 + renderData.dx * ((float)(x) + Rand(1)) +  renderData.dy * ((float)(y) + Rand(1));
   vec3 O = renderData.E + vec3(Rand(0.4f) - 0.2f, Rand(0.4f) - 0.2f, Rand(0.4f) - 0.2f);
   vec3 D = normalize(P - O); //direction of the first ray, the one from the camera towards the pixel we are considering

   for (int depth = 1; depth <= MAXDEPTH; depth++){
    float t;
    Vec hitLightPoint;
    PathVert vert;
    if (!checkIfRayIntersectSomething(t)){
        //we didn't find any object.. but we still may have found the light which is an object non represented in the scene
        //the depth check avoids me rendering the light as a white plane
        if (depth > 1 && checkRayLightIntersection(O, D, hitLightPoint)){
            //update the vertex since we realized it's the light
            vert.p = hitLightPoint;
            vert.n = Vec(0, -1, 0);//cause the light is pointing down
            path.verts[depth - 1] = vert;
            path.vertCount++;
            return true; //light hit, path completed    
        }
        return false; //nothing hit, path non valid
    }
    //otherwise I got a hit into the scene
    vert.p = O + D * t; //reach the hitPoint
    vert.n = methodToFindTheNormal();
    vert.color = CalculateColor(vert.p); //according to the material properties (only diffuse objects so far)
    path.verts[depth - 1] = vert;
    path.vertCount++;

    //since I have the light, and a path terminates when it hits the light, I have to check out also if my ray hits this light,
    //and if does, I have to check whether it first hits the light or the object just calculated above
    //moreover with the "depth > 1" check, I avoid again rendering the light which otherwise would be visible as a white plane

    if (depth > 1 && checkRayLightIntersection(O, D, hitLightPoint)){
        float distFromObj = length(vert.p);
        float distFromLight = length(hitLightPoint);
        if (distFromLight < distFromObj){
            //update the vertex since we realized it's the light
            vert.p = hitLightPoint;
            vert.n = Vec(0, -1, 0);
            vert.color = Vec(1, 1, 1);// TODO light color? or light emission?

            path.verts[depth - 1] = vert;
            return true; //light hit, path completed
        }
    }
    if (depth == MAXDEPTH) return false;
       Vec newDir = BSDFDiffuseReflectionCosineWeighted(vert.n, D);//explained later
       D = newDir;
       O = vert.p;
   }
   return false;
}

```

The BSDFDiffuseReflectionCosineWeighted() just calculate the new directions, tested and working. What remains last is the Sample method which calculates the final color of the pixel.

```
Vec Sampling(Path &path){

   Vec color(1, 1, 1);

   for (int vert = 0; vert < path.vertCount - 1; vert++) { //considers the last vertex as the light
      const PathVert &currVert = path.verts[vert];
      const PathVert &nextVert = path.verts[vert + 1];
      Vec wo = (nextVert.p - currVert.p).norm();
      double cosTheta = fabs(wo.dot(currVert.n));
      float PDF = cosTheta/PI;
      if (cosTheta <= 1e-6) return Vec();
      //considering only DIFFUSE objects
      color = color.mult(currVert.color * (cosTheta / M_PI) / PDF);
   }
   return color.mult(Vec(10.0f, 10.0f, 10.0f)); //multiplication for the light emission?
}

```

Result with 16SPP is:

[![enter image description here](https://i.sstatic.net/HNOah.jpg)](https://i.sstatic.net/HNOah.jpg)

As you can see the results are not that bad but there is one main problem: shadows are missing.. tried many combinations but no improvements. There is some error in the algorithm itself. Can you help me understanding why? Thanks in advance.

EDIT: this is my target reference:

[![enter image description here](https://i.sstatic.net/QHG2z.jpg)](https://i.sstatic.net/QHG2z.jpg)

EDIT 2: this is what happens in case the **depth > 1** check is omitted:

[![enter image description here](https://i.sstatic.net/hiqEY.jpg)](https://i.sstatic.net/hiqEY.jpg)

EDIT 3: after fixing the **distFromLight** and **distFromObj** I got the following results:

[![some clue of shadows](https://i.sstatic.net/JvGhf.jpg)](https://i.sstatic.net/JvGhf.jpg)

One of the 2 problems as suggested by @username_2 could be too soft shadows. So I tried to lower the size of the light, increasing his power to compensate. Results are not good and shown here:

[![enter image description here](https://i.sstatic.net/8aYPH.png)](https://i.sstatic.net/8aYPH.png)

Moreover as requested I can post some more code:

```
bool checkRayLightIntersection(Vec O, Vec D, Vec & hitLightPoint){
  //getting the 4 corners of my light
  Vec P1 = Vec(lightPos.x - 20, lightPos.y, lightPos.z + 10);
  Vec P2 = Vec(lightPos.x + 20, lightPos.y, lightPos.z + 10);
  Vec P3 = Vec(lightPos.x + 20, lightPos.y, lightPos.z - 10);
  Vec P4 = Vec(lightPos.x - 20, lightPos.y, lightPos.z - 10);

  //the majority of the methods first find out where the ray intersects the plane that the rectangle lies on Ax + By + Cz + D = 0
  //in our case the equation of that plane is easy -> D = 20
  // answers.google.com/answers/threadview?id=18979

  float t = -(-O.y + lightPos.y) / (-D.y);
  if (t > 0){
    Vec hitPoint = O + D * t;
    Vec V1 = (P2 - P1).norm();
    Vec V2 = (P3 - P2).norm();
    Vec V3 = (P4 - P3).norm();
    Vec V4 = (P1 - P4).norm();
    Vec V5 = (hitPoint - P1).norm();
    Vec V6 = (hitPoint - P2).norm();
    Vec V7 = (hitPoint - P3).norm();
    Vec V8 = (hitPoint - P4).norm();

    if (V1.dot(V5) > 0.0 && V2.dot(V6) > 0.0 && V3.dot(V7) > 0.0 && V4.dot(V8) > 0.0){
        hitLightPoint = hitPoint;
        return true;
    }
  }
  return false;
}

```

Instead, the **checkIfRayIntersectSomething** method is part of the framework itself (it has just another name), it's not done by me but it's tested and working.

EDIT 4: tried to render a small part of the scene with high number of spp. Tried 512 spp for the bottom right corner where should appear shadows according to the target image. Results show that shadows are not there:

[![Bottom right corner at 512spp](https://i.sstatic.net/ohZJu.png)](https://i.sstatic.net/ohZJu.png)

EDIT 5:

I hereby post the code used to render the target image, it does at the same time my **Sampling()** and **GeneratePath()** methods using a recursive structure.

```
Vec Sample(vec3 O, vec3 D, int depth)
{
  vec3 color(0, 0, 0);
  float t;
  if (checkIfIntersectSomething(D, t))
  {

    vec3 I = O + t * D; //get to the intersection point on the object
    vec3 diffuse = getMaterial(I);

    vec3 L = vec3(-1 + Rand(2.0f), 20, 9 + Rand(2.0f)) - I; //(-1,20,9) is Hard-code of the light position, and I add Rand(2.0f) on X and Z axis
    //so that I have an area light instead of a point light
    L = normalize(L);
    float ndotl = dot(I.getNormal(), L); //the closer the dotProdutc is to 1.0, the more the light and surface face each other
    if (ndotl > 0)
    {
        if (!checkRayLightIntersection(L)) {
            float dist = distFromLight(I);
            color += diffuse * ndotl * vec3(1000.0f, 1000.0f, 850.0f) * (1.0f / (dist * dist));
        }
    }

    // russian roulette
    float Psurvival = CLAMP((diffuse.r + diffuse.g + diffuse.b) * 0.33333f, 0.2f, 0.8f);

    // continue random walk 
    float rand = Rand(1.0f);
    if (depth < 10 && rand < Psurvival)
    {
        //Besides russian roulette, I also do another weight, because rays that go towards the horizon will bring back very little energy
        //so I make a random distribution that favours those rays who are close to the normal of the hit point, this is DiffuseReflectionCosineWeighted(). It creates a Random bounce but proportional to N dot R
        vec3 R = DiffuseReflectionCosineWeighted(I.getNormal());//there is a weight

        float prob = 1.0;
        float cosTheta = fabs(dot(I.getNormal(), R));
        if (cosTheta > 1e-6) prob = cosTheta / M_PI;

        color += diffuse * Sample(I + R * EPSILON, R, depth + 1) * (1.0f / Psurvival); //the cosTheta() of the attenuation of the rendering equation gets simplified with the cosTheta of the "prob"
        //the PI of the prob gets simplified with the BRDF where we are using the ideal BRDF = diffuse/PI
    }
}
return color;

```

}

EDIT 6: username_2 was right, I had a problem calculating the **distanceFromObject** and **distanceFromLight**. After solving this, we realized that some sort of shadows appeared (shown in one of the images above). He then suggested that such a broad light might have been washing out the shadows, and so it was. By reducing the light size and by using a high amount of SPP we realized that shadows are there and visibile: <https://i.sstatic.net/Eez7y.jpg> . As a drawback tho, way less rays reach the light, giving back an image with high noise, but this is another kind of problem and I created another question for it. Thanks to @username_2 and @username_3 for their help

**Tags:** c++, lighting, shadow, pathtracing, global-illumination

> **Comments:**
> **username_1**: I can see shadows. Therefore i do not concurr with your analysis. However secondary light might have too high reflectivity. Could you try using a simpler scene.  
> **username_0**: @username_1 edited the description with a target reference image  
> **username_1**: Using a simpler scene helps you debug the effect. To me it looks like the shadows are there your scene is just not color corrected and you bounce too nuch energy.  
> **username_0**: @username_1 I just posted some of the code because I am using a much more complex framework. That's why I was avoiding rendering more simple scenes, because it requires me some effort finding out how exactly the scene loader works.  Ok, may you suggest me code-wisely what I could change to color the scene correctly?  
> **username_2**: It might help to restrict your image to a very small region in which you expect to have a very obvious shadow. Then you can get a more detailed look at that region in a much shorter rendering time, to see if it really is zero shadow, or just a less obvious softened shadow.  
> **username_3**: Seeing your updated screenshot (3rd edit), is it still rendered with 16spp? With a only small area light, and no bidirectional path tracing or next event estimation, it's expected to have a lot of noise. Have you tried a much higher number, like 1000spp?  
> **username_0**: @username_3 yes it is still rendered with 16 spp. No I didn't try with 1000spp because it would take a huge amount of time and this project is about a research topic which aims to good quality images with low number of spp.  
> **username_2**: Comments are not for extended discussion; this conversation has been [moved to chat](http://chat.stackexchange.com/rooms/38357/discussion-on-question-by-username_0-path-tracer-not-rendering-shadows).  
> **username_2**: @SlippD.Thompson and username_0, this is not an appropriate place to have a discussion about behaviour on an external site. You are both very welcome on this site, but complaints relating to other sites will need to be addressed there, not here. If you use the chat room created to continue discussing the original path tracing question, please do so respectfully. Heated conflict over differences tends to decrease understanding rather than increase it.  

---

## 2 answers

### Answer by username_2 on 2016-04-08. Score: 6 | Accepted answer

## The problem appears to be unintentionally transparent surfaces

Although the image is grainy, it is sufficiently clear to estimate that all of the darker regions are due to surfaces facing away from the light, rather than due to shadows cast onto surfaces facing towards the light. So it does seem that there is a problem, and the lack of shadows is not just due to having soft shadows from a large area light that don't show up well, or excessively reflective surfaces that provide too much ambient light. Somehow light is getting through your surfaces.

## The probable source of this problem

Since you check for intersection with the light and the objects separately, the crucial point where this could go wrong is where you check

```
if (distFromLight < distFromObj)

```

You correctly terminate the path and return the light colour if this check is true (that is, if the light is closer than the nearest object in that direction). So the problem is likely to be with one of the two variables you are comparing. If they were both correct, then you would see shadows.

These are calculated on the preceding lines

```
float distFromObj = length(vert.p);
float distFromLight = length(hitLightPoint);

```

As pointed out in [username_3 username_3's answer](https://computergraphics.stackexchange.com/a/2281/231), these calculations may be redundant.

`length(vert.p)` should still be available as `t`, used 11 lines previously in the calculation of `vert.p`. This not only avoids unnecessary calculation, but eliminates a potential source of error. Then this length is likely to be correct unless there is a problem with `checkIfRayIntersectSomething()`.

This leaves only `length(hitLightPoint)` as a source of error, unless there is a more subtle problem. It would be useful to be able to eliminate the possibility of problems with `length()` or `hitLightPoint`, which is calculated by `checkRayLightIntersection()`.

## We need to see the code to be able to help

`length()`, `checkIfRayIntersectSomething()` and `checkRayLightIntersection()` are not currently included with the code in your question, so we can't analyse these. If you are able to share these that would help narrow down the source of the problem.

## A simple way to check whether this is the real problem

You could also verify whether this analysis of the problem is correct by rendering a scene which will eliminate the other possibilities (shadows too soft to distinguish, or shadows washed out by too much reflected light from the rest of the scene). If the light getting through the surfaces is the source of your problem, then it will apply even if there is no path from the light to a surface, provided that surface is facing the light. So render a simple scene that has a light with a plane beneath it, and another plane beneath that, separated by a gap big enough to easily be seen. The lower plane is completely occluded by the upper plane, so should receive no light (no soft shadows and no reflected light from elsewhere). If the lower plane is lit, then you know that this is indeed the problem and you can focus your efforts here.

If the lower plane is *not* lit, then either this is not the problem, or there is a problem with the scale and the occluding plane may need to be at a different distance from the light in order for the problem to show up. You can use your existing scene as a source of estimated distances for which this problem occurs, and you can use inclined planes in order to see the effect for a variety of ratios of object distance to light distance within a single image.

Sharing the images resulting from such simple scenes may help us identify the problem more precisely.

---

I notice that in a comment on the question you mention that you do not have a straightforward way of generating scenes yourself. Perhaps there are much simpler scenes already publicly available in the same file format, that you can use to test this?

If not, it sounds like the only thing you have control over is the light. In that case, you could try putting the light underground. If surfaces are not transparent, the scene should be uniformly black. If surfaces are being lit despite an occluding surface, you may see evidence for it this way that would be hard to discern when there is lots of secondary light bouncing around.

> **Comments:**
> **username_0**: your answer was very useful, thanks first of all ! You where right, **distFromLight** and  **distFromObj** where wrongly calculated. I fixed it. Something has been improved but the problem is not fixed yet. I posted one picture showing the new results. I thought maybe the error relies on too soft shadows due to a too big light, therefore I tried to lower the dimension of the light. No matter how increase the power of the light, but in this case results are bad (posted a second picture about it). Hope it helps you. In the meantime I am working on the .obj loader for a simpler scene.  
> **username_2**: A smaller light means more of the rays will never reach the light, so you will need to increase the number of rays to give a better quality image. This will also increase the time it takes to render. It might be worth instead making the light as large as possible (an infinite or very large plane). This way, if the problem is transparent surfaces, you should see even more evidence for it, with the added benefit that the largest possible light will reduce the graininess of the image.  
> **username_2**: I've added an additional final paragraph about putting the light underground as a test (see edited answer) that may help if you are unable to change the scene, but able to move the light.  
> **username_0**: Ok, I followed your suggestion. I tried to put the light beneath the scene and nothing is lit up. No light passes through surfaces and therefore surfaces are not transparent. At least we can exclude this possibility.. so it can be that shadows are too soft or washed out by too much light reflected from the rest of the scene. How do you suggest me to proceed?  
> **username_2**: Have you tried rendering just a small section of the image yet? For example, the bottom right sixteenth of the image contains are strong shadow in the target image. Rendering just that small image would allow you to use a much higher number of samples per pixel, to give a clearer image and more idea of what is going on.  
> **username_0**: just did it, shadows are not there.. you can find the piece of image above at EDIT 4  
> **username_0**: Let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/38237/discussion-between-username_0-and-username_2).  
> **username_0**: EDIT 6 contains the final explanation  

---

### Answer by username_3 on 2016-04-07. Score: 5

> is it correct having such a big light?

I don't see any problem with having a big area light. That said, it also depends on the scale of your scene. If the light is large compared to it, shadows will tend to be more diffuse, like under an overcast sky.

> shadows are missing.. [...] Can you help me understanding why?

I haven't found any blatant mistake, but there are two elements that look suspicious to me.

```
if (depth > 1 && checkRayLightIntersection(O, D, hitLightPoint)) {
    // ...
    return true; //light hit, path completed
}

```

Although this part seems to be correct as far as I can tell, I would try to make sure there is no bug introducing a bias. For every bounce, it checks if there is a ray hitting the light first, and if so it breaks out of the loop: in case the code is incorrect and it happens to hit the light more often than it should, the shadows will disappear.

Moreover, in the first if:

```
return false; //nothing hit, path non valid

```

and the related loop:

```
for (int y = 0; y < SCRHEIGHT; y++) for (int x = 0; x < SCRWIDTH; x++)
{     
  Path path;
   if(generatePath(x,y, path)){
      Sample(path);
   }
}

```

Make sure you still take the paths that hit nothing into account when averaging the color of the pixel, otherwise this will also introduce a bias.

As username_1 said in the comment, you should try using a simpler scene: a few spheres or cubes, no texture. You can also reduce the number of bounces to one or two to see only direct lighting. If necessary, maybe even use a constant sky dome first instead of a light area.

---

Other remarks:

```
struct Path {
   PathVert verts[MAX_DEPTH]; //maxDepth is 15 for now
   int vertCount;
   int x, y; //which pixel this path is referring to
};

```

Do you need to store x and y?

```
float distFromObj = length(vert.p);
float distFromLight = length(hitLightPoint);

```

Don't you already have two `t` that you can compare?

> **Comments:**
> **username_0**: First of all thanks for your answer :)  So, the method to check the intersection between the ray and the square light is right. I have already been asking about it cause of a small error I couldn't find. The method and the relative solution can both be found here:  http://stackoverflow.com/questions/36180741/intersection-of-ray-and-rectangle-in-c/36186088#36186088  I also edited the question adding a picture that shows the result in case I omit the **depth > 1** check. Maybe it can help understanding where the problem is.. thanks once more  

---

