<p>The problem, I think, is that you are comparing two different kinds of depth values. For want of better terms, the rasterizer's depth buffer stores 'parallel' depth, whereas raymarching calculates 'true' depth - that is, the rasterizer stores the distance to the plane parallel to the view plane which contains the point, but the raymarcher calculates the <em>actual</em> 3D distance from camera to point.
<a href="https://i.sstatic.net/ad4zF.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ad4zF.png" alt="enter image description here"></a></p>

<p>You can see from the picture, the relation should be $rasterDepth = rayDepth*cos(A)$</p>

<p>So try this - in your <code>getRayDirection</code> function, <em>before</em> you do your view rotation, save your fragment ray's original normalized Z value somewhere, maybe an extra field in your Ray struct. Let's call it <code>cosA</code>.</p>

<p>Then in your <code>rayMarch</code> function the relevant code will be</p>

<pre><code>if( depth * cosA &gt; rasterized_depth){
        break;
}
</code></pre>

<p>Hopefully that should fix the problem.</p>

CC BY-SA 4.0
2018-05-30T19:10:52.350Z
BINGO we got a winner!  This was exactly the problem,  I'll edit my question to show how I fixed this, I sort of had an intuition that it was going to be something like this, but there's no way I would have been able to figure this out any time soon with out your help!
Krupip
CC BY-SA 4.0
2018-05-30T18:22:04.123Z
2018-05-30T18:22:04.123Z
russ
<p>I'm trying to apply raymarching on top of 3D rasterization, ordering what 3D object gets displayed based on the depth of a rasterization pass and the raymarch depth.   After getting the transform right for <em>position</em> (<a href="https://www.scratchapixel.com/lessons/3d-basic-rendering/ray-tracing-generating-camera-rays/generating-camera-rays" rel="nofollow noreferrer">using the information here</a>) I was able to have the position of the raymarched spheres coherent with the rasterized cubes.   I ran into issues however with depth being different at the edges of the screen compared to the middle, despite position never changing.</p>
<p>These gifs should show what I'm talking about</p>
<p><a href="https://i.sstatic.net/rZakQ.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/rZakQ.gif" alt="enter image description here" /></a></p>
<p><a href="https://i.sstatic.net/4fD5Z.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/4fD5Z.gif" alt="enter image description here" /></a></p>
<p>higher res versions
<a href="https://ibb.co/ebji0y" rel="nofollow noreferrer">https://ibb.co/ebji0y</a>
<a href="https://ibb.co/dhUGLy" rel="nofollow noreferrer">https://ibb.co/dhUGLy</a></p>
<p>Something odd is going on with depth, but I don't understand what I'm doing wrong?  Here is my fragment shader for my depth texture render pass:</p>
<pre><code>#version 330 core
layout(location = 0) out vec4 FragColor;
layout(location = 1) out float DepthColor;
in vec2 TexCoord;

// texture samplers
uniform sampler2D texture0;
uniform sampler2D texture1;
uniform float near;
uniform float far;

float LinearizeDepth(float depth)
{
    float z = depth * 2.0 - 1.0; // back to NDC
    return (2.0 * near * far) / (far + near - z * (far - near));
}
void main()
{

    float depth = LinearizeDepth(gl_FragCoord.z);
    DepthColor = depth;
    FragColor = mix(texture(texture0, TexCoord), texture(texture1, TexCoord), 0.3);
}
</code></pre>
<p>and here is my shader for the final raymarch pass:</p>
<pre><code>#version 330 core
out vec4 FragColor;

in vec2 TexCoords;

uniform sampler2D u_screen_texture;
uniform sampler2D u_depth_texture;
uniform float u_fov;
uniform vec3 u_rotation;
uniform vec3 u_origin;
uniform vec3 u_resolution;

const float inf = uintBitsToFloat(0x7F800000u);

vec3 rotVecX(in vec3 vector, in float theta){
    mat3 rotmat = mat3(1, 0, 0,
                       0, cos(theta), -sin(theta),
                       0, sin(theta), cos(theta));
    return vector * rotmat;
}

vec3 rotVecY(in vec3 vector, in float theta){
    mat3 rotmat = mat3(cos(theta), 0, sin(theta),
                       0, 1, 0,
                       -sin(theta), 0, cos(theta));
    return vector * rotmat;
}

vec3 calculateFragementRay(in vec2 fragCoord, in vec3 resolution){
    vec2 uv = fragCoord;
    uv.x = (uv.x * 2.0) - 1.0;
    uv.y = (2.0 * uv.y) - 1.0;
    if(resolution.x &gt;= resolution.y){
        uv.x *= resolution.x/resolution.y;
    }else{
        uv.y *= resolution.y/resolution.x;
    }
    float tan_fov = tan(u_fov/2.0);
    vec2 pxy = uv * tan_fov;
    vec3 ray_dir = normalize(vec3(pxy, 1));
    return ray_dir;
}

vec3 getRayDirection(in vec2 fragCoord, in vec3 resolution, in vec3 rotation){
    // Normalized pixel coordinates (from -1 to 1)
    vec3 ray_dir = calculateFragementRay(fragCoord, resolution);
    ray_dir = rotVecX(ray_dir, rotation.x);
    ray_dir = rotVecY(ray_dir, rotation.y);

    return ray_dir;
}

struct Ray{
    vec3 origin;
    vec3 direction;
};

Ray getRay(in vec2 fragCoord, in vec3 resolution, in vec3 rotation, in vec3 origin){
    vec3 ray_dir = getRayDirection(fragCoord, resolution, rotation);
    vec3 ray_origin = origin;
    return Ray(ray_origin, ray_dir);
}

struct RayMarchSettings{
    int max_steps;
    float max_distance;
    float min_distance;
    float depth_weight;
};

struct Sphere{
    vec3 origin;
    float radius;
};

float signedDistance(vec3 p, in Sphere sphere){
    return (length(p - sphere.origin) - sphere.radius);
}

const int NULL_ID = -1;
const int SPHERE_ID = 0;
const int VSPHERE_ID = 1;

Sphere test_sphere = Sphere(vec3(0.0, 4.0, 4.0),1.0);
Sphere volume_sphere = Sphere(vec3(0.0, 1.0, 0.0),2.0);
float signedDistance(vec3 ray_endpoint, out int obj_id){
    float d0 = signedDistance(ray_endpoint, test_sphere);
    float d1 = signedDistance(ray_endpoint, volume_sphere);
    float minval = min(d0, d1);
    obj_id = minval == d0 ? SPHERE_ID : VSPHERE_ID;
    return minval;
}

bool rayMarch(Ray ray, in RayMarchSettings settings, in float rasterized_depth, out float out_depth, out int out_obj_id){
    float depth = 0.0;
    for( int i = 0; i &lt; settings.max_steps; ++i){
        vec3 ray_endpoint = ray.origin + ray.direction * depth;
        int obj_id;
        float ray_distance = signedDistance(ray_endpoint, obj_id);
        depth += ray_distance * settings.depth_weight;

        if( depth &gt; rasterized_depth){
            break;
        }
        if(ray_distance &lt;= settings.min_distance){
            out_depth = depth;
            out_obj_id = obj_id;
            return true;
        }

        if(depth &gt; settings.max_distance){
            break;
        }
    }
    out_depth = inf;
    out_obj_id = NULL_ID;
    return false;
}

float checkerBoardTexture(in vec2 uv){
    vec2 q = floor(uv);
    return clamp(mod( q.x+q.y, 2.0 ), 0.25, 1.0);
}
float fogify(in float dist){
    return  1.0/ (1.0 + dist*dist * 0.1);
}

RayMarchSettings rm_settings = RayMarchSettings(128, 100000.0, 0.0001, 0.5);

void main()
{

    vec3 origin = vec3(u_origin.x, u_origin.y, u_origin.z);
    origin = origin.zyx;
    Ray ray = getRay(TexCoords, u_resolution, u_rotation, origin);
    vec3 texture_color = texture(u_screen_texture, TexCoords).rgb;
    float rasterized_depth = texture(u_depth_texture, TexCoords).r;

    float depth = inf;
    int obj_id = NULL_ID;


    vec3 color;
    if(rayMarch(ray, rm_settings,rasterized_depth,  depth, obj_id)){
        vec3 endpoint = ray.origin + ray.direction * depth;
        float pattern_value = checkerBoardTexture(endpoint.xz * 4.0);

        if(obj_id == SPHERE_ID){
            color = vec3(1.0)*pattern_value;
        }
        if(obj_id == VSPHERE_ID){
            color = vec3(0.0,  1.0, 0.0)*pattern_value;
        }

        switch(obj_id){
            case SPHERE_ID:
            color = vec3(1.0)*pattern_value;
            break;
            case VSPHERE_ID:
            color = vec3(0.0,  1.0, 0.0)*pattern_value;
            break;
            default:
            color = vec3(1.0,0.0,1.0)*pattern_value;
        }
     }else{
        float re_rasterized_depth = rasterized_depth == inf ? 0.0 : rasterized_depth;
        color = texture_color +re_rasterized_depth/100.f;
     }
    FragColor = vec4(color, 1.0);
}
</code></pre>
<p>The <code>u_depth_texture</code> is where the depth comes from (and what <code>DepthColor</code> corresponds to)  The <code>u_screen_texture</code> is just a texture of the rasterization pass information (output of <code>FragColor</code>).</p>
<h1>UPDATE:</h1>
<p>So <a href="https://computergraphics.stackexchange.com/a/7675/6530">this answer</a> outlined exactly my problem.  I had to make the following changes:</p>
<pre><code>vec3 getRotatedRayDirection(vec3 ray_dir, in vec3 rotation){
    // Normalized pixel coordinates (from -1 to 1)
    //vec3 ray_dir = calculateFragementRay(fragCoord, resolution);
    ray_dir = rotVecX(ray_dir, rotation.x);
    ray_dir = rotVecY(ray_dir, rotation.y);
    return ray_dir;
}

struct Ray{
    vec3 origin;
    vec3 direction;
    float cosA;
};

Ray getRay(in vec2 fragCoord, in vec3 resolution, in vec3 rotation, in vec3 origin){

    vec3 ray_dir = calculateFragementRay(fragCoord, resolution);
    vec3 rot_ray_dir = getRotatedRayDirection(ray_dir, rotation);
    vec3 ray_origin = origin;
    float cosA = ray_dir.z; // a.z/length(a), already normalized so length(a) == 1.0. 
    return Ray(ray_origin, rot_ray_dir, cosA);
}
</code></pre>
<p>Above I basically added cosA as a seperate value onto the ray, as suggested by the linked answer, with the appropriate code changes to avoid re-running <code>calculateFragmentRay</code>.  However I needed to figure out which cosine corresponded to the one in thier picture, I knew it was a lateral dimension, it happened to be Z in this case, and I used the standard method of retrieving cosine from a normalized vector.</p>
<p>After that I did exactly what the linked answer said and made the following change:</p>
<pre><code>if( (depth * ray.cosA) &gt; rasterized_depth){
    break;
}
</code></pre>
<p>And here are the results! No depth artifacts!</p>
<p><a href="https://i.sstatic.net/IptJC.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/IptJC.gif" alt="enter image description here" /></a></p>
<p><a href="https://i.sstatic.net/kIzIU.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/kIzIU.gif" alt="enter image description here" /></a></p>

CC BY-SA 4.0
2018-05-31T04:02:55.990Z
Glad it worked :) By the way, you don't need to divide by length(rayDir) to get cosA - since you already normalized it in calculateFragmentRay.
russ
CC BY-SA 4.0
2018-05-31T04:20:26.743Z
@russ good catch, I'll edit it out
Krupip
<opengl><rendering><glsl><rasterizer><raymarching>
How to align Ray Marching on top of traditional 3D rasterization?
Krupip
# How to align Ray Marching on top of traditional 3D rasterization?

(Asked by: username_0 on 2018-05-30)

I'm trying to apply raymarching on top of 3D rasterization, ordering what 3D object gets displayed based on the depth of a rasterization pass and the raymarch depth. After getting the transform right for *position* ([using the information here](https://www.scratchapixel.com/lessons/3d-basic-rendering/ray-tracing-generating-camera-rays/generating-camera-rays)) I was able to have the position of the raymarched spheres coherent with the rasterized cubes. I ran into issues however with depth being different at the edges of the screen compared to the middle, despite position never changing.

These gifs should show what I'm talking about

[![enter image description here](https://i.sstatic.net/rZakQ.gif)](https://i.sstatic.net/rZakQ.gif)

[![enter image description here](https://i.sstatic.net/4fD5Z.gif)](https://i.sstatic.net/4fD5Z.gif)

higher res versions
<https://ibb.co/ebji0y>
<https://ibb.co/dhUGLy>

Something odd is going on with depth, but I don't understand what I'm doing wrong? Here is my fragment shader for my depth texture render pass:

```
#version 330 core
layout(location = 0) out vec4 FragColor;
layout(location = 1) out float DepthColor;
in vec2 TexCoord;

// texture samplers
uniform sampler2D texture0;
uniform sampler2D texture1;
uniform float near;
uniform float far;

float LinearizeDepth(float depth)
{
    float z = depth * 2.0 - 1.0; // back to NDC
    return (2.0 * near * far) / (far + near - z * (far - near));
}
void main()
{

    float depth = LinearizeDepth(gl_FragCoord.z);
    DepthColor = depth;
    FragColor = mix(texture(texture0, TexCoord), texture(texture1, TexCoord), 0.3);
}

```

and here is my shader for the final raymarch pass:

```
#version 330 core
out vec4 FragColor;

in vec2 TexCoords;

uniform sampler2D u_screen_texture;
uniform sampler2D u_depth_texture;
uniform float u_fov;
uniform vec3 u_rotation;
uniform vec3 u_origin;
uniform vec3 u_resolution;

const float inf = uintBitsToFloat(0x7F800000u);

vec3 rotVecX(in vec3 vector, in float theta){
    mat3 rotmat = mat3(1, 0, 0,
                       0, cos(theta), -sin(theta),
                       0, sin(theta), cos(theta));
    return vector * rotmat;
}

vec3 rotVecY(in vec3 vector, in float theta){
    mat3 rotmat = mat3(cos(theta), 0, sin(theta),
                       0, 1, 0,
                       -sin(theta), 0, cos(theta));
    return vector * rotmat;
}

vec3 calculateFragementRay(in vec2 fragCoord, in vec3 resolution){
    vec2 uv = fragCoord;
    uv.x = (uv.x * 2.0) - 1.0;
    uv.y = (2.0 * uv.y) - 1.0;
    if(resolution.x >= resolution.y){
        uv.x *= resolution.x/resolution.y;
    }else{
        uv.y *= resolution.y/resolution.x;
    }
    float tan_fov = tan(u_fov/2.0);
    vec2 pxy = uv * tan_fov;
    vec3 ray_dir = normalize(vec3(pxy, 1));
    return ray_dir;
}

vec3 getRayDirection(in vec2 fragCoord, in vec3 resolution, in vec3 rotation){
    // Normalized pixel coordinates (from -1 to 1)
    vec3 ray_dir = calculateFragementRay(fragCoord, resolution);
    ray_dir = rotVecX(ray_dir, rotation.x);
    ray_dir = rotVecY(ray_dir, rotation.y);

    return ray_dir;
}

struct Ray{
    vec3 origin;
    vec3 direction;
};

Ray getRay(in vec2 fragCoord, in vec3 resolution, in vec3 rotation, in vec3 origin){
    vec3 ray_dir = getRayDirection(fragCoord, resolution, rotation);
    vec3 ray_origin = origin;
    return Ray(ray_origin, ray_dir);
}

struct RayMarchSettings{
    int max_steps;
    float max_distance;
    float min_distance;
    float depth_weight;
};

struct Sphere{
    vec3 origin;
    float radius;
};

float signedDistance(vec3 p, in Sphere sphere){
    return (length(p - sphere.origin) - sphere.radius);
}

const int NULL_ID = -1;
const int SPHERE_ID = 0;
const int VSPHERE_ID = 1;

Sphere test_sphere = Sphere(vec3(0.0, 4.0, 4.0),1.0);
Sphere volume_sphere = Sphere(vec3(0.0, 1.0, 0.0),2.0);
float signedDistance(vec3 ray_endpoint, out int obj_id){
    float d0 = signedDistance(ray_endpoint, test_sphere);
    float d1 = signedDistance(ray_endpoint, volume_sphere);
    float minval = min(d0, d1);
    obj_id = minval == d0 ? SPHERE_ID : VSPHERE_ID;
    return minval;
}

bool rayMarch(Ray ray, in RayMarchSettings settings, in float rasterized_depth, out float out_depth, out int out_obj_id){
    float depth = 0.0;
    for( int i = 0; i < settings.max_steps; ++i){
        vec3 ray_endpoint = ray.origin + ray.direction * depth;
        int obj_id;
        float ray_distance = signedDistance(ray_endpoint, obj_id);
        depth += ray_distance * settings.depth_weight;

        if( depth > rasterized_depth){
            break;
        }
        if(ray_distance <= settings.min_distance){
            out_depth = depth;
            out_obj_id = obj_id;
            return true;
        }

        if(depth > settings.max_distance){
            break;
        }
    }
    out_depth = inf;
    out_obj_id = NULL_ID;
    return false;
}

float checkerBoardTexture(in vec2 uv){
    vec2 q = floor(uv);
    return clamp(mod( q.x+q.y, 2.0 ), 0.25, 1.0);
}
float fogify(in float dist){
    return  1.0/ (1.0 + dist*dist * 0.1);
}

RayMarchSettings rm_settings = RayMarchSettings(128, 100000.0, 0.0001, 0.5);

void main()
{

    vec3 origin = vec3(u_origin.x, u_origin.y, u_origin.z);
    origin = origin.zyx;
    Ray ray = getRay(TexCoords, u_resolution, u_rotation, origin);
    vec3 texture_color = texture(u_screen_texture, TexCoords).rgb;
    float rasterized_depth = texture(u_depth_texture, TexCoords).r;

    float depth = inf;
    int obj_id = NULL_ID;

    vec3 color;
    if(rayMarch(ray, rm_settings,rasterized_depth,  depth, obj_id)){
        vec3 endpoint = ray.origin + ray.direction * depth;
        float pattern_value = checkerBoardTexture(endpoint.xz * 4.0);

        if(obj_id == SPHERE_ID){
            color = vec3(1.0)*pattern_value;
        }
        if(obj_id == VSPHERE_ID){
            color = vec3(0.0,  1.0, 0.0)*pattern_value;
        }

        switch(obj_id){
            case SPHERE_ID:
            color = vec3(1.0)*pattern_value;
            break;
            case VSPHERE_ID:
            color = vec3(0.0,  1.0, 0.0)*pattern_value;
            break;
            default:
            color = vec3(1.0,0.0,1.0)*pattern_value;
        }
     }else{
        float re_rasterized_depth = rasterized_depth == inf ? 0.0 : rasterized_depth;
        color = texture_color +re_rasterized_depth/100.f;
     }
    FragColor = vec4(color, 1.0);
}

```

The `u_depth_texture` is where the depth comes from (and what `DepthColor` corresponds to) The `u_screen_texture` is just a texture of the rasterization pass information (output of `FragColor`).

# UPDATE:

So [this answer](https://computergraphics.stackexchange.com/a/7675/6530) outlined exactly my problem. I had to make the following changes:

```
vec3 getRotatedRayDirection(vec3 ray_dir, in vec3 rotation){
    // Normalized pixel coordinates (from -1 to 1)
    //vec3 ray_dir = calculateFragementRay(fragCoord, resolution);
    ray_dir = rotVecX(ray_dir, rotation.x);
    ray_dir = rotVecY(ray_dir, rotation.y);
    return ray_dir;
}

struct Ray{
    vec3 origin;
    vec3 direction;
    float cosA;
};

Ray getRay(in vec2 fragCoord, in vec3 resolution, in vec3 rotation, in vec3 origin){

    vec3 ray_dir = calculateFragementRay(fragCoord, resolution);
    vec3 rot_ray_dir = getRotatedRayDirection(ray_dir, rotation);
    vec3 ray_origin = origin;
    float cosA = ray_dir.z; // a.z/length(a), already normalized so length(a) == 1.0. 
    return Ray(ray_origin, rot_ray_dir, cosA);
}

```

Above I basically added cosA as a seperate value onto the ray, as suggested by the linked answer, with the appropriate code changes to avoid re-running `calculateFragmentRay`. However I needed to figure out which cosine corresponded to the one in thier picture, I knew it was a lateral dimension, it happened to be Z in this case, and I used the standard method of retrieving cosine from a normalized vector.

After that I did exactly what the linked answer said and made the following change:

```
if( (depth * ray.cosA) > rasterized_depth){
    break;
}

```

And here are the results! No depth artifacts!

[![enter image description here](https://i.sstatic.net/IptJC.gif)](https://i.sstatic.net/IptJC.gif)

[![enter image description here](https://i.sstatic.net/kIzIU.gif)](https://i.sstatic.net/kIzIU.gif)

**Tags:** opengl, rendering, glsl, rasterizer, raymarching

> **Comments:**
> **username_1**: Glad it worked :) By the way, you don't need to divide by length(rayDir) to get cosA - since you already normalized it in calculateFragmentRay.  
> **username_0**: @username_1 good catch, I'll edit it out  

---

## 1 answers

### Answer by username_1 on 2018-05-30. Score: 4 | Accepted answer

The problem, I think, is that you are comparing two different kinds of depth values. For want of better terms, the rasterizer's depth buffer stores 'parallel' depth, whereas raymarching calculates 'true' depth - that is, the rasterizer stores the distance to the plane parallel to the view plane which contains the point, but the raymarcher calculates the *actual* 3D distance from camera to point.
[![enter image description here](https://i.sstatic.net/ad4zF.png)](https://i.sstatic.net/ad4zF.png)

You can see from the picture, the relation should be $rasterDepth = rayDepth\*cos(A)$

So try this - in your `getRayDirection` function, *before* you do your view rotation, save your fragment ray's original normalized Z value somewhere, maybe an extra field in your Ray struct. Let's call it `cosA`.

Then in your `rayMarch` function the relevant code will be

```
if( depth * cosA > rasterized_depth){
        break;
}

```

Hopefully that should fix the problem.

> **Comments:**
> **username_0**: BINGO we got a winner!  This was exactly the problem,  I'll edit my question to show how I fixed this, I sort of had an intuition that it was going to be something like this, but there's no way I would have been able to figure this out any time soon with out your help!  

---

