<p>The problem, I think, is that you are comparing two different kinds of depth values. For want of better terms, the rasterizer's depth buffer stores 'parallel' depth, whereas raymarching calculates 'true' depth - that is, the rasterizer stores the distance to the plane parallel to the view plane which contains the point, but the raymarcher calculates the <em>actual</em> 3D distance from camera to point.
<a href="https://i.sstatic.net/ad4zF.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ad4zF.png" alt="enter image description here"></a></p>

<p>You can see from the picture, the relation should be $rasterDepth = rayDepth*cos(A)$</p>

<p>So try this - in your <code>getRayDirection</code> function, <em>before</em> you do your view rotation, save your fragment ray's original normalized Z value somewhere, maybe an extra field in your Ray struct. Let's call it <code>cosA</code>.</p>

<p>Then in your <code>rayMarch</code> function the relevant code will be</p>

<pre><code>if( depth * cosA &gt; rasterized_depth){
        break;
}
</code></pre>

<p>Hopefully that should fix the problem.</p>

CC BY-SA 4.0
2018-05-30T19:10:52.350Z
BINGO we got a winner!  This was exactly the problem,  I'll edit my question to show how I fixed this, I sort of had an intuition that it was going to be something like this, but there's no way I would have been able to figure this out any time soon with out your help!
Krupip
CC BY-SA 4.0
2018-05-30T18:22:04.123Z
2018-05-30T18:22:04.123Z
russ
<p>I'm trying to apply raymarching on top of 3D rasterization, ordering what 3D object gets displayed based on the depth of a rasterization pass and the raymarch depth.   After getting the transform right for <em>position</em> (<a href="https://www.scratchapixel.com/lessons/3d-basic-rendering/ray-tracing-generating-camera-rays/generating-camera-rays" rel="nofollow noreferrer">using the information here</a>) I was able to have the position of the raymarched spheres coherent with the rasterized cubes.   I ran into issues however with depth being different at the edges of the screen compared to the middle, despite position never changing.</p>
<p>These gifs should show what I'm talking about</p>
<p><a href="https://i.sstatic.net/rZakQ.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/rZakQ.gif" alt="enter image description here" /></a></p>
<p><a href="https://i.sstatic.net/4fD5Z.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/4fD5Z.gif" alt="enter image description here" /></a></p>
<p>higher res versions
<a href="https://ibb.co/ebji0y" rel="nofollow noreferrer">https://ibb.co/ebji0y</a>
<a href="https://ibb.co/dhUGLy" rel="nofollow noreferrer">https://ibb.co/dhUGLy</a></p>
<p>Something odd is going on with depth, but I don't understand what I'm doing wrong?  Here is my fragment shader for my depth texture render pass:</p>
<pre><code>#version 330 core
layout(location = 0) out vec4 FragColor;
layout(location = 1) out float DepthColor;
in vec2 TexCoord;

// texture samplers
uniform sampler2D texture0;
uniform sampler2D texture1;
uniform float near;
uniform float far;

float LinearizeDepth(float depth)
{
    float z = depth * 2.0 - 1.0; // back to NDC
    return (2.0 * near * far) / (far + near - z * (far - near));
}
void main()
{

    float depth = LinearizeDepth(gl_FragCoord.z);
    DepthColor = depth;
    FragColor = mix(texture(texture0, TexCoord), texture(texture1, TexCoord), 0.3);
}
</code></pre>
<p>and here is my shader for the final raymarch pass:</p>
<pre><code>#version 330 core
out vec4 FragColor;

in vec2 TexCoords;

uniform sampler2D u_screen_texture;
uniform sampler2D u_depth_texture;
uniform float u_fov;
uniform vec3 u_rotation;
uniform vec3 u_origin;
uniform vec3 u_resolution;

const float inf = uintBitsToFloat(0x7F800000u);

vec3 rotVecX(in vec3 vector, in float theta){
    mat3 rotmat = mat3(1, 0, 0,
                       0, cos(theta), -sin(theta),
                       0, sin(theta), cos(theta));
    return vector * rotmat;
}

vec3 rotVecY(in vec3 vector, in float theta){
    mat3 rotmat = mat3(cos(theta), 0, sin(theta),
                       0, 1, 0,
                       -sin(theta), 0, cos(theta));
    return vector * rotmat;
}

vec3 calculateFragementRay(in vec2 fragCoord, in vec3 resolution){
    vec2 uv = fragCoord;
    uv.x = (uv.x * 2.0) - 1.0;
    uv.y = (2.0 * uv.y) - 1.0;
    if(resolution.x &gt;= resolution.y){
        uv.x *= resolution.x/resolution.y;
    }else{
        uv.y *= resolution.y/resolution.x;
    }
    float tan_fov = tan(u_fov/2.0);
    vec2 pxy = uv * tan_fov;
    vec3 ray_dir = normalize(vec3(pxy, 1));
    return ray_dir;
}

vec3 getRayDirection(in vec2 fragCoord, in vec3 resolution, in vec3 rotation){
    // Normalized pixel coordinates (from -1 to 1)
    vec3 ray_dir = calculateFragementRay(fragCoord, resolution);
    ray_dir = rotVecX(ray_dir, rotation.x);
    ray_dir = rotVecY(ray_dir, rotation.y);

    return ray_dir;
}

struct Ray{
    vec3 origin;
    vec3 direction;
};

Ray getRay(in vec2 fragCoord, in vec3 resolution, in vec3 rotation, in vec3 origin){
    vec3 ray_dir = getRayDirection(fragCoord, resolution, rotation);
    vec3 ray_origin = origin;
    return Ray(ray_origin, ray_dir);
}

struct RayMarchSettings{
    int max_steps;
    float max_distance;
    float min_distance;
    float depth_weight;
};

struct Sphere{
    vec3 origin;
    float radius;
};

float signedDistance(vec3 p, in Sphere sphere){
    return (length(p - sphere.origin) - sphere.radius);
}

const int NULL_ID = -1;
const int SPHERE_ID = 0;
const int VSPHERE_ID = 1;

Sphere test_sphere = Sphere(vec3(0.0, 4.0, 4.0),1.0);
Sphere volume_sphere = Sphere(vec3(0.0, 1.0, 0.0),2.0);
float signedDistance(vec3 ray_endpoint, out int obj_id){
    float d0 = signedDistance(ray_endpoint, test_sphere);
    float d1 = signedDistance(ray_endpoint, volume_sphere);
    float minval = min(d0, d1);
    obj_id = minval == d0 ? SPHERE_ID : VSPHERE_ID;
    return minval;
}

bool rayMarch(Ray ray, in RayMarchSettings settings, in float rasterized_depth, out float out_depth, out int out_obj_id){
    float depth = 0.0;
    for( int i = 0; i &lt; settings.max_steps; ++i){
        vec3 ray_endpoint = ray.origin + ray.direction * depth;
        int obj_id;
        float ray_distance = signedDistance(ray_endpoint, obj_id);
        depth += ray_distance * settings.depth_weight;

        if( depth &gt; rasterized_depth){
            break;
        }
        if(ray_distance &lt;= settings.min_distance){
            out_depth = depth;
            out_obj_id = obj_id;
            return true;
        }

        if(depth &gt; settings.max_distance){
            break;
        }
    }
    out_depth = inf;
    out_obj_id = NULL_ID;
    return false;
}

float checkerBoardTexture(in vec2 uv){
    vec2 q = floor(uv);
    return clamp(mod( q.x+q.y, 2.0 ), 0.25, 1.0);
}
float fogify(in float dist){
    return  1.0/ (1.0 + dist*dist * 0.1);
}

RayMarchSettings rm_settings = RayMarchSettings(128, 100000.0, 0.0001, 0.5);

void main()
{

    vec3 origin = vec3(u_origin.x, u_origin.y, u_origin.z);
    origin = origin.zyx;
    Ray ray = getRay(TexCoords, u_resolution, u_rotation, origin);
    vec3 texture_color = texture(u_screen_texture, TexCoords).rgb;
    float rasterized_depth = texture(u_depth_texture, TexCoords).r;

    float depth = inf;
    int obj_id = NULL_ID;


    vec3 color;
    if(rayMarch(ray, rm_settings,rasterized_depth,  depth, obj_id)){
        vec3 endpoint = ray.origin + ray.direction * depth;
        float pattern_value = checkerBoardTexture(endpoint.xz * 4.0);

        if(obj_id == SPHERE_ID){
            color = vec3(1.0)*pattern_value;
        }
        if(obj_id == VSPHERE_ID){
            color = vec3(0.0,  1.0, 0.0)*pattern_value;
        }

        switch(obj_id){
            case SPHERE_ID:
            color = vec3(1.0)*pattern_value;
            break;
            case VSPHERE_ID:
            color = vec3(0.0,  1.0, 0.0)*pattern_value;
            break;
            default:
            color = vec3(1.0,0.0,1.0)*pattern_value;
        }
     }else{
        float re_rasterized_depth = rasterized_depth == inf ? 0.0 : rasterized_depth;
        color = texture_color +re_rasterized_depth/100.f;
     }
    FragColor = vec4(color, 1.0);
}
</code></pre>
<p>The <code>u_depth_texture</code> is where the depth comes from (and what <code>DepthColor</code> corresponds to)  The <code>u_screen_texture</code> is just a texture of the rasterization pass information (output of <code>FragColor</code>).</p>
<h1>UPDATE:</h1>
<p>So <a href="https://computergraphics.stackexchange.com/a/7675/6530">this answer</a> outlined exactly my problem.  I had to make the following changes:</p>
<pre><code>vec3 getRotatedRayDirection(vec3 ray_dir, in vec3 rotation){
    // Normalized pixel coordinates (from -1 to 1)
    //vec3 ray_dir = calculateFragementRay(fragCoord, resolution);
    ray_dir = rotVecX(ray_dir, rotation.x);
    ray_dir = rotVecY(ray_dir, rotation.y);
    return ray_dir;
}

struct Ray{
    vec3 origin;
    vec3 direction;
    float cosA;
};

Ray getRay(in vec2 fragCoord, in vec3 resolution, in vec3 rotation, in vec3 origin){

    vec3 ray_dir = calculateFragementRay(fragCoord, resolution);
    vec3 rot_ray_dir = getRotatedRayDirection(ray_dir, rotation);
    vec3 ray_origin = origin;
    float cosA = ray_dir.z; // a.z/length(a), already normalized so length(a) == 1.0. 
    return Ray(ray_origin, rot_ray_dir, cosA);
}
</code></pre>
<p>Above I basically added cosA as a seperate value onto the ray, as suggested by the linked answer, with the appropriate code changes to avoid re-running <code>calculateFragmentRay</code>.  However I needed to figure out which cosine corresponded to the one in thier picture, I knew it was a lateral dimension, it happened to be Z in this case, and I used the standard method of retrieving cosine from a normalized vector.</p>
<p>After that I did exactly what the linked answer said and made the following change:</p>
<pre><code>if( (depth * ray.cosA) &gt; rasterized_depth){
    break;
}
</code></pre>
<p>And here are the results! No depth artifacts!</p>
<p><a href="https://i.sstatic.net/IptJC.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/IptJC.gif" alt="enter image description here" /></a></p>
<p><a href="https://i.sstatic.net/kIzIU.gif" rel="nofollow noreferrer"><img src="https://i.sstatic.net/kIzIU.gif" alt="enter image description here" /></a></p>

CC BY-SA 4.0
2018-05-31T04:02:55.990Z
Glad it worked :) By the way, you don't need to divide by length(rayDir) to get cosA - since you already normalized it in calculateFragmentRay.
russ
CC BY-SA 4.0
2018-05-31T04:20:26.743Z
@russ good catch, I'll edit it out
Krupip
<opengl><rendering><glsl><rasterizer><raymarching>
How to align Ray Marching on top of traditional 3D rasterization?
Krupip
# How to align Ray Marching on top of traditional 3D rasterization?

(Asked by: username_0 on 2018-05-30)

I'm trying to apply raymarching on top of 3D rasterization, ordering what 3D object gets displayed based on the depth of a rasterization pass and the raymarch depth.   After getting the transform right for position (using the information here) I was able to have the position of the raymarched spheres coherent with the rasterized cubes.   I ran into issues however with depth being different at the edges of the screen compared to the middle, despite position never changing.
These gifs should show what I'm talking about


higher res versions
https://ibb.co/ebji0y
https://ibb.co/dhUGLy
Something odd is going on with depth, but I don't understand what I'm doing wrong?  Here is my fragment shader for my depth texture render pass:
#version 330 core
layout(location = 0) out vec4 FragColor;
layout(location = 1) out float DepthColor;
in vec2 TexCoord;

// texture samplers
uniform sampler2D texture0;
uniform sampler2D texture1;
uniform float near;
uniform float far;

float LinearizeDepth(float depth)
{
    float z = depth * 2.0 - 1.0; // back to NDC
    return (2.0 * near * far) / (far + near - z * (far - near));
}
void main()
{

    float depth = LinearizeDepth(gl_FragCoord.z);
    DepthColor = depth;
    FragColor = mix(texture(texture0, TexCoord), texture(texture1, TexCoord), 0.3);
}

and here is my shader for the final raymarch pass:
#version 330 core
out vec4 FragColor;

in vec2 TexCoords;

uniform sampler2D u_screen_texture;
uniform sampler2D u_depth_texture;
uniform float u_fov;
uniform vec3 u_rotation;
uniform vec3 u_origin;
uniform vec3 u_resolution;

const float inf = uintBitsToFloat(0x7F800000u);

vec3 rotVecX(in vec3 vector, in float theta){
    mat3 rotmat = mat3(1, 0, 0,
                       0, cos(theta), -sin(theta),
                       0, sin(theta), cos(theta));
    return vector * rotmat;
}

vec3 rotVecY(in vec3 vector, in float theta){
    mat3 rotmat = mat3(cos(theta), 0, sin(theta),
                       0, 1, 0,
                       -sin(theta), 0, cos(theta));
    return vector * rotmat;
}

vec3 calculateFragementRay(in vec2 fragCoord, in vec3 resolution){
    vec2 uv = fragCoord;
    uv.x = (uv.x * 2.0) - 1.0;
    uv.y = (2.0 * uv.y) - 1.0;
    if(resolution.x >= resolution.y){
        uv.x *= resolution.x/resolution.y;
    }else{
        uv.y *= resolution.y/resolution.x;
    }
    float tan_fov = tan(u_fov/2.0);
    vec2 pxy = uv * tan_fov;
    vec3 ray_dir = normalize(vec3(pxy, 1));
    return ray_dir;
}

vec3 getRayDirection(in vec2 fragCoord, in vec3 resolution, in vec3 rotation){
    // Normalized pixel coordinates (from -1 to 1)
    vec3 ray_dir = calculateFragementRay(fragCoord, resolution);
    ray_dir = rotVecX(ray_dir, rotation.x);
    ray_dir = rotVecY(ray_dir, rotation.y);

    return ray_dir;
}

struct Ray{
    vec3 origin;
    vec3 direction;
};

Ray getRay(in vec2 fragCoord, in vec3 resolution, in vec3 rotation, in vec3 origin){
    vec3 ray_dir = getRayDirection(fragCoord, resolution, rotation);
    vec3 ray_origin = origin;
    return Ray(ray_origin, ray_dir);
}

struct RayMarchSettings{
    int max_steps;
    float max_distance;
    float min_distance;
    float depth_weight;
};

struct Sphere{
    vec3 origin;
    float radius;
};

float signedDistance(vec3 p, in Sphere sphere){
    return (length(p - sphere.origin) - sphere.radius);
}

const int NULL_ID = -1;
const int SPHERE_ID = 0;
const int VSPHERE_ID = 1;

Sphere test_sphere = Sphere(vec3(0.0, 4.0, 4.0),1.0);
Sphere volume_sphere = Sphere(vec3(0.0, 1.0, 0.0),2.0);
float signedDistance(vec3 ray_endpoint, out int obj_id){
    float d0 = signedDistance(ray_endpoint, test_sphere);
    float d1 = signedDistance(ray_endpoint, volume_sphere);
    float minval = min(d0, d1);
    obj_id = minval == d0 ? SPHERE_ID : VSPHERE_ID;
    return minval;
}

bool rayMarch(Ray ray, in RayMarchSettings settings, in float rasterized_depth, out float out_depth, out int out_obj_id){
    float depth = 0.0;
    for( int i = 0; i < settings.max_steps; ++i){
        vec3 ray_endpoint = ray.origin + ray.direction * depth;
        int obj_id;
        float ray_distance = signedDistance(ray_endpoint, obj_id);
        depth += ray_distance * settings.depth_weight;

        if( depth > rasterized_depth){
            break;
        }
        if(ray_distance <= settings.min_distance){
            out_depth = depth;
            out_obj_id = obj_id;
            return true;
        }

        if(depth > settings.max_distance){
            break;
        }
    }
    out_depth = inf;
    out_obj_id = NULL_ID;
    return false;
}

float checkerBoardTexture(in vec2 uv){
    vec2 q = floor(uv);
    return clamp(mod( q.x+q.y, 2.0 ), 0.25, 1.0);
}
float fogify(in float dist){
    return  1.0/ (1.0 + dist*dist * 0.1);
}

RayMarchSettings rm_settings = RayMarchSettings(128, 100000.0, 0.0001, 0.5);

void main()
{

    vec3 origin = vec3(u_origin.x, u_origin.y, u_origin.z);
    origin = origin.zyx;
    Ray ray = getRay(TexCoords, u_resolution, u_rotation, origin);
    vec3 texture_color = texture(u_screen_texture, TexCoords).rgb;
    float rasterized_depth = texture(u_depth_texture, TexCoords).r;

    float depth = inf;
    int obj_id = NULL_ID;


    vec3 color;
    if(rayMarch(ray, rm_settings,rasterized_depth,  depth, obj_id)){
        vec3 endpoint = ray.origin + ray.direction * depth;
        float pattern_value = checkerBoardTexture(endpoint.xz * 4.0);

        if(obj_id == SPHERE_ID){
            color = vec3(1.0)*pattern_value;
        }
        if(obj_id == VSPHERE_ID){
            color = vec3(0.0,  1.0, 0.0)*pattern_value;
        }

        switch(obj_id){
            case SPHERE_ID:
            color = vec3(1.0)*pattern_value;
            break;
            case VSPHERE_ID:
            color = vec3(0.0,  1.0, 0.0)*pattern_value;
            break;
            default:
            color = vec3(1.0,0.0,1.0)*pattern_value;
        }
     }else{
        float re_rasterized_depth = rasterized_depth == inf ? 0.0 : rasterized_depth;
        color = texture_color +re_rasterized_depth/100.f;
     }
    FragColor = vec4(color, 1.0);
}

The u_depth_texture is where the depth comes from (and what DepthColor corresponds to)  The u_screen_texture is just a texture of the rasterization pass information (output of FragColor).
UPDATE:
So this answer outlined exactly my problem.  I had to make the following changes:
vec3 getRotatedRayDirection(vec3 ray_dir, in vec3 rotation){
    // Normalized pixel coordinates (from -1 to 1)
    //vec3 ray_dir = calculateFragementRay(fragCoord, resolution);
    ray_dir = rotVecX(ray_dir, rotation.x);
    ray_dir = rotVecY(ray_dir, rotation.y);
    return ray_dir;
}

struct Ray{
    vec3 origin;
    vec3 direction;
    float cosA;
};

Ray getRay(in vec2 fragCoord, in vec3 resolution, in vec3 rotation, in vec3 origin){

    vec3 ray_dir = calculateFragementRay(fragCoord, resolution);
    vec3 rot_ray_dir = getRotatedRayDirection(ray_dir, rotation);
    vec3 ray_origin = origin;
    float cosA = ray_dir.z; // a.z/length(a), already normalized so length(a) == 1.0. 
    return Ray(ray_origin, rot_ray_dir, cosA);
}

Above I basically added cosA as a seperate value onto the ray, as suggested by the linked answer, with the appropriate code changes to avoid re-running calculateFragmentRay.  However I needed to figure out which cosine corresponded to the one in thier picture, I knew it was a lateral dimension, it happened to be Z in this case, and I used the standard method of retrieving cosine from a normalized vector.
After that I did exactly what the linked answer said and made the following change:
if( (depth * ray.cosA) > rasterized_depth){
    break;
}

And here are the results! No depth artifacts!

**Tags:** opengl, rendering, glsl, rasterizer, raymarching
> **Comments:**
> **username_1**: Glad it worked :) By the way, you don't need to divide by length(rayDir) to get cosA - since you already normalized it in calculateFragmentRay.  
> **username_0**: @russ good catch, I'll edit it out  

---

## 1 answers

### Answer by username_1 on 2018-05-30. Score: 4 | Accepted answer

The problem, I think, is that you are comparing two different kinds of depth values. For want of better terms, the rasterizer's depth buffer stores 'parallel' depth, whereas raymarching calculates 'true' depth - that is, the rasterizer stores the distance to the plane parallel to the view plane which contains the point, but the raymarcher calculates the actual 3D distance from camera to point.

You can see from the picture, the relation should be $rasterDepth = rayDepth*cos(A)$
So try this - in your getRayDirection function, before you do your view rotation, save your fragment ray's original normalized Z value somewhere, maybe an extra field in your Ray struct. Let's call it cosA.
Then in your rayMarch function the relevant code will be
if( depth * cosA > rasterized_depth){
        break;
}

Hopefully that should fix the problem.

> **Comments:**
> **username_0**: BINGO we got a winner!  This was exactly the problem,  I'll edit my question to show how I fixed this, I sort of had an intuition that it was going to be something like this, but there's no way I would have been able to figure this out any time soon with out your help!  

---

