<p>I've done something similar. The way I did it was by using a height map image to change the positions of vertices in a grid in a vertex shader. I generated a 2D grid of vertices that were evenly spaced on the CPU. I uploaded the vertices and a texture and in the vertex shader, I sampled the texture and used the resulting brightness as the height. So I'd keep the x and y positions of each vertex and change the z based on the image.</p>

<p>Now there are other ways you could utilize this to get something more like in the video. You could either have a 2D grid of single vertices, or even a grid of small 3D models. Change the z position based on the color of a height map. But additionally, you could upload a small noise texture and vary the x and y positions by a small amount based on the noise, which might vary per frame. I would see if you could vary the x and y by a large enough amount to see, but small enough that no point ever crossed over a neighbor point. That would get you some more movement and randomness.</p>

CC BY-SA 3.0
2017-11-30T05:53:25.160Z
OK I'll give your suggestions a try, but the BTLNews article [I've linked to in the question](http://www.btlnews.com/crafts/visual-fx/level-256-vfx-produces-the-final-moments-of-karl-brant-for-nerdist-com/) explains that they've started with a 3D representation or possibly a point cloud from a hacked Kinect box. So they start with an actual heigh of the surface, rather than generate one from a color or brightness. Possibly they re-interpolate it to a regular x-y grid of points with continuous z (z being "out of the page").
uhoh
CC BY-SA 3.0
2017-11-17T00:24:38.170Z
2017-11-17T00:24:38.170Z
user1118321
<p>I don't know the name of this technique, but the idea is to adjust the density and location of identical dots or small objects in order to produce a gray tone image or even a suggestion of three dimensionality. It's better to use these two images below as examples.</p>

<p>I'd like to do something like this on my own, but by writing relatively simple algorithm rather than a program or package. I'll be starting with a 3D representation of an object's surface inside a python script running in  Blender. <em>My question isn't how to implement, but how to understand what's happening</em>.</p>

<p>Could someone explain what this is called and share some amount of insight into how this works? I don't need a full-blown derivation, just some idea to get started on something simple.</p>

<p><strong>In the screenshot</strong>, the objects (in software) follow the face contour in 3D, but their position (and brightness) is fluid and their separation varies continuously. While they appear to be approximately on a grid, I'm not sure if they are a fixed projected grid (I think I've read somewhere it was done using a Kinect) and move because of the movement of the model, or if they move algorithmically as well.</p>

<p><strong>edit:</strong> More info on the Kinect in the Below the Line article <a href="http://www.btlnews.com/crafts/visual-fx/level-256-vfx-produces-the-final-moments-of-karl-brant-for-nerdist-com/" rel="nofollow noreferrer">Level 256 VFX Produces The Final Moments of Karl Brant for Nerdist.com</a>.</p>

<blockquote>
  <p>“Leveraging the power of open source applications like Processing, Python, and bash scripting, we were able to take the point cloud data from the Kinect, manipulate it, and get it into Nuke for integration with the live-action plates for texture re-projection,” Kleinberg explained. “The process evolved and grew over time, and when we finally saw the fruits of our labor on the screen, we were excited to see Karl Brant fully realized, taking on an expressive life of his own.”</p>
</blockquote>

<p>I'm familliar with something simple like using a random number to decide if the pixel should be on or off depending on the gray value; if it's 0.3, then if a random number (chosen for each pixel, but not necessarily randomized every frame) is below 0.3, the pixel should be ON (seems to be called 1-bit <a href="https://en.wikipedia.org/wiki/Dither" rel="nofollow noreferrer">dithering</a>). But I think there is a lot more going on here than that. </p>

<p><strong>below:</strong> Cropped screen shot at <code>11:23</code> in the short film <a href="https://www.youtube.com/watch?v=jFotiDbj-D4" rel="nofollow noreferrer"><em>The Final Moments of Karl Brandt</em></a>:</p>

<p><a href="https://i.sstatic.net/FBas6.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/FBas6.jpg" alt="enter image description here"></a></p>

CC BY-SA 3.0
2017-11-16T13:52:59.227Z
Need some help with the tags.
uhoh
CC BY-SA 3.0
2017-11-16T14:25:16.117Z
The two images you have show completely unrelated techniques. The first is placing 3D objects on a surface. The second just downsamples a 2D image to 1-bit using [dithering](https://en.wikipedia.org/wiki/Dither). Which are you interested in?
Dan Hulme
CC BY-SA 3.0
2017-11-16T14:27:14.963Z
@DanHulme ya OK I see what you mean - there is no effort to show 3D in the 2nd one, it's just coming from shading (original lighting) of the model. I've removed it.
uhoh
CC BY-SA 3.0
2017-11-17T00:24:22.590Z
The kinect can produce depth images for you. The depth image essentially gives you a set of points in 3d space. A point cloud much like what we see above. You'll have to downsample, the resolution will be much too high. Then to get the shade of them (steeper areas should have darker colors) you can take the gradient of the image for your normals. Not sure how his lights are positioned or what kind of lights he's using exactly though. There does seem to be some sort of blur going on or maybe that's the fog.
Andrew Wilson
<rendering><pixel-graphics>
What is this technique called and how can I reproduce it algorithmically?
uhoh
# What is this technique called and how can I reproduce it algorithmically?

(Asked by: username_0 on 2017-11-16)

I don't know the name of this technique, but the idea is to adjust the density and location of identical dots or small objects in order to produce a gray tone image or even a suggestion of three dimensionality. It's better to use these two images below as examples.

I'd like to do something like this on my own, but by writing relatively simple algorithm rather than a program or package. I'll be starting with a 3D representation of an object's surface inside a python script running in Blender. *My question isn't how to implement, but how to understand what's happening*.

Could someone explain what this is called and share some amount of insight into how this works? I don't need a full-blown derivation, just some idea to get started on something simple.

**In the screenshot**, the objects (in software) follow the face contour in 3D, but their position (and brightness) is fluid and their separation varies continuously. While they appear to be approximately on a grid, I'm not sure if they are a fixed projected grid (I think I've read somewhere it was done using a Kinect) and move because of the movement of the model, or if they move algorithmically as well.

**edit:** More info on the Kinect in the Below the Line article [Level 256 VFX Produces The Final Moments of Karl Brant for Nerdist.com](http://www.btlnews.com/crafts/visual-fx/level-256-vfx-produces-the-final-moments-of-karl-brant-for-nerdist-com/).

> “Leveraging the power of open source applications like Processing, Python, and bash scripting, we were able to take the point cloud data from the Kinect, manipulate it, and get it into Nuke for integration with the live-action plates for texture re-projection,” Kleinberg explained. “The process evolved and grew over time, and when we finally saw the fruits of our labor on the screen, we were excited to see Karl Brant fully realized, taking on an expressive life of his own.”

I'm familliar with something simple like using a random number to decide if the pixel should be on or off depending on the gray value; if it's 0.3, then if a random number (chosen for each pixel, but not necessarily randomized every frame) is below 0.3, the pixel should be ON (seems to be called 1-bit [dithering](https://en.wikipedia.org/wiki/Dither)). But I think there is a lot more going on here than that.

**below:** Cropped screen shot at `11:23` in the short film [*The Final Moments of Karl Brandt*](https://www.youtube.com/watch?v=jFotiDbj-D4):

[![enter image description here](https://i.sstatic.net/FBas6.jpg)](https://i.sstatic.net/FBas6.jpg)

**Tags:** rendering, pixel-graphics

> **Comments:**
> **username_0**: Need some help with the tags.  
> **username_1**: The two images you have show completely unrelated techniques. The first is placing 3D objects on a surface. The second just downsamples a 2D image to 1-bit using [dithering](https://en.wikipedia.org/wiki/Dither). Which are you interested in?  
> **username_0**: @username_1 ya OK I see what you mean - there is no effort to show 3D in the 2nd one, it's just coming from shading (original lighting) of the model. I've removed it.  
> **username_2**: The kinect can produce depth images for you. The depth image essentially gives you a set of points in 3d space. A point cloud much like what we see above. You'll have to downsample, the resolution will be much too high. Then to get the shade of them (steeper areas should have darker colors) you can take the gradient of the image for your normals. Not sure how his lights are positioned or what kind of lights he's using exactly though. There does seem to be some sort of blur going on or maybe that's the fog.  

---

## 1 answers

### Answer by username_3 on 2017-11-17. Score: 2

I've done something similar. The way I did it was by using a height map image to change the positions of vertices in a grid in a vertex shader. I generated a 2D grid of vertices that were evenly spaced on the CPU. I uploaded the vertices and a texture and in the vertex shader, I sampled the texture and used the resulting brightness as the height. So I'd keep the x and y positions of each vertex and change the z based on the image.

Now there are other ways you could utilize this to get something more like in the video. You could either have a 2D grid of single vertices, or even a grid of small 3D models. Change the z position based on the color of a height map. But additionally, you could upload a small noise texture and vary the x and y positions by a small amount based on the noise, which might vary per frame. I would see if you could vary the x and y by a large enough amount to see, but small enough that no point ever crossed over a neighbor point. That would get you some more movement and randomness.

> **Comments:**
> **username_0**: OK I'll give your suggestions a try, but the BTLNews article [I've linked to in the question](http://www.btlnews.com/crafts/visual-fx/level-256-vfx-produces-the-final-moments-of-karl-brant-for-nerdist-com/) explains that they've started with a 3D representation or possibly a point cloud from a hacked Kinect box. So they start with an actual heigh of the surface, rather than generate one from a color or brightness. Possibly they re-interpolate it to a regular x-y grid of points with continuous z (z being "out of the page").  

---

