The most interesting thing about what the Lytro Illum can do that has an advantage over traditional photography, is that you can play with the depth in the scene. This feature has the effect of allowing the viewer to feel more immersed in the scene by having the added 3D information of experiencing the illusion of depth off a 2D screen. For the most part, the Illum's software is able to detect the depth of objects in the scene correctly. However, there are a few conditions that confuse the processing which will result in some artifacts that will need to be manually corrected by pulling the depth map into Photoshop (or any other image editing software.) In this post, I will discuss some challenging situations that the software has trouble with and cover how to correct the depth maps for processing.
Note: In the images shown below, areas shaded in blue are what the software is choosing as the near depth and the orange tinted areas are what is being interpreted as the far depth.
Texture & Patterns
As you can see from the depth map above, the software had a hard time choosing the correct depths for this busy scene. The part of the wall that had been stuccoed over is so much smoother and brighter than the surrounding that this area became processed as being a different depth than the bricks.
Contrast: Low & High
Since the texture and tone of the stalactites are very similar to each other, it had trouble processing the correct depth in the upper left corner of the image for part of the far stalactite that borders the near one. You can also see that the edge detection for this object type is a bit poor since the tip and edges of the formations are being mapped as part of the far depth. The bottom right corner of the image is also not creating the correct depth map either but for different reasons. If there are parts of the image that lack detail and either reach near pure black or pure white, the software is unable to estimate the depth correctly. This area will become a depth artifact when processing and will either appear to be part of your foreground or background.
At the moment there does not appear to be a contestant rule for which the software chooses. In the stalactite image, it incorrectly made part of the blackness of the cave a near depth. While in contrast the image to the right, the software chose to make the mossy rock part of the far depth.
The software also has trouble with areas that are near white. The image below is a good example, as the bright parts of the clouds are being interpreted as being near.
Fixing the Depth Map
For users that have image editing software, Lytro has enabled a feature that allows them to easily correct the depth map.
- After selecting the image that you would like to fix, go to File>Export and select Depth Map from the drop down menu. Select where you would like your file to be exported, and click Export. You can then open the files in an image editor that allows you to manually paint the correct depth tone, like Photoshop.
- Open both of the files that are created in your image editor and use the color image as a layer on top of the gray scale depth image. Photoshop may pop-up with an warning saying "The target document has a different depth than the source document. This may result in lower than expected quality. Are you sure you want to proceed?" Just ignore this and click Yes as you are just using the image as a reference layer so you can better see what you are editing.
- In your Layers pallet, drag the opacity of your reference layer down to about 20% or to a percent where you can see both layers. For the depth map, the lighter tone represents the far depth and darker tone represents the near depth.
- Select the Depth Map (Background layer) and the Eye dropper tool. Set the Eye dropper tool to only sample the Current Layer from the drop down menu. Use the Eye dropper to select an area with the same depth of the area that you would like to fix.
- Now switch to the Paint Brush to correct the desired area. You can resample depth area colors by holding Alt/Option to temporarily switch back and forth between the Eye dropper and Paint brush. If you have large areas that need to be corrected, it can be easier to use the selection tools on the Reference layer image and then switch back to the Depth layer to paint in the desired area more precisely. For this, you will want to turn the opacity of the Reference layer image back up to 100% temporarily while making the selection.
- Once you have addressed all problem areas, you can then delete the Reference image layer and save.
- You may now go back to the Lytro Desktop app. Navigate to File>Import. Under the Import drop down menu choose Selected Depth Maps, click Browse to find the file you would like to update, and then click Import. After that it will just take a couple minutes to update the processing on your lightfield.