<p>When doing a filter operation such as a blur, in many cases the filter itself will be normalized so its values sum to 1.0, precisely to avoid this problem.</p>

<p>In cases where out-of-range outputs do need to be handled, the most common way is what you called capping. In graphics it's more commonly known as clamping or saturating. This is the default behavior if, for instance, you run a pixel shader on a GPU and output out-of-range values.</p>

<p>Renormalizing the output image might be necessary in some special cases depending on what you're doing. I can't think of anytime that I've seen the modulo behavior used—it would be a pretty weird operation to do on a visual image.</p>

CC BY-SA 3.0
2018-02-10T04:36:49.960Z
Thanks for this. I was actually working with some images converted to numpy matrices which are uint8 and it automatically uses modulo when you exceed the unsigned byte length so that's why I thought maybe modulo was used in graphics somewhere. Can I ask for clarification on the clamping? So is it common to add/multiply images together? If so, in my samples working with grayscale images it would seem like the final image would just instantly oversaturate and "white-out" so to speak. Is this normal and do we still just clamp the values?
zoombini
CC BY-SA 3.0
2018-02-10T18:56:30.607Z
Yes, it's pretty common, though we usually think of images as representing fractional values in [0, 1] rather than integers 0 to 255. Alpha blending, for instance, involves adding and multiplying. Depending on the images and the operations you're doing, saturating to white may indeed happen, but in other cases (like alpha blending layers of images on top of each other) it's not really an issue. Also, if you work with HDR images, you can leave them unclamped while you do a bunch of operations on them, then apply a [tone mapping](https://en.wikipedia.org/wiki/Tone_mapping) function at the end.
Nathan Reed
CC BY-SA 3.0
2018-02-10T02:26:32.000Z
2018-02-10T02:26:32.000Z
Nathan Reed
<p>While clamping or saturating may be the most common way of handling this situation, it's usually not the best. (But wrapping/modulus is one of the worst.) Another way of handling it is to either convert the input to 16 or 32-bit ints or floats and do all calculations in the higher bit depth. You may want to down-convert the final output, if required by your processing tools. Or, if you can choose the output format you can leave it in a higher bit depth. Most professional tools these days do their work in higher bit depths.</p>

<p>Regarding your question:</p>

<blockquote>
  <p>is it common to add/multiply images together?</p>
</blockquote>

<p>Yes, it's quite common in tools that allow compositing. In Photoshop, for example, there are blend modes that are even named "Add" and "Multiply" and they do just that. (Though they assume the values are normalized to a 0-1 range.) If you are working with 8-bit per channel images, they will clamp, but if you are working with 32-bit per channel images, they will simply have values greater than 1. If done with linear RGB, this mimics light fairly naturally, and you can apply natural photographic operations to the results, like decreasing the exposure and it will bring those blown-out highlights back into the displayable range. As we move into the era of High Dynamic Range output devices, this type of processing will only become more common and useful.</p>

CC BY-SA 3.0
2018-02-10T15:41:30.773Z
Thanks! Normalization into 0-1 floating point values makes sense to me because then multiplying won't cause the threshold to be exceeded. What would you do in the case of adding/subtracting images together? Would you clamp the values between 0-1 after the arithmetic?
zoombini
CC BY-SA 3.0
2018-02-10T15:43:40.317Z
Yes, once you've completed all the operations you need to on the image, only then do you clamp it. So if you're adding 2 images, then blurring the result, or scaling it, or whatever, you do all of those things at the higher bit depth and then clamp the final output (if needed).
user1118321
CC BY-SA 3.0
2018-02-10T15:53:01.867Z
I guess this doesn't work for grayscale, right? Let's say I have a standard 8-bit RGB image, 8-bits for each channel.

Image 1 is a solid red color image where every pixel is [200, 0, 0].
Image 2 is a solid dark red color image where every pixel is [100, 0, 0].

When I up-convert to a higher bit depth, do I literally just use 16-bits for each channel, so my final image would be an image where every pixel is now [300, 0, 0]? Now my choices are (A) Leave it at this 16-bit per channel depth or (B) Convert back to 8-bit in which case the image will clamp back to [255, 0, 0]. Is that right?
zoombini
CC BY-SA 3.0
2018-02-10T15:59:25.987Z
This absolutely works for grayscale. It also works for other color spaces like HSV, YCbCr, La*b*, etc. And yes, you have the right idea about what I'm suggesting.
user1118321
CC BY-SA 3.0
2018-02-10T06:40:20.057Z
2018-02-10T06:40:20.057Z
user1118321
<p>I'm currently taking a graphics course and we've recently covered cross-correlation with regards to using a filter matrix that is applied to a region of pixels in a sort of continual raster-type scan across a larger input image to produce some output image.</p>

<p>The formula is denoted here:
<a href="https://i.sstatic.net/ZTbzk.png" rel="noreferrer"><img src="https://i.sstatic.net/ZTbzk.png" alt=""></a></p>

<p><em>G is the output image, h is the kernel/box filter matrix, F is the input image</em></p>

<p>The classic one that was introduced is a 3x3 box filter for blurring an input image. The box filter is a 3x3 matrix with each cell value being equally set to 1/9. We've been dealing primarily with grayscale images with values 0 (black) to 255 (white).</p>

<p>In this case, the largest possible output pixel value would be if the 3x3 input image area contained all white pixels in which case we'd have:</p>

<p><code>[(1/9) * 255 ] * 9 = 255</code></p>

<p>This is fine, but what if our kernel filter matrix instead had 9 cell values all equal to 0.5?</p>

<p>Then the largest potential output value would be:</p>

<p><code>[0.5 * 255] * 9 = 1147.5</code></p>

<p>Once we've completely finished producing our new output image matrix, how do we handle values above 255 and below 0?</p>

<p>It seems like there are three possible approaches:</p>

<h2>Capping</h2>

<p>Cap all output values to 0 and 255. Any value below 0 is capped to 0, any value above 255 is capped to 255.</p>

<h2>Modulo</h2>

<p>Use modulo. In this case it would be 1147.5 % 255 = 125.5 rounded to 126.</p>

<h2>Scaling</h2>

<p>Take the highest output value and use it to scale all the values proportionally. If our example output image pixel values were 1147.5, 100, and 255, then we would perform the following operations:</p>

<ul>
<li><code>(1147.5 / 1147.5) * 255 = 255</code></li>
<li><code>(100 / 1147.5)    * 255 = 22.2 = 22</code></li>
<li><code>(255 / 1147.5)    * 255 = 56.6 = 57</code></li>
</ul>

<p>I just don't get which one of these I'm supposed to be using for common graphics operations. I have the same question when combining images together (such as multiply, divide, screen, etc).</p>

<mathematics><matrix><blur>
Modulo vs Scaling vs Capping when performing filters
zoombini
# Modulo vs Scaling vs Capping when performing filters

(Asked by: username_0 on 2018-02-10)

I'm currently taking a graphics course and we've recently covered cross-correlation with regards to using a filter matrix that is applied to a region of pixels in a sort of continual raster-type scan across a larger input image to produce some output image.

The formula is denoted here:
[![](https://i.sstatic.net/ZTbzk.png)](https://i.sstatic.net/ZTbzk.png)

*G is the output image, h is the kernel/box filter matrix, F is the input image*

The classic one that was introduced is a 3x3 box filter for blurring an input image. The box filter is a 3x3 matrix with each cell value being equally set to 1/9. We've been dealing primarily with grayscale images with values 0 (black) to 255 (white).

In this case, the largest possible output pixel value would be if the 3x3 input image area contained all white pixels in which case we'd have:

`[(1/9) * 255 ] * 9 = 255`

This is fine, but what if our kernel filter matrix instead had 9 cell values all equal to 0.5?

Then the largest potential output value would be:

`[0.5 * 255] * 9 = 1147.5`

Once we've completely finished producing our new output image matrix, how do we handle values above 255 and below 0?

It seems like there are three possible approaches:

## Capping

Cap all output values to 0 and 255. Any value below 0 is capped to 0, any value above 255 is capped to 255.

## Modulo

Use modulo. In this case it would be 1147.5 % 255 = 125.5 rounded to 126.

## Scaling

Take the highest output value and use it to scale all the values proportionally. If our example output image pixel values were 1147.5, 100, and 255, then we would perform the following operations:

* `(1147.5 / 1147.5) * 255 = 255`
* `(100 / 1147.5) * 255 = 22.2 = 22`
* `(255 / 1147.5) * 255 = 56.6 = 57`

I just don't get which one of these I'm supposed to be using for common graphics operations. I have the same question when combining images together (such as multiply, divide, screen, etc).

**Tags:** mathematics, matrix, blur


---

## 2 answers

### Answer by username_1 on 2018-02-10. Score: 2 | Accepted answer

When doing a filter operation such as a blur, in many cases the filter itself will be normalized so its values sum to 1.0, precisely to avoid this problem.

In cases where out-of-range outputs do need to be handled, the most common way is what you called capping. In graphics it's more commonly known as clamping or saturating. This is the default behavior if, for instance, you run a pixel shader on a GPU and output out-of-range values.

Renormalizing the output image might be necessary in some special cases depending on what you're doing. I can't think of anytime that I've seen the modulo behavior used—it would be a pretty weird operation to do on a visual image.

> **Comments:**
> **username_0**: Thanks for this. I was actually working with some images converted to numpy matrices which are uint8 and it automatically uses modulo when you exceed the unsigned byte length so that's why I thought maybe modulo was used in graphics somewhere. Can I ask for clarification on the clamping? So is it common to add/multiply images together? If so, in my samples working with grayscale images it would seem like the final image would just instantly oversaturate and "white-out" so to speak. Is this normal and do we still just clamp the values?  
> **username_1**: Yes, it's pretty common, though we usually think of images as representing fractional values in [0, 1] rather than integers 0 to 255. Alpha blending, for instance, involves adding and multiplying. Depending on the images and the operations you're doing, saturating to white may indeed happen, but in other cases (like alpha blending layers of images on top of each other) it's not really an issue. Also, if you work with HDR images, you can leave them unclamped while you do a bunch of operations on them, then apply a [tone mapping](https://en.wikipedia.org/wiki/Tone_mapping) function at the end.  

---

### Answer by username_2 on 2018-02-10. Score: 2

While clamping or saturating may be the most common way of handling this situation, it's usually not the best. (But wrapping/modulus is one of the worst.) Another way of handling it is to either convert the input to 16 or 32-bit ints or floats and do all calculations in the higher bit depth. You may want to down-convert the final output, if required by your processing tools. Or, if you can choose the output format you can leave it in a higher bit depth. Most professional tools these days do their work in higher bit depths.

Regarding your question:

> is it common to add/multiply images together?

Yes, it's quite common in tools that allow compositing. In Photoshop, for example, there are blend modes that are even named "Add" and "Multiply" and they do just that. (Though they assume the values are normalized to a 0-1 range.) If you are working with 8-bit per channel images, they will clamp, but if you are working with 32-bit per channel images, they will simply have values greater than 1. If done with linear RGB, this mimics light fairly naturally, and you can apply natural photographic operations to the results, like decreasing the exposure and it will bring those blown-out highlights back into the displayable range. As we move into the era of High Dynamic Range output devices, this type of processing will only become more common and useful.

> **Comments:**
> **username_0**: Thanks! Normalization into 0-1 floating point values makes sense to me because then multiplying won't cause the threshold to be exceeded. What would you do in the case of adding/subtracting images together? Would you clamp the values between 0-1 after the arithmetic?  
> **username_2**: Yes, once you've completed all the operations you need to on the image, only then do you clamp it. So if you're adding 2 images, then blurring the result, or scaling it, or whatever, you do all of those things at the higher bit depth and then clamp the final output (if needed).  
> **username_0**: I guess this doesn't work for grayscale, right? Let's say I have a standard 8-bit RGB image, 8-bits for each channel.  Image 1 is a solid red color image where every pixel is [200, 0, 0]. Image 2 is a solid dark red color image where every pixel is [100, 0, 0].  When I up-convert to a higher bit depth, do I literally just use 16-bits for each channel, so my final image would be an image where every pixel is now [300, 0, 0]? Now my choices are (A) Leave it at this 16-bit per channel depth or (B) Convert back to 8-bit in which case the image will clamp back to [255, 0, 0]. Is that right?  
> **username_2**: This absolutely works for grayscale. It also works for other color spaces like HSV, YCbCr, La*b*, etc. And yes, you have the right idea about what I'm suggesting.  

---

