<p>By <strong>simulating what the subpixels look like</strong>, and optionally converting that to greyscale, we can obtain the intended (even better: color-artifact-free) effect of the original subpixel rendering.</p>
<p><strong>Someone please add this to a screen magnifier software to help read small text and inspect graphic renders.</strong></p>
<p>Results for going from the top left to the bottom right image look nice:</p>
<h1><a href="https://i.sstatic.net/Hl3pByeO.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Hl3pByeO.png" alt="original" /></a> → <a href="https://i.sstatic.net/A2IPUVC8.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/A2IPUVC8.png" alt="greyscale" /></a></h1>
<h1>      ↓                 ↑</h1>
<h1><a href="https://i.sstatic.net/JjF8WB2C.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/JjF8WB2C.png" alt="simulated subpixels" /></a> → <a href="https://i.sstatic.net/TMvFk6RJ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/TMvFk6RJ.png" alt="simulated subpixels, greyscale" /></a></h1>
<p>The algorithm is:</p>
<ul>
<li>Create an image that simulates what the screen looks like at 3x magnification, i.e. render each subpixel as three dedicated pixels of only that primary hue (red, green, <strong>or</strong> blue).</li>
<li>Optionally, convert to greyscale (and brighten 3x because the simulated image can't be bright without HDR).</li>
<li>Display the result at a high zoom level, otherwise the subpixels of the displaying device would interfere a tiny bit. Compensating for that as well is a different story.</li>
</ul>
<p>Python code:</p>
<pre class="lang-python prettyprint-override"><code>import numpy as np
import matplotlib.pyplot as plt

def subpixel_render_and_save_all(image_path):
    # Load the image as an array and normalize to RGB values [0, 255]
    img = plt.imread(image_path)
    
    # Ignore alpha channel for now. TODO handle alpha channel
    if img.shape[2] == 4:
        img = img[:, :, :3]
    
    img = (img * 255).astype(np.uint8)
    
    height, width, _ = img.shape

    # Create a base for the subpixel-rendered image (3x larger dimensions)
    img_subpixels = np.zeros((height * 3, width * 3, 3), dtype=np.uint8)

    # Place each color channel in its respective vertical subpixel column
    img_subpixels[:, 0::3, 0] = np.repeat(img[:, :, 0], 3, axis=0)  # Red
    img_subpixels[:, 1::3, 1] = np.repeat(img[:, :, 1], 3, axis=0)  # Green
    img_subpixels[:, 2::3, 2] = np.repeat(img[:, :, 2], 3, axis=0)  # Blue
    
    # Convert to grayscale using luminance method
    # img_subpixels_grayscale = (0.2989 * img_subpixels[:, :, 0] +
                                # 0.5870 * img_subpixels[:, :, 1] +
                                # 0.1140 * img_subpixels[:, :, 2]).astype(np.uint8)

    # Convert to grayscale
    img_subpixels_grayscale = (img_subpixels[:, :, 0] +
                               img_subpixels[:, :, 1] +
                               img_subpixels[:, :, 2]).astype(np.uint8)

    # Downscale by averaging each 3x3 block to a single pixel
    img_subpixels_grayscale_downscaled = img_subpixels_grayscale.reshape(height, 3, width, 3).mean(axis=(1, 3)).astype(np.uint8)

    base_filename = image_path.rsplit('.', 1)[0]
    plt.imsave(f&quot;{base_filename}_subpixels.png&quot;, img_subpixels)
    plt.imsave(f&quot;{base_filename}_subpixels_grayscale.png&quot;, img_subpixels_grayscale, cmap=&quot;gray&quot;)
    plt.imsave(f&quot;{base_filename}_subpixels_grayscale_downscaled.png&quot;, img_subpixels_grayscale_downscaled, cmap=&quot;gray&quot;)

# Example usage
subpixel_render_and_save_all(&quot;input_image.png&quot;)

</code></pre>
<p>This concrete method is similar to <a href="https://computergraphics.stackexchange.com/questions/9639/upscaling-screenshots-with-subpixel-rendered-text/14255#comment14816_9639">Simon F's mysterious comment</a>.</p>

CC BY-SA 4.0
2024-10-25T11:11:59.757Z
2024-11-22T08:59:49.807Z
2024-11-22T08:59:49.807Z
root
<p>Upscaling a screenshot with subpixel antialiased text produces unpleasant color fringes, e.g. <a href="https://computergraphics.stackexchange.com/q/2034/12022">why does black text have orange and blue pixels</a> This is especially annoying when trying to show an application on a projector, when you have to zoom in with a tool like the Windows <a href="https://en.wikipedia.org/wiki/Magnifier_(Windows)" rel="nofollow noreferrer">Magnifier</a> to show details to the audience.</p>

<p>There are some solutions available to exploit the subpixel geometry of a display when <em>downscaling</em> an image, e.g. <a href="https://computergraphics.stackexchange.com/a/1431/12022">https://computergraphics.stackexchange.com/a/1431/12022</a>. Are there any solutions for the opposite, i.e. taking an image with subpixel rendering and producing an upscaled image without color artifacts? I imagine this task is much more difficult in the presence of both subpixel (text) and non-subpixel (the rest of the screenshot) content in the same image, though.</p>

CC BY-SA 4.0
2020-03-10T18:44:49.023Z
You would have to know the pixels for the text.
lightxbulb
CC BY-SA 4.0
2020-03-10T23:23:51.733Z
I hoped there would be some heuristics (based on stuff like edge detection on different color channels) to find pixels likely corresponding to text, although I couldn't find any.
Kristóf Marussy
CC BY-SA 4.0
2020-03-11T04:47:17.623Z
Possibly a neural network can help. Any simpler heuristic will probably also include other elements than the text.
lightxbulb
CC BY-SA 4.0
2020-03-13T08:48:29.117Z
If you can (a) assume a certain RGB layout for the display (and I imagine you could quickly determine this based on the first vertical edge) and (b) assume fixed background and foreground colours (hopefully white and black) *then* you should be able to replace your source image with a 'monochrome' image which is 3x the resolution in X.
Simon F
CC BY-SA 4.0
2020-03-14T12:10:24.107Z
@SimonF In my case, I would have actual screen captures (text with various colors mixed with GUI elements and images). I know the subpixel layout (queried from the display). I guess a GAN could be trained to do screenshot superresolution, but I am more interested in real-time (think Window Magnifier) solutions. I was hoping that, if my eyes can easily see sub-pixel text as a single color, there is a simple signal processing operation that could do the same...
Kristóf Marussy
CC BY-SA 4.0
2024-10-25T11:23:43.587Z
@SimonF That's a cool thought. Roughly how would you take the background and foreground colors into account? (In an algorithm like [in my post](https://computergraphics.stackexchange.com/a/14255/22474) or another one.)
root
CC BY-SA 4.0
2024-10-30T10:09:16.747Z
@root  I haven't read it too thoroughly but I think the post below looks ok.  Presumably if you do have different foreground and background colours, you will need to calc the 'blend' between the lower and upper values instead of just assuming they are, say, 0 and 255
Simon F
CC BY-SA 4.0
2024-11-02T15:07:12.537Z
@SimonF I haven't thought yet how to use that 'blend' in the reconstruction algorithm. That might depend on how the original subpixel rendering algorithm took the background and foreground color into account in the first place. So those are some things to look into and think about.
root
CC BY-SA 4.0
2024-11-07T09:20:54.603Z
@root Well, _ideally_, the blend (even if the background and foreground were 255 and 0) would be non-linear due to sRGB/gamma etc etc, but it wouldn't surprise me if the implementations were just using a linear blend for speed (or naivety)
Simon F
CC BY-SA 4.0
2024-11-07T14:15:04.723Z
The conflict is that the brightness of a subpixel on one hand tries to achieve a lightness map with a subpixel resolution, but on the other hand to respect foreground and background hues. For example, a red subpixel, due to its x-coordinate, might be best for rendering a certain bright detail at subpixel resolution, but the foreground and background might be blue and green, respectively, so a bright red subpixel might look more out of place than if the foreground/background are black/white? Also orange on dark orange can't freely use all subpixels. Hues dictate subpixel-resolved lightness maps
root
CC BY-SA 4.0
2024-11-22T08:41:45.320Z
@SimonF I forgot to tag you above. Similarly: Different hues yield different lightness "textures"&shifts. E.g. a red pixel on white or black looks shifted left, a blue pixel looks shifted right. (If you subpixel-shift it, it may look shifted in the opposite direction, depending on the shift direction.) A green pixel on white background looks "striped" in terms of lightness; the area to its left looks bluish, the area to its right looks reddish. Vertical white-to-black edges look bluish, black-to-white ones look reddish. (Because our perception doesn't group subpixels into the original pixels.)
root
CC BY-SA 4.0
2020-05-28T20:20:33.463Z
If you use Windows, you can go to Control Panel / Appearance and Personalization / Fonts and click "Adjust ClearType text" link on the left. It opens "ClearType Text Tuner" wizard, where you can uncheck "Turn on ClearType" checkbox. After that you should not see colored edges on enlarged screenshots from that machine. Removing colored edges from existing screenshots is more difficult, I use Adjustment / Black and White tool in paint.net on manually selected regions for that.
Vladimir Reshetnikov
<image-processing><subpixel-rendering><font-rendering>
Upscaling screenshots with subpixel rendered text
Kristóf Marussy
# Upscaling screenshots with subpixel rendered text

(Asked by: username_0 on 2020-03-10)

Upscaling a screenshot with subpixel antialiased text produces unpleasant color fringes, e.g. [why does black text have orange and blue pixels](https://computergraphics.stackexchange.com/q/2034/12022) This is especially annoying when trying to show an application on a projector, when you have to zoom in with a tool like the Windows [Magnifier](https://en.wikipedia.org/wiki/Magnifier_(Windows)) to show details to the audience.

There are some solutions available to exploit the subpixel geometry of a display when *downscaling* an image, e.g. <https://computergraphics.stackexchange.com/a/1431/12022>. Are there any solutions for the opposite, i.e. taking an image with subpixel rendering and producing an upscaled image without color artifacts? I imagine this task is much more difficult in the presence of both subpixel (text) and non-subpixel (the rest of the screenshot) content in the same image, though.

**Tags:** image-processing, subpixel-rendering, font-rendering

> **Comments:**
> **username_1**: You would have to know the pixels for the text.  
> **username_0**: I hoped there would be some heuristics (based on stuff like edge detection on different color channels) to find pixels likely corresponding to text, although I couldn't find any.  
> **username_1**: Possibly a neural network can help. Any simpler heuristic will probably also include other elements than the text.  
> **username_2**: If you can (a) assume a certain RGB layout for the display (and I imagine you could quickly determine this based on the first vertical edge) and (b) assume fixed background and foreground colours (hopefully white and black) *then* you should be able to replace your source image with a 'monochrome' image which is 3x the resolution in X.  
> **username_0**: @username_2 In my case, I would have actual screen captures (text with various colors mixed with GUI elements and images). I know the subpixel layout (queried from the display). I guess a GAN could be trained to do screenshot superresolution, but I am more interested in real-time (think Window Magnifier) solutions. I was hoping that, if my eyes can easily see sub-pixel text as a single color, there is a simple signal processing operation that could do the same...  
> **username_4**: If you use Windows, you can go to Control Panel / Appearance and Personalization / Fonts and click "Adjust ClearType text" link on the left. It opens "ClearType Text Tuner" wizard, where you can uncheck "Turn on ClearType" checkbox. After that you should not see colored edges on enlarged screenshots from that machine. Removing colored edges from existing screenshots is more difficult, I use Adjustment / Black and White tool in paint.net on manually selected regions for that.  
> **username_3**: @username_2 That's a cool thought. Roughly how would you take the background and foreground colors into account? (In an algorithm like [in my post](https://computergraphics.stackexchange.com/a/14255/22474) or another one.)  
> **username_2**: @username_3  I haven't read it too thoroughly but I think the post below looks ok.  Presumably if you do have different foreground and background colours, you will need to calc the 'blend' between the lower and upper values instead of just assuming they are, say, 0 and 255  
> **username_3**: @username_2 I haven't thought yet how to use that 'blend' in the reconstruction algorithm. That might depend on how the original subpixel rendering algorithm took the background and foreground color into account in the first place. So those are some things to look into and think about.  
> **username_2**: @username_3 Well, _ideally_, the blend (even if the background and foreground were 255 and 0) would be non-linear due to sRGB/gamma etc etc, but it wouldn't surprise me if the implementations were just using a linear blend for speed (or naivety)  
> **username_3**: The conflict is that the brightness of a subpixel on one hand tries to achieve a lightness map with a subpixel resolution, but on the other hand to respect foreground and background hues. For example, a red subpixel, due to its x-coordinate, might be best for rendering a certain bright detail at subpixel resolution, but the foreground and background might be blue and green, respectively, so a bright red subpixel might look more out of place than if the foreground/background are black/white? Also orange on dark orange can't freely use all subpixels. Hues dictate subpixel-resolved lightness maps  
> **username_3**: @username_2 I forgot to tag you above. Similarly: Different hues yield different lightness "textures"&shifts. E.g. a red pixel on white or black looks shifted left, a blue pixel looks shifted right. (If you subpixel-shift it, it may look shifted in the opposite direction, depending on the shift direction.) A green pixel on white background looks "striped" in terms of lightness; the area to its left looks bluish, the area to its right looks reddish. Vertical white-to-black edges look bluish, black-to-white ones look reddish. (Because our perception doesn't group subpixels into the original pixels.)  

---

## 1 answers

### Answer by username_3 on 2024-10-25. Score: 1

By **simulating what the subpixels look like**, and optionally converting that to greyscale, we can obtain the intended (even better: color-artifact-free) effect of the original subpixel rendering.

**Someone please add this to a screen magnifier software to help read small text and inspect graphic renders.**

Results for going from the top left to the bottom right image look nice:

# [original](https://i.sstatic.net/Hl3pByeO.png) → [greyscale](https://i.sstatic.net/A2IPUVC8.png)

# ↓                 ↑

# [simulated subpixels](https://i.sstatic.net/JjF8WB2C.png) → [simulated subpixels, greyscale](https://i.sstatic.net/TMvFk6RJ.png)

The algorithm is:

* Create an image that simulates what the screen looks like at 3x magnification, i.e. render each subpixel as three dedicated pixels of only that primary hue (red, green, **or** blue).
* Optionally, convert to greyscale (and brighten 3x because the simulated image can't be bright without HDR).
* Display the result at a high zoom level, otherwise the subpixels of the displaying device would interfere a tiny bit. Compensating for that as well is a different story.

Python code:

```python
import numpy as np
import matplotlib.pyplot as plt

def subpixel_render_and_save_all(image_path):
    # Load the image as an array and normalize to RGB values [0, 255]
    img = plt.imread(image_path)
    
    # Ignore alpha channel for now. TODO handle alpha channel
    if img.shape[2] == 4:
        img = img[:, :, :3]
    
    img = (img * 255).astype(np.uint8)
    
    height, width, _ = img.shape

    # Create a base for the subpixel-rendered image (3x larger dimensions)
    img_subpixels = np.zeros((height * 3, width * 3, 3), dtype=np.uint8)

    # Place each color channel in its respective vertical subpixel column
    img_subpixels[:, 0::3, 0] = np.repeat(img[:, :, 0], 3, axis=0)  # Red
    img_subpixels[:, 1::3, 1] = np.repeat(img[:, :, 1], 3, axis=0)  # Green
    img_subpixels[:, 2::3, 2] = np.repeat(img[:, :, 2], 3, axis=0)  # Blue
    
    # Convert to grayscale using luminance method
    # img_subpixels_grayscale = (0.2989 * img_subpixels[:, :, 0] +
                                # 0.5870 * img_subpixels[:, :, 1] +
                                # 0.1140 * img_subpixels[:, :, 2]).astype(np.uint8)

    # Convert to grayscale
    img_subpixels_grayscale = (img_subpixels[:, :, 0] +
                               img_subpixels[:, :, 1] +
                               img_subpixels[:, :, 2]).astype(np.uint8)

    # Downscale by averaging each 3x3 block to a single pixel
    img_subpixels_grayscale_downscaled = img_subpixels_grayscale.reshape(height, 3, width, 3).mean(axis=(1, 3)).astype(np.uint8)

    base_filename = image_path.rsplit('.', 1)[0]
    plt.imsave(f"{base_filename}_subpixels.png", img_subpixels)
    plt.imsave(f"{base_filename}_subpixels_grayscale.png", img_subpixels_grayscale, cmap="gray")
    plt.imsave(f"{base_filename}_subpixels_grayscale_downscaled.png", img_subpixels_grayscale_downscaled, cmap="gray")

# Example usage
subpixel_render_and_save_all("input_image.png")

```

This concrete method is similar to [username_2 F's mysterious comment](https://computergraphics.stackexchange.com/questions/9639/upscaling-screenshots-with-subpixel-rendered-text/14255#comment14816_9639).


---

