Lossy magic (png-vslp) Part I

Lossless optimization of png's can achieve reasonable savings, however, substantial compression gains can be made with very slight lossy processing (VSLP). Increasingly png images are produced from vector artwork or touched up in photo editors, introducing common artefacts that bloat up png files but add minimal visual information. Principally I'm talking about excessive palette data and in this context I use this as short-hand for all 4 channels: RGB and (most significantly) alpha. Minimizing this obviously reduces the palette size and may allow small images to be saved with just a 256 palette at only 8bits per pixel (vs. 32bpp). But reducing the palette also greatly improves the compressibility of the file, since we have larger areas of equal colour and alpha values, which is nice for the pre-filters and deflate to crunch on. There are many programs out there for producing lossy pngs, but they tend to discard excessive information and the results are visually obvious. To clarify, I'm talking about removing redundant data that will not be visible in the vast majority of situations (hence VSLP).

Removing this data by hand is pretty time consuming, so I hope the process can be automated by using ImageMagick or some custom program (any volunteers?). I've not had much success using this software, so the following is my list of Tweaks for VSLP which someone might want to script. Note, the first assumption (and significant saving) is that we will only deal with pngs using 8 bits per channel; all 16pbc images will be downsampled. The following convention is used: (Suggested variable name and [default value]) ...

1. Tweak alpha transparency data.
It's the decent alpha capabilities that we all love about pngs, but easy gains with least visual impact can be made by tweaking here. The human eye is not so good at judging transparency and yet we preserve high fidelity when scaling and re-touching images in photo editing software.
I think reducing alpha data should be the first target, with highest yield vs visual impact.

1.1. Transparent threshold (alpha_threshold_trans [2%])
Below a certain level of transparency the human eye cannot pick up on the subtle effect on the background colour, so we can set that alpha value to fully transparent. This is clearly dependent on the contrast between the background and foreground colours being blended. I have found that below 2% (i.e. A<=5, for 0-255) represents a reasonable threshold point. Below this threshold we set A=0, above the threshold we leave A=A. Example: Scaling an image with full alpha from 128 to 32 pixels will produce many pixels with an A=1 which is not visible or desireable, but may significantly hurt compression.

1.2. Opaque threshold (alpha_threshold_opaq [98%])
An identical case as above can be made for the threshold at which pixels appear effectively opaque. Again from experience, it would seem that near to 2% fully opaque there is little point preserving alpha data (i.e. A>=250, for 0-255) . Above this threshold we set A=255, below A=A. Example as above.

1.3. Transparent colour (alpha_color [1])
Thresholding alpha values may well reduce the palette and transparency tables, but it yields an additional benefit. Often images contain redundant colour data in fully (/highly) transparent regions which may be useful during editing but are not required for the finished image. This can be removed by setting all fully transparent pixels (A=0 after thresholding), to a common background colour. Sensible colour options are: [0] leave colours as they are; [1] set to black [RGB=0,0,0]- which is highly efficient for most images; [2] Pick the most highly used colour in the image (often white or black).

I have numerous other VSLP tweaks that I routinely apply, and will write them up in time. Cheers.

New Addition

Currently my OOo activities are on pause, with the new addition of Baby Beth to our family. Hopefully in a month things will have settled down.

18MB for images archives

After installing OOo 2.0.2RC1, I was keen to play with IconSwitching and check out the 'crystal' theme. I was surprised to see the image archives now weigh in at 18MB- ouch! But this is good: just as I was losing faith in my hacking efforts, I was reassured that they are a useful contribution.
I've also noticed that jimmac has a CWS set aside for producing a Tango theme for OOo3.0 - cool! I really like the Tango set, but alas the png files are unoptimized. When (/If?) I get my icon audit finished, I might try and give Tango some TLC.
Development of pngcrush has started again, so it will be interesting to see if it can reclaim it's crown from optipng.

Ongoing Icon audit

I am working on an icon audit, to create a catalogue for developers and Theme creators. I hope it will also serve as a possible spec, to remove some of the duplications in OOo icons. Currently I estimate a 3:1 reduction in files, so ~2.5K unique images, of which probably ~1k are core to a theme (this matches kendy's 908 icons). It's very much a work in progress, but I will make the draft available to download, which should be updated once a week (Fridays?). Currently the catalogue contains <300 images, which is the result of ~2months work (in any scraps of my freetime). At this rate it will take me another 14months to complete!! Either I need to get some more time (26hr days?) , get more efficient, or hope someone wants to collaborate...

Icon dependencies

With the upcoming Icon Switching in OOo 2.0.2, we should be able to slim down 'images.zip', by splitting out the High Contrast (HC) icons, which now exist as a seperate theme. However, this has not been fully addressed as hardcoding remains for HC icons causing issues when they aren't present. I've been tracking these down by trial & error. Yesterday I caught a tricky one, involving 4 icons... much rejoicing after many hours of hacking. The result is a new release of my optimized images.zip (version 2, beta3), which fixes the problems I have encountered so far. Obviously it would be best if a developer trawls the source code for these dependencies, but I wouldn't have a clue where to start. For more information on these dependency issues see: i61432, i61498, i61500, i61634, i61804.

Lossy tweak for a crystal icon

I mentioned earlier that some of the crystal set contains icons saved at 16bits per sample, and that many icons would benefit from some lossy tweaking. Well here's an example: /res/commandimagelist/lc_linewidth.png
(Also used for lc_formatline.png & lc_fillstyle.png).

Original was 1,860kb. After lossless optimization (only 'advdef' can handle 16bps), compressed to 1,658kb (84%). Reducing to 8bps (32bpp) and some minor tweaks to remove a few unnecessary palette colours (lossy+losslesss), gives us a final file of 856kb (46%).

Running various compression methods gets us so far, but the best approach is for the icon designers to consider efficiency at the production stage. The increasing trend for blitting png icons from an svg (or other vector file), moves the designer even further away from pixel pushing and the ability to hand optimize a file. Maybe this is a dying craft only for the true geeks?

Progress with 'custom_image' themes

After contacting kendy, I've been working on optimizing the crystal and industrial icon themes. They are generally much larger than the default theme, since the images are richer with more complex alpha transparencies. As a result, the potential for lossless optimization is not as good. The crystal set has now been finished and can be found here (1.32 to 1.15MB ~87%). The industrial set can be found here (1.83 to 1.35MB ~74%). Both themes would benefit from some hand tweaking, since at least some icons have been saved as 48bpp(!) so cannot be optimized with my method. However, I think it is a better use of my time to press on with the Icon Audit and tweaking the Default set.