Resizing artifacts with Lanczos

[Drarakel]

I'm experiencing strong artifacts (ringing) when resizing with Lanczos lately (with a current Q16 IM). While I know that there can be some ringing (depends of course on the images), I'm struck by the amount here.
Maybe some pros can comment - if I made something wrong or if Lanczos / the windowed resize filters have a special weakness here (besides the 'normal' ringing).

Example: test.png
(It looks 'ugly' at original size, as it's an output from Ghostscript at 4x size - to compensate the non-working antialiasing there. I have to downsize this supersampled image to 25% in order to get a normal, antialiased image.)

First, a 'normal' resize:

Code: Select all

convert test.png -resize 25% result1.png

Looks fine.
The normal Lanczos resize with more lobes:

Code: Select all

convert test.png -set option:filter:lobes 8 -resize 25% result2.png

Still fine. If I magnify that, I of course see that there's some ringing. But it's ok.

Code: Select all

convert result2.png -crop 79x93+4+9 +repage -scale 400% result2s.png

The above commands show no severe artifacts yet, but in general, the results will be wrong (even if it's not noticable in THIS example). The reason is that my input image is a sRGB image. (I stripped the metadata to keep the filesizes low, but I can assure you that the image had been converted with the "sRGB IEC61966-2.1" profile.)
One solution is to make gamma corrections in order to do the resizing in linear space. So, I modify my 'normal' resize:

Code: Select all

convert test.png -gamma 0.45455 -resize 25% -gamma 2.2 result3.png

And with more lobes:

Code: Select all

convert test.png -gamma 0.45455 -set option:filter:lobes 8 -resize 25% -gamma 2.2 result4.png

A non-acceptable result, if you ask me. I don't even need to zoom in that image to see that the last result has three halos/ghosts around these letters. Well, doing a gamma correction after resize probably accentuates the artifacts from Lanczos. But is it really supposed to be that bad?
For comparison - again a magnified image of the letter 'A' here:

Code: Select all

convert result4.png -crop 79x93+4+9 +repage -scale 400% result4s.png

I can avoid some of the severe artifacts by just using the default Lanczos settings (not using more lobes). But even with the default of 3 lobes, I still see one distinct halo around the letters. I don't see that halo on the first result images - without the gamma corrections. But again: Without the gamma corrections, IM apparently assumes that the input is already in linear space, and that's just wrong with most images. (Yes, I have read that thread.)

One problem could be that such a manual gamma correction is not totally correct yet. But even if I could convert my sRGB images to a 100% correct linear space (and I don't know how to do that with ImageMagick) - I'm not sure if the accentuated resize artifacts would be gone.

Has anyone some hints here (besides using a more blurry resizing filter)?

(Edit: Corrected one typo in the 'result4' command.)

[anthony]

Technically you should be removing gamma correction before resizing, then adding it again afterwards.

However the amount of ringing is greater when using a 8 lobed lanczos. It is the way it is.

IM does not do automatic gamma compensation simply because not all image formats have a gamma setting.
If you want to change this, then it is up to Magick.

[Drarakel]

However the amount of ringing is greater when using a 8 lobed lanczos. It is the way it is.

Sure. I was not surprised that lanczos with 8 lobes itself had more ringing (when compared to the same command with less lobes).

I was surprised that a (theoretically more correct) lanczos resize in linear space (and I have to use gamma corrections there to do that if I use a sRGB image) showed much more ringing than the same resize in sRGB space. And I used the lanczos with 8 lobes to produce an example where the difference is more striking. The differences between 'result2.png' and 'result4.png' (both with 8 lobes) are much more noticable than the differences between 'result1.png' and 'result3.png' (both with 3 lobes). But even with the 3-lobes-resize, the difference is there. And, well, I'm a bit worried with the result of the 3-lobes-resize in linear space (result3.png), as it has more ringing than I was used to. (The perception of that of course also depends a bit on the monitor settings).
I had hoped that there was a trick that allows one to do the resize in linear space while keeping the resizing artifacts in the final image at a 'normal' level (the ringing in 'result1.png' or 'result2.png' is ok). But that's perhaps not possible.

[anthony]

It seems to be just the way it works.

Have you tried this with some other image processing package?

[Drarakel]

I now was able to do the correct gamma operations for sRGB - instead of using the approx. average gamma of 2.2. So, the resizing can be done in complete linear RGB. And the resizing artifacts from Lanczos also have diminished a bit - compared to 'result3.png' and 'result4.png'. (Only by a small degree - but it's better than nothing.)
I will again use 8 lobes, as the differences can be seen a bit better there. A rather complicated (and slow) commandline is now needed:

Code: Select all

convert test.png -fx "(p>0.04045) ? pow((p+0.055)/1.055,2.4) : p/12.92" -set option:filter:lobes 8 -resize 25% -fx "(p>0.0031308) ? pow(p,1.0/2.4)*1.055-0.055 : p*12.92" result4b.png
convert result4b.png -crop 79x93+4+9 +repage -scale 400% result4bs.png

(Taking the math from e.g. here.)
Theoretically, the 'normal' ImageMagick colorspace options could also be used - but there are some issues with that at the moment.
Anyway, the magnified result with that:

(One can see the difference by switching forth and back the result images.)

Have you tried this with some other image processing package?

Well, I don't use this professionally. Up to now, I mostly used the 'normal' resize methods and don't cared about gamma. I guess, with a lot of tools one has a hard time trying to do image operations in linear RGB space (when one has sRGB as input). According to Eric Brasseur's article, the situation on Mac OS X is better as more tools seem to be "gamma-compliant" there. But I'm using Windows.. I could perhaps test it with Photoshop, but I'm not sure if I could directly compare the artifacts with resizing in linear space - as the resizing algorithms are probably also different there (bicubic?).

[anthony]

I am not actually certain sRGB is just a gamma corrected linear RGB.

You may need to look at the conversion of sRGB to RGB before the gamma correction and resizing. I believe this would require you to use some color profiles. I myself have never needed to do this.

[Drarakel]

As I said, I didn't use this much either. So, forgive me if I use some terms wrongly (especially since technical English language doesn't belong to my skills)..

You may need to look at the conversion of sRGB to RGB before the gamma correction and resizing. I believe this would require you to use some color profiles.

Did you look at this article?
http://www.4p8.com/eric.brasseur/gamma.html#solutions
The color space/color profile itself shouldn't matter here - but the gamma does. With most digital (RGB) images, one can probably assume a gamma of 2.2 or a similar value. (An image in linear space would appear way too dark on normal monitors.) With an image that was converted with a sRGB profile before, I just have the advantage, that I know the exact gamma curve. And ignoring the gamma and doing the operations (like resize or rotate) as if the images were in linear space can then yield some wrong values (some interpolated pixels appear too dark).

Again, as I said, up to now, I just used the 'normal' resize operations (didn't do the gamma compensation). As most of my pictures are real-world-photographs, I think, the 'error' usually wasn't very big there. I mean, I often use ImageMagick for 'normal' resizes - and the resizing methods/filters are very, very good here! (And I also noticed that there were some additional improvements in that area lately. A big thanks to Anthony for that! And also to Nicolas Robidoux.)

But.. I had noticed that e.g. after downsizing images by a large factor, some text inserts were too faint after that or some colors were a bit off or something like that. Such things can be avoided if one adds operations to the commands that convert the images to linear space (and back).
And I'm hoping that one will be able to do the exact transformations for common images (sRGB) in an efficient way in IM (not with "-fx"). Probably, just "-colorspace sRGB" has to be fixed (and perhaps the name be changed as it's a bit misleading). But that gets a bit off-topic now...

[anthony]

But.. I had noticed that e.g. after downsizing images by a large factor, some text inserts were too faint after that or some colors were a bit off or something like that. Such things can be avoided if one adds operations to the commands that convert the images to linear space (and back).

This may be off topic (again) but extreme down-sampling with text often results in the text becoming too mixed with the surrounding colors, and thus becoming faint. Their is very little that can be done about that, except posibly trying to use morphology to make the text a little thicker,
perhaps with some masking to limit the effects.

And I'm hoping that one will be able to do the exact transformations for common images (sRGB) in an efficient way in IM (not with "-fx"). Probably, just "-colorspace sRGB" has to be fixed (and perhaps the name be changed as it's a bit misleading). But that gets a bit off-topic now...

The typical way of converting things like sRGB to RGB and so on is with profiles.
See IM Examples Color Profiles
http://www.imagemagick.org/Usage/formats/#color_profile
often images give little or no indication of the color profile used, or does not include the profile information assuming that if you need it you have it.

I do hope someone with more knowledge of color profile handling can help you with this.

[Drarakel]

This problem has nothing to do with color management with color profiles!
I know about profiles, thanks. But the point is that, for example, if you have an image that was already converted to sRGB (yeah, with profiles) and simply do a resize after that, you will get (slightly) wrong values. In my last post, I didn't write about the 'regular' negative effects that can occur with resizing, but about the additional problems that can occur when this is done without gamma compensations (and for the gamma compensations, you don't need profiles).
I can only point again to the mentioned article and directly quote:

Technically speaking, the problem is that "the computations are performed as if the scale of brightnesses was linear while in fact it is a power scale." In mathematical terms: "a gamma of 1.0 is assumed while it is 2.2."

And to 'fix' that with such images in IM, one could use:
- "-gamma" before and after the operations. That will do it with some images. Though with images that are really in sRGB space, that isn't very precise (and IMO, it's not worth the effort to do the gamma compensations when one adds again an error there).
- Theoretically, I could also do a conversion with color profiles - if I would find a destination profile that uses the gamma value 1.0. But I don't want to do an actual color space conversion and change the additional properties like the gamut and such things - but just the gamma. (With Photoshop, one could do a conversion with the sRGB profile by just modifying the gamma value - i.e. by using a custom profile. See here. But that's more of a workaround solution and isn't directly possible in IM anyway.)
- Or one can try to use the exact formula for sRGB images. At the moment, it's possible only with a slow "-fx" operation in IM. (Or does anyone have another solution?) Well, there's this "-colorspace sRGB" that theoretically just does this gamma change. But that operation is broken at the moment. (And, it's really hidden - it isn't explained that it would actually just change the gamma curve from a sRGB image into an image with gamma 1.0.)

So, I have already my solution for this gamma problem (and am hoping that "-colorspace sRGB" can be fixed).
Sorry for such a lengthy response, but apparently you didn't quite believe me there - or my former responses weren't enough.


Anyway, all this wasn't the original reason for starting the thread. I started the thread because I noticed that WHEN DOING such a gamma-compliant resize, that the artifacts from Lanczos are much more accentuated. And I wondered if there's something special in the Lanczos algorithm (as I don't know much about that) that would explain that. And I also wondered if I perhaps had made an error somewhere.
But after making additional tests and also with your comments, I am now convinced that there is no way getting around this completely. My conclusion if one wants to avoid these stronger artifacts: One can either use a normal Lanczos resize without the gamma compensations, and will usually have no severe resizing artifacts - but some slight off looking colors (depends on the image of course). Or one can use a gamma-compliant command, but perhaps use a softer resizing filter (and manually sharpen after that).

Or, to put it the other way round: The artifacts from Lanczos (or another windowed filter) are only then toned down if the resize is done in a slightly wrong manner (if the input image is handled as if it were in linear space, but in reality is not).

[anthony]

This problem has nothing to do with color management with color profiles!

Fair enough... I'll try to remove the foot from my mouth now.

Anyway, all this wasn't the original reason for starting the thread. I started the thread because I noticed that WHEN DOING such a gamma-compliant resize, that the artifacts from Lanczos are much more accentuated. And I wondered if there's something special in the Lanczos algorithm (as I don't know much about that) that would explain that. And I also wondered if I perhaps had made an error somewhere.

My understanding is that is just the way lanczos is. It does have quite a strong ringing effect when it deals with sharp boundaries. Using a larger window (large lobes or support), or a raw Sinc, is a lot worse! But the effect is made worse if applied in in a gamma corrected image.

But after making additional tests and also with your comments, I am now convinced that there is no way getting around this completely. My conclusion if one wants to avoid these stronger artifacts: One can either use a normal Lanczos resize without the gamma compensations, and will usually have no severe resizing artifacts - but some slight off looking colors (depends on the image of course). Or one can use a gamma-compliant command, but perhaps use a softer resizing filter (and manually sharpen after that).

May I suggest you try a Mitchell (actually Mitchell-Netravali) Filter, and then do a small unsharp step afterward.
The unsharp is how photoshop handles image resizing. I have not talked about this much as I have not really gone into any experiments with it, though I do mention it as an aside under the "Blur" expert option.

Or, to put it the other way round: The artifacts from Lanczos (or another windowed filter) are only then toned down if the resize is done in a slightly wrong manner (if the input image is handled as if it were in linear space, but in reality is not).

I agree, but IM has little way of known if the input image is gamma corrected or not, as most do not mention it. By then again about half the image processing in IM is with generated images that do not contain gamma correction.

Basically your damned if you gamma correct and you are damned if you don't.

I have similar problems in other areas of IM. Image Distortion, and Ordered Dither are two areas which I have been heavily involved with.

[Drarakel]

I agree, but IM has little way of known if the input image is gamma corrected or not, as most do not mention it.

At least it would be possible to know the gamma if a color profile is embedded (e.g. sRGB) or e.g. if a PNG file has a sRGB chunk - and theoretically IM could use that for automatic gamma compensations. But personally, I don't need that. I'm perfectly fine with explicitly adding the operations for the gamma compensations - as long as these operations exist (which would be the case if "-colorspace sRGB" would work). (While this option is broken, I wrote my own app to do this.)

May I suggest you try a Mitchell (actually Mitchell-Netravali) Filter, and then do a small unsharp step afterward.

Yeah, it seems, that's the way to go. I already use Mitchell for some pictures, but probably will have to use it to a greater extent (especially in such gamma compliant resizes).


Thanks for your input!

[anthony]

Please let us know how it goes.

Hmmm what exactly is your sRGB conversion solution?

[Drarakel]

Hmmm what exactly is your sRGB conversion solution?

Do you mean the formula that I'm using for converting the sRGB gamma into a linear gamma?
I took the formula from here:
http://www.sjbrown.co.uk/2004/05/14/gam ... rendering/
http://en.wikipedia.org/wiki/SRGB#The_r ... sformation
ImageMagick has this formula already (strangely enough, it's listed with the name "sRGB" here):
http://www.imagemagick.org/script/comma ... colorspace
(Though the implementation in IM has errors at the moment. See my report.)

[adx]

I just had to say - one thing that can provoke a resize filter is sharp-edged text: This situation (a step from 0 to full intensity) can't occur in "ordinary" sampled image data (properly bandlimited/antialised and not clipping highlights). Filters which work really close to the Nyquist limit (like a high-lobed Lanczos) don't like it. But nor do they like high downsize ratios which have a similar effect - it's just a tradeoff between steepness of cutoff (frequency domain) and ringing.

My excitement on finding that image resizing is generally done wrong (not in linear space) was also dampened when I realised that this wrongness also hides ringing. The best I have done in situations where it is a big problem is to use a less 'ringey' filter (eg bessel) with some post-sharpening. This has the same effect of boosting / retaining high frequencies (as a sharp cutoff filter), but also boosts the aliasing which is rarely a big problem. This adds another degree of freedom to the compromise and can give excellent results. (I have been battling away at finding a good image downsizer for years!)

Another problem I have had with IM Lanczos is the square fiter kernel: It gives a kind of "diagonal noise" look to textures with a lot of sharp content (eg leaves from a distance). If I get time some day I'll experiment with round kernels of varying sizes.

Anyway, I do almost all my image downsizing now using IM and the 0.4545 2.2 transform and the default Lanczos filter, it sounds horrible but dragging and dropping images onto batch files is much faster than using some GUI app, and it produces superb output (and with one of the best tuned jpeg compressors). Thanks for the proper sRGB functions, that was something I wanted to try but hadn't yet got round to working out how to do in IM.

[NicolasRobidoux]

Another problem I have had with IM Lanczos is the square fiter kernel: It gives a kind of "diagonal noise" look to textures with a lot of sharp content (eg leaves from a distance). If I get time some day I'll experiment with round kernels of varying sizes.

Please try the brand new distort Lanczos and LanczosSharp cylindrical methods. These cylindrical Jinc-Jinc "Lanczos" 3-lobe are Lanczos-like (although a bit more blurry) and they do not suffer from "Lanczos diagonal noise." In addition, their haloing looks "more natural" than the haloing of standard Sinc-Sinc Lanczos. They are so new that they've only been tested for enlarging, but I've been really impressed with the preliminary comparative results. (I think that Anthony has not been as impressed as me, so take my "!" with a large grain of salt. On the other hand, none of us has had the time to do a lot of testing lately.)

For example,

Code: Select all

convert test.png -filter Lanczos +distort SRT '0,0 .25,0' +repage -crop 357x118+1+1 result.png

to resize to 25%. Some alternate "negative lobe" methods in decreasing order of cost: Swap Lanczos for LanczosSharp, Lanczos2, Lanczos2Sharp, Mitchell, and Robidoux. And we've not even looked at the results using filters with more than 3 lobes with EWA distort. (Note: Anthony suggests dropping the +repage and cropping at +0+0 instead of +1+1. I'll try this later. See viewtopic.php?f=2&t=17410. I actually know more about the internals of IM than how to use it!)

Even in this example, if you look carefully, you will notice that diagonal lines have less "checkerboard ringing" nearby than with plain (orthogonal) Lanczos:

Code: Select all

convert test.png -resize 25% result1.png

On the other hand, maybe you'll feel that Lanczos (used cylindrically) is a bit on the blurry side? Instead of using unsharp mask after the fact, you may instead "tighten" the blur kernel, as in

Code: Select all

convert test.png -filter Lanczos -define filter:blur=.8956036897402794 +distort SRT '0,0 .25,0' +repage -crop 357x118+1+1 result.png

This will make it run faster too.

LanczosSharp and Lanczos2Sharp are names given to methods with "special" blurs. (The spectral effect of scaling the blur is to scale the frequency response surface extent by its reciprocal.) (Note to Anthony and Fred: This blur is not chosen randomly: It is the second local minimizer of the deviation from the identity when doing no-op (the first minimizer is almost exactly equal to 1).) Most likely too sharp (aliased) for upsampling, but to my eye sharper than regular (tensor) Lanczos3 but with less ringing for this reduction by a factor of 4 of your test image.

General note: This is totally empirical, but it appears that with Lanczos distort (Jinc-Jinc 3 lobes EWA), blur taking values between about .8 and 1.05 gives OK results. For example, there are some grounds to use blur=0.82750567990790052. With this small a blur, the amount of data used by EWA Jinc-Jinc is actually considerably smaller than used by tensor Sinc-Sinc. Of course, the smaller the blur, the more likely Moire will rear its ugly pattern, but for your test image, no problem: The reduced image actually looks kind of crisp for this value of blur which is pretty close to the reasonable limit. On the other hand, text-like and nature-like are not good candidates for Moire, so no surprise there. (Lanczos2 does not fare so well if blur is reduced a lot.)

The more recent the svn, the better (they are very recent additions, and LanczosSharp and its relative Lanczos2Sharp were tweaked... yesterday and today, respectively).
They are very recently developed methods, so we don't know much about how they actually perform in real life. For example, I'm guessing that Lanczos2Sharp may turn out to be an OK all-around method for downsampling. (Robidoux should be a little faster and give similar results; apologies for the tacky filter name: I'm trying to get tenure.) These methods appear to be original to IM, with the exception of (distort) Jinc-Jinc Lanczos2 which was published in a 1993 Helsinki U. of T. Masters thesis by Andreas Gustafsson (under the name "Lanczos2D") which seems to have gone under the radar.

(I say "appear to be original" because they seem to be the result of doing the "obvious next step" with respect to both filtering kernels and averaging ellipse computation in the field of Elliptical Weighted Averaging (EWA) resampling, and then asking "What can be improved without making things worse?" (Answer: we can try to make cylindrical methods do better with horizontal and vertical lines, which are not naturally preserved when one averages radially like they are with tensor (orthogonal) methods.) Someone has got to have thought of that before Anthony, Fred and I!)

Warning: If you call these filters through -resize, you will not get their cylindrical versions. You need to call them through distort to get the desired effect.

Apologies for sending you the "untested way" but the lack of diagonal artifacts when the results are compared to those obtained with any other method with negative lobes is exactly what has been jumping at me in my preliminary testing of these new methods. I am guessing (hoping) that this will jump at you too. (Too much grading to do to fully test any time soon )