BiCubic Interpolation does not match any known filter!

[anthony]

Do you know of any application that uses a different interpolation for alpha, when doing the same operation on color channels?

Note that images can have multiple alpha's, as different masking, but only one (the primary) can be applied to colors. Excatly how multiple alphas should be telt with is a problem.

Note that the division 1/alpha is actually 1/{resultant alpha} and is because the color channel themselves are multipled by alpha*weight during the interpolation. The same alpha blending happens in composition.

See SVG composition formulas.
http://www.w3.org/TR/2009/WD-SVGComposi ... ompositing

It is done to ensure that you don't get contributions of 'fully transparent colors' into the result visible color, and so producing halos, such as shown in the very OLD bug reports
http://www.imagemagick.org/Usage/bugs/resize_halo/
and what was previously possible with blur and default channel setting (before the switch to using morphology blur)
http://www.imagemagick.org/Usage/bugs/blur_trans/

[anthony]

Cristy seems to have installed Nicholas's Formula. The interpolation is no longer a extreme 'cardinal' but now equivelent to the Catmul-Rom Key's Cubic Filter.

I think it was its extreme depth that made me think it was actually a lagrange cubic as I mentioned in my IM Example Filter notes. I shall correct it.

In IMv6 I have now renamed 'Bicubic' to be 'Catrom'. Bicubic remains available in both CLI and in the API, though is depreciated as being in-precise. Use of this interpolation in examples should now be avoided.

The new list of Interpolations is now...

Code: Select all

convert -list interpolate
Average
Average4
Average9
Average16
Background
Bilinear
Blend
Filter
Integer
Mesh
Nearest
NearestNeighbor
Spline

with Average4 equivalent to Average, and Nearest equivalent to NearestNeighbor

[anthony]

Great the 'blend' interpolation addition now seems to have broken Not my fault, honest!


ASIDE: For easier examples -interpolative-resize is now also available in the latest IMv6 from SVN, just as it is in IMv7. WARNING: The option call name will change later in IMv7 development! I only did this backport for convenience.

Code: Select all

convert xc: -bordercolor gray10 -border 2  -interpolate catrom -interpolative-resize 150 pixel_2d_catrom.png

[NicolasRobidoux]

Do you know of any application that uses a different interpolation for alpha, when doing the same operation on color channels?

I have done several quick google searches, and have not found anything particularly informative. It looks like lots of people stick to bilinear, which makes the issue moot.

[NicolasRobidoux]

...
Note that the division 1/alpha is actually 1/{resultant alpha} and is because the color channel themselves are multipled by alpha*weight during the interpolation.
...

This makes complete sense.

I feel very strongly about clamping negative interpolated alphas to the same value as what alpha=zero is clamped to when reciprocals are used.
If this is done, I feel less strongly about systematically using a monotone method to interpolate the alpha.
But I feel pretty strongly about this too, because alpha truly is special: it is combined nonlinearly with the other channels. As mentioned, it is a little bit like what physicists call "an envelope", which actually is a little bit like a window function for a Sinc- or Jinc-windowed method. (This analogy is a bit stretched, but still conveys the right gist.) In any case, when something is used nonlinearly, it generally needs to be "regularized" more strongly than if not. Regularizing, here, means keeping within it's natural range, but also making sure that the result is smoother. This is why, with interpolate methods, like Catmull-Rom and Mitchell, that have negative lobes, I suggest that you use cubic B-spline smoothing (or quadratic B-spline smoothing or even bilinear if you want to keep things "tight") for the alpha channel.

This is entirely based on back of the envelope computations. I have performed no experiment.

-----

Barring actual careful testing, this is actually my recommendation:

I recommend that if you use reciprocals you are better off using bilinear for the alpha channel with every single colour interpolator, throughout the whole toolchain. One reason is that if you interpolate between alpha=0 (at x=0, say) and alpha=1 (at x=1, say), clamping values alpha close to zero to MagickEpsilon will have the overall effect of

multiply by alpha
divide by MagickEpsilon
multiply by alpha when viewed

for an overall colour*alpha*(alpha/MagickEpsilon)=colour*x^2/MagickEpsilon when viewed, which means that when using an alpha channel bilinear creates smooth enough transitions while being interpolatory: You formally get a C^1 "viewed" result (formally because 1/MagickEpsilon is very large, and consequently the slope of the transparency multiplier goes from 0 to 1 over a very short distance, namely MagickEpsilon, which is almost as bad as having a slope discontinuity).

So, at this point, I'd say use bilinear for the alpha channel across the board (possible exception: nearest neighbour with the colour channels: then use nearest neighbour with alpha too). If you feel that this is not good enough, we can try other things. But then we'll have a sane baseline to compare the expensive/more complicated methods to. And I'd actually wager that complicated will lose more often than not.

(Systematic bilinear with alpha certainly would make things run faster.)

Set me straight if I'm wrong.

[NicolasRobidoux]

My goal, here, is more about preventing things going "badly" wrong than pushing the limit on how nice one can make things when things are already nice.

[NicolasRobidoux]

Anthony: I had not seen your interesting and old discussion of the issues. My comment is this: Given "Either blur colors, or blur transparency, do not blur both." you should not be surprised by my suggestion to decouple the resampling of the transparency from the resampling of the colours. In addition, my suggestion to use a "tight" resampler for the transparency across the board when resampling colours, namely bilinear, should not surprise you either: I don't want the alpha to extend past where it was to start with, because otherwise it is likely to make "spurious colours" visible.

[anthony]

My goal, here, is more about preventing things going "badly" wrong than pushing the limit on how nice one can make things when things are already nice.

Note that this same discussion would also apply to resize and distort resampling!

Would you want alpha to be handled using bilinear for those?

The same goes for any convolution. We can ignore composition for this as they rarely use negative weights in blending pixels (unless you are extrapolating a blend).

[anthony]

I don't want the alpha to extend past where it was to start with, because otherwise it is likely to make "spurious colours" visible.

Then you will NEVER be able to sharpen a 'shaped' edge. Especially a very blurry edge!

Alpha like colors much be able to get closer too and reach limits of the transparency range. Of course negative alpha does not make sense. But then neither does a color becoming blacker than black and whiter than white, and yet HDRI requires this.

Colors get 'clamped' too!

[NicolasRobidoux]

...
Note that this same discussion would also apply to resize and distort resampling!

Would you want alpha to be handled using bilinear for those?

Probably not, because these methods are used a lot to downsample, and "leading" artifacts are different then than when upsampling.

[NicolasRobidoux]

Maybe I'm wrong. I will happily use the same interpolator for colours and alpha. You have more experience fixing this issue than I do. I know that.

However:

...
Alpha like colors much be able to get closer too and reach limits of the transparency range. Of course negative alpha does not make sense. But then neither does a color becoming blacker than black and whiter than white, and yet HDRI requires this.

Colors get 'clamped' too!

Blacker than black or whiter than white shades do make sense, because they can understood as meaning that you have non-optimal black and white points in your digital photograph. Digital cameras, even expensive ones, often leave in dark colours that are actually noise, and many digital photographs benefit from raising the black point more than a bit (this is even true in 8-bit). Overexposure means that you set your white point too low when you took the shot. There is a connection, I believe, to the common raw processor buttons "recover highlights/shadows". Once you start playing with the black point and the white point, you have to be able to deal with negative colours and colours which are past your current choice of white point.

Transparency outside of [0,1] makes absolutely no sense.

Also: Transparency is used nonlinearly: It is a multiplier. Always.

This is not a goose/gander thing. Nonlinear things have a way of blowing up in your face that linear things can't.

Finally: It would seem that if my idea should be tried, it should be tried with a small piece of IM first. Interpolators is such a small piece.

On the other hand, you have a lot on your plate already, so why try something different now?

Because Nicolas got thinking?

I think all the time, so this is a bad reason. And I don't have the full, or even the big, picture.

-----

Anthony: Truly: If you are sure, or even kinda sure, that using the same interpolator is better, I'm in no mood to argue, and I realize that my position is weak.

[magick]

Transparency outside of [0,1] makes absolutely no sense.

Nicolas, we'll be investigating this assertion on a case by case basic in the near future. As we mentioned, when we enforced transparency clamping in a wholesale fashion, multiple algorithms failed to produce proper results as visually demonstrated by demo.pl in the PerlMagick/demo folder. Perhaps in a week, we'll try a few algorithms and investigate why clamping is causing the failure.

[NicolasRobidoux]

Transparency outside of [0,1] makes absolutely no sense.

Let me moderate this statement:

Transparency outside of [0,1] makes absolutely no sense to me.

[Jason S]

Transparency outside of [0,1] makes absolutely no sense to me.

Consider two ways you could process an image:

1. Apply a background color to the image, then resize it.
2. Resize the image, then apply a background color to it.

Do you expect to get the same result in both cases? If you allow the intermediate image to have transparency outside of [0,1], you can get the same result. Otherwise, you won't (assuming the resampling filter takes negative values).

At least, that's the conclusion I came to when I looked into this issue recently.

Do you know of any application that uses a different interpolation for alpha, when doing the same operation on color channels?

ImageWorsener used to have a feature to let you use a different filter for the alpha channel, but I removed it, because it produced nonsense results. I saw no way to prevent it from taking pixels that have no meaningful color, and making them visible.

[NicolasRobidoux]

Jason S: Thank you very much for your comment!

Transparency outside of [0,1] makes absolutely no sense to me.

Consider two ways you could process an image:

1. Apply a background color to the image, then resize it.
2. Resize the image, then apply a background color to it.

Do you expect to get the same result in both cases? If you allow the intermediate image to have transparency outside of [0,1], you can get the same result. Otherwise, you won't (assuming the resampling filter takes negative values).

Insightful and to the point!
Now, the question becomes: Is it maximally useful for these two processes to give the same answer? It certainly is reasonable, but there may be a hidden cost.
Of that, I'm not sure.

ImageWorsener used to have a feature to let you use a different filter for the alpha channel, but I removed it, because it produced nonsense results. I saw no way to prevent it from taking pixels that have no meaningful color, and making them visible.

Did you get nonsense results when you used bilinear on the alpha with another filter on the colour channels? My point is to use a "safer" (monotone, tight support...) filter on the alpha.

(Also: You fed alpha multiplied colours into the filter, and "unmultiplied" them after resampling? Or you always used alpha multiplied colours?)