image size convention in resize.c

[NicolasRobidoux]

Notation: When resizing, the sampled (input) image has m rows and n
columns, the output image has M rows and N columns, output pixels
are indexed with (I,J) with 0<=I<=M-1 and 0<=J<=N-1, and coordinates
are set so that they equal the index right at a pixel
location. (x,y) is a position in the "input" image.

Suggestion:

Depending on whether resize is enlarging or reducing, resize should
switch from the

"Match corners" convention:

The pullback of output pixel location (I,J) is (x,y) where

x = -0.5 + (I+0.5) * m/M

y = -0.5 + (J+0.5) * n/N

(this is the convention used all the time by IM right now) to the

"Match centers" convention:

The pullback of output pixel location (I,J) is (x,y) where

x = I * (m-1)/(M-1)

y = J * (n-1)/(N-1)

The switch is to happen in each direction independently depending on
whether m<M (resp. n<N).

"Match corners" is best when downsampling, but not when upsampling.

Quick and dirty motivation: Of the two conventions, "Match corners"
makes the interior of the image the largest within the result when
reducing, and "Match centers" makes the interior the largest within
the result when enlarging. This reduces boundary artifacts (less abyss
is used) and makes the result slightly less blurry (because each input
pixel occupies slightly more space in the output image no matter
what).

Because both conventions agree when M=m or N=n, the swith is seamless.

Perverse but harmless (IMHO) side effect: If you enlarge then reduce
back to the same size, or do the opposite, the final result will not
be perfectly aligned with the original: it will be as if it was
slightly enlarged and then cropped. (Not surprising since my goal is
to "minimize the abyss contribution"/"minimize extrapolation".)

Cristy and Anothy:

Can I go ahead and modify the code?

This is a minor code change.

Nicolas Robidoux
Laurentian University

[fmw42]

Perhaps it might be better implemented as a -define or -set option, if that is possible.

I think it might be best to keep the reversibility option available.

Anthony has pointed out this issue in his discussion with -distort and -fx, but I don't think anything has been implemented in this regard, yet. It is a good idea as both types of resizing (and distortion operators) can benefit at times from both methods, depending upon the user's goals.

But I don't know how big an effort is involved.

Just my two pennies worth for the discussion.

Let's here from Anthony. Whatever he feels is best or practical is OK with me.

Fred

[NicolasRobidoux]

Opinions:

A switch would be nice.

The default should be:

For resize, switch depending on what is done in each direction (so, when enlarging in one direction and reducing in the other, both conventions would be used, depending on the direction).

For everything else, "match corners" (the current default) because the bad side effects of using "corners" when upsampling are milder than the side effects of using "centers" when downsampling.

Nicolas Robidoux
Laurentian University

[magick]

If you use a -define you can start coding now because your change will not activate without the option. If you want to do this automatically, let's wait for Anthony to comment.

[anthony]

I am not quite certain I follow the change. Though as Fred Weinhaus mentioned it does sound a lot like my distortion scaling discussion, where I worked out the correct way of doing -resize using -fx and distrotions.
Specificially this example from
http://www.imagemagick.org/Usage/transform/#fx

Code: Select all

  convert  rose:   -size 120x80 xc: +swap  \
           -fx 'v.p{  (i+.5)*v.w/w-.5, (j+.5)*v.h/h-.5  }' \
           fx_scaled.png

For those given access to my private development area (Fred, Nicholas, Cristy) a deeper discussion of this example is in
http://wraith.rcs.griffith.edu.au/~anth ... _notes.txt

The BIG difference between -resize and -distort however is that resize as no virtual pixel input, where distort does.
Careful testing of the change is vital. The current Resize method was from Paul Heckberts "zoom" program, and has been used for a long time. I am loathed to change it without good cause.

However if the change based on either a -define run time switch OR perhaps a #define compile time switch. Then I see no problem with trying it out.

I think a #define switch would be better in this case as it means we can at some point decide to remove one half of the change permanently. Where as a runtime -define will probably require both styles to be preserved and tested for LIFE!

This is why my change in Distort (from HQEWA to Clamped EWA) was done with a compile time #define as once the switch was finalised I wanted to full remove the older very blurry HQEWA code, and not have to try and preserve it later.

The important thing to remember is to ensure that before the commit, you make sure the old code is still worked as before, so that if it is decided not to go ahead, nothing is lost, and we don't need to try and recode the old code from the SVN.

[NicolasRobidoux]

I'll wait until I have a substantial quiet block of time to make changes. The "new" IM resize is more complex than the old version on which I did this switch years ago.

[anthony]

The only major change should be the way filter calls were made. The rest of the code should be essentially the same.

[NicolasRobidoux]

I'm going to take my time on this:

I'm realizing now that to do this just right, I need to be extremely careful. It's not only the sampling positions that need to be changed, but also the way the filter kernel is scaled depending on the "resize factors."

And there are other things to double check (cache...).

[anthony]

Do you consider it worth the effort? Either for IM users or for you personally?

[NicolasRobidoux]

1) Personal/professional: To do careful quantitative comparative testing of resampling methods, it is preferable to be able to switch image size convention (because some methods are more "friendly" to one or another of the conventions, and this levels the field; there is another reason, having to do with that reconstruction tests are better done with both box filtering and point sampling, and they each are more naturally done with one of the conventions). If I modify the code so it switches automatically, it is easy to modify it for personal use (or through a -define) to hard wire one or the other of the conventions.

In other words: I'll have to do it for myself and my students anyway. And I don't think I'm alone doing research in the field who'd like to control which convention is used. It's a small visual difference, but a huge one numerically when you compare methods that don't suck.

2) When enlarging in one or both directions, this will lead to a slightly better result, computed slightly faster. Esp. near the boundary. But yeah, it's definitely a tweak.

[NicolasRobidoux]

Note however that the difference jumps at you when enlarging a small image.

[anthony]

Yes that would be true. Okay then a run time -define would be better, over the #define compile time directive.

If the difference is quite large, it may be better to make new X and Y pass functions, so resize has two X pass and two Y pass functions depending on the -define setting.

That may be the cleanest method of implementation, as only one (the top level ResizeImage()) handles the -define. It would also probably make debugging easier too.

Of course if it is only a difference in the 'taps' than it may only need a small adjustment in one place. in each pass. In that case the internal function probably should have a boolean argument/variable/control to select which method to use, whcih is again controls from that top level ResizeImage() function. You do not want to lookup and string compare the define Artifacts for each and every row/column in the image.

Do you have something that explains the difference between the two resize 'tapping' methods?

[NicolasRobidoux]

Here are some examples of the difference between the two image size conventions (done with my first private pass at changing the code):

The 8x enlargement of the IM built-in rose picture:

Code: Select all

convert rose: -resize 560x560 rose_corners.png

resulted in

.

The resulf of the same operation with the "match centers" convention is

.

Maybe a -define option with no automatic switch because it may be a matter of taste which convention is better?

Although I know what I like:

"Mathematically", match corners is better for downsampling and match centers is better for upsampling because this eliminates extrapolation. But mathematics is not the answer to everything.

However, I think that "match centers" does look better for this simple enlargement, at least in the following sense: Near the top and bottom of the right boundary of the "match corners" (IM default) result, there are slightly too thick "flat" bands which do not have the same characteristic as the rest of the image.

Likewise, if you look really carefully at the bottom of the "match corners" result (IM default), you'll notice that basically the boundary pixels were simply "smoothly stretched." In other words, unless I'm imagining things, it is perceptible that the enlargement is not "a window into a reality which extends to infinity." Instead, one can faintly see that the reality from which the enlargement was created was itself a window of finite extent. This manifests itself in an increase in vertical features near horizontal boundaries, and vice versa. This is unavoidable to some extent, but one would like "feature statistics" to minimally vary as one approaches the boundary.

The "match centers" result does NOT give this impression: One really has the impression that one simply cropped a much larger image. In other words, the boundary effects are basically invisible.

And the fact that what is interesting in the image (what's in the interior as opposed to the boundary itself) is larger is an added bonus. Generally, the first/last row/columns of pixels (that is, the boundary pixels) are not where the action is. Why give it more space than necessary? When enlarging with the "match centers" convention, they are there, they are just not "thickened" as much as the interior pixels. There actually is a side effect of this which is really subtle in this test: To my eye, "match centers" actually is less aliased and has slightly less haloing even in the interior of the image. (I am not sure that this would still be the case with other enlargement ratios.)

I don't have an example off-hand, but enlarging an image with high frequency (but not necessarily tile: photograph of densely packed blades of grass, say) patterns near the boundary should make pretty clear that "match centers" is better: The current IM convention will "smooth out and stretch" the patterns near the boundary, while the "match centers" one will not stretch them and will smooth them out less. Another example of image making obvious the difference between the two would be one with a bunch of "rays" radiating from the center of the image. Near the boundary, the "rays" should become somewhat bent, becoming somewhat perpendicular to the boundary, and this effect will be more obvious with the current IM default. (Apologies for not taking the time to hunt for such an image and testing this myself right away.)

[NicolasRobidoux]

Things are a bit different for man-made images, because generally they are made with a priori awareness of the total extent of the image. For such (the dragon picture, say) I could understand one preferring the current IM default (match corners) even when enlarging, because it is better at preserving the perceived relationship between the size of objects within the canvas and the canvas itself, and because generally there is no action near the boundary for such images. (Playing devil's advocate, here.)

(current IM default)

(match centers convention)

Note how close to the bottom of the image the tail of the dragon gets with the "match centers" convention.

[anthony]

Hmmm.. doing the resize using the general distort...

The resize 'corners default' is the equivalent to this...
convert rose: +distort Affine '0,0 0,0 %w,0 560,0' show:

Note that this is a two point affine distort to preserve aspect ratio.
See http://www.imagemagick.org/Usage/distor ... ol_escapes

It may be however that the above only seems to match 'corners default' as Distort preserves the shading of the edges into transparency, where Resize does not.



Looking at the resulting image from your example the current default 'matched corners' still seems to be the more 'natural' method.

The other 'matched corners' seems to have a distinct loss of information around the edge. Sort of like the image was enlarged slightly bigger and trimmed.



However I can see that for some situations this change would be desirable, so I would make the -define available, but 'turned off' by default. That would make backward compatibility a non-issue as well.

If you would like to write a non-technical blurb, I will include that in the top section of IM examples, Resize Images. A quick 'reference' would also need to be added to the offical command line options page, as well as in the sources for the "ResizeImage()" function.

NOTE: this define would also effect other operations that call ResizeImages internally. For example "montage", -polaroid, as a couple that comes to mind.