AffineImageTransformation broken

[provos]

AffineImageTransformation is still broken for transformations that rotate and translate at the same time:

Code: Select all

  if ((affine_matrix->sx*affine_matrix->sy) > 0)
    {
      transform.tx=0;
      transform.ty=0;
    }
  if ((affine_matrix->tx == 0.0) && (affine_matrix->ty == 0.0))
    {
      transform.tx=(-min.x);
      transform.ty=(-min.y);
    }

should be replaced with

Code: Select all

 transform.tx -= min.x;
 transform.ty -= min.y;

Niels.

[anthony]

We know about this and it is at the top of the to-do list. Even before a perspective transform builtin.

Thanks for the patch, I am sure cristy will put it in quickly, though I have a feeling it may not be the only thing wrong. How must testing have you done?

I have a slow DIY version that does work correctly detailed in IM Examples, under both Distorting Images and Bugs, Testing. It was hoped that the DIY version would not only explain the internals of affine transforms, but also help bug fix it.

[provos]

I did no testing on the patch whatsoever but looked at the code a long time last night. Unfortunately, changing ImageMagick itself was not an option so I had to reimplement using DrawAffineImage.

Niels.

[anthony]

After some experimentation I found this is also not quite correct.

Correct solution is to add offsets to the bounds determination code first,
then use the transformed location of the 0,0 pixel as part of the translation modification.

I also added a few comments to the code, tested and commited the change to the Subversion Code.

Hopefully by tomorrows build it will be in the beta testing site. But it does now seem to be working.

PS: I also updated the Affine DIY section of IM exampes to show this merger of image offsets into the affine matrix to allow direct (no offset) image to image translation.

Thank you for your efforts.

[provos]

Thank you very much for fixing. I guess I did not completely understand how the extent stuff fit in there.

Niels.

[anthony]

The extent stuff was for determining the bounds and position of the destination image, but it ignored the offset of the source image.

My use of extent[0] was as that was the translated location of the real image pixel 0,0, allowin me to determine direct image to image translates which included the offset info.

Note that the DIY one is EXACLY the same, just going from destination to source.

The Draw Affine routine does the affine matrix inversion internally and I copied the formual I found in there today onto the DIY page. Just what I was lookign for to complete that IM Examples section.

Now to do something simular for perspective transforms. Though the question is should IM be given the 16 box arguments, or the 8 perspective matrix arguments.

[rmabry]

Now to do something simular for perspective transforms. Though the question is should IM be given the 16 box arguments, or the 8 perspective matrix arguments.

It would be cool if there were a few ways to give the args. Another nice thing would be to be able to feed it the coordinates of four points on the image that should be mapped to four points on the output. There would need to be some error checking, of course, for impossible requests.

By the way, are there limits given in the case of this or the affine transform that prevent someone for asking (inadvertently) for an image that is ridiculously tall or wide?

Rick

[anthony]

For Affine transforms and other options I am currently creating 'helper' scripts to convert one argument format to another. But if perspective 'matrix' input is going to be used, then IM needs to know some why of inverting a perspective matrix. Currently I only have a helper script to generate the perspective matrix arguments from 2 x 4 coordinate pairs (eg 16 numbers to 8 numbers). A problem.

IM has (and I now detail on IM Examples) a affine matrix inverter which it uses internally
in 'draw.c'.

No no limits. I for example use to use -tile 99999x1 in monatge (version 5), but that produces a recidulious image in IM v6. No limits, the user may in fact really mean it
A 'zero size' limit may be a must though!

In the mean time I have downloaded and built the latest SRC RPM from the alpha area, and ran it through all IM Examples. I can now say that Affine Transformations has passed ALL my tests.

The only thing that I have left is some posible adjustments to the bounds, for correct float to interger adjustments as the image size and canvas offsets can only handle integer values. This however would only result in minor (less than a pixel) adjustments to the grossly correct results it is now achieving.

That is the final image may need to be up to 2 pixels larger to prevent the corners from being clipped, and the offsets may need to be one pixel less. The translation adjustments however should remian about right.

Can you please take a look and see what you think.

[rmabry]

In the mean time I have downloaded and built the latest SRC RPM from the alpha area, and ran it through all IM Examples. I can now say that Affine Transformations has passed ALL my tests.

Which version is that? I grabbed a beta yesterday (6.3.2-1) and don't want to rebuild if I don't need to.

The only thing that I have left is some posible adjustments to the bounds, for correct float to interger adjustments as the image size and canvas offsets can only handle integer values. This however would only result in minor (less than a pixel) adjustments to the grossly correct results it is now achieving.

That is the final image may need to be up to 2 pixels larger to prevent the corners from being clipped, and the offsets may need to be one pixel less. The translation adjustments however should remian about right.

For me the issue of the bounds is very important. Just lately I have relied on being able to calculate the positions of the points (including the four corners of the image) as they are transformed by the affine matrix given to AffineTransform. I do this to know where to crop an image to extract certain parts after the transformation. I need it accurate to the exact pixel.

This sort of thing will need to be stable before I can use it reliably. One thing I noticed is that I will have to take into account the fact that there is rounding in the ImageMagick code for treating a pixel with coordinates (x,y) as being centered at (x+.5, y+.5). I hope that has remained consistent. Adding pixels around the edges to fix certain jaggies is one way to go, but can't it also be handled using the virtualpixel notions?

In any case, all I need is a solid, stable, clear way to predict where a pixel will end up following an affine transformation.

Can you please take a look and see what you think.

I will, just tell me the first version it shows up in.

Thanks for your work on this, Anthony, I know this is an important matter to finish. But please be careful that it is mathematically predictable for delicate use!

Are you making changes to any of the other functions in that family, such as Rotate, XShear, YShear?

Rick

[anthony]

The one I downloaded was yesterday (sunday Australian time), and was marked 6.3.2-1 in the alpha area. If the creation date is or after 22 january, you should be fine.

The .5 rounding I think is actualy more because the bounding boxed are not checked properly. The best test would be to try a null transform to make sure it comes out right.
I am not ceratin it is consistant throught IM, as it seems to get added as problems are pointed out. I believe it was actually added before we even had pixel interpolation, in which case it may be now a mistake due to advances in other areas.

I'll have to study it more carefully, for bounding boxed but no longer have much time. Whcih is why I asked you to take a look at you obviously have a stronger interest in this.

I have been readin about the 'mesh' interpolation, (currently broken, see IM Examples Bugs, Testing) and once it is fixed I may be recomending it as the better interpolation for any form of rotates or distionts.

[rmabry]

The best test would be to try a null transform to make sure it comes out right.

With the default interpolation (bilinear), the null (identity) transform comes out exactly correct. So do flips and flops and 90-degree rotations, all a good sign. X- and y-shears look perfect. Pure scaling also looks good to me.

Bicubic interpolation gives nice, symmetric results with the identity transform, with a slight decrease of total pixel rgba values due, I gather, to rounding issues. (I haven't looked very deeply.) So, for instance, if a single blue pixel (0,0,255) on a black background undergoes an identity transform, under bicubic interpolation it is smeared out among its home pixel and its 8 neighbors, the sum of the blue components coming back as 253. Not bad, I guess. (What an interesting problem interpolation is. Mathematically it is nothing difficult, but deciding how to make things look best is much harder. How best to smear out a single isolated blue pixel so that it looks the same from a distance? Changing to red probably calls for a different solution, ideally.)

I have been readin about the 'mesh' interpolation, (currently broken, see IM Examples Bugs, Testing) and once it is fixed I may be recomending it as the better interpolation for any form of rotates or distionts.

In the meantime (and in any case), shouldn't AffineTransform in PerlMagick have an interpolate option?

Rick

[magick]

In the meantime (and in any case), shouldn't AffineTransform in PerlMagick have an interpolate option?

Done. ImageMagick 6.3.2.-8 Beta available sometime tommorrow.

[rmabry]

X- and y-shears look perfect.

I should have said this modulo interpolation. With very stretchy shears, the interpolation has a lot to do with the results, obviously. The "perfect" aspect only meant with regard to positions of the (centers of the) transformed pixels, my particular first concern, because I want to be able to predict exactly where everything will appear.

And I'm not concerning myself with the translation part, which is interesting in terms of its difficulties. At present, how would someone translate an image 1/2 a pixel to the right, for instance? (Never mind why someone might want to do so or how awful the result might look...) It seems to me that it cannot be done with AffineTransform.

Among the various interpolation schemes, I don't see one that would strike me as the most natural for an affine or perspective transform, namely one that would use the inverse transform of the boundary of each individual pixel to gather the exact areas (to be used as weights) cut from each pixel in the inverse-transformed region. If such code were written even reasonably well, it might be slower by far than the bilinear and similarly crude methods, but certainly faster than the filter method (which must do a CropImage for each pixel, if I am not mistaken). Is there a name for such an interpolation scheme? Perhaps I haven't described it sufficiently well.

Thanks for adding the -interpolation option to PerlMagick. Can a -color (-fill) option also be added (as with Rotate)?

Rick

[magick]

We added the background option to AffineTransform() in ImageMagick 6.3.2-2 Beta available sometime tommorrow.

[anthony]

Interpolation is very difficult, and I am trying to study and get 'mesh' interpolation working.

This should produce much sharper edges for rotated images as it subdivided the inter-pixel area in into two triangles so as to miniize the slope for at least one triangle. That is produce a sharp diagonal when one pixel is very different from the others.

Bilinear in contrast just averages out all the pixel colors in the very center of the inter-pixel space, whcih sort of 'blurs' a 'hash' or 'dither pattern' in images.

----

As for the next step in testing, I would try very very slight partical pixel level translations
of a single pixel image and check the look and size of the image at various numbers. EG: .3 .5 .8 etc and even small negative movements, to ensure the final image size and placements and the look of the single 'pixel' is also correct.

If that works then the low level stuff should also be right. However a quick look

Code: Select all

    convert xc:blue -affine 1,0,0,1,.3,0 -transform info:
    xc:blue XC 1x1 1x1+0+0 DirectClass 16-bit 

Also

Code: Select all

convert pattern:gray50 -interpolate bilinear -affine 1,0,0,1,.5,0 -transform x:

shows that the translation does not enlarge the destination image correctly as it remains a single pixel in size. Of course as the size is wrong the single pixel lookup is not a proper representation of the result, and the 'hash' shows no sign of the bilinear interpolation, whcih should produce an 'averaged grey' in the last example (I think).