fft not working in IM 6.6.2.2 HDRI

[fmw42]

IM 6.6.2.2 Q16 HDRI Mac OSX Tiger.

I was just notified that several of my scripts for processing images with FFT are not working. I don't know how many releases this has been a problem since I have not tested FFT (either in HDRI or non-HDRI in a while). I am not sure if these are related to recent changes in HDRI to do clipping or changes in the way you do rounding (recent topics in the forum).


There are several issues

1) output image types that do not support multi-images, such as pfm (png), no longer create the second image automatically from either -fft or +fft without adding +adjoin. This was not the case earlier. However, I can modify my scripts to deal with that if it is something hard to put back. Let me know.

2) A simple round trip (using either -fft -ift or +fft +ift) with no stored intermediate images seems to work fine.

input: square31.png


convert square31.png -fft -ift square31_fftmp_rt.png

compare -metric rmse square31.png square31_fftmp_rt.png null:
0 (0)

convert square31.png +fft +ift square31_fftri_rt.png

compare -metric rmse square31.png square31_fftri_rt.png null:
0 (0)


3) However when saving an output image with -fft and looking at the results there are issues:

convert square31.png -fft +adjoin square31_fft_mp.png
convert square31_fft_mp-0.png -evaluate log 10000 square31_fft_mp_spec10000.png

The phase image (-1) looks like this:

and it should look like this:


The spectrum generated from the magnitude (log processed as the magnitude is too dark) looks like this:

but it should look more like this:


4) When doing a round trip with -fft -ift (for magnitude and phase) and saving the intermediate results as pfm or miff, the result seems fine, EVEN THOUGH THERE SEEMS TO BE A PROBLEM WITH MAG/PHASE AS DESCRIBED IN 3) above. Don't understand this.

convert square31.png -fft +adjoin square31_fft_mp.pfm
convert square31_fft_mp-0.pfm square31_fft_mp-1.pfm -ift square31_fft_mp_rt_pfm.png
compare -metric rmse square31.png square31_fft_mp_rt_pfm.png null:
0 (0)

likewise with miff

convert square31.png -fft square31_fft_mp.miff
convert square31_fft_mp.miff -ift square31_fft_mp_rt_miff.png
compare -metric rmse square31.png square31_fft_mp_rt_miff.png null:
0 (0)

5) But it fails terribly when using +fft +ift (for real and imaginary)

convert square31.png +fft +adjoin square31_fft_ri.pfm
convert square31_fft_ri-0.pfm square31_fft_ri-1.pfm -ift square31_fft_ri_rt_pfm.png
compare -metric rmse square31.png square31_fft_ri_rt_pfm.png null:
8230.5 (0.125589)

likewise with miff

convert square31.png +fft square31_fft_ri.miff
convert square31_fft_ri.miff -ift square31_fft_ri_rt_miff.png
compare -metric rmse square31.png square31_fft_ri_rt_miff.png null:
8230.5 (0.125589)

Let me know what I can do to help track this down further or test any fixes.

Fred

[magick]

We get the expected phase image (-1). Not sure why it fails for you.

Go to http://sourceforge.net/projects/imagemagick/ and grab a couple of legacy releases and see if you find a release that produces the expected results. If so, we know we introduced a bug in recent releases and we can compare the old results to the new to track the bug. If you do find a release that works, post it here and we'll get to work tracking the bug. We tried a few releases ourselves and each produces similar results to your posting which suggests the bug has been in ImageMagick all along. If you decide the bug has been in ImageMagick since we first started supporting Fourier transforms, let us know and we will investigate further.

[fmw42]

I will go back some releases and test. But I know that it was working when I made all my scripts and when I made the Examples page for Anthony and the version on my web site. So I expect I will be able to find it. It just may take some time for me depending upon how many releases back I need to go.

Do you just need to know any one release that works or the most current one that works?

My examples were made about july/aug 2009, so I will start with 6.5.4.10 which was about then.

[magick]

Do you just need to know any one release that works or the most current one that works?

Use a binary search method. The most recently version that works is best because we can then use a diff on the source modules that work against the current release to spot the patch that broke Fourier transforms.

[snibgo]

(Has anyone thought of using the examples pages etc as a regression test suite? Then new releases could be readily tested for breaking old functionality.

Just a thought, and probably a stupid one.)

[fmw42]

Not stupid. I was going to ask Magick what he does for testing. Probably just a simple round trip, which seems to work for some reason. Seems to be saving fft to intermediate images and then ift on those files that fail. Perhaps he needs to do that at a minimum.

I think Anthony may use his page at http://www.imagemagick.org/Usage/fourier/ and at http://www.imagemagick.org/Usage/fourier/fft_math/ to make sure things are up to date and working. I have sent him a message to see if he is having trouble, though his pages look fine as far as I can see.

[magick]

We can reproduce the problem you posted and will have a patch within the next few days. Thanks.

[fmw42]

Magick points out a typo in my command above. Rather than

convert square31.png +fft +adjoin square31_fft_ri.pfm
convert square31_fft_ri-0.pfm square31_fft_ri-1.pfm -ift square31_fft_ri_rt_pfm.png
compare -metric rmse square31.png square31_fft_ri_rt_pfm.png null:

it should be

convert square31.png +fft +adjoin square31_fft_ri.pfm
convert square31_fft_ri-0.pfm square31_fft_ri-1.pfm +ift square31_fft_ri_rt_pfm.png
compare -metric rmse square31.png square31_fft_ri_rt_pfm.png null:

Nevertheless it still does not give 0 (0) as it should though a different value than 8230, more like 5553

[fmw42]

1) output image types that do not support multi-images, such as pfm (png), no longer create the second image automatically from either -fft or +fft without adding +adjoin

Seems I am in error here, too.

My testing does show that pfm does support multi-image format, so no +adjoin is needed.

My memory must be going! (Senior Moment). Also getting sloppy and not even double checking my own notes.

My apologies to Magick for these two errors on my part.

[fmw42]

These bugs seem to be fixed per my testing in IM 6.6.2-3 Q16 HDRI beta. Namely, the round trip issue and my scripts now work correctly again.

Thanks, Magick, for the fast response and hard work.

Fred

[anthony]

(Has anyone thought of using the examples pages etc as a regression test suite? Then new releases could be readily tested for breaking old functionality.

Just a thought, and probably a stupid one.)

I used to do this all the time, but small changes such as font handling, or GIF file quantization/dithering can pop up a huge number of 'major' changes, and it gets tiring comparing old and new images.

I occasionally re-build examples in various pages to test and check for changes, especially in major pages at the top of the Examples index. But I don't do this not all the time.

Building the latest development (SVN) release of IM, and I'll then run it though the Fourier page.
Last done (shown at bottom of page) 7 May 2010 so it was not really a long time ago.


However, this page does not include multiply/divide using either magnitude/phase (non HDRI capabile) or real/imaginary (HDRI only) FFT image styles. These were demonstrated in a sub-page
http://www.imagemagick.org/Usage/fourier/fft_math/

That was last processed 6 October 2009 using IM v6.5.6-9

Proper IM examples of this usage is waiting for the next stage of FFT development, the addition of a set of -fourier and +fourier methods for applying FFT specific operations on either real images or FFT converted images. (Currently just a proposal and waiting a programmer).

[anthony]

Re-building the FFT examples using the current SVN beta v6.6.2-3 of today...

No major differences seen. (many minor ones in text and slight format changes) but nothing to do with IM HDRI.

Looking at the FFT math sub-page...
Only changes in text font!

No problems can now be found. Presumably Cristy has fixed the problem in the SVN.

[fmw42]

I just finished testing all my FFT HDRI scripts with IM 6.6.2.3 HDRI and they all seem to be working fine again.

Thanks again Magick.

[ccp]

Hi,
I am an IM newbee experimenting with your scripts and reading your FT tutorial which I find very useful. Thanks first of all.

Now I came along the same problem where an FFT round trip (as separate steps) does not really work. I am using
ImageMagick-6.6.3-Q16
which I downloaded from
http://www.imagemagick.org/script/insta ... hp#windows
and then compiled all together with fftw.

May you point me to the version/source which worked fine for you.

Conrad

[fmw42]

It has been a while since I last tested and there were some issues a bit ago. But I do not recall the versions. However, I need to go out for a while this morning, but I will test again this afternoon.

What image type are you saving the intermediate and final results?

Are you using -fft -ift or +fft +ift?

Perhaps you should show me your exact command line?

Also, are you in Q8, Q16 and with or without HDRI?

Fred