-fx twice as slow with upgrade.

[dognose]

I've been testing out the new version, and I've noticed that while most features
process at about the same speed, -fx is noticeably slower. Over twice as slow in fact.

the function I'm using is this:
-fx 'v.p{0,u*v.h}'

My old version:
Version: ImageMagick 6.2.8 03/14/07 Q16 file:/usr/share/ImageMagick-6.2.8/doc/index.html

The new version:
Version: ImageMagick 6.3.8 01/30/08 Q16 http://www.imagemagick.org

both are x86_64 builds.. though I can't tell how to confirm that.

Is it a problem with my build, or is -fx just slower now? Is there any way to speed it up?

I was just learning about the Custom Image Filters option, but I really wouldn't know where to start with that.

[magick]

We switched from straight pixel interpolation to pixel resampling which is an improved algorithm but is generally slower. The -fx option is interpreted and therefore expected to be slow. What we have someone working on is an Fx compiler to speed up the operations or an option to convert the Fx expression to C code which could be compiled and loaded as an ImageMagick plug-in.

[dognose]

That would be great. I was just wondering if that would be possible.

I'm sure others would appreciate the filter for -fx 'v.p{0,u*v.h}' to recolor images.. it's in a bunch of the examples and I use it a lot. That and distortion maps (-fx 'p{v*w,j}') . I nominate those as an early test cases

Or, maybe someone here can translate those into Image Filters in C ?

[magick]

It would be quite simple to parse FX expressions into a C image filter and its something we've wanted to do for quite sometime. Its delay is simply the lack of time. We have over 300 items on our list of things to-do and in the mean-time we have questions, bug fixes, etc. There is no ETA on this problem right now but it will be available at some point it in the future.

[magick]

We tried -fx on a 3Ghz linux box against a 2736x3648 image and it completed in 3.7 seconds. What is the stats on your images and how long does it take -fx to complete?

[dognose]

Hmm, that's certainly faster than mine. On the test I used:

image1 1000x1000
image2 4x1
Intel(R) Xeon(R) CPU X3220 @ 2.40GHz x86_64 CentOs 5
(it's quad core, but I know it's only using one core)

with 6.3.8 it took 10.1 seconds
with 6.2.8 it took 4.6 seconds

[magick]

Are you compiling with -O3? We have a Redhat x86_64 with 2 x PE2950 Xeon 5160 3.0GHz/4MB 1333FSB and your command ran is less than a second.

[dognose]

Code: Select all

Host system type : x86_64-redhat-linux-gnu
                  Option                        Value
-------------------------------------------------------------------------------
Shared libraries  --enable-shared=yes           yes
Static libraries  --enable-static=yes           yes
Module support    --with-modules=yes            yes
GNU ld            --with-gnu-ld=yes             yes
Quantum depth     --with-quantum-depth=16       16
High Dynamic Range Imagery
                  --enable-hdri=no              no
Delegate Configuration:
BZLIB             --with-bzlib=yes              yes
DJVU              --with-djvu=no                no
DPS               --with-dps=yes                no
FlashPIX          --with-fpx=yes                no
FontConfig        --with-fontconfig=yes         yes
FreeType          --with-freetype=yes           yes
GhostPCL          None                          pcl6 (unknown)
GhostXPS          None                          gxps (unknown)
Ghostscript       None                          gs (8.15.2)
result_ghostscript_font_dir='none'
Ghostscript fonts --with-gs-font-dir=default
Ghostscript lib   --with-gslib=yes              yes
Graphviz          --with-gvc=yes                no
JBIG              --with-jbig=yes               no
JPEG v1           --with-jpeg=yes               yes
JPEG-2000         --with-jp2=yes                no
LCMS              --with-lcms=yes               no
LQR               --with-lqr=yes                yes
Magick++          --with-magick-plus-plus=yes   yes
OpenEXR           --with-openexr=yes            no
PERL              --with-perl=yes               /usr/bin/perl
PNG               --with-png=yes                yes
RSVG              --with-rsvg=yes               no
TIFF              --with-tiff=yes               yes
result_windows_font_dir='none'
Windows fonts     --with-windows-font-dir=
WMF               --with-wmf=yes                yes
X11               --with-x=yes                  yes
XML               --with-xml=yes                yes
ZLIB              --with-zlib=yes               yes
X11 Configuration:
      X_CFLAGS        =
      X_PRE_LIBS      =
      X_LIBS          = -L/usr/lib64
      X_EXTRA_LIBS    =
Options used to compile and link:
  PREFIX          = /usr
  EXEC-PREFIX     = /usr
  VERSION         = 6.3.8
  CC              = gcc
  CFLAGS          = -I/usr/include/lqr-1 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -O3 -g -pipe -Wall -Wall -W -pthread
  MAGICK_CFLAGS   = -I/usr/include/lqr-1 -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -O3 -g -pipe -Wall -Wall -W -pthread
  CPPFLAGS        = -I/usr/include
  PCFLAGS         =
  DEFS            = -DHAVE_CONFIG_H
  LDFLAGS         = -L/usr/lib64 -lfreetype
  MAGICK_LDFLAGS  = -L/usr/lib64 -L/usr/lib64 -lfreetype
  LIBS            = -lMagick -ltiff -lfreetype -ljpeg -L/usr/lib64/lqr-1 -L/lib64 -llqr-1 -lglib-2.0 -lfontconfig -lXext -lX11 -lbz2 -lz -lm -lpthread
  CXX             = g++
  CXXFLAGS        = -O3 -g -pipe -Wall -Wall -W -pthread

The move from -O2 to -O3 had a slight improvement to 9.7 seconds.

[anthony]

Ithe function I'm using is this:
-fx 'v.p{0,u*v.h}'

FYI, that function is now a fast builtin known as -clut. IM Examples was updated to demonstrating this builtin for color table lookup
http://imagemagick.org/Usage/color/#clut

[dognose]

Ahh. -clut is the answer. it's much faster for me (down to 3 seconds) I have to look into it more as it's got the interpolation thing too.

[anthony]

What actually happened...

Interpolation is the original routines that was used to do color lookups from images where the lookup could be a floating point number falling between two pixels (the interpolation). No scale or other factors provided in the lookup. Just a single floating point coordinate into the source image.

However as I was developing a General Image Distortion routine, interpolated lookup was not doing to cut it. Basically as a image gets smaller (scaled smaller because of the distortion, such as when you View Distant Horizons) You do not want just a single 'point' lookup, but want a color of a larger area of the source image. That is a 'scaled' or 'resize filter' lookup of the color. IE a point and a 'scale'.

For example instead of

as you would get with a simple 'point' interpolated lookup, you want to get something like

which is a scaled 'area re-sampling' pixel lookup. As You can see it was vital to getting higher quality distorted images. (Quality and Speed of this 'horizon' type image will be improved further later)

This code also included normal interpolated lookup as part of its process (a -filter point mode - see the IM examples Resize Filters discussion)

That resampling code I developed for -distort had a much more well defined API interface, so Cristy rolled them together as a common interface for a lot of other image processing routines, such as -fx even though they don't need or make use of the full area re-sampling aspects, as they do not have any scaling factors as part of the lookup process.

I did not ask for this, nor did I expect this to happen, as they are very distinct modes of pixel color lookup. But I understand his motives on the matter.

NOTE I am not finished with the re-sampling code, having put it on hold while I completed an overhaul of the resize filter code (which was needed for further development). Currently I am taking a break from IM coding, so I can do other things (I have a life!) but when I get back to it I will look at streamlining the interface for unscaled 'point' filtered lookups.

However I have no ETA on this. It probably will not happen until I get back from Kiteflying in China this next April.

[fmw42]

clut is definitely the best thing for this kind of thing as it does the same but is a real compiled function, but as a point of information, you might want to try

... -filter point -fx ...

and see if that speeds things up again for fx

[dognose]

Wow, that's pretty interesting, thanks Anthony

the -filter point didn't seem to help at all on this one.

The various -interpolate options have a huge effect on the speed of the command though. Luckly I can use the fastest option (NearestNeighbor or Integer) which cut the execution time in half.

Thanks again for all your help. I'm down to 1.6 seconds from 10.6 seconds on a heavily used routine.

Will the -interpolate option affect most or many commands? Can I change the default method systemwide? I have a lot of other process running slower as well that I need to work on.

[anthony]

Will the -interpolate option affect most or many commands?

Mostly distort type commands, like -fx, -swirl, -implode, -clut. Things that lookup colors from a 'floating point' position. Resize Images use a more powerful but slower resize filter type system. It is closely related to interpolation but involves a 'scaled' lookup, rather than a 'unscaled' lookup.

Can I change the default method system wide? I have a lot of other process running slower as well that I need to work on.

No. not without changing the source code.