Legacy ImageMagick Discussions Archive

Hello forum,
I just wondered if my imagemagick is slow or if just the task is too big.

I have an Ubuntu 64bit running on a 8x4Core (yes, 32 Cores!) Xeon Server with 256GByte Ram.
So this machine is top of the range I ever worked with.
I am running :

Code: Select all

convert -version

Version: ImageMagick 6.6.9-7 2012-08-17 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2011 ImageMagick Studio LLC
Features: OpenMP

I have an image with 25.000 x 4000 pixel in cmyk-tif
I have to resize it to 125.000 x 20.000 pixel still in cmyk-tif

Both images a located on a ramdisk, so this is no harddisk speed issue.

I do a standard resize without any options, so it is done with the standard bicubic resize and the result is just what I want.
The new image is around 10GByte of Data, so it is miles away of a swapping problem and in fact the monitoring of the memory usage is just in this area.
But I need more or less two minutes for this resizes command.
Monitoring the CPU usage shows a max usage of 16% in total, so there is space for improvement.
Is the time of two minutes "normal" on a machine like this?
Is there a chance to change the settings for a speed improvement?

Or (this is just 1% of the total work, all other stuff is also done with imagemagick, so this would be ok...)
is there another program which would do this specific job faster?

Thanks for your help!
Roland

Let's try a few things. The resize algorithm requires two intermediate staging buffers, let's use an algorithm that only requires one:

time convert cmyk.tif -sample 125000x4000 resize.tif

Next, try reducing the number of threads:

export MAGICK_THREAD_LIMIT=16
time convert cmyk.tif -resize 125000x4000 resize.tif

If you get a performance boost, the default 32 threads are too many. If this does not help, chances are the pixels are being cached to disk. Caching to disk is single threaded and two orders of magnitude slower than memory. To verify lets check with cache debugging:

time convert -debug cache cmyk.tif -resize 125000x4000 resize.tif|& grep -i open

We get

time convert -debug cache -define registry:temporary-path=/data/tmp cmyk.tif -resize 125000x4000! resize.tif |& grep -i open
2013-02-17T09:53:19-05:00 0:00.000 0.000u 7.0.0 Cache convert[9755]: cache.c/OpenPixelCache/3526/Cache
open cmyk.tif[0] (Heap Memory, 25000x4000x4 1.4901GiB)
2013-02-17T09:53:21-05:00 0:01.900 1.850u 7.0.0 Cache convert[9755]: cache.c/OpenPixelCache/3526/Cache
open cmyk.tif[0] (Heap Memory, 125000x4000x4 7.4506GiB)
2013-02-17T09:53:34-05:00 0:14.810 123.930u 7.0.0 Cache convert[9755]: cache.c/OpenPixelCache/3667/Cache
open cmyk.tif[0] (/data/tmp/magick-9755_oS0gOZzieLW[-1], Map, 125000x4000x4 7.4506GiB)

Which confirms the pixels are cached to disk. If so you may get a modest performance boost with ImageMagick 6.8.3-1, the current release. It uses 2 threads to cache to and from disk. On our system we increased the cache memory limits so it would keep the pixels in memory:

export MAGICK_MEMORY_LIMIT=24GB
identify -list resource

Code: Select all

  File       Area     Memory        Map       Disk   Thread  Throttle       Time
--------------------------------------------------------------------------------
   768   25.185GB  22.352GiB  23.455GiB  unlimited       12         0  unlimited

Now the same command caches to memory:

time convert -debug cache -define registry:temporary-path=/data/tmp cmyk.tif -resize 125000x4000! resize.tif | & grep -i open
2013-02-17T09:55:45-05:00 0:00.010 0.000u 7.0.0 Cache convert[9802]: cache.c/OpenPixelCache/3526/Cache
open cmyk.tif[0] (Heap Memory, 25000x4000x4 1.4901GiB)
2013-02-17T09:55:47-05:00 0:01.920 1.870u 7.0.0 Cache convert[9802]: cache.c/OpenPixelCache/3526/Cache
open cmyk.tif[0] (Heap Memory, 125000x4000x4 7.4506GiB)
2013-02-17T09:56:00-05:00 0:14.810 122.060u 7.0.0 Cache convert[9802]: cache.c/OpenPixelCache/3526/Cache
open cmyk.tif[0] (Heap Memory, 125000x4000x4 7.4506GiB)

On our 32GB system with 3GHZ x 4 Xeon chips, it ran in just over a minute-- that's on 4 threads. We would expect better results with an increase in threads.

If you are forced to cache pixels to disk, you can cache the pixels to your RAMdrive with the -define option as above, e.g. -define registry:temporary-path=/myramdrive. Although with the amount of memory on your system, it should cache all pixels to memory instead of disk.

There are a huge number of computation going on when resizing. A fast chip can make the difference in performance.

Legacy ImageMagick Discussions Archive

Resize speed

Resize speed

Re: Resize speed