convert creates a 4.5GB temp file and very slow to process

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
webereinc

Re: convert creates a 4.5GB temp file and very slow to process

Post by webereinc »

I, too, am having a problem with convert taking an exceptionally long time converting > 1MB tiffs into any other type of format (e.g. jpg or gif) even if I try to do a resize or depth reduction. Is there any help out here?
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: convert creates a 4.5GB temp file and very slow to process

Post by magick »

Have you read http://www.imagemagick.org/script/architecture.php? If an image resource demands exceed the limits of your computer, the image pixels are cached to disk which can be 1000 times slower than in memory. You can see if this is the case by adding -debug cache to your command line and look for the pixel cache being opened on disk. A fix is to add more memory to your computer. Also be sure you are using a modern version of ImageMagick which include numerous performance improvements and support for running algorithms in multiple cores in parallel.

If you post a URL to your image and the ImageMagick command line you are using, we will download it and see if we can reproduce the problem.
webereinc

Re: convert creates a 4.5GB temp file and very slow to process

Post by webereinc »

Hey magick,

After reading the writeup of the architecture, I confirmed that the convert process is opening the pixel cache on disk as suggested. A sample of the types of TIFs I need to convert can be found at http://www.weberenterprises.com/images/plat.tif

The command I am using is: convert -resize 38% plat.tif plat.gif

I am trying to resize large scanned black and white tif images to 8.5" x 11" pages that can be served up via the web.

Sorry for hi-jacking this thread - please let me know if I should start a new one.

Thanks
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: convert creates a 4.5GB temp file and very slow to process

Post by magick »

We're using ImageMagick 6.5.3-10 and your image converted to PNG in 18 seconds on our 3Ghz Linux box with 2GB of memory. It took 36 seconds to convert to GIF. However, it only took 3 seconds if we resized with -sample rather than -resize.
webereinc

Re: convert creates a 4.5GB temp file and very slow to process

Post by webereinc »

Hey Magick,

Thanks for the feedback... Some more details:

I am developing the app on a Windows box, but it will be ultimately deployed to a Linux server. The laptop is only a 1.8 GHz machine with 1GB of memory. The server is more robust.

Would it be beneficial to deploy the Q8 version as opposed to the Q16 considering that all of these images to be converted are either scanned in text documents or scanned in black-and-white plats and subdivisions? Is there a way to force convert to NOT do full 16-bit processing of the pixels considering that the source images are black-and-white only? I've tried setting the input parameters with no success.

I will try pulling down the latest version (I currently have Version: ImageMagick 6.5.0-0 2009-03-09 Q16 OpenMP http://www.imagemagick.org installed) and see if that makes any difference.

Thanks in advance!

- Ed
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: convert creates a 4.5GB temp file and very slow to process

Post by magick »

The Q8 version of ImageMagick uses 1/2 the resources of Q16 for memory and disk. Use it to conserve memory or disk and speed up processing. However, you will lose some accuracy and you will not be able to process 16-bit images-- well you can but they will first be reduced to 8-bit pixels internally.

Our aspiration is to create a bit-depth neutral pixel cache but so far we have not done it because in most cases it would require extra processing time switching bit depths between the pixel buffer and the pixel cache and we do not want to slow down ImageMagick any further. The upside is that you could select the bit-depth you needed at run time including run-time support for HDRI images. Currently you must pick a bit-depth and whether to use HDRI at compile time.
webereinc

Re: convert creates a 4.5GB temp file and very slow to process

Post by webereinc »

Using the Q8 version with the -resample vs. -resize parameter greatly reduced the processing time on my laptop development environment! Now, how do I go about getting the Q8 version installed on my Ubuntu server? The Q16 is currently installed.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: convert creates a 4.5GB temp file and very slow to process

Post by fmw42 »

I am not an expert at this, so all I can do is point you to documentation. see http://www.imagemagick.org/script/advan ... lation.php
Post Reply