Extremely slow TIFF conversion (Ubuntu)

[slowtiff2]

I am seeing general slowness with ImageMagick, sometimes a coredump, and seems to really only happen when dealing with TIFF files.

This occurs from the command-line and using PHP.

I read about a possible issue with OpenMP, and so compiled from source "--without-openmp".

So two questions:

• how do I verify that OpenMP is "off"?

• General request for help troubleshooting.

This is a VM running on Azure.

Distributor ID: Ubuntu
Description: Ubuntu 16.04.5 LTS
Release: 16.04
Codename: xenial


Here's a command showing how long it's actually taking. The source TIFF is black and white, and 2mb in size.

Attempting to convert causes "Aborted (core dumped)"

Code: Select all

time /usr/local/bin/convert -monitor +dither -map pattern:gray50 339483.tiff out.jpg
load image[339483.tiff]: 19600 of 19601, 100% complete
load image[gray50]: 31 of 32, 100% complete
classify image colors[gray50]: 31 of 32, 100% complete

Aborted (core dumped)

real    12m25.320s
user    0m2.162s
sys     0m2.896s

Here's a command getting image height:

Code: Select all

time /usr/local/bin/convert 339483.tiff -format "%w x %h" info:
22907 x 19601
real    0m15.548s
user    0m0.677s
sys     0m0.981s

edit:
Version: ImageMagick 7.0.8-14 Q16 x86_64 2018-11-19 https://imagemagick.org
Copyright: © 1999-2018 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI
Delegates (built-in): bzlib djvu fftw fontconfig freetype jbig jng jpeg lcms lqr lzma openexr pangocairo png tiff wmf x xml zlib

[snibgo]

What version of IM?

how do I verify that OpenMP is "off"?

Code: Select all

convert -version

Look under "Features:", and check for "OpenMP".

Your tiff is quite large, 22907 x 19601 pixels. With IM v6 Q16 integer, this needs 6 GB memory. If you don't have enough, this should use disk, which is slower but shouldn't cause a crash.

[fmw42]

how do I verify that OpenMP is "off"?

You can also just limit the threads to 1.

Code: Select all

MAGICK_THREAD_LIMIT=1 convert ....

or

Code: Select all

convert -limit thread 1 ....

[slowtiff2]

Thanks to both of you.

I have confirmed that OpenMP is not listed in the features (see my edit above with output of -version).

Including "-limit thread 1" seems to run faster (but I have a very small number of tests) so I will look at adding that to the overall configuration.

Here's the crappy part:
Overall, everything related to ImageMagick seems faster today. Was I blaming ImageMagick when really it was something else? quite possible, though I do not know what that could have been.

Will ImageMagick automatically use disk if RAM is not available? I wonder if there was some slowness with disk I/O? Unfortunately this is an azure VM and I have zero control (and really no access) to see what was going on.


/dev/sda:
Timing cached reads: 16908 MB in 1.99 seconds = 8482.64 MB/sec
Timing buffered disk reads: 98 MB in 3.05 seconds = 32.15 MB/sec


I guess I continue on and see if it comes up again.

Thank you again... very appreciative of your fast response and your time helping.

[snibgo]

If your IM doesn't have OpenMP, then it doesn't multithread, and "-limit threads 1" will make no difference.

On a virtual machine, you share physical resources with other processes. So performance (as measured by a wall clock) will vary.

If you don't have enough RAM for pixel caches, IM will use disk, provided disk usage isn't limited by policy.xml or elsewhere. Use "-list resource" to see the limits.

Disks that physically spin are massively slower than RAM. And this could be worse on a virtual machine -- many processes may be competing for the slow disk access.