Problem converting large JPGs

[pnt103]

I'm trying to resize several large B&W images using convert, but it keeps core-dumping. Small files work OK, though. Typical image file is around 12MB, 19200 x 13100 pixels, 16-bit greyscale, JPG format. This is using ImageMagick 6.6.9-7 on a machine running Ubuntu 12.04.5 LTS, with 6GB RAM and 1.8TB of free space on the disk partition I'm using.

I've checked ulimit, and there are no limits on memory or disk use, tried setting MAGICK_TMPDIR to the directory I'm in (and which the images are in) so there's plenty of disk space available. Here's what happens:

$ convert -debug cache LR01.jpg -resize 900 LR01_900.jpg
2018-08-01T17:08:27+01:00 0:00.000 0.000u 6.6.9 Cache convert[26572]: cache.c/DestroyPixelCache/1540/Cache
destroy
2018-08-01T17:08:27+01:00 0:00.000 0.000u 6.6.9 Cache convert[26572]: cache.c/ExtendCache/4020/Cache
extend LR01.jpg[0] (/raid2TB/pete/tmp/magick-0iQ1wOxO[4], disk, 2.5423GBB)
2018-08-01T17:08:27+01:00 0:00.000 0.000u 6.6.9 Cache convert[26572]: cache.c/OpenPixelCache/4246/Cache
open LR01.jpg[0] (/raid2TB/pete/tmp/magick-0iQ1wOxO[4], disk, 19285x13183 2.5423GBB)
Segmentation fault (core dumped)

The ultimate goal is to create two reduced-size copies for web use, one about 900 pixels wide for display, and another as a smaller thumbnail. What am I missing?

[fmw42]

What is your version of libjpeg? You can find that from

Code: Select all

convert -list format

Perhaps you need to upgrade it.

Otherwise, check your policy.xml file for other limitations such as on width or height or memory, etc, though you seem to have perhaps checked that already.

If you want someone to test, then post your input JPG to some free hosting service that will not modify it and put the URL here.

On Linux, the version 6.6.9.7 would appear to be ancient. But they do not change versions when they patch. So the more important fact would be the date associated with the version. Be sure you have the latest patch from your Linux distro.

[magick]

Agreed. Try upgrading to the latest release. If that fails, let us know and we will investigate further.

[pnt103]

Thank you replying so quickly.

"convert -list format" just lists (a lot of) formats without giving versions:

Code: Select all

JPEG* JPEG      rw-   Joint Photographic Experts Group JFIF format (80)
JPG*  JPEG      rw-   Joint Photographic Experts Group JFIF format (80)
JPX*  JP2       rw-   JPEG-2000 File Format Syntax

My libjpg is /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2

ImageMagick itself is 8:6.6.9.7-5ubuntu3.9 with libMagickCore.so.4.0.1 and libMagickWand.so.4.0.1, or from "convert -version":

Code: Select all

Version: ImageMagick 6.6.9-7 2017-03-14 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2011 ImageMagick Studio LLC
Features: OpenMP

Clearly a fairly old but not (?) ancient version, probably because although I only installed it a few days ago, I did it using apt-get and my version of Ubuntu Linux isn't very new (12.04 LTS, whereas the current is 18.04 LTS - 2 or 3 LTS versions different). I wonder why they don't update application version numbers. I'm not sure what difference upgrading the O/S would make to installing a pre-built ImageMagick - but upgrading the O/S would entail big changes to the whole system.

Anyway, it's the current version as far as Ubuntu thinks I should have. Otherwise I think I'd have to build ImageMagick from source; not impossible but I'd like to avoid that if I can.

For completeness, here's the important part of policy.xml:

Code: Select all

<policymap>
  <!-- <policy domain="system" name="precision" value="6"/> -->
  <!-- <policy domain="resource" name="temporary-path" value="/tmp"/> -->
  <!-- <policy domain="resource" name="memory" value="2GiB"/> -->
  <!-- <policy domain="resource" name="map" value="4GiB"/> -->
  <!-- <policy domain="resource" name="area" value="1gb"/> -->
  <!-- <policy domain="resource" name="disk" value="16eb"/> -->
  <!-- <policy domain="resource" name="file" value="768"/> -->
  <!-- <policy domain="resource" name="thread" value="4"/> -->
  <!-- <policy domain="resource" name="throttle" value="0"/> -->
  <!-- <policy domain="resource" name="time" value="3600"/> -->
</policymap>

If my files are roughly 19200 x 13500 pixels in 16-bit greyscale, I reckon they should be a little over half a gig uncompressed, so the limits seem OK. The machine has 6GB RAM, 6GB swap space, over 80GB free in /tmp, and 1.7GB free on the drive the images are on.

Any suggestions? Do you have any particular reason to think upgrading would make a difference to what should be a straightforward operation?

[snibgo]

Your policy.xml has all lines commented-out. What does "convert -list resource" say?

I've never used patched versions of IM (except where I have made the patches). You version has obviously been partially patched (because the version number is in the source code, and that hasn't been patched). My question would be: how do we know that a consistent set of patches has been applied?

[pnt103]

Oh, spit. Sorry, guys, it's too hot here, and I only just realised that policy has all the relevant parts commented out. And apparently that was the problem. I edited it to remove the comments and adjust the values, and now I can do what I want. I've not tried everything, but "so far, so good". fmw42, thank you for pointing me in the right direction.

[pnt103]

Your policy.xml has all lines commented-out. What does "convert -list resource" say?

Yes, I realised that just before you posted. For the sake of completeness:

Code: Select all

$ convert -list resource
File         Area       Memory          Map         Disk    Thread         Time
-------------------------------------------------------------------------------
 768          3GB         3GiB         8GiB   13.877788EiB         4         3600

Before I removed the comment marks and edited the values it was:

Code: Select all

$ convert -list resource
File         Area       Memory          Map         Disk    Thread         Time
-------------------------------------------------------------------------------
 768     1.5321GB    5.7077GiB    11.415GiB    unlimited         2    unlimited

Just needs a bit of tweaking now.

[fmw42]

I would also point out that your libjpeg at 80 is old. The current version is 90. So you might also consider upgrading it. Also the date of you patch of ImageMagick is 3/14/2017. That is over a year ago. So perhaps there is a more current patch. You should check with your Linux distro people to find out if there is a newer version, if you still have trouble.