Page 1 of 1
Problem converting large JPGs
Posted: 2018-08-01T09:48:30-07:00
by pnt103
I'm trying to resize several large B&W images using convert, but it keeps core-dumping. Small files work OK, though. Typical image file is around 12MB, 19200 x 13100 pixels, 16-bit greyscale, JPG format. This is using ImageMagick 6.6.9-7 on a machine running Ubuntu 12.04.5 LTS, with 6GB RAM and 1.8TB of free space on the disk partition I'm using.
I've checked ulimit, and there are no limits on memory or disk use, tried setting MAGICK_TMPDIR to the directory I'm in (and which the images are in) so there's plenty of disk space available. Here's what happens:
$ convert -debug cache LR01.jpg -resize 900 LR01_900.jpg
2018-08-01T17:08:27+01:00 0:00.000 0.000u 6.6.9 Cache convert[26572]: cache.c/DestroyPixelCache/1540/Cache
destroy
2018-08-01T17:08:27+01:00 0:00.000 0.000u 6.6.9 Cache convert[26572]: cache.c/ExtendCache/4020/Cache
extend LR01.jpg[0] (/raid2TB/pete/tmp/magick-0iQ1wOxO[4], disk, 2.5423GBB)
2018-08-01T17:08:27+01:00 0:00.000 0.000u 6.6.9 Cache convert[26572]: cache.c/OpenPixelCache/4246/Cache
open LR01.jpg[0] (/raid2TB/pete/tmp/magick-0iQ1wOxO[4], disk, 19285x13183 2.5423GBB)
Segmentation fault (core dumped)
The ultimate goal is to create two reduced-size copies for web use, one about 900 pixels wide for display, and another as a smaller thumbnail. What am I missing?
Re: Problem converting large JPGs
Posted: 2018-08-01T11:50:32-07:00
by fmw42
What is your version of libjpeg? You can find that from
Perhaps you need to upgrade it.
Otherwise, check your policy.xml file for other limitations such as on width or height or memory, etc, though you seem to have perhaps checked that already.
If you want someone to test, then post your input JPG to some free hosting service that will not modify it and put the URL here.
On Linux, the version 6.6.9.7 would appear to be ancient. But they do not change versions when they patch. So the more important fact would be the date associated with the version. Be sure you have the latest patch from your Linux distro.
Re: Problem converting large JPGs
Posted: 2018-08-02T04:08:22-07:00
by magick
Agreed. Try upgrading to the latest release. If that fails, let us know and we will investigate further.
Re: Problem converting large JPGs
Posted: 2018-08-02T08:02:35-07:00
by pnt103
Thank you replying so quickly.
"convert -list format" just lists (a lot of) formats without giving versions:
Code: Select all
JPEG* JPEG rw- Joint Photographic Experts Group JFIF format (80)
JPG* JPEG rw- Joint Photographic Experts Group JFIF format (80)
JPX* JP2 rw- JPEG-2000 File Format Syntax
My libjpg is /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2
ImageMagick itself is 8:6.6.9.7-5ubuntu3.9 with libMagickCore.so.4.0.1 and libMagickWand.so.4.0.1, or from "convert -version":
Code: Select all
Version: ImageMagick 6.6.9-7 2017-03-14 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2011 ImageMagick Studio LLC
Features: OpenMP
Clearly a fairly old but not (?) ancient version, probably because although I only installed it a few days ago, I did it using apt-get and my version of Ubuntu Linux isn't very new (12.04 LTS, whereas the current is 18.04 LTS - 2 or 3 LTS versions different). I wonder why they don't update application version numbers. I'm not sure what difference upgrading the O/S would make to installing a pre-built ImageMagick - but upgrading the O/S would entail big changes to the whole system.
Anyway, it's the current version as far as Ubuntu thinks I should have. Otherwise I think I'd have to build ImageMagick from source; not impossible but I'd like to avoid that if I can.
For completeness, here's the important part of policy.xml:
Code: Select all
<policymap>
<!-- <policy domain="system" name="precision" value="6"/> -->
<!-- <policy domain="resource" name="temporary-path" value="/tmp"/> -->
<!-- <policy domain="resource" name="memory" value="2GiB"/> -->
<!-- <policy domain="resource" name="map" value="4GiB"/> -->
<!-- <policy domain="resource" name="area" value="1gb"/> -->
<!-- <policy domain="resource" name="disk" value="16eb"/> -->
<!-- <policy domain="resource" name="file" value="768"/> -->
<!-- <policy domain="resource" name="thread" value="4"/> -->
<!-- <policy domain="resource" name="throttle" value="0"/> -->
<!-- <policy domain="resource" name="time" value="3600"/> -->
</policymap>
If my files are roughly 19200 x 13500 pixels in 16-bit greyscale, I reckon they should be a little over half a gig uncompressed, so the limits seem OK. The machine has 6GB RAM, 6GB swap space, over 80GB free in /tmp, and 1.7GB free on the drive the images are on.
Any suggestions? Do you have any particular reason to think upgrading would make a difference to what should be a straightforward operation?
Re: Problem converting large JPGs
Posted: 2018-08-02T08:35:58-07:00
by snibgo
Your policy.xml has all lines commented-out. What does "convert -list resource" say?
I've never used patched versions of IM (except where I have made the patches). You version has obviously been partially patched (because the version number is in the source code, and that hasn't been patched). My question would be: how do we know that a consistent set of patches has been applied?
Re: Problem converting large JPGs
Posted: 2018-08-02T08:57:18-07:00
by pnt103
Oh, spit. Sorry, guys, it's too hot here, and I only just realised that policy has all the relevant parts commented out. And apparently that was the problem. I edited it to remove the comments and adjust the values, and now I can do what I want. I've not tried everything, but "so far, so good". fmw42, thank you for pointing me in the right direction.
Re: Problem converting large JPGs
Posted: 2018-08-02T09:03:11-07:00
by pnt103
snibgo wrote: ↑2018-08-02T08:35:58-07:00
Your policy.xml has all lines commented-out. What does "convert -list resource" say?
Yes, I realised that just before you posted. For the sake of completeness:
Code: Select all
$ convert -list resource
File Area Memory Map Disk Thread Time
-------------------------------------------------------------------------------
768 3GB 3GiB 8GiB 13.877788EiB 4 3600
Before I removed the comment marks and edited the values it was:
Code: Select all
$ convert -list resource
File Area Memory Map Disk Thread Time
-------------------------------------------------------------------------------
768 1.5321GB 5.7077GiB 11.415GiB unlimited 2 unlimited
Just needs a bit of tweaking now.
Re: Problem converting large JPGs
Posted: 2018-08-02T11:16:58-07:00
by fmw42
I would also point out that your libjpeg at 80 is old. The current version is 90. So you might also consider upgrading it. Also the date of you patch of ImageMagick is 3/14/2017. That is over a year ago. So perhaps there is a more current patch. You should check with your Linux distro people to find out if there is a newer version, if you still have trouble.