Page 1 of 1
identify -format '%k-%T' hangs on large files
Posted: 2015-05-24T05:55:27-07:00
by grigory
Hey guys,
I have a web service using Imagemagick (Core2Duo + 8GB RAM).
When I run the following command on a 15MB GIF image (I tried different images):
/usr/local/bin/identify -format '%k-%T|' /home/webiste/image.gif
identify just hangs most of the time.
At first, it takes 100% CPU, then it takes 500MB of memory... 800MB... then it uses all the memory, all the swap, and then my server goes down in a minute.
Any ideas on how to solve this issue?
Thanks!
Re: identify -format '%k-$T' hangs on large files
Posted: 2015-05-24T06:36:34-07:00
by snibgo
%k calculates the number of unique colours. I don't know how it works. Perhaps it sorts all the pixels, and runs though these aggregating counts.
But a GIF can contain no more than 256 colours, so a faster method would be to simply walk through all the pixels, incrementing a counter for the appropriate palette entry.
How many pixels does your image have? What is the image -- a photograph, or a graphic with large areas of block colour?
Re: identify -format '%k-%T' hangs on large files
Posted: 2015-05-24T06:48:08-07:00
by grigory
Well, I use a GIF made from video. Its size is 2.23MB. Then I resize the file to bigger one so it's 15-20MB in size.
After that I try to use identify on that big GIF, and it hangs. I would say it has 2000x1000 or maybe even 2500x1500 in pixels.
Re: identify -format '%k-$T' hangs on large files
Posted: 2015-05-24T06:57:23-07:00
by snibgo
That isn't large. I can do %k on a 35 million pixel GIF in about 3 seconds.
What version of IM, on what platform? I use IM v6.9.1-0 on Windows 8.1.
EDIT: What is the value of $T? If that contains percent signs, that might explain the problem.
Re: identify -format '%k-%T' hangs on large files
Posted: 2015-05-26T06:38:15-07:00
by grigory
Sorry, I use '%k-%T', not '%k-$T'. Posted wrong string here.
I use CentOS and 6.9.1.-2 version of ImageMagick with OpenMP:
Version: ImageMagick 6.9.1-2 Q16 x86_64 2015-05-11
http://www.imagemagick.org
Copyright: Copyright (C) 1999-2015 ImageMagick Studio LLC
License:
http://www.imagemagick.org/script/license.php
Features: DPC OpenMP
Delegates (built-in): jng jpeg png zlib
Re: identify -format '%k-%T' hangs on large files
Posted: 2015-05-26T09:52:06-07:00
by fmw42
If this is a multi-frame gif, then how may frames do you have. IM may be running out of memory and swapping to disk, since it likely has to read every frame into memory rather than read each frame one at a time.
You could try converting to miff streaming format and that might then process each frame separately. Try
Code: Select all
convert yourlarger.gif miff:- | convert - -format "%k-%T" info:
or start with your smaller gif and do the resize in line.
Code: Select all
convert yoursmaller.gif -resize ... miff:- | convert - -format "%k-%T" info:
Does that help?
Re: identify -format '%k-%T' hangs on large files
Posted: 2015-05-31T08:00:51-07:00
by grigory
Hi fmw42,
Will your solution take longer to show the result?
I mean that miff streaming because of separately processing.
Re: identify -format '%k-%T' hangs on large files
Posted: 2015-05-31T10:43:36-07:00
by fmw42
grigory wrote:Hi fmw42,
Will your solution take longer to show the result?
I mean that miff streaming because of separately processing.
I do not know, probably not much. But did you even try it to see if it works?
Re: identify -format '%k-%T' hangs on large files
Posted: 2015-06-03T07:44:05-07:00
by grigory
Yes, I've tried.
First solution takes more time (56 seconds for 10 repeats in a row vs. 47 seconds for my previous command). Here are results:
/usr/local/bin/convert /gifs/00005.gif miff:- | convert - -format "%k-%T|" info:
#real 0m55.736s
#user 0m45.543s
#sys 0m8.853s
and
/usr/local/bin/identify -format '%k-%T|' /gifs/00005.gif
#real 0m46.814s
#user 0m41.527s
#sys 0m3.420s
And I can't test you second solution because script doesn't know the path to previous smaller image to use it.
Re: identify -format '%k-%T' hangs on large files
Posted: 2015-06-03T08:33:57-07:00
by fmw42
And I can't test you second solution because script doesn't know the path to previous smaller image to use it.
Why do you not know where the smaller image resides, if you say above that you resize it in your old command sequence?
Nevertheless, if piping slows down one command it will slow down the other.
Sorry, I have no other ideas.