Legacy ImageMagick Discussions Archive

Hello!
I have a big problem, and i couldn't find solution. I triied about 10-20 options for this, but i still can't limit the memory usage for the jp2 conversion.
About 300-400 MB is free all the time, but the conversion use all the ramaining memory in my PC, and i am afraid of the failure in my server, because there is a web application.
I use the im4java lib to process the conversions in batch mode (10-12 at one time), and i thought the memory allocation error is lib's failure, but then i tested the normal cmd line conversion.

Example:
25/85 MB tif image with 4207x6522 8-bit sRGB.
Referring to the formula [(Quantum * height * width * channel)/8] => 16 * 4207 * 6522 * 3 = 157MB * 2 /*because of the pixel clone*/ ~ 320MB
And it's fine the total memory usage is about 340MB (i think because of the windows' task manager working with 1000 not with 1024) when i convert the file into a jpeg. But when i try the jp2 it's 1,2 GB.
Same is for the bigger, 8889x15120 image, but in this case, the memory usage always the available memory - 400MB, which is in my case ~5GB, intead of the ~1,5GB. It's like the memory allocation is growing exponential, and i don't know a multiplier "magic number".
I tried:

• the image magick -limit options but it's not working.
  the JasPer limitations with the -define option. Example: ftp://ftp.tuwien.ac.at/graphics/ImageMa ... w/jp2.html
  also tried the quality with the normal -quality (but this isn't working at all, i think because of the delegating) and the -delegate jp2:quality=50. Result: the file is smaller, but the memory usage is still the maximum.

Anyone has an idea how can i calculate/limit the memory usage?

What is your IM version and platform? Is this on your computer or a server? What was your exact command line?

fmw42 wrote:What is your IM version and platform? Is this on your computer or a server? What was your exact command line?

Testing on my computer
- OS: win 8.1 x64
- ImageMagick 6.9.3-8 Q16 x64
Resources:
8G RAM - about 4-5 GB available, depends on the number of webapps running.
8G swap (pagefile)
CPU: Intel(R) Core(TM) i5-3210M @ 2.50GHz

The default command after the im4java parse (http://im4java.sourceforge.net/): ColorSeparation is necessary, because the 2bit tiff's after the JP2 convert are black. Only visible after IrfanView's color correction.

I'm testing the conversions with this command and expanding with the limitations (see above). The server has more resources, so i can use parallel conversions. The implementation in the java lib is a simple loop, and when the big images are coming the server runs out of the memory, because of the high memory usage by the JasPer library.

Note: As i said it's not the java lib's fault, because the memory usage for 1 file in the loop is the same as the normal command line start with one file.

I have just found that the jasper lib changed to openjpeg2 from the 6.8.8-2 version. But the memory usage by this lib is still ridiculous

We leverage the OpenJPEG2 delegate library to read and write JPEG-2000 images. The OpenJPEG2 API allocates the entire image in memory. If you spot opportunities in the API to only allocate a few rows of pixels at a time, let us know and we will investigate rewriting the JP2 coder to lower its memory requirements.