fax tif makes IM consume gigs of memory, pins cpu

[mmlenz]

I can't post the image itself at the moment because it contains private info (I'll see if I can find out from the customer what software is generating it and also get an sample image) but here is a verbose identify. Maybe someone can spot something in the meantime.

OS is Ubuntu 14.04 up-to-date. I also get the exact same results on Debian 6.x running (ImageMagick 6.6.0-4 2014-04-04 Q16)

Code: Select all

$ identify -version
Version: ImageMagick 6.7.7-10 2014-03-06 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC
Features: OpenMP    

Code: Select all

$ identify -verbose source.tif 
Image: source.tif
  Format: TIFF (Tagged Image File Format)
  Class: DirectClass
  Geometry: 2191x2894+0+0
  Resolution: 1x1
  Print size: 2191x2894
  Units: Undefined
  Type: Bilevel
  Base type: Bilevel
  Endianess: MSB
  Colorspace: Gray
  Depth: 1-bit
  Channel depth:
    gray: 1-bit
  Channel statistics:
    Gray:
      min: 0 (0)
      max: 1 (1)
      mean: 0.936169 (0.936169)
      standard deviation: 0.244452 (0.244452)
      kurtosis: 10.7345
      skewness: -3.56854
  Colors: 2
  Histogram:
    404738: (  0,  0,  0) #000000 gray(0,0,0)
   5936016: (255,255,255) #FFFFFF gray(255,255,255)
  Rendering intent: Undefined
  Gamma: 1
  Interlace: None
  Background color: gray(255,255,255)
  Border color: gray(223,223,223)
  Matte color: gray(189,189,189)
  Transparent color: gray(0,0,0)
  Compose: Over
  Page geometry: 2191x2894+0+0
  Dispose: Undefined
  Iterations: 0
  Scene: 0 of 2
  Compression: Group4
  Orientation: TopLeft
  Properties:
    date:create: 2014-08-04T07:28:31-05:00
    date:modify: 2014-08-04T07:28:31-05:00
    signature: bef3ea6ca0aa1839b89b59ca71838b058b88a3626d9852dfc33a8ede2c9e8c7e
    tiff:endian: msb
    tiff:photometric: min-is-white
    tiff:rows-per-strip: 29
  Artifacts:
    filename: source.tif
    verbose: true
  Tainted: False
  Filesize: 110KB
  Number pixels: 6.341M
  Pixels per second: 158.5MB
  User time: 0.040u
  Elapsed time: 0:01.039
  Version: ImageMagick 6.7.7-10 2014-03-06 Q16 http://www.imagemagick.org
Image: source.tif
  Format: TIFF (Tagged Image File Format)
  Class: DirectClass
  Geometry: 1696x2298+0+0
  Resolution: 200x200
  Print size: 8.48x11.49
  Units: PixelsPerInch
  Type: Bilevel
  Base type: Bilevel
  Endianess: MSB
  Colorspace: Gray
  Depth: 1-bit
  Channel depth:
    gray: 1-bit
  Channel statistics:
    Gray:
      min: 0 (0)
      max: 1 (1)
      mean: 0.921342 (0.921342)
      standard deviation: 0.269205 (0.269205)
      kurtosis: 7.79857
      skewness: -3.13027
  Colors: 2
  Histogram:
    306564: (  0,  0,  0) #000000 gray(0,0,0)
   3590844: (255,255,255) #FFFFFF gray(255,255,255)
  Rendering intent: Undefined
  Gamma: 1
  Interlace: None
  Background color: gray(255,255,255)
  Border color: gray(223,223,223)
  Matte color: gray(189,189,189)
  Transparent color: gray(0,0,0)
  Compose: Over
  Page geometry: 1696x2298+0+0
  Dispose: Undefined
  Iterations: 0
  Scene: 1 of 2
  Compression: Group4
  Orientation: TopLeft
  Properties:
    date:create: 2014-08-04T07:28:31-05:00
    date:modify: 2014-08-04T07:28:31-05:00
    signature: d7c405e7825755af7b205ead19568f0ce6ac7e17a760e52c9b62ded92f2a86e8
    tiff:endian: msb
    tiff:photometric: min-is-white
    tiff:rows-per-strip: 57
  Artifacts:
    filename: source.tif
    verbose: true
  Tainted: False
  Filesize: 110KB
  Number pixels: 3.897M
  Pixels per second: 1.68MB
  User time: 2.360u
  Elapsed time: 0:03.320
  Version: ImageMagick 6.7.7-10 2014-03-06 Q16 http://www.imagemagick.org

This is the command that makes IM freak out.

Code: Select all

$ convert source.tif -resample 200 -resize 400x400 low-%d.jpg

So I figured the tif was corrupted in some way so I just do a simple:

Code: Select all

$ convert source.tif temp-source.tif

Which works fine and produces a slightly smaller file. However running the same convert above only with temp-source.tif produces the exact same memory/cpu consuming freakout.

[snibgo]

Resolution: 1x1

That's probably the problem. Then you resample 200, which might magnify it by 200x200. Yeuch.

[mmlenz]

I was looking through our code and the reason I was doing the resample is because we also get fax tifs with the non-symmetrical resolution (200x100). So if you use that convert command (without the resample) on one of those 200x100 fax tifs you end up with an image that is squashed vertically. I know we also get tifs in that are not (1x1) which work fine with the resample (my guess is they are probably 200x200). This is the first time we've encountered these 1x1 images.

Any thoughts on a single convert that would work for both types and produce a normal looking jpeg?

EDIT: I just noticed that the first image is 1x1 and second image in the tif is 200x200. WTH?

[snibgo]

I would have to experiment with a load of images to find a single-command solution. But "1x1 undefined" is a bad density. Typical good densities are "72 dpi" or "35 pixels per centimeter". Doing any processing with a bad density will give bad results. I suggest you test for this, and (for example) "set density 200x200", or something.

[mmlenz]

I ended up using the output from:

Code: Select all

$ identify -format '%[fx:resolution.x/resolution.y]' image.tif[0]

to test == 1 If the resolutions are symmetrical I don't do the resampling. If it's != 1 (faxed) then I resample. convert appears to "do the right thing" when the resample is left off for the 1x1.

[mmlenz]

Kind of related but is there a way to have convert output the total resources used for a run? I need to put some sane limits on how much memory and disk space IM can use.

[snibgo]

Perhaps "-debug resource".