identify -unique take minutes for Tiff in 32 bits

[RichardDumoulin]

Hi!

I have the same problem with :

Code: Select all

ImageMagick 6.8.9-10 Q16 on Linux
ImageMagick 7.0.5-10 Q16 on Windows 10

I try to get the number of colors for a TIFF image using the command :

Code: Select all

identify -verbose -unique A28I2434.tif

If I save the file in Lightroom or Photoshop in 8 bits I don't have any problem. it is really fast. If I save as a Tiff in 16 bits in LR or PS, the command "hang" at this point for few minutes :

Code: Select all

E:\>identify -verbose -unique A28I2434.tif
Image: A28I2434.tif
  Format: TIFF (Tagged Image File Format)
  Mime type: image/tiff
  Class: DirectClass
  Geometry: 5472x3648+0+0
  Resolution: 240x240
  Print size: 22.8x15.2
  Units: PixelsPerInch
  Type: TrueColor
  Endianess: LSB
  Colorspace: sRGB
  Depth: 16-bit
  Channel depth:
    Red: 16-bit
    Green: 16-bit
    Blue: 16-bit
  Channel statistics:
    Pixels: 19961856
    Red:
      min: 0 (0)
      max: 65535 (1)
      mean: 31163.6 (0.475527)
      standard deviation: 9577.9 (0.146149)
      kurtosis: 0.701174
      skewness: 0.573125
      entropy: 0.955511
    Green:
      min: 0 (0)
      max: 65535 (1)
      mean: 32485.4 (0.495695)
      standard deviation: 8213.19 (0.125325)
      kurtosis: 1.30813
      skewness: 0.29047
      entropy: 0.942269
    Blue:
      min: 0 (0)
      max: 65535 (1)
      mean: 28153.5 (0.429595)
      standard deviation: 8877.65 (0.135464)
      kurtosis: 1.07805
      skewness: 0.451484
      entropy: 0.94801
  Image statistics:
    Overall:
      min: 0 (0)
      max: 65535 (1)
      mean: 30600.8 (0.466939)
      standard deviation: 9089.61 (0.138699)
      kurtosis: 0.898127
      skewness: 0.412885
      entropy: 0.948597


--- hang for few minute here then I get : ----


Colors: 19943960
  Rendering intent: Perceptual
  Gamma: 0.454545
  Chromaticity:
    red primary: (0.64,0.33)
    green primary: (0.3,0.6)
    blue primary: (0.15,0.06)
    white point: (0.3127,0.329)
  Matte color: grey74
  Background color: white
  Border color: srgb(223,223,223)
  Transparent color: none
  Interlace: None
  Intensity: Undefined
  Compose: Over
  Page geometry: 5472x3648+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: Zip
  Orientation: TopLeft
  Properties:
    aux:ApproximateFocusDistance: 135/10
    aux:Firmware: 1.0.2
    aux:FlashCompensation: 0/1
    aux:ImageNumber: 0
    aux:Lens: EF500mm f/4L IS USM +1.4x III
    aux:LensID: 143
    aux:LensInfo: 700/1 700/1 0/0 0/0
    aux:LensSerialNumber: 0000000000
    aux:OwnerName: Richard Dumoulin
    aux:SerialNumber: 032111000028
    crs:AlreadyApplied: True
    crs:AutoLateralCA: 0
    crs:AutoWhiteVersion: 134348800
    crs:Blacks2012: 0
    crs:BlueHue: 0
    crs:BlueSaturation: 0
    crs:CameraProfile: Adobe Standard
    crs:Clarity2012: 0
    crs:ColorNoiseReduction: 25
    crs:ColorNoiseReductionDetail: 50
    crs:ColorNoiseReductionSmoothness: 50
    crs:Contrast2012: 0
    crs:ConvertToGrayscale: False
    crs:DefringeGreenAmount: 0
    crs:DefringeGreenHueHi: 60
    crs:DefringeGreenHueLo: 40
    crs:DefringePurpleAmount: 0
    crs:DefringePurpleHueHi: 70
    crs:DefringePurpleHueLo: 30
    crs:Dehaze: 0
    crs:Exposure2012: -0.37
    crs:GrainAmount: 0
    crs:GreenHue: 0
    crs:GreenSaturation: 0
    crs:HasCrop: False
    crs:HasSettings: True
    crs:Highlights2012: 0
    crs:HueAdjustmentAqua: 0
    crs:HueAdjustmentBlue: 0
    crs:HueAdjustmentGreen: 0
    crs:HueAdjustmentMagenta: 0
    crs:HueAdjustmentOrange: 0
    crs:HueAdjustmentPurple: 0
    crs:HueAdjustmentRed: 0
    crs:HueAdjustmentYellow: 0
    crs:LensManualDistortionAmount: 0
    crs:LensProfileEnable: 0
    crs:LensProfileSetup: LensDefaults
    crs:LuminanceAdjustmentAqua: 0
    crs:LuminanceAdjustmentBlue: 0
    crs:LuminanceAdjustmentGreen: 0
    crs:LuminanceAdjustmentMagenta: 0
    crs:LuminanceAdjustmentOrange: 0
    crs:LuminanceAdjustmentPurple: 0
    crs:LuminanceAdjustmentRed: 0
    crs:LuminanceAdjustmentYellow: 0
    crs:LuminanceSmoothing: 0
    crs:ParametricDarks: 0
    crs:ParametricHighlights: 0
    crs:ParametricHighlightSplit: 75
    crs:ParametricLights: 0
    crs:ParametricMidtoneSplit: 50
    crs:ParametricShadows: 0
    crs:ParametricShadowSplit: 25
    crs:PerspectiveAspect: 0
    crs:PerspectiveHorizontal: 0
    crs:PerspectiveRotate: 0.0
    crs:PerspectiveScale: 100
    crs:PerspectiveUpright: 0
    crs:PerspectiveVertical: 0
    crs:PerspectiveX: 0.00
    crs:PerspectiveY: 0.00
    crs:PostCropVignetteAmount: 0
    crs:ProcessVersion: 6.7
    crs:RedHue: 0
    crs:RedSaturation: 0
    crs:Saturation: 0
    crs:SaturationAdjustmentAqua: 0
    crs:SaturationAdjustmentBlue: 0
    crs:SaturationAdjustmentGreen: 0
    crs:SaturationAdjustmentMagenta: 0
    crs:SaturationAdjustmentOrange: 0
    crs:SaturationAdjustmentPurple: 0
    crs:SaturationAdjustmentRed: 0
    crs:SaturationAdjustmentYellow: 0
    crs:Shadows2012: 0
    crs:ShadowTint: 0
    crs:SharpenDetail: 25
    crs:SharpenEdgeMasking: 0
    crs:SharpenRadius: +1.0
    crs:Sharpness: 25
    crs:SplitToningBalance: 0
    crs:SplitToningHighlightHue: 0
    crs:SplitToningHighlightSaturation: 0
    crs:SplitToningShadowHue: 0
    crs:SplitToningShadowSaturation: 0
    crs:Temperature: 5050
    crs:Tint: +27
    crs:ToneCurveName: Linear
    crs:ToneCurveName2012: Linear
    crs:ToneMapStrength: 0
    crs:UprightCenterMode: 0
    crs:UprightCenterNormX: 0.5
    crs:UprightCenterNormY: 0.5
    crs:UprightFocalLength35mm: 35
    crs:UprightFocalMode: 0
    crs:UprightFourSegmentsCount: 0
    crs:UprightPreview: False
    crs:UprightTransformCount: 6
    crs:UprightVersion: 151388160
    crs:Version: 9.9
    crs:Vibrance: 0
    crs:VignetteAmount: 0
    crs:WhiteBalance: As Shot
    crs:Whites2012: 0
    date:create: 2017-06-06T15:54:32-04:00
    date:modify: 2017-06-06T15:54:37-04:00
    dc:format: image/tiff
    exif:ApertureValue: 5.65564
    exif:CustomRendered: 0
    exif:DateTimeDigitized: 2017:06:04 09:26:00
    exif:DateTimeOriginal: 2017:06:04 09:26:00
    exif:ExposureBiasValue: 0
    exif:ExposureMode: 1
    exif:ExposureProgram: 1
    exif:ExposureTime: 0.00125
    exif:Flash: 16
    exif:FNumber: 7.1
    exif:FocalLength: 700
    exif:FocalPlaneResolutionUnit: 3
    exif:FocalPlaneXResolution: 1520
    exif:FocalPlaneYResolution: 1520
    exif:ISOSpeedRatings: 800
    exif:MaxApertureValue: 5
    exif:MeteringMode: 5
    exif:PixelXDimension: 5472
    exif:PixelYDimension: 3648
    exif:SceneCaptureType: 0
    exif:ShutterSpeedValue: 9.64386
    exif:SubSecTimeDigitized: 27
    exif:SubSecTimeOriginal: 27
    exif:WhiteBalance: 0
    icc:copyright: Copyright (c) 1998 Hewlett-Packard Company
    icc:description: sRGB IEC61966-2.1
    icc:manufacturer: IEC http://www.iec.ch
    icc:model: IEC 61966-2.1 Default RGB colour space - sRGB
    photoshop:ColorMode: 3
    photoshop:DateCreated: 2017-06-04T09:26:00.027
    photoshop:ICCProfile: sRGB IEC61966-2.1
    signature: ffd7504ef8f3e100e6cd8af05c4346e880943cc7c8ed897849bde538e9234158
    tiff:alpha: unspecified
    tiff:artist: Richard Dumoulin
www.RichardDumoulin.com
    tiff:copyright: All rights reserved
Richard Dumoulin
www.RichardDumoulin.com
    tiff:endian: lsb
    tiff:make: Canon
    tiff:model: Canon EOS-1D X Mark II
    tiff:photometric: RGB
    tiff:rows-per-strip: 31
    tiff:software: Adobe Photoshop CS6 (Windows)
    tiff:timestamp: 2017:06:06 15:54:30
    xmp:CreateDate: 2017-06-04T09:26
    xmp:CreatorTool: Adobe Photoshop Lightroom 6.9 (Windows)
    xmp:Label: Yellow
    xmp:MetadataDate: 2017-06-06T15:54:30-04:00
    xmp:ModifyDate: 2017-06-06T15:54:30-04:00
    xmp:Rating: 5
    xmpMM:DerivedFrom: A0F5FDABFBDA6B7AB8F450FC67DC79DA
    xmpMM:DocumentID: xmp.did:98914fbd-198b-174a-827a-67f08ac3cdff
    xmpMM:InstanceID: xmp.iid:6472EA1EF14AE711AC4DF0E85EAF138E
    xmpMM:OriginalDocumentID: A0F5FDABFBDA6B7AB8F450FC67DC79DA
  Profiles:
    Profile-8bim: 7858 bytes
    Profile-icc: 3144 bytes
    Profile-xmp: 24274 bytes
  Artifacts:
    identify:unique-colors: true
    verbose: true
  Tainted: False
  Filesize: 108.187MiB
  Number pixels: 19.9619M
  Pixels per second: 31.9901MB
  User time: 0.609u
  Elapsed time: 0:01.624
  Version: ImageMagick 7.0.5-10 Q16 x64 2017-06-03 http://www.imagemagick.org
identify: Incompatible type for "RichTIFFIPTC"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/925.
identify: Unknown field with tag 34864 (0x8830) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/925.
identify: Unknown field with tag 34866 (0x8832) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/925.
identify: Unknown field with tag 42032 (0xa430) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/925.
identify: Unknown field with tag 42033 (0xa431) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/925.
identify: Unknown field with tag 42034 (0xa432) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/925.
identify: Unknown field with tag 42036 (0xa434) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/925.
identify: Unknown field with tag 42037 (0xa435) encountered. `TIFFReadCustomDirectory' @ warning/tiff.c/TIFFWarnings/925.
identify: Incompatible type for "RichTIFFIPTC"; tag ignored. `TIFFFetchNormalTag' @ warning/tiff.c/TIFFWarnings/925.

Could you please let me know if I do something wrong or if it is a bug?

I have the same problem on Linux with an older version of ImageMagick.

I did try differents parameters to save the file (no compression, zip compression, adobe rgb, srgb) and it is related to saving in 16 bits only.

Thanks in advance for your hekp!

[snibgo]

What does "identify -verbose -unique A28I2434.tif" do? What do you want it to do?

"-unique" isn't listed as a command-line option in the documentation http://www.imagemagick.org/script/comma ... ptions.php

[RichardDumoulin]

Hi!

Thanks for the reply snibgo.

I want to get the number of colors in the image.

I found initially the problem using PerlMagick command : my ($colors) = $image->Get('colors');

The full command that I use was :
my ($width, $height, $xres, $yres, $filesize, $colors) = $image->Get('width', 'height', 'x-resolution', 'y-resolution', 'filesize', 'colors');

By doing many tests, I found that the slow down was related to getting the number of colors.

I tried the command : identify -verbose -unique to reproduce the behavior to find if it's a problem in Perl Magick or in Image Magick.

From what I see, it looks like a problem occuring in Image Magic and only with 16 bits Tiff.

[snibgo]

To count the unique colours, IM must first find them. With large images eg 5000x7500 pixels and millions of colours, this takes lots of time and memory.

This is partly unavoidable, but perhaps it can be improved.

Meanwhile, reducing the data to 8 bits/channel/pixel certainly improves time and memory, eg:

Code: Select all

convert AGA_2774_sRGB.tiff -depth 8 miff:- |convert - -unique-colors info:

- MIFF 1429307x1 1429307x1+0+0 8-bit sRGB 0.375u 0:00.375

[RichardDumoulin]

Thanks for the reply!

Converting to 8 bits may speed it but it is strange that a Tiff in 32 bits takes few minutes to count the colors?

I would expect that loading the file to memory would take most of the time (which is required for any operations)...

[snibgo]

I would expect that sorting the colours takes the most time. There is a big difference between sorting 1 million colours and sorting 35 million colours.

Why are you counting colours? If you need an accurate count, then the hard work must be done, without simplifying to 8-bit.

[RichardDumoulin]

Hi Snibgo!

Thanks for the reply and info...

I extract the number of colors to know if it's a black and white image, a greyscale image or ... a color image.

The number of colors is also displayed on the web site.

[snibgo]

I extract the number of colors to know if it's a black and white image, a greyscale image or ... a color image.

You can get that from IM directly, more or less. Try:

Code: Select all

identify -verbose myimage.ext

Look near the top. If it says the image is grayscale, then it has no colours. If it has only black and white, it will show a histogram with only those colours.

[RichardDumoulin]

Hi Snibgo!

Thanks I will try this solution!!

Really appreciate your help!