benchmark example/request - color convert to cmyk, scale up and raster a giant rgb tiff

[magier0815]

Hello Folks,

this is my first post after trying to achieve it on my own. But I just don't get what I need.
A little help would be appreciated.

Here are the specs of my hugh source file:

TIFF Directory at offset 0xc8cd7352 (3368907602)
Image Width: 239286 Image Length: 4693
Resolution: 4616, 4616 pixels/inch
Bits/Sample: 8
Sample Format: unsigned integer
Compression Scheme: None
Photometric Interpretation: RGB color
Orientation: row 0 top, col 0 lhs
Samples/Pixel: 3
Rows/Strip: 1
Planar Configuration: single image plane
SubIFD Offsets: 3368951976

The source file contains an image/scan of a cmyk ink jet printout with pure primary colors (no color mixing of any sort, quasi pure spot cmyk). Also the src file is nearly sRGB but false colors wouldn't really matter in the output. It's more a matter of geometry (preserving faint geometric details) and ideally preserving "pure" cmyk colors in the output, because of geometry considerations.

The output needed is defined in the following:

1) four files, one for each separation of CMYK with the following tiff spec (don't consider the actual size, it's just a format example, resolution is as needed/printed)
TIFF Directory at offset 0x3f45122 (66343202)
Subfile Type: (0 = 0x0)
Image Width: 22658 Image Length: 23418
Resolution: 600, 800 pixels/inch
Bits/Sample: 1
Compression Scheme: None
Photometric Interpretation: min-is-white
Samples/Pixel: 1
Rows/Strip: 23418
Planar Configuration: single image plane

please mind the congruency of "Image Length" with "Rows/Strip". I could correct/reformat that later if it is to troublesome in the first place.

the longer edge/side of the output needs to be in the 800dpi direction!

2) the already hugh input file should first be upscaled 5 times (to preserve the details, single pixel lines overpainted etc.)
And yes, I'm aware of the output size of about 7 meters in the longer direction. It wouldn't be a problem if it where 14 meters if possible.

3) converted to cmyk with a profile e.g. ISOcoated_v2_300_eci.icc (or the printers four channel profile I'd provide)

4)rastered/dithered with e.g. error diffusion or something more appropriate to preserve geometric details

5) output to the four separation (cmyk) files 1-bit as defines in 1)

I've already tried quiet some parameter combinations but didn't achieve the result I'd need. The best approach I found until now was the following but also didn't work: https://poizan.dk/blog/2014/02/28/monoc ... agemagick/

ImageMagick version: 6.5.4-7 2012-04-10 Q16 OpenMP on Red Hat Enterprise Linux Workstation release 6.5 (Santiago)

but could also provide version 7.0.3-1 Q16 x86_64 if preferred
Version: ImageMagick 7.0.3-1 Q16 x86_64 2016-09-20 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2016 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Features: Cipher DPC HDRI Modules OpenMP
Delegates (built-in): bzlib cairo djvu fftw fontconfig fpx freetype gslib jng jpeg lcms ltdl lzma pangocairo png ps tiff webp wmf x xml zlib


Thanks in advance four all your help
Magier0815

ps: wouldn't this be quiet a benchmark for ImageMagick?

[snibgo]

First, the processes:

1. Aside from rows per strip, this seems to be:

Code: Select all

convert huge.tif -colorspace CMYK -threshold 50% -separate out-%01d.tif

2. If "upscaled 5 times" means multiplying both width and height by 5, then:

Code: Select all

convert huge.tif -resize 500% even_more_huge.tif

3.

Code: Select all

convert huge.tif -profile sRGB.icc -profile ISOcoated_v2_300_eci.icc huge_cmyk.tif

4. I'm not sure what you want.

5. As 1.

IM v6.5.4-7

That version is prehistoric. I suggest you use a recent version.

The size, 1 giga-pixels for the input and 25 giga-pixels for result (2), is no problem provided your computer is large enough. If you don't have enough memory it will use disk, which will take a very long time.

[magier0815]

Thanks for your effort, but this doesn't do what I need.

The output I get is just monochrome and 8 bit see the following tiffinfo and all out* files are (bit) equal

$ tiffinfo out-*
out-0.tif:
TIFF Directory at offset 0x37be4 (228324)
Image Width: 6576 Image Length: 4384
Resolution: 4616, 4616 pixels/inch
Bits/Sample: 8
Compression Scheme: LZW
Photometric Interpretation: min-is-black
FillOrder: msb-to-lsb
Orientation: row 0 top, col 0 lhs
Samples/Pixel: 1
Rows/Strip: 4384
Planar Configuration: single image plane
Page Number: 0-4
Predictor: horizontal differencing 2 (0x2)
out-1.tif:
TIFF Directory at offset 0x37be4 (228324)
Image Width: 6576 Image Length: 4384
Resolution: 4616, 4616 pixels/inch
Bits/Sample: 8
Compression Scheme: LZW
Photometric Interpretation: min-is-black
FillOrder: msb-to-lsb
Orientation: row 0 top, col 0 lhs
Samples/Pixel: 1
Rows/Strip: 4384
Planar Configuration: single image plane
Page Number: 1-4
Predictor: horizontal differencing 2 (0x2)
out-2.tif:
TIFF Directory at offset 0x37be4 (228324)
Image Width: 6576 Image Length: 4384
Resolution: 4616, 4616 pixels/inch
Bits/Sample: 8
Compression Scheme: LZW
Photometric Interpretation: min-is-black
FillOrder: msb-to-lsb
Orientation: row 0 top, col 0 lhs
Samples/Pixel: 1
Rows/Strip: 4384
Planar Configuration: single image plane
Page Number: 2-4
Predictor: horizontal differencing 2 (0x2)
out-3.tif:
TIFF Directory at offset 0x37be4 (228324)
Image Width: 6576 Image Length: 4384
Resolution: 4616, 4616 pixels/inch
Bits/Sample: 8
Compression Scheme: LZW
Photometric Interpretation: min-is-black
FillOrder: msb-to-lsb
Orientation: row 0 top, col 0 lhs
Samples/Pixel: 1
Rows/Strip: 4384
Planar Configuration: single image plane
Page Number: 3-4
Predictor: horizontal differencing 2 (0x2)

Magier0815

[snibgo]

The output I get is just monochrome and 8 bit see the following tiffinfo and all out* files are (bit) equal

The outputs from (1) should be monochrome, 1 sample per pixel. To also be 1 bit/sample, add "-depth 1", for example:

Code: Select all

f:\web\im>convert rose: -colorspace CMYK -separate -monochrome -depth 1 out-%01d.tif

[magier0815]

@ snibgo, thanks for your help.

the actual solution in use now:

Code: Select all

convert -resize 100%x125% -profile sRGB_IEC61966-2-1_black_scaled.icc Mosaic.tif -intent saturation -profile ISOcoated_v2_300_eci.icc -separate -monochrome -depth 1 -rotate "-90" +compress -density 1000x800 125x100/out-%d.tiff

results for the cyan separation in

Code: Select all

TIFF Directory at offset 0xa399d5a (171547994)
  Image Width: 5731 Image Length: 239258
  Resolution: 1000, 800 pixels/inch
  Bits/Sample: 1
  Compression Scheme: None
  Photometric Interpretation: min-is-black
  FillOrder: msb-to-lsb
  Orientation: row 0 top, col 0 lhs
  Samples/Pixel: 1
  Rows/Strip: 239258
  Planar Configuration: single image plane
  Page Number: 0-4
  White Point: 0.3127-0.329
  PrimaryChromaticities: 0.640000,0.330000,0.300000,0.600000,0.150000,0.060000

This took hours to run, since I actually only have 64G ram installed. Going to increase that to 128 or even 256 (my limit would be 1T, but don't want to go that far). What was quiet impressive, at times the ram wasn't saturated, there were 3200% cpu load, all 32 cores where saturated! Impressive!!!!

The actual ink jet printout of about 6.5m in length is definitely acceptable for my needs!

Ideas how to improve feasibility of this uncommon task:

• reduce the quantum level to 8 bits, should half the size of memory required. false colors don't matter to a certain extend, are present anyways. solely geometry matters and ideally preserving CMYK primaries without to much mixing.
• store the intermediate results to disk (CMYK separeted channels) and sequentially process every channel of CMYK separately. But which file format should I use?
• Reorder processing steps
• Reflect on the implications of the rotation. The direction (+/-) of the rotation doesn't matter, solely needs to be 90 degrees. So what is better in my case: +90 or -90, concerning runtime / memory consumption?
• The Photometric Interpretation: min-is-black is wrong in my case, should be min-is-white. This isn't really a problem, since I can correct that with tiffset. Only the parameter isn't correct, the image data itself (the levels 0==white and 1==black) are correct.
• Further Ideas?

LG
magier0815

[snibgo]

convert -resize 100%x125% -profile sRGB_IEC61966-2-1_black_scaled.icc Mosaic.tif -intent saturation -profile ISOcoated_v2_300_eci.icc -separate -monochrome -depth 1 -rotate "-90" +compress -density 1000x800 125x100/out-%d.tiff

The command order is bad. Read the file, then process it, then write it. You have "-resize" and "-profile" before you have an image to operate on.


IM Q8 will use half the memory of Q16 for the pixel data.

store the intermediate results to disk (CMYK separeted channels) and sequentially process every channel of CMYK separately.

IM v7 may store single-channel images in memory more efficiently than v6. I haven't tested this.

But which file format should I use?

Generally, either miff or mpc. IM can memory-map mpc files, so they don't need to be read into memory, which will help your situation. But the format is specific to an IM version, eg you can't write a mpc with Q8 and read it with Q16.