IM gives profile error when doing composite

[datro]

ImageMagick 7.0.7-22 Q16 HDRI x64 on Windows 10 64-bit

IM is giving me the following error in my Windows batch script:

magick: color profile operates on another colorspace `icc' @ error/profile.c/ProfileImage/1018.

when I do the following sequence of processing on a 16-bit grayscale TIFF image which has an embedded Gray Gamma 2.2 profile (this is a simplified version of the actual code):

Code: Select all

magick -quiet input.tif embedded.icc
magick -quiet input.tif
-profile Gray-elle-V2-g10.icc
-filter Cubic -distort Resize 10800
( -clone 0 -gamma 3 -define convolve:scale=100%,100 -morphology Convolve DoG: ... -gamma 0.333333 )
( -clone 0 -define convolve:scale=100%,100 -morphology Convolve DoG: ... )
-delete 0
( -clone 1 -auto-level )
-compose over -composite
-profile embedded.icc
-set density 720x720
-set comment "Created by ImageMagick"
output.tif

Here's explanation of the steps, line by line:
// save the embedded profile to local disk
// read in the image to memory
// convert to linear gray space using a local profile
// resize the image
// create sharpened image in gamma 3 ("background" for the upcoming composite)
// create sharpened image ("source" for the upcoming composite)
// delete the original image at index 0
// create blend mask based on the "source" image
// blend background and source using mask (composite)
// convert back to original non-linear gray space using the previously saved profile
// set new resolution in meta-data
// add a comment to meta-data
// write the image to output file

As best I can tell, IM seems to have a problem at the profile conversion step (back to the original non-linear gray space). The error message seems to suggest IM was perhaps in an incompatible color space after the composite. Is IM making some bad colorspace assumptions during the cloning and composite steps? Or am I not understanding how to use "-profile" for the conversions in and out of linear space for grayscale files?

Now here's the interesting part: I modified line two above as follows:

Code: Select all

magick -quiet input.tif -type Grayscale

and this eliminates the profile error message. My guess is that the addition of "-type Grayscale" somehow removes the confusion for IM, but I'd like some confirmation that including this is the proper way to fix the error, or if not, some guidance on the proper way to address it.

Thanks,
Dave

[fmw42]

What was the message and was it just a warning or an error?

[datro]

As indicated above, the error is:
magick: color profile operates on another colorspace `icc' @ error/profile.c/ProfileImage/1018.

Magick aborts at this point and the output file is not created.

Thanks,
Dave

[fmw42]

Try to find out where it fails.

Start with

magick -quiet input.tif -profile Gray-elle-V2-g10.icc tmp.miff

or

magick -quiet input.tif embedded.icc
magick -quiet input.tif -profile Gray-elle-V2-g10.icc -profile embedded.icc tmp.miff

do either of those fail?

If not, the add in other line. Or put in +write tmpX.png (X=0,1,2...) in various places and see how many get created.

Perhaps post an example input tif file that fails your command and your full command line. Then we can test it.

[snibgo]

I suggest you simplify the command, to the minimum that shows the problem. Then isolate the option that needs to be removed to cure the problem.

I suspect the issue is that IM (or LCMS) is confused about whether the image is colour or grayscale, and whether one or both the profiles should apply to colour or grayscale images.

[datro]

OK, the following should provide more specific information about where the error is occurring.
In a nutshell, the error occurs on the "-profile embedded.icc" which is converting back to the original non-linear gray space.

I have provided the necessary files and three simple Windows batch test scripts to test it for yourself here:
https://www.dropbox.com/sh/svxba869nhqa ... lD-za?dl=0

The following FAILS (test_magick.bat):

Code: Select all

@echo off

magick -quiet input.tif embedded.icc
magick -quiet input.tif ^
-profile Gray-elle-V2-g10.icc ^
-define filter:c=0.1601886205085204 -filter Cubic -distort Resize 500 ^
( -clone 0 -gamma 3 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 -gamma 0.333333 ) ^
( -clone 0 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 ) ^
-delete 0 ^
( -clone 1 -auto-level ) ^
-compose over -composite ^
-profile embedded.icc ^
output.tif

echo done.

The following WORKS (test_magick_nosharpen.bat - cloning and composite have been removed):

Code: Select all

@echo off

magick -quiet input.tif embedded.icc
magick -quiet input.tif ^
-profile Gray-elle-V2-g10.icc ^
-define filter:c=0.1601886205085204 -filter Cubic -distort Resize 500 ^
-profile embedded.icc ^
output.tif

echo done.

And finally, here is "test_magick_writes.bat" which is identical to the first failing script, but I've added "+write tmpX.tif" (X=0,1,2) on the processing steps starting with the last clone in the sequence:

Code: Select all

@echo off

magick -quiet input.tif embedded.icc
magick -quiet input.tif ^
-profile Gray-elle-V2-g10.icc ^
-define filter:c=0.1601886205085204 -filter Cubic -distort Resize 500 ^
( -clone 0 -gamma 3 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 -gamma 0.333333 ) ^
( -clone 0 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 ) ^
-delete 0 ^
( -clone 1 -auto-level ) +write tmp0.tif ^
-compose over -composite +write tmp1.tif ^
-profile embedded.icc +write tmp2.tif ^
output.tif

echo done.

After running the above script, I have files tmp0.tif and tmp1.tif, but not tmp2.tif. It means the error occurs on the "-profile embedded.icc" when cloning and composite are in the processing sequence.

As mentioned in an earlier post, if I add "-type Grayscale" at the beginning of the command, it works. It seems to me the cloning and/or composite is introducing the confusion for IM. Let me know if there is further testing I can do here.

Thanks,
Dave

[fmw42]

Just guessing, but it might be that the profiles are confusing ImageMagick when trying to go to grayscale profiles rather than grayscale colorspace.

Also try replace -gamma 3 with -evaluate pow 0.333333 and -gamma 333333 with -evaluate pow 3.

If that fails, then post a link on the bugs forum pointing to this post.

[datro]

Important Update:

After some further testing, I now see that the problem lies with the cloning, not specifically with the later "-profile".
I added +writes on each of the cloning steps to see what is output from each cloning step. When I look at the first cloned image, IT HAS BEEN CONVERTED TO Colorspace sRGB even though it is still Type Grayscale! Here's what Magick Identify reports on this tmp image:

Colorspace: sRGB
Type: Grayscale
Base type: TrueColor

So it looks like from that point forward in the processing we are in sRGB, but at the end when we try to do "-profile embedded.icc" to get back to non-linear grayspace, IM throws the error, complaining that there is a mis-match in colorspace.

So, my conclusion now is that the defect exists in "-clone". It should NOT do a colorspace conversion to sRGB on a Grayscale input image. What I would expect to see from the cloned image is Colorspace=LinearGray, Type=Grayscale in my case. (I'm not sure why the -clone causes the "TrueColor" base type data to be added).

I will submit a new post to the "Bugs" forum and link to this thread.

Dave

[fmw42]

I tried your command in IM 7.0.7.22 Q16 HDRI Mac OSX Sierra.

This works without adding back the embedded profile.

Code: Select all

magick -quiet input.tif embedded.icc
magick -quiet input.tif \
-profile Gray-elle-V2-g10.icc +write tmp1.png \
-define filter:c=0.1601886205085204 -filter Cubic -distort Resize 500 +write tmp2.png \
\( -clone 0 -gamma 3 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 -gamma 0.333333 +write tmp3.png \) \
\( -clone 0 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 +write tmp4.png \) \
-delete 0 \
\( -clone 1 -auto-level +write tmp5.png \) \
-compose over -composite +write tmp6.png \
output.tif

But if fails when I add it back.

Code: Select all

magick -quiet input.tif embedded.icc
magick -quiet input.tif -strip \
-profile embedded.icc -profile Gray-elle-V2-g10.icc +write tmp1.png \
-define filter:c=0.1601886205085204 -filter Cubic -distort Resize 500 +write tmp2.png \
\( -clone 0 -gamma 3 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 -gamma 0.333333 +write tmp3.png \) \
\( -clone 0 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 +write tmp4.png \) \
-delete 0 \
\( -clone 1 -auto-level +write tmp5.png \) \
-compose over -composite +write tmp6.png \
-profile embedded.icc \
output.tif

Note that all tmp images are create, but no output.tif. I even tried stripping the image of all meta data and putting back the original profile. I also tried output as PNG and JPG, but they failed also.

I do not think it is a clone issue, but one having to do with the profiles you are using. Though I am not sure why.

This seems to me to be a bug and needs the IM developers to review comment further.

[datro]

Fred,
Thanks for your testing and further information. Please see my last post on this thread (before this one) for more information in case you didn't see it.

I still think it is a -clone issue, because when I did a -verbose listing of info from each tmp file that I created on each step I can see that the first -clone does a conversion into sRGB colorpace (and from there the processing apparently stays in sRGB until we hit the "-profile" at the end when the error is thrown. I think -clone is the real source of the problem, not the profiles. If I take out the clone and composite steps, it works just fine, including assigning the original non-linear profile at the end.

Dave

[snibgo]

I added +writes on each of the cloning steps to see what is output from each cloning step. When I look at the first cloned image, ...

I'm not sure what you mean. Did the "+write" create a file that you then examined? Or did you "+write info:"?

[fmw42]

Using MIFF for the write files

Code: Select all

magick -quiet input.tif embedded.icc
magick -quiet input.tif -strip \
-profile embedded.icc -profile Gray-elle-V2-g10.icc +write tmp1.miff \
-define filter:c=0.1601886205085204 -filter Cubic -distort Resize 500 +write tmp2.miff \
\( -clone 0 -gamma 3 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 -gamma 0.333333 +write tmp3.miff \) \
\( -clone 0 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 +write tmp4.miff \) \
-delete 0 \
\( -clone 1 -auto-level +write tmp5.miff \) \
-compose over -composite +write tmp6.miff \
-profile embedded.icc \
output.tif

Code: Select all

magick identify

Code: Select all

tmp1.miff MIFF 600x761 600x761+0+0 16-bit Grayscale Gray 915071B 0.010u 0:00.009
tmp2.miff MIFF 500x634 500x634+0+0 16-bit Grayscale Gray 635839B 0.000u 0:00.000
tmp3.miff MIFF 500x634 500x634+0+0 16-bit Grayscale Gray 635839B 0.010u 0:00.000
tmp4.miff MIFF 500x634 500x634+0+0 16-bit Grayscale Gray 635839B 0.000u 0:00.009
tmp5.miff MIFF 500x634 500x634+0+0 16-bit Grayscale Gray 635839B 0.000u 0:00.000
tmp6.miff MIFF 500x634 500x634+0+0 16-bit TrueColor sRGB 1.81576MiB 0.000u 0:00.

and

Code: Select all

magick identify -verbose tmp6.miff

Code: Select all

 Format: MIFF (Magick Image File Format)
  Class: DirectClass
   Colorspace: sRGB
  Type: TrueColor

It is a puzzle to me how doing a compose over -composite of two grayscale image creates a true color sRGB image. It does the same in IM 6.9.9.34.


However, If I run the same command in IM 6.9.9.34, I get no message and I get an output file.

Code: Select all

convert -quiet input.tif embedded.icc
convert -quiet input.tif -strip \
-profile embedded.icc -profile Gray-elle-V2-g10.icc +write tmp1.miff \
-define filter:c=0.1601886205085204 -filter Cubic -distort Resize 500 +write tmp2.miff \
\( -clone 0 -gamma 3 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 -gamma 0.333333 +write tmp3.miff \) \
\( -clone 0 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 +write tmp4.miff \) \
-delete 0 \
\( -clone 1 -auto-level +write tmp5.miff \) \
-compose over -composite +write tmp6.miff \
-profile embedded.icc \
output.tif

Code: Select all

identify tmp*.miff

Code: Select all

tmp1.miff MIFF 600x761 600x761+0+0 16-bit Grayscale Gray 915031B 0.010u 0:00.000
tmp2.miff MIFF 500x634 500x634+0+0 16-bit Grayscale Gray 635799B 0.000u 0:00.000
tmp3.miff MIFF 500x634 500x634+0+0 16-bit Grayscale Gray 635799B 0.000u 0:00.009
tmp4.miff MIFF 500x634 500x634+0+0 16-bit Grayscale Gray 635799B 0.000u 0:00.000
tmp5.miff MIFF 500x634 500x634+0+0 16-bit Grayscale Gray 635799B 0.000u 0:00.009
tmp6.miff MIFF 500x634 500x634+0+0 16-bit TrueColor sRGB 1.81572MiB 0.000u 0:00.000

So it appears to be an IM 7 bug.

[datro]

I added +writes on each of the cloning steps to see what is output from each cloning step. When I look at the first cloned image, ...

I'm not sure what you mean. Did the "+write" create a file that you then examined? Or did you "+write info:"?

Each write created a differently-named "intermediate" tif file which I then examined with magick identify -verbose. That's how I noticed that on the first clone an unexpected conversion to sRGB was being done.

Here's the script:

Code: Select all

@echo off

magick -quiet input.tif embedded.icc
magick -quiet input.tif ^
-profile Gray-elle-V2-g10.icc +write tmp0_profileIn.tif ^
-define filter:c=0.1601886205085204 -filter Cubic -distort Resize 500 +write tmp1_resize.tif ^
( -clone 0 -gamma 3 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 -gamma 0.333333 +write tmp2_clone_first.tif ) ^
( -clone 0 -define convolve:scale=100%,100 -morphology Convolve DoG:3,0,0.4981063336734057 +write tmp3_clone_second.tif ) ^
-delete 0 ^
( -clone 1 -auto-level +write tmp4_clone_third.tif ) ^
-compose over -composite +write tmp5_composite.tif ^
-profile embedded.icc +write tmp6_profileOut.tif ^
output.tif

echo done.

More info: Based on more testing I've just now completed, I am wondering if using -gamma on the first clone is somehow involved here.
My "tmp2_clone_first.tif" data reveals the conversion to sRGB, but "tmp3_clone_second" (without the -gamma settings) results in the expected Gray colorspace. The third clone also results in Gray, but then the Composite results in sRGB (most likely since one of the two images being composited is sRGB).

Does "-gamma" result in a colorspace conversion??

Dave

[snibgo]

Each write created a differently-named "intermediate" tif file which I then examined with magick identify -verbose.

So you don't know it was the clone. It could be the clone, or writing to a tif, or reading from a tif. If you use "-verbose +write info" that will exclude the tif read/write from the problem.

"-gamma" is, in some sense, a colorspace conversion. But "-evaluate Pow" isn't. I suggest you follow Fred's suggestion, and use Pow instead.

[datro]

Taking a deep breath here...this problem is getting more complex, but I still think there is a bug.
After additional testing, I can report the following (using more concise and informative scripts).

Case 1 (Resize included, but no processing on the clones)
This WORKS:

Code: Select all

magick -quiet input.tif embedded.icc
magick -quiet -verbose input.tif ^
-profile Gray-elle-V2-g10.icc ^
-define filter:c=0.1601886205085204 -filter Cubic -distort Resize 500 +write tmp1 ^
( -clone 0 +write tmp2 ) ^
( -clone 0 ) ^
-delete 0 ^
( -clone 1 -auto-level ) ^
-compose over -composite +write tmp3 ^
-profile embedded.icc ^
output.tif

Code: Select all

input.tif TIFF 600x761 600x761+0+0 16-bit Grayscale Gray 942748B 0.078u 0:00.076
Gray-elle-V2-g10.icc ICC 1x1 1x1+0+0 16-bit sRGB 512B 0.000u 0:00.000
Affine Projection:
  -distort AffineProjection \
      '0.833333,-0.000000,-0.000000,0.833114,0.000000,0.000000'
Affine Distort, FX Equivelent:
  -size 502x636 -page -1-1 xc: +insert \
  -fx 'ii=i+page.x+0.5; jj=j+page.y+0.5;
       xx=+1.200000*ii +0.000000*jj +0.000000;
       yy=+0.000000*ii +1.200315*jj +0.000000;
       v.p{ xx-v.page.x-.5, yy-v.page.y-.5 }' \
input.tif=>tmp1 TIFF 600x761=>500x634 16-bit Grayscale Gray 663416B 0.031u 0:00.007
input.tif=>tmp2 TIFF 600x761=>500x634 16-bit Grayscale Gray 663416B 0.000u 0:00.006
input.tif=>tmp3 TIFF 600x761=>500x634 16-bit Grayscale Gray 663504B 0.000u 0:00.006
embedded.icc ICC 1x1 1x1+0+0 16-bit sRGB 408B 0.000u 0:00.000
input.tif=>output.tif TIFF 600x761=>500x634 16-bit Grayscale Gray 663400B 0.156u 0:00.066

As you can see in the verbose output above, everything stays in "...Grayscale Gray..." throughout. (It does seem a bit odd that IM reports the profiles as being "sRGB" but I don't think that is the source of the problem...I suspect it is just an oddity of how -verbose works).

Case 2 (No resize, simple gamma conversion on the first clone)
This also WORKS:

Code: Select all

magick -quiet input.tif embedded.icc
magick -quiet -verbose input.tif ^
-profile Gray-elle-V2-g10.icc ^
( -clone 0 -gamma 3 -gamma 0.3333333333333333 +write tmp2 ) ^
( -clone 0 ) ^
-delete 0 ^
( -clone 1 -auto-level ) ^
-compose over -composite +write tmp3 ^
-profile embedded.icc ^
output.tif

Code: Select all

input.tif TIFF 600x761 600x761+0+0 16-bit Grayscale Gray 942748B 0.078u 0:00.078
Gray-elle-V2-g10.icc ICC 1x1 1x1+0+0 16-bit sRGB 512B 0.000u 0:00.000
input.tif=>tmp2 TIFF 600x761 600x761+0+0 16-bit Grayscale Gray 942616B 0.000u 0:00.007
input.tif=>tmp3 TIFF 600x761 600x761+0+0 16-bit Grayscale Gray 942704B 0.063u 0:00.010
embedded.icc ICC 1x1 1x1+0+0 16-bit sRGB 408B 0.000u 0:00.000
input.tif=>output.tif TIFF 600x761 600x761+0+0 16-bit Grayscale Gray 942600B 0.297u 0:00.097

As before, everything stays in "...Grayscale Gray..."

Case 3 (Resize included, AND simple gamma conversion on the first clone)
This FAILS!

Code: Select all

magick -quiet input.tif embedded.icc
magick -quiet -verbose input.tif ^
-profile Gray-elle-V2-g10.icc ^
-define filter:c=0.1601886205085204 -filter Cubic -distort Resize 500 +write tmp1 ^
( -clone 0 -gamma 3 -gamma 0.3333333333333333 +write tmp2 ) ^
( -clone 0 ) ^
-delete 0 ^
( -clone 1 -auto-level ) ^
-compose over -composite +write tmp3 ^
-profile embedded.icc ^
output.tif

Code: Select all

input.tif TIFF 600x761 600x761+0+0 16-bit Grayscale Gray 942748B 0.078u 0:00.078
Gray-elle-V2-g10.icc ICC 1x1 1x1+0+0 16-bit sRGB 512B 0.000u 0:00.000
Affine Projection:
  -distort AffineProjection \
      '0.833333,-0.000000,-0.000000,0.833114,0.000000,0.000000'
Affine Distort, FX Equivelent:
  -size 502x636 -page -1-1 xc: +insert \
  -fx 'ii=i+page.x+0.5; jj=j+page.y+0.5;
       xx=+1.200000*ii +0.000000*jj +0.000000;
       yy=+0.000000*ii +1.200315*jj +0.000000;
       v.p{ xx-v.page.x-.5, yy-v.page.y-.5 }' \
input.tif=>tmp1 TIFF 600x761=>500x634 16-bit Grayscale Gray 663416B 0.031u 0:00.008
input.tif=>tmp2 TIFF 600x761=>500x634 16-bit Gray 1.84195MiB 0.000u 0:00.006
input.tif=>tmp3 TIFF 600x761=>500x634 16-bit sRGB 1931510B 0.063u 0:00.008
embedded.icc ICC 1x1 1x1+0+0 16-bit sRGB 408B 0.000u 0:00.000
magick: color profile operates on another colorspace `icc' @ error/profile.c/ProfileImage/1018.

In this case, something has happened on the first clone (with the gamma conversion) output (tmp2). Instead of "Grayscale Gray" being reported, we now see only "Gray". Furthermore, the composite output is now in sRGB!

I wrote a simple script to report selected "magick identify" information from each of the above tmp files and it produces the following:

Code: Select all

Image: tmp1
Colorspace: Gray
Type: Grayscale
Gamma: 0.454545
Profile: Gray-elle-V2-g10.icc
--------------------------
Image: tmp2
Colorspace: sRGB
Type: Grayscale
Gamma: 0.454545
Profile: Gray-elle-V2-g10.icc
--------------------------
Image: tmp3
Colorspace: sRGB
Type: Grayscale
Gamma: 0.454545
Profile: Gray-elle-V2-g10.icc
--------------------------

On tmp2 I see the Colorspace as "sRGB" and the Type as "Grayscale". Something is not right here and it appears this incongruity just carries forward and ultimately convinces the composite to create sRGB output which apparently causes the last "-profile" to barf and IM aborts.

@Alan and Fred: I also tested the above scenarios with "-evaluate pow .." as you suggested; NO CHANGE. Same results whether the clone is using -gamma or -evaluate pow.

For me, the data above demonstrates there is some weird interaction going on between the resize and the clone with a gamma (or eavluate pow) conversion, and this results in incorrect colorspace behavior downstream in the processing. Where do we go from here?

Thanks,
Dave