[Resolved] possible bug linear gray still has gamma=0.4545

[Dabrosny]

If I understand you correctly, you want a new colorspace lineargray or some such thing. But this would only be understood internally by IM. I do not believe any image format would know what to do with that (lineargray named) colorspace. Glennrp can correct me.

Almost none of our internal colorspaces are understood by the PNG or JPG format for example (but all are understood by IM when read back in the MIFF format). This fact would not change at all.

We need this internally so that our internal processing will behave as expected, especially when converting from grayscale to a color colorspace, so that IM knows whether gamma compression is needed as part of the conversion or not.

It appears that -grayscale rec709luminance is what you want, but you want it to be called something like lineargray.

rec709luminance is not the name of a colorspace in IM; if it were then I would have the colorspace i want (although its name would not be ideal in my opinion).

Since it is currently labeled gray with gamma=1, those image formats that can deal with it are fine and internally IM can handle it.

IM can handle it but it isn't doing so. It's much cleaner to use the colorspace to identify whether the grayscale data represents linear or sRGB-nonlinear for example, rather than only relying on gamma, which is not done for the color colorspaces. Glennrp seems to be on the same page on this one based on his prev reply.

The only issue I see is that IM may not know the difference between gray gamma=1 from rec709luminance and gray gamma=1 from rec601luminance.

As I said, once the conversions are applied correctly (which involves a different set of coefficients), rec709luminance values should be numerically identical to rec601luminance values.
The problem you are referring to already exists for the linear RGB colorspace (we can't tell whether it's RGB based on rec601 or rec709 primaries) but is no longer a problem after combined into a single linearw luminance channel. As RGB values they are not the same because of different primary chromaticities but *after* they are combined into Y that difference no longer has an effect.

[Dabrosny]

Glennrp, what's the next step for your proposal to add two explicit colorspaces in addition to the generic gray?

Fred, just to explain my last response further

As I said, once the conversions are applied correctly (which involves a different set of coefficients), rec709luminance values should be numerically identical to rec601luminance values.
The problem you are referring to already exists for the linear RGB colorspace (we can't tell whether it's RGB based on rec601 or rec709 primaries) but is no longer a problem after combined into a single linearw luminance channel. As RGB values they are not the same because of different primary chromaticities but *after* they are combined into Y that difference no longer has an effect.

The -grayscale rec709 luminance can only be applied correctly (i.e. correctly producing a luminance as a result) to values that already use rec709 primaries (like sRGB), while -grayscale rec601luminance can only be applied correctly to values that already use rec601 primaries. The coefficients are different because the assumed pre-existing colorspace is assumed to be different. The purpose of both transformations is to convert those two different colorspaces to the same resulting 1-channel colorspace: linear Luminance!

[fmw42]

I still do not understand what good this does. It is only good internally to IM. No other output format will understand these colorspaces. IM can already produce CIE Y linear gray via -grayscale rec709luminance as colorspace gray with gamma=1. Since it is what you want (correct me if I am wrong), what else is needed for internal IM processing. IM will continue to process it as linear since the gamma=1 (unless you mix it with a non-linear gray image). Don't get me wrong, I am not against new colorspaces, but I just don't see any benefit you gain by adding a new colorspace that does just what IM already has with -grayscale rec709luminance and cannot be understood by external image formats for use in other applications or for storage.

What am I missing here? How do you want to make use of the new colorspace that adds new functionality. That is, what do you want to do with the new colorspace that you cannot already do now? Can you give me a simple examples of the kind of processing you would do that you cannot do now.

[Dabrosny]

I still do not understand what good this does. It is only good internally to IM. No other output format will understand these colorspaces. IM can already produce CIE Y linear gray via -grayscale rec709luminance as colorspace gray with gamma=1. Since it is what you want (correct me if I am wrong), what else is needed for internal IM processing. IM will continue to process it as linear since the gamma=1 (unless you mix it with a non-linear gray image). Don't get me wrong, I am not against new colorspaces, but I just don't see any benefit you gain by adding a new colorspace that does just what IM already has with -grayscale rec709luminance and cannot be understood by external image formats for use in other applications or for storage.

What am I missing here? How do you want to make use of the new colorspace that adds new functionality. That is, what do you want to do with the new colorspace that you cannot already do now? Can you give me a simple examples of the kind of processing you would do that you cannot do now.

First of all, IM will not in general "continue to process it as linear since the gamma=1", or at least not today. When I indicate that I want to convert it to sRGB colorspace using -colorspace sRGB it will fail to apply the gamma compression to the linear image, so the data will not, in fact, be in the sRGB colorspace, though it will appear to be by its colorspace and also it will have changed image->gamma to 0.454545 which also makes it appear nonlinear.

Okay maybe we're just going 'round in circles. All the functionality is in IM if you can figure out how to invoke it correctly at each stage. The benefit of having a linear gray colorspace is the IM will already *know* that when you do a -colorspace sRGB on a LinearGray image, it should automatically, by default, do an sRGB gamma compression in order to actually have the data be correct in the sRGB colorspace. Without that, the *user* has to tell IM what to do in each case, because IM doesn't know.

By your logic, we don't need need both RGB and sRGB colorspaces so long as IM provides functions that we can use when we need it in order to apply or remove the sRGB gamma compression. Or why don't we simply use RGB but set gamma=2.2 instead of having sRGB? That would be just like what you're proposing we do for single-channel images, which should, as Glennrp points out, not be treated differently from color images with regard to this sort of thing.

[fmw42]

First of all, IM will not in general "continue to process it as linear since the gamma=1", or at least not today. When I indicate that I want to convert it to sRGB colorspace using -colorspace sRGB it will fail to apply the gamma compression to the linear image, so the data will not, in fact, be in the sRGB colorspace, though it will appear to be by its colorspace and also it will have changed image->gamma to 0.454545 which also makes it appear nonlinear.

Yes it will, if all the fixes are made so that it behaves as it was intended in IM 6.8.7.1

Okay maybe we're just going 'round in circles. All the functionality is in IM if you can figure out how to invoke it correctly at each stage. The benefit of having a linear gray colorspace is the IM will already *know* that when you do a -colorspace sRGB on a LinearGray image, it should automatically, by default, do an sRGB gamma compression in order to actually have the data be correct in the sRGB colorspace. Without that, the *user* has to tell IM what to do in each case, because IM doesn't know.

Same as above. It should be doing that now, but there are bugs that were not there earlier.

By your logic, we don't need need both RGB and sRGB colorspaces so long as IM provides functions that we can use when we need it in order to apply or remove the sRGB gamma compression. Or why don't we simply use RGB but set gamma=2.2 instead of having sRGB? That would be just like what you're proposing we do for single-channel images, which should, as Glennrp points out, not be treated differently from color images with regard to this sort of thing.

The difference is that linear RGB is a valid colorspace at least for PNG. But PNG only knows about gray being linear or non-linear by its gamma value. I do not believe the PNG standard supports special gray colorspaces. Also sRGB is not exactly the same as applying gamma=2.2.


Again I am not opposed to adding new colorspaces. If you can convince the IM developers, that is great. So as I said before, post an enhancement request on the Developers forum and succinctly explain why you think it is needed.

[Dabrosny]

The difference is that linear RGB is a valid colorspace at least for PNG. But PNG only knows about gray being linear or non-linear by its gamma value. I do not believe the PNG standard supports special gray colorspaces. Also sRGB is not exactly the same as applying gamma=2.2.

It has a way of indicating srgb with a special chunk, but I don't think png has a way of indicating specifically that the file having red, blue, and green channels is in a *linear* colorspace, which is what IM means by -colorspace RGB. Except of course gamma=1, which is fine for png but in IM we indicate it mainly by colorspace "RGB" rather than just gamma, so similarly I think we should indicate linearity for single-channel images by colorspace as well.

Again I am not opposed to adding new colorspaces. If you can convince the IM developers, that is great. So as I said before, post an enhancement request on the Developers forum and succinctly explain why you think it is needed.

Okay, thanks, maybe I'll start a new one and quote Glennrp's response from here.

Or maybe I'll first focus on fixing the current default for -colorspace gray, which is actually more important to me.

[fmw42]

Or maybe I'll first focus on fixing the current default for -colorspace gray, which is actually more important to me.

What is wrong? The current default for -colorspace gray should be the same as nonlinear rec709luma. IM mostly assumes gray is non-linear. You must use either -grayscale rec709luminance or use -colorspace gray -colorspace RGB to get a linear gray image.

imbh convert rose: -colorspace gray rose_gray1.png
imbh convert rose: -colorspace rec709luma rose_gray2.png
imbh compare -metric rmse rose_gray1.png rose_gray2.png null:
0 (0)

imbh convert rose: -colorspace gray rose_gray1.png
imbh convert rose: -grayscale rec709luma rose_gray3.png
imbh compare -metric rmse rose_gray1.png rose_gray3.png null:
0 (0)




imbh convert rose: -grayscale rec709luminance rose_linear1.png
Image: rose_linear1.png
Geometry: 70x46+0+0
Page geometry: 70x46+0+0
Class: PseudoClass
Colorspace: Gray
Type: Grayscale
Depth: 8/1-bit
Alpha: False
Channels: gray
Rendering intent: Undefined
Gamma: 1
Colors: 200
Gray:
min: 5 (0.0196078)
max: 255 (1)
mean: 49.9761 (0.195985)
standard deviation: 53.9087 (0.211407)



imbh convert rose: -colorspace gray -colorspace RGB rose_linear2.png
Image: rose_linear2.png
Geometry: 70x46+0+0
Page geometry: 70x46+0+0
Class: PseudoClass
Colorspace: Gray
Type: Grayscale
Depth: 8/1-bit
Alpha: False
Channels: gray
Rendering intent: Undefined
Gamma: 1
Colors: 203
Gray:
min: 5 (0.0196078)
max: 255 (1)
mean: 43.3106 (0.169845)
standard deviation: 54.5687 (0.213995)


I would think that these should be the same, but are not quite. The means and std are different.

imbh compare -metric rmse rose_linear2.png rose_linear1.png null:
2886.83 (0.0440501)

So looks like perhaps another bug.

[snibgo]

Yes! I agree with you 100%. But then why doesn't the colorspace of a grayscale image show as "RGB" or "sRGB" in IM? Instead it shows as "gray", from which we can't tell whether its values represent linear (RGB) data or nonlinear (sRGB) data.

Yes. I'd like IM to show the colorspace as "RGB" or "sRGB" as appropriate. I don't know why IM has a "gray" colorspace. I can't see that it needs one.

An image that is gray may be in RGB, sRGB, Lab, HSL, or any of the other colorspaces. IM should keep track of the colorspace, and provide the correct transformation from one colorspace to another, whether an image happens to be gray or not. Mostly, it does so. (Sometimes it doesn't and we need to explicitly "-set colorspace XX". IM can be enhanced so it always keeps track of the colorspace; adding new colorspaces is a separate issue.)

I'm struggling to see the benefits of new colorspaces GrayLog and GrayLinear. I suppose they would enable operations like "-colorspace GrayLinear" to provide an alternative to "-grayscale Rec709Luminance".

Do you want a gray equivalent of RGB, sRGB, Lab, HSL, and all the other colorspaces?

[fmw42]

I don't know why IM has a "gray" colorspace. I can't see that it needs one.

In IM 6, since grayscale images have 3 channels, there is really no difference. But it tells other applications that since it is gray, to only create one channel. The same will be true in IM 7.

An image that is gray may be in RGB, sRGB, Lab, HSL, or any of the other colorspaces. IM should keep track of the colorspace, and provide the correct transformation from one colorspace to another, whether an image happens to be gray or not.

Yes that could be an issue. Was it gray rec709luma or gray rec601luma? IM does not know nor if it is one channel from Lab vs YCbCr or any one channel from any colorspace..

[Dabrosny]

Yes! I agree with you 100%. But then why doesn't the colorspace of a grayscale image show as "RGB" or "sRGB" in IM? Instead it shows as "gray", from which we can't tell whether its values represent linear (RGB) data or nonlinear (sRGB) data.

Yes. I'd like IM to show the colorspace as "RGB" or "sRGB" as appropriate. I don't know why IM has a "gray" colorspace. I can't see that it needs one.

An image that is gray may be in RGB, sRGB, Lab, HSL, or any of the other colorspaces.

I'd be okay with that so long as IM had some way of telling that a given image was meant to be single-channel, and how that channel is related to the full colorspace (e.g. does it represent the L in Lab or HSL, etc.).

Do you want a gray equivalent of RGB, sRGB, Lab, HSL, and all the other colorspaces?

I wouldn't add one for each color colorspace unless someone says it serves some purpose.

The only grayscale colorspaces I think are fundamentally useful are colorimetric, meaning that they can be mapped directly (though often nonlinearly) to lightness as perceived by humans. This would include (linear) luminance and nearly any nonlinear function thereof. If two colors (as represented in any colorspace having a reasonable color gamut) look just as light as one another to a human, that means they should have the same luminance Y as one another, as well as the same Y^(2.2), Y^(1/3)-223.4, etc, as one another.

That would include L* from LAB but not any of the Lumas like those from YCbCr, YIQ, etc. because these are arbitrarily defined as a weighted sum of separate nonlinear R', G', and B', which makes them pretty much unrelated to how humans or any other animals perceive lightness.

[fmw42]

From Magick:

The proper equivalent to:

convert rose: -grayscale rec709luminance rose_linear1.png

is

convert rose: -colorspace RGB -intensity Rec709luminance -colorspace gray rose_linear2.png

and not

convert rose: -colorspace gray -colorspace RGB rose_linear2.png

[fmw42]

My original post about

convert xc:"gray(50%)" gray_linear.png

producing gamma=0.4545 rather than 1 is now fixed in IM 6.8.7.1 release. It now produces gamma=1.