[SOLVED] HDRI is now default for IM7 but not IM6 (still Q16)

[NicolasRobidoux]

Cristy and Anthony:
I don't have the time to collect precise examples buttressing the following opinion. If you are not convinced, let me know, and I'll add such to this thread as I run into them.

As usual, this is just an opinion. (But I'm opinionated )

Very strong opinion: The default ImageMagick distribution should be switched from 16-bit to HDRI.

This, of course, does not mean that I suggest that we should get rid of 8- and 16-bit compiles. Just that when a Ubuntu user, say, installs ImageMagick from the package manager, she should get HDRI by default.

Rationale:

Part 0:
Nowadays' consumer hardware has frighteningly fast FPUs. What's somewhat slowish is conversion from floating point to integers. So, although I've not benchmarked, it could be that the HDRI version runs comparably fast, at least when the images are reasonably small. (Memory usage is king in the speed department.)

("Frightening" is the word for someone whose first programming job involved a punch card reader which was physically larger than his office, and whose "internet" connection speed to the very first Canadian VAX---at a now extinct particle physics center---was measured in bauds. Ma Bell bankrupted the center.)

Part 1:
ImageMagick is getting serious about colourspaces.
Now, if you convert, say, an sRGB image imported with a recent ICC profile in Perceptual Rendering Intent mode, to XYZ (a favorite colourspace of many people) with libvips (which I know does colourspace correctly), the probability that you will get out of "nominal" gamut colours (below 0 or above 100) is extremely high. They won't overshoot by much (no more than about 4, usually), but they will. If the XYZ values are clipped when storing into 16-bit, the transformation won't be reversible.
Now, maybe your brand new XYZ implementation prevents this somehow? (I have not checked.)
But the same issue, I expect, will occur with all esoteric colourspaces (Lab, LCh, ...).
Summary: If your default implementation is not HDRI, you are likely to get bug reports from people who use fancy colourspaces.
Nip those in the bud.

Part 2:
With negative lobe filters, we need negative transparency to get best results (documented elsewhere).

Part 3:
The clipping that occurs between the horizontal and vertical passes of orthogonal resize (-resize, the most commonly used) causes filters with negative lobes to have avoidable artifacts. These could be eliminated by making an exception to the between pass storage used by such methods (Anthony mentioned that this was a possibility) but then methods without negative lobes would be slowed down without noticeable gain. In addition, these artifacts are less noticeable when downsampling, which is, I would guess, a major use of ImageMagick. So, one can argue that for many uses, the current implementation is better (speed!).
Making HDRI the default eliminates the issue. The artifacts disappear in the default version. If you want speed, grab an alternate (8- or 16-bit) version, but be aware of the possible consequences.

[magick]

We agree with your rationale for HDRI. We haven't considered it as a default build previously because of the memory issue, performance issues, and some unexpected results when pixels go out of band. HDRI doubles the memory requirements and in some cases there is diminished performance. We are talking about a default setting. Any user and the Linux distributors can readily override the HDRI build with the configure script command line argument --disable-hdri.

Cristy votes yes. Anthony? Fred?

[fmw42]

As I install manually from source, I have no problem either way. The one possible issue for binaries, might be that you either need to include FFTW or let the user know it is not available in the binary and what one needs to do to install it. Perhaps that is a non-issue, since I have not used binaries in quite some time and most people likely do not care about FFT processing.

I think the real issue is your Linux distributors (redhat, etc) who may oppose such automatic changes in their bundle. So I would suggest you pose this to them first. They may not like any potential slow down, though they would gain from quality of results.

[NicolasRobidoux]

...unexpected results when pixels go out of band...

I've been thinking about that some operations pretty much require that pixel values be "in-band" to work correctly, and the best solution I've come up with is that such operations should silently clamp some of the bands to usable ranges before proceeding. Clearly, there could be #defines that only turn on such clamping when one is in HDRI mode. +sigmoidal-contrast is one such operation if I understand correctly (-sigmoidal-contrast is not, again if I understand correctly).

[magick]

Use the -clamp option to clamp HDRI pixels.

[NicolasRobidoux]

Cristy: My suggestion was that some operations should automatically -clamp in HDRI to ensure they work properly.

[magick]

Ok, if you identify which options should be automatically clamped, we'll add a patch to the Subversion trunk.

[fmw42]

Cristy: My suggestion was that some operations should automatically -clamp in HDRI to ensure they work properly.

Sorry to stick my nose in. But that seems to be opposed to the philosophy of HDRI. Can we be sure that any operation will always need clamping. Might some other user think differently. I know I had to clamp before doing -evaluate log, as that seems to be one of those kinds of operations. But I am perfectly willing to -clamp when that seems appropriate or necessary.

I would defer on this if you two and Anthony think it is a good idea. But somewhere in the docs, it should be pointed out which ones are being automatically clamped in HDRI mode.

[NicolasRobidoux]

Fred: I'm only thinking that this is appropriate for operations that may either flat out break (division by zero, log of a negative value, discontinuous/unstable behavior when there should be none, ...) or for which there is a very strong, actually inescapable, reason never ever to use it with out of range values.
If and when I run into an operation that needs automatic clamping to range, I'll flag it in Bugs, giving the reasons why. Then, we can decide together on a case by case basis.
As pointed out by you and Cristy, "most of the time" can be taken care of by manually inserting -clamp. I'll only flag candidates for "always".

[fmw42]

Fred: I'm only thinking that this is appropriate for operations that may either flat out break (division by zero, log of a negative value, discontinuous/unstable behavior when there should be none, ...) or for which there is a very strong, actually inescapable, reason never ever to use it with out of range values.
If and when I run into an operation that needs automatic clamping to range, I'll flag it in Bugs, giving the reasons why. Then, we can decide together on a case by case basis.
As pointed out by you and Cristy, "most of the time" can be taken care of by manually inserting -clamp. I'll only flag candidates for "always".

OK.

By the way, how are you handling the fact that -sigmoidal-contrast becomes degenerate at exactly or very very near 0? It also appears that with the version in use now (I presume the old version even for IM 6) that I can use values smaller than 1 and still see differences between say 0.1 and 1. So my objection that may have caused the original change may have been wrong or misinterpreted for that change. I did not care that a value of 0 was black or gray. My original tests, that may have been in error, seemed to me that -sigmoidal-contrast was clamped at 1. That was my objection. But as I say, it is not doing so now. So there may be no reason to go back to the newer version and try to fix the + form, if you are finding that the older version is working correctly for you.

[NicolasRobidoux]

Fred: I'm looking into the sigmoidal source code very carefully (but part-time).
I am pretty certain it will change.

[fmw42]

Fred: I'm looking into the sigmoidal source code very carefully (but part-time).
I am pretty certain it will change.

OK. What seems to be the issue for you?

[NicolasRobidoux]

Fred: Most likely, I'll "go back" to something very much like the new version, except that I'll make sure that all pieces fit together just right.
I'm starting to get the details.

[anthony]

I have no oppion as to use of HDRI, by default. But it does open up the posibility of users getting more weird or unexpected results.

The real question is now many existing scripts, programs, and API's would break by a drastic change like that!

Besides -clamp, I would like to see -round also added to round values within the Q level being used.

We could probably do it for IMv7, but IMv6 should remain Q16 by default.


One other possibility is to look at, is using HDRI with normalized values 0.0 to1.0, rather than a Q range.
However this would very likely break many things that have an assumption about the use of integers!
Most named colors for example.

Color preservation when doing no-op image conversions may also become more difficult.

[NicolasRobidoux]

There is something logical to IM6 remaining Q16 and IM7 going to HDRI by default.