-fx return raw values?

[TedBaker]

Is there a simple way to use the -fx command with raw values rather than normalised, i.e. 0-1

for example

convert rose: -fx 'debug(u)' null

I would like to work with the raw 16bit value stored, or even its 8-bit equivalent, if possible.

[fmw42]

try

Code: Select all

convert rose: txt:

or

Code: Select all

convert rose: -fx 'debug(65535*u)' null:

[snibgo]

The constant Fred give, 65535, is correct for Q16 IM. For portability across any Q-number IM, use QuantumRange.

[fmw42]

The constant Fred give, 65535, is correct for Q16 IM. For portability across any Q-number IM, use QuantumRange.

Good point.

But if he is on Q16 and wants 8-bit values, then it would need to be

Code: Select all

convert rose: -fx 'debug(255*u)' null:

The txt: output provides 8-bit values in the srgb(...) values and quantum range in the raw values right after the coordinates.

[TedBaker]

Thanks for those, was just wondering if there was switch as it can be a bit laborious, with long fx statements.

With the precision of the value give by of u, will i get the exact value when I do 65535*u?

[fmw42]

u is a float in the range of 0 to 1. So multiplying by 65535 would produce a floating point value. I do not know if it is being rounded or truncated to an integer. But I suspect that the IM developers have done whatever is correct to make it properly an integer.

[snibgo]

I do not know if it is being rounded or truncated to an integer. But I suspect that the IM developers have done whatever is correct to make it properly an integer.

It isn't deliberately rounded or truncated to an integer, but the current "-precision" setting may round it to an integer. The output may not be an integer, eg it could be "29183.999999999996". If an integer is required, a common technique is to add 0.5 and take the floor of the result, eg

Code: Select all

convert toes.png -precision 19 -fx debug(floor(QuantumRange*u+0.5)) null: