At first I thought it would be a trivial task but after spending two evenings trying i give up and hope somebody can help. I want to apply 3 simple operations to each pixel i.e.:
1. divide each channel value by a floating number
2. floor the result
3. multiply by a floating number
In 8 bit mode this works alright however the results in 16 bit are completely different:
Example:
The pixel in Q8 is (116, 106, 34)
Step 1. Divide each value by 64 -> (1.8125, 1.65625, 0.53125)
Step 2. Floor each value --> (1,1,0)
Step 3. Multiply each value by 85.333 --> (85.33333333333333, 85.33333333333333, 0) --> Final result (86, 86, 0)
Now in Q16 this operation obviously is not that straight forward and I can't figure out what I'm doing wrong or whether it's IM that rounds up values somewhere
I've tried scaling Q16 values down to 8bit but still getting different results:
The pixel in Q16 is (29812, 27242, 8738)
Step 1.
Code: Select all
-evaluate divide 16448 -evaluate multiply 257 --> (2, 2, 0)
I can't understand why the B in the above result is still 0. Is IM rounding up all the values after each evaluate ?
Somebody might ask what a big deal it's only 1 level difference? Well if the channel value after flooring is equal 0 then whatever it's multiplied by it still will be 0 right?
Can somebody please advise how to can I implement these 3 steps in one command (ideally without using -fx operator)
Thanks