Page 3 of 4

Re: Get perceptual hash value for image using command line

Posted: 2016-08-20T19:28:26-07:00
by snibgo
Ah, thanks. I wondered whether 11 was mathematical or a coincidence.

I've expanded my trials to include 2-channel colorspaces CbCr CC DbDr IQ and UV that are really YCbCr etc without the Y channel.

Re: Get perceptual hash value for image using command line

Posted: 2016-08-21T10:30:33-07:00
by snibgo
Okay, so the maximum PH is 11. What is the minimum? I thought the minimum was 0.0 but I was wrong. For example:

Code: Select all

identify -verbose -define identify:moments logo:
We have negative PH in the Chroma channel. When this happens, it throws my proportional difference calculation.

Are negative values correct, or is there a bug?

Re: Get perceptual hash value for image using command line

Posted: 2016-08-21T13:38:03-07:00
by fmw42
I believe the negative values occur when the moment is (slightly) larger than 1, so that the negative log of the moment is negative. I have puzzled what to do about that for some time with no conclusion. I do not know if it would be better to clamp to 0 or not. Open for consideration and recommendations.

Re: Get perceptual hash value for image using command line

Posted: 2016-08-21T14:11:37-07:00
by snibgo
Ah, thanks, that makes sense.

So, when I clamp negative values to zero, my comparison tests (656 images, 30 colorspaces) give slightly worse results. So, no, I don't think they should be clamped.

I have seen values around -4. Is there a theoretical minimum? If not, maybe they should be clamped to -11, to be symmetrical with the maximum.

Re: Get perceptual hash value for image using command line

Posted: 2016-08-21T15:51:25-07:00
by fmw42
snibgo wrote: I have seen values around -4. Is there a theoretical minimum? If not, maybe they should be clamped to -11, to be symmetrical with the maximum.
Theoretical minimum? For what? The moment or the hash value -log(moment)? Do you mean biggest negative value? If so, I do not recall that it is clamped at -11.

Re: Get perceptual hash value for image using command line

Posted: 2016-08-21T16:16:07-07:00
by snibgo
Sorry, yes, I mean the perceptual hash value. So the range of possible values is -infinity to +11.0? Okay, I can live with that.

Re: Get perceptual hash value for image using command line

Posted: 2016-08-21T17:13:39-07:00
by fmw42
snibgo wrote:Sorry, yes, I mean the perceptual hash value. So the range of possible values is -infinity to +11.0? Okay, I can live with that.
If you find a clamp to some largest negative value makes sense (perhaps -11), let me know. We can change the code later after we get your conclusions.

I am still not sure what to do about the negative values. It has bothered me since I first saw them. But they have been rare for me.

I do not think you will get really large negative values since the moments are typically less than 1 and probably do not exceed one by too much. No proof on that, though.

Re: Get perceptual hash value for image using command line

Posted: 2016-08-21T20:01:39-07:00
by snibgo
I'm not keen on clipping values without good reason.

The most negative PH value in my current test is -4.02589. 3276 of the 367360 values are negative, so about 1% of the values. Only 88 of those are less than -1.0.

15170 values (4% of the values) are exactly 11.0.

Half the values are below 4.72312. The mean is 4.96005.

Re: Get perceptual hash value for image using command line

Posted: 2016-08-23T02:45:15-07:00
by snibgo
I've uploaded Perceptual hash tests, which describes the tests I did, results, conclusions, and recommendations for IM internals.

Recommendations in a nutshell: use at least two colorspaces. Default to using xyY and HSB. Provide facilities to use any colorspaces (and any number of them), and to omit any channels.

Feel free to comment, anyone.

Re: Get perceptual hash value for image using command line

Posted: 2016-08-23T08:54:24-07:00
by fmw42
Why HSB? I though you suggested that H was not good, especially for red images?

I am just reading your tests. Note that the IM PHASH uses HCLp not HCL and I do not see HCLp in your list of tests. There is a difference between HCLp and HCL.

Re: Get perceptual hash value for image using command line

Posted: 2016-08-23T09:21:48-07:00
by fmw42
Seems to me that if there is potential for H to cause problems then one might want to use your second choice of xyY and YDbDr.

Do you have a list of your attacks, especially the hue changes?

Can you clarify? Did you rotate all the images and crop off the background padding and use those for you tests or was this rotation just one attack?

What did you do in regards to vertical and horizontal crops as attacks? Please clarify.

By the way, that was an massive and exhaustive set of tests. Thanks for this more thorough testing and your conclusions.

Re: Get perceptual hash value for image using command line

Posted: 2016-08-23T09:37:02-07:00
by snibgo
Yes, for some work I did, CL+sRGB performed better than HCL+sRGB. That dataset wasn't artificial (it was edited photos), and the majority of tweaks were colour or tone shifts.

I intend to get my hands on that data again, and try my test on it, or at least a subset.

This test set for my web page had far fewer colour shifts. The second-best pair was xyY and YDbDr, which might work far better on the edited photos. This is why I suggest xyY and HSB should be the defaults, but that users must be able to change them. I don't think one particular combination of colorspaces can be the best for all situations.
fmw42 wrote:There is a difference between HCLp and HCL.
Is there?

Code: Select all

%IM%convert hald:10 ( -clone 0 -colorspace HCLp ) ( -clone 0 -colorspace HCL ) -delete 0 -metric RMSE -format %[distortion] -compare info:

0

%IM%convert hald:10 ( -clone 0 -colorspace HCLp ) ( -clone 0 -colorspace HCL ) -delete 0 -metric AE -format %[distortion] -compare info:

0

Re: Get perceptual hash value for image using command line

Posted: 2016-08-23T09:50:01-07:00
by snibgo
fmw wrote:Do you have a list of your attacks, especially the hue changes?
My attacks are on that web page, at http://im.snibgo.com/phashtest.htm#tweakImages.bat

The script is Windows BAT, but fairly understandable. For example, the code:

Code: Select all

  for %%V in (-20 -15 -10 -5 5 10 15 20) do %IM%convert ^
    %%F ^
    +depth ^
    ^( +clone ^
       -brightness-contrast %%V,0 ^
       +write %OUTDIR%\!BASE!_bri_%%V.%EXT% ^
       +delete ^
    ^) ^
    -brightness-contrast 0,%%V ^
    %OUTDIR%\!BASE!_con_%%V.%EXT%
This takes one of the standard images (%%F) and loops through %%V. For each %%V (which is one of -20, -15 etc) it creates two variations, using %%V as the first or second parameter for -brightness-contrast.

The rotates and crops work in similar ways.

Re: Get perceptual hash value for image using command line

Posted: 2016-08-23T09:53:59-07:00
by fmw42
Thanks. I see it now at

Code: Select all

for %%V in (90 95 97.5 102.5 105 110) do %IM%convert ^
    %%F ^
    -modulate 100,100,%%V ^
    %OUTDIR%\!BASE!_hue_%%V.%EXT%

Re: Get perceptual hash value for image using command line

Posted: 2016-08-23T10:34:14-07:00
by fmw42
HCLp is more like PS HCL than plain HCL. I will see if I can find that information.

Looking at gems.c lines 841 (HCL) and 904 (HCLp). They look the same. But HCLp should be different according to

viewtopic.php?f=2&t=21663#p88910
viewtopic.php?f=2&t=21646
viewtopic.php?f=1&t=23803&hilit=HCLp#p101115
viewtopic.php?f=2&t=21646#p112265