Smart crop function in IM?

[snibgo]

In IM 7 HDRI, the image becomes totally white using -blur 0x20, ...

That shouldn't happen, of course. v7.0.1-0 behaves as expected.


On v7 and Sobel:

Comparing the output of slopeMag.bat on toes.png, v6.9.0-0 and v7.0.1-0 (Cygwin compilation), both Q32 HDRI, the RMSE difference is:

Code: Select all

0.0755406 (1.15268e-006)

They are virtually identical. (But this is an old v7. It may have broken since then.))

Comparing v6.9.0-0 Q32 HDRI and v6.9.3-7 Q16 integer, the difference is:

Code: Select all

277.132 (0.00422876)

This is reasonable, the expected difference.

Here is a simplified version, for HDRI. It gives virtually the same result, in v6 or v7.

Code: Select all

%IM7DEV%magick ^
  %INFILE% ^
  -alpha off ^
  -define convolve:scale="^!" ^
  ( -clone 0 -morphology Convolve Sobel:0 ) ^
  ( -clone 0 -morphology Convolve Sobel:90 ) ^
  -delete 0 ^
  -evaluate Pow 2 ^
  -evaluate-sequence Mean ^
  -evaluate Pow 0.5 ^
  -separate -evaluate-sequence Max ^
  %AUTO% ^
  +depth ^
  -define quantum:format=integer ^
  %OUTFILE%

For Windows, I need to escape the "!", hence "^!"

All my tests were repeated with %AUTO% either blank or "-auto-level".

[fmw42]

Yes that was the simpler sobel that I was suggesting would not need the solarize, +level, but have no bias. But it only works for HDRI due to it allowing the negative slope values. For both HDRI and non-hdri, one should use -bias and solarize.

Did you find any significant difference with and without -auto-level?

Did you compare Q16 HDRI in both IM 6 and IM 7? I am about the compile IM 6 HDRI and will report back.

[fmw42]

I ran a version of the sobel operator with the blur in IM 6.9.8.10 Q16 HDRI and it ran fine. Doing the same in IM 7.0.6.1 Q16 HDRI show a different result after the blur. See my bug report at viewtopic.php?f=3&t=32268

[fmw42]

snibgo:

I tried running the following images with your simpler sobel method in IM 6.9.8.10 Q16 HDRI. Do you get similar results (especially for the white cat image)? I did not expect much for it due to the flowers and grass texture detail. Note my different list of thresholds.

Code: Select all

infile="car_lady_dog_small.jpg"
infile="white_cat_small.jpg"
infile="sofa_cat_small.jpg"
infile="dragonfly_small.jpg"

Code: Select all

inname=`im6hdri convert "$infile" -format "%t" info:`
list="99 97 95 90 85"
for thresh in $list; do
cropvals=`im6hdri convert $infile \
-alpha off \
-define convolve:scale="^!" \
\( -clone 0 -morphology Convolve Sobel:0 \) \
\( -clone 0 -morphology Convolve Sobel:90 \) \
-delete 0 \
-evaluate Pow 2 \
-evaluate-sequence Mean \
-evaluate Pow 0.5 \
-separate -evaluate-sequence Max \
-auto-level \
+depth \
-define quantum:format=integer \
-blur 0x20 -equalize -threshold $thresh% \
-format "%@" info:`
echo "cropvals=$cropvals;"
im6hdri convert  "$infile" \
-crop $cropvals +repage \
${inname}_${thresh}_v9.jpg
done
im6hdri convert  *_v9.jpg -reverse -bordercolor gray -border 5 +append ${inname}_results.jpg
rm -f *_v9.jpg

I do not seem to get much different with or without HDRI at least for the first image.

[snibgo]

My script now has an option "ctdUSE_CONN_COMP". If this is zero, I get the same result as you for the white cat.

When that option is one, after thresholding it eliminates all white connected components except the largest. This gives a smaller result, so lowering the threshold compensates.

So far, it seems that 90 is a good threshold when ctdUSE_CONN_COMP=0; 85 is a good threshold when ctdUSE_CONN_COMP=1.

Below, I show six thresholds.

Code: Select all

set SRC=white_cat1.jpg

set ctdUSE_CONN_COMP=0

call %PICTBAT%cropToDetail %SRC% x_99.png . . 99
call %PICTBAT%cropToDetail %SRC% x_97.png . . 97
call %PICTBAT%cropToDetail %SRC% x_95.png . . 95
call %PICTBAT%cropToDetail %SRC% x_90.png . . 90
call %PICTBAT%cropToDetail %SRC% x_85.png . . 85
call %PICTBAT%cropToDetail %SRC% x_80.png . . 80

%IM%convert ^
  x_99.png x_97.png x_95.png x_90.png x_85.png x_80.png ^
  -bordercolor gray -border 5 +append ^
  -quality 40 vt_cat0.jpg

set ctdUSE_CONN_COMP=1

call %PICTBAT%cropToDetail %SRC% x_99.png . . 99
call %PICTBAT%cropToDetail %SRC% x_97.png . . 97
call %PICTBAT%cropToDetail %SRC% x_95.png . . 95
call %PICTBAT%cropToDetail %SRC% x_90.png . . 90
call %PICTBAT%cropToDetail %SRC% x_85.png . . 85
call %PICTBAT%cropToDetail %SRC% x_80.png . . 80

%IM%convert ^
  x_99.png x_97.png x_95.png x_90.png x_85.png x_80.png ^
  -bordercolor gray -border 5 +append ^
  -quality 40 vt_cat1.jpg

[fmw42]

You have a good point. In my implementation there could be multiple areas in the thresholded result that are grouped by the trim %@. So I will have to do similarly and add an option for CCL. Thanks. Just wanted to be sure we got similar results, especially for this image.

I also found that using -statistic standard_deviation 3x3 in place of the sobel gives good, but slightly different results. In many cases it looks better. You might want to try that also and add an option.

Here is a comparison for blur=20 and thresholds=99 97 95 90 85

Sobel:


Std:



Sobel:


Std:



Sobel:


Std:



Sobel:


Std:


I suspect that slight changes of thresholds will make the two sets closer. Or perhaps, as you say, it could be different numbers of threshold regions that may also be grouping differently.

[snibgo]

Thanks, yes, I'll explore other methods of slope magnitude.

One I've recently devised uses comet blur, see http://www.imagemagick.org/Usage/convolve/#comet

On white_cat1.jpg, comet sigma 10, this gives a slope magnitude:



And this blurs and equalises better, creating less emphasis on the top-left corner, concentrating on the face:



For a range of thresholds (99,97,95,90,85,80), with all components:



For the same range of thresholds, with just the largest component:



Purrfect.

[fmw42]

Just to be sure I understand, are you blurring before getting the sobel (slope/magnitude) or as before doing the blur afterwards. Those results look very good.

[snibgo]

The processing is as before: get the magnitude, blur, equalize and threshold.

The "comet" magnitude is also technically a blur. We can "comet blur" in four directions: right, down, left, and up. The right blur minus the left blur gives me the slope in the x-direction. Down blur minus up blur gives the y slope.

Then, from the two slopes, magnitude = sqrt ((dx^2 + dy^2)/2) as before.

[fmw42]

Thanks. I understand now what you are doing. Clever idea.

[snibgo]

The same method on the other cat gives:



However, this is with sigma=1 for the comet. I haven't yet found one method and set of parameters that gives good results for all these images at all thresholds.

[Bonzo]

I haven't yet found one method and set of parameters that gives good results for all these images at all thresholds.

I think that is the main problem with smart cropping. The site the OP linked to gave you the option to select different versions of the crop.

Perhaps it would be an idea to average a couple of different methods or settings for the crop area?

[snibgo]

I think cropping is always best done by human intelligence. At least, until computers understand human faces and expressions and what they are looking at and how picture elements interact and so on.

But some users don't have that luxury. Perhaps they process thousands of images per day and need a fully automated, hands-off system.

Some of these images are easy. Almost any method crops the blurred background around the insect. But the cats are awkward. (Aren't they always?)

[fmw42]

snibgo:

I am not able to reproduce your results. Do I have the same comet edge method that you used? I am using 6.9.8.10 Q16 HDRI (not Q32)

Code: Select all

blur=20
comet=1
#comet=10
infile="sofa_cat_small.jpg"
inname=`convert $infile -format "%t" info:`
list="99 98 95 90 85"
for thresh in $list; do
cropvals=`im6hdri convert $infile \
\( -clone 0 -morphology Convolve Comet:0x${comet}+0 \) \
\( -clone 0 -morphology Convolve Comet:0x${comet}+90 \) \
\( -clone 0 -morphology Convolve Comet:0x${comet}+180 \) \
\( -clone 0 -morphology Convolve Comet:0x${comet}+270 \) \
-delete 0 \
\( -clone 0,2 -compose minus -composite \) \
\( -clone 1,3 -compose minus -composite \) \
-delete 0-3 \
-evaluate Pow 2 \
-compose plus -composite \
-evaluate Pow 0.5 \
-separate -evaluate-sequence Max \
-auto-level \
-blur 0x20 -equalize -threshold $thresh% \
-format "%@" info:`
im6hdri convert $infile -crop $cropvals +repage ${inname}_smarttrim_comet_c${comet}_$thresh.jpg
done
im6hdri convert ${inname}_smarttrim_comet_c${comet}*.jpg -reverse -bordercolor gray -border 5 +append ${inname}_smarttrim_comet1_c${comet}_b${blur}_f1-2-5-10-15hdri.jpg

Comet blur 10 and blur=20:


Comet blur 1 and blur=20:

[fmw42]

Here is another idea -- using a corner detector. I have not tested all my corner detectors (harris, shi-tomasi, morphologic) from my script corners. But here I just use the easiest, which is morphologic (http://www.site.uottawa.ca/~laganier/pu ... s/coin.pdf). It needs some improvement. But you will get the idea. Typical mthresh values are 40 for all but the sofa cat (50) and white cat (35). So there is some adjustment needed per image.

Code: Select all

blur=20
region=1
mthresh=40
sharp=1
infile="car_lady_dog_small.jpg"
infile="white_cat_small.jpg"
infile="sofa_cat_small.jpg"
infile="dragonfly_small.jpg"
inname=`convert $infile -format "%t" info:`
list="95 90 85 80 75 70"
for thresh in $list; do
cropvals=`convert $infile -sharpen 0x1 \
\( -clone 0 -morphology dilate plus:$region -morphology erode diamond:$region \) \
\( -clone 0 -morphology dilate cross:$region -morphology erode square:$region \) \
-compose difference -composite -auto-level -threshold $mthresh% \
-separate -evaluate-sequence Max \
-auto-level +write ${inname}_m$mthresh.png \
-blur 0x20 -equalize -threshold $thresh% \
-format "%@" info:`
convert $infile -crop $cropvals +repage ${inname}_smarttrim_corners_s1__m${mthresh}_$thresh.jpg
done
convert ${inname}_smarttrim_corners_s1__m${mthresh}*.jpg -reverse -bordercolor gray -border 5 +append ${inname}_smarttrim_corners_m${mthresh}_s1_b${blur}_f5-10-15-20-25-30.jpg
rm -f ${inname}_smarttrim_corners_s1__m${mthresh}*.jpg

Below I show the corner image for the mthresh value used followed by the trim on the image for the variations in thresholds of 95 90 85 80 75 70. I have not tried blurs other than 20, yet.