Smart crop function in IM?

[fmw42]

@Fred: Good stuff. Your page http://www.fmwconcepts.com/imagemagick/ ... /index.php has "Animation" in places I think you intended "Arguments:".

Thanks. Cut and paste errors. I have fixed it.

[snibgo]

Snibgo, what is a slope magnitude image?

It's a form of edge-detection.

At each pixel, the magnitude of a slope is the square root of the squared slope in the x-direction plus the squared slope in the y-direction. (Pythagorus: h^2 = x^2+y^2.) For each individual slope, I use a Sobel edge detector, which for the x-direction is the weighted average of 3 pixels to the right minus the weighted average of 3 pixels to the left. See http://www.imagemagick.org/Usage/convolve/#sobel

If you just pick an arbitrary N and M, do you always get good results on these images?

No. This isn't surprising. For example, if the image has detail in the horizontal direction (such as the insect), but I specify a tall narrow rectangle, it is bound to include loads of background data which can overpower the sharp detail, so the chosen offset misses the most valuable detail.

That's the point of my previous post: it doesn't pick arbitrary N and M. The command finds N and M (and offsets) of the smallest rectangle that contain all the pixels that when averaged (blurred) have a slope magnitude above a certain threshold. So the commands find the the best rectangle, and there is no need to sub-image search for a white rectangle of dimensions supplied by the user.

However, under this scheme, the user does have to supply a threshold, which is far less intuitive than the dimension NxM. The command shown calculates for five arbitrary thresholds, appending them for viewing convenience.

Another possibility: the user supplies one dimension, either width or height, and a script finds the optimal other dimension (and offsets). I can't immediately see how to do this.

[fmw42]

Your magnitude slope gradient is just the original basic sobel gradient magnitude as per https://en.wikipedia.org/wiki/Sobel_operator. Anthony has an example of that also. So I understand that. I chose to use the largest from the 8-directional sobel. See anthony's convolve where he has

Code: Select all

convert face.png   -define convolve:scale='!' \
           -define morphology:compose=Lighten \
           -morphology Convolve  'Sobel:>'   face_sobel_maximum_2.png

So we are basically doing the same thing.

Your blur, equalize and threshold, then, is as I suspected a type of fuzzy trim. That seems to be a better concept that picking (optimal) dimensions.

Why did you need to equalize before thresholding? Did you simply find that it worked better than not doing it? Or did you have some specific motivation for adding that in? I tried doing a simple -fuzz -trim, but it did not work well. Perhaps adding in your blur, equalize and threshold will work better in my metrics as well.

[fmw42]

Another possibility: the user supplies one dimension, either width or height, and a script finds the optimal other dimension (and offsets). I can't immediately see how to do this.

I experimented with finding the best W width slice of the image, then from that result finding the best H height slice. That is a separable process. But the best vertical section does not always contain the best WxH region. This would seem to me to be a flaw in concepts that I have read where the trim one column from the left or right, whichever, is lower in detail, until they get the best W. Then doing the same by rows from the extracted group of columns. So I am not sure any separable process would work well.

[snibgo]

Your magnitude slope gradient is just the original basic sobel magnitude as per https://en.wikipedia.org/wiki/Sobel_operator.

Yes. I got it from one of Anthony's pages.

Why did you need to equalize before thresholding?

Because I want to turn a certain percentage of pixels black, and the rest white. Without equalizing, I can't control how many pixels become black.

Yes, my "-equalize, -threshold, -format %@" is a fuzzy trim. The blur is important, so the trim ignores small areas of high detail. I don't know the optimum value of blur sigma, and there may be a better way of doing this, such as threshold with no blur, but then use "-connected-components" to ignore small areas.

[fmw42]

snibgo:

I tried to emulate your results with 3 different sobel magnitude techniques and none of them match your results. For your smallest size, you get a more square result. Was your command line for computing s.png in your slope-magnitude method different from what is on Anthony's page and presumably my method 1?

Were you using HDRI? I am using Im 6.9.8.10 Q16 Mac OSX non-hdri

Method 1:
infile="car_lady_dog_small.jpg"
inname=`convert "$infile" -format "%t" info:`
list="99 97 95 90 75"
for thresh in $list; do
cropvals=`convert "$infile" \
-define convolve:scale=50%! -bias 50% \
\( -clone 0 -morphology Convolve Sobel:0 \) \
\( -clone 0 -morphology Convolve Sobel:90 \) \
-delete 0 -solarize 50% -level 50,0% \
-compose Lighten -composite +write tmp_v1.png \
-blur 0x20 -equalize -threshold $thresh% \
-format "%@" info:`
echo "cropvals=$cropvals;"
convert "$infile" \
-crop $cropvals +repage \
${inname}_${thresh}_v1.jpg
done
convert *_v1.jpg -reverse -bordercolor gray -border 5 +append results_v1.jpg

Sobel Magnitude:


Result:



Method 2:
infile="car_lady_dog_small.jpg"
inname=`convert "$infile" -format "%t" info:`
list="99 97 95 90 75"
for thresh in $list; do
cropvals=`convert "$infile" \
-define convolve:scale=100%! \
\( -clone 0 -morphology Convolve Sobel:0 -evaluate pow 2 \) \
\( -clone 0 -morphology Convolve Sobel:90 -evaluate pow 2 \) \
-delete 0 -compose plus -composite -evaluate pow 0.5 +write tmp_v2.png \
-blur 0x20 -equalize -threshold $thresh% \
-format "%@" info:`
echo "cropvals=$cropvals;"
convert "$infile" \
-crop $cropvals +repage \
${inname}_${thresh}_v2.jpg
done
convert *_v2.jpg -reverse -bordercolor gray -border 5 +append results_v2.jpg

Sobel Magnitude:


Result:



Method 3:
infile="car_lady_dog_small.jpg"
inname=`convert "$infile" -format "%t" info:`
list="99 97 95 90 75"
for thresh in $list; do
cropvals=`convert "$infile" \
-define convolve:scale=! -define morphology:compose=Lighten \
-morphology Convolve Sobel:\> +write tmp_v3.png \
-blur 0x20 -equalize -threshold $thresh% \
-format "%@" info:`
echo "cropvals=$cropvals;"
convert "$infile" \
-crop $cropvals +repage \
${inname}_${thresh}_v3.jpg
done
convert *_v3.jpg -reverse -bordercolor gray -border 5 +append results_v3.jpg

Sobel Magnitude:


Result:



EDIT:

I noticed that a larger blur makes the results more square and get smaller faster as one approaches 100% threshold.

Method 6: same as method 1 but using -blur 0x50 rather than -blur 0x20

infile="car_lady_dog_small.jpg"
inname=`convert "$infile" -format "%t" info:`
list="99 97 95 90 75"
for thresh in $list; do
cropvals=`convert "$infile" \
-define convolve:scale=50%! -bias 50% \
\( -clone 0 -morphology Convolve Sobel:0 \) \
\( -clone 0 -morphology Convolve Sobel:90 \) \
-delete 0 -solarize 50% -level 50,0% \
-compose Lighten -composite +write tmp_v1.png \
-blur 0x50 -equalize -threshold $thresh% \
-format "%@" info:`
echo "cropvals=$cropvals;"
convert "$infile" \
-crop $cropvals +repage \
${inname}_${thresh}_v6.jpg
done
convert *_v6.jpg -reverse -bordercolor gray -border 5 +append results_v6.jpg

Result:

[fmw42]

See my edits above regarding the blur amount

[snibgo]

-compose Lighten -composite

That would give the maximum of the slopes in the X and y directions. I calculate the magnitude, which is the sqrt of the average of the squared slopes (average rather than sum, so output is in range 0 to 100%).

Code: Select all

%IM32f%convert ^
  %INFILE% ^
  -alpha off ^
  -define convolve:scale="50%%^!" -bias 50%% ^
  ( -clone 0 -morphology Convolve Sobel:0 ) ^
  ( -clone 0 -morphology Convolve Sobel:90 ) ^
  -delete 0 -solarize 50%% -level 50,0%% ^
  +level 0,70.71067811865475%% ^
  -evaluate Pow 2 ^
  -compose plus -composite  ^
  -evaluate Pow 0.5 ^
  -separate -evaluate-sequence Max ^
  -auto-level ^
  +depth ^
  -define quantum:format=integer ^
  %OUTFILE%

Yes, I use HDRI because slopes are small, and I don't want to lose precision taking the square root etc.

I then "-separate -evaluate-sequence Max" to get the greatest slope from the three channels.

I had a bug in my posts above: after "-auto-level" I had "-auto-gamma". This changes the results slightly.

The slope magnitude is now:

[snibgo]

list="99 97 95 90 75"

I used 85, not 75.

[fmw42]

You added +level 0,70.71067811865475%% to your sobel -- why? Also you added -separate -evaluate-sequence Max to get the brightest values from the channels. So those seems to be the main differences as you point out. I also tried in my tests to add -auto-level, but I did not see any visual differences in the limited test that I did.

Also, I do not think you need to solarize if you are doing square and sort. But I could be wrong if using HDRI. I will test your code later today or tomorrow on non-hdri and HDRI.

Sorry about the typo of 75 vs 85. I copied it wrong when I wrote down your list of thresholds.

Thanks

[snibgo]

You added +level 0,70.71067811865475%% to your sobel -- why?

That divides by sqrt(2). Without that, the magnitude would have a range 0 to 141%.

Also, I do not think you need to solarize ...

But there is "-bias 50%", so zero slope is IM value 50%. "-solarize 50%% -level 50,0%%" makes zero slope be represented by IM value 0%.

I copied that code years ago from Anthony, who wrote it even more years ago for integer IM. I need precision, and I should re-write it properly for HDRI.

[fmw42]

snibgo:

I tried your command in IM 6 Q16 non-hdri as follows and it works fine, but does not match your results.

Code: Select all

infile="car_lady_dog_small.jpg"
inname=`convert "$infile" -format "%t" info:`
list="99 97 95 90 75"
for thresh in $list; do
cropvals=`convert "$infile" \
-define convolve:scale=50%! -bias 50% \
\( -clone 0 -morphology Convolve Sobel:0 \) \
\( -clone 0 -morphology Convolve Sobel:90 \) \
-delete 0 -solarize 50% -level 50,0% \
+level 0,70.71067811865475% \
-evaluate Pow 2 \
-compose plus -composite \
-evaluate Pow 0.5 \
-separate -evaluate-sequence Max \
-auto-level \
+depth \
-define quantum:format=integer \
+write tmp_v8.png \
-blur 0x20 -equalize -threshold $thresh% \
-format "%@" info:`
echo "cropvals=$cropvals;"
convert "$infile" \
-crop $cropvals +repage \
${inname}_${thresh}_v8.jpg
done
convert *_v8.jpg -reverse -bordercolor gray -border 5 +append results_v8.jpg

But when I run it in IM 7 Q16 HDRI, I get 0 for the width and height and an error "magick: geometry does not contain image `' @ warning/attribute.c/GetImageBoundingBox/240."

Code: Select all

infile="car_lady_dog_small.jpg"
inname=`magick "$infile" -format "%t" info:`
list="99 97 95 90 75"
for thresh in $list; do
cropvals=`magick "$infile" \
-define convolve:scale=50%! -bias 50% \
\( -clone 0 -morphology Convolve Sobel:0 \) \
\( -clone 0 -morphology Convolve Sobel:90 \) \
-delete 0 -solarize 50% -level 50,0% \
+level 0,70.71067811865475% \
-evaluate Pow 2 \
-compose plus -composite \
-evaluate Pow 0.5 \
-separate -evaluate-sequence Max \
-auto-level \
+depth \
-define quantum:format=integer \
+write tmp_v8.png \
-blur 0x20 -equalize -threshold $thresh% \
-format "%@" info:`
echo "cropvals=$cropvals;"
magick "$infile" \
-crop $cropvals +repage \
${inname}_${thresh}_v8.jpg
done
magick *_v8.jpg -reverse -bordercolor gray -border 5 +append results_v8.jpg

I even tried adding -define compose:clamp=on before the -compose plus -composite.

Have I missed something? Or did you do something else that you might have forgot about in your post?

P.S. I kept the 75 for comparison to my previous attempts.

EDIT: It seems all my different sobel methods fail in IM 7 HDRI. So I am missing something or some clamping is needed somewhere?

[fmw42]

You added +level 0,70.71067811865475%% to your sobel -- why?

That divides by sqrt(2). Without that, the magnitude would have a range 0 to 141%.

But you then also do -pow 0.5 to take the sqrt. And do a -auto-level. Without testing, I would have thought that the +level would not matter in HDRI.

Also, I do not think you need to solarize ...

But there is "-bias 50%", so zero slope is IM value 50%. "-solarize 50%% -level 50,0%%" makes zero slope be represented by IM value 0%.[/quote]

Yes, I overlooked the -bias. I was thinking you had zero bias. Then in HDRI you get both positive and negative edges and so the square and sort take care of that.

[snibgo]

I haven't tried these commands with v7.


I've uploaded a page Crop to detail with a Windows BAT script, cropToDetail.bat. The script takes width and height as parameters. The user can supply both, or just one, or no dimensions at all.

I've added a refinement to the basic processing: after thresholding, use just the largest two connected components (which should be the black background and the largest white component).

When only one dimension is given, the script iteratively finds the threshold that gives that dimension, and hence gets the other dimension.

When no dimensions are given, the script removes the most uninteresting 90% (approx) pixels.

[fmw42]

I put in +write statements after each step in my script. In IM 7 HDRI, the image becomes totally white using -blur 0x20, which causes the trim to get 0 dimensions. I will test later under IM 6 HDRI