Smart crop function in IM?

[maqe]

I use IM to generate a bunch of thumbnails of different sizes and formats. Does IM have a function for smart crop like https://imagga.com/solutions/cropping-api.html ? If not, have any of you used any third party library for this?

[fmw42]

There is no "smart" crop.

But ImageMagick has all kinds of crop options. You can crop to the largest size and pad. You can crop to the smallest size and trim off the extra. You can crop so that it does not respect aspect and thus get exactly the coordinates you want. see all the sections of
http://www.imagemagick.org/Usage/crop/#crop
and all the sections of
http://www.imagemagick.org/Usage/thumbnails/#creation

Also there is liquid rescale that has smart trimming of parts of the image. See http://www.imagemagick.org/script/comma ... id-rescale and http://www.fmwconcepts.com/misc_tests/l ... index.html

[maqe]

I see. So I guess I need to look for a third party service do analyze the image so I know the coordinates for the best crop result.

[snibgo]

IM contains low-level objective operations for processing raster images. It doesn't contain higher-level subjective operations such as "crop to the interesting area".

IM tools can be used to build those high-level operations, once someone has defined "interesting".

[snibgo]

For example, suppose we want to crop an image to 200 pixels wide. We want to find the most "interesting" sub-image that is 200 columns wide.



We might say that a column of pixels that is different to its neighbour column is "interesting". When they are the same, it is not interesting. Hence, the "-compose Difference" reduced to a single channel and scaled to a single pixel high is a measure of interestingness.

Then, the mean of a rolling window 200 pixels wide will have one or more peaks that show the central position of the most interesting windows.

Windows BAT script:

Code: Select all

set SRC=barbara_test.jpg

%IM%convert ^
  %SRC% ^
  ( +clone ) ^
  -geometry +1+0 -compose Difference -composite ^
  -grayscale RMS -scale "x1^!" ^
  -statistic mean 200x1 -auto-level ^
  -fill Black +opaque White ^
  -format %%@ ^
  info:

The result is "2x1+558+0". So we want a crop 200 pixels wide, starting at x=559-100=458. [EDIT: I meant x, not y.]

Code: Select all

%IM%convert ^
  %SRC% ^
  -crop 200x+458+0 +repage ^
  -quality 40 barbara200.jpg

This is fast and quite effective. Clearly, the definition of "interesting" can be more complex. It might include face-detection, which is very complex.

[fmw42]

Another way to do smart cropping is to find the subsection with the most edge content using canny edge detector. In unix syntax:

Code: Select all

infile="white_cat1.jpg"
ww=250
hh=250
inname=`convert "$infile" -format "%t" info:`
convert "$infile" -canny 0x1+10%+30% tmp1.gif
coords=`compare -metric rmse -subimage-search -dissimilarity-threshold 1 tmp1.gif \( -size ${ww}x${hh} xc:white \) null: 2>&1 | cut -d\  -f4`
xoff=`echo "$coords" | cut -d, -f1`
yoff=`echo "$coords" | cut -d, -f2`
convert "$infile" -crop ${ww}x${hh}+${xoff}+${yoff} +repage ${inname}_crop.jpg

Code: Select all

infile="car_lady_dog_1.jpg"
ww=250
hh=300
inname=`convert "$infile" -format "%t" info:`
convert "$infile" -canny 0x1+10%+30% tmp1.gif
coords=`compare -metric rmse -subimage-search -dissimilarity-threshold 1 tmp1.gif \( -size ${ww}x${hh} xc:white \) null: 2>&1 | cut -d\  -f4`
xoff=`echo "$coords" | cut -d, -f1`
yoff=`echo "$coords" | cut -d, -f2`
convert "$infile" -crop ${ww}x${hh}+${xoff}+${yoff} +repage ${inname}_crop.jpg

Speed-up can be had by doing the compare on a pyramid of resolutions -- two often is enough. Or by using my rmsecorr or normcrosscorr to do the subimage search.

[Bonzo]

Out of interest I gave your code a try snibgo but get an error on the first piece of code:

convert: geometry does not contain image `' @ warning/attribute.c/GetImageBoundingBox/240.

Version 7.0.5-Q16 on a Windows 10 PC ( same error with magick and convert ).

I have a feeling I have seen a similar error before using magick if the operators are in the wrong order; but it should have worked with the convert as it is still using V6?

[fmw42]

You get that error when the crop has zero size for one or both of Width and Height

[Bonzo]

Thank you fmw42 as my crop dimensions from the command ( found using a php version of the code ) are 0x0+1+0

I get the same error using snibgo's image as well.

I also had a go at your codes but I am not sure how to implement the cut in Windows or php

[snibgo]

My command isn't robust, because "-trim" or "-format %@" can result in a 0x0 image, which IM then objects to.

A cure is to insert "-compose Over -bordercolor Black -border 1" before the "-format". Then parse out the x-offset, and subtract 1 for the crop. (This doesn't totally solve the problem: a totally black input image is totally uninteresting, and trims down to 0x0.)

Here is a more robust version (Windows BAT syntax):

Code: Select all

set SRC=\prose\pictures\barbara_test.png

rem REQ_W is required width of "interesting" region.
rem REQ_W should be less than actual width of image.

set REQ_W=200

for /F "usebackq tokens=3 delims=x+" %%X in (`%IM%convert ^
  %SRC% ^
  ^( +clone ^) ^
  -geometry +1+0 -compose Difference -composite ^
  -grayscale RMS -scale "x1^!" ^
  -statistic mean %REQ_W%x1 -auto-level ^
  -fill Black +opaque White ^
  -compose Over ^
  -bordercolor Black -border 1 ^
  -format %%@ ^
  info:`) do set /A CX=%%X-1-(%REQ_W%/2)

%IM%convert ^
  %SRC% ^
  -crop %REQ_W%x+%CX%+0 +repage ^
  -quality 40 out.jpg

I expect IM v7 can do this in a single command.

[Bonzo]

Thank you for updating the code snibgo; it works well on your example but for some reason fails on my test image. I get a crop of -crop 200x+-100+0 in my command display. I understand it will not work on every image but I was just interested in seeing how well it worked.

[fmw42]

I have written a bash unix shell script to do a minimally intelligent (or semi-smart) version of a smart crop. See http://www.fmwconcepts.com/imagemagick/ ... /index.php for examples.

[snibgo]

@Bonzo: The "-geometry +1+0 -compose Difference -composite" gives a spurious result at the left edge, so the cure is to then remove the left edge: "-crop x+1+0 +repage".

@Fred: Good stuff. Your page http://www.fmwconcepts.com/imagemagick/ ... /index.php has "Animation" in places I think you intended "Arguments:".


A method to find a 300x200 area that is "interesting" (ie contains the most detail), with my published Windows BAT scripts:

Code: Select all

set SRC=_MG_5357S.jpg

%IM%convert -size 300x200 xc:white w.png

call %PICTBAT%slopemag %SRC% s.png

call %PICTBAT%srchImg s.png w.png

%IM%convert %SRC% -crop %siCOMP_CROP% +repage -quality 40 insect300x200.jpg

Similarly:

[snibgo]

Command and scripts above, from Fred and me, answer the question: "Where in this image is the NxM rectangle with the most detail?"

But how do we choose N and M?

My commands create a slope magnitude image, s.png, which is bright where this is detail, and dark where there is no detail. So each pixel has a lightness, which is a score. We can process this (or any similar image) to find the smallest rectangle that contains the T% top-scoring pixels.

s.png is noisy, so we blur it first.

Code: Select all

for /F "usebackq" %%L in (`%IM%convert ^
  s.png ^
  -blur 0x20 ^
  -equalize ^
  ^( +clone -threshold 99%% -format "t99=%%@\n" +write info: +delete ^) ^
  ^( +clone -threshold 97%% -format "t97=%%@\n" +write info: +delete ^) ^
  ^( +clone -threshold 95%% -format "t95=%%@\n" +write info: +delete ^) ^
  ^( +clone -threshold 90%% -format "t90=%%@\n" +write info: +delete ^) ^
  ^( +clone -threshold 85%% -format "t85=%%@\n" +write info: +delete ^) ^
  b.png`) do set %%L

%IM%convert ^
  %SRC% ^
  ( +clone -crop %t99% +write c99.png +delete ) ^
  ( +clone -crop %t97% +write c97.png +delete ) ^
  ( +clone -crop %t95% +write c95.png +delete ) ^
  ( +clone -crop %t90% +write c90.png +delete ) ^
  ( +clone -crop %t85% +write c85.png +delete ) ^
  NULL:

%IM%convert ^
  c99.png c97.png c95.png c90.png c85.png ^
  -bordercolor gray(50%%) -border 3 ^
  +append ^
  a.png

[fmw42]

But how do we choose N and M?

My commands create a slope magnitude image

In my script, I must admit, that I do not get good results for all sizes of N and M. So that is a great question.

Snibgo, what is a slope magnitude image? If you just pick an arbitrary N and M, do you always get good results on these images?