How would you go about separating elements from their background?

[coloring]

I would like to separate "islands" (icons) from a "sea" (background). The difficulty is that the background can have more than one color depending on the area.

In other words, I need to make "islands of content" stand out from the "background sea".

I suppose the technique would be somewhat similar to how scanned documents are post-processed to whiten the page and darken the fonts.

The idea is to later quickly determine the coordinates of those "islands" by scanning for a single color.

I cannot upload any images right now, but I may do so later on when I get better internet.

[snibgo]

I suppose the technique would be somewhat similar to how scanned documents are post-processed to whiten the page and darken the fonts.

Yeah, maybe.

The trick is to define what visually distinguishes foreground and background.

For more specific advice, you know what to do.

[fmw42]

Background foreground separation can be very hard unless the background is a relatively constant color.

[anthony]

Also how 'fuzzy' and 'shaped' the border is between background and foreground. For example rotated square images? Or other odd shapes.
What other 'junk' is on the page... Text?

It is all a matter of defining what is foreground and background in a way the computer can understand.

The trouble with computers, of course,
is that they're very sophisticated idiots. -- Doctor Who, "Robot"

[coloring]

In this case, the borders are not fuzzy at all. I am trying to isolate widgets such as fields and buttons from a screenshot.

[fmw42]

Post an example image and also identify your IM version, date, and platform.

[coloring]

Code: Select all

C:> magick.exe --version
Version: ImageMagick 7.0.8-16 Q16 x64 2018-12-10 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2018 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Visual C++: 180040629
Features: Cipher DPC HDRI
Delegates (built-in): bzlib cairo flif freetype gslib heic jng jp2 jpeg lcms lqr
 lzma openexr pangocairo png ps raw rsvg tiff webp xml zlib

I'd like to either overlay a yellow border around those "islands" like in the image, or crop out the widgets inside the yellow border and save them as separate images.

[fmw42]

Get rid of the background image. Then threshold or use -fill -opaque on the image to make all the text white and the gray into black. Blur the image a little so the text is all connected and re-threhsold. Then use connected components to extract the bounding boxes around those regions. Then use the bounding boxes to crop your original image or draw yellow boxes given he bounding box coordinates. Sorry I do not code in Windows scripting.

See https://imagemagick.org/script/connected-components.php

[coloring]

Thanks! Non-GNU Linux or macOS are fine too. From there I can figure out how to translate it for Windows.

[fmw42]

Here is some code that seems to work for me on my Mac. I have posted the tmp images to show intermediate steps. The -define connected-components:area-threshold=20 is there to remove some small black spots interior to the white, but may not be necessary.

The main idea is to

1) replace white with red
2) replace the darker gray with red
3) extract the Chroma channel from HCL
4) filltoborder with black to remove the image texture outside your gray area
5) threshold at 0 to binarize
6) use connected components to filter out some small areas
7) horizontal only blur 0x1.5 via morphology and threshold 0 to binarize again
8 ) do a morphology open to remove some long horizontal lines from the original white lines
9) do connected components again to get only the gray(255) areas and save only the bounding boxes for them
10) loop over all the white region bounding boxes and convert the boxes from WxH+X+Y to X1,Y1 X2,Y2 for drawing and save as a string
11) draw all the bounding boxes with one draw command

Code: Select all

bboxArr=(`convert image_1.gif \
-fill red -opaque white \
-fill red -opaque "gray(216)" \
-colorspace HCL -channel g -separate +channel +write tmp0.png \
-fill black -bordercolor black -draw "color 20,10 filltoborder" -alpha off \
-threshold 0 \
-define connected-components:area-threshold=6 \
-define connected-components:mean-color=true \
-connected-components 4 \
-morphology convolve blur:0x1.5 -threshold 0 \
-morphology open octagon:1 -type bilevel +write tmp1.png \
-define connected-components:verbose=true \
-define connected-components:area-threshold=20 \
-define connected-components:mean-color=true \
-connected-components 4 tmp2.png | grep "gray(255)" | awk '{print $2}'| tr "x" "+"`)
num="${#bboxArr[*]}"
echo "$num"
draw_args=""
for ((i=0; i<num; i++)); do
bbox="${bboxArr[$i]}"
ww=`echo "$bbox" | cut -d+ -f1`
hh=`echo "$bbox" | cut -d+ -f2`
x1=`echo "$bbox" | cut -d+ -f3`
y1=`echo "$bbox" | cut -d+ -f4`
y1=$((y1-4))
x2=$((ww+x1))
y2=$((hh+y1+6))
draw_args="$draw_args rectangle $x1,$y1 $x2,$y2"
done
convert image_1.gif -fill none -stroke red -strokewidth 2 -draw "$draw_args" -alpha off image_1_proc.gif

tmp00.png


tmp0.png


tmp1.png


tmp2.png



Output