Legacy ImageMagick Discussions Archive

Hello Everyone!

I'm new to the software, And I need some help regarding a project.

I basically need to remove the black background of over 700000 tif images.

Thing is, it's basically paper (with information) on a black background, and this background has a lot of white dots spread all over it.

But the white paper's position and size, and the image's resolution are really dynamic. Is there a way for me to do this using the command lines?

I've have images as examples of how they are and how they should become in the google Drive link below.

https://drive.google.com/open?id=1R_Ekp ... h9pxrz8fVm

Thanks for the help!

Please always provide your Imagemagick version, date of version and platform since syntax may differ.

I took your first image and used the following connect-components processing:

Code: Select all

convert Original_Image.tif -threshold 50% -type bilevel \
-define connected-components:area-threshold=5000 \
-define connected-components:mean-color=true \
-connected-components 4 \
-morphology smooth octagon:5 \
-transparent black Original_Image_result.png

http://www.fmwconcepts.com/misc_tests/t ... result.png

See
https://imagemagick.org/script/connected-components.php
https://imagemagick.org/Usage/morphology/

Hello fmw42!

Thanks for the patience!

The version I'm running is the ImageMagick-7.0.8-Q16, date of the version seems to be 2019-01-27, according to last log change, and platform is Windows 7 64-bits.

I tried running the lines, and studied the links provided, along with some extras.

But the images I'm trying to crop were actualy microfilms originaly, and were converted into .tif files in a scanner. Due to that, image quality can be reeeeeeally bad, as in lots of blurry text, pixaled black dots on white paper and highly varying resolutions and positions of the paper in the image.

Also, the examples I used were blank pages, but the actual images contain a lot of text (even tough most of it is unreadable and blurry). So applying these lines, starting from

Code: Select all

-connected-components 4

command, the text inside the actual paper was erased.

Ideally, an automatic crop that identifies where the paper begins and where it ends, and then crops that out (without modifyig the contents of what's written in the paper) would be perfect. I have another example of a blank image, this time with the pixalated effect I described at the edges of the the paper.

https://drive.google.com/file/d/1yCS_50 ... sp=sharing

Again, thanks for the help and the patience!

Do the same, but use the result as a mask to put into the alpha channel of the image.

Unix Syntax:

Code: Select all

convert Original_Image_3.tif \
\( +clone -morphology close octagon:9  -type bilevel \
-define connected-components:area-threshold=5000 \
-define connected-components:mean-color=true \
-connected-components 4 \) \
-alpha off -compose copy_opacity -composite \
Original_Image_3.png

Windows Syntax:

Code: Select all

convert Original_Image_3.tif ^
( +clone -morphology close octagon:9  -type bilevel ^
-define connected-components:area-threshold=5000 ^
-define connected-components:mean-color=true ^
-connected-components 4 ) ^
-alpha off -compose copy_opacity -composite ^
Original_Image_3.png

fmw42, that was PERFECT! Thank you!

But I still need to remove the white background, as to reduce the image resolution...the final image would have it's 0x0 pixel coordinates as the start of the paper, and resolution would be the coordinates at the end of the paper, or a crop rectangle that would cover all of the paper if the paper is skewed. That's why I suggested the crop function. Any final line that would manage that or any software that can do batch removal of background?

Once again, I'm REALLY thankful for the help, this will save my skin.

add -trim just before the output

Code: Select all

convert Original_Image_3.tif ^
( +clone -morphology close octagon:9  -type bilevel ^
-define connected-components:area-threshold=5000 ^
-define connected-components:mean-color=true ^
-connected-components 4 ) ^
-alpha off -compose copy_opacity -composite ^
-trim ^
Original_Image_3.png

That will not necessarily make all pixel be on your image if the image is rotated

Hey fmw42!

It worked like a charm...for the most part.

One of the films cotains really noisy backgrounds, like so:

https://drive.google.com/open?id=1lS4pM ... 5k89SgxWRf

So the command lines written basically do nothing, I assume due to these numerous white dots and random white lines in the background.

As far as I've been able to grasp reading the IM Usage page, in these lines, you basicaly "close" the black dots in the white page, defined as a continuous octagon of 9 white pixels . Then, you define the foreground as a continuous 4 white pixels, turn off the alpha to make the background opaque, then trim it all away.

Am I on the mark or do I need to study more? I'm still having trouble understanding the "-define" command, especially the "mean-color=true" (english is not my native language)

Is there a way to make the lines consider the noise on the background of the image?

Once again, thanks for all the help! This is already beyond amazing for what I'm trying to do.

You have blacked out your text. So the result will not be good. But the main issue is that you have opposite polarity for the background. So replace -morphology close with -morphology open.

Code: Select all

convert Original_Image3.tif \
\( +clone -morphology open octagon:9  -type bilevel +write tmp1.png \
-define connected-components:area-threshold=5000 \
-define connected-components:mean-color=true \
-connected-components 4 +write tmp2.png \) \
-alpha off -compose copy_opacity -composite \
-trim +repage \
Original_Image3.png

Is that what you want?

Without

Code: Select all

-define connected-components:mean-color=true

the output will be gray levels associated with the region indices. With it, the output will have the original image colors after being merged.

I managed to find a blank page that ilustrates the problem more clearly, so it should be better for testing. As you'll be able top see, there is even a white line on the right side that makes it even more dificult (caused likely due to a bad scanning). Some images even have their bottom parts with the start of another page, but I don't think there is anything to be done about that.

I tried this code on it but the results are as follows...I think opening up the morphology just causes it to fill more white spots than there were at first, up until the border of the image.

https://drive.google.com/open?id=1-6zjh ... ow6MFlvqp5 <- Original
https://drive.google.com/open?id=1mFdNB ... MlYdBXDKAp <- tmp1
https://drive.google.com/open?id=1Kl-EY ... 3C6_4z4esj <- tmp2
https://drive.google.com/open?id=11RrxI ... 3kbJ75X_Qs <-End image

The first code you gave me is already working for the images with less noise. The bigger issue are these noisier ones, since the majority are like that.

I'm trying to study in the IM Usage page, but I have no formal training in image manipulation, so a lot of the concepts go over my head...even so, what I already learned is fascinating, and I'm trying to find more material to study. If you could recommend any, that would be awesome.

Sorry to keep asking, but I'm really desperate to solve this

Try this. I am not sure I can do better. I do not get any horizontal stripes. What is your IM version, date of version and platform?

Code: Select all

convert Original_Image4.tif \
\( +clone -morphology open octagon:3  -type bilevel +write tmp1.png \
-define connected-components:area-threshold=5000 \
-define connected-components:mean-color=true \
-connected-components 4 \
-morphology close octagon:15 +write tmp2.png \) \
-alpha off -compose copy_opacity -composite \
-trim +repage \
Original_Image4.png

I'm using the ImageMagick-7.0.8-27-Q16-x64-dll, downloaded from the homepage of ImageMagick, on a Windows 7 Pro 64-bits. The date of the version, according to the log files, is 2019-01-27.

The white stripe I mentioned is actualy in the original image, verticaly, on the right. I just assumed it would interfere with the processing. Does not seem to be the case.

The final image should also be in TIF format (noticed you've been outputting for PNG since I did that in my original post, my bad). Converting to TIF at the end of the code make these lines in the automatic black background that is generated in a TIF appear. But for some reason, if I convert to TIF after first converting it to PNG (as per your original code) the background remains white and there are no lines on it (which is weird).

Thing is, the code now seems to be invading the paper and erasing parts of the text that it confuses with background, resulting in things like this:

https://drive.google.com/open?id=1AKNzy ... bzk3jApNn1 <-Original border of the paper
https://drive.google.com/open?id=1r0qR- ... tnIjEDgBOI <-Result
https://drive.google.com/open?id=1d-MUb ... tuD4zQcOM5 <-Text in the image
https://drive.google.com/open?id=1lAQj- ... p69cc20w9E <-Result
https://drive.google.com/open?id=15lExx ... 8gXrAIBtXj<- Figure of a Wing
https://drive.google.com/open?id=1oM12V ... i-jDT0MDkB<- Result

https://drive.google.com/open?id=1RBRMi ... NvWa0GUlW1 <- Another border, but this one realy ilustrates how bad these images can be. Believe it, this is text, but it just looks like a bunch of black blotches

https://drive.google.com/open?id=11cHX0 ... 4ykhygjTt_ <- The result of its conversion, looking like swiss cheese.

Making the output a PNG and converting to TIF afterwards is no problem, since its all batch files anyway. The last problem, it seems, is this part where it doesn't properly recognize the boundaries of the paper and what to keep inside the image.

I do not think I can help further. This is a very hard problem and your images are so noise. I do not understand where the green comes from. Is that from coloring the transparent regions?

Actualy, the green is just the visualisation option from IrphanView. Since these images are PNG and the background is transparent, I just made it so the IrphanView showed this green background in order to see a better contrast.

The help you gave so far is already amazing, way beyond what I'm capable of doing for sure.

That first answer you gave me back in 2019-02-08 already worked wonders for the less noisy films, and the code with the "open" and "close" command seems to be the solution for the noisier films, needing just a little bit of fine tunning in order to make it regognize background and frontground.

I'll try and mess around with the morphology configs and see if it gets better. If you could make just one last try to help (as you clearly understand this better than I do), that would also be amazing.

Once again thanks for the amazing help!

For simple ones like the wing, just do CCL with a very small area threshold

Code: Select all

convert Wing_original.PNG -type bilevel \
-define connected-components:area-threshold=1 \
-define connected-components:mean-color=true \
-connected-components 4 \
Wing_original_proc.png

What do you expect for result for

https://drive.google.com/open?id=1AKNzy ... bzk3jApNn1

What do I do with the green in the input?

What do you expect for result for

https://drive.google.com/open?id=1RBRMi ... NvWa0GUlW1

All small black or white spots removed or only the white ones in the outside black area. If the latter, that needs CCL to be enhanced so that you can specify color as well as area-threshold.

But try

Code: Select all

convert Original_border2.PNG -type bilevel \
-define connected-components:area-threshold=25 \
-define connected-components:mean-color=true \
-connected-components 4 \
-trim +repage \
Original_border2_proc.png

Hey fmw42!

Just to clarify, these images were crops (printscreens) of sections of the results the last code gave me. The green areas are transparencies, and I've left them like that just as a visual cue for what happened, since the it seems the "open morphology" command made holes of transparency inside the paper (which was what I was trying to avoid).

I've messed around with the command lines and the diferent settings for area threshold, conected components, morphology, etc, along with trying to make sense of the tutorials on the usage page.

But I could not generate a command line that managed to do the job the code you wrote me on 02-08-2019 did for about 5.000 images of the 700.000 (those were the images with less noise). Since noisiness and resolutions of the images are so diverse, what seems to work for one image does not work for the other.

I've gathered a bunch of full images, with varying degrees of quality, if you want to check. With the selection below, I'm confident that a line that works for all of them will work for the whole package.

Here's the link: https://drive.google.com/drive/folders/ ... sp=sharing

I'll try and check on another forum dedicated to batch image manipulation for another solution. In the mean time, I'll also keep studying to try and make a command line that works for at least most of them.

Thanks for all the help all the same! It was amazing

Legacy ImageMagick Discussions Archive

Dynamic removal of black background

Dynamic removal of black background

Re: Dynamic removal of black background

Re: Dynamic removal of black background

Re: Dynamic removal of black background

Re: Dynamic removal of black background

Re: Dynamic removal of black background

Re: Dynamic removal of black background

Re: Dynamic removal of black background

Re: Dynamic removal of black background

Re: Dynamic removal of black background

Re: Dynamic removal of black background

Re: Dynamic removal of black background

Re: Dynamic removal of black background

Re: Dynamic removal of black background

Re: Dynamic removal of black background