Finding isolated pixels.

[rnbc]

Hi!

I'm trying to automatically detect hot pixels in a sensor. In order to do so I defined a criteria for "isolated pixel" which states that a pixel is more isolated if the difference to all it's neighbors is high. If the pixel has any close neighbor, even if only one, than it's not isolated.

I'm calculating difference in terms of luminance, where luminance = sum of all channels. That's debatable, but anyway... my problem is implementing this.

The definition would be something like, in terms of -fx:

Code: Select all

convert.exe \
Lenna.tif \
-fx '\
min(\
abs(r+g+b-p[-1,0].r-p[-1,0].g-p[-1,0].b),\
abs(r+g+b-p[1,0].r-p[1,0].g-p[1,0].b),\
abs(r+g+b-p[0,-1].r-p[0,-1].g-p[0,-1].b),\
abs(r+g+b-p[0,1].r-p[0,1].g-p[0,1].b),\
abs(r+g+b-p[-1,1].r-p[-1,1].g-p[-1,1].b),\
abs(r+g+b-p[1,1].r-p[1,1].g-p[1,1].b),\
abs(r+g+b-p[1,-1].r-p[1,-1].g-p[1,-1].b),\
abs(r+g+b-p[-1,-1].r-p[-1,-1].g-p[-1,-1].b)\
)' \
-evaluate Multiply 0.1 \
-depth 8 \
-interlace line \
-quality 100 \
-sampling-factor 1x1 \
Lenna-isolated-pixels.tif

And isolated pixels should appear whiter than the others. I specifically introduced a white pixel in the middle and a few other artifacts to test this. But it's not working, the image should appear almost black except for a few pixels...

Any ideas?

[fmw42]

Do your colorspace conversion first, so you don't have to do in fx and slow that down. You can convert to colorspace gray or get channel 1 of colorspace OHTA which is the simple average of the r,g,b channels.

Have you tried using -lat (local area threshold)?

You might also try my isonoise script and then get the difference between the input and output images to find the noise pixels.

Perhaps you can post your test image for others to work with.

[rnbc]

Hi fmw42. Sorry I didn't add to the discussion concerning HDR images the other day... I read it, but I had nothing to add really

Just to clarify what an isolated pixel is, and is not:



This image contains 9 isolated pixels, the others with neighbors in the diagonals, or vertical/horizontal are not isolated.

I don't see how I can do it without the fx operator, or without writing some C code

The question is, why doesn't the fx operator work in the way I'm using it?

[rnbc]

It seems as if -fx runs left to right, up to down, and changes pixel values in place, so that I'm comparing with an already changed image, instead of comparing inside the original image and generating a new image.

How can I compare inside the original image? Generating a new image with the result of the comparison, of course.

[fmw42]

It seems as if -fx runs left to right, up to down, and changes pixel values in place, so that I'm comparing with an already changed image, instead of comparing inside the original image and generating a new image.

How can I compare inside the original image? Generating a new image with the result of the comparison, of course.

I don't think this is the case about the way fx works.

In the mean time, try this:



convert lena_test.png -colorspace gray -lat 5x5+55% lena_test_lat5x55.png

[fmw42]

fx min() only allows two arguments. try this


convert lena_test.png -colorspace gray -monitor -fx " \
min(min(min(min(min(min(min \
(abs(u-p[-1,0]), \
abs(u-p[1,0])), \
abs(u-p[0,-1])), \
abs(u-p[0,1])), \
abs(u-p[-1,1])), \
abs(u-p[1,1])), \
abs(u-p[1,-1])), \
abs(u-p[-1,-1]))" \
+monitor lena_test_fx.png



You might also be able to use custom -morphology shapes or correlations to match each of your patterns above faster than using -fx. However, I have not experimented with doing that.

see
http://www.imagemagick.org/Usage/morphology/#user
http://www.imagemagick.org/Usage/convolve/#counting
http://www.imagemagick.org/Usage/convol ... ate_search

[rnbc]

Thanks, that was the problem, min only accepts two arguments! Dumb me!

I'll try to devise some faster method, but this one already works miracles for what I want

[fmw42]

I am not sure that min() really only allows 2 arguments.

I tried:


convert lena_test.png -colorspace gray -monitor -fx " \
min( \
abs(u-p[-1,0]), \
abs(u-p[1,0]), \
abs(u-p[0,-1]), \
abs(u-p[0,1]), \
abs(u-p[-1,1]), \
abs(u-p[1,1]), \
abs(u-p[1,-1]), \
abs(u-p[-1,-1]))" \
+monitor lena_test_fx2.png

And did not get any errors but got the following image which is not the same as the previous one. Perhaps it only used the first two arguments in the min?




Perhaps we need to hear from Anthony to get a better understanding.

[rnbc]

With only 2 arguments I get the expected results. Not so with more... so there is a problem somewhere.

Anyway, this is not an efficient way of doing things. Better do it by:

- cloning the image
- moving the clone 1 pixel
- comparing the entire image with the clone with "-compose difference"
- generate 8 comparison images this way
- take the max() form this set of 8 images, at each pixel location generating the isolated pixel map (the "alpha" channel), with "-compose Darken"

I haven't implemented that part efficiently yet, still using the "fx" method. Soon...! I'll post when done!

The rest, to attenuate the isolated pixels, goes like this:

Code: Select all

time nice +99 /usr/local/apps/ImageMagick/installed/bin/convert.exe SDIMa_26009-isolated.tif -alpha off SDIMa_26009-alpha.tif -compose copyopacity -composite SDIMa_26009-transparent.tif
time nice +99 /usr/local/apps/ImageMagick/installed/bin/convert.exe SDIMa_26009-isolated.tif -gaussian-blur 9x3 SDIMa_26009-blurred.tif
time nice +99 /usr/local/apps/ImageMagick/installed/bin/composite.exe SDIMa_26009-transparent.tif SDIMa_26009-blurred.tif SDIMa_26009-clean.tif
time nice +99 /usr/local/apps/ImageMagick/installed/bin/convert.exe SDIMa_26009-clean.tif -depth 8 -interlace line -quality 100 -sampling-factor 1x1 SDIMa_26009-final.jpg

Of course the alpha image can be contrast-enhanced, etc... to better remove the hot pixels, if needed.

[fmw42]

This shows that fx min() gives strange result with more than 2 arguments.

this seems to find the last argument
convert xc: -format "%[fx:min(10,8,2,4)]" info:
4

but this seems to find the correct argument
convert xc: -format "%[fx:min(2,4,8,10)]" info:
2


whereas this is correct with 2 args
convert xc: -format "%[fx:min(2,4)]" info:
2

and

convert xc: -format "%[fx:min(4,2)]" info:
2

[magick]

Most / all -fx methods take 1 or 2 parameters. The expression is reparsed for each pixel so for efficiency we keep the parser simplistic.

[fmw42]

- cloning the image
- moving the clone 1 pixel
- comparing the entire image with the clone with "-compose difference"
- generate 8 comparison images this way

That is similar to how I did several of my morphology script operations. You can see the code in my morphology script. Similarly in my statsfilt script. Perhaps those scripts will help you do that more efficiently.

I create 8 different images using -roll and saved in mpc format. Then used -compose darken or -compose lighten to find the min and max.

You can do something similar by creating the rolled images and compose diff to get the 8 abs diff images, then find the min or max as appropriate using -evaluate-sequence min or max, which allow any number of images and was not available when I did my scripts. see http://www.imagemagick.org/script/comma ... e-sequence

The rest, to attenuate the isolated pixels, goes like this:

Once you get the isolated pixels, you could create a mask. Then process the image with a median filter (rather than the gaussian blur) and use the mask to blend the two so that only the isolated pixels get replaced with the median. That is what I did in my isonoise script.

[anthony]

I would simple use a hit-n-miss morphology with a peaks:1.5 kernel. It does exactly as you say, only returning a pixel that has a separation between the center (peak) and the largest neighbour.

convert lena_test.png -morphology HMT Peaks:1.5 show:

In fact the amount of separation between the largest neighbour and the peak pixel is what the operation returns! In other words it is a direct comparison of the results of separate 'background' and 'foreground' pixel tests.
See Hit and Miss Morphology
http://www.imagemagick.org/Usage/morphology/#hmt
And specifically HMT with Gray-scale Images
http://www.imagemagick.org/Usage/morpho ... _greyscale

It would certainly be a lot faster than any FX method, and you can adjust the neighbourhood very easily.
http://www.imagemagick.org/Usage/morphology/#peaks

NOTE this is different to using a "Correlation" (actually the same as convolution in this case), which uses averages, as as such can find a peak even when one neighbour is only slightly equal or larger than the 'origin' pixel. This is also why LAT can fail.

[rnbc]

Anthony, the Morphology method seems to work rather well, thanks! I'll test it on some real images with real hot pixels and report the results. The problem is that hot pixels are not as "hot" as in my example images, and the "fx" method seems more sensitive. Then again, only practice will tell

I understand they should do the same though, at least for hot pixels. The "fx" method also detects "dark/stuck" pixels, which tend to be rarer, but exist sometimes (not in any of my cameras though).

[rnbc]

I found this worked great:

Code: Select all

convert Lenna-128-isolated.tif \( +clone -morphology HMT Peaks:1.5 \) +swap -compose subtract -composite Lenna-128-subtract.tif