Evaluate Blurriness/Sharpness Factor

[7eicher]

Dear friends,

first let me say, I like IM and I find it amazing how much work you guys have spent into!

I came to this forum with the hope to find an answer to my question. In my current project I have shot thousands of images, which are used for photogrammetry 3D reconstruction of a house. The images need to be sharp for this process, but unfortunately the shots arn't that perfect for all images. So some images have a slight motion blur and I would like to detect it and remove the blurry images from the pipeline. In Stackoverflow someone suggested to use color deviation and/or edge-amount to detect the difference between the images, but that didn't work out well, as the images seem too different, in color and objects.

Does anybody have an idea how to detect a value X to measure the sharpness of an image. I have to treat hundreds of images and want to automate this process, to give me at least and indication. I had a look at blind deconvolution and it sounds promising, but I have no idea how to go further.

Does anybody have expperience in this field or ideas what to look at. I dont expect a ready to go code, but if, yes this would be fantastic off course .

Take care,

Markus

[snibgo]

If you have two photographs of the same scene, at the same exposure, finding the sharpest one is easy: it has the highest standard deviation.

Is this your situation? Or are the photographs all different? Or do they overlap?

[7eicher]

They are all shot from different position with a certain degree of overlapp, round about 25% or more. Therefore standard deviation might not be so usefull, as new objects become visable in the scene and others leave.

Here are the images for one of the rooms: https://www.dropbox.com/sh/6gzb7djwzif1 ... wX6Ta?dl=0

[snibgo]

Skimming through them, I can't see any motion blur. But they are smallish (1000x667). Perhaps you have larger versions with blur.

Even so, any blur will probably be small. You might use techniques to match areas between photos, then find the sharpest pair-wise. This may not work well, as matching photos will distort them and thus may create larger blurs than the motion blur.

If motion blur is linear, perhaps repeating the blur in the same direction doesn't make it much worse, so that could give you a detection method.

Perhaps finding the maximum gradient between pixels is a suitable measure. Photos with maximum gradient below a threshold are rejected.

You have 5700 photos. Manually identifying the ones with motion blur might take 3 hours, and this might be the fastest method. But that is just one room. How many rooms?

[7eicher]

Yes, these images are only thumbnails. The originals have 7900x4300 pixel in RAW (NEF) and most of them dont have motion blur, but some do little. Very few a lot. In total we have round about 13.000 images, 4 rooms + exterior. The problem with the manual check is, it will take a lot of time, just to load the big images. And as you have seen allready, you need to load the original size to see motion blur with eye. So loading 36Mpixel in TIF 16bit Tonemapped (100Mb+ per image) is a bit heavy. Actually I would like to have an algorithm which suites also for further projects, so I think I would invest a bit of time.

>> If motion blur is linear, perhaps repeating the blur in the same direction doesn't make it much worse, so that could give you a detection method.

Sounds interesting. The motion blur is usually linear, but I dont know the direction. So I would simply try to apply an unblur in several directions, with decent degree. If I understand you right, than I check the difference of standard deviation between original and pseudo-unblurred image per direction. If one of the direction has higher deviation, it means, here we have some blur in the original image. Right?

[snibgo]

I don't know much about unblurring. It usually creates ringing, making calculations of sharpness difficult. But blurring an image in one direction, at various different angles, might tell us something. For example, if we blur vertically and the standard deviation drops, then blur horizontally and it doesn't, then the input was already blurred horizontally.

Can you make a crop of one of the blurred images, about 500x500, and put the result here? The best quality, eg 16-bit PNG or TIFF. Then we can experiment.

[7eicher]

Sure I can do. Here is are some sources, cropped and original, exr, tif8 and tif16 (toned): https://www.dropbox.com/sh/tnjkomqf7556 ... 89ITa?dl=0

I think best would be test based on CRB_7983_tonemapped_crop.tif, 16bit tonemapped, 1000px. I will do some tests with IM and post them here. Meanwhile I tried a commercial product called SmartBlur.net, which performs blind deconvolution and got some results. You can find it also in the Dropbox, its file SmartBlur-unblurred.jpg.

I will do now some IM blurring in different directions and see how standard deviation changes.

Take care,

Markus

PS: I liked your site with the IM examples and scripts, interesting!

[snibgo]

The scheme works.

If we look at both CRB_7983 images, we see vertical lines are quite sharp but horizontal lines are not. So we have blur in the vertical direction.

I use "-motion-blur" as a simple blur that is directional. It simulates motion that accelerates to the left.

Windows BAT syntax.

Code: Select all

set BLUR=0x3

%IM%convert xc:White -bordercolor Black -border 10 ^
  -motion-blur %BLUR%+0 ^
  -auto-level ^
  -scale 1000%% ^
  mb_sample.png

This shows the blur I am simulating.

Code: Select all

set SRC=CRB_7983_tonemapped_crop.tif

%IM%convert ^
  %SRC% ^
  -format "Orig: %%[fx:standard_deviation]\n" ^
  -write info: ^
  ( +clone ^
    -motion-blur %BLUR%+0 ^
    -format "   0: %%[fx:standard_deviation]\n" ^
    -write info: +delete ) ^
  ( +clone ^
    -motion-blur %BLUR%+90 ^
    -format "  90: %%[fx:standard_deviation]\n" ^
    -write info: +delete ) ^
  ( +clone ^
    -motion-blur %BLUR%+180 ^
    -format " 180: %%[fx:standard_deviation]\n" ^
    -write info: +delete ) ^
  ( +clone ^
    -motion-blur %BLUR%+270 ^
    -format " 270: %%[fx:standard_deviation]\n" ^
    -write info: +delete ) ^
  NULL:

The output is:

Code: Select all

Orig: 0.192653
   0: 0.190436
  90: 0.192007
 180: 0.190813
 270: 0.191995

For the 0 degree blur (ie horizontal), the SD drops by about 20 parts in 2000.

For the 90 degree blur (ie vertical), the SD drops by about 6 parts in 2000.

Hence, the image is already blurred vertically.

The 45-degree angles should also be measured because an image photographed with a 45-degree blur would show equal SD drops for 0 and 90 degrees.

Note: strictly, this doesn't tell us the image is blurred. An sharp artificial image of vertical lines would show an SD drop when blurred horizontally but not when blurred vertically, so this test would say falsely that the image was blurred vertically. I suspect that for real photographs, the test is probably good, but more testing is needed to confirm.


PS. Thanks for your comment on my site. I'll add this topic to the "to-do" list.

[7eicher]

The appraoch works geniously, very good Alan!

I tested it, each 30deg. and it works out well as long as the motion blur is linear. Thats the deviation for a pretty sharp image:

Orig: 0.219353
0: 0.21501
30: 0.212929
60: 0.210584
90: 0.210118
120: 0.210869
150: 0.212393
180: 0.214804
210: 0.213837
240: 0.212407
270: 0.212422
300: 0.212947
330: 0.213658

One can see, 1) the max. difference between any of the directions and the original is limited and 2) the max. difference between any of the directions is also small compared to the image with motion-blur.

I plan to write a small script to walk over all my images and make some cross checks. I will let you know.

Take care and thanks so far!

Markus

[7eicher]

I should mention that this approach doesnt cover linear-motion-blur in the direction of the image forth or backwards. It only works when moving right,left,up or down.

[snibgo]

We can convert the SD drops to percentages. For example at 90 degrees: -100*(0.210118-0.219353)/0.219353 = 4.2%.

Then we can plot the percentage drops on a polar chart. (I use OpenOffice, a "net" chart.)



We can see a difference between vertical and horizontal, in the ratio about 2:1. CRB_7983_tonemapped_crop.tif had a ratio of about 3:1.

Yes, this tests only linear motion, eg the camera twists about a vertical axis. It doesn't test for the camera moving forwards or backwards, or twisting the camera on an axis towards the object. These could be tested by using different types of blur.

[snibgo]

Just another thought:

CRB_7983.tif[0] shows specular highlights, smeared in a nearly vertical direction about 30 pixels long, in patterns. A script might be written that identified this pattern, and each element, the 30-pixel smear. This flags the image as suffering from camera shake.

Even better, it defines the point spread function (PSF) or kernel for a deconvolution that would remove the camera shake. Then the image wouldn't need to be discarded.

[snibgo]

Yes, that theory works. With Gimp, I cropped out one of the smears (the one that lies on pixel coordinate 4512,2583), painted out adjacent smears, and adjusted the levels. Here is the resulting kernel:

After 10 iterations of a Richardson-Lucy deconvolution, the image is much sharper (though still with smearing, and added ringing, so the kernel isn't perfect). Here's a crop of the result:

[7eicher]

Interesting approach.

I can't see the images. Could you upload them somewhere?

[snibgo]

Sorry about that. Blooming dropbox. Try these URLs:

Crop that shows camera shake:
https://dl-web.dropbox.com/get/unblur_h ... 0R9rA&dl=1
Kernel:
https://dl-web.dropbox.com/get/k_knl2.p ... 9t-lQ&dl=1
Unblurred with that kernel:
https://dl-web.dropbox.com/get/unblr_cr ... DhZsw&dl=1