Create overlay mask to correct exposure

[royeven]

I'm working with a project [1] where I want to take photographs of something covered by glass. My problem is that the lighting is uneven and I get various reflections from the glass. It also doesn't help that the glass platen is angled in relation to the light source, so that the light intensity is higher at the part of the glass platen close to the light and lower at the parts further away from the light source. This gives me very uneven exposure of the photographed item. I have done my very best to fix the uneven lighting conditions, but because of the geometries and the placement of the camera I have not been able to get a satisfactory solution.

I would therefore try do compensate for the uneven exposure in software. Because of the large amount of pictures where the exposure will need to be corrected, it's crucial that I can make a progam that can be executed via the command line, preferably in linux, and imagemagick seems like the best place to start. A search of similar situations and discussions mostly turned up subjects where this sort of correcting is done on a picture-to-picture basis, which is impossible in my case. I would like to develop a method where this can be done "automagically". The closest thing I have found to a solution is a discussion implying that this is something I really shouldn't be doing:
viewtopic.php?f=1&t=14267

But I would very much like to get something like this working, and am willing to invest quite some time in doing it. My problem is that I'm out on very deep waters here, as I'm not familiar with photo editing or the techniques. That's why I turn to you for hints and tips to what I need to do or for handy references to other tools that might help me in achieving the goal.

My theory is that I can take a photograph of a black piece of cardboard positioned directly underneath the glass. The areas that reflects too much light will be brighter than the areas where the light intensity is correct. And I think it should be possible to use this to calculate the amount of exposure correction needed for each pixel. The camera operates in manual mode, so the f-stop and the exposure time is the same from picture to picture. The lighing, the glass platen and the camera is fixed, and doesn't move, so it should be possible to create a mask to correct the exposure.

But I don't know how to calculate the amount of correction needed for each pixel. Has anyone here been in a similar situation? I could really need some starting ideas and hints on how to solve this problem.

In advance, thanks for your help
Regards,
royeven

[1] I'm building a book scanner. A video of the design can be found here: http://www.youtube.com/watch?v=a4-qMc2QSOw

Edit:
Oh, I forgot to mention that I would love to work with raw image files, preferably *.ppm

[snibgo]

Interesting project, and very clear explanation on the video.

In general, the starting point is to get the lighting as even as possible because software correction will never be perfect. In this situation, where users can install whatever lights they want, you need a general solution. As you say in the video, the machine should be covered in a black cloth to eliminate lighting variation.

Yes, I should think a solution is possible. The user would scan blank white paper and feed the results into a script, which generates a mask that can be used for all subsequent scans. (I say, "a mask", but it might need more than one. And maybe the setup should involve more than one scan, perhaps with grey and black paper.)

For development, the first step is to scan some samples, so we can see what the problems are. Scan both sides, so we can see the lighting variation -- I expect is it roughly symmetrical. Scan plain white paper and, if you can, also plain grey and plain black. If your camera can only produce JPEGs we'll have to work with that, but if you can get raw images that would be very much better.

Then put the results on a web site somewhere, and post the URL here. (Dropbox works well.)

[snibgo]

.

[fmw42]

I believe you want something like vignette correction using a mask and -compose divide. It is like evening out the background.

see
http://www.imagemagick.org/Usage/photos/#vignettation
http://www.imagemagick.org/Usage/compose/#divide

Take a photo of a white card and modify it so that the brightest values become pure white, which is where you think it is correct and darker is where it needs to be made brighter. Then use -compose divide to correct the photos. Be careful of the image and mask order.

If you use convert

convert whitecard image -compose divide -composite result
or

convert image whitecard +swap -compose divide -composite result

or though not recommended

composite -divide image whitecard result.

or use the appropriate Divide_Dst, Divide_Src

see
http://www.imagemagick.org/Usage/compose/#compose

[royeven]

Hi again and thanks for the tips from both of you! I have spent my time off from work for the last week working on improving the design, but my attempts have not been successful. First of all I feel I should emphasize that it's not me in the video. The design of the scanner is the product of the guy behind http://www.diybookscanner.org. I'm just one of many trying to build a scanner of my own and adjusting (and hopefully improving) the design. I realize that the better lighting conditions I have to start out with, the better the final result will be. In this case though, I'm afraid that difficult lighting conditions comes as part of the package. I'll try to explain the problem.

This is my setup with cameras and lights


In order to get least geometry distortions (distortions in perspective and angle), the camera has to be mounted perpendicularly to the glass platen. This captures the page as a flat surface. Unfortunately, in order to also get even lighting, you have to have the light where the camera lens is. In the above picture, the bottom of the book platen is further away from the light than the top. Since the light intensity (watt light received pr. square inch) lessens when you get further away from the light[1], the light intensity is higher at the top than at the bottom, which gives an uneven exposure of the image. As far as my investigations go, this can not be fixed! In order to fix it, the light must come from behind the camera or from the side of the camera, which would give a very bright reflection in the glass platen, which is worse than my current setup.

When I photograph an entirely plack piece of cardboard, this is the result:
https://dl.dropboxusercontent.com/u/188 ... _black.ppm
The image is in ppm format, so I was not able to make it display directly in the broser. If there are anybody out there willing to help me but not being able to open ppm images, send me a PM and I'll post the images as jpg. I use ppm because it's lossless, but solving the problem is more important to me.

It doesn't look too bad. The horizontal line across the image is the bottom of the v-shaped glass, i.e. where the spine of the book would be. I have only photographed the right-handed side of the calibration photos since the left-handed side would be symetrical, pluss I would like to search for a method to correct the exposure that works independently of the lighting, size of the platen, etc. At first glance it looks like it's evenly lit, but when playing with the level-tool or treshold-tool in gimp (or photoshop) it becomes clear that there is a circular (elliptical?) area at the top that receives much more light than the rest of the image. I think that's because of the light intensity trouble described above.

Here is a photograph of an 18% grey card:
https://dl.dropboxusercontent.com/u/188 ... n_gray.ppm
and here of a white card:
https://dl.dropboxusercontent.com/u/188 ... _white.ppm

I don't have a gray card or a white card big enough to fill the entire platen, so my idea is to make the correction in two steps:
1. Use the photograph of the black cardboard to fix the exposure
2. Use the grey card or the white card to fix the color distortion by step 1 and the conversion from NEF-files to PPM-files[2].

I have read very carefully the part of the convert-function that fmw42 mentioned, and a great deal of theory on vignette distortions, and the more I read, the more uncertain i become? Is vignette-correction the correct course for this quest? I don't want to make any parts of the images brighter; some parts are overexposed as it is. I would like to make the brightest pixels from the black calibration photo darker, so that the entire photo becomes the same color. And I would like to apply the same exposure adjustment to every other page (i.e. all the right-handed pictures in the photo series). The left-handed photo series must be given the same treatment, but with a different bleck correction image and a different correction mask.

For reference, here are some images of book pages for the project. The book is copyrighted, so I don't want to post many pages here:
https://dl.dropboxusercontent.com/u/188 ... /page1.ppm
https://dl.dropboxusercontent.com/u/188 ... /page3.ppm
https://dl.dropboxusercontent.com/u/188 ... /page5.ppm

I have been trying the convert tips from fmw42, but wasn't able to get it to work, most likely because it's a method designed to make the darkest points brighter, which I don't really want to do (or do I?), and because I don't have a white card big enough to fill the platen. Isn't it better to use black as calibration photo in this case, since the white card is reflective in itself, and the extra exposure from the circular (or elliptical?) area on top will be less "analyzeable" when it blends in with the already high exposure from the white card?

I'm hoping for tips from more experienced photographers and users of imagemagick than myself. I'm really, really lost here.

Regards,
royeven

[1] The light intensity is inversely proportional to the square of the distance between the platen and the light source
[2] I have a Nikon D3200. Its raw format is *.NEF. The conversion from NEF to PPM doesn't use the metadata stored in the NEF to reproduce the correct white balance from the camera, so I need to use a grey card for that. But the grey card is equally affected by the wxposure distortions as the rest of the photos are.

[snibgo]

Gosh, dropbox is slow tonight. While I'm downloading, a few thoughts.

Code: Select all

convert page1.ppm page1.png

... then you can view page1.png with any image viewer.

Does the D3200 capture more than 8 bits? If it does, you should create 16-bit files, not 8-bit files. The more information you get for calibration, the better. Don't post JPG files. They lose data.

Do you use dcraw to convert from the NEF? If so, "-T -6" will give 16-bit tiff files, which is generally more useful than PPM. Dcraw can also give you the camera white-balance settings. Or exiftool can give you this, also from the NEF file. You should probably have the camera set to manual white balance, but it probably doesn't matter. (I assume the D3200 works like the D800, which doesn't change the NEF pixel data for different white balances. It's just a setting in the file, which dcraw can use or ignore.)

Will your normal targets be white paper, or black paper? White, I suppose. In that case, a white calibration target is probably more useful. Then we can give you a process to make the entire target an even shade that is nearly white. (And it will be white, rather than blue or any other colour.)

The calibration target needs to cover the entire field. At a pinch, a card that covers over half the field would do, so you would photograph it twice, then blend the two photos. But it really is much easier with one card. It needs to be free of any blemishes. You gray card has a smudge, in the bottom-right of the photo. The conversion process would create a similar smudge on all the final images, which is a bad idea.

Okay, I can see the photos now. You seem to have set the camera on auto.

I'm off to bed soon. By the time I'm up again, Fred will probably have cracked it.

[fmw42]

I have not been following this very closely. I really am not much of a photographer. But as snibgo said, the calibration card(s), should cover the same are of the platen that your photographed page does. If you do not match the calibration card to the location of each photographed page, then the inverse distance squared light fall off cannot be matched in you photograph. You will need to have some way of marking or identifying where the photographed page is located relative to the calibration card or make sure they are the same size, orientation and location, so that the calibration card can be used to remove the uneven lighting.

[snibgo]

Okay, it's past my bedtime, but here is a good first try. (Windows batch script; for Unix, change ^ to \, and %% to %.

Code: Select all

convert ^
  page1.ppm ^
  calibration_white.ppm  ^
  +swap ^
  -compose Divide -composite ^
  -level 50%%,95%% ^
  page1.png

[anthony]

See...
Vignette Removal
http://www.imagemagick.org/Usage/photos/#vignettation

Of special note is some mathematical analysis of vignette in a special 'curved film' pinhole camera.
viewtopic.php?f=1&t=18944

It will give some mathematical bases to why and what is happening, though lenses (camera detail) can distort that mathematics. This is why pictures of calibration cards can be much more useful.

[snibgo]

The lightness of an object is affected by the distance from the light source, but also the angle it presents to both the light and the camera, and the surface characteristics of the object.


Let's start from this basic transformation. All scripts are for IM 6.8.5 under Windows 7.

Code: Select all

convert ^
  page1.ppm ^
  calibration_white.ppm ^
  -compose Divide_Src -composite ^
  page1.png

The result needs rotating and cropping. I'll ignore that for now. There are two other problems: the image has a slight blue or cyan colour cast, and it has chromatic aberation.

The blue cast might be because the book has bluer paper than the calibration card. Whatever the cause, it is easily removed, but I'll come back to that one later.

Chromatic aberation is a result of different wavelengths of light refracting differently. You can see it by zooming tightly to one of the letters near a corner of the image. Zoom in to fill the screen with the letter. You will see colour fringing around the letter. For most lenses, it can be corrected with a barrel distortion on two of the three channels.

Code: Select all

set CA_RED=0.0008770 -0.0036186 0.0037030 0.9997046
set CA_BLUE=-0.0006181 0.0014701 -0.0006352 0.9998279

convert ^
  page1.png ^
  -separate ^
  ( -clone 0 -distort Barrel "%CA_RED%"  ) -swap 0,-1 +delete ^
  ( -clone 2 -distort Barrel "%CA_BLUE%" ) -swap 2,-1 +delete ^
  -combine ^
  page1ca.png

These numbers are valid for your lens, at the same focus you used, and at the same zoom setting (if any). I got them from tca_correct.exe, which is part of the Hugin toolset. The correction takes about 50 seconds on my laptop.

Even if you will convert the image to greyscale, chromatic aberation correction will sharpen up the image slightly. This is real sharpening, correcting a fault inherent to all lenses, not to be confused with artificial sharpening such as "unsharp masks".

Now we've done that, let's colour-balance (aka "white balance") the image so the paper is a neutral gray. For this, I use Gimp eyedropper, sample average, radius 13, use info window. Zooming in (about 400%), I see cyan (blue-green) speckles. These are probably the paper texture, shown up by the oblique lighting. Typical RGB values (over a radius 13) are 253,254,254. So we could multiply the red channel by 243/253. But I want to avoid clipping, so instead I'll multiply both B and G by 253/254, which is 0.9961.

Code: Select all

convert ^
  page1ca.png ^
  -channel G -evaluate Multiply 0.9961 ^
  -channel B -evaluate Multiply 0.9961 ^
  page1cawb.png

This takes 20 seconds. There are still cyan blotches, but for normal viewing they are balanced by the slightly red background. If you are making the final version greyscale, don't bother with this step.

Modern books have consistent paper colour throughout the book, but older books often vary. You might (or might not) want to change the colour balance to compensate for this.

The blacks aren't totally black. They vary considerably between about 36% and 48%, but the text at the top is lighter than the text at the bottom, up to about 60%. Yikes. We know the paper is lighter at the top than the bottom, but it seems the ink is even worse. Perhaps because ink is shinier than paper. We probably need different corrections for black versus white, although it would be best if the black calibration chart was ink. For now, we'll ignore this problem. Let's make everything below 50%, black. The cyan blotches are above 90%, so we can get rid of them at the same time.

Code: Select all

convert ^
  page1cawb.png ^
  -level 50%%,90%% ^
  page1cawbl.png

Now the easy stuff. I'll crop a reasonable distance within the page boundary, to allow a tolerance when turning pages. I get the coordinates from Gimp. While looking at this, I see the page is very slightly barrel-shaped. I can't detect this on the text. At the same time, we rotate.

Code: Select all

convert ^
  page1cawbl.png ^
  -crop 4286x2556+1020+408 +repage ^
  -rotate 90 ^
  page1cawbc.png

The result isn't bad. I see two problems: the text bottom-right is higher than bottom-left. Text on the right is lighter than text on the left. Both problems are correctable.

Let's combine all the operations into one command.

Code: Select all

convert ^
  page1.ppm ^
  calibration_white.ppm ^
  -compose Divide_Src -composite ^
  -separate ^
  ( -clone 0 -distort Barrel "%CA_RED%"  ) -swap 0,-1 +delete ^
  ( -clone 2 -distort Barrel "%CA_BLUE%" ) -swap 2,-1 +delete ^
  -combine ^
  -channel G -evaluate Multiply 0.9961 ^
  -channel B -evaluate Multiply 0.9961 ^
  -level 50%%,90%% ^
  -crop 4286x2556+1020+408 +repage ^
  -rotate 90 ^
  page1_corrected.png

This takes 1 minute 6 seconds.

The same command works fine on page5.ppm. The result for page3.ppm isn't so good -- the paper isn't white. I strongly suspect your camera chose a different exposure for this page.

I strongly recommend that you use manual camera settings (shutter speed, aperture, ISO and white balance), and the same ones for the calibration and the entire book. If you use auto, a page that has denser text will get an increased exposure, so the background and text will both become lighter.

[snibgo]

As an aside, I might mention that dcraw can do white balancing in much the same way as ImageMagick. Or it can take the WB factors the camera put into the NEF file. We can read WB data from the NEF file, using dcraw or exiftool. (Probably also using IM.)

Dcraw can also correct for chromatic aberation, but only the first-order factor. IM is far better. Some cameras with some lenses "know" the CA factors for that lens, and will use that data when creating the JPG, but not the NEF file. It might put the data into the NEF file, but not in a way we can read.

Using non-CPU lenses on a Nikon D800, the camera doesn't correct CA. But the Nikon View NX2 software does. I don't know if View NX2 knows about the lens, or if the camera buried the settings in the file. I haven't compared CA results from View NX2 compared with the tca_correct and IM solution outlined above.

[snibgo]

Looking at the result, text lines are not the same length. In addition, lines at the top slope downwards, and lines at the bottom slope up. The lines may also be curved, but that is harder to see.

This might be because (a) the printing is skew, (b) the camera wasn't square on to the page (a perspective problem), or (c) the lens has barrel distortion. You can get lens barrel distortion data using Hugin tools, but you need more photographs. I'll cover perspective here.

Page 5 is fairly full of text. We want the boundaries of the text to be rectangular. Using Gimp, and looking at complete lines of text, I can extract coordinates that should match.

top-left x: 1473
bot-left x: 1458

top-right x: 4941
bot-right x: 4977

top-left y: 531
top-right y: 515

bot-left y: 2777
bot-right y: 2755

So I'll distort:

Code: Select all

--From--   ---To---
1473 531   1465 523
4941 515   4959 523
4977 2755  4959 2766
1458 2777  1465 2766

For convenience, I'll put these in a file called dist.txt:

Code: Select all

1473 531   1465 523
4941 515   4959 523
4977 2755  4959 2766
1458 2777  1465 2766

Then we can correct perspective distortion with

Code: Select all

-distort perspective @dist.txt

in our command:

Code: Select all

"%IMG685%convert" ^
  page5.ppm ^
  calibration_white.ppm ^
  -compose Divide_Src -composite ^
  -separate ^
  ( -clone 0 -distort Barrel "%CA_RED%"  ) -swap 0,-1 +delete ^
  ( -clone 2 -distort Barrel "%CA_BLUE%" ) -swap 2,-1 +delete ^
  -combine ^
  -channel G -evaluate Multiply 0.9961 ^
  -channel B -evaluate Multiply 0.9961 ^
  -distort perspective @dist.txt ^
  -level 50%%,90%% ^
  -crop 4286x2556+1020+408 +repage ^
  -rotate 90 ^
  page5_corrected.png

If the distortion is caused by your setup, rather than a printing error, "-distort perspective @dist.txt" should be valid for all pages.

We can now see more clearly that the bottom line is slightly curved, although the top lines aren't. A lens shouldn't create such asymmetrical barrel distortion. What lens are you using? I suspect it isn't a Nikon prime lens.