Legacy ImageMagick Discussions Archive

Hi there.
I would appreciate some advice/direction:

I would like to transform the color balance from a sensor with "no IR-filter" (Canon 60Da) to a "normal" one, say Canon 5DMark I. I have written up a web page describing the problem and a potential solution, including a couple of images: https://sites.google.com/site/alistarga ... -to-normal

Now of course I can adjust one image in GIMP, but I have several thousands of images, so I would prefer an ImageMagick solution I can script into a loop instead of going via Photoshop/Lightroom.

In brief, I think my approach is: under a range of bright to dark twilight conditions, use TWO cameras to take bracketed exposures of a gray card. Then using snibgo's Gray balance approach (http://im.snibgo.com/colchkcha.htm#graybal) create a library of transforms indexed to the solar depression and exposure value. With that in hand, for any picture past or present, I can push it through the appropriate transform. I know it won't be exact, but it will be closer to what my eye sees than the current excess of reds.

Am I on the right path? Is there something obvious I've missed?

Alister.
P.S. This forum boosts my confidence that humanity will make it! Thanks to all who contribute to the community.

Interesting problem. A general assumption in image processing is that we are dealing with real colours, those inside the CIE horseshoe. In principle, we know the transformation each camera makes from "scene referred" to "camera referred", so we know the inverse of each transformation, so we can transform the output from either camera to look as if it was made by the other. Or we can take short cuts and consider just gray balance (my page you linked) and contrast and saturation and so on.

But the general assumption isn't true for you. One of your cameras captures non-real colours, colours you can't see, beyond the red end of the spectrum. It transforms those colours into visible ones, colours you can see on the computer screen or whatever. I guess that it also "shuffles up" ordinary visible red colours, to make room for the colours that were infrared, but I could be wrong.

I have no experience of IR, so I can only speculate.

Two important factors are: the amount of IR in the illumination, and the IR reflectivity of the object. About half the radiation from the sun is IR. Does this proportion vary by time of day, or moisture in the air? I don't know. What proportion of light from blue sky or cloud is IR? I don't know, but I suspect not much. How much from a sunset? Different artificial lights will have different proportions of IR (eg inefficient incandescent versus modern fluorescent or LED). Green vegetation is a good IR reflector. In addition, all objects above absolute zero emit IR radiation, so cameras sensitive to IR can see people even when there is no light source.

You might get hold of a photographic IR filter, the type that screws to the front of a lens. Or even just a piece of IR gel. (Or improvise with a rectangular container of water!) Then you can take photos of the same scene with the same camera and lens, with and without the IR filter. If you also have an ordinary camera with a sensor IR filter, then take the same photo, and you have three images. You can create a series of tests with different illuminations (midday, before sunset, after sunset, artificial lights), and of different objects (gray card, people, foliage, landscape, skyscape). Test with auto-exposure, but also the same manual exposure for all three photos.

(We have a naming problem: the IR filter at the sensor blocks IR light ("cut-IR"), but the IR filter at the front of the lens blocks everything except IR ("pass-IR").)

These tests will show you what the Canon 60Da does to images in the presence or absence of IR. The photos with the "pass-IR" filter measure how much IR there is. I would expect an obvious tonal shift, and probably a red-shift in the areas where there is most IR present. For example, as shown on your page, the 60Da shows the red sky as lighter and redder, but doesn't change the blue sky.

For a constant subject (eg sunsets with black silhouette foreground) I expect a transformation from one camera to the other might be quite simple. Two stages:

1. Create a mask that is white where you have most IR, black where there is none, and gray in between. This is made from the image, according to redness and brightness. For example: 2. Create a transformation from "ordinary camera" to "IR camera". This might be a red shift and lighter tones, eg "-modulate 120,100,80".

3. Composite with the mask, eg: Sorry, I've rambled a bit.

Thanks for the reply! I concur with your speculations. Fortunately for my current need, I don't need to worry about the complete range of subjects. I too thought about those times when something is visible in IR (Sun setting through forest fire smoke) but not in the normal camera, the transform falls apart in the strict sense - will just have to live with that.

Hopefully I will have time this evening to give your code snippets a try. As for gray card imaging under twilight, looks like my weather won't permit that for a number of days.

Much appreciated,
Alister.

Good stuff.

To expand slightly on this point:

snibgo wrote:2. Create a transformation from "ordinary camera" to "IR camera". This might be a red shift and lighter tones, eg "-modulate 120,100,80".

The transformation can go in the opposite direction, of course, and is likely to be more complex. If you have two photos of the same subject under different conditions, eg with and without a filter, there are many ways of finding a transformation that makes one look like the other. These methods include, roughly increasing in complexity and accuracy:
Colour checker charts: Gray balance
Setting the mean
Gain and bias
Set mean and stddev
Colours to matrix and polynomials
Matching histograms

Making progress! While I wait to borrow a 6x4 color chart from one of my friends, I played with the "forward" transform, that is from my IR-sensitive 60Da to the non-IR "normal" shot. At first I got a little confused with the modulate command until I realized the values were not R,G,B. Then I saw the matrix method on the IM webpage and realized it was much closer to the direction of the colchkcha method.
mask: negate it: transform with matrix: In essence, I am reducing red by giving it 90% R then feeding in it 15% G and 10% B, dropping the green a bit by mixing in some red and blue, and finally reducing the blue a bit. I iterated those matrix numbers around until I got something I liked, posted on the updated page:
https://sites.google.com/site/alistarga ... -to-normal

Next step is to implement the methods and do it properly. I've downloaded the zipped bats and had a look. At first I was hoping that I would not have to go via the CygWin implementation http://im.snibgo.com/customim.htm (I am currently running IM with the Windows env.), but it appears that to run the methods which are in c, then CygWin is required. Given my current schedule/calendar, it will likely be a few days before the configs/makes are working properly. Thanks so much for detailing how to get there!

My process module cols2mat (see Colours to matrix and polynomials) calculates the 6x6 colour matrix that best transforms the first image to the second. For example, taking the first two images on your page: The result noIRtoIR.jpg is:

The text output is: Adding line-breaks to the matrix for legibility: The last three lines are constant. The final column is an addition to each channel, and we see they are calculated to be quite small. The top-left 3x3 sub-matrix can be directly compared to the numbers you found manually.

By default, cols2mat find the best matrix to transform the entire image. If you have a mask of which parts of the image are affected by IR, you can use that to make the rest of the image transparent, and use the "weightAlpha" option to get a matrix optimized for just the IR parts.


My process modules are written in C. To use them, you must be able to compile, link and install IM from source. I do that with Cygwin tools on Windows, documented on my pages.

Looks even better, as I would expect! And gives me the confidence that the effort I put into copying your CygWin implementation will really pay off.

I'm really looking forward to getting it going!

Thanks so much.

WOOHOO! Holy smokes, thank you so much for all your help snibgo. It was a bit of a steeple chase fixing the compile errors and the changes in MagickCore function calls from V6.9x to V7.0x, but I am so grateful for your guidance, and happy with myself at not having given up in the face of "yet another compiler error".

Here is a result fresh oout of the computational oven, using images from the other night, where I ran 2 cameras, side by side with the identical exposure settings, 60Da, 5DMark II, transformed 60Da image:
https://sites.google.com/site/alistarga ... l/Trio.jpg

Hot damn!

The images are out of focus because I was switching the pair of cameras from shooting the twilight sky to shooting a nearby Xrite ColorChecker passport. I intend to use those to calibrate different exposures and solar depression angles, so that I can take any 60Da image past or future and transform it to something pretty close to a normal camera's colour balance. In the meantime, a single pair produces a transform matrix (using cols2mat method Cross above) is a fantastic start.

I intend to post updates to my web page https://sites.google.com/site/alistarga ... -to-normal when I finish the calibration process. The colour matrix here is: c2matrix=0.53761,0.237494,0.0655367,0,0,-0.0618967,0.0230337,0.522579,0.442921,0,0,-0.0514341,-0.197834,0.169792,1.01265,0,0,-0.0241132,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1

I'm really pumped that this all worked, with much appreciation to snibgo!
Alister.

That's a good result, Alister. Thanks for sticking with it. With your encouragement, I've updated my modules to build under v7.0.7-28. They all build and run, and most of them run correctly. When I've fixed those that don't, I'll update my web pages.