Adjust exposure based on image metadata

[ashenkin]

Hello all,

As part of a photogrammetry project, I took a few hundred photos of an object using manual settings on a D800, saving images in NEF RAW format. However, I forgot to turn off auto-iso (argh!). So, all my photos have the same aperture and shutter speed, but they differ in their ISO. I would like to correct for this such that the images will all have the same exposure as I originally intended.

I think the adjustment in EV units would go something like this: exposure_adjustment = -log2(ISO/baseISO). This is in units of "EV" (which admittedly i'm not super clear on). So, if we standardize to ISO 1000 for example, I would have to read the ISO used for each image in its metadata, and then apply an EV adjustment of -log2(ISO/1000) for each image. Any thoughts about how to carry this out in imagemagick? I've seen an exposure script, but it adjusts based on percentages, not EV units.

Thanks,
Allie

ImageMagick 7.0.7-4 Q16 x64
Windows 10 Pro (linux possible if necessary)
R scripting environment preferred, but Perl, Python, others possible...

detailed version info:
$ magick.exe convert -version
Version: ImageMagick 7.0.7-4 Q16 x64 2017-09-23 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2015 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Visual C++: 180040629
Features: Cipher DPC Modules OpenMP
Delegates (built-in): bzlib cairo flif freetype jng jp2 jpeg lcms lqr openexr pangocairo png ps rsvg tiff webp xml zlib

[snibgo]

To test whether your camera raw is linear, see my page Linear camera raw. I tested my own Nikon D800 and it is linear, at one ISO setting. I guess your will be too. Strictly, the camera should be tested at other ISO settings as well.

"Linear" means the raw values are proportional to the amount of light. One EV (aka "one stop") is a doubling or halving of light, so the raw values double or halve.

So if you go from say, ASA 100 to ASA 400, this is two stops, so we expect the values to double twice, ie increase by a factor of four.

So the formula for adjusting raw values is simple:

Code: Select all

newRaw = oldRaw * newISO/oldISO

Beware that this might create values greater than 100%, which usually clip to white (just as if you had over-exposed in the camera).

So the IM command could be:

Code: Select all

magick in.tiff -evaluate Multiply %[fx:1000/ActualISO] out.tiff

I stress these are raw values, not sRGB values.

You can get the actualISO from the metadata:

Code: Select all

convert in.tiff -format %[EXIF:ISOSpeedRatings] info:

Sorry, I don't know R or Perl, or IM with Python.

Does that answer the question?

[ashenkin]

Thanks very much, @snibgo! That sounds great, and yes, I'm working with RAW here. Do you think it'd work reading and writing in RAW format (NEF preferred), or do i have to convert to tiff as your command line indicates above?

[snibgo]

[I've edited the above slightly.]

IM can't read the RAW file. It delegates that work to libRaw or dcraw or ufraw or whatever. But IM will tell the delegate to convert to sRGB, so I suggest you use a delegate directly, and ensure you tell it you want the raw values. The delegate (eg dcraw) can make a TIFF file that IM can read.

[ashenkin]

got it - thanks snibgo.

[snibgo]

Thinking futher, maybe dcraw's "-b" option can do what you want directly, eg ...

Code: Select all

%DCRAW% -v -4 -r 1 1 1 1 -g 1 1 -o 0 -b 4 -T -R 1 -O outtiff.tiff in.nef

... might multiply values by 4, ie 2 stops, but I've never used "-b", so test this first.

[ashenkin]

ahh... but that wouldn't allow me to read in the current iso of the photo, correct? i would have to embed that in a larger script... i assumed the magick command above was reading the iso of the current image...

also, what is the "-R" option there? don't see it in dcraw 9.27 anywhere...

[ashenkin]

Actually, i have a D810... forgot... here's an example of the dcraw output i'm getting by running the following:

Code: Select all

$ for i in *.NEF; do dcraw -v -4 -r 1 1 1 1 -g 1 1 -o 0 -T -O tiff/"$i".tiff "$i"; done

Loading Nikon D810 image from DSC_5240.NEF ...
Platform 0: Intel(R) Corporation OpenCL 2.1
        Device 0: Intel(R) HD Graphics 530  <---
Scaling with darkness 600, saturation 16383, and
multipliers 1.000000 1.000000 1.000000 1.000000
AHD interpolation (CL)...
Building histograms...
Writing data to tiff/DSC_5240.NEF.tiff ...

all the tiffs are super dark and green tinted... not sure if that's just windows not knowing what to do with a raw tiff, or if dcraw is doing something i didn't intend... thoughts?

before: https://i.imgur.com/ERB3OXC.jpg
after: https://i.imgur.com/2U48vtX.jpg

[snibgo]

"-R n" is an extra option by me to dcraw (see dcraw and WB), which prevents normalisation of the -r white balance multipliers. It is used to prevent the WB mechanism from causing clipping. In that command, "-R n" is redundant as you have "-r 1 1 1 1", so the channel multipliers are all 1 so they won't be normalised. And that's why the result looks green.

For photogrammetry, you probably want the same white balance applied to all images. If the camera was set to a fixed WB, then you can use dcraw's "-w". Otherwise, you might use dcraw "-i" to find the automatic WB of all the images, than pick a typical one or an average to give to "-r".

The result looks dark because the image is in linear RGB ("raw") colorspace, rather than sRGB, but your viewer doesn't realise this so it doesn't compensate. You can convvert to sRGB with:

Code: Select all

magick in.tiff -set colorspace RGB -colorspace sRGB out.tiff

[ashenkin]

Thanks snibgo, for all your help. I'm going to try to avoid compiling dcraw if possible, as i'm on windows and it's always a pain somehow. If i have to though, i'll try. Or i'll do it on a linux vm.

Ok, so things are looking better. Now, dcraw ends up with a darker looking image, though the colors are correct. The sRGB image is also a bit dark, but better. Not sure what's happening with the brightness... any thoughts there?

dcraw command:

Code: Select all

$ for i in *.NEF; do dcraw -v -4 -r  1.925781 1.000000 1.359375 0.000000 -g 1 1 -o 0 -T -O tiff/"$i".tiff "$i"; done 

magick command for sRGB:

Code: Select all

$ for i in *.tiff; do magick "$i" -set colorspace RGB -colorspace sRGB ../sRGB/"$i"; done

before (NEF): https://i.imgur.com/ERB3OXC.jpg
dcraw output (tiff): https://i.imgur.com/XmVhOCJ.jpg
sRGB (tiff): https://i.imgur.com/r6hCnc7.jpg

Thanks,
Allie

[snibgo]

You have dcraw "-4" option, which switches on "-W", which switches off the auto-brighten operation. Try "-6" instead of "-4". This will brighten the image so that 1% of the pixels are clipped.

Alternatively, leave "-4", and auto-brighten in IM:

Code: Select all

$ for i in *.tiff; do magick "$i" -set colorspace RGB -auto-level -colorspace sRGB ../sRGB/"$i"; done

[ashenkin]

so, i'm not really clear on what brightening actually does (i should learn more about image and raw file formats), but will any of this defeat the original purpose of starting from an original exposure and then compensating based on iso? i'm concerned that i'm somehow not properly bringing over the original exposure information from the .nef to the .tiff...

[snibgo]

Well, yes, for photogrammetry where you have a number of images with supposedly the same exposure (or they are normalised to the same exposure) you don't want any automatic adjustment of individual images. You want to treat them all the same. If an individual image looks dark, that doesn't matter, provided they match as a set. You can brighten the entire set if you want, eg by "-evaluate Multiply 1.1".

But I'm not really sure what you are doing. You've shown an image of a flower. What other images go with it?

[ashenkin]

Sorry for the silence - had to focus on some other things for a while, but now back to this. I'm uploading all the raw DNGs (converted from the original NEF) here: https://drive.google.com/drive/folders/ ... sp=sharing.

So, overall, I'm trying to equalize the exposure across images, given that the camera was auto-adjusting ISO (as I had forgotten to turn that off).

[snibgo]

Those images have been processed by Adobe DNG Converter. I don't know if that has changed pixel values. I'll assume not.

I don't know if the Nikon D810 is linear. I'll assume it is.

You can find the image that was taken with the lowest ISO, and normalise all exposures to that one. For example, DSC_5238 is ISO 1250, and DSC_5226 is ISO 2500. So DSC_5266 received one stop more exposure, so should be darkened by one stop to match 5238. Put it another way, 1250/2500 = 0.5, so we multiply 5266 by 0.5, in linear colorspace. Windows BAT syntax:

Code: Select all

%dcraw% -v -6 -T -W -o 0 -O x.tiff DSC_5238.dng

%IMG7%magick ^
  x.tiff ^
  -set colorspace RGB ^
  -colorspace sRGB ^
  x_5238.tiff

%dcraw% -v -6 -T -W -o 0 -O x.tiff DSC_5226.dng

%IMG7%magick ^
  x.tiff ^
  -set colorspace RGB ^
  -evaluate Multiply 0.5 ^
  -colorspace sRGB ^
  x_5226.tiff

Now the images are as if they were taken at the same ISO setting. You might want to process them further, eg increase saturation or whatever.