10bit float log dpx to 16bit float lin exr

[danielHeen]

Hello!

I'm trying to convert a 10bit float log dpx to linear 16bit float exr. I have version 6.6.0-8 with exr and hdri enabled. It seems that if I keep the log colorspace the image maintains floating-point and looks the same, but if I convert to rgb(linear) it clips colors at 1.0

Code: Select all

convert input.dpx -depth 16 -define quantum:format=floating-point output.exr

results in a floating-point log exr.

Code: Select all

convert input.dpx -depth 16 -define quantum:format=floating-point -colorspace rgb output.exr

results in clipped image

Any help would be appreciated,
-Daniel

[fmw42]

IM only supports "half" format EXR (i.e. 16bit)

You can save to TIFF or PFM if you want to keep the fractional or negative values. see NetPBM for PFM

[danielHeen]

Hey!

Thanks for the reply.
Isn't half 16 also supposed to be float?
What I find strange is that if I leave the colorspace to log, it does what I ask, but if I convert to rgb it acts funny and clips the image.

-Daniel

[magick]

We'll need to investigate where the clipping occurs. We'll get back with you in a day or two.

[danielHeen]

Thanks!

I also have a problem with srgb colorspace. Will post in separate thread.

-Daniel

[fmw42]

Hey!

Thanks for the reply.
Isn't half 16 also supposed to be float?
What I find strange is that if I leave the colorspace to log, it does what I ask, but if I convert to rgb it acts funny and clips the image.

-Daniel

Yes, 16-bit float is the half format. So I guess it should work fine. I have not had much use for EXR with respect the FFT in HDRI mode, I think due to the log format. So I just used MIFF, PFM or TIFF.

Sorry I don't know why converting colorspace would cause a problem. It may be an issue/oversight with IM -colorspace in HDRI

[magick]

Let's eliminate EXR for a moment. I need you to define "it acts funny and clips the image." Type

• convert input.dpx input.pfm
  identify -verbose input.pfm
  convert input.dpx -colorspace rgb input.pfm
  identify -verbose input.pfm

Given the statistical distribution, what exactly is unexpected?

[danielHeen]

Hi!

What I meant by acting funny is only that it clips the image in RGB while it doesn't when maintaining Log.

I tried converting to pfm file and it looks as it should but I'm not sure if it's a float. Also did with -colorspace rgb and the result was identical. Did a diff on the results.

here is the output from identify:

Code: Select all

Image: output.pfm
  Format: PNM (Portable anymap)
  Class: DirectClass
  Geometry: 2048x778+0+0
  Resolution: 72x72
  Print size: 28.4444x10.8056
  Units: Undefined
  Type: TrueColor
  Base type: TrueColor
  Endianess: MSB
  Colorspace: RGB
  Depth: 32/16-bit
  Channel depth:
    red: 16-bit
    green: 16-bit
    blue: 16-bit
  Channel statistics:
    Red:
      min: 0.5 (0)
      max: 65535.5 (1)
      mean: 8101.6 (0.123622)
      standard deviation: 15062 (0.229831)
      kurtosis: 8.97854
      skewness: 3.14801
    Green:
      min: 0.5 (0)
      max: 65535.5 (1)
      mean: 10350.3 (0.157935)
      standard deviation: 16388.2 (0.250067)
      kurtosis: 5.4406
      skewness: 2.48443
    Blue:
      min: 0.5 (0)
      max: 65535.5 (1)
      mean: 10130.8 (0.154586)
      standard deviation: 17503.5 (0.267086)
      kurtosis: 4.22932
      skewness: 2.28789
  Image statistics:
    Overall:
      min: 0.5 (0)
      max: 65535.5 (1)
      mean: 7145.67 (0.109036)
      standard deviation: 14773 (0.225421)
      kurtosis: 8.90801
      skewness: 3.07319
  Rendering intent: Undefined
  Interlace: None
  Background color: white
  Border color: rgb(223,223,223)
  Matte color: grey74
  Transparent color: black
  Compose: Over
  Page geometry: 2048x778+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: Undefined
  Orientation: Undefined
  Properties:
    date:create: 2010-03-26T14:49:32+01:00
    date:modify: 2010-03-26T14:49:32+01:00
    signature: 80b2c6d91adf4c1b67b030965507821996adb2ef5ddcfa4fe44c285277e9dd4b
  Artifacts:
    verbose: true
  Tainted: False
  Filesize: 19.12MB
  Number pixels: 1.593M
  Pixels per second: 17.7M
  User time: 0.100u
  Elapsed time: 0:01.089
  Version: ImageMagick 6.6.0-8 2010-03-26 Q16 http://www.imagemagick.org

Thanks,
-Daniel

[magick]

We need to reproduce the problem. Post a URL to an image and the command you use to convert it. Point to a specific pixel or pixels that are clipped when you think they should have values either less than 0 or greater than the quantum range (e.g. 255 for Q8).

[danielHeen]

Hello!

Here is a link to an example: http://www.stormstudios.no/page/imagemagick/dpx2exr.zip
*input.dpx is a log file.
*nuke_lin.exr is an example of what I want to achieve.
*output.exr is what I got.

Area of interest are the trees next to the man in frame.
In nuke after log/lin conversion of input-dpx you can adjust exposure up and down still maintaining details in the trees. After conversion with IM there is no more data in the "clipped" area.
As you can see in nuke_lin.exr nuke preserves the data after export.

commands tried:

Code: Select all

convert input.dpx -colorspace rgb -depth 16 -define quantum:format=floating-point output.exr

and:

Code: Select all

convert input.dpx -colorspace rgb -define quantum:format=floating-point output.exr

and:

Code: Select all

convert input.dpx -colorspace rgb -depth 16 output.exr

and:

Code: Select all

convert input.dpx -colorspace rgb output.exr

Tanks,
-Daniel

[magick]

Set the gamma before you convert to RGB:

• convert input.dpx -set gamma 0.59 -colorspace rgb output.exr

The result has a normalized RMSE less than 0.0014 when compared to Nuke.

[danielHeen]

Yes, they visually look the same, but if you tweak the exposure the trees are clipped and have no detail. I did an "identify -verbose" on the two and there is a difference.
As you can see, the IM one isn't float.

nuke one:

Image: nuke_lin.exr
Format: EXR (High Dynamic-range (HDR))
Class: DirectClass
Geometry: 1920x1080+0+0
Resolution: 72x72
Print size: 26.6667x15
Units: Undefined
Type: TrueColorMatte
Endianess: Undefined
Colorspace: RGB
Depth: 16-bit
Channel depth:
red: 16-bit
green: 16-bit
blue: 16-bit
alpha: 1-bit
Channel statistics:
Red:
min: -72.3114 (-0.00111103)
max: 227197 (3.4668)
mean: 29969.2 (0.457301)
standard deviation: 34071.6 (0.519899)
kurtosis: 2.5449
skewness: 1.51379
Green:
min: -21.0622 (-0.000329018)
max: 232573 (3.54883)
mean: 39078 (0.596291)
standard deviation: 43847.3 (0.669068)
kurtosis: 2.02638
skewness: 1.40877
Blue:
min: -167.997 (-0.00257111)
max: 227197 (3.4668)
mean: 60044 (0.916212)
standard deviation: 57299.4 (0.874332)
kurtosis: -0.136489
skewness: 0.750582
Alpha:
min: 0.5 (0)
max: 0.5 (0)
mean: 0 (0)
standard deviation: 0 (0)
kurtosis: 0
skewness: 0
Image statistics:
Overall:
min: -167.997 (-0.00257111)
max: 232573 (3.54883)
mean: 32272.8 (0.492451)
standard deviation: 45362.6 (0.692189)
kurtosis: 2.47488
skewness: 1.64097
Rendering intent: Undefined
Interlace: None
Background color: white
Border color: rgba(223,223,223,1)
Matte color: grey74
Transparent color: none
Compose: Over
Page geometry: 1920x1080+0+0
Dispose: Undefined
Iterations: 0
Compression: Undefined
Orientation: Undefined
Properties:
date:create: 2010-03-29T13:51:46+02:00
date:modify: 2010-03-29T13:51:46+02:00
signature: e286098d394617cbbf0066dca822bcde550479d31b4e1f93257e090afab13895
Artifacts:
verbose: true
Tainted: False
Filesize: 8.449MB
Number pixels: 2.074M
Pixels per second: 7.68M
User time: 0.200u
Elapsed time: 0:01.270
Version: ImageMagick 6.6.0-8 2010-03-26 Q16 http://www.imagemagick.org

IM one:

Image: output.exr
Format: EXR (High Dynamic-range (HDR))
Class: DirectClass
Geometry: 1920x1080+0+0
Resolution: 72x72
Print size: 26.6667x15
Units: Undefined
Type: TrueColorMatte
Endianess: Undefined
Colorspace: RGB
Depth: 16-bit
Channel depth:
red: 16-bit
green: 16-bit
blue: 16-bit
alpha: 1-bit
Channel statistics:
Red:
min: 0.5 (0)
max: 65535.5 (1)
mean: 25650.5 (0.391401)
standard deviation: 24191.7 (0.369142)
kurtosis: -1.30987
skewness: 0.419172
Green:
min: 0.5 (0)
max: 65535.5 (1)
mean: 30304.8 (0.462422)
standard deviation: 26950.5 (0.411238)
kurtosis: -1.66383
skewness: 0.161841
Blue:
min: 0.5 (0)
max: 65535.5 (1)
mean: 38167.5 (0.582399)
standard deviation: 28832.4 (0.439955)
kurtosis: -1.77315
skewness: -0.276085
Alpha:
min: 0.5 (0)
max: 0.5 (0)
mean: 0 (0)
standard deviation: 0 (0)
kurtosis: 0
skewness: 0
Image statistics:
Overall:
min: 0.5 (0)
max: 65535.5 (1)
mean: 23530.7 (0.359055)
standard deviation: 27208.4 (0.415174)
kurtosis: -1.3946
skewness: 0.594497
Rendering intent: Undefined
Interlace: None
Background color: white
Border color: rgba(223,223,223,1)
Matte color: grey74
Transparent color: none
Compose: Over
Page geometry: 1920x1080+0+0
Dispose: Undefined
Iterations: 0
Compression: Undefined
Orientation: Undefined
Properties:
date:create: 2010-03-30T08:46:21+02:00
date:modify: 2010-03-30T08:46:21+02:00
signature: d629347715c9472a8b6673388b43b0636fd36e1cbb696ce2c9a94e1aad3001ce
Artifacts:
verbose: true
Tainted: False
Filesize: 16.61MB
Number pixels: 2.074M
Pixels per second: 20.74M
User time: 0.090u
Elapsed time: 0:01.100
Version: ImageMagick 6.6.0-8 2010-03-26 Q16 http://www.imagemagick.org

Thanks again,
-Daniel

[magick]

The Log to RGB color conversion is based on DPX specification and in code it looks like this:

• logmap=ScaleMapToQuantum((MagickRealType) (MaxMap*(reference_white+
  log10(black+((MagickRealType) i/MaxMap)*(1.0-black))/((gamma/density)*
  0.002/film_gamma))/1024.0));

If we put debug statements in we find that logmap ranges from 0 to 65535 which means any transformation stays in the 0 .. 65535 range.

Linear to RGB has 4 settings: gamma, film-gamma, reference-black, and reference-white. However, these settings still keep the mappings in the 0 .. 65535 range. Without more knowledge about how Nuke is transforming from Linear to RGB, we can't make any further progress.

Your posting did alert us that there is an erroneous 0.5 in the identify program for HDRI (e.g. red min is 0.0 not 0.5). We get a patch for that problem sometime tomorrow.

[danielHeen]

Hi!

I am no C++ programmer my self, but I can provide you with some source code that might help.

Just to be safe and prevent a potential lawsuit I can give you a link so you can download nuke trial and look at the example code and headers.
If you go into the archive you don't need to register. Just use the password provided by The Foundry on the page.

http://www.thefoundry.co.uk/pkg_downloa ... 98B0171453

Once downloaded and extracted, the files you are looking for are in: Nuke6.0v2/include/DDImage(headers) and Nuke6.0v2/Documentation/NDK/examples(cpp code, readers and writers)

and here is the source for djv_view/convert which unfortunately doesn't work either, but perhaps you can get an idea based on other code?: http://sourceforge.net/projects/djv/fil ... z/download

Thanks,
-Daniel

[danielHeen]

Hello,

Anyone got a chance to look at this?

-Daniel