Legacy ImageMagick Discussions Archive

Hi there all,
I've been using Magick++ for a system which handles compositing image layers. The source images sent to Magick++ are in EXR format and contain 16bit linear colorspace data (these are coming from Unity and are HDR images) I'm currently facing two issues:

1 -The output PSD I create contains these layers correctly but it seems the data is clamped/incorrect. See the example pixel value in the source EXR and PSD file Magick++ outputs. I understand Photoshop handles 16bit a little strangely.

Original source EXR pixel value:

Magick++ output PSD pixel value:

Is this data clamped in the output file because I have built the 16 bit version of Magick++, and would it be expected to correctly hold the original pixel values if this was rebuilt with Magick++ set to 32bit quantum depth? Currently I have to switch mode to 32 bit in Photoshop when opening the output file in order to start making adjustments to layers, but my EXR's are captured using a half-precision 16bit per channel format.

2 -The file size of the output PSD's is larger than expected for the number of layers. Does anyone know of any further parameters/config to ease this? I'll put a snippet of my output function I've written below.

Code: Select all

bool merge_images_layered_psd(string output_path)
{
	bool completed = false;

	try
	{
		list<Image> layersToFlatten;

		for (pair<string,string> imageInfo : images_to_layer)
		{
			string extension = PathFindExtension(imageInfo.first.c_str());

			Image image;

			try 
			{
				image.read(imageInfo.first);
			}
			catch (Magick::ErrorConfigure &error)
			{
				cerr << error.what() << endl;
			}

			image.label(imageInfo.second);
			if (extension == ".exr")
			{
				image.colorSpaceType(ColorspaceType::RGBColorspace);
				image.type(ImageType::TrueColorAlphaType);
			}

			image.colorSpaceType(ColorspaceType::RGBColorspace);
			image.alphaChannel(AlphaChannelOption::SetAlphaChannel);
			image.attribute("alpha", "Activate");
			image.attribute("colorspace:auto-grayscale", "false");
			
			layersToFlatten.push_back(image);
		}

		Image flattened;
		flattenImages(&flattened, layersToFlatten.begin(), layersToFlatten.end());

		flattened.label("CompositeLayer");
		flattened.colorSpaceType(ColorspaceType::RGBColorspace);
		flattened.alphaChannel(AlphaChannelOption::SetAlphaChannel);
		flattened.attribute("alpha", "Activate");
		flattened.attribute("colorspace:auto-grayscale", "false");


		list<Image> layersForPsd;
		layersForPsd.push_back(flattened);

		for (auto img : layersToFlatten)
		{
			layersForPsd.push_back(img);
		}

		writeImages(layersForPsd.begin(), layersForPsd.end(), output_path, true);
		completed = true;
	}
	catch (exception &error_)
	{
		cout << "Caught exception: " << error_.what() << endl;
		return 1;
	}

	images_to_layer.clear();
	return completed;
}

For reference, an output PSD containing 3 layers at 1920x1080 shows up at around 63.5Mb, resaving the file in Photoshop reduces the file size by almost half. When I resave these files in 32bit mode (as my adjustments need to be done in 32bit to function correctly with HDR data it seems) they are usually around the same size as the 16 bit output file Magick++ produced.

Thanks to anyone who reads this and can offer advice, really appreciate it as I've been working on this for a while and this roadblock is one of the final hurdles!

I suspect your input EXR contains data that is over 100% of quantum. I suppose your IM is built without HDRI, so it can only process positive integers, so it clamps values over 100% to exactly 100%.

The file size in bytes depends on the compression.

Hey, thanks for the response -

Do you know if there's a way to define compression for the output photoshop file? currently I just feed the output path string which ends with ".psd" to writeImages() to get my output file as in the example code in my post, but if you mean I can specify a compression for the PSD to use (I guess that whatever it saves it out using by default in Photoshop will work since that doesn't reduce my quality anyway) that'd be awesome!

Hmm I was pretty sure my project was setup with HDRI ticked in the wizard for building, but I'll take a look and try rebuild (I could try test writing some fixed negatives to the input EXR from a shader too for a quick test). ran into some problems and took a while to get it building the first time

Thanks a bunch!

At the command line, we can use "-compress Zip" and so on, and we get different sizes for the output PSD. I don't use Photoshop, and don't know if it can read all these, or whether IM's compression is better or worse than PS's compression.

Testing for HDRI is trivial, for example write a program that does the equivalent to this:

Code: Select all

magick xc:gray(200%) txt:

# ImageMagick pixel enumeration: 1,1,65535,gray
0,0: (131070)  #FFFFFFFFFFFF  gray(200%)

My version of magick hasn't capped the value at 100%, so it is clearly built with HDRI.

convert -version on IM 6 or magick -version on IM 7 will tell you if you have HDRI compile.

Thanks for both of these replies - I've added to the psd creatyion function to specify compressType on each of the layers. Seems ZIP does not work for PSD files for now but RLE compression made a nice impact on the output file size so I'm pleased to see that

I've checked the verison of IM I have built to ensure HDRI is enabled, and have rebuilt with Q32 to be sure I can support 32 bit PSD files. I'm still seeing my output file open in photoshop as 16 bit and so is losing some data. I am using Magick++ included as part of a dll which I create for my project, but have also checke dagainst magick.exe using the command line from my VisualMagick local build. My EXRs provided can be either fixed as either 16 bit half precision or full float, I have tried both with no difference. I noticed in command line the EXR files are detected my magick.exe as 32-bit and RGB, but the output file is still 16-bit. Below is the example test from command line.

Can anyone suggest why my output PSD is still not 32 bit? As I think this is where it clamps and loses the HDR data.

An excerpt of my current code for writing the PSD file is below

Code: Select all

		list<Image> layersToFlatten;

		for (pair<string,string> imageInfo : images_to_layer)
		{
			string extension = PathFindExtension(imageInfo.first.c_str());

			Image image;

			try 
			{
				image.read(imageInfo.first);
			}
			catch (Magick::ErrorConfigure &error)
			{
				cerr << error.what() << endl;
			}

			image.label(imageInfo.second);
			if (extension == ".exr")
			{
				image.type(ImageType::TrueColorAlphaType);
			}

			image.colorSpaceType(ColorspaceType::RGBColorspace);
			image.alphaChannel(AlphaChannelOption::SetAlphaChannel);
			image.attribute("alpha", "Activate");
			image.attribute("colorspace:auto-grayscale", "false");
			image.compressType(CompressionType::RLECompression);
			image.depth(32);
			
			layersToFlatten.push_back(image);
		}

		Image flattened;
		flattenImages(&flattened, layersToFlatten.begin(), layersToFlatten.end());

		flattened.label("CompositeLayer");
		flattened.colorSpaceType(ColorspaceType::RGBColorspace);
		flattened.alphaChannel(AlphaChannelOption::SetAlphaChannel);
		flattened.attribute("alpha", "Activate");
		flattened.attribute("colorspace:auto-grayscale", "false");
		flattened.compressType(CompressionType::RLECompression);
		flattened.depth(32);

		list<Image> layersForPsd;
		layersForPsd.push_back(flattened);

		for (auto img : layersToFlatten)
		{
			layersForPsd.push_back(img);
		}

		writeImages(layersForPsd.begin(), layersForPsd.end(), output_path, true);

I took a little look deeper in the PSD writing code in IM

Forgive me if I'm misreading this and please feel free to correct but if any IM devs are present, this looks like PSD writing with IM is only capable of writing 8-bit or 16-bit depth files, is this the case?

I think you are reading that correctly. If you are feeling adventurous, you might remove the lines ...

Code: Select all

  if (next_image->depth > 8)
    next_image->depth=16;

... and rebuild IM and see what happens when you try to write 32-bit/channel/pixel PSD files.

Yep - there's a reasonable amount more work to be done to get it working than that but trying to add 32 bit support now

For anyone interested I managed to add this functionality.

A couple of changes to write correct bit depth info in the header (as well as the overrides above being commented out) was enough to get the channel data to be correctly written into the file, but the issue from there on was an undocumented data block of 112 bytes stored in the color mode data section of the PSD file format. It isn't described in the file spec online, which only specifies this data section as used for duotone or indexed color mode images (with a fixed size of 768 bytes for this.)

For now copying this data block from a functional 32 bit PSD file has been enough to get IM writing 32 bit HDR compliant PSD files, and I've been able to combine EXR's/TIFFs/HDR files as the layers from here. The 112 byte data block did not change when testing against a large sample of different configuration 32 bit PSD files but would still be nice to have more meaningful documentation that describes this section

I have noticed (this morning) the same issue with converting EXR images to PSD and PSB.
Any color data in any channel above 1.0 is clamped to 1.0.
I am converting from half-float 16 bit EXR files. The resulting PSD (or PSB) image is 16 bit with an unexpected 8bit layer error, and clamped to 1.0.

Has there been any further progress with adding 32 PSD and PSB files to IM at this point?

Cheers,
- Jimbo

@Jimbo, What is your version of ImageMagick and on what platform. To save to higher bit-depth you will need HDRI to go outside the quantum range, I believe.

@fmw42, as you stated in the other thread "Change DPI while keeping resolution,"
https://www.imagemagick.org/discourse-s ... 23#p168523
my version of ImageMagick is Q16 HDRI. I'm on a Mac OS X v10.14.5 (Mojave).

I have not tried Q32 because I'm not sure how to add that to what I am doing with Homebrew. I will look into that.

But the Q16 HDRI does support the HDRI values outside of normalized imagery (above 1 and below 0). And as you can see by Weary Wanderer's statement above from August 8, 2019, He managed to cobble together a 32bit PSD from editing the code. I have sent him a message, but have yet to hear back.

So, here are the other issues I've found.
- I believe there are three types of images that support HDRI work in IM that can be read by Photoshop. TIFF (which has a 40k pixel dimension limit), PFM and EXR. Neither EXR nor PFM have dpi in their specifications and therefore can't be used for very large imagery (over 1,000 inches) as a PSB in Photoshop.

- Currently, the PSD and PSB files created by IM, which both have dpi specs available, are limited to a 16bit per channel output and clamp color to a normalized quantum (0 to 1). Not ideal for photography or CGI.

Without PSD/PSB output being fixed to accept 32bits per channel, I need to see what other options are out there besides IM.

Thanks.
Jimbo

Jimbo wrote:.. TIFF (which has a 40k pixel dimension limit), ...

Does it? IM can happily write and read 1x100000 and 100000x1 32-bit HDRI pixel TIFFs.

Interesting. I wonder if anything else can read them. I will try and report.
Also, snibgo, what version of IM are you using?

Update: started working on this, and the first tiff limit I found was mentioned by Photoshop. tiff file format has a 4 Gbyte limit.
More to come...
- J

Legacy ImageMagick Discussions Archive

RE: Combining HDR/EXR files into a single PSD

RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD

Re: RE: Combining HDR/EXR files into a single PSD