Page 1 of 2
Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-08-08T02:08:35-07:00
by dalex_257
The Adobe Lightroom 2 and Photoshop CS3 produce PSD and TIFF files (when converting from RAW in PS CS3) that ImageMagick has trouble to get metadata from. By metadata I refer to EXIF, IPTC and XMP information. The identify command in verbose mode shows that both those formats have 8BIM profile which is a binary image resource (as defined by Photoshop format document) and ImageMagick does seem to have some logic parsing those but it is somehow flawed. In files produced by Lightroom and Photoshop CS3 ImageMagick extracts the whole image resource table at once if trying to extract it as 8BIM. Even if the resource table contains other 8BIM resources with thumbnails, embedded ICC profiles and EXIF/XMP metadata - those are not recognised at all.
It would be nice to fix this to allow extraction of the PSD tags as well as embedded metadata from those 8BIM resources. The XMP in PSD files are documented and implemented in a code in Adobe XMP SDK which is released under BSD license (the PSD format reader is quite well commented) so I guess its a matter of implementing it in IM (XMP resource ID is 0x424, EXIF resource ID is 0x422 and IPTC - 0x404 all taken from XMP SDK).
Also by some reason when trying to extract XMP resource/profile from PSD/TIF files with convert command, ImageMagick looks up the APP1 resource which I believe is wrong (outdated?).
Edit: forgot to add that the ExifTool by Phil Harvey does display all the metadata correctly from both of those formats (so I guess the code could be analysed and ported to ImageMagick).
Edit2: this is happening in the most recent released version of ImageMagick 6.4.2-6.
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-08-08T05:05:23-07:00
by magick
Can you post a URL to a couple of PSD and TIFF images that illustrate the problem. We will use them as out test images to verify and fix the bug you reported. Thanks.
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-08-08T06:59:44-07:00
by dalex_257
magick wrote:Can you post a URL to a couple of PSD and TIFF images that illustrate the problem. We will use them as out test images to verify and fix the bug you reported. Thanks.
Here you go - files exported from LR 2 with all defaults but reduced to 8bit per channel and reduced in size to 800x600 (to save the space).
Archive:
http://www.dalex.clara.net/LR2Test.zip
This contains the following files:
LR2Test-1.tif
LR2Test-2.tif
LR2Test-1.psd
LR2Test-2.psd
All of these don't show any EXIF/IPTC/XMP information with identify -verbose. For all of these, extraction of only 8BIM profile can be extracted with convert, no EXIF or XMP. All of those have EXIF (8BIM resource ID 0x422) and XMP (8BIM resource ID 0x424) embedded with the lots of camera/lens metadata and some other Lightroom settings. All commands used from the latest IM version (downloaded yesterday).
Please keep me informed by adding to this thread.
Edit: URL updated - now shared on my personal site without limitations
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-08-09T15:51:45-07:00
by dalex_257
dalex_257 wrote:magick wrote:Can you post a URL to a couple of PSD and TIFF images that illustrate the problem. We will use them as out test images to verify and fix the bug you reported. Thanks.
Archive:
http://www.dalex.clara.net/LR2Test.zip
Did the file helped? Do you know the reason of the problem and is it going to be fixed?
Thanks
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-08-09T16:55:36-07:00
by magick
We are just starting on this problem, we may need a week or two depending on the demands on our time.
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-08-11T00:49:06-07:00
by dalex_257
magick wrote:We are just starting on this problem, we may need a week or two depending on the demands on our time.
Thanks.
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-08-20T11:10:06-07:00
by dalex_257
magick wrote:We are just starting on this problem, we may need a week or two depending on the demands on our time.
Any progress with this problem?
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-08-20T11:31:38-07:00
by magick
Download and try ImageMagick 6.4.3. We extract the embedded profiles from the Photoshop resource block. If you continue to have problems, let us know.
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-08-22T04:25:41-07:00
by dalex_257
magick wrote:Download and try ImageMagick 6.4.3. We extract the embedded profiles from the Photoshop resource block. If you continue to have problems, let us know.
The 6.4.3 seems to be unavailable. The source links on your FTP does not exist for Windows and for UNIX the tar archive is 6.4.2-10.
I've tried a few mirrors but thay all refer to 6.4.2 versions.
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-08-22T06:01:58-07:00
by magick
Grab the latest ImageMagick source from
ftp://ftp.imagemagick.org/pub/ImageMagick. Some mirrors take a week or two before they catch up with the authoritative site.
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-08-27T03:32:30-07:00
by dalex_257
Thanks, it works now for PSD and TIFs. However, there is a nagging issue with JPEGs profuced by Lightroom (or Photoshop ACR). When trying to extract the XMP profile from them (by 'convert aaa.jpg bbb.xmp' command) the generated XMP file is invalid and includes extra bytes in header that do not belong there. Doing so for TIF and PSD produces correct XMP. The example of the beginning of such XMP extracted from JPEG:
Code: Select all
http://ns.adobe.com/xap/1.0/NULL<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 4.2-c020 1.124078, Tue Sep 11 2007 23:21:40 ">
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<rdf:Description rdf:about=""
The first URL and following NULL character do not belong in XMP really. Again for how XMPs are stored in JPEGs, I wil refer you to Adobe XMP SDK sources. I did the sample of such JPEG which can be found in here
http://www.aletan.com/img/v1/p98083919.jpg
EDIT: It seems also that writing XMPs in JPEG does not work well either. I have tried to convert the PSD file exported from Lightroom to JPEG (that should preserve the profiles) and also by manually extracting the XMP from PSD and adding it to JPEG. When both of those JPEGs opened in Lightroom or Photoshop, none has any metadata apart from a few EXIF fields (not all of them).
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-08-27T10:57:08-07:00
by magick
The JPEG XMP profile includes the XMP namespace followed by a NULL followed by the XMP packet. We need to strip the namespace from the profile. Will have a fix in the ImageMagick subversion trunk within a day or two. Thanks.
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-08-29T07:19:05-07:00
by dalex_257
magick wrote:The JPEG XMP profile includes the XMP namespace followed by a NULL followed by the XMP packet. We need to strip the namespace from the profile. Will have a fix in the ImageMagick subversion trunk within a day or two. Thanks.
Great, thanks. I'll monitor the releases for the change.
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-12-09T18:01:33-07:00
by ThomasH
i've got the same problem with 6.4.x, and what troubles me is that it used to work just fine with older imagemagick versions. also, the problem seems to be with TIF files in generall, not necessarily created by lightroom. i'm both seeing profile problems with adobe photoshop CS2 and imagemagick 6.3.6 created files!
when doing an identify, the profiles seem to be read just fine:
Code: Select all
[b]identify -verbose test.tif[/b]
...
Compression: None
Orientation: TopLeft
Properties:
comment: Test
create-date: 2008-12-10T01:20:26+01:00
modify-date: 2008-10-01T13:02:30+02:00
signature: edited
tiff:artist: edited
tiff:make: Canon
tiff:model: Canon EOS 30D
tiff:rows-per-strip: 1
tiff:software: Adobe Photoshop CS2 Windows
tiff:timestamp: 2008:09:22 05:05:45
Profiles:
Profile-8bim: 7046 bytes
Profile-icc: 3144 bytes
IEC 61966-2.1 Default RGB colour space - sRGB
Profile-iptc: 2048 bytes
unknown[2,0]:
Created Date[2,55]: 20080813
Created Time[2,60]: 201607+0100
Country[2,101]: Britain
Country Code[2,100]: GBR
Image Name[2,5]: Test
Keyword[2,25]: Test
...
accessing the profiles fails:
Code: Select all
[b]convert test.tif xmp:test.xmp[/b]
convert: no APP1 data is available `test.xmp' @ meta.c/WriteMETAImage/2318.
[b]convert test.tif xmp:test.iptc[/b]
convert: no IPTC profile available `test.iptc' @ meta.c/WriteMETAImage/2249.
this also fails for files that were previously edited by imagemagick, and where imagemagick was previously able to manipulate the profiles:
Code: Select all
[b]identify -verbose 001.tif[/b]
...
Compression: None
Orientation: TopLeft
Properties:
comment: edited
create-date: 2008-12-10T01:49:15+01:00
modify-date: 2008-05-16T16:09:09+02:00
signature: e4377aa1ac2dbdcee2270ac2c62a1bbed87908d2d1677190e46e34e00e1f85b6
tiff:artist: unknown
tiff:copyright: edited
tiff:document: edited
tiff:make: Canon
tiff:model: Canon EOS 30D
tiff:rows-per-strip: 1
tiff:software: ImageMagick 6.3.6 10/19/07 Q16 http://www.imagemagick.org
tiff:timestamp: 2008:05:13 22:31:54
Profiles:
Profile-8bim: 15168 bytes
Profile-icc: 560 bytes
Adobe RGB (1998)
Profile-iptc: 152 bytes
Profile-iptc: 152 bytes
unknown[2,0]:
Keyword[2,25]: edited1
Keyword[2,25]: edited2
Keyword[2,25]: edited3
Keyword[2,25]: edited4
Keyword[2,25]: edited5
Byline[2,80]: edited
unknown[2,92]: edited
Copyright String[2,116]: edited
Caption[2,120]: edited
Artifacts:
verbose: true
Tainted: False
...
[b]convert 001.tif iptc:test.iptc[/b]
convert: no IPTC profile available `test.iptc' @ meta.c/WriteMETAImage/2249.
[b]convert 001.tif xmp:test.xmp[/b]
convert: no APP1 data is available `test.xmp' @ meta.c/WriteMETAImage/2318.
any help very much appreciated - i need a way to be able to programmatically extract and modify the XMP data quite urgently...
update: using an old 6.3.4 imagemagick installation, i can read, extract and modify the xmp profiles just fine...
Re: Metadata in PSD and TIFF files produced by Lightroom 2 bug
Posted: 2008-12-09T19:04:18-07:00
by magick
We're using ImageMagick 6.4.7-5 and cannot reproduce the problem you posted. We extracted the XMP profile from a number of TIFF images. Post a URL to a couple of your TIFF images that fail and the command you are using. We need to reproduce the problem before we can help.
