How to handle tifs over 2 GB?

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
GiantCowFilms
Posts: 17
Joined: 2017-02-28T22:08:29-07:00
Authentication code: 1151

How to handle tifs over 2 GB?

Post by GiantCowFilms »

I'm having trouble running this command on some large tifss. It appears to work on the tiffs as long as they are under 2 GB as soon as they go over it stops working.

Code: Select all

magick "C:\Path\To\Huge.tiff" -page -12+2 -background black -flatten "C:\Path\To\Huge.tiff"
Running it on the huge tiff file gives the following error:

Code: Select all

magick: Integer overflow. `TIFFStripSize' @ error/tiff.c/TIFFErrors/572.
Also noteworthy is the fact that I seem to have this problem with morgify as well but not the convert command.

I read somewhere that it might be possible to fix this using bigTiff. If that is the case, how do I compile image magick with big tiff on windows? I have already completed a standard compile using VS.
Last edited by GiantCowFilms on 2017-03-08T11:44:40-07:00, edited 1 time in total.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: How to handle tifs over 2 GB?

Post by fmw42 »

-page -12 2
That does not seem to be a proper value for -page. See http://www.imagemagick.org/script/comma ... s.php#page
GiantCowFilms
Posts: 17
Joined: 2017-02-28T22:08:29-07:00
Authentication code: 1151

Re: How to handle tifs over 2 GB?

Post by GiantCowFilms »

fmw42 wrote: 2017-03-07T19:48:54-07:00
-page -12 2
That does not seem to be a proper value for -page. See http://www.imagemagick.org/script/comma ... s.php#page
Yeah, that wasn't my actual value (which is generated by some code that runs the command), so I tried to give an example of a valid one from memory, but clearly that didn't work. I can assure you the actual value is valid.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: How to handle tifs over 2 GB?

Post by fmw42 »

Large tiffs may need more memory than your available RAM and thus thrash to disk, which will be incredibly slow. See
http://www.imagemagick.org/Usage/files/#massive or perhaps you need to install BigTIff delegate library. See http://www.awaresystems.be/imaging/tiff/bigtiff.html and http://bigtiff.org/
Johndeer
Posts: 5
Joined: 2017-02-26T12:09:48-07:00
Authentication code: 1151

Re: How to handle tifs over 2 GB?

Post by Johndeer »

GiantCowFilms wrote: 2017-03-07T18:41:53-07:00 I'm having trouble running this command on some large tifss. It appears to work on the tiffs as long as they are under 2 GB as soon as they go over it stops working.

Code: Select all

magick "C:\Path\To\Huge.tiff" -page -12 2 -background black -flatten "C:\Path\To\Huge.tiff"
Running it on the huge tiff file gives the following error:

Code: Select all

magick: Integer overflow. `TIFFStripSize' @ error/tiff.c/TIFFErrors/572.
Also noteworthy is the fact that I seem to have this problem with morgify as well but not the convert command.

I read somewhere that it might be possible to fix this using bigTiff. If that is the case, how do I compile image magick with big tiff on windows? I have already completed a standard compile using VS.
I never managed to make any version of IM work with really big TIff pictures. The TIFF format uses 32bit offsets, and as such, it is limited to 4 gigabytes. Many implementations handle these offsets using signed integers, and thus support files of up to 2 gigabytes, but the only real limit resulting from the format specification is 4 gigabytes. I'm pretty sure the IM integrer (IM uses libtiff iirc) is somehow limited to 2GB. I ended up converting my TIFF files > 2GB with Irfanview into uncompressed PNGs. The PNG integrer is so capacious that you're never gonna reach any kind of limits.
User avatar
glennrp
Posts: 1147
Joined: 2006-04-01T08:16:32-07:00
Location: Maryland 39.26.30N 76.16.01W

Re: How to handle tifs over 2 GB?

Post by glennrp »

PNG is also limited to 2Gx2G
Johndeer
Posts: 5
Joined: 2017-02-26T12:09:48-07:00
Authentication code: 1151

Re: How to handle tifs over 2 GB?

Post by Johndeer »

glennrp wrote: 2017-03-08T17:21:31-07:00 PNG is also limited to 2Gx2G
Yes, but I think the file size and the file dimensions have been kinda mixed up here. In theory, the PNG integrer allows files as large as 4.61 Exa-pixels (!) (the 2Gx2G dimension you mentioned) but I'm not sure if there's a proper limit for the file size itself.
It seems IM fails to handle TIFF files over 2GB (the file size, whatever the dimensions of the picture), which was the point of this topic. It's probably related to the libtiff library. On the other hand, IM handles huge PNG files without breaking a sweat (as long as there's enough RAM available of course). I tried a couple of v6 and v7 versions on different computers and it seems to always be the case
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: How to handle tifs over 2 GB?

Post by fmw42 »

What version of libtiff? Have you tried BigTIff
GiantCowFilms
Posts: 17
Joined: 2017-02-28T22:08:29-07:00
Authentication code: 1151

Re: How to handle tifs over 2 GB?

Post by GiantCowFilms »

fmw42 wrote: 2017-03-09T09:52:40-07:00 What version of libtiff? Have you tried BigTIff
I would like to try BigTiff, but have no idea how to build Imagick with it.

I'm not sure which version. The one that is bundled with the source when downloaded. I'm using Imagick 7.0.5.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: How to handle tifs over 2 GB?

Post by fmw42 »

You have to add the libtiff delegate to Imagemagick, not Imagick. If it was a bundle, then you probably cannot change it. You would have to rebuild Imagemagick from source and include all the current delegates plus add bigtiff delegate.
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: How to handle tifs over 2 GB?

Post by fmw42 »

It may be a limitation of the dimensions or filesize in your Imagemagick policy.xml file. You can edit that file.
mideal
Posts: 3
Joined: 2017-03-13T06:07:11-07:00
Authentication code: 1151

Re: How to handle tifs over 2 GB?

Post by mideal »

ahem...
Which OS are you running?
Which filesystem is used? FAT32 has a file size limit of 2GB!?
User avatar
dlemstra
Posts: 1570
Joined: 2013-05-04T15:28:54-07:00
Authentication code: 6789
Contact:

Re: How to handle tifs over 2 GB?

Post by dlemstra »

Are you using the 32 or 64 bit version?
.NET + ImageMagick = Magick.NET https://github.com/dlemstra/Magick.NET, @MagickNET, Donate
Post Reply