Montage 10,000 images on VPS

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
Denver
Posts: 6
Joined: 2017-04-24T11:09:10-07:00
Authentication code: 1151

Montage 10,000 images on VPS

Post by Denver »

Hi everyone,

I have a directory of 10,000 150x150 png images that I would like to montage into a 100x100 grid using the following code:

Code: Select all

montage -limit memory 2048 -limit map 2048 *.png -geometry +0+0 -tile 100x100 montage.png
Understandably I guess this is too much for my server to handle. I receive the following error:

Code: Select all

montage.im6: UnableToAcquireCacheView `Too many open files' @ fatal/cache-view.c/AcquireAuthenticCacheView/121.
Is there anything I can change to make this work?
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Montage 10,000 images on VPS

Post by snibgo »

What does "-list resource" say? On my computer:

Code: Select all

f:\web\im>%IM%convert -list resource
Resource limits:
  Width: 214.7MP
  Height: 214.7MP
  Area: 8.4474GP
  Memory: 3.9336GiB
  Map: 7.8672GiB
  Disk: unlimited
  File: 1536
  Thread: 8
  Throttle: 0
  Time: unlimited
So I couldn't open 10,000 files in IM. But changing the file limit in policy.xml might fix that.
snibgo's IM pages: im.snibgo.com
Denver
Posts: 6
Joined: 2017-04-24T11:09:10-07:00
Authentication code: 1151

Re: Montage 10,000 images on VPS

Post by Denver »

So I just checked and upped the max files to 10,001 as well as tweaked my query

Code: Select all

File         Area       Memory          Map         Disk    Thread         Time
-------------------------------------------------------------------------------
10001     4.1983GB     1.955GiB      3.91GiB    unlimited         2    unlimited

Code: Select all

montage -limit memory 1900 -limit map 1900 *.png -geometry +0+0 -tile 100x100 montage.png
However, I still get the following

Code: Select all

montage.im6: UnableToAcquireCacheView `Too many open files' @ fatal/cache-view.c/AcquireAuthenticCacheView/121

The resulting image should be 15,000x15,000 – that's obviously big, but not too big to be possible, right?
User avatar
magick
Site Admin
Posts: 11064
Joined: 2003-05-31T11:32:55-07:00

Re: Montage 10,000 images on VPS

Post by magick »

What is your system limit? Try `ulimit -a`. You may need to up the file limit or reduce the ImageMagick file limit to something less than the system limit.
Denver
Posts: 6
Joined: 2017-04-24T11:09:10-07:00
Authentication code: 1151

Re: Montage 10,000 images on VPS

Post by Denver »

Cool just tried this:

Code: Select all

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 15845
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 15845
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
I have done this montage before with 28x28 sized images (rather than the current 150x150) and the total number of opened images wasn't an issue, so not exactly sure how to move forward.
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Montage 10,000 images on VPS

Post by snibgo »

I was mistaken about the file limits. I can montage 10,000 PNG files with no problem. Sorry, I don't know what your problem is.
snibgo's IM pages: im.snibgo.com
Denver
Posts: 6
Joined: 2017-04-24T11:09:10-07:00
Authentication code: 1151

Re: Montage 10,000 images on VPS

Post by Denver »

So I upped my system limit to 10,001 but the same issue is persisting :(
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Montage 10,000 images on VPS

Post by snibgo »

What version of IM are you using? On what platform?
snibgo's IM pages: im.snibgo.com
Denver
Posts: 6
Joined: 2017-04-24T11:09:10-07:00
Authentication code: 1151

Re: Montage 10,000 images on VPS

Post by Denver »

ImageMagick 6.7.7-10 2016-06-01 Q16
and Ubuntu 14.04
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Montage 10,000 images on VPS

Post by snibgo »

That explains why I couldn't find the error message in the source code.

6.7.7 is very old. I suggest you upgrade and try again.
snibgo's IM pages: im.snibgo.com
Denver
Posts: 6
Joined: 2017-04-24T11:09:10-07:00
Authentication code: 1151

Re: Montage 10,000 images on VPS

Post by Denver »

Wow, I didn't realize I wasn't on the current version. Updated and it now works. Thank you!
Post Reply