need help working with png - 124560px by 28800px

[lois54]

I have to put together the rendered images that make up this huge project. I am using amazon web services to spin up compute machines for this. My question is about what the best environment will be to do this efficiently. Bigger is not always better. on one of the processes I found that convert was only using 10% of the cpu. I am using env MAGICK_TMPDIR=<tmpfile> in big empty partition>" and that keeps it from filling up my root directory. I need to know if I need more cores and do I what do I need to do to be sure that I am using them. Should I be choosing a compute optimized system or looking for better IOPS?

Are there command options that will help? I have looked at the docs but I am afraid there being so much I will miss what I need to know.

The project is a print at ~300dpi that will be 8 feet high by 35 ft long. will be delivering to the print shop 2 files since the image will be put on two different surfaces. Half on a wall and half on a sliding wall/barn door. I have rendered it in 621 pieces PNG 3922x1800 and cropped them to 3824x1765.

So far I have succeeded in using convert <file> -append to put all the images together.

Any help will be appreciated. This is the image https://flic.kr/p/Fr5sgN

Thank you!
Peace, Lois

[magick]

Large images typically exceed the memory resources of the host so instead the pixels are cached to disk. For optimal performance, you want an instance with modest memory and plenty of disk. Want to speed up the conversion? Then use an instance with fast disk. Likely order of performance might be NAS, local SATA, local SAS, and optimal performance, local SSD.

If your image fits into memory (add -debug cache to your command-line to confirm), performance is controlled by the speed of the memory and processor. The faster the memory / processor, the faster the conversion. If you are processing the image other than conversion (.e. resize, sharpen, etc.), multiple processors are desirable since ImageMagick can run these operations in parallel across all your processors.

[snibgo]

So far I have succeeded in using convert <file> -append to put all the images together.

So, what do you now need to do that is causing problems?

Is your image 8 bits/channel/pixel, or 16 or what?

What version of IM are you using? On what platform? What is the Q-number?

(If your image is 8 bits/channel/pixel, Q8 IM will need only half the memory of Q16.)

How much memory does your computer have? Q8 needs 4 bytes per pixel. I expect your append operation needs to hold all the input images and the output in memory at the same time. This is approx 8 G pixels, so Q8 needs 32 GB, and Q16 needs 64 GB.

If you don't have that much memory, it will use disk, which will be very slow. A SSD (solid state disk) may improve performance.

If you don't have enough memory, VIPS may be a more suitable tool for such large images.

[lois54]

Thanks!
The version I am using is 6.7.8-9 Features:OpenMP
Sorry I forgot to include that. I thought of it and then ... Well I am not getting enough sleep...
I am trying to decide which computer essentially. Since I am trying to choose which type of machine to use in the cloud. I have choices of size of memory and maybe I would be best off with the huge 32G or better machine but it needs to run much faster for me because it is much more expensive per hour.
Image is 8bits
I have another one and I need to see the image to crop it to the right dimensions.
The Q is whatever is the default. I installed this with yum.

[snibgo]

The Q is whatever is the default.

What does "convert -version" say? Probably Q16.

Image is 8bits

Then you only need Q8 for appending, which will use 50% less memory.

[lois54]

Thank you! I compiled a new copy of the newest IM setting the Q to 8. (I wish all my compiles went so smoothly.) I appreciate this help very much.

One more question. If I need to do gradient masking on the images should I go back to a Q16 version? I can do that pretty easily if it would be better for masking layers and overlaying them into my bigger image.

Since as I am appending together some of my cropped images I am seeing lines that are due to slight variations in rendering with Apophysis 7X. I am hoping to reduce the visual problem with the gradient as I have done successfully with photoshop on a smaller scale.

[snibgo]

If the image is like a photograph (detail at multiple levels; no flat colours) then general image processing (adjusting colours and tonal levels, compositing etc) should probably be done with Q16. If you use Q8, you might introduce banding.

The general image processing could be done on the small rectangles. When you want to assemble them, use ImageMagick Q8 to reduce memory requirements.