Rotate Images in a Parallel For (ImageMagick.MagickResourceLimitErrorException)

[pablobhz]

Hello Everyone.
I"ve been trying to rotate 5 images simultaneoulsy, using a Parallel.For

However, i'm always having a Magick.OutOfResource (something like it) exception.
Is there anything i can do to change the max memory MagickNET can allocate ?

This is the code that shows the problem:

Code: Select all

Parallel.For(0, ImagesToRotate.Count, i =>
             {
                 MagickImage test = new MagickImage(ImagesToRotate.ElementAt(i).Value);
                 var ImgRotated = new ImageProcessing.RotateImage(test, RotationIndex).BytesFromRotatedImage;
                 //Inside RotateImage a MagickImage object is created
                 RotatedStuff.Add(ImagesToRotate.ElementAt(i).Key, ImgRotated);                 
                 //Images.Add(ImgRotated.rotatedImage);
             });

And this is the ImageRotate Class:

Code: Select all

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using ImageMagick;

namespace DI_Plot_2018.ImageProcessing
{
    class RotateImage
    {
        public MagickImage rotatedImage { get; private set; }
        public byte[] BytesFromRotatedImage { get; private set; }
        public RotateImage(byte[] BytesToRotate, int RotationIndex)
        {
            using (MagickImage img = new MagickImage(BytesToRotate))
            {
                switch (RotationIndex)
                {

                    case 0:
                        {
                            rotatedImage = img;
                            break;
                        }
                    case 1:
                        {
                            img.Rotate(90);                            
                            break;
                        }
                    case 2:
                        {
                            img.Rotate(180);                            
                            break;
                        }
                    case 3:
                        {
                            img.Rotate(270);                            
                            break;
                        }
                    case 4:
                        {
                            img.Flip();
                            img.Rotate(90);                            
                            break;

                        }
                    case 5:
                        {
                            img.Flip();
                            img.Rotate(180);                            
                            break;
                        }
                    case 6:
                        {
                            img.Flip();
                            img.Rotate(270);                            
                            break;
                        }
                    case 7:
                        {                            
                            RotationIndex = 0;
                            break;
                        }
                }
                rotatedImage = img;
            }                           
        }
    }
}

The images are 5 20MB tiff images.

Thanks in advance for any help.

EDIT3:
Tried to set:

Code: Select all

          ResourceLimits.Memory = 2147483648; //2GB
            ResourceLimits.Thread = 20;
            

No results - same problem.
EDIT2: Tried to do the same thing using a regular loop and had the same error.
EDIT: Exception Details
ImageMagick.MagickResourceLimitErrorException
HResult=0x80131500
Message=memory allocation failed `' @ error/tiff.c/ReadTIFFImage/1773
Source=Magick.NET-Q8-x86
StackTrace:
at ImageMagick.NativeInstance.CheckException(IntPtr exception, IntPtr result)
at ImageMagick.MagickImage.NativeMagickImage.ReadBlob(MagickSettings settings, Byte[] data, Int32 length)
at ImageMagick.MagickImage.Read(Byte[] data, Int32 length, MagickReadSettings readSettings, Boolean ping)
at ImageMagick.MagickImage.Read(Byte[] data, MagickReadSettings readSettings)
at ImageMagick.MagickImage..ctor(Byte[] data)
at DI_Plot_2018.ImageProcessing.RotateImage..ctor(Byte[] BytesToRotate, Int32 RotationIndex) in C:\Users\Pablo\source\repos\DI-Plot 2018\ImageProcessing\RotateImage.cs:line 16
at DI_Plot_2018.StartProcessing.BeginProcessKeyValuePair.<>c__DisplayClass2_0.<RotatedImages>b__0(Int32 i) in C:\Users\Pablo\source\repos\DI-Plot 2018\StartProcessing\BeginProcessKeyValuePair.cs:line 45
at System.Threading.Tasks.Parallel.<>c__DisplayClass17_0`1.<ForWorker>b__1()

[snibgo]

Is there anything i can do to change the max memory MagickNET can allocate ?

Resource limits are set in policy.xml. If the problem is that one of those limits is reached, you can edit that file, eg to remove that limit entirely.

[pablobhz]

Is there anything i can do to change the max memory MagickNET can allocate ?

Resource limits are set in policy.xml. If the problem is that one of those limits is reached, you can edit that file, eg to remove that limit entirely.

And where do i find policy.xml ?

[snibgo]

Typically, in the same directory as binaries magick.exe etc, IM's installation directory. But the binaries may be in a "bin" directory, so the xmls are in ..\etc from there.

[pablobhz]

As soon I get home I’ll be checking it
However, there are some details I can share/

I’ve made the same thing on an older version (can’t remember right now, I was dealing with the same software) and I didn’t have any trouble .

I’ve made an regular loop and tried to add the rotated image to an List<MagickImage> but I got the same exception.

I’m reading the tiff into an List<byte[]> first só I can work with all objects on the ram.

[pablobhz]

Typically, in the same directory as binaries magick.exe etc, IM's installation directory. But the binaries may be in a "bin" directory, so the xmls are in ..\etc from there.

Sorry but i couldn't find any policy.xml file.
The only XML i found were related to instances description when you initialize them.

Shouldn't ResourceLimits do what you're telling me to (set the resource limits to something higher) ?

[snibgo]

policy.xml sets limits that can't be overridden. So if those limits are too low, you can't get past them.

I suggest you try:

Code: Select all

magick -debug all rose: NULL: 2>&1 |grep .xml

This does nothing useful (it reads a built-in image and writes it to a bit-bucket) but the debug output will be something like:

Code: Select all

  Searching for configure file: "C:\ProgramFiles\ImageMagick-7.0.7-28-Q16-HDRI\coder.xml"
  Searching for configure file: "C:\Users\Alan\AppData\Local\ImageMagick\coder.xml"
  Loading coder configuration file "C:\ProgramFiles\ImageMagick-7.0.7-28-Q16-HDRI\coder.xml" ...
  Searching for configure file: "C:\ProgramFiles\ImageMagick-7.0.7-28-Q16-HDRI\delegates.xml"
  Searching for configure file: "C:\Users\Alan\AppData\Local\ImageMagick\delegates.xml"
  Loading delegate configuration file "C:\ProgramFiles\ImageMagick-7.0.7-28-Q16-HDRI\delegates.xml" ...
  Loading delegate configuration file "C:\Users\Alan\AppData\Local\ImageMagick\delegates.xml" ...

This tells me the two locations (system-wide, and a user directory) that IM expects to find its XML files.

[pablobhz]

policy.xml sets limits that can't be overridden. So if those limits are too low, you can't get past them.

I suggest you try:

Code: Select all

magick -debug all rose: NULL: 2>&1 |grep .xml

This does nothing useful (it reads a built-in image and writes it to a bit-bucket) but the debug output will be something like:

Code: Select all

  Searching for configure file: "C:\ProgramFiles\ImageMagick-7.0.7-28-Q16-HDRI\coder.xml"
  Searching for configure file: "C:\Users\Alan\AppData\Local\ImageMagick\coder.xml"
  Loading coder configuration file "C:\ProgramFiles\ImageMagick-7.0.7-28-Q16-HDRI\coder.xml" ...
  Searching for configure file: "C:\ProgramFiles\ImageMagick-7.0.7-28-Q16-HDRI\delegates.xml"
  Searching for configure file: "C:\Users\Alan\AppData\Local\ImageMagick\delegates.xml"
  Loading delegate configuration file "C:\ProgramFiles\ImageMagick-7.0.7-28-Q16-HDRI\delegates.xml" ...
  Loading delegate configuration file "C:\Users\Alan\AppData\Local\ImageMagick\delegates.xml" ...

This tells me the two locations (system-wide, and a user directory) that IM expects to find its XML files.

I'm using MagickNET, not ImageMagick. I don't have ImageMagick installed here.
Should i install and try to search the file ?

Also i believe my problem is something else, because i tried with other library (OpenCV) and had the same problems
I'll try to change my target platform to x64 and see if the problem persists.

[snibgo]

Maybe you simply don't have enough memory to rotate the images at the same time.

[pablobhz]

Even rotating in a regular for i had the issue (not doing in parallel).
I fixed my problem by rebuilding the project entirely on x64.

It takes 2GB ram basically to rotate all those heavy images.

Anyway thanks in advance !