Perl Thread Problem: perl: magick/image.c:1469: DestroyImage

[Jay]

Hey,

I am using Image Magick and it works great!
However today I tried to add multi threading to my script.
Without threads (no error):

Code: Select all

print start(1);
print start(2);

With threads (error: perl: magick/image.c:1469: DestroyImage: Assertion `image->signature == 0xabacadabUL' failed.)

Code: Select all

use threads;
my $thr = threads->create('start', '1');
my $thr2 = threads->create('start', '2');
print $thr->join();
print $thr2->join();

It seems that as soon as $thr->join() is called my script crashes and displays this exception.

Do I have to use MagickWandGenesis?
How can I solve this with perl?

Thanks
Jay

[magick]

Download the very latest ImageMagick release, 6.5.0-10. We have a few patches for threading issues. If that still fails, post a small script we can use to reproduce the problem and we will come up with a fix within a day or two. Thanks.

[Jay]

I am using the Debian 64bit release 6.3.7.9 and could not find any newer version.
Is there any newer version?

[magick]

Not sure how Debian distributions work, however, you can always download the source and install like this:

• tar xvfz ImageMagick-6.5.1-0.tar.gz
  cd ImageMagick-6.5.1-0
  ./configure
  make
  make install

[Jay]

I did:

Code: Select all

wget ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-6.5.1-0.tar.gz
tar xvfz ImageMagick-6.5.1-0.tar.gz
cd ImageMagick-6.5.1-0
./configure

configuring ImageMagick 6.5.1-0
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking whether build environment is sane... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gcc... gcc
checking for C compiler default output file name...
configure: error: in `/opt/www/jan/image_magick/ImageMagick-6.5.1-0':
configure: error: C compiler cannot create executables
See `config.log' for more details.

Code: Select all

apt-get install imagemagick

Reading package lists... Done
Building dependency tree
Reading state information... Done
imagemagick is already the newest version.

Code: Select all

/usr/bin/convert -version
Version: ImageMagick 6.3.7 02/18/08 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2008 ImageMagick Studio LLC

EDIT:
it seems libc6-dev was missing so i installed it:

Code: Select all

apt-get install libc6-dev

now

Code: Select all

./configure 

worked however

Code: Select all

make

failed (/usr/bin/ld: cannot find -lperl)
solution:

Code: Select all

apt-get install libperl-dev

and now it works!

Code: Select all

make
make install

However I guess perl cant find it:

test.pl

Code: Select all

use Image::Magick;

Code: Select all

Can't load '/usr/local/lib/perl/5.8.8/auto/Image/Magick/Magick.so' for module Image::Magick: /usr/lib/libgomp.so.1: cannot allocate memory in static TLS block at /usr/lib/perl/5.8/DynaLoader.pm line 225.
 at test.pl line 1
Compilation failed in require at test.pl line 1.
BEGIN failed--compilation aborted at test.pl line 1.
&Image::Magick::constant not defined. The required ImageMagick libraries are not installed or not installed properly.
END failed--call queue aborted at test.pl line 1.

Solution:

Code: Select all

./configure --disable-openmp
make
make install

test.pl

Code: Select all

use Image::Magick;
my $test=new Image::Magick;
print $test->GetAttribute('version');

finally shows the right version

Code: Select all

perl test.pl
ImageMagick 6.5.1-0 2009-04-01 Q16 h

[Jay]

I tried to shrink my code: (causes a crash)

main.pl

Code: Select all

use strict;
use Image::Magick;
require "sub.pl";
my $x = new Image::Magick;
$x -> ReadImage("xc:white");
startThreads();
my $y = new Image::Magick;
$y -> ReadImage("xc:white");
print "DONE!";
1;

sub.pl

Code: Select all

use threads;

sub startThreads{

	 sub start_thread {
			my @args = @_;
			print('Thread started: ', join(' ', @args), "\n");
			for(my $j=0;$j<10;$j++){
			for(my $i=1;$i<10;$i++){
				my $x=1/$i;
			}
			}
			print('Thread ended: ', join(' ', @args), "\n");
			return "juhu";
		}
	my $thr = threads->create('start_thread', '1');
	my $thr2 = threads->create('start_thread', '2');
	print $thr->join();
	print $thr2->join();

}

1;

perl main.pl
Thread started: 1
Thread ended: 1
Thread started: 2
Thread ended: 2
perl: magick/image.c:1611: DestroyImage: Assertion `image->signature == 0xabacadabUL' failed.
Aborted


As you can see my last line print "DONE!"; is not executed.
I only use threads in sub.pl and dont access ImageMagick within any thread.

[magick]

You exit the program while the threads are still running. If you wait for the threads to complete, the program exits without crashing. Use is_running() and sleep() or just sleep() and your program exits without complaint.

[Jay]

print $thr->join(); should print out my start_thread sub return value of return "juhu";
but it does not because of image magick

i tried to create an easier example:

Code: Select all

use strict;
use threads;
use Image::Magick;


my $x = new Image::Magick;
$x -> ReadImage("xc:white");
#undef $x;
startThreads();

print "DONE!\n";

sub startThreads{
  print "Start of start threads \n";
  my $thr = threads->create('threadSub', '1');
  my $thr2 = threads->create('threadSub', '2');
  print $thr->join();
  #$x -> ReadImage("xc:white");
  print $thr2->join();
  print "End of start threads \n";
}

sub threadSub {
  my @args = @_;
  print('Thread started: ', join(' ', @args), "\n");
  sleep(1);
  print('Thread ended: ', join(' ', @args), "\n");
  return "return value (".join(' ', @args).") \n";
}


1;

result:

Code: Select all

Start of start threads
Thread started: 1
Thread started: 2
Thread ended: 1
return value (1)
Thread ended: 2
perl: magick/image.c:1611: DestroyImage: Assertion `image->signature == 0xabacadabUL' failed.
Aborted

in line 8 I wrote: #undef $x;
if you change it to undef $x;
the code will work:

Code: Select all

Start of start threads
Thread started: 1
Thread started: 2
Thread ended: 1
return value (1)
Thread ended: 2
return value (2)
End of start threads
DONE!

now please change line 8 back to #undex $x and change line 20
from #$x -> ReadImage("xc:white"); to $x -> ReadImage("xc:white");
you will see this output:

Code: Select all

Start of start threads
Thread started: 1
Thread started: 2
Thread ended: 1
return value (1)
Thread ended: 2
return value (2)
End of start threads
perl: magick/image.c:1611: DestroyImage: Assertion `image->signature == 0xabacadabUL' failed.
Aborted

so I guess the imageMagick fails because of this:

$x is created global in the main perl thread
two further threads are started

1. thread is done and ->join() destroys it. ImageMagick destroys $x
2. thread is done and ->join() destroys it. ImageMagick and can't destroy $x and dies.

Now here is my question: why does it destroy a global that is neither initialized nor used in this thread?

[Jay]

[Magick-announce] ImageMagick 6.3.2 Released

2006-10-02 6.3.0-0 Cristy <quetzlzacatenango at image...>
* Destroy MagickCore API environment when END{} is called in PerlMagick
(patch provided by Dmitry Karasik).

Maybe this is done everytime any thread ends.

[magick]

Perl is creating a new reference for $x, one for each thread. The reference points to a single image object. The assertion is thrown when the same image object is destroyed more than once. We're curious why a new reference is created for a global variable for each thread. This could be a problem with Perl threads. We're also not sure PerlMagick has an opportunity to intercept the new references as they are created so we can clone the image object. If we clone, DESTROY() will work because it will not be destroying the same image object each time.

It will take some time to investigate if the problem is with Perl threads (likely) or with PerlMagick (certainly possible). In the mean-time, if you have any insight, let us know.

[Jay]

I could create a really wearied work around.
As I am coding Perl only for 1 month I can't guarantee that this is the best solution but it works for me

I wrote a new Packege:
SleepThread.pm

Code: Select all

package SleepThread;

use strict;
use threads;
use threads::shared;

my @sThreads=();

sub new {
	my $ref = {};
	bless($ref,shift);
	$ref->init(shift);
	push @sThreads,$ref;
	return $ref;
}

sub EndAll {
	for (@sThreads)
	{
		$_->cancel();
	}
}

sub init {
	my $this=shift;
	my $waitingFunction = shift;
	my @pPointer=();
	share @pPointer;
	$this -> {waitForMe}="0";
	$this -> {pPointer}=\@pPointer;
	share($this -> {waitForMe});
	$this->{thread} = threads->create('sleepingThread',($this,$waitingFunction));
}

sub start {
	my ($this,@params)=@_;
	$this->{waitForMe} = 1;
	push @{$this -> {pPointer}},@params;
	cond_signal $this->{waitForMe};
}

sub cancel {
	my $this=shift;
	$this->{waitForMe} = 2;
	cond_signal $this->{waitForMe};
}

sub join {
	my $this=shift;
	return $this->{thread}->join();
}

sub run {
	my ($this,@params)=@_;
	$this->start(@params);
	return $this->join();
}

sub sleepingThread{
	my $this = shift;
	my $waitingFunction = shift;
	lock $this->{waitForMe};
	cond_wait $this->{waitForMe};
	if ($this->{waitForMe} == 1)
	{
		return $waitingFunction->( @{$this -> {pPointer}} );
	}
}

now i changed my main.pl

Code: Select all

use strict;
use threads;
use threads::shared;

#1 change
use SleepThread;

use Image::Magick;


#2 change
my $st = SleepThread->new(\&startThreads);


my $x = new Image::Magick;
$x -> ReadImage("xc:white");

#3 change 
#startThreads("Test Value");
$st->run("Test Value");

print "DONE!\n";

sub startThreads{
  print $_[0]."\n";
  print "Start of start threads \n";
  my $thr = threads->create('threadSub', '1');
  my $thr2 = threads->create('threadSub', '2');
  print $thr->join();
  #$x -> ReadImage("xc:white");
  print $thr2->join();
  print "End of start threads \n";
}

sub threadSub {
  my @args = @_;
  print('Thread started: ', join(' ', @args), "\n");
  sleep(1);
  print('Thread ended: ', join(' ', @args), "\n");
  return "return value (".join(' ', @args).") \n";
}


1;

result:

Code: Select all

Test Value
Start of start threads
Thread started: 1
Thread started: 2
Thread ended: 1
return value (1)
Thread ended: 2
return value (2)
End of start threads
DONE!

How does that work?

It hides my code in a thread so ImageMagick cant see it and waits for execution.
As soon as I need it I resume this thread.
This resumed thread is free to start new threads.

[Jay]

Okay now I completely understand the problem:

Threads don't share references so any reference like an image magick object or a hash-reference is cloned and turned back in a new reference.
So you could check wether the reference changed!

Code: Select all

use threads;
use Image::Magick;

my $x = new Image::Magick;
$x -> ReadImage("xc:white");

print scalar($x)."\n";
startThreads();

sub startThreads{
  print "Start of start threads \n";
  my $thr = threads->create('threadSub', '1');
  my $thr2 = threads->create('threadSub', '2');
  print $thr->join();
  print $thr2->join();
  print "End of start threads \n";
}

sub threadSub {
  my @args = @_;
  print scalar($x)."\n";
}
1;

Result:

Code: Select all

Image::Magick=ARRAY(0x604290)
Start of start threads
Image::Magick=ARRAY(0x975f10)
Image::Magick=ARRAY(0xa7d0d0)
perl: magick/image.c:1611: DestroyImage: Assertion `image->signature == 0xabacadabUL' failed.
Aborted

As you can see $x changed to a new reference!
So how could you solve this?

Every time a new Image is Created (f.e. by ReadImage) you store the scalar value of the hash reference as a string -
f.e. $objName="".scalar($obj);
Inside of your destroy function you check whether these hash references fit or not.
If they don't fit the ImageObject muss not be destroyed.

Does this make sense to you?

[magick]

The solution to the problem is to implement the XS CLONE() method or tag each image so that it is not destroyed more than once. Some superficial analysis suggests this is a difficult problem so its now on our bug fix list. Unfortunately we currently do not have an ETA on a fix. In the mean-time if you discover a simple and elegant solution, let us know.

[Jay]

I found a solution (I hope it won't slow down my script too much)

Code: Select all

my $x = new Image::Magick;
$x -> ReadImage("xc:white");

my @blobs = $x->ImageToBlob();
undef $x;

#threading

my $x = new Image::Magick;
$x->BlobToImage(@blobs);

[Jay]

could you fix this bug?