Integrating PerlMagick with Perl OpenGL (POGL)

[grafman]

I'm the primary owner of CPAN's OpenGL module, and would like to work with PerlMagick's authors to tightly integrate our two modules.

POGL supports mapping GPU memory to CPU memory (via Frame Buffer Objects and Vertex Buffer Objects). POGL provides an OpenGL::Array object that wraps C buffers, which allows Perl developers to efficiently share/pass C arrays without copying/converting/casting the data.

What I'd like to do is add/expose an interface in PerlMagick (via XS) to do the following:

1. Use a C pointer (to an RGB or RGBA buffer from POGL) and create a PerlMagick image object or blob (directly caching/using the C pointer - not a copy), and

2. Get a C pointer to an RGA/RGBA buffer from a PerlMagick image object or blob (a direct reference, not a copy).

This would provide a mapped buffer between the GPU and IM, and would be much faster than using GetPixels, converting the resulting Perl array into a C array, do a GPU operation, to convert it back to a Perl array, then finally back to SetPixels.

IM provides a perfect compliment to POGL by loading/creating/saving textures, as well as providing data processing for vertex data (stored as images).

You can read about POGL's performance at http://graphcomp.com/opengl/benchmarks/

Ideally, I'd like to see this added to PerlMagick (I'd be happy to help); alternately, I could add the IM interfaces within POGL. I'd prefer the former; the latter is a fallback if no one has the time to implement this in PerlMagick.

If anyone knows how to contact PerlMagick's authors (Kyle Shorter?), please send them my way.

Thanks! - Bob Free

[magick]

PerlMagick has a BlobToImage() and ImageToBlob() methods that should do what you want. In addition ReadImage() and WriteImage() has a blob=> parameter to accept in-memory image formats including raw RGB and RGBA bytes.

[grafman]

PerlMagick has a BlobToImage() and ImageToBlob()

Very familiar with IM blobs - and if I were using the C interface, it'd be exactly what I need. However, in PerlMagick, blobs are _copied_ and returned/passed by Perl reference. I'm looking for an XS API that returns/accepts a C pointer to a blob's data (without alloc/copy/release).

Perl OpenGL (POGL) has APIs that accept C pointers, bypassing the copy/convert/casting of Perl arrays/references. OpenGL::Array (OGA) objects wrap typed C buffers; oga->ptr() gives you a C pointer (not a Perl reference); this gets mapped to GPU memory. In other words, if you change a value in OGA, you are directly changing a value on the GPU (via DMA).

It's this memory mapping that allows Perl OpenGL performance to be comparable to C, faster than SDL, and much faster than Python (PyOpenGL).

What I want to do is use POGL's mapped C pointers as IM image buffers, so that when you do an IM operation, it will be directly modifying GPU memory, and vice versa.

In other words, what I need is something like cptr = $im->ImageToPointer() and $im->PointerToImage(cptr) - which keeps the pointer, rather that alloc/copy/release.

The following is the XS code POGL uses to return a C pointer:

Code: Select all

void *
ptr(oga)
	OpenGL::Array	oga
	CODE:
	RETVAL = oga->data;
	OUTPUT:
	RETVAL

The following is an example of XS code that accepts a C pointer:

Code: Select all

void
glBitmap_c(width, height, xorig, yorig, xmove, ymove, bitmap)
	GLsizei	width
	GLsizei	height
	GLfloat	xorig
	GLfloat	yorig
	GLfloat	xmove
	GLfloat	ymove
	void *	bitmap
	CODE:
	glBitmap(width, height, xorig, yorig, xmove, ymove, bitmap);

Really trivial XS code. So the question is, is there a PerlMagick author/contributor that can work with me to add this PerlMagick, or will I need to add IM XS wrapper APIs to OpenGL. I'd prefer the former.

Thanks! - Bob

[magick]

Write the XS modules you want added to PerlMagick and we'll add it assuming it does not cause an excessive burden to the code base. Post your patch here and we'll get it into the most recent ImageMagick beta within a day or two.

[grafman]

Write the XS modules you want added to PerlMagick and we'll add it assuming it does not cause an excessive burden to the code base.

Excellent - thanks!!! I'm out of town at the moment, but will put this together as soon as I return - Bob

[grafman]

This posting has generated some interest - and I've been getting requests for additional info - so here's the background, and what I'm planning to do...

I've been doing 3D development since 1973, and haved worked with OpenGL for a while. I stumbled on CPAN's OpenGL module early last year, and found that no one had been maintaining it for the past 5+ years - so it lacked support for most of the recent OpenGL extensions, like Frame Buffer Objects (FBOs), Vertex Buffer Objects (VBOs) and Vertex/Fragment Programs (shaders).

I was given control over CPAN's OpenGL module earlier this year and posted a major update, which includes added support for over 50 new extensions. I've also posted a number of benchmarks that demonstrate: Perl OpenGL (POGL) performance is comparable to C, faster than SDL::OpenGL and much faster than PyOpenGL.

I've been using IM with OpenGL for quite a while - mainly for loading/saving textures - but since PerlMagick lacks a method for getting/setting image buffers via a C pointer (understandable, as POGL is the only Perl module that I'm aware of that uses C pointers), there was a significant performance bottleneck.

Therefore, I began looking into ways that I could more tightly integrate POGL with PerlMagick:

1. Add interfaces in PerlMagick that would expose C pointers to IM's image buffers.

2. Using these pointers, provide an efficient method in POGL to use IM to load/save textures via FBOs. This will allow Perl developers to easily write high-perfomance GPU effects using PerlMagick - without the need to modify/update or add plug-ins to IM.

3. Using these pointers, provide methods for mapping IM buffers to VBOs, allowing PerlMagick to draw directly to GPU memory, and the GPU to draw directly to PerlMagick objects. This will alllow Perl developes to use PerlMagick for GPGPU processing.

4. As a separate project, I'd like to eventually add GPU features within IM to improve performance and add new image processing features.

The result will be significant improvements in performance and functionality for both modules. I believe there's a natural synergy between PerlMagick and POGL, and look forward to working with the PerlMagick community to make this happen! - Bob

[grafman]

I've just submitted 3 new PerlMagick APIs to ImageMagick's developer team for consideration:

• Get('Quantum') - returns IM's cache depth.
• ImageToPointer() - returns a C pointer to IM's image cache.
• SyncPointer() - syncs IM's image cache, after the GPU has written to it.

Benchmarks indicate about an 188X performance improvement over ImageToBlob:

• ImageToBlob + glTexImage2D_s: 44.789FPS
• GetPixels + glTexImage2D_p: 45.595FPS
• ImageToPointer + glTexImage2D_c: 8614.444 FPS

The performance improvement is due the reduction/elimination of copy/converting/casting data buffers, required by traditional Perl interfaces.

I will be releasing a POGL update to coincide with the beta release containing these enhancements.

[grafman]

With magick's support, the following APIs have been checked into trunk and the latest beta build (6.3.5):

• Get('Quantum') - returns IM's cache depth.
• GetImagePixels() - returns a C pointer to IM's image cache.
• SyncImagePixels() - syncs IM's image cache, after the GPU has written to it.

I'll be posting a new POGL update to coincide with 6.3.5's release (hopefully in a couple of weeks), which will include sample code for PerlMagick/OpenGL interoperability.