Using php proc_open() with multiple input streams

[CodeFan]

0 down vote favorite
share [g+] share [fb] share [tw]


Seems like I'm having problems using the streams which are piped to a process when using the proc_open() php function.

The process I'm starting is simply the convert ImageMagick utility to compose 3 images on top of each other. When only 1 input stream is used (STDIN) and a variable is dumped into that stream the convert program works fine and returns its output which can be stored in a variable like so:

Code: Select all

$cmd = BIN_PATH.DIRECTORY_SEPARATOR.'convert ';
$cmd .= ' -size SOMESIZE ';
$cmd .= ' -background black ';
$cmd .= ' -fill white ';
$cmd .= ' -stroke none ';
$cmd .= ' -gravity center ';
$cmd .= ' -trim ';
$cmd .= ' -interline-spacing SOMELINEHEIGHT ';
$cmd .= ' -font SOMEFONT ';
$cmd .= ' label:"SOMETEXT" ';
$cmd .= ' miff:- ';
$ctext_opacity = shell_exec($cmd);

First I run the convert and store the output in the $ctext_opacity variable. Then the next command is called through proc_open() and the $ctext_opacity variable is piped trough the STDIN and used as an input image:

Code: Select all

$cmd = BIN_PATH.DIRECTORY_SEPARATOR.'convert ';
$cmd .= '-size SOMESIZE ';
$cmd .= ' xc:\'rgb(230, 225, 50)\' ';
$cmd .= ' -gravity center ';
$cmd .= ' - '; // ImageMagick uses dash(-) for STDIN
$cmd .= ' -alpha Off ';
$cmd .= ' -compose CopyOpacity ';
$cmd .= ' -composite ';
$cmd .= ' -trim ';
$cmd .= ' miff:- ';
$chighlight = '';
$descriptorspec = array(
    0 => array("pipe", "r"),
    1 => array("pipe", "w")
);
$process = proc_open($cmd, $descriptorspec, $pipes);

if (is_resource($process)) {
    fwrite($pipes[0], $ctext_opacity);
    fclose($pipes[0]);

    while (!feof($pipes[1])) {
        $chighlight .= fgets($pipes[1]); // HERE WE FEED THE OUTPUT OF "CONVERT" TO $chighlight
    }
    //echo $chighlight; die();
    fclose($pipes[1]);

    $return_value = proc_close($process);
}

The above command is called 3 times and 3 separate images are generated and stored in 3 variables. The next command is supposed to accept those 3 variables as input images (the ImageMagic syntax specifies the alternative io streams like fd:N where N is the number of the stream which I spawn through proc_open()). However I seem to be writing to the input streams or reading from the STDOUT incorrectly which results most probably in unflushed output from the process which causes it to hang without terminating.

Code: Select all

$cmd = BIN_PATH.DIRECTORY_SEPARATOR.'convert ';
$cmd .= ' -size SOMESIZE ';
$cmd .= ' xc:transparent ';
$cmd .= ' -gravity center ';
$cmd .= ' - -geometry -2-2 -composite ';
$cmd .= ' fd:3 -geometry +2+2 -composite ';
$cmd .= ' fd:4 -composite ';
$cmd .= 'png:- ';

$descriptorspec = array(
    0 => array("pipe", "r"),
    1 => array("pipe", "w"),
    2 => array("pipe", "a"),
    3 => array("pipe", "r"),
    4 => array("pipe", "r")
);
$process = proc_open($cmd, $descriptorspec, $pipes);
if (is_resource($process)) {
    $read = null;
    $rd = array($pipes[1]);
    $write = array($pipes[0], $pipes[3], $pipes[4]);
    $wt = array($pipes[0], $pipes[3], $pipes[4]);
    $im_args = array($cshade, $chighlight, $ctext);
    $except = null;
    $readTimeout = 1;
    $ctext_deboss = '';

    $numchanged = stream_select($read, $write, $except, $readTimeout);
    foreach($write as $w) {
        $key = array_search($w, $wt);
        fwrite($wt[$key], $im_args[$key]);
        fclose($wt[$key]);

        $read = array($pipes[1]);
        $rd = array($pipes[1]);
        $write = null;
        $except = null;
        $readTimeout = 1;
        $ctext_deboss = '';

        $numchanged = stream_select($read, $write, $except, $readTimeout);
        foreach($read as $r) {
            while (!feof($r)) {
                $ctext_deboss .= fgets($pipes[1]);
            }
        }
        fclose($pipes[1]);

        $return_value = proc_close($process);
        echo $ctext_deboss; die();
    }
}

I can't seem to transfer the 3 & 4 pipes' contents as convert throws an error with empty/incorrect data

[anthony]

The problem with IMv6 is that IM reads a pipe until the pipe closes (recieves an EOF). I think it also forcably closes the input when finsihed, even when thier is no read need. Imagemagick version 6 was just not really designed with pipelines in mind. It may not be flushing output correct. Imagemagick version 7 will be. That is read images with a well defined 'End of Image', but without closing that pipeline.

Solutions.

IM can read multiple images (of the same file format) from a pipeline. So if you have a number of images using a 'streaming image' format (PbmPlus, MIFF, TXT), then you can send it all those images, one afetr another before closing your input pipe.

Also IM can read from multiple file descriptors using the fd:N coder. That is FD:0 is another name for standard input. So if you open multiple descriptors with popen, you can send different images into IM using defferent file descriptors. (EG descriptor 4,5,6 etc) You also can use -write to write to file descriptors too! The order to send/recieve images is command line option order.

See fd
http://www.imagemagick.org/Usage/files/#fd

I will need to look at your examples more closely, and will get back to you on them. I have not used the PHP popen much.

[CodeFan]

Thanks for the thorough explanation. Indeed the solution I came accross was to use a concatenated png file with the 3 images appended to each other (each output as a png as well). That way the concatenated images are still stored in a php variable and no intermediate files are used. That seemed to shorten the generation timespan from 3.5 to about 2.5 seconds. Here's the working code:

Code: Select all

		    	$khh = BIN_PATH.DIRECTORY_SEPARATOR.'convert ';
		    	$khh .= '						-size '.$imageSize.' ';
		    	$khh .= '						xc:\'rgb(230, 225, 50)\' ';
		    	$khh .= '						-gravity center ';
		    	$khh .= '						- ';
		    	$khh .= '						-alpha Off ';
		    	$khh .= '						-compose CopyOpacity ';
		    	$khh .= '						-composite ';
		    	$khh .= '						-trim ';
    	                 $khh .= '						png:- ';
		    	
				$chighlight = '';
		    	$descriptorspec = array(
		    			0 => array("pipe", "r"),
		    			1 => array("pipe", "w")
		    	);
		    	$process = proc_open($khh, $descriptorspec, $pipes);
		    	 
		    	if (is_resource($process)) {
		    		fwrite($pipes[0], $ctext_opacity);
		    		fclose($pipes[0]);
		    		
		    		while (!feof($pipes[1])) {
		    			$chighlight .= fgets($pipes[1]);
		    		}
		    		
		    		fclose($pipes[1]);
		    		 
		    		$return_value = proc_close($process);
		    	}

....................................................................................

		    	$khh = BIN_PATH.DIRECTORY_SEPARATOR.'convert ';
		    	$khh .= '						-size '.$imageSize.' ';
		    	$khh .= '						xc:'.$fontColor.' ';
		    	$khh .= '						-gravity center ';
		    	$khh .= '						- ';
		    	$khh .= '						-alpha Off ';
		    	$khh .= '						-compose CopyOpacity ';
		    	$khh .= '						-composite ';
		    	$khh .= '						-trim ';
		    	if($_GET['ctr']==='3') {
		    	    $khh .= '						png:- ';
		    	} else {
		    	    $khh .= '						png:- ';
		    	}
		    	$ctext = '';
		    	$descriptorspec = array(
		    			0 => array("pipe", "r"),
		    			1 => array("pipe", "w")
		    	);
		    	$process = proc_open($khh, $descriptorspec, $pipes);
		    	
		    	if (is_resource($process)) {
		    		fwrite($pipes[0], $ctext_opacity);
		    		fclose($pipes[0]);
		    	
		    		while (!feof($pipes[1])) {
		    			$ctext .= fgets($pipes[1]);
		    		}
		    	
		    		fclose($pipes[1]);
		    		 
		    		$return_value = proc_close($process);
		    	}

....................................................................................

		    	$khh = BIN_PATH.DIRECTORY_SEPARATOR.'convert ';
		    	$khh .= '						-size '.$imageSize.' ';
		    	$khh .= '						xc:\'rgb(50, 85, 20)\' ';
		    	$khh .= '						-gravity center ';
		    	$khh .= '						- ';
		    	$khh .= '						-alpha Off ';
		    	$khh .= '						-compose CopyOpacity ';
		    	$khh .= '						-composite ';
		    	$khh .= '						-trim ';
	    	    $khh .= '						png:- ';
		    	$cshade = '';
		    	$descriptorspec = array(
		    			0 => array("pipe", "r"),
		    			1 => array("pipe", "w")
		    	);
		    	$process = proc_open($khh, $descriptorspec, $pipes);
		    	
		    	if (is_resource($process)) {
		    		fwrite($pipes[0], $ctext_opacity);
		    		fclose($pipes[0]);
		    	
		    		while (!feof($pipes[1])) {
		    			$cshade .= fgets($pipes[1]);
		    		}
		    	
		    		fclose($pipes[1]);
		    		 
		    		$return_value = proc_close($process);
		    	}

....................................................................................

		    	$khh = BIN_PATH.DIRECTORY_SEPARATOR.'convert ';
		    	$khh .= '						-size '.$sizeX.'x'.$sizeY;
		    	$khh .= '						xc:transparent ';
		    	$khh .= '						-gravity center ';
		    	$khh .= '						png:- -geometry -'.$i.'-'.$i.' -composite ';
		    	$khh .= '						png:- -geometry +'.$i.'+'.$i.' -composite ';
		    	$khh .= '						png:- -composite ';
	    	        $khh .= '						png:- ';
		    	
		    	$ctext_deboss = '';
		        $descriptorspec = array(
		    			0 => array("pipe", "r"),
		    			1 => array("pipe", "w")
		    	);
		    	$process = proc_open($khh, $descriptorspec, $pipes);
		    	
		    	if (is_resource($process)) {
		    		fwrite($pipes[0], $cshade.$chighlight.$ctext);
		    		fclose($pipes[0]);
		    	
		    		while (!feof($pipes[1])) {
		    			$ctext_deboss .= fgets($pipes[1]);
		    		}
		    	
		    		fclose($pipes[1]);
		    		 
		    		$return_value = proc_close($process);
		    	}

....................................................................................

The last proc_open just concatenates the 3 previously generated images ($cshade.$chighlight.$ctext) and feeds it into the stdin which does seem to work pretty well unlike the numbered file descriptors which I can't seem to handle right (or at least the php part). Later the 3 images are pulled consecutively by specifying just png:- as input.

Any optimization tips are greatly appreciated. Currently the final image being generated is a result of about 16 intermediate operations.
Create an emboss effect in true color.
Create an emboss effect mask (the goal is to have transparent text color but still preserve the highlight/shade of the emboss)
Add the two together
Add perspective distortion to the no collorfill embossed text
Create the same distortion effect but as a mask for the whole text
Add the text mask to a base image to have the text cut-out from the base image
Add the no colorfilled distorted text on top of the cut-out base image to finally have embossed perspective distorted text with the colorfill part transparent all the way through the base image

The 2.5 seconds are somewhat slow for a real time updating text input to generate the preview (considered the fact there's also an average 1.5 second delay for retrieving the image from the server). That being generated on a dedicated server machine (not sure about the specifics but I'd guess it's a pretty fast pc). What would be the most common pitfalls that drag the performance of imagemagick? Would a pc component upgrade do any good and which component is most critical to how fast the images are being generated?

[anthony]

Thanks for the thorough explanation. Indeed the solution I came accross was to use a concatenated png file with the 3 images appended to each other (each output as a png as well). That way the concatenated images are still stored in a php variable and no intermediate files are used. That seemed to shorten the generation timespan from 3.5 to about 2.5 seconds. Here's the working code:

I did not know you can simply concatinate PNG images! Im certainnlay can't seem to read concatenated PNG images.

MIFF and PPM images on the other hand are designed to concatanate together.

I'll have to study this further!

[CodeFan]

Indeed a png works just as good. In fact I couldn't get the miff to work AFAIC remember . I can send the preview generator file if anyone is interested.