Very Slow Performance to get one pixel RGB

[beshoo]

Dear user ,

I am sorry if i post in the wrong form , please correct me if !

Any way , i have an Image 2000*2000 bmp format.
I have an Array of 3000 coordinate XY. and i want to collect the RGB color value for each pixel.

will , i know the command line , which is

Code: Select all

/usr/bin/convert ./twitter.bmp -format '%[pixel:p{232,345}]' info:- 

which is work great ! but very slow !

How i determinate that ?

ok let see this Bash code :

Code: Select all

#!/bin/bash
for (( c=1; c<=3000; c++ ))
 do
   x=$(( $RANDOM % 2000 ));
   y=$(( $RANDOM % 2000 ));
   /usr/bin/convert ./twitter.bmp -format '%[pixel:p{'$x','$y'}]' info:- >> testfile.txt 
done

this code will generate random XY and trying to simulate my array situation, goal is fetching RGB color value of the pixel form the image in a loop 3000 time and store the RGB values in a Text file.
this code take more than 3 min to complete !

try it by your self and see how much time it take , my server is very powerful , cor i7 * 4 and 64 GB of ram !

I am sure there is some thing wrong with the code ,
Ps i did test PHP

Code: Select all

<?
$startTime = microtime(true);

for ($i =0 ; $i <= 3000 ; $i ++){
$x = rand(0,2000);
$y = rand(0,2000);

exec("/usr/bin/convert ./twitter.bmp -format '%[pixel:p{".$x.",".$y."}]' info:-  2>&1", $json);
}
$endTime = microtime(true);  
$elapsed = $endTime - $startTime;
echo "Execution time : $elapsed seconds \n\n\n";

?>

and this one :

Code: Select all

<?
$startTime = microtime(true);

$image = new Imagick('./twitter.bmp');

for ($i =0 ; $i <= 3000 ; $i ++){
$x = rand(0,2000);
$y = rand(0,2000);


$pixel = $image->getImagePixelColor($x, $y);
$colors = $pixel->getColor();
}
$endTime = microtime(true);

print_r($colors);

    $elapsed = $endTime - $startTime;
    echo "Execution time : $elapsed seconds \n\n\n";


?>

all of these samples take a huge time to complete .

So what the magic behind the imagemagick !

any way lets see what imagemagick can do regarding this !

Thank you and best regards.

[fmw42]

fx: or pixel: operations are relatively slow. So not advised for loops over large numbers of items.

What is your IM version and platform? Please always provide that, since syntax differs and new functions may be available. If you are using IM 6.8.3.10 and can make all other pixels transparent, then you can get all the non-transparent pixels quickly using sparse-color:

Code: Select all

convert image sparse-color:

will give a space separate list

or

Code: Select all

convert sparse_source.gif sparse-color: | tr " " "\n"

will give a line for each color

If you are on an older system, make all other pixels transparent and do

Code: Select all

convert image txt: | grep -v "none"

will give a space separate list



Otherwise, you might just do

Code: Select all

convert image txt:

and save as string or text file and then search the textual data for the pixel coordinates you want as an OS script (shell or batch depending upon your platform)

[beshoo]

I am very sorry for lack of information
this oparation is very important , users will upload images to the server and my server will do the math and retrun some math information base on the RGB color value , user will not wate 2 min ! i have to find a solution to serve the information within max 3 or 5 sec !

My server FULL information :

Code: Select all

My server : Linux - CentOs 6.1 64 Bit

ImageMagick Version

Code: Select all

root@server [/]# /usr/bin/convert -version
Version: ImageMagick 6.7.2-7 2015-07-29 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2011 ImageMagick Studio LLC
Features: OpenMP  

please have a look , the image i am working on :
http://trafficlayer.net/gmap/tmp/twitter.bmp

that is what i have , and i dont know how can i make the gray color "#E5E3DF" transparent , any way i have the location of XY to be on the Colored Lines.

see the green lines , some times it has multi colors , Red , orange , Black but the background is "#E5E3DF" for sure.

Code: Select all

convert image txt:

take very long time one min .. download my image and test it by your self


My server Information :
Processor

Code: Select all

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 45
model name      : Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
stepping        : 7
microcode       : 1808
cpu MHz         : 1999.984
cache size      : 15360 KB
physical id     : 0
siblings        : 12
core id         : 0
cpu cores       : 6
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb pln pts dtherm tpr_shadow vnmi flexpriority ept vpid xsaveopt
bogomips        : 3999.96
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

Memory

Code: Select all

             total       used       free     shared    buffers     cached
Mem:      65801204   63896028    1905176      25692    2301604   58034960
-/+ buffers/cache:    3559464   62241740 
Swap:      1999868     654196    1345672 
root@server [/]# cat /proc/meminfo
MemTotal:       65801204 kB
MemFree:         1683200 kB
Buffers:         2301632 kB
Cached:         58035168 kB
SwapCached:       295452 kB
MemCommitted:   198180864 kB
VirtualSwap:           0 kB
Active:         18471276 kB
Inactive:       43362292 kB
Active(anon):     874396 kB
Inactive(anon):   631064 kB
Active(file):   17596880 kB
Inactive(file): 42731228 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       1999868 kB
SwapFree:        1345676 kB
Dirty:              6184 kB
Writeback:             0 kB
AnonPages:       1220268 kB
Mapped:            71300 kB
Shmem:             25692 kB
Slab:            2073360 kB
SReclaimable:    1965892 kB
SUnreclaim:       107468 kB
KernelStack:       11424 kB
PageTables:        26028 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    34900468 kB
Committed_AS:    2759012 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      395044 kB
VmallocChunk:   34324353000 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        4272 kB
DirectMap2M:     2058240 kB
DirectMap1G:    65011712 kB

[fmw42]

This takes about 12 s on my Mac Mini and captures 278214 color values

Code: Select all

time convert twitter.bmp -fuzz 10% -transparent "#E5E3DF" sparse-color: | tr " " "\n"

But will likely get too many or too few coordinates. You will need to tune the fuzz value to make transparent the background gray and other areas that are not of interest.

If you want every pixel other than exactly #E5E3DF, then leave off the -fuzz or set the value to 0.


Unfortunately, your IM version is too old for sparse-color:. So I would recommend that you upgrade.


The only other thing I can suggest is to crop each pixel and get its color

Code: Select all

convert twitter.bmp -crop 1x1+50+50 +repage txt: | tail -n +2 | cut -d\  -f4

#E5E3DF

or

Code: Select all

convert twitter.bmp -crop 1x1+50+50 +repage txt: | tail -n +2 | cut -d\  -f6

srgb(229,227,223)

But I do not know if this will be any faster than use pixel:



Otherwise, I am at a loss to recommend anything else.

[snibgo]

#!/bin/bash
for (( c=1; c<=3000; c++ ))
do
x=$(( $RANDOM % 2000 ));
y=$(( $RANDOM % 2000 ));
/usr/bin/convert ./twitter.bmp -format '%[pixel:p{'$x','$y'}]' info:- >> testfile.txt
done

You are running convert, and re-reading the entire image into memory, for every pixel you find. And then you are appending (opening and closing) a text file for every pixel. Of course that takes a long time.

There are many solutions. For example, you could generate a text script so there was only one convert, and one read of the file. Open the output txt file, write the info about the 3000 pixels, and close the output file.

EDIT: on a Core i7, Windows 8.1, Cygwin bash, 2000x2000 pixel image, the above takes 197 seconds. But we can easily reduce this to 2.7 seconds (including the time to start up and shut down bash):

Code: Select all

#!/bin/bash

echo sq2000.bmp > randPix.scr
for (( c=1; c<=3000; c++ ))
 do
   x=$(( $RANDOM % 2000 ));
   y=$(( $RANDOM % 2000 ));
   echo -format '%[pixel:p{'$x','$y'}]' +write info:
done >> randPix.scr

convert @randPix.scr NULL: >testfile2.txt

As Fred says, "pixel:" isn't fast. If you wanted a million results, you could dump the entire image to a text file, one pixel per line, and pick a million random lines.

[beshoo]

I like to thank you for your kindly reply ,snibgo and fmw42.


SOLVED