fx complex if statement crashes imagemagick

[cnschulz]

Hello,

Im using ImageMagick 6.9.1-2 Q8 x64 2015-04-14 as part of a ruby script to process some mapping information. One step uses the "convert" utility to make a reference image in 100 shades. This is failing on my system giving a windows crash message "this program has stopped working...". I have tried putting the -fx operators in an external file as I though the command line was too long. This had no effect. The interesting thing is if I reduce the number of "ifs" down to about 55 from 100 then it works (but obviosly doesnt product the intended result). Memory doesnt seem to be an issue as im watching memory usage as the command runs (I still have 2GB free). So, can anybody suggest a cause and/or solution? Is there a better way to write the fx operators? Many thanks.

Code: Select all

convert -size 1x256 canvas:black -fx "j==200 ? 1.00000 : (j==199 ? 1.00000 : (j==198 ? 1.00000 : (j==197 ? 1.00000 : (j==196 ? 1.00000 : (j==195 ? 1.00000 : (j==194 ? 1.00000 : (j==193 ? 1.00000 : (j==192 ? 1.00000 : (j==191 ? 1.00000 : (j==190 ? 1.00000 : (j==189 ? 1.00000 : (j==188 ? 1.00000 : (j==187 ? 1.00000 : (j==186 ? 1.00000 : (j==185 ? 1.00000 : (j==184 ? 1.00000 : (j==183 ? 1.00000 : (j==182 ? 1.00000 : (j==181 ? 1.00000 : (j==180 ? 1.00000 : (j==179 ? 1.00000 : (j==178 ? 1.00000 : (j==177 ? 1.00000 : (j==176 ? 1.00000 : (j==175 ? 1.00000 : (j==174 ? 0.98462 : (j==173 ? 0.96923 : (j==172 ? 0.95385 : (j==171 ? 0.93846 : (j==170 ? 0.92308 : (j==169 ? 0.90769 : (j==168 ? 0.89231 : (j==167 ? 0.87692 : (j==166 ? 0.86154 : (j==165 ? 0.84615 : (j==164 ? 0.83077 : (j==163 ? 0.81538 : (j==162 ? 0.80000 : (j==161 ? 0.78462 : (j==160 ? 0.76923 : (j==159 ? 0.75385 : (j==158 ? 0.73846 : (j==157 ? 0.72308 : (j==156 ? 0.70769 : (j==155 ? 0.69231 : (j==154 ? 0.67692 : (j==153 ? 0.66154 : (j==152 ? 0.64615 : (j==151 ? 0.63077 : (j==150 ? 0.61538 : (j==149 ? 0.60000 : (j==148 ? 0.58462 : (j==147 ? 0.56923 : (j==146 ? 0.55385 : (j==145 ? 0.53846 : (j==144 ? 0.52308 : (j==143 ? 0.50769 : (j==142 ? 0.49231 : (j==141 ? 0.47692 : (j==140 ? 0.46154 : (j==139 ? 0.44615 : (j==138 ? 0.43077 : (j==137 ? 0.41538 : (j==136 ? 0.40000 : (j==135 ? 0.38462 : (j==134 ? 0.36923 : (j==133 ? 0.35385 : (j==132 ? 0.33846 : (j==131 ? 0.32308 : (j==130 ? 0.30769 : (j==129 ? 0.29231 : (j==128 ? 0.27692 : (j==127 ? 0.26154 : (j==126 ? 0.24615 : (j==125 ? 0.23077 : (j==124 ? 0.21538 : (j==123 ? 0.20000 : (j==122 ? 0.18462 : (j==121 ? 0.16923 : (j==120 ? 0.15385 : (j==119 ? 0.13846 : (j==118 ? 0.12308 : (j==117 ? 0.10769 : (j==116 ? 0.09231 : (j==115 ? 0.07692 : (j==114 ? 0.06154 : (j==113 ? 0.04615 : (j==112 ? 0.03077 : (j==111 ? 0.01538 : (j==110 ? 0.00000 : (j==109 ? 0.00000 : (j==108 ? 0.00000 : (j==107 ? 0.00000 : (j==106 ? 0.00000 : (j==105 ? 0.00000 : (j==104 ? 0.00000 : (j==103 ? 0.00000 : (j==102 ? 0.00000 : (j==101 ? 0.00000 : (j==100 ? 0.00000 : (0.0)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))" -sigmoidal-contrast 4.0x50% c:/nswtopo-master/nsw.vegetation-spot5-clut.png

[fmw42]

IM -fx probably allows only a limited number of characters in the fx command or a limited number of chained ternary functions.

Why don't you just make a look up table image and use -clut to map your image rather than the slow -fx. See http://www.imagemagick.org/Usage/color_mods/#clut. You just need to create your 256 values into 1 pixel colors (or grayshades) and append them all together. See http://www.imagemagick.org/Usage/layers/#append. You can make a colortable file of the list as

xc:color0
xc:color1
xc:color2
...
xc:color255

Then feed that file to (unix syntax) --- what is your platform?

Code: Select all

convert image \( @colortable.txt +append \) -clut result

see also color names and values at http://www.imagemagick.org/script/color.php

[cnschulz]

Thanks for that. Also verified not working in ImageMagick 6.9.1-10 Q16 x64 2015-07-25.

The problem with your reply is that I am a total noob and debugging a script someone else has written. Ill forward your message to him. Thanks again.

[fmw42]

Sorry, I did not read your post carefully enough. Looking more closely, it seems that all you want to do is make a reference image, that looks like a look up table. You want to assign a grayvalue to each element of 256 pixel tall image. That is really only the append part of my command above. So you just need to put in each graylevel you want for each of the 256 pixels in a saved file list and use -append to make it a column in stead of a row.

Note that you have to convert your fx values from the range of 0 to 1 to the range 0 to 255 or 0 to 100%. The latter may be better since you just need to multiply by 100. You need a color value (presumably grayscale0 for each of your 0 to 255 input image grayscale values. You can use gray(xx%) for the colors. You also need to fill in the values that you are missing above 200 and below 100 with black I presume, since you are starting with canvas:black (same as xc:black). Or you can make your image have 100 values between 200 and 100 and then append black to the top and bottom afterwards.

Code: Select all

convert @colortable.txt -append result

is all you need to make your image. The color table file would have 256 rows and would contain the following values (where Y increases downward)

Code: Select all

xc:gray(0%) for Y=0
the same for all pixels Y= 1 to 109
xc:gray(0%) for Y=110
xc:gray(1.538%) for Y=111
... all your other values as appropriate in percent
xc:gray(98.462%) for Y=174
xc:gray(100%) for Y=175
the same for all pixels Y=176 to 199
xc:gray(100%) for Y=200
xc:gray(0%) for Y=201
the same for all pixels Y=202 to 254
xc:gray(0%) for Y=255

You could also write a script to process each of your conditions to create the file rather than doing it manually as above.

You have not mentioned what your platform is? IM syntax and scripting is different for Unix and Windows.

[snibgo]

The documentation http://www.imagemagick.org/script/fx.php says:

ternary conditional expression, return value y if x != 0, otherwise z; only one ternary conditional permitted per statement

The OP command is almost exactly equivalent to (Windows BAT syntax):

Code: Select all

convert ^
  -size 1x256 gradient:black-white ^
  -level 43.1640625%%,68.5546875%% ^
  -sigmoidal-contrast 4.0x50%% ^
  v.png

EDIT: This seems to be an exact equivalent:

Code: Select all

convert ^
  -size 1x110 canvas:Black ^
  ( -size 1x66 gradient:black-white ^
    -fx "int(u*66)/65" ) ^
  -size 1x80 canvas:White ^
  -append ^
  v2.png

[fmw42]

I have used chained ternary conditions with success, despite what the docs say. But typically only 2 or three chained ones.

[snibgo]

In fx.c, function FxEvaluateSubexpression is recursive. I see nothing that creates a limit, other than stack depth.

Experiments show that v6.9.1--6 pre-build Windows binary fails at 59 recursions. The Cygwin-compiled v6.9.0-0 fails at 120 recursions.

Code: Select all

f:\web\im>convert rose: -format "%[fx:(9999==1?10:(9999==2?10:(9999==3?10:(9999=
=4?10:(9999==5?10:(9999==6?10:(9999==7?10:(9999==8?10:(9999==9?10:(9999==10?10:(
9999==11?10:(9999==12?10:(9999==13?10:(9999==14?10:(9999==15?10:(9999==16?10:(99
99==17?10:(9999==18?10:(9999==19?10:(9999==20?10:(9999==21?10:(9999==22?10:(9999
==23?10:(9999==24?10:(9999==25?10:(9999==26?10:(9999==27?10:(9999==28?10:(9999==
29?10:(9999==30?10:(9999==31?10:(9999==32?10:(9999==33?10:(9999==34?10:(9999==35
?10:(9999==36?10:(9999==37?10:(9999==38?10:(9999==39?10:(9999==40?10:(9999==41?1
0:(9999==42?10:(9999==43?10:(9999==44?10:(9999==45?10:(9999==46?10:(9999==47?10:
(9999==48?10:(9999==49?10:(9999==50?10:(9999==51?10:(9999==52?10:(9999==53?10:(9
999==54?10:(9999==55?10:(9999==56?10:(9999==57?10:(9999==58?10:(9999==59?10:(999
9==60?10:(9999==61?10:(9999==62?10:(9999==63?10:(9999==64?10:(9999==65?10:(9999=
=66?10:(9999==67?10:(9999==68?10:(9999==69?10:(9999==70?10:(9999==71?10:(9999==7
2?10:(9999==73?10:(9999==74?10:(9999==75?10:(9999==76?10:(9999==77?10:(9999==78?
10:(9999==79?10:(9999==80?10:(9999==81?10:(9999==82?10:(9999==83?10:(9999==84?10
:(9999==85?10:(9999==86?10:(9999==87?10:(9999==88?10:(9999==89?10:(9999==90?10:(
9999==91?10:(9999==92?10:(9999==93?10:(9999==94?10:(9999==95?10:(9999==96?10:(99
99==97?10:(9999==98?10:(9999==99?10:(9999==100?10:(9999==101?10:(9999==102?10:(9
999==103?10:(9999==104?10:(9999==105?10:(9999==106?10:(9999==107?10:(9999==108?1
0:(9999==109?10:(9999==110?10:(9999==111?10:(9999==112?10:(9999==113?10:(9999==1
14?10:(9999==115?10:(9999==116?10:(9999==117?10:(9999==118?10:(9999==119?10:99))
))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))
)))))))))))))))))))))))))))))))))))))]" info:
99

[fmw42]

Interesting that there is a difference in the chain limit. Perhaps that has to do with the number of characters allow in a command or in -fx between window and unix.

[snibgo]

I was running them both in Windows, so no difference there. It may be a difference between v6.9.0-0 and v6.9.1--6, or a difference between the compiler toolsets.

[fmw42]

I thought you said you were using Cygwin for one of the tests. That is unix, is it not?

[snibgo]

I tested all versions by running them from identical Windows BAT scripts.

Pre-built Windows binaries, Q16 non-HDRI, versions v6.9.0-0 and 6.9.1--6, both break at 59 recursions. They fail silently.

The v6.9.0-0, Q16 non-HDRI, I compiled with Cygwin tools breaks at 120 recursions. This fails with the message:

Code: Select all

5 [unknown (0xD48)] convert 3628 cygwin_exception::open_stackdumpfile: Dumping stack trace to convert.exe.stackdump

The stackdump is:

Code: Select all

Exception: STATUS_STACK_OVERFLOW at rip=0043622B3F6
rax=00000000000010D8 rbx=0000000000045740 rcx=0000000000043648
rdx=0000000007FFFFF7 rsi=0000000000047864 rdi=0000000000047860
r8 =0000000000000000 r9 =0000000000000000 r10=00000006000AA568
r11=FEFEFEFEFEFEFEFF r12=00000001801EE300 r13=0000000000000000
r14=0000000000000000 r15=0000000000000004
rbp=0000000000047868 rsp=0000000000045630
program=c:\cygwin64\home\Alan\iminst16i\bin\convert.exe, pid 3628, thread unknown (0xD48)
cs=0033 ds=002B es=002B fs=0053 gs=002B ss=002B
Stack trace:
Frame        Function    Args
00000047868  0043622B3F6 (0043613C536, 00000045740, 00000047864, 00000047860)
00000047868  000000020D8 (00000045740, 00000047864, 00000047860, 00000047868)
00000047868  00600082400 (00000047864, 00000047860, 00000047868, 001801EE300)
00000047868  0043613C536 (00000000000, 00000000000, 00000000000, 00000045740)
00000047868  0043613C7F4 (00000000000, 00000000000, 00000000000, 00000047860)
0000004999C  0043613C7F4 (00000000000, 00000000000, 00000000000, 00000049980)
0000004BAC1  0043613CA2C (00000000000, 00000000000, 00000000000, 0000004BAA3)
0000004DBEA  0043613E77C (00000000000, 00000000000, 00000000000, 0000004DBC0)
0000004FD0F  0043613CA2C (00000000000, 00000000000, 00000000000, 0000004FCE3)
00000051E38  0043613E77C (00000000000, 00000000000, 00000000000, 00000051E00)
00000053F5D  0043613CA2C (00000000000, 00000000000, 00000000000, 00000053F23)
00000056086  0043613E77C (00000000000, 00000000000, 00000000000, 00000056040)
000000581AB  0043613CA2C (00000000000, 00000000000, 00000000000, 00000058163)
0000005A2D4  0043613E77C (00000000000, 00000000000, 00000000000, 0000005A280)
0000005C3F9  0043613CA2C (00000000000, 00000000000, 00000000000, 0000005C3A3)
0000005E522  0043613E77C (00000000000, 00000000000, 00000000000, 0000005E4C0)
End of stack trace (more stack frames may be present)

So, STATUS_STACK_OVERFLOW. Perhaps there is a compiler switch to increase the size of the stack.

[magick]

We can reproduce the problem you posted and have a patch in ImageMagick 6.9.2-1 Beta, available by sometime tomorrow. Thanks.