Black line appears sometimes on images resized by convert

[kouiskas]

I work for deviantArt and we use ImageMagick on a boatload of images, as one might expect from a large traffic website where users can submit an unlimited amount of image files. We have a cluster dedicated to running image conversion jobs. For a while now we've had a bug in ImageMagick haunt us: http://fc01.deviantart.net/fs70/i/2010/ ... urlove.jpg As you can see, that image has a horizontal black line. That line wasn't there on the original JPEG image that was resized to 900x1214 using convert.

The bug isn't reproducible, if we run the same command on the same original image, there's a very high probability that it will succeed and the black line won't be there in the output image anymore. It's somehow linked to server load and could be related to low available memory conditions.

We suspect that it's linked to an internal ImageMagick process that creates two temporary files instead of one while convert runs and will stitch the 2 temporary images into one. We've witnessed that the black line appears where the separation between the two temporary files was. We think that there might be a bug in that stitching step that combines the two temporary files created by IM under specific memory or load conditions. Someone familiar with the ImageMagick source code might know where that stitching process's code is, I'd be happy to take a look myself to try and understand the situation better.

This is what the command we run usually looks like:
convert -limit memory 512 -limit map 1024 -type TrueColor -quality 85 -define jpeg:size=1800x2376 blacklinetest.jpg[900X1188] -strip JPG:test.jpg

We're currently running ImageMagick 6.5.7-4 2009-11-03 Q8. Before someone points out that we should update, we would only do it if someone can show us that this bug is known and has already been addressed in a more recent release. If it's news to the wizards, then there's no point for us to go through the painful and risky process of updating IM on our production image resizing cluster.

[magick]

Post a URL to your original image. We need to reproduce the problem ourselves before we can comment.

[kouiskas]

http://showmeyourlove.deviantart.com/#/d2rgl6f

Click on "download image" on the right hand side of the page to get the original.

[magick]

We're using ImageMagick 6.6.2-7, the current release. We ran this command first;

• convert -limit memory 512 -limit map 1024 -type TrueColor -quality 85 -define jpeg:size=1800x2376 blacklinetest.jpg[900X1188] -strip reference.jpg

and verified there was no black line.

Next we ran these commands 1000 times on a 4 processor Fedora system:

• convert -limit memory 512 -limit map 1024 -type TrueColor -quality 85 -define jpeg:size=1800x2376 blacklinetest.jpg[900X1188] -strip image.jpg
  compare -metric rmse image.jpg reference.jpg null:

Each time compare returned 0db which means that the generated image.jpg exactly matched the reference image meaning no black line was injected.

[amvonet]

We have the same issue. I work for AMVONET (its an elearning virtual class platform). We have the same issue except it is with a ton of different black lines (and it doesnt affect all images, but roughly 95% of the images that are converted.) We only have preview's converted with image magick so the actual images are fine (when previewed or published in our app). So its just the image previews that have the issue and like devianart mentions we can never duplicate it so its super rare. It just happened again today (the 2nd time in like a year or so) let me know if you want a screenshot.

[NicolasRobidoux]

Forgive me for making a wild guess, but could this come from "inconsistent branching when using the results of floating point computations", as discussed in the article by David Monniaux "The pitfalls of verifying floating-point computations," article which is a reference of the Floating point Wikipedia entry (http://en.wikipedia.org/wiki/Floating_point)?

I am referring to the following myth:

If x<1 tests true at one point, then x<1 tests stays true later if I never modify x

(see the second page of the article).

It would explain the possible dependence on load (was x kept in register between two uses or was it booted off and reloaded?).

(Of course I should not make such assertions without checking the source code...)

[magick]

The problem could be one of a thousand things. In general, unless we can reproduce a problem, it is quite speculative to guess where the black lines are introduced. The best path forward would be to identify cases where it fails and provide us with the image and the command line that fails. Even if it fails intermittently, we might be able to reproduce the problem by running the command ten of thousands of times until it fails. If you do come up with a test case, post it here along with the output of these commands:

• convert -list configure
  convert -list format

[NicolasRobidoux]

... We have the same issue except it is with a ton of different black lines (and it doesnt affect all images, but roughly 95% of the images that are converted.) We only have preview's converted with image magick so the actual images are fine (when previewed or published in our app). So its just the image previews that have the issue and like devianart mentions we can never duplicate it so its super rare. It just happened again today (the 2nd time in like a year or so) let me know if you want a screenshot.

You mean that when it happens (which is almost never), 95% of the images being processed at that time have the glitch?

Could we see a screenshot?

[ethaniel]

This is weird, we have a same setup (many server resizing images under heavy load). And we just bumped into the same problem, 4 years later.
Black lines on images. Random black lines.

kouiskas, did you find a solution?

[dlemstra]

Are you using the latest version of ImageMagick?