Random black dots problem
Random black dots problem
Hi guys and gals,
I have a bit of a problem with ImageMagick that I've been wrestling with for a few hours, and I think it's time I got some help.
When editing an image with the imagick PHP extension I occasionally get a seemingly random spattering of black dots on my output image.
For example, I open this image:
(additional links in case of 404:
http://s8.postimg.org/izkvrqgph/c5cdc3e ... b8c0e2.jpg
http://i.imgur.com/ee8Gqy3.jpg)
Might result in something like this:
(additional links in case of 404:
http://s28.postimg.org/kgweuthod/image.jpg
http://i.imgur.com/MrKlz5s.jpg)
Notice the random-ish black dots all across the face. Sometimes this problem occurs, sometimes it doesn't. The black dots are different every time and don't appear to be related to anything in particular, though they do seem to follow areas within the image that have the same colour.
The problem seems to occur more often when I am processing many images at the same time, and less often if I just process a single image.
In the above example I am simply using haldClutImage(), though the same problem occurs using other methods, it really doesn't seem to matter which methods I am using.
For the sake of completeness this is the clut I am using:
(additional links:
http://postimg.org/image/js6bzh9fb/
http://i.imgur.com/mPcXIVO.jpg)
The hald clut was produced some time ago in an earlier version of ImageMagick, I am really hoping the clut isn't the problem as I'm not sure I have the heart to re-do all my cluts!
I am using the following:
PHP Version 5.5.9-1ubuntu4.14
imagick module version 3.3.0
ImageMagick 6.9.3-3 Q16 x86_64 2016-02-06 (as reported in phpinfo() and identify)
I have re-compiled ImageMagick (using default configure settings) to upgrade from the old version in the Ubuntu repository, I have re-compiled the imagick PECL PHP extension also.
I could have sworn I had a similar issue in the past but I can't find it for the life of me, the closest related problem I could find was this one: https://www.imagemagick.org/discourse-s ... hp?t=25554. The end result being that it would be fixed it a subsequent version, so maybe my issue has a different source.
Could anybody give me some pointers or things to try?
I have a bit of a problem with ImageMagick that I've been wrestling with for a few hours, and I think it's time I got some help.
When editing an image with the imagick PHP extension I occasionally get a seemingly random spattering of black dots on my output image.
For example, I open this image:
(additional links in case of 404:
http://s8.postimg.org/izkvrqgph/c5cdc3e ... b8c0e2.jpg
http://i.imgur.com/ee8Gqy3.jpg)
Might result in something like this:
(additional links in case of 404:
http://s28.postimg.org/kgweuthod/image.jpg
http://i.imgur.com/MrKlz5s.jpg)
Notice the random-ish black dots all across the face. Sometimes this problem occurs, sometimes it doesn't. The black dots are different every time and don't appear to be related to anything in particular, though they do seem to follow areas within the image that have the same colour.
The problem seems to occur more often when I am processing many images at the same time, and less often if I just process a single image.
In the above example I am simply using haldClutImage(), though the same problem occurs using other methods, it really doesn't seem to matter which methods I am using.
For the sake of completeness this is the clut I am using:
(additional links:
http://postimg.org/image/js6bzh9fb/
http://i.imgur.com/mPcXIVO.jpg)
The hald clut was produced some time ago in an earlier version of ImageMagick, I am really hoping the clut isn't the problem as I'm not sure I have the heart to re-do all my cluts!
I am using the following:
PHP Version 5.5.9-1ubuntu4.14
imagick module version 3.3.0
ImageMagick 6.9.3-3 Q16 x86_64 2016-02-06 (as reported in phpinfo() and identify)
I have re-compiled ImageMagick (using default configure settings) to upgrade from the old version in the Ubuntu repository, I have re-compiled the imagick PECL PHP extension also.
I could have sworn I had a similar issue in the past but I can't find it for the life of me, the closest related problem I could find was this one: https://www.imagemagick.org/discourse-s ... hp?t=25554. The end result being that it would be fixed it a subsequent version, so maybe my issue has a different source.
Could anybody give me some pointers or things to try?
- fmw42
- Posts: 25562
- Joined: 2007-07-02T17:14:51-07:00
- Authentication code: 1152
- Location: Sunnyvale, California, USA
Re: Random black dots problem
What happens if you do the same in a command line or use the command line with PHP exec()? Does it happen? If not, then I suspect it is some flaw in your environment or Imagick as opposed to Imagemagick. One thing is to be sure you have enough disk space in the Imagemagick TEMP directory. If you are running out of space due to lots of simultaneous processing, then that could be the cause, but most often that shows up as an empty bottom portion of the image. Your random black region issue looks like an overshoot and wrap around of grayscale. I would not expect that if all you are doing is simply a hald-clut, unless your hald image is faulty. Looks like you are using PNG for the hald image. But be sure you do not use JPG. Also what process did you use in creating the hald image. If that processing had overshoot and wrap around in graylevel that that could cause this problem.
How did you create the hald image?
What is your platform? Is this a server?
How did you create the hald image?
What is your platform? Is this a server?
Re: Random black dots problem
The hald image was generated a long time ago, using a command like this image was then modified in Photoshop and saved as a PNG (I don't want my careful colours to be messed up by JPEG compression!). I can't imagine that the hald image is causing the problem, as the problem is intermittent and produces different results every time, though I am prepared to be wrong about this.
I have used these exact hald images in the past without this black dot problem. The project got shelved for some years and has since been resurrected on a new machine.
I have tried running the command and the PHP code (the getImageGeometry() is only there because it is called every time from within the actual PHP application, and I wanted to be inclusive).
In neither of these tests was I able to replicate the problem, indicating that it is indeed something within the web environment that is causing the issue.
As far as I can tell there is plenty of space available for temp files, around 49GB, I can't imagine this isn't enough.
I added a lot of tracing code to the application to log every single call to any Imagick method, in order to verify that there aren't any errant method calls sneaking in somewhere, the output of this log looks like this:
The project is running on a linux virtual machine used as a dev environment,
The virtual machine normally has 2GB of ram assigned, I restarted it with 4GB and it made no difference. "free" command indicates that there is a little over 1GB of RAM unused when at rest.
I performed a test of processing many images and noting which "filters" (as they are referred to in the PHP application) exhibit the black dot problem. I then examined the PHP code for each of the filters to see what the problematic ones have in common, and every single one of the problem filters make use of hald clut, and none of the other filters use hald clut. I think I can say without any doubt that the hald clut is the problem.
I made a gallery of a few images with the same filter applied that exhibit the same problem, just in case the pattern of it might indicate the underlying problem:
(additional links:
https://i.imgsafe.org/331127b.png
http://i.imgur.com/jZMVVGg.png
https://www.imageupload.co.uk/image/cFrs
http://pasteboard.co/1nEnzA6k.png)
I would agree with your hypothesis of the problem being some kind of overshoot in colour levels, my original thought was it being some kind of overflow or clipping.
I look forward to hearing any further thoughts!
Code: Select all
convert hald:8 hald.png
I have used these exact hald images in the past without this black dot problem. The project got shelved for some years and has since been resurrected on a new machine.
I have tried running the command
Code: Select all
convert testinput.jpg clut.png -hald-clut testoutput.png
Code: Select all
<?php
$i = new Imagick();
$i->readImage("testinput.jpg");
$dim = $i->getImageGeometry();
$i->haldClutImage(new Imagick("clut.png"));
$i->writeImage("test-".rand().".png");
?>
In neither of these tests was I able to replicate the problem, indicating that it is indeed something within the web environment that is causing the issue.
As far as I can tell there is plenty of space available for temp files, around 49GB, I can't imagine this isn't enough.
I added a lot of tracing code to the application to log every single call to any Imagick method, in order to verify that there aren't any errant method calls sneaking in somewhere, the output of this log looks like this:
as you can see, nothing exceptional or unexpected.readImage c5cdc3e8fc6475878f9f115164cb58e9a3b8c0e2.jpeg
getImageGeometry
haldClutImage clut.png
writeImage
The project is running on a linux virtual machine used as a dev environment,
Code: Select all
$ uname -a
Linux VirtualBox 3.19.0-32-generic #37~14.04.1-Ubuntu SMP Thu Oct 22 09:41:40 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
I performed a test of processing many images and noting which "filters" (as they are referred to in the PHP application) exhibit the black dot problem. I then examined the PHP code for each of the filters to see what the problematic ones have in common, and every single one of the problem filters make use of hald clut, and none of the other filters use hald clut. I think I can say without any doubt that the hald clut is the problem.
I made a gallery of a few images with the same filter applied that exhibit the same problem, just in case the pattern of it might indicate the underlying problem:
(additional links:
https://i.imgsafe.org/331127b.png
http://i.imgur.com/jZMVVGg.png
https://www.imageupload.co.uk/image/cFrs
http://pasteboard.co/1nEnzA6k.png)
I would agree with your hypothesis of the problem being some kind of overshoot in colour levels, my original thought was it being some kind of overflow or clipping.
I look forward to hearing any further thoughts!
- fmw42
- Posts: 25562
- Joined: 2007-07-02T17:14:51-07:00
- Authentication code: 1152
- Location: Sunnyvale, California, USA
Re: Random black dots problem
It is the temp directory that Imagemagick uses, not your whole disk space. So find where IM is putting your temp files and watch that directory to see if it is getting full, though I doubt that that would cause the problems you see.
Do you have more than one IM version installed. Are you sure that Imagick is using the version of Imagemagick you think it is.
What does Imagick::getVersion return?
Again can you reproduce the problem using command line in PHP exec(). That will tell you if it is an Imagemagick issue or an Imagick issue.
Push comes to shove provide an example input image like the one above and the modified hald image and I can test that with the current IM release 6.9.3.3.
Try
<?php
echo "<pre>";
system("type -a convert");
echo "</pre>";
?>
To see if you have more than one version that your $PATH knows about.
I tried you hald image on the logo: image ten times in a loop and it works perfectly fine.
I suspect your problem is with your environment or with Imagick version. So again try the same thing but in PHP exec().
Also check to see that you have current versions of libjpeg and libpng? Perhaps one is corrupt?
Do you have more than one IM version installed. Are you sure that Imagick is using the version of Imagemagick you think it is.
What does Imagick::getVersion return?
Again can you reproduce the problem using command line in PHP exec(). That will tell you if it is an Imagemagick issue or an Imagick issue.
Push comes to shove provide an example input image like the one above and the modified hald image and I can test that with the current IM release 6.9.3.3.
Try
<?php
echo "<pre>";
system("type -a convert");
echo "</pre>";
?>
To see if you have more than one version that your $PATH knows about.
I tried you hald image on the logo: image ten times in a loop and it works perfectly fine.
I suspect your problem is with your environment or with Imagick version. So again try the same thing but in PHP exec().
Also check to see that you have current versions of libjpeg and libpng? Perhaps one is corrupt?
Re: Random black dots problem
Try
convert clut.png clut.jpg
Now read clut.jpg instead of clut.png. Does that fix the problem?
convert clut.png clut.jpg
Now read clut.jpg instead of clut.png. Does that fix the problem?
Re: Random black dots problem
I'm not completely sure how to determine how to keep an eye on ImageMagicks's temp file creation. I grep'd policy.xml which suggested that the tmp dir is /tmp, I watched the folder while I generated all the images within the application and it didn't look like any temp files are being created there. If you want to give me instructions so I can return some proper information then I would be happy to do so.
I don't think I have multiple versions installed, though it's not impossible. Imagick::getVersion() says
The version that comes from the Ubuntu package repo was 6.7 or so, so I am somewhat sure that I only have 1 version installed as I grabbed the latest source and compiled it myself. I also compiled the Imagick extension to the latest version.
I ran a hald clut convert command using exec from within the web environment in a loop 20 times without any problem. I should note that my previous PHP test (that was successful) was performed via CLI.
I converted the hald clut image to a JPEG and tested that from the web environment and encountered no problems, as a test I also converted the hald clut image to a 8 bits per channel PNG and that was also OK. I tried re-saving the problematic clut image in Photoshop in case it had been corrupted somehow, but it made no difference.
It would seem that the problem is with the 16 bits/channel hald clut, but only within the web environment using Imagick.
I don't think I have multiple versions installed, though it's not impossible. Imagick::getVersion() says
Code: Select all
array (size=2)
'versionNumber' => int 1683
'versionString' => string 'ImageMagick 6.9.3-3 Q16 x86_64 2016-02-06 http://www.imagemagick.org' (length=68)
I ran a hald clut convert command using exec from within the web environment in a loop 20 times without any problem. I should note that my previous PHP test (that was successful) was performed via CLI.
I converted the hald clut image to a JPEG and tested that from the web environment and encountered no problems, as a test I also converted the hald clut image to a 8 bits per channel PNG and that was also OK. I tried re-saving the problematic clut image in Photoshop in case it had been corrupted somehow, but it made no difference.
It would seem that the problem is with the 16 bits/channel hald clut, but only within the web environment using Imagick.
- fmw42
- Posts: 25562
- Joined: 2007-07-02T17:14:51-07:00
- Authentication code: 1152
- Location: Sunnyvale, California, USA
Re: Random black dots problem
Did you check your versions of libjpeg and libpng?
see
see
convert list format
Re: Random black dots problem
Sorry, I overlooked that.fmw42 wrote:Did you check your versions of libjpeg and libpng?
see
convert list format
Code: Select all
$ convert -list format
Format Mode Description
-------------------------------------------------------------------------------
3FR r-- Hasselblad CFV/H3D39II
A* rw+ Raw alpha samples
AAI* rw+ AAI Dune image
AI rw- Adobe Illustrator CS2
ART* rw- PFS: 1st Publisher Clip Art
ARW r-- Sony Alpha Raw Image Format
AVI r-- Microsoft Audio/Visual Interleaved
AVS* rw+ AVS X image
B* rw+ Raw blue samples
BGR* rw+ Raw blue, green, and red samples
BGRA* rw+ Raw blue, green, red, and alpha samples
BGRO* rw+ Raw blue, green, red, and opacity samples
BIE* rw- Joint Bi-level Image experts Group interchange format (2.0)
BMP* rw- Microsoft Windows bitmap image
BMP2* -w- Microsoft Windows bitmap image (V2)
BMP3* -w- Microsoft Windows bitmap image (V3)
BRF* -w- BRF ASCII Braille format
C* rw+ Raw cyan samples
CAL* rw- Continuous Acquisition and Life-cycle Support Type 1
Specified in MIL-R-28002 and MIL-PRF-28002
CALS* rw- Continuous Acquisition and Life-cycle Support Type 1
Specified in MIL-R-28002 and MIL-PRF-28002
CANVAS* r-- Constant image uniform color
CAPTION* r-- Caption
CIN* rw- Cineon Image File
CIP* -w- Cisco IP phone image format
CLIP* rw+ Image Clip Mask
CMYK* rw+ Raw cyan, magenta, yellow, and black samples
CMYKA* rw+ Raw cyan, magenta, yellow, black, and alpha samples
CR2 r-- Canon Digital Camera Raw Image Format
CRW r-- Canon Digital Camera Raw Image Format
CUR* rw- Microsoft icon
CUT* r-- DR Halo
DATA* rw+ Base64-encoded inline images
DCM* r-- Digital Imaging and Communications in Medicine image
DICOM is used by the medical community for images like X-rays. The
specification, "Digital Imaging and Communications in Medicine
(DICOM)", is available at http://medical.nema.org/. In particular,
see part 5 which describes the image encoding (RLE, JPEG, JPEG-LS),
and supplement 61 which adds JPEG-2000 encoding.
DCR r-- Kodak Digital Camera Raw Image File
DCX* rw+ ZSoft IBM PC multi-page Paintbrush
DDS* rw+ Microsoft DirectDraw Surface
DFONT* r-- Multi-face font package (Freetype 2.5.2)
DNG r-- Digital Negative
DPX* rw- SMPTE 268M-2003 (DPX 2.0)
Digital Moving Picture Exchange Bitmap, Version 2.0.
See SMPTE 268M-2003 specification at http://www.smtpe.org
DXT1* rw+ Microsoft DirectDraw Surface
DXT5* rw+ Microsoft DirectDraw Surface
EPDF rw- Encapsulated Portable Document Format
EPI rw- Encapsulated PostScript Interchange format
EPS rw- Encapsulated PostScript
EPS2* -w- Level II Encapsulated PostScript
EPS3* -w+ Level III Encapsulated PostScript
EPSF rw- Encapsulated PostScript
EPSI rw- Encapsulated PostScript Interchange format
EPT rw- Encapsulated PostScript with TIFF preview
EPT2 rw- Encapsulated PostScript Level II with TIFF preview
EPT3 rw+ Encapsulated PostScript Level III with TIFF preview
ERF r-- Epson Raw Format
FAX* rw+ Group 3 FAX
FAX machines use non-square pixels which are 1.5 times wider than
they are tall but computer displays use square pixels, therefore
FAX images may appear to be narrow unless they are explicitly
resized using a geometry of "150x100%".
FITS* rw- Flexible Image Transport System
FRACTAL* r-- Plasma fractal image
FTS* rw- Flexible Image Transport System
G* rw+ Raw green samples
G3* rw- Group 3 FAX
GIF* rw+ CompuServe graphics interchange format
GIF87* rw- CompuServe graphics interchange format (version 87a)
GRADIENT* r-- Gradual linear passing from one shade to another
GRAY* rw+ Raw gray samples
GROUP4* rw- Raw CCITT Group4
H* -w- Image expressed as a 'C/C++' char array
HALD* r-- Identity Hald color lookup table image
HDR* rw+ Radiance RGBE image format
HISTOGRAM* -w- Histogram of the image
HRZ* rw- Slow Scan TeleVision
HTM* -w- Hypertext Markup Language and a client-side image map
HTML* -w- Hypertext Markup Language and a client-side image map
ICB* rw- Truevision Targa image
ICO* rw+ Microsoft icon
ICON* rw- Microsoft icon
IIQ r-- Phase One Raw Image Format
INFO -w+ The image format and characteristics
INLINE* rw+ Base64-encoded inline images
IPL* rw+ IPL Image Sequence
ISOBRL* -w- ISO/TR 11548-1 format
ISOBRL6* -w- ISO/TR 11548-1 format 6dot
JBG* rw+ Joint Bi-level Image experts Group interchange format (2.0)
JBIG* rw+ Joint Bi-level Image experts Group interchange format (2.0)
JNG* rw- JPEG Network Graphics
See http://www.libpng.org/pub/mng/ for details about the JNG
format.
JNX* r-- Garmin tile format
JPE* rw- Joint Photographic Experts Group JFIF format (80)
JPEG* rw- Joint Photographic Experts Group JFIF format (80)
JPG* rw- Joint Photographic Experts Group JFIF format (80)
JPS* rw- Joint Photographic Experts Group JFIF format (80)
JSON -w+ The image format and characteristics
K* rw+ Raw black samples
K25 r-- Kodak Digital Camera Raw Image Format
KDC r-- Kodak Digital Camera Raw Image Format
LABEL* r-- Image label
M* rw+ Raw magenta samples
M2V rw+ MPEG Video Stream
M4V rw+ Raw MPEG-4 Video
MAC* r-- MAC Paint
MAGICK* rw- Predefined Magick Image (LOGO, ROSE, etc.); output same as 'H'
MAP* rw- Colormap intensities and indices
MASK* rw+ Image Clip Mask
MAT rw+ MATLAB level 5 image format
MATTE* -w+ MATTE format
MEF r-- Mamiya Raw Image File
MIFF* rw+ Magick Image File Format
MKV rw+ Multimedia Container
MNG* rw+ Multiple-image Network Graphics (libpng 1.2.50)
See http://www.libpng.org/pub/mng/ for details about the MNG
format.
MONO* rw- Raw bi-level bitmap
MOV rw+ MPEG Video Stream
MP4 rw+ MPEG-4 Video Stream
MPC* rw+ Magick Persistent Cache image format
MPEG rw+ MPEG Video Stream
MPG rw+ MPEG Video Stream
MRW r-- Sony (Minolta) Raw Image File
MSL* --- Magick Scripting Language
MSVG -w+ ImageMagick's own SVG internal renderer
MTV* rw+ MTV Raytracing image format
MVG* rw- Magick Vector Graphics
NEF r-- Nikon Digital SLR Camera Raw Image File
NRW r-- Nikon Digital SLR Camera Raw Image File
NULL* rw- Constant image of uniform color
O* rw+ Raw opacity samples
ORF r-- Olympus Digital Camera Raw Image File
OTB* rw- On-the-air bitmap
OTF* r-- Open Type font (Freetype 2.5.2)
PAL* rw- 16bit/pixel interleaved YUV
PALM* rw+ Palm pixmap
PAM* rw+ Common 2-dimensional bitmap format
PANGO* --- Pango Markup Language
PATTERN* r-- Predefined pattern
PBM* rw+ Portable bitmap format (black and white)
PCD* rw- Photo CD
PCDS* rw- Photo CD
PCL rw+ Printer Control Language
PCT* rw- Apple Macintosh QuickDraw/PICT
PCX* rw- ZSoft IBM PC Paintbrush
PDB* rw+ Palm Database ImageViewer Format
PDF rw+ Portable Document Format
PDFA rw+ Portable Document Archive Format
PEF r-- Pentax Electronic File
PES* r-- Embrid Embroidery Format
PFA* r-- Postscript Type 1 font (ASCII) (Freetype 2.5.2)
PFB* r-- Postscript Type 1 font (binary) (Freetype 2.5.2)
PFM* rw+ Portable float format
PGM* rw+ Portable graymap format (gray scale)
PICON* rw- Personal Icon
PICT* rw- Apple Macintosh QuickDraw/PICT
PIX* r-- Alias/Wavefront RLE image format
PJPEG* rw- Joint Photographic Experts Group JFIF format (80)
PLASMA* r-- Plasma fractal image
PNG* rw- Portable Network Graphics (libpng 1.2.50)
See http://www.libpng.org/ for details about the PNG format.
PNG00* rw- PNG inheriting bit-depth, color-type from original if possible
PNG24* rw- opaque or binary transparent 24-bit RGB (zlib 1.2.8)
PNG32* rw- opaque or transparent 32-bit RGBA
PNG48* rw- opaque or binary transparent 48-bit RGB
PNG64* rw- opaque or transparent 64-bit RGBA
PNG8* rw- 8-bit indexed with optional binary transparency
PNM* rw+ Portable anymap
PPM* rw+ Portable pixmap format (color)
PREVIEW* -w- Show a preview an image enhancement, effect, or f/x
PS rw+ PostScript
PS2* -w+ Level II PostScript
PS3* -w+ Level III PostScript
PSB* rw+ Adobe Large Document Format
PSD* rw+ Adobe Photoshop bitmap
PTIF* rw+ Pyramid encoded TIFF
PWP* r-- Seattle Film Works
R* rw+ Raw red samples
RADIAL-GRADIENT* r-- Gradual radial passing from one shade to another
RAF r-- Fuji CCD-RAW Graphic File
RAS* rw+ SUN Rasterfile
RAW r-- Raw
RGB* rw+ Raw red, green, and blue samples
RGBA* rw+ Raw red, green, blue, and alpha samples
RGBO* rw+ Raw red, green, blue, and opacity samples
RGF* rw- LEGO Mindstorms EV3 Robot Graphic Format (black and white)
RLA* r-- Alias/Wavefront image
RLE* r-- Utah Run length encoded image
RMF r-- Raw Media Format
RW2 r-- Panasonic Lumix Raw Image
SCR* r-- ZX-Spectrum SCREEN$
SCREENSHOT* r-- Screen shot
SCT* r-- Scitex HandShake
SFW* r-- Seattle Film Works
SGI* rw+ Irix RGB image
SHTML* -w- Hypertext Markup Language and a client-side image map
SIX* rw- DEC SIXEL Graphics Format
SIXEL* rw- DEC SIXEL Graphics Format
SPARSE-COLOR* -w+ Sparse Color
SR2 r-- Sony Raw Format 2
SRF r-- Sony Raw Format
STEGANO* r-- Steganographic image
SUN* rw+ SUN Rasterfile
SVG -w+ Scalable Vector Graphics
SVGZ -w+ Compressed Scalable Vector Graphics
TEXT* rw+ Text
TGA* rw- Truevision Targa image
THUMBNAIL* -w+ EXIF Profile Thumbnail
TIFF* rw+ Tagged Image File Format (LIBTIFF, Version 4.0.3)
TIFF64* rw- Tagged Image File Format (64-bit) (LIBTIFF, Version 4.0.3)
TILE* r-- Tile image with a texture
TIM* r-- PSX TIM
TTC* r-- TrueType font collection (Freetype 2.5.2)
TTF* r-- TrueType font (Freetype 2.5.2)
TXT* rw+ Text
UBRL* -w- Unicode Text format
UBRL6* -w- Unicode Text format 6dot
UIL* -w- X-Motif UIL table
UYVY* rw- 16bit/pixel interleaved YUV
VDA* rw- Truevision Targa image
VICAR* rw- VICAR rasterfile format
VID* rw+ Visual Image Directory
VIFF* rw+ Khoros Visualization image
VIPS* rw+ VIPS image
VST* rw- Truevision Targa image
WBMP* rw- Wireless Bitmap (level 0) image
WMV rw+ Windows Media Video
WPG* r-- Word Perfect Graphics
X3F r-- Sigma Camera RAW Picture File
XBM* rw- X Windows system bitmap (black and white)
XC* r-- Constant image uniform color
XCF* r-- GIMP image
XPM* rw- X Windows system pixmap (color)
XPS r-- Microsoft XML Paper Specification
XV* rw+ Khoros Visualization image
Y* rw+ Raw yellow samples
YCbCr* rw+ Raw Y, Cb, and Cr samples
YCbCrA* rw+ Raw Y, Cb, Cr, and alpha samples
YUV* rw- CCIR 601 4:1:1 or 4:2:2
* native blob support
r read support
w write support
+ support for multiple images
Code: Select all
$ dpkg --get-selections | grep jpeg
libjpeg-dev:amd64 install
libjpeg-progs install
libjpeg-turbo-progs install
libjpeg-turbo8:amd64 install
libjpeg-turbo8:i386 install
libjpeg-turbo8-dev:amd64 install
libjpeg8:amd64 install
libjpeg8:i386 install
libjpeg8-dev:amd64 install
libmjpegutils-2.1-0 install
libopenjpeg2:amd64 install
Code: Select all
$ dpkg --get-selections | grep png
libpng12-0:amd64 install
libpng12-0:i386 install
libpng12-dev install
Re: Random black dots problem
Try replacing libpng-1.2.50 with the current libpng-1.2.56. The "1.2.50" that you are running is
probably not really libpng-1.2.50 but rather Ubuntu's "patched" 1.2.50 which might truncate the
palette, due to an incorrectly-applied recent security patch.
probably not really libpng-1.2.50 but rather Ubuntu's "patched" 1.2.50 which might truncate the
palette, due to an incorrectly-applied recent security patch.
Re: Random black dots problem
This might be something I am unable to do. While I have control over the development environment I don't have much control over the production server so I won't be able to replace libpng in that location.glennrp wrote:Try replacing libpng-1.2.50 with the current libpng-1.2.56. The "1.2.50" that you are running is
probably not really libpng-1.2.50 but rather Ubuntu's "patched" 1.2.50 which might truncate the
palette, due to an incorrectly-applied recent security patch.
Is the best guess that the local libpng is faulty? If so, maybe I should just use a different format for my hald cluts? Assuming this is so, what would be a good format to use? It would ideally be something I can open with Photoshop, something lossless, and something with minimal compression so the files can be loaded quickly. TIFF?
Also given the above assumption, is it worth having the cluts in a 16 bits per channel format? Or should I just limit them to 8? It is intended to be applying the cluts to both 8 and 16 bpc images, if that makes any difference...
-
- Posts: 12159
- Joined: 2010-01-23T23:01:33-07:00
- Authentication code: 1151
- Location: England, UK
Re: Random black dots problem
An easy test is: convert the clut to tiff format. (Use "-compress None" to eliminate that variable.) If that never makes the black marks, that's a workaround.
snibgo's IM pages: im.snibgo.com
Re: Random black dots problem
Would you post the result of "pngcheck -v clut.png" (or post clut.png)? From the CLUT image (a JPEG) embedded in your original post, it appears to have more than 256 colors, so the libpng palette bug is probably not relevant.
EDIT: Never mind, I found your link, and here's the pngcheck report:
$ pngcheck -v clut.png | grep -v IDAT
File: clut.png (1073321 bytes)
chunk IHDR at offset 0x0000c, length 13
512 x 512 image, 48-bit RGB, non-interlaced
chunk gAMA at offset 0x00025, length 4: 0.45471
chunk iCCP at offset 0x00035, length 334
profile name = Photoshop ICC Profile, compression method = 0 (deflate)
compressed profile = 311 bytes
chunk cHRM at offset 0x0018f, length 32
White x = 0.31271 y = 0.32912, Red x = 0.64 y = 0.33001
Green x = 0.3 y = 0.6, Blue x = 0.15 y = 0.06001
chunk pHYs at offset 0x001bb, length 9: 2835x2835 pixels/meter (72 dpi)
chunk tIME at offset 0x001d0, length 7: 5 Nov 2013 17:49:02 UTC
chunk tEXt at offset 0x001e3, length 80, keyword: File Name
chunk iTXt at offset 0x0023f, length 14665, keyword: XML:com.adobe.xmp
uncompressed, no language tag
no translated keyword, 14644 bytes of UTF-8 text
chunk tEXt at offset 0x03b94, length 24, keyword: Software
chunk tEXt at offset 0x03bb8, length 15, keyword: Writer
zlib: deflated, 32K window, maximum compression
chunk IEND at offset 0x1060a1, length 0
No errors detected in clut.png (140 chunks, 31.8% compression).
$
So, there is no PLTE chunk, therefore the use of a modified libpng-1.2.50 is not relevant.
There are some ancillary PNG chunks (gAMA, iCCP, cHRM, pHYs, tIME, iTXt, tEXt),
but I don't think any of those have any bearing on the problem.
EDIT: Never mind, I found your link, and here's the pngcheck report:
$ pngcheck -v clut.png | grep -v IDAT
File: clut.png (1073321 bytes)
chunk IHDR at offset 0x0000c, length 13
512 x 512 image, 48-bit RGB, non-interlaced
chunk gAMA at offset 0x00025, length 4: 0.45471
chunk iCCP at offset 0x00035, length 334
profile name = Photoshop ICC Profile, compression method = 0 (deflate)
compressed profile = 311 bytes
chunk cHRM at offset 0x0018f, length 32
White x = 0.31271 y = 0.32912, Red x = 0.64 y = 0.33001
Green x = 0.3 y = 0.6, Blue x = 0.15 y = 0.06001
chunk pHYs at offset 0x001bb, length 9: 2835x2835 pixels/meter (72 dpi)
chunk tIME at offset 0x001d0, length 7: 5 Nov 2013 17:49:02 UTC
chunk tEXt at offset 0x001e3, length 80, keyword: File Name
chunk iTXt at offset 0x0023f, length 14665, keyword: XML:com.adobe.xmp
uncompressed, no language tag
no translated keyword, 14644 bytes of UTF-8 text
chunk tEXt at offset 0x03b94, length 24, keyword: Software
chunk tEXt at offset 0x03bb8, length 15, keyword: Writer
zlib: deflated, 32K window, maximum compression
chunk IEND at offset 0x1060a1, length 0
No errors detected in clut.png (140 chunks, 31.8% compression).
$
So, there is no PLTE chunk, therefore the use of a modified libpng-1.2.50 is not relevant.
There are some ancillary PNG chunks (gAMA, iCCP, cHRM, pHYs, tIME, iTXt, tEXt),
but I don't think any of those have any bearing on the problem.
- fmw42
- Posts: 25562
- Joined: 2007-07-02T17:14:51-07:00
- Authentication code: 1152
- Location: Sunnyvale, California, USA
Re: Random black dots problem
The PNG HALD image is fine as it is. My tests showed no problem in generating the output. Also if you get no issues with using PHP exec() and the command line command in it, then it is not your HALD image, nor your libpng or libjpeg
So what happens if you start with a tiff and save to a tiff? Get a new tiff image and do not convert from jpg or png to tiff. Do you get the same results.
My gut feeling is that it is still in your OS or PHP environment or your Imagick? Have you tried an older version of Imagick?
So what happens if you start with a tiff and save to a tiff? Get a new tiff image and do not convert from jpg or png to tiff. Do you get the same results.
My gut feeling is that it is still in your OS or PHP environment or your Imagick? Have you tried an older version of Imagick?
- fmw42
- Posts: 25562
- Joined: 2007-07-02T17:14:51-07:00
- Authentication code: 1152
- Location: Sunnyvale, California, USA
Re: Random black dots problem
Perhaps the version of Imagemagick you are using is not compatible with the version of Imagick?
Re: Random black dots problem
I'm sorry for the delay in replying, I had some connectivity issues.
But, while I had the down time I did some testing. And I discovered that by converting my cluts to 8bpc TIFFs the problem goes away. Possibly the same would apply if I used 8bpc PNG also, but I haven't tested that, I'm just happy that this particular issue has gone away.
But, while I had the down time I did some testing. And I discovered that by converting my cluts to 8bpc TIFFs the problem goes away. Possibly the same would apply if I used 8bpc PNG also, but I haven't tested that, I'm just happy that this particular issue has gone away.