convert(1) command can't handle multibyte characters

[Sonam]

Hi,

convert(1) command can't handle multibyte characters properly and results the garbled characters.

It's easy to reproduce.. Create a text file which contains multibyte characters and process it with convert(1) as follows.

$ convert label:@test.txt test.png
convert: FreeType library is not available
`/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
convert: FreeType library is not available
`/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
:
$

You will see the garbled characters in test.png.

$ cat text.txt
ÆüËܸì

[fmw42]

convert: FreeType library is not available

Looks like you need to install the FreeType libraray. See http://www.imagemagick.org/download/delegates/

[Sonam]

Freetype Library is already installed on the system.

bash-3.2# pkginfo | grep freetype
system SUNWfreetype2 FreeType2 Font library

bash-3.2# /usr/sfw/bin/convert label:"HELLO" 1.png
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
bash-3.2#

The above command creates a correct png file despite the warning messages.

The issue is only when multibyte characters are used.

This error is seen with both 2byte char(Japanese) is in the text file and 1byte char(alphabet) is in the text file.

convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.

although there is this error, the text file successfully gets converted to png when the text file contains 1byte char(alphabet)

Regards,
Sonam

[fmw42]

I don't know much about Freetype to comment. But if it is not installed in a place where IM can find it, then IM won't recognize that it is there. Perhaps it is not an acceptable version? Just some things to check.

[Sonam]

What should be the correct location of the freetype fonts for IM to work properly ?

Important thing to notice here is that the text file successfully gets converted to png when the text file contains 1byte char(alphabet).
It's not working only with 2byte char(Japanese).

[fmw42]

What should be the correct location of the freetype fonts for IM to work properly ?

Probably the same locations as where you have installed IM.

Perhaps it is sufficient to be in some location that is in your $PATH. I am not really sure.

[gmr]

Any ideas on the issue. The issue is the same as Sonam had mentioned.

bash-3.2# /usr/sfw/bin/convert label:@text_UTF-8.txt out.png
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
bash-3.2# ls /usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z
/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z
bash-3.2#

So the file exists on the system, even though the error says the freetype library is not available.

So to put a straight question, does imagemagick support multi-byte character conversion?
If yes, can I get pointers on what to look for regarding the issue?

Thanks

[Jason S]

does imagemagick support multi-byte character conversion?

Testing ImageMagick 6.8.6-5 on Linux...

Code: Select all

$ echo -e 'X\xe2\x98\xbaX' > label.txt
$ convert label:@label.txt test.png

My result: X☺X
(I do not get any warnings about FreeType.)

So, it interpreted the label file as UTF-8, and the non-Latin character was correctly processed. This confirms that IM does support multi-byte UTF-8 characters, at least in some situations.

Code: Select all

$ echo -e 'X\xa9X' > label.txt
$ convert label:@label.txt test.png

My result: X©X

This file is not valid UTF-8, and IM interprets it as Latin-1 or similar. It must be doing some autodetection of the encoding.

Code: Select all

$ echo -e 'X\xe4\xb8\x8eX' > label.txt
$ convert label:@label.txt test.png

My result: X?X

This is a CJK character, encoded in UTF-8. Since it was displayed as a single question mark instead of three garbage characters, I assume this is a limitation of the font that was used, and not a problem with interpreting the UTF-8 encoding.

[fmw42]

Any ideas on the issue. The issue is the same as Sonam had mentioned.

bash-3.2# /usr/sfw/bin/convert label:@text_UTF-8.txt out.png
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
convert: FreeType library is not available `/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z'.
bash-3.2# ls /usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z
/usr/openwin/lib/X11/fonts/TTbitmaps/Arial-Bold14.pcf.Z
bash-3.2#

So the file exists on the system, even though the error says the freetype library is not available.

So to put a straight question, does imagemagick support multi-byte character conversion?
If yes, can I get pointers on what to look for regarding the issue?

Thanks

If freetype exists, it is not where IM expects to find it. Perhaps your PATH does not point to that location.

To check about IM, on current versions use

convert -version

and see if freetype is listed as delegates

On older systems, type

convert -list configure

look at the line starting with DELEGATES and see if it is listed there.

[gmr]

The running version of imagemagick is 6.8.3-5. "convert -version" lists freetype in the delegates list, as below.

Features: DPC Modules
Delegates: bzlib fftw freetype jng jpeg lcms lzma openexr pango png ps tiff x xml zlib

I also tried the given examples in the previous update ... but I was not able to convert some of them. The copyright symbol was converted correctly. But none of the other characters were converted correctly.

Code: Select all

$ echo -e 'X\xa9X' > label.txt
$ convert label:@label.txt test.png

My result is also as expected : X©X. But, where as the following convert did not yield the result as expected. I got the conversion text as a question mark.

Code: Select all

$ echo -e 'X\xe2\x98\xbaX' > label.txt
$ convert label:@label.txt test.png

Any thoughts on where might be the issue ?? There was a mention of the fonts. How do I check on whether the available fonts are an issue?

Thanks for the help!
gmr!

[fmw42]

convert label:@label.txt test.png

The text file must support UTF8.

[magick]

And the font must support the Unicode character set such as Arial Unicode, and extended MS font.

[Jason S]

Use the -font and -pointsize options to select a font that contains the characters you need. "convert -list font" will list the known fonts.

[gmr]

Updating to have the closure remarks.

Thanks. Having explicitly pointing to the font file, did the trick.