Convert 2-bit to png after deskew has odd results

[muccigrosso]

On Q16, with TIFF, the results are different.

Code: Select all

convert xc: xc:black +append t.tif

identify -verbose says 16/1-bit (2 colors)

but

Code: Select all

convert t.tif -resize 1000x1! t2.tif

identify -verbose 16-bit grayscale with 644 colors.

So I think the resulting bit depth and colors depends upon the output format.

The point is, if you create more colors than your input bit-depth and the output format supports it, you will get a higher bit depth result with more colors.

One more thing, if you instead force the depth on that first image to 1 and then resize and show it, all is well and you see all the grays, even though the reported depth is 1-bit sRGB. (I thickened up the bar to make it easier to see.)

Code: Select all

convert xc: xc:black +append -depth 1 -resize 1000x100! show:

But if you write that to a png or tiff instead of showing it, then you're back to black and white, as the output files are 1-bit like the original.

What's more, the tiff and png have different rules for deciding which pixels go black and which white. The tiff divides the bar in half, while the png creates more white (3/4?).

[fmw42]

So is there a question?

[muccigrosso]

So is there a question?

Well, yeah, a few.

I for one don't quite have clarity on this. In particular, why doesn't using standard in/out not preserve the bit-depth of the intermediate image? If that's by design, is it noted somewhere?

Also, as you wrote:

The point is, if you create more colors than your input bit-depth and the output format supports it, you will get a higher bit depth result with more colors.

But it seems that the output format remembers the bit depth of the input file, even if that format supports higher bit depths, so that in my case the output tiff and png file have 1-bit depth, even though both those formats can handle a lot more.(I'd call them the same format, even if the depth varies.) Again, by design? And is it documented somewhere?

Third, why does the intermediate file in the test case when seen with show: contain grays, yet info: reports only 1-bit depth? If info: is like doing a file output, OK, but why then isn't show:? In the absence of a -depth option, show: in fact shows an image that you won't get if you write to a file, unless you use a filetype that forces the increased bit depth.

Finally why do the 1-bit tiff and png test-case files look different? They appear to be using different thresholds for creating black pixels. Why?

Thanks for the replies. I'm learning a lot.

[fmw42]

Please write your questions with respect to commands that you feel are not working as we discussed above.

Imagemgick will preserve bit depth if you do not add more colors than that bit depth allows and the output format supports that. We showed that the number of colors increases to 644, you get either 8-bit or 16-bit output depending upon PNG or TIFF

What intermediate file? What was the command and how did you test the intermediate file for bit-depth. Best to look at identify -verbose for that.

Which command created the 1-bit tiff and 1-bit test case?

See the top-most post in this forum "IMPORTANT: Please Read This FIRST Before Posting" at http://www.imagemagick.org/discourse-se ... f=1&t=9620 for how to ask good questions. Try to ask a question with a command line and possible input and output or some means that we can reproduce your issue.

[muccigrosso]

Please write your questions with respect to commands that you feel are not working as we discussed above.

Imagemgick will preserve bit depth if you do not add more colors than that bit depth allows and the output format supports that. We showed that the number of colors increases to 644, you get either 8-bit or 16-bit output depending upon PNG or TIFF

What intermediate file? What was the command and how did you test the intermediate file for bit-depth. Best to look at identify -verbose for that.

Which command created the 1-bit tiff and 1-bit test case?

See the top-most post in this forum "IMPORTANT: Please Read This FIRST Before Posting" at http://www.imagemagick.org/discourse-se ... f=1&t=9620 for how to ask good questions. Try to ask a question with a command line and possible input and output or some means that we can reproduce your issue.

Again, I appreciate the help, so let me try again. Questions are in bold.

1. In the simple conversion after deskew case with the files I linked to above:

Code: Select all

convert -deskew 40% test.pgm test_deskew.png

the output png returns to the bit depth of the original pgm file (1 bit), even though png as a format can handle 8 bits.

Does that mean that if a format can handle the bit depth of the original file, it will be limited to that, even if the that means throwing away info?

That's a little different from "if you create more colors than your input bit-depth and the output format supports it, you will get a higher bit depth result with more colors", since png does support 8 bits, but in this case the output file doesn't have it.

2. You can see a similar thing in my modification of the example case you gave. In your example:

Code: Select all

convert xc: xc:black +append t.tiff

the bit depth of the output t.tiff is 16-bit Grayscale Gray, so that's not quite analogous to what I had, since we start out here with a high bit depth. So I forced the original depth to 1 to mimic the pgm I started with:

Code: Select all

convert xc: xc:black +append -depth 1 -resize 1000x100! <output>

Now, if I make the output "show:", the image has lots of grays in it. Same thing if I make the output a jpg. If I instead make the output "info:" or "-write mpr:output" or a png or tiff file, the result is 1-bit sRGB in the case of "info:" or the png/tiff output image is black and white (1 bit). If I write the mpr:output to a png or tiff file, I get the same results as just writing the file directly.

Why does "show:" preserve the increased depth? jpg has to be 8-bit, so that's why that file retains the depth does. Same with show:? I'd have thought not.

Also why does a simple info: say "1-bit sRGB", but -verbose output of the same shows a histogram with lots of grays and a depth of "1/16-bit"?

3. In the case of the output png and tiff, the commands are

Code: Select all

convert xc: xc:black +append -depth 1 -resize 1000x100! output.png
convert xc: xc:black +append -depth 1 -resize 1000x100! output.tiff

Why do the output files have different patterns of black and white? Specifically, the tiff has more black (50% threshold, it looks like), while png has less (75% threshold?).


So, overall, here's my current understanding of the process:

1. Start with a 1-bit image.
2. Process it in some way and thereby increase the bit depth. (Above, I used deskew, you used a resize, and I also tested with blur.)
3. Output the result to a format that can support 1-bit depth, and the extra depth goes away. These include png and tiff (and info:?). These formats do not all reduce the depth in the same way, so that, for example, the resultant png looks different from the resultant tiff.
4. Output it to a format that can't support 1-bit depth, and you get higher depth, as in jpg (and show:?).

[fmw42]

1)

Code: Select all

convert test.pgm -deskew 40% test.png

results in 16/1-bit

There were no increase in the number of colors in the output that would warrant an increase in depth. So IM keeps the depth.


2) Your command creates a two pixel black and white image.

Code: Select all

convert xc: xc:black +append t.tiff

So it is still bilevel 1-bit. So no change in depth 16/1-bit.


3) Improper syntax. Try

convert xc: xc:black +append -resize 1000x100! -depth 1 output.png
convert xc: xc:black +append -resize 1000x100! -depth 1 output.tiff

They produce the same results for me. I am not sure why your command gives different results. But the depth should be set last. Possibly difference in how libpng and libtif behave. IM relies upon these delegates to read and write files.



Deskew apparently did not increase the number of colors. Whereas, -resize will due to resampling from neighboring pixels.


Use identify -verbose on your images to see about bit-depth and number of colors.

[muccigrosso]

1)

Code: Select all

convert test.pgm -deskew 40% test.png

results in 16/1-bit

There were no increase in the number of colors in the output that would warrant an increase in depth. So IM keeps the depth.

Well, this is the crux of the question. Yes, the resultant png here still has two colors, but the deskewed image should have some grays, shouldn't it? Here's a screen shot of the result of the command but with -depth 8 as the final step, where you can clearly see the grays on the letter edges. If I move the depth option to before the deskew, I get exactly the same result.

Code: Select all

convert -deskew 40% test.pgm -depth 8 test_depth.png

Try a clearer case: apply a blur to the pgm, so you know there's some gray and then save it directly to a png or tiff. It'll be 1-bit: the png is all white and the tiff has some black blobs (consistent with the different thresholds). View it with show: instead or save to jpg and it'll be nice and gray like blurred images are.

I'm not sure exactly what IM is basing it on, but clearly it's returning to the bit-depth of the original image for some formats, even though the resultant image has a greater bit depth.

2) Your command creates a two pixel black and white image.

Code: Select all

convert xc: xc:black +append t.tiff

So it is still bilevel 1-bit. So no change in depth 16/1-bit.

Deskew apparently did not increase the number of colors. Whereas, -resize will due to resampling from neighboring pixels.

Use identify -verbose on your images to see about bit-depth and number of colors.

This was your command and yes, that's right, it's 1-bit. I don't understand all the variations in these depth and colorspace and type values that identify returns with and without -verbose, but if I stick in a -depth 1 at the end of your command sequence, I get Type: Bilevel which perhaps is making the difference and causing the resultant file to go back to 1-bit. Clearly something in these files is making a difference.

One more test…create a file with your command, then make a copy with -depth 1:

Code: Select all

convert xc: xc:black +append t_original.tiff
convert t_original.tiff -depth 1 t_one.tiff

Now resize both of them:

Code: Select all

for i in t*.tiff; do convert $i -resize 1000x100! "$i.new.tiff"; done

The output from the 1-bit file is now black and white, but from the original file it is gray. Identify results for the two output files are:

Code: Select all

original: TIFF 2x1 2x1+0+0 16-bit Grayscale Gray 262B 0.000u 0:00.000
1-bit   : TIFF 2x1 2x1+0+0 1-bit Bilevel Gray 260B 0.000u 0:00.000

Identify -verbose shows relevant differences only here, with the original file on the right:

3) Improper syntax. Try

Code: Select all

convert xc: xc:black +append -resize 1000x100! -depth 1 output.png
convert xc: xc:black +append -resize 1000x100! -depth 1 output.tiff

They produce the same results for me. I am not sure why your command gives different results. But the depth should be set last. Possibly difference in how libpng and libtif behave. IM relies upon these delegates to read and write files.

Yes, mine look the same if I do this. I'm often confused about command order, but doesn't what you do here force the conversion to 1-bit after the resize vs mine which does the depth reduction before resizing? It also - I assume - uses the same routine to reduce the depth. To make it clearer, how about we write out to a file at depth 1, then resize:

Code: Select all

convert xc: xc:black +append -depth 1 inter.pgm
convert inter.pgm -resize 1000x100! output.png
convert inter.pgm -resize 1000x100! output.tiff

Now this looks like my original example, where the png and tiff are different.

Again, I don't know what exactly is going on, but IM is clearly using something about the input file to determine the bit depth of the output file, and using show: hides that fact in some cases because show: shows the full depth of the processed image no matter what the input image's depth.

[fmw42]

Your first command with -deskew does not add any new colors. They are just black and white. PGM is a linear gray whereas PNG is non-linear. That is why you see some changes in the results. But no new colors are added, just blacks and whites. There is no blurring.

The bit depth that any format supports is only relevant if more colors are or the output format does not support the lower bit-depth.

Type of Bilevel just means it is 1-bit (black and white). No other gray colors.


This command

Code: Select all

convert xc: xc:black +append t.tiff

Is just appending a white pixel with a black pixel. TIFF supports bilevel 1-bit, so it gets the lowest bit depth that is needed to support the number of colors. In this case -depth 1 makes no difference since the data is only 1-bit.

Resizing adds new colors due to the need to do a weighted average of groups of input pixels to get one output pixel. If the resulting number of colors is more that black and white, the image will be increased to 8-bit or 16-bit as we demonstrated before. Your resize command is not just black and white any more.

Code: Select all

convert t.tiff -resize 1000x100! new.tiff

You can see that from the number of colors listed in identify -verbose new.tiff Colorspace: Gray
Type: Grayscale
Endianess: LSB
Depth: 16-bit
Channel depth:
gray: 16-bit
Colors: 644

644 colors is more than 256, so it becomes 16-bit for Tiff.

If you add -depth 1, it will threshold the image somewhere (hopefully at 50%) and you will get back only black and white pixels. So the result will become 1-bit, since tiff supports 1-bit (bilevel) images. JPG for example only supports 8-bits per pixel per channel.

You can achieve different depths either by using -depth or by using -type. For example -depth 1 and -type bilevel will produce 1-bit result.

Whether you can use one or the other or neither depends upon the output format and what it supports. See https://www.imagemagick.org/Usage/formats/

[snibgo]

I'll just make a point about this:

... but doesn't what you do here force the conversion to 1-bit after the resize vs mine which does the depth reduction before resizing?

"-depth 1" doesn't change any data, it doesn't reduce the data to 1 bit. It is a setting that is used when writing some output formats. We can demonstrate this with "-depth 1 -depth 8" in a command. The first "-depth" will be ignored.

Provided "-depth" occurs in the command before the write, it doesn't actually matter where in the command it occurs. However, for readability, it is best placed immediately before the write.


On writing to stdout: I don't think this makes any difference to the depth. What matters is the format. If you write to "-" with no format specified (eg by writing to "PNG:-") then the output format will be the same as the input format.

[muccigrosso]

I'll just make a point about this:

... but doesn't what you do here force the conversion to 1-bit after the resize vs mine which does the depth reduction before resizing?

"-depth 1" doesn't change any data, it doesn't reduce the data to 1 bit. It is a setting that is used when writing some output formats. We can demonstrate this with "-depth 1 -depth 8" in a command. The first "-depth" will be ignored.

It might not change the data, but it seems to change the output file, no? And the properties of that output file affect what it looks like after processing. Isn't that what my example with and without -depth 1 shows?

That is, two images with the same pixels produce different outputs after processing.

Let's create two pixel-identical files:

Code: Select all

> convert xc: xc:black +append t_original.tiff
> identify t_original.tiff
t_original.tiff TIFF 2x1 2x1+0+0 16-bit Grayscale Gray 262B 0.000u 0:00.000
> convert t_original.tiff -depth 1 t_one.tiff
> identify t_one.tiff t_one.tiff TIFF 2x1 2x1+0+0 1-bit Bilevel Gray 260B 0.000u 0:00.000
t_one.tiff TIFF 2x1 2x1+0+0 1-bit Bilevel Gray 260B 0.000u 0:00.000

This creates two identical images, right? But they have different color features (for lack of a better term) according to identify.

Now, process each of them by resizing and saving to another tiff file with exactly the same command:

Code: Select all

convert t_original.tiff -resize 1000x100! t_original_new.tiff
convert t_one.tiff -resize 1000x100! t_one_new.tiff

These output files are NOT the same. The one made from the original file preserves the grays, while the one made from the depth-reduced file does not; it's black and white.

This is my only point here: IM decides on the depth of the output file based on the depth (or something) of the input file. Or am I missing something?

Provided "-depth" occurs in the command before the write, it doesn't actually matter where in the command it occurs. However, for readability, it is best placed immediately before the write.


On writing to stdout: I don't think this makes any difference to the depth. What matters is the format. If you write to "-" with no format specified (eg by writing to "PNG:-") then the output format will be the same as the input format.

So by "format" we also mean "bit depth" here, right? For example, png has a potential bit-depth of 16 bits, yet in our case IM writes to a png that has a depth 1 bit.

On where in the command the depth setting occurs, the data suggest otherwise. Again, not that the pixels are different, but that the resultant depth/color/type is different so that subsequent modifications do not have the same effect. An example…

First create a file and then resize it. This should create a file with lots of grays:

Code: Select all

> convert xc: xc:black +append -resize 1000x100! test_depth.png
> identify test_depth.png
test_depth.png PNG 1000x100 500x100+0+0 16-bit Grayscale Gray 1518B 0.000u 0:00.000

Yep, visual inspection shows a file with a nice gradation of grays.

Now, convert the file to depth 1 along the way and that changes. First before and then after the resizing:

Code: Select all

> convert xc: xc:black +append -depth 1 -resize 1000x100! test_depth_first.png
> identify test_depth_first.png
test_depth_first.png PNG 1000x100 500x100+0+0 8-bit Gray 2c 349B 0.000u 0:00.000
> convert xc: xc:black +append -resize 1000x100! -depth 1 test_depth_last.png
> identify test_depth_last.png
test_depth_last.png PNG 1000x100 500x100+0+0 8-bit Gray 2c 349B 0.000u 0:00.000

In the first example, the black is 25% of the bar, but in the second example, it's 50% (= threshold for conversion to black is different).

So, unless I'm missing something, the -depth 1 option makes a difference and it also matters where it occurs. That is, pixel-identical images produce different output depending on their color/depth properties (which I admit to not really understanding).

[muccigrosso]

Your first command with -deskew does not add any new colors. They are just black and white. PGM is a linear gray whereas PNG is non-linear. That is why you see some changes in the results. But no new colors are added, just blacks and whites. There is no blurring.

OK, let's blur the file instead of deskewing it:

1. Take my original test.pgm and blur it.

Code: Select all

convert -blur 0,8 test.pgm show:

Here's what it looks like with show: (which I'm suggesting shows all the grays):



As expected there are a lot of grays.

2. Instead of sending to show:, save this as another file, a png:

Code: Select all

convert -blur 0,8 test.pgm blur.png

The output file is totally white (because the grays are too light to be converted to black). The output file is listed by identify as

Code: Select all

1-bit Gray

while the conversion without saving to file, but instead to "info:" says:

Code: Select all

8-bit Gray 2c

So writing to a png removes the increased depth and sets the threshold a little high.

Now if I convert instead to tiff:

Code: Select all

convert -blur 0,8 test.pgm blur.tiff

I get a file that has some black blobs in it, again consistent with the way IM seems to set a lower threshold for tiff output. identify says:

Code: Select all

1-bit Bilevel Gray

So, again, the increased depth is thrown away and pixels converted to black/white depending on a different algorithm.

Converting to jpg instead of png or tiff results in a file which has all the grays in it.

In sum, when I take this 1-bit pgm file and perform an action on it which increases the bit depth (now, a blur), the resulting output in png or tiff is returned to the original 1-bit depth, with different thresholds for conversion to black. If I convert that pgm to 8-bit and then do the blur, the grays are preserved in the final png/tiff.

My question: is this expected behavior (throwing away the increased depth because the input file's depth is lower)?

Also: do you see the source of my confusion?

[snibgo]

If you care about where the threshold occurs, you should use "-threshold 50%" or whatever. Relying on the file-writer part of IM to threshold 1-bit in a particular way seems unwise.

If you use "-threshold 50%", you can put "-depth 1" anywhere you want (as far as I can see) and get the same result.

If you just put "-depth 1" with no threshold, you are effectively saying you don't care where the threshold is.

[muccigrosso]

If you care about where the threshold occurs, you should use "-threshold 50%" or whatever. Relying on the file-writer part of IM to threshold 1-bit in a particular way seems unwise.

If you use "-threshold 50%", you can put "-depth 1" anywhere you want (as far as I can see) and get the same result.

If you just put "-depth 1" with no threshold, you are effectively saying you don't care where the threshold is.

The threshold isn't really what concerns me, though I find it interesting that the two converters use different thresholds.

My confusion stems from the fact that for some outputs IM reduces the bit depth of the output image based on the original image's characteristics. Note too that those characteristics are not limited to the file format, at least as I understand that word. Two input tiff files can produce different output depending on their color characteristics. See the screen shot of the verbose identify output on my two files up above. I would have said that tiff is the format, but maybe I'm using that word incorrectly.

So, to beat this dead horse:

1. Is this influence of the input file the expected behavior?
2. Is the expected behavior (whatever it is) documented somewhere? I thought I had read that when pixels get altered, the output file won't necessarily match the input, so in this case I expected the output to automatically go to an increased bit depth.

Further, the Usage page on file handling says this about show:

These are special output formats that will which will directly display the image result to your screen. Instead of saving the image into a file, it just displays the result.

But this isn't quite true here as show: provides the entire bit depth, unlike the output files in png and tiff format.

3. Is this the expected behavior of show:?

I can work around this easily by just specifying the output bit depth I want, but I'm trying to understand what IM is doing.

Again, thanks.

[snibgo]

... so in this case I expected the output to automatically go to an increased bit depth.

That expectation will often not be met. When we care about the bit depth of the output, and we don't know for sure that the input bit depth is sufficient, we should use "+depth" or whatever.

When the input has depth 1 (whether the format is tiff or png or whatever), the output is usually 1 but might be more. If we care about the output bit depth, we should set it.

When an image is sent to "show:", it isn't first saved as tiff or png or whatever, so doesn't undergo any bit-reductions according to those file formats. As far as I know, "show:" works as expected.

[fmw42]

Post your input and output images and exact command line. Note that deskew will add a background color and the edges of your original image will likely be interpolated to new colors.