file size of jpeg images created on x86_64 and s390x differ

[Petr]

Code: Select all

x86_64$ convert test.jpg test1.x86_64.jpg
x86_64$ ls -l test1.x86_64.jpg
-rw-r--r-- 1 linux users 715665 Jan 22 10:27 test1.x86_64.jpg
x86_64$

Code: Select all

s390x$ convert test.jpg test1.s390x.jpg
s390x$ ls -l test1.s390x.jpg
-rw-r--r-- 1 linux users 715527 Jan 22 04:24 test1.s390x.jpg
s390x$

For test.jpg can stand e. g. git/cups/doc/images/smiley.jpg, and my original testcase was real photo I can provide. Resulting images viewed on x86_64 via display do NOT look different at the first human-eye sight.

Is that expected? Can be and can be proved in some way that resulting images are equal despite different file sizes? I did

Code: Select all

s390x$ convert test1.s390x.jpg test1.s390x.png

and

Code: Select all

x86_64$ convert test1.s390x.jpg test1.s390x.png

but, according to hexdump, test1.s390x.png and test1.x86_64.png differ in IDAT chunk (and in file sizes too).

[Petr]

According to jpegdump, resulting images differ in Huffman tables.

Code: Select all

$ jpegdump -v -v test1.x86_64.jpg > test1.x86_64.jpg.dump
$ jpegdump -v -v test1.s390x.jpg > test1.s390x.jpg.dump
$ diff -a -upr test1.{x86_64,s390x}.jpg.dump
--- test1.x86_64.jpg.dump	2015-01-20 16:11:43.055645497 +0100
+++ test1.s390x.jpg.dump	2015-01-20 16:11:31.048497603 +0100
@@ -1,5 +1,5 @@
 
-test1.x86_64.jpg:
+test1.s390x.jpg:
 offset $0 SOI
 offset $2 APP0 (length 16)
   JFIF version 0101, 300 x 300 dpi
@@ -365,13 +365,13 @@ offset $5b75 SOF0 (baseline DCT Huffman)
 offset $5b88 DHT (length 30)
   table 0
     bits  1 (codes=  0) 
-    bits  2 (codes=  1) $05 
-    bits  3 (codes=  5) $00 $02 $03 $04 $06 
-    bits  4 (codes=  1) $01 
-    bits  5 (codes=  1) $07 
-    bits  6 (codes=  1) $08 
-    bits  7 (codes=  1) $09 
-    bits  8 (codes=  1) $0a 
+    bits  2 (codes=  0) 
+    bits  3 (codes=  7) $00 $01 $02 $03 $04 $05 $06 
+    bits  4 (codes=  1) $07 
+    bits  5 (codes=  1) $08 
+    bits  6 (codes=  1) $09 
+    bits  7 (codes=  1) $0a 
+    bits  8 (codes=  0) 
     bits  9 (codes=  0) 
     bits 10 (codes=  0) 
     bits 11 (codes=  0) 
@@ -393,17 +393,17 @@ offset $5ba8 DHT (length 87)
     bits  9 (codes=  5) $08 $15 $23 $52 $71 
     bits 10 (codes=  4) $33 $62 $81 $91 
     bits 11 (codes=  8) $16 $24 $72 $82 $a1 $b1 $c1 $f0 
-    bits 12 (codes=  4) $09 $43 $92 $d1 
-    bits 13 (codes=  4) $a2 $b2 $e1 $f1 
-    bits 14 (codes=  2) $17 $25 
+    bits 12 (codes=  3) $43 $92 $d1 
+    bits 13 (codes=  7) $09 $17 $25 $a2 $b2 $e1 $f1 
+    bits 14 (codes=  0) 
     bits 15 (codes=  2) $34 $53 
-    bits 16 (codes= 19) $c2 $d2 $e2 $f2 $18 $26 $44 $63 $0a $35 $73 $27 $54 $83 $45 $55 $65 $84 $93 
+    bits 16 (codes= 19) $c2 $d2 $e2 $f2 $18 $26 $44 $63 $0a $35 $73 $27 $45 $83 $54 $55 $65 $84 $93 
 offset $5c01 DHT (length 28)
   table 1
     bits  1 (codes=  0) 
     bits  2 (codes=  3) $00 $02 $03 
-    bits  3 (codes=  1) $04 
-    bits  4 (codes=  1) $01 
+    bits  3 (codes=  1) $01 
+    bits  4 (codes=  1) $04 
     bits  5 (codes=  1) $05 
     bits  6 (codes=  1) $06 
     bits  7 (codes=  1) $07 
@@ -427,17 +427,17 @@ offset $5c1f DHT (length 69)
     bits  7 (codes=  3) $05 $41 $42 
     bits  8 (codes=  5) $14 $51 $52 $61 $71 
     bits  9 (codes=  6) $23 $81 $91 $a1 $b1 $f0 
-    bits 10 (codes=  5) $06 $62 $c1 $d1 $e1 
-    bits 11 (codes=  4) $15 $33 $72 $f1 
-    bits 12 (codes=  2) $24 $82 
-    bits 13 (codes=  1) $43 
-    bits 14 (codes=  3) $16 $34 $92 
-    bits 15 (codes=  2) $07 $53 
-    bits 16 (codes=  7) $a2 $c2 $b2 $d2 $25 $35 $73 
+    bits 10 (codes=  6) $06 $33 $62 $c1 $d1 $e1 
+    bits 11 (codes=  3) $15 $72 $f1 
+    bits 12 (codes=  0) 
+    bits 13 (codes=  2) $24 $82 
+    bits 14 (codes=  0) 
+    bits 15 (codes=  2) $43 $16 
+    bits 16 (codes= 11) $92 $07 $34 $35 $53 $a2 $c2 $25 $b2 $d2 $73 
 offset $5c66 SOS (length 12)
   components 3
     id 1 dc table 0, ac table 0
     id 2 dc table 1, ac table 1
     id 3 dc table 1, ac table 1
   spectral selection 0 to 63, bit position high 0, low 0
-offset $aeb8f EOI
+offset $aeb05 EOI

When -define jpeg:optimize-coding=false parameter to convert is used, then Huffman tables do not differ.

Code: Select all

--- test1.x86_64.jpg.dump	2015-01-20 16:26:52.177027461 +0100
+++ test1.s390x.jpg.dump	2015-01-20 16:26:53.842053605 +0100
@@ -1,5 +1,5 @@
 
-test1.x86_64.jpg:
+test1.s390x.jpg:
 offset $0 SOI
 offset $2 APP0 (length 16)
   JFIF version 0101, 300 x 300 dpi
@@ -440,4 +440,4 @@ offset $5d38 SOS (length 12)
     id 2 dc table 1, ac table 1
     id 3 dc table 1, ac table 1
   spectral selection 0 to 63, bit position high 0, low 0
-offset $b4b31 EOI
+offset $b48e0 EOI

Nevertheless even then file size of resulting image differs as can be also seen from EOI offsets difference, if I understand correctly.

[Petr]

Original test was performed with
ImageMagick 6.8.8-1
libjpeg-turbo 1.3.1

I tested also with
ImageMagick 6.8.8-1
jpeg 9a

With this setup, I get smiley.jpg converted to the same image (checked with md5sum). That is not true for my original (real photo) testcase:

Code: Select all

-rw-r--r-- 1 linux users  715719 Jan 22 08:19 test1.s390x.IJG.jpg
-rw-r--r-- 1 linux users  715527 Jan 22 04:24 test1.s390x.jpg
-rw-r--r-- 1 linux users  715681 Jan 22 14:19 test1.x86_64.IJG.jpg
-rw-r--r-- 1 linux users  715665 Jan 22 10:27 test1.x86_64.jpg

where *IJG* was converted with jpeg 9a and others with libjpeg-turbo.

[snibgo]

If you are converting identical input files (and the filenames are identical) on different machines but with identical versions of IM (including all libraries), I would expect the output JPEG files to be identical.

I don't understand your tests. Are you saying that, with these conditions, you get different results?

[Petr]

Yes. The resulting files differs in file size.

[fmw42]

Do you have the same versions of your delegate libraries (esp libjpeg) on both systems?

[Petr]

Yes.

[Petr]

Namely: original test with /usr/lib64/libjpeg.so.8.0.2 from libjpeg-turbo project on both s390x and x86_64. Second test (comment from 2015-01-22T14:32:14+01:00) was trough /usr/lib64/libjpeg.so.9.1.0 from IJG jpeg library and ImageMagick built against it (again, on both s390x and x86_64).

[fmw42]

Are you forcing the quality to the same value on both systems by using -quality XX, to be sure they two systems use the same compression?

[Petr]

Code: Select all

x86_64$ for i in 10 30 50 100; do convert test.jpg -quality $i test1.$i.jpg; ls -l test1.$i.jpg; done
-rw-r--r-- 1 linux users 75854 Jan 27 11:15 test1.10.jpg
-rw-r--r-- 1 linux users 153799 Jan 27 11:15 test1.30.jpg
-rw-r--r-- 1 linux users 215478 Jan 27 11:15 test1.50.jpg
-rw-r--r-- 1 linux users 1170498 Jan 27 11:15 test1.100.jpg

but

Code: Select all

s390x$ for i in 10 30 50 100; do convert test.jpg -quality $i test1.$i.jpg; ls -l test1.$i.jpg; done
-rw-r--r-- 1 linux users 75854 Jan 27  2015 test1.10.jpg
-rw-r--r-- 1 linux users 153799 Jan 27  2015 test1.30.jpg
-rw-r--r-- 1 linux users 215457 Jan 27  2015 test1.50.jpg
-rw-r--r-- 1 linux users 1170468 Jan 27  2015 test1.100.jpg

[Petr]

If that would be interesting for you, I tried the differences for quality=`seq 1 100`, and I get the following. First column is quality percent and second number is file size.

Code: Select all

--- s390x.txt	2015-01-27 11:33:37.132312580 +0100
+++ x86_64.txt	2015-01-27 11:33:20.876111770 +0100
@@ -10,92 +10,92 @@
 10 75854
 11 80157
 12 84067
-13 88451
+13 88449
 14 92433
 15 96950
-16 100357
+16 100366
 17 104702
-18 109013
+18 109005
 19 112482
 20 115672
 21 119904
 22 124432
-23 128504
+23 128538
 24 131738
 25 134972
 26 138449
 27 142211
-28 146328
+28 146342
 29 149546
 30 153799
-31 157691
+31 157680
 32 159895
-33 163988
-34 165988
-35 170279
-36 174223
-37 175925
-38 179534
-39 182936
-40 185192
-41 189434
-42 191592
-43 194950
-44 199716
+33 163989
+34 165982
+35 170316
+36 174221
+37 175921
+38 179546
+39 182942
+40 185207
+41 189448
+42 191616
+43 194928
+44 199685
 45 201569
 46 205684
 47 206994
 48 209626
-49 214870
-50 215457
-51 217054
-52 224394
-53 228390
-54 230940
-55 233959
+49 214837
+50 215478
+51 217036
+52 224314
+53 228366
+54 230935
+55 233948
 56 237844
 57 240776
-58 244407
-59 246163
+58 244383
+59 246143
 60 249779
 61 255611
 62 258001
 63 262202
-64 265171
-65 270680
-66 277145
-67 281531
-68 290082
-69 298677
-70 309321
-71 316846
-72 324213
-73 335201
-74 345570
-75 349804
-76 358629
-77 374720
-78 384279
-79 393731
-80 409572
+64 265153
+65 270647
+66 277140
+67 281517
+68 290070
+69 298675
+70 309339
+71 316849
+72 324221
+73 335227
+74 345571
+75 349778
+76 358587
+77 374719
+78 384242
+79 393690
+80 409507
 81 426035
 82 444737
-83 460661
-84 480883
-85 507693
-86 527867
+83 460605
+84 480852
+85 507713
+86 527862
 87 543086
-88 568144
-89 588634
-90 614272
-91 635603
-92 649735
-93 682037
-94 715719
-95 751637
-96 793214
-97 834146
-98 875977
-99 980328
-100 1170468
+88 568155
+89 588668
+90 614241
+91 635553
+92 649728
+93 682035
+94 715681
+95 751641
+96 793314
+97 834139
+98 875922
+99 980391
+100 1170498

[Petr]

But: For quality=`seq 1 100`, there's no difference in file sizes when git/cups/doc/images/smiley.jpg stands for original image (test.jpg). Today's tests was performed with IJG jpeg library as a backend. According to initial comment of this topic, there would be at least one difference (for default quality) when libjpeg-turbo would be used.

[Petr]

http://upload.wikimedia.org/wikipedia/c ... 5_ubt.jpeg

is one of the online image that exposes the problem for me. File size indicates the issue for -quality=69,70,71,97,99,100 (only few bytes) plus md5sum indicates the issue for -quality=37,38,68,93,94,95,96.

http://upload.wikimedia.org/wikipedia/c ... ka_NZ.jpeg

is one of the online image that does NOT expose the problem. All md5sums correspond for -quality=1..100.