JPEG compression

[khavish]

The JPEG compression algorithm split the image into 8x8 blocks



My thinking is that if we split an image into 8x8 blocks, feed each block to libjpeg and then merge the JPEG blocks into a lossless format(say PNG) , we should theoretically obtain the same JPEG if we had converted the original image directly into JPEG.However this isn't the case in practice.

Image used : https://drive.google.com/file/d/1ncp_gZ ... sp=sharing

Here is my code:

Code: Select all

#!/bin/bash
# crop the image in 8x8 blocks(a.k.a tiles) and save offsets in filename
convert bench.png -crop 8x8 bench_%d.png
num=`convert bench_*.png -format "%n\n" info: | tail -n 1`

#Convert the tiles to JPEG
for ((i=0; i< $num; i++)); do
offset=`convert bench_$i.png -format "%X%Y" info:`
convert bench_$i.png -quality 80 bench_${i}_${offset}.jpg
done

#Merge the JPEG tiles into a single PNG image

list=""
for ((i=0; i < $num; i++)); do
offset=`echo bench_${i}_*.jpg | sed -n 's/^bench.*[+]\(.*\)[+]\(.*\).jpg$/+\1+\2/p'`
echo "$i $offset"
list="$list ( bench_${i}_${offset}.jpg -set page $offset )"
done
convert $list -layers merge bench_merge_tiles_q80.png
convert bench.png -quality 80 bench_q80_direct.jpg
#Compared the generated merged JPEG in PNG with the directly converted whole image 
compare -metric SSIM  bench_q80_direct.jpg bench_merge_tiles_q80.png /dev/null 

What am i missing here ?

[snibgo]

With a sampling factor 4:4:4 (ie no downsamping), there is no difference.

With downsampling, you need larger blocks, eg 16x16. Even then, there is a difference, but only up to about 1% RMSE even with low "-quality" numbers. I guess the difference is because the downsampling is done before splitting into blocks.

Aside: I would put "+repage" after "-layers merge", though it doesn't matter here.

A revised script (for my old version of IM which doesn't have "-metric SSIM") is:

Code: Select all

#!/bin/bash

# Set qual <= 100:
qual=2

# Choose a combination of sampl and blksz:
sampl=4:2:2
blksz=16x16

sampl=4:4:4
blksz=8x8

sampl=4:2:0
blksz=16x16


rm bench_*.png
rm bench_*.jpg

# crop the image in ${blksz} eg 8x8 blocks(a.k.a tiles) and save offsets in filename
convert bench.png -crop ${blksz} bench_%d.png
num=`convert bench_*.png -format "%n\n" info: | tail -n 1`

#Convert the tiles to JPEG
for ((i=0; i< $num; i++)); do
offset=`convert bench_$i.png -format "%X%Y" info:`
convert bench_$i.png -quality ${qual} -define jpeg:sampling-factor=${sampl} bench_${i}_${offset}.jpg
done

#Merge the JPEG tiles into a single PNG image

list=""
for ((i=0; i < $num; i++)); do
offset=`echo bench_${i}_*.jpg | sed -n 's/^bench.*[+]\(.*\)[+]\(.*\).jpg$/+\1+\2/p'`
echo "$i $offset"
list="$list ( bench_${i}_${offset}.jpg -set page $offset )"
done
convert $list -layers merge +repage bench_merge_tiles_q${qual}.png
convert bench.png -quality ${qual} -define jpeg:sampling-factor=${sampl} bench_q${qual}_direct.jpg
#Compared the generated merged JPEG in PNG with the directly converted whole image 
compare -metric RMSE bench_q${qual}_direct.jpg bench_merge_tiles_q${qual}.png NULL:

[khavish]

Ahh...i forgot about automatic chroma subsampling in IM.Thanks for the reminder and the help.


Happy new Year

[rubles04]

Why JPEG compression processes image by 8x8 blocks instead of applying Discrete Cosine Transform to the whole image?

[snibgo]

I don't know, but I suggest:

1. DCT time to process a block is proportional to at least N*log(N) where N is the number of pixels per block. So best speed comes from small blocks.

2. DCT can be done in parallel, so smaller blocks are better for multithreading.

3. The lossy compression occurs at the DCT stage, and is constant within a block. Image areas with no high-frequency detail (eg blue sky) compress more easily than areas with HF detail (eg grass), so we don't want a block to straddle these mixed types.