Extract a region of an huge jpeg

[alex88]

Hi guys,

i've a very big image (30000x30000) in jpeg, and i need to extract a region (like 100x100+500+500) from the image.

I've tried with the -size argument, but using something like:

Code: Select all

convert -define jpeg:size=2000x2000 -extract 100x100+0+0 image.jpg output.jpg

i get 100x100 pixel but no the same 100x100 pixel that i get from the original image (that loads fully in memory and is damn slow). Is possible to read just that part of the image to extract it without fully reading the image?

[fmw42]

try

convert -define jpeg:size=200x200 image.jpg[100x100+0+0] output.jpg

or

convert -define jpeg:size=200x200 image.jpg -extract 100x100+0+0 output.jpg

[alex88]

convert -define jpeg:size=200x200 image.jpg[100x100+0+0] output.jpg

that worked in 0.559s but, it gets a 100x100 image, but not the 100x100 first pixels of the real image. In this case:

Your output vs Real output

The real one are the 100x100 pixels of the big image. It seems that that command takes the first 100x100 pixel of a resized image.. isn't it?

convert -define jpeg:size=200x200 image.jpg -extract 100x100+0+0 output.jpg

with this i just got the original image resized to 1500x1500 px, putting -extract 100x100+0+0 before image.jpg gives me same result as above.

[fmw42]

what version of IM are you using? perhaps upgrade?

also see http://www.imagemagick.org/Usage/formats/#jpg_read

but I don't really use it much so cannot say if there is a bug in your version or not.

[alex88]

what version of IM are you using? perhaps upgrade?

also see http://www.imagemagick.org/Usage/formats/#jpg_read

but I don't really use it much so cannot say if there is a bug in your version or not.

i'm using version 6.6.7, bytheway probably reading "just enough of the input" doesn't mean reading the first pixels or data sequentially.

Also from that link:

Code: Select all

 Note that this modifier causes the JPEG library to skip the reading of whole columns and rows of pixels

That's why it resizes the image. Probably there is no way to read just a small part of a jpeg without decoding the whole image?

[fmw42]

Sorry I am not an expert on this issue, but I thought the -define was supposed to take care of that. I don't know why it would resize. I did not think that was part of the operation of the -define. I guess you will need to wait for one of the IM experts to answer that.

[alex88]

Np, thank you anyway. I hope someone will help for this because i don't want to learn all that huffman related things to read a jpeg from hex

[fmw42]

I tried with an image 1280x960 and was able to get it to work without resizing by making the -define 5x the desired output size (1x and 2x did not work for me as it gave some resizing)

The docs on the page I referenced say:

"Because of this is it recommended taht you specify at least double the final 'resize' of the image, to avoid these Aliasing Artifacts, just as shown in the above example."

Also their example seems to take the upper left corner by default, since as far as I know, the -define does not allow any offset.

So this worked:

convert -define jpeg:size=500x500 P1050408.JPG[100x100+0+0] P1050408_sub.JPG

This also worked


convert -define jpeg:size=500x500 P1050408.JPG[100x100+100+100] P1050408_sub2.JPG


Perhaps the problem is with such a large jpg as you have --- 30,000 x 30,000?

[Drarakel]

Do not use "-define jpeg:size" - this is only good as a first step when drastically downsizing images, or when one only needs e.g. the metadata.
You can use "-extract" as alternative to 'file[size]', but the extract option must be before the input image.

With very large images, "stream" is one possibility.
See also here:
http://www.imagemagick.org/Usage/files/#massive
http://www.imagemagick.org/script/stream.php
Example:

Code: Select all

stream -map rgb -storage-type char -extract 100x100+0+0 image.jpg - | convert -depth 8 -size 100x100 rgb:- output.jpg

The command gets a bit more difficult, and you also have to know the number of channels (rgb/cmyk), but this usually saves some time (especially quick with e.g. BMPs or uncompressed TIFFs) and memory.

[anthony]

Hi guys,

i've a very big image (30000x30000) in jpeg, and i need to extract a region (like 100x100+500+500) from the image.

I've tried with the -size argument, but using something like:

Code: Select all

convert -define jpeg:size=2000x2000 -extract 100x100+0+0 image.jpg output.jpg

i get 100x100 pixel but no the same 100x100 pixel that i get from the original image (that loads fully in memory and is damn slow). Is possible to read just that part of the image to extract it without fully reading the image?

Using -define jpeg:size is wrong and misleading the solution.
That asks the JPEG library to read a image that was resized to at least 2000x2000 pixels. That is the library will skip reading images.

You have two solutions to get the actual 100x100 pixels (without resize). from the image.

1/ use stream to extract the region as raw RGB values. Stream does not read the whole image in if it can help it, but pulls it in either one or just a few rows at a time (with JPEG probably 8 rows at a time as JPEG uses 8x8 pixel fourier transformed cells to hold the image.

2/ Use a JPEG specific (non-IM) program to extract just the appropriate set of 8x8 cells without decoding those cells. That image can then be read in as a smaller 'cropped' JPEG image, to then get the exact 100x100 region you want.

Hmmm I am sure that is posible, but after a quick look though jpegtran and similar files it does not appear to allow you to do this. Still I am pretty sure this is posible!

Anyone else have information?

[alex88]

With very large images, "stream" is one possibility.
See also here:
http://www.imagemagick.org/Usage/files/#massive
http://www.imagemagick.org/script/stream.php
Example:

Code: Select all

stream -map rgb -storage-type char -extract 100x100+0+0 image.jpg - | convert -depth 8 -size 100x100 rgb:- output.jpg

The command gets a bit more difficult, and you also have to know the number of channels (rgb/cmyk), but this usually saves some time (especially quick with e.g. BMPs or uncompressed TIFFs) and memory.

Thank you drakael, that worked fine. The problem is that it seems that the time depends on the image size, for example:

• That with a 12000x12000 image: real 0m5.313s
  That with a 30000x30000 image: 0m25.867s

that's a quite big difference, and if i want to process more larger images i think it will grow up fast.

2/ Use a JPEG specific (non-IM) program to extract just the appropriate set of 8x8 cells without decoding those cells. That image can then be read in as a smaller 'cropped' JPEG image, to then get the exact 100x100 region you want.

Hmmm I am sure that is posible, but after a quick look though jpegtran and similar files it does not appear to allow you to do this. Still I am pretty sure this is posible!

Anyone else have information?

1) A specific JPEG program like?

2) Before this i was using this code in java:

Code: Select all

public static BufferedImage readImg (String path, int startx, int starty, int w, int h){
		File input = new File(path);
		ImageInputStream iis = null;
		try {
			iis = ImageIO.createImageInputStream(input);
		} catch (IOException e) {
			e.printStackTrace();
		}
		Iterator iter = ImageIO.getImageReaders(iis);
		ImageReader reader =(ImageReader)iter.next();
		reader.setInput(iis,true);
		ImageReadParam param = reader.getDefaultReadParam();
		int imageIndex = 0;
		Rectangle rect = new Rectangle(startx, starty, w, h); 
		param.setSourceRegion(rect);
		BufferedImage bi=null;
		try {
			bi = reader.read(imageIndex, param);
		} catch (IOException e) {
			e.printStackTrace();
		}
		return bi;
	}

and then write down the bi image with simple imageIO. That was working, in that 30000x30000 image it was extracting the data in 200ms, the problem is that going on with the image increases the time required, e.g., top left 100x100 block, 200ms, bottom right block, 40seconds.

[anthony]

By extracting the specific 8x8 blocks, of the raw JPEG image you should be able to 'seek' to the right place in the data, and then extract just the data blocks needed. The problem is I have never delat with raw JPEG file format. So I am makign some big assumptions.

[alex88]

My problem is how do you check those 8x8 blocks, they won't have the same size in the raw file and afaik there isn't a library that permit that but i think is theorically possible.

This guy here made a quick explanation of how it works but it would be hard and long to make it that way.

[anthony]

That is why I was looking at the special programs such as jpegtran. It has routines to rotate JPEG 90 degrees without decoding it (assuming it's size is multiples of 8x8). I though it had a simular 'crop' option, but can't see on in the manpage. Butr then there has been a number of varients of that particular command. Maybe it only my varient doesn't have it.

[alex88]

Maybe the Lossless crop 'n' drop function? let me try..