Create minimal TIFF from uint16_t RGB in C (GCC) and ImageMagick?

[BrianP007]

Exec Summary:
From inside a C program with uint16_t arrays for R,G and B, (or a single, interlaced, "uint48_t" array), what is the most direct IM way to write a usable tif/png/??? to disk with PURE, RAW DATA and a minimal header?

Gory Details:
DCraw creates a TIF file which essentially behaves like a normal TIF, but has only a 1376 byte TIF fig leaf on top of raw, 48 bit/pixel, uint16_t data. I am trying to create a tif/png/??? file with enough header to hold the XY resolution and be read like a jpg by Photoshop (no .RAW interrogation).

You can use (Identify xyz.tif) to get the XY resolution of DCRaw.TIFF. Since the XRes * YRes * 6B/pix + 1376 == file_size, you can just jettison the top header with an fseek and have access to the pure data in C. Best of both worlds!


In PerlMagick, this is a fairly direct implement to crate a tif from 48bit RAW data (without shelling out to convert.exe)

Code: Select all

sub raw_to_tif()  {
	$raw='raw/ac-254020.wb.gray.mlv.7360x4912.raw';
	($tif=$raw) =~ s/^.*\/(.*)\.raw$/$1.tif/;  # Create "$BASE.tif
	($xres, $yres)=&fname_to_xyres($raw);  # Get XRESxYRES f/fname
	$tsize=$xres . "x$yres";  # 7360x4912
	print("FN2XY -> XRes=$xres, YRes=$yres -> $tsize\n");
	my $im = Image::Magick->new(size=>"$tsize", type=>'TrueColor', depth=>16);  
	
	$err=$im->Read(filename=>"RGB:$raw");    warn $err  if $err;
	$err=$im->Write(filename=>"TIFF:$tif");  warn $err  if $err;
	$tsize= -s $tif;
	$hsize= $tsize - (6 * $xres * $yres);  # Header=fileSize - uint48Size
	printf("$tif created, size=%.6fMB -> tif_hdr=$hsize\n", $tsize/1E6);
	undef $im;  # Nuke after last use
}

I:\br3\ac-254020>bb.pl
Running C:\bin\bb.pl Sun Oct 18 20:03:06 2015
FN2XY -> XRes=7360, YRes=4912 -> 7360x4912
ac-254020.wb.gray.mlv.7360x4912.tif created, size=216.953480MB -> tif_hdr=39560
Elapsed time = 608.035 ms

############################################################################

I write to .raw files with the XY res embedded in the filename so I don't have to remember it (brain buffer < 8 bytes <:o). Photoshop handles the .raw file extension, type in xres, yres, 3 channels, 16 bit and PC format (not Mac!) and a few more dialog about color space, .. and voila, I am only one, save-as interrogation from having a usable file.

From inside a C program with uint16_t arrays for R,G and B, (or a single, interlaced, "uint48_t" array), what is the most direct IM way to write a usable tif/png/??? to disk with PURE, RAW DATA and a minimal header (like DCraw)?


------------------------------------------------------------------------------------

Searching for (create tif) finds zero results. The only google hit for (c imagemagick c api write rgb tif 16bit rgb sample program -convert) is "Writing images from raw data?" from 2002

search.php?keywords=create++tif+&fid[0]=6
Search found 0 matches: create tif

[snibgo]

I don't know how to write data from memory to an image file, but it probably involves "blobs".

Doing the job by first reading the tiff file, then writing the image to another file, is simple: use ReadImage() and WriteImage(). See the demonstration program "core.c".

[BrianP007]

>> From snibgo <<

>> I don't know how to write data from memory to an image file
This is essentially what the question boils down to: How to take RGB Short Int data from memory and create a tif/png with IM.

>> probably involves "blobs"
I already have the Binary Large OBject, 219MB of RAM with uint16 data. This part is done

>> simple: use ReadImage() ...
Maybe I did not explain this well enough. There is NO IMAGE, just a bunch of bits in memory. I need to create an image, not read an existing one... There are 9 million reds, 9M blues and 18M Greens. All are 14 bit unsigned ints in uint16_t, C memory arrays. That's it.

There are no headers, no EXIF, no segment markers, no SOI, no EOI, no APP*, no COMments, no frames, no gamma curve, no color space, no chromaticities, no black or white points. It is just raw electron counts from photosites.


>> simple: use ... WriteImage()
This should be the last step.
1) CREATE_IMAGE from bits in memory <<<<<<< MISSING LINK <<<<<<<<<<<
2) Write TIFF file to disk in a form Photoshop can understand


There seems to be a chicken and egg situation. From http://www.imagemagick.org/source/core.c:
/* Initialize the image info structure and read an image. */
...
image_info=CloneImageInfo((ImageInfo *) NULL);
(void) strcpy(image_info->filename,argv[1]);
images=ReadImage(image_info,exception);

"CLONE" implies taking "ImageInfo" from one place (presumably an IMAGE) and copying it elsewhere.
Since we have not told it: A) 16bit Quanta, B) XY Resolution or C) RAW:: format , it has no possibility of decrypting the apparently random mountain of bits without info NOT CONTAINED in the file

The Example Perl Program I supplied was a real, working script starting with a barrel of bits and creating a working TIFF. Exiftool and Identify both work on it (see below). The runtime is a real run-time.

I have been looking for an example C Program which starts with RAW, 16bit/channel RGB data and creates any standard format, NON-COMPRESSED, image. A WORKING C PROGRAM, not a snippet of an API or a vague reference to something somewhere on the web.

The last thread I could find about creating a TIFF from nothing but ~random sensor data was from 2002 and is closed so I can't even ask the person if they figured it out.

Still searching ...


# =================================================================
The PerlMagick script created a REAL IMAGE!

I:\br3\ac-254020>identify ac-254020.wb.gray.mlv.7360x4912.tif
ac-254020.wb.gray.mlv.7360x4912.tif TIFF 7360x4912 7360x4912+0+0 16-bit sRGB 217
MB 0.000u 0:00.000

I:\br3\ac-254020>exiftool ac-254020.wb.gray.mlv.7360x4912.tif
BaseName : ac-254020.wb.gray.mlv.7360x4912
BitsPerSample : 16 16 16
Compression : Uncompressed
Directory : .
ExifByteOrder : Little-endian (Intel, II)
ExifToolVersion : 10.02
FileAccessDate : 2015:10:18 19:51:04-05:00
FileCreateDate : 2015:10:18 19:51:04-05:00
FileExtension : tif
FileModifyDate : 2015:10:18 20:37:58-05:00
FileName : ac-254020.wb.gray.mlv.7360x4912.tif
FilePermissions : rw-rw-rw-
FileSize : 207 MB
FileType : TIFF
FileTypeDescription : Tagged Image File Format
FileTypeExtension : tif
FillOrder : Normal
ImageHeight : 4912
ImageSize : 7360x4912
ImageWidth : 7360
MIMEType : image/tiff
Megapixels : 36.2
Orientation : Horizontal (normal)
PageNumber : 0 1
PhotometricInterpretation : RGB
PlanarConfiguration : Chunky
PrimaryChromaticities : 0.6399999858 0.3300000132 0.300000012 0.600000
024 0.150000006 0.05999999867
RowsPerStrip : 1
SamplesPerPixel : 3
StripByteCounts : (Binary data 29471 bytes, use -b option to ext
ract)
StripOffsets : (Binary data 46597 bytes, use -b option to ext
ract)
WhitePoint : 0.3127000035 0.3289999963

[BrianP007]

I don't know how to write data from memory to an image file, but it probably involves "blobs".

Doing the job by first reading the tiff file, then writing the image to another file, is simple: use ReadImage() and WriteImage(). See the demonstration program "core.c".

I am still looking for a real program which creates a 48bit/channel tiff file with raw guts (like the DCraw.tif with 1376 bytes of headed on top of pure, raw, 3Xuint16_t, RGB data.

Here is an example of a full, working C program which creates a PNG from NOTHING, NO STARTER IMAGE to CLONE. Unfortunately, GD does not support 16bit quanta or I would be using it.

Check out the Run-Time; 694 MICRO-SECONDS! I wish all of my programs ran that fast (and did something useful).

Code: Select all

#include "gd.h"
#include "stdio.h"
#include "sys/stat.h"
#include "stdlib.h"
#include "string.h"
#include "stdint.h"
#include <math.h>
#include <time.h>
#include <windows.h>

// gcc -O4 -ffast-math -m64 -Ofast -march=corei7-avx -mtune=corei7-avx I:/br3/ac-254020/gd.image.from.nothing.demo.c -lgd -o I:/br3/ac-254020/gd.image.from.nothing.demo.exe

void main(int argc, char *argv[])  {
LARGE_INTEGER cpu_freq;  // CPU clock ticks per second
LARGE_INTEGER stime, etime;  // Clock ticks at start and end
struct stat statbuf;
FILE *out;  // FILE pointer to OUTPUT Image
gdImagePtr im;
char img_fn[]="Demo.png";
int red, fsize;
float dtime;  

	QueryPerformanceFrequency(&cpu_freq);  // Gets clock_tics/sec? -> 3.4E6, !9
	QueryPerformanceCounter(&stime);  // Main Program start time

	im = gdImageCreateTrueColor(100, 100);
	red = gdImageColorAllocate(im, 255, 0, 0);  
	gdImageSetPixel(im, 50, 50, red);
	
	out=fopen(img_fn, "wb");
	gdImagePng(im, out);
	fflush(out);  fclose(out);
	gdImageDestroy(im);
	stat(img_fn, &statbuf);    // Stat the file to get its size
	fsize = statbuf.st_size;  // File SIZE in bytes

	QueryPerformanceCounter(&etime);  // Main Program start time
	dtime = (etime.QuadPart-stime.QuadPart) * 1.0E0f / cpu_freq.QuadPart;
	printf("%s Image %s has size=%d bytes in %.3f ms\n", 
		argv[0], img_fn, fsize, dtime*1.0E3f);
	fflush(stdout);
	exit(0);
}  // End Main().

I:\br3\ac-254020> gcc -O4 -ffast-math -m64 -Ofast -march=corei7-avx -mtune=corei7-avx I:/br3/ac-254020/gd.image.from.nothing.demo.c -lgd -o I:/br3/ac-254020/gd.image.from.nothing.demo.exe

I:\br3\ac-254020>gd.image.from.nothing.demo.exe
gd.image.from.nothing.demo.exe Image Demo.png has size=136 bytes in 0.694 ms

Code: Select all

I:\br3\ac-254020>exiftool  Demo.png
BaseName                        : Demo
BitDepth                        : 8
ColorType                       : RGB
Compression                     : Deflate/Inflate
Directory                       : .
ExifToolVersion                 : 10.02
FileAccessDate                  : 2015:10:18 23:24:20-05:00
FileCreateDate                  : 2015:10:18 23:24:20-05:00
FileExtension                   : png
FileModifyDate                  : 2015:10:19 00:00:34-05:00
FileName                        : Demo.png
FilePermissions                 : rw-rw-rw-
FileSize                        : 136 bytes
FileType                        : PNG
FileTypeDescription             : Portable Network Graphics
FileTypeExtension               : png
Filter                          : Adaptive
ImageHeight                     : 100
ImageSize                       : 100x100
ImageWidth                      : 100
Interlace                       : Noninterlaced
MIMEType                        : image/png
Megapixels                      : 0.010
PixelUnits                      : meters
PixelsPerUnitX                  : 3780
PixelsPerUnitY                  : 3780

[snibgo]

There are 9 million reds, 9M blues and 18M Greens.

This sounds like a Bayer mosaiced image, eg NEF data from a Nikon DSLR. I suppose you would also want to de-mosaic the image, yes?

I have been looking for an example C Program which starts with RAW, 16bit/channel RGB data and creates any standard format, NON-COMPRESSED, image. A WORKING C PROGRAM, not a snippet of an API or a vague reference to something somewhere on the web.

The best example I know of, that does exactly that (including demosaicing), is dcraw. True, it starts from a file, not an array. Perhaps you could remove the code you don't need.

[BrianP007]

Snibgo,
I am not actually thinking about demosaicing at this point. I just want to work with the channels in their rawest possible form.

As far as DCraw goes, I have built it here, optimized for my exact chipset, and seen a 6x performance over the generic (i386???) binary. I spent about an hour trying to hack a crop into it so it didn't crunch the entire image, just a tight face crop to get the skin tones. ACR supports cropping and dramatically speeds up the process. DCraw is not what I would consider an approachable piece of code.

But, starting with 10,145 lines of code and filtering down to the 10 I need seems like starting with the blueprint of the Pentagon when you need a dog house.

Somebody out there has to have written a simple, demo program which does nothing but make a tiny tiff from scratch.

How hard could it be to translate 3 lines of PerlMagick into C? ::
$im = Image::Magick->new(size=>"7360x1912", type=>'TrueColor', depth=>16);
$im->Read(filename=>"RGB:bb.raw");
$im->Write(filename=>"TIFF:bb.tif");

If I can't find a Reindeer, I'll make one instead! << He Grinched!

Brian

[snibgo]

... make a tiny tiff from scratch.

But then you have an example that reads from a file and writes to another, like ReadImage() and WriteImage() that I mentioned above.

[BrianP007]

... make a tiny tiff from scratch.

But then you have an example that reads from a file and writes to another, like ReadImage() and WriteImage() that I mentioned above.

>> But then you have an example that reads from a file...

You are re-re-re-repeating non-useful info. Did you read my line-by-line analysis of ReadImage() above. READ-IMAGE needs an image to read. We have already established that THERE IS NO IMAGE TO READ.

THE TOPIC IS CREATING an IMAGE FROM SCRATCH, NOT READING AN EXISTING IMAGE. It would be efficient to focus exclusively on NEW INFORMATION regarding CREATING an IMAGE.

TYVM!

[snibgo]

Sorry, but I'm struggling to understand what info you want. NewMagickImage() creates a new image from scratch.

[BrianP007]

Snibgo,

>> I'm struggling to understand what info you want.

I am struggling to boil the question down to a more basic form...
What is the simplest method to create a NEW IMAGE starting with ~random sensor
bits which does not involve copying/cloning an existing image???

In your first 3 posts on this question, you only mentioned Reading/writing an
existing image. This 4th post mentions NEW with respect to an image for the
first time. Making progress!!!


SEE PERL SCRIPT (from 1st post). It creates a TIF from raw data in 3 lines.
HOW to translate THIS PERL TO C LANGUAGE USING IMAGE__MAGICK???
$im = Image::Magick->new(size=>"7360x4912", type=>'TrueColor', depth=>16);
$im->Read(filename=>"RGB:bb.raw"); << Contains ~random sensor gibberish
$im->Write(filename=>"TIFF:bb.tif"); << READ by photoshop and file explorer

The Working Perl Script shows EVERYTHING needed. It compiles and runs and shows
a run-time. Anybody with Perl can cut the script, paste it onto their system and
run it, supplying only the sensor bits which can not be programmed.

A working C program, ready to compile and run like the Perl script,
would be ideal. One, critical, line from ~40 lines of code needed to build
an executable is a step forward. But, having a PISTON when you need a CAR does
not get you very far...

search.php?keywords=NewMagickImage
Search found 1 match: NewMagickImage <<>> Searched query: newmagickimage
Nobody has ever discussed this until today.

googling(NewMagickImage imagemagick NewMagickImage example code), only the source code for the function
itself is found in the first 5 pages. google presents no evidence anybody has ever used it.

Not enough to go on...

Brian