How often have you taken a picture with a digital camera, only for it to come out looking like this? The sky’s too bright, the shaded parts are too dark, and however much one juggles with it with software, it won’t come out right.
 


The eye managed to adapt to it all right, effortlessly moving over the scene. With film, if one were a darkroom wizard, one could play tricks with the processing. But digital – what do we get for our multi-megapixels and our 16777216 colours?

It’s that big number that’s playing tricks with us. It is the cube of 256, so that we have have the product of 256 red shades by 256 green and 256 blue. If they’re all fully saturated at 255, we have while, and if fully unsaturated at 0, we have black, and if the three colours are all equal at any other level we have a ‘perfect’ gray.

“So what’s the problem?” says an impatient, grouchy voice. “Ain’t that enough levels for anyone?” Take this mushroom / toadstool that I photographed about a month ago. While part of the surroundings is hidden in shadow, the cap colour is all compressed into the top few values of RGB.
  

Playing with gamma and contrast, these levels get pulled out into something like a contour map, an effect known as “posterization”.

Now from the “Luminous Landscape” website I learn that all digital camera sensors actually have much more step sensitivity than that, starting from 12-bit (4096 gray levels), up to 16-bit (65536 gray levels) colour. Now many of these initially save what is known as a RAW image, but generally what goes on to your memory card is but an 8-bit JPEG image. However, use of RAW images has not made its way into the amateur market, partly because there is no standard format for them. If at this point you want to find out more, here are four articles from that website.

Bit-depth       Understanding RAW files     The RAW Truth      The RAW flaw 
 
= = = = = = = = = = = = = = = = = = = = = = =

Now when it comes to scientific data, the problem is much more severe. One’s photographs may be astronomical or microscopical, or data from scattered X-rays or neutrons. In these cases 8-bit is simply not enough!


Recently, at our Centre for Advanced Microscopy, we had a visitor from the Keyence company showing us one of their digital optical microscopes.

One of the features of their higher end models is the ability to operate with 16-bit colour. Here is a micrograph of part of the visitor’s business card. Below is an 8-bit image, and the detail in the black and white regions is largely lost. Above is the 16-bit image (processed and rendered to 8-bit), which shows the fibre structure in white, red, and black. 

However, coming to X-ray work, detectors at beamlines output 12- or 16-bit images. However, every system seems to have its own format. Coming back from one site, our processed picture looked like the left one in this pair.



When we measured the data values round a circle, we got a graph something like this.



These should be smooth(ish) peaks. But looking carefully at all the wiggles in the curve, one finds that they occur at multiples of 256. This suggests that the problem is due to the way the 16-bit data are stored. There is not even a standard to say whether the byte containing the lower 8 bits comes before or after that containing the upper 8.

How there exists a program which can cope with many of these hassles. It is called FIT2D, and you can have a look at its ESRF reference manual online to see what a formidable beast it is. It is, alas, a somewhat “geekish” program to use.

For a reasonable stop-gap solution (enough to get a paper sent off) we used FIT2D to convert the image to BSL FORMAT. The manual says:

The BSL format is essentially the same as the OTOKO format developed at Hamburg. It consists of an ASCII header file and a number of binary image data files. The images are stored in floating point reals. (This may cause problems when files are moved between non-IEEE and IEEE float point reals systems.)
This may also cause problems between little-endian and big-endian systems since the endianess of the stored data is not recorded. 

(Now “endianess” refers to whether the bits in a byte are ordered with the most significant bit first or last. They are opposite in PCs and (most) Unix systems.)

Our own in-house software (written by El Jefe himself) could now give us the right-hand image.