RAW HDR Processing

Comparing the Original to the processed Images

Every travel photographer knows the scenario. While most photographers advise not to shoot during the harsh light of the day, we often have little choice. Often I find myself at a place and I know I will have to move on, either because I have a travel schedule or because I am on a weekend trip and have to get back to my daily routine. Believe me, if I could afford it, I would follow the good advice and spent a lot of time on each location waiting for perfect conditions to make my photos shine.
In situations such as this, the best would be to put your camera on a tripod and bracket your exposure. You can then combine the pictures on your computer. However you cannot take your tripod everywhere (it may not be allowed) and it is very cumbersome.
In the past I have often written about RAW processing and I have stated that you can make good use of the dynamic range captured by your camera. Almost every Digital SLR camera and many of the better Point and Shoot models alow you to record RAW files. A RAW file is the data captured by your Camera's sensor that is not processed yet. It does not contain color information (yet), which is very useful to adjust white balance at the computer and apply sharpening and other enhancements the way you like it (check my other Worklfow Tutorials for more information). A 12-bit file contains 16 times more levels then a 8-bit JPG file. This is the main reason why I only shoot RAW.

Photomatix

While Photoshop offers tools to generate HDR (high dynamic range) images and blend multiple exposures, I find neither of them adequate for my needs. The tone mapping is very limited and not very useful. Photoshop also detects the exposure of an image and initially refuses to merge multiple files that were created from a single RAW file (they all contain the same EXIF data, so Photoshop sees no difference in Exposure). While there are ways around this (delete exif information), I have also evaluated Photomatix and found it extremely useful for the task at hand.
Photomatix automates the process of creating HDR images or stacking images and makes the entire process very easy.

Some definitions

Skip the next paragraph if you don't care.
An HDR image contains 32-bit of brightness information with virtually no limitation on how many levels you can represent (floating point numbers).
Tone Mapping describes the process of mapping such an image back into either 8-bit (or 16-bit color) space for display and/or printing.

Why should you care?

As always, nature cannot be copied in all its perfection. Your eyes are the ultimate image sensors. They have a nonlinear response curve enabling you to see very bright areas in the presence of dark shades. Your camera does not have that capability. It has a very limited response curve, leaving you either with blown out highlights, too dark shadows or both. Worse than that, when the image is processed, even more information is lost (8-bit JPG).
You might wonder why I would want to convert a 12-bit image to 32-bit just to output it as a 16-bit TIF. Well here is how I think it works.
RAW vs. Bitmap comparison
Both, the 16-bit and the 8-bit image have the same minimum and maximum brightness levels. The only difference being that the 16-bit image offers a higher resolution on the levels. The RAW file however contains information beyond the levels of the bitmap images, and those are the ones we are trying to recover. So we shift the bitmap scale left and right, create multiple files and combine them accordingly.

Ok, what now

For extremely high dynamic range scenes (e.g. a dark cathedral with bright windows) you are probably better of taking multiple photographs with different exposures (use Aperture Priority or you will mess up your Depth of Field and sharpness), but I find that I can get away with some processing on a single RAW file. In fact I can easily push my RAW files to +/- 2 f-stops in post processing (that would be equivalent to taking 5 images each one stop apart).

Lets process

RAW converter Exposure
Open up your RAW file in your favorite RAW converter (I use Adobe Camera RAW. The arrow points to the exposure setting). Don't touch any of the settings except the Exposure slider (you can also fine tune white balance, sharpness, ... but thats covered elsewhere).
Play around with the exposure slider. Move it to the left and see the image become darker. Watch your highlights and see how much information we can recover. Find out how low you need to go (but don't go much beyond 2 f-stops).
It will depend on your camera how far you can push this.
Remember the number (e.g. -2).
Then move your slider to the right and see how much you can pull out of the shadows (without seing too much noise).
Remember that number (e.g. 2).
Subract the two numbers (2 - (-2) = 4).
In our example we are going to cover a 4 f-stop range (thats about as far as this works on RAW files for most Canon Cameras I guess (I hear the new line has 14-bit analog to digital converters, so it may go further).
Now divide those into n equal steps, not much larger then about 1 f-stop.
In the example above this results in 5 images:
(-2, -1, 0, 1, 2)

Example2:
Min = -1.6
Max =  0.8
Result = (-1.6, -0.8, 0, 0.8)

Enter these numbers into the Exposure compensation box and convert one image for each (5 images in the first example). I usually save them as TIF files so I don't lose information to compression (until the very end when I save everything as JPG).

Processing with Photomatix

Photomatix is the perfect tool for combining these images into a single HDR image and then use Tone mapping to output them. 
The software offers far too many options to cover them all at length. I encourage you to play around with it a bit.
We can either generate a HDR image from the 5 tiff images or we can simply combine them and let Photomatix figure out the shadows and highlights.
The HDR options offers far more control and more dramatic results.

H&S - Auto

Lets start with the simpe technique. Sometimes I like this better since HDR images take some getting used to. Although they probably resemble what we see, we are not used to this kind of presentation.

Photomatix Batch Averaging
I usually use the batch processing mode. In this mode, you can automatically process all images in a folder. For this test, I have put all 5 files into a folder and pointed the batch processing window to this folder (LOCATION).
Then I told Photomatix that I had 5 images (if you process more then one image, make sure they have the same number of source files).
Since we generated all of those images from a single source file, we do not have to select Align bracketed images. This option is very usefull if you composed multiple images (exposure bracketing).
Then I selected Highlight and Shadow - Auto (telling Photomatix to figure out the best settings).
Thats it. Photomatix will put the results file in the same folder (see above or below for a comparison image).

HDR

This option is a bit more involved, but Photomatix makes it as easy as it can get.
Photomatix Batch HDR conversion
Simply go to HDR -> Generate and select your images. In the next dialog photomatix will ask you for the exposure values (still remember the settings from your RAW converter?).
Press OK and Photomatix will generate an HDR image for you.
HDR files contain far more information then your monitor possibly can display, so don't worry if the image doesn't look good (yet). This is where the Tone Mapping comes in. The Tone Mapping is the real strength of Photomatix. It can be bought as a Photoshop Plug in (I bought the bundle, but I use the standalone program most of the time).

Photomatix Tone Mapping
On the left you can see the HDR file (linear light levels). On the right you can see the Tone Mapping Window. It applies a linear curve to the image mapping all brightness levels into the bitmap image space.
The Window is very intuitive. You can simply hover with your mouse over any of the sliders to get a short explanation.
Play around with it for a while and save settings you like. Load the default or older settings and compare the results.
When you are ready, simply press the OK button. Photomatix will output a TIFF file of chosen output depth (I went with 16-bit in this example).

Don't worry about what you pick here. You can always save the HDR (.hdr) and come back to this step again and again until you are completely happy. I have not modified the standard settings much for this example, however the results are already very impressive:
Comparing original, H/S and HDR image
The picture on the left uses my standard Photoshop settings for processing images (they are already better then in camera JPG settings, since they tend to recover some of the Highlights and Shadows).
The picture in the middle was generated with the Automatic Highlight and Shadow settings. If you look closely, you will find that some of the shadows of the roof have been brought to live and that the building (outside in the bright sun) now looks better (not over exposed).
With the HDR image, we were able to pull the structure of the wood out of the shadows (the roof looks way better) and even the distant building has regained a lot of its texture.
The Original Image in this example already has some detail pulled out of the shadows and highlights.

Conclusions:

Always shoot RAW images. As a best case scenario, you can just batch process them in Photoshop and still get better results then your camera would give you with very little effort.
If you come across a difficult image, you can use the proper tools to restore it accordingly.
Photomatix is an excellent software. To use its full potential, pick a difficult scene and shoot a lot of exposures.
I use it primarily for recovering highlights and shadows (either with H/S or HDR) from my photographs. It is the best tool for the job.

Download an evaluation version of Photomatix and start playing: Photomatix Download

Wow, Amazing

Thats amazing. I will check out the evaluation right now.

All Those Bits

Good article, Andre. How bits are used is always somewhat of a mystery. A couple notes...

- When a RAW file is automatically converted to JPEG or TIFF, it tends to clip off some of the highlights. While a 16-bit TIFF could represent the entire dynamic range, doing so tends to leave the entire image looking a bit underexposed and so it still needs to be curved-up for display or printing. Also, these conversions limit the color gamut to whatever RGB space is being used.

- A RAW image contains linear information (essentially a count of photons per pixel) while JPEG and TIFF are usually gamma-compressed to match the response curve of the human eye. You get about about 12 stops of exposure within 8-bits of data on these setting which is about equivalent to 12-bits of dynamic range in RAW. Each additional bit will add about another 2 stops of exposure and precision.

I use EasyHDR for my own night photography HDR.

Thanks Brian

Thanks for the notes Brian. I have completely forgotten about the linear information in RAW. There is a pretty good Plugin by Fred Miranda that processes your RAW files in linear mode and supposedly you can get good results with it too but I haven't seen any dramatic results either.

A TIFF file that I convert with a RAW editor cannot be tweaked beyond a certain point anymore. I can never recover as much of the highlights as I can from the RAW file with Photomatix.
So it does appear to get clipped beyond some point.

BTW: Since you mentioned the color space. I have found that I can get the best (most dramatic) results by using "Pro Photo RGB" throughout the editing process and making the conversion (Adobe Engine, Perceptual) to sRGB one of my last steps. I can push the color saturation much more this way without getting any posterization effects (especially annoying on sunset pictures when the sun is visible).

Thanks again for the good pointers.

Your Raw HDR article

This is what I need to know altho it is still beyond me. It was the second such article I've read & I'm starting to get the idea.
I've shot jpegs for years and process them with iPhoto but it's time to learn more.
I have Capture One & Photoshop 7 software to post process the images from my new Nikon D80 but I'm lost when I open them up. I don't understand the terminology yet and the Help menus just make me more confused.
I will check out your Workflow bits. I find your writing easy to understand- now I need to get the terms.
Any pointers for me from what I've said?
Thanks. I keep checking back.

Capture One

I have tried Capture one a long time ago, but not recently.

Basically you need to adjust the Exposure up and down, but keep all other settings the same. Adjust the exposure in equal steps (read the article again, hopefully this point will become apparent) and then store them on your hard drive. Download the trial version of Photomatix and try the steps to create a HDR image. It is pretty straightforward.

To see some results of what you are doing, start with an image where you know you can pull some details out of the shadows and highlights (large dynamic range). Photomatix is really easy to use, I think you will get a hang of it quite fast.

To learn more about RAW processing, you could start browsing through some of the Workflow Articles. Although they are written mainly with ACR (Adobe Camra RAW) in mind, I have made a few remarks regarding the advantages of RAW (for instance white balance can be adjusted later, the automatic isn't always right).

Photomatix

Andre, Thanks for the article. Do you have better results using this method than just opening up the single raw in Photomatix and letting it generate the pseudo hdr file. Then doing the Tone Mapping.

The latest version is 2.4.1 (although there is a newer 2.5 Beta) so maybe the original version of Photomatix did not allow this.

Thanks Mike

I have to admit that I haven't been aware that Photomatix could even open RAW files. Thanks for the tip.
All things being equal, I don't know if I would ever use this, as I prefer to fine tune my pictures with a full fledged RAW editor.
However I don't tune much on hdr images (maybe some sharpening but thats about it).
I do remember that HDRSoft was not exactly promoting generating HDR files from a single RAW, since the software can do so much more, so maybe thats a new feature.
I will try it out and see how this compares. Thanks again.

Black spots

When I take multiple shots of the same scene and try to merge them with PM, I get black spots around leaves that were blowing in the wind or water ripples, anything that was moving. Is this "normal"? Those scenes just don't allow themselves to be HDR'd? OR is there a processing step/slider that will help reduce this effect?

thanks,
-tim

Movement

Tim: Thats a pretty good question. The problem really doesn't exist if you generate 3 exposures from a single RAW file (like in the article above). It's a really good example for using just a single RAW (since all exposures come from the same file, there is no difference between them).
Movement is probably one of the hardest things to process in multiple exposures. Generally it's not impossible, but requires a lot of work. You need to create layer masks for each of the photographs and fix it in post processing.
A shot like that would probably be done best in Photoshop blending manual.
The first steps are the same (create some files with different exposure). Then you stack them in Photoshop (layers) and you start painting the layer masks very carefully (thus selecting what image will show).
I used this once to remove people from a scene (taking a whole lot of shots and then painting them out of the picture). It's a cumbersome process.
However if you can clearly separate between certain areas of the image (e.g. top with bright leafs and bottom with lots of shadows) you might get away with simply stacking two images and drawing a gradient on your layer mask to blend from one image to another.

OK

ok, it likes good!
I'll go & try it!
The best & bright lights,
Belosz

Distortion

I keep getting a lot of distortion.. or maybe noise? I'm not sure what to call it. In the darker shadow areas of my images after I do the tone mapping. I'm using the same programs as you. and the tiffs look fine after I save them from the RAW. Any suggestions?

Thanks
-Jake

Bill/Jake? I recommend to

Bill/Jake?
I recommend to play around with the parameters during the mapping. You can set the black and white point. Set the preview to maximum size (1024).
If you are unsuccessful, you could send me your example to check it out.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

More information about formatting options