How white balance and highlight recovery works

This article references our freeware white balance plug-in that can be downloaded free of charge. It lets you quickly white balance and recover highlights in situations where you don’t have access to the RAW data.

How it works

The desired white balance occurs when R = G = B for an area which should be a neutral color (e.g. grey, white). To achieve this, we can change the gain on the RGB channels / multiply all the red, green and blue pixels by some set of values. For example, if R=3 G=2 B=1 then G needs to multiplied by 1.5 and B by 3 to get the desired white balance. There are two issues:

1- Doing this will increase noise (bad). Multiplying the channels by some value will also multiply noise by that value and bring it up. We can’t really avoid this unless optical filters are used in front of the lens to set white balance (for still photography I don’t feel it’s necessary as you can use HDR techniques).

2- Changing the gain on each channel will cause some values to exceed white level (also bad). One of the channels will stay the same and have no gain applied to it.

Highlight recovery works by taking the values above white level and intelligently guessing what the values in the unchanged channel are. Suppose the neutral value is R=3 G=2 B=1 and there is some highlight in the image that is R=3 G=3 B=1.5. The highlight, after white balance, will be R=3 G=4.5 B=4.5. Suppose clipping occurs at 3. Normally we would just clip the data at 3. But instead of throwing the data away, we can do highlight recovery here.

Assuming that the highlight is white, the original scene value would have to be R=4.5 G=4.5 B=4.5. So we basically (A) make an assumption about the color of highlight and then (B) work backwards to perform highlight recovery. This means that highlight recovery will not work perfectly if the highlights aren’t white. But in most real world scenes they are, so it will work most of the time. You can turn highlight recovery down or off in situations where it won’t work.

In a nutshell, that is the basic explanation about how highlight recovery works. In the freeware plugin, I add a rolloff that de-saturates highlights to deal with values close to white level. This is to avoid an abrupt transition between pixels where highlight recovery is and isn't being used.

Why RAW is superior

One of the reasons RAW is superior when it comes to white balance is because it avoids white balance being done twice. Shooting JPEG will apply white balance in the camera and again when you adjust it. As explained earlier, white balance is a destructive process so it is best to only do it once.

The camera’s digital signal processing may also do things that will screw up the white balance process. Any colors that fall outside your working color space (e.g. sRGB, Adobe RGB) will be clipped; a RAW workflow would avoid this pitfall.

Processing shortcuts in the camera’s color matrix and a non-standard transfer function can also screw up the white balance process.

So shoot RAW if you can. But if for some reason you don’t have RAW data to work with (e.g. scanned images, JPEGs from third parties, etc. etc.) then check out the plug-in.