False coloring a video in real time using lookup tables

I've been working on OpenGLES very intensively for video rendering and applying real-time filters for months, and I haven't yet put up a blogpost on one of my favorite filters yet. I love it because it shows how fast and simple things can be in the GPU if you know how to use it well;
What you have to do is given a greyscale image, false color it with a provided mapping from grayscale to RGB values; i.e.,

Grey [0,255] --> R[0,255], G[0,255], B[0,255]

In a traditional CPU world, this is fairly simple. All you have to do is loop the image for each pixel, and use the mapping to convert the grey pixel to an RGB. But this is insanely slow, and cannot be done in real time. In comes the GPU! It's perfect for repetitive tasks, like this simple image processing problem where you have to apply the same function to every pixel of an image.
Here's what the output looks like. The center image is the original greyscale image. My favorite color palette is the one on the top right. Fun fact, the palette is called Inferno.

Ideally, I should've taken these from a thermal imaging camera, but I'd have to wait till after the weekend to click pictures in office. It becomes a lot more obvious why these color palettes are used in thermal imagers once you see how minute differences in heat signatures which may be overlooked in a grey(black-hot or white-hot) image become very visible in these false-colored images.

Here's how this is done in OpenGLES; it's fairly straightforward really*. 

The meat is in the fragment shader. You pass your original image, and the palette mapping, an array of 256 RGB values. Each index holds the corresponding mapping of grey to color.

float grey = texture2D(sTexture, vTextureCoord).r;
gives the grey value at the pixel coordinate vTextureCoord. Note the r picks only the r values in ARGB.
gl_FragColor = texture2D(lutTexture, vec2(grey, 0.5))
then picks the color mapped in the lutTexture. Note, the texture2D acts as an image array accessor picking the 2d value supplied by the vec2.


* Like everything else on Earth, if you know how something works, it's really simple. This took me a good half week to figure out. I'll argue if you know the internals of GLES, this is a rookie-level problem. But if you're new to the GL world, this'll make you pull your hair!


Popular posts from this blog

[Breaking News] AI takes over universe to calculate Pi

Firebase Auth | The Debug vs Release Signature Problem

Finding My Parter