Java's Primitive Datatypes are Signed!
It took me a whole 6 hours to write a color conversion from the YUV420P color space to the RGB color space. I thought I had a fairly good understanding of the various color formats (YUV420P, SP, 422, etc etc) and how to access individual Y, U, and V components; but I spent the better half of the 6 hours reading and re-reading the conversions and theory. The converted output's colors were completely messed up, and the images looked something like this:
Note, you needn't know what each of these really is or the formula of the conversion behind this. The crux of the problem was converting the given byte[] array input, to an int[] array output. I struggled because of a very simple yet hair-pulling gotcha; but it took me hours to realize where I was going wrong. All primitives in Java are signed! If you come from a Python-like world, or are ignorant of signed-ness, you'd expect the byte 0x0 to print 0, and 0xFF to print 255.
In Java however, where everything is signed, ((byte) 0xFF) is -1, and not 0x255. Further, simply trying to print 0xFF prints 255; which is because 0xFF is an int. Now, what I wanted to do was simply access a signed byte as an unsigned byte.
I am however super embarrassed at spending 6 hours on this. This shows how developers tend to forget the underlying mechanisms because of all the abstractions high-level languages. On the upside, I recapped all color formats and the formulas behind them!
Here's the source I've put on Github Gists. Note, there's room for more optimization; the conversion to int and then back to byte for each R, G, and B component isn't necessary.
The colors are all messed up; but notice you don't see the 4 quadrant ghosts as you'd usually see in YUV conversions that go wrong. This hinted that my conversions/element access wasn't wrong.*
In Java however, where everything is signed, ((byte) 0xFF) is -1, and not 0x255. Further, simply trying to print 0xFF prints 255; which is because 0xFF is an int. Now, what I wanted to do was simply access a signed byte as an unsigned byte.
Here's the source I've put on Github Gists. Note, there's room for more optimization; the conversion to int and then back to byte for each R, G, and B component isn't necessary.
Comments
Post a Comment