Posts

Two years at Tonbo Imaging

This year, compared to last year has been a lot more of software development and making the cameras a finished product - a device that can be sold with the highest quality standards. If you've worked on taking a proof-of-concept stage demo unit to a high-grade production ready, you'd know how time-consuming this is.
Media Libray for Android - Lib Graf
This is probably the company's first Media Library. A lot of the older systems use the same code copy-pasted from one git repository to another for media streaming or creating viewer applications with minor changes for different products. This a step towards stopping that and is specifically for Android devices. Works on Android Handheld and Wear devices.
Read more at rish.space/projects/lib-grafCamera SDK & Camera-Client Architecture
Tonbo has made over 50 different camera devices ranging from 80-pixel tiny cameras to handheld units to ones that go on helicopters. There's a lot of similar code over dozens of repositorie…

Limited by Nature

TLDR; We've been told the sky is the limit to imagination ever since we were kids, but really, all we can think of is limited by what we feel, see or experience from nature.

What makes us human? Is it our ability to feel happy, pleasure, sorrow, those triggered chemicals in our body we call emotions? Or is it our consciousness, our ability to think? We put a great deal of stress on our ability to philosophize, our nature of questioning everything we've been given - the only species to question it's existence - to think of genius or tackle things from a new angle and work collectively towards the betterment of human society.

I'm convinced by my will to not just happily accept that all-we-see-is-all-life-is, that the space we live in, the way we perceive time, the taken-for-granted lifespan given to us - none of it is as simple as we see it, that it probably is a form of a simulation and we'relimited by what we've been programmed into thinking. It's just that…

You say you want to use your phone?

Image
If you have a brother, you know the satisfaction derived from constantly nagging, pinching, purposefully singing loudly when he's asked you to stay shut because he's working on something, saying why to everything on and on and on, asking silly questions, and asking them on repeat, obnoxiously staring at him, etc. etc. Well, now that he's in college and I live in Bangalore, I can't do all that. But I can make his phone unusable!

Paaras: "Bro. I'm really busy. Don't annoy me okay?"
Rish: "Of course not brother dear."
Paaras: "What?"
Rish: "Annoy you I will, and annoy you I shall. B|"
Paaras: "Go away."

...

Another 36 minutes later.

I wrote a Python script that mimics touch commands over adb to your Android device. It called my brother every 10 seconds, making his phone unusable when he was connected to the Internet. Except of course, he blocked me after this.
The script on Github.

An Architecture for a hardware-software system II

This is our solution at Tonbo Imaging to the question I put up here.

Very broadly, I'll break our solution into:

1. The various components and gluing them together
A whole lot of Managers that handle various aspects of the software, the Views, a CommandParser, and MessageResponse Queue, all glued together by an internal RxBus - asynchronously sending messages from various threads to one another.

2. Device Capabilities + Autogen UI
Each camera broadcasts its capabilities, a json file containing a list of commands it can take. Each command contains its parameters, their datatypes and the list/range of values they can take. A lot of the UI is autogenerated from this.

3. Versioning
This was fairly tricky and we're still not 100% sure we're happy with what we have. We have to take care of the versioning between various components:
- camera OS ⇌ camera software
- camera software ⇌ clients
- camera software ⇌ web

4. Camera-Graphics-Display-Streaming 
I'll add the graphics librar…

An Architecture for a hardware-software system

Image
You are to design a single camera software, various client applications, and a communication model for a family of cameras. Each camera runs the Android OS, and will be crafted to make several variants; each of which takes up different forms. Each camera has various channels and interfaces to communicate with.

Variables in the camera variants:
headless or on-device display
ie, the camera may be wireless only; it may have an LED as an indicator, or it may have an OLED or LCDperform on-device analytics
not each camera is allowed to perform on-board analytics; some could do stabilization, some face detection, some motion detection, some may perform all. This is limited by what algorithms the customer buys.camera sensor resolution, fps, camera lens
every variant has a different sensor and lens specificationmultiple camera sensors
some cameras might have multiple sensors; thermal, near IR, or visible lightmotor-driven PTZ
the camera might have a pan-tilt-zoom driven by motors... and many more th…

Lessons from drumming classes

Image
Now, my drums teacher is quite a strict man. He's been teaching for over 10 years and has performed all over the country for another 10 years prior to that. Not everyone approves of his way of teaching, and he ensures he admits only those students fine with his strict ways. I've been attending two-hour classes once every Thursday, for the past 4 months now, and he has not yet let me play simple quarter note exercises. Most drum classes teach this during the first couple of weeks; some even start with these on their first day. He, on the other hand, has forced me to learn how to read notes, practice every exercise from the slowest 60 beats per minute to 240 beats per minute, and engraved in my mind that a great drummer is not one that can play a jazzy riff or perform a fancy dynamic, but one that understands drumming is all about timekeeping.

Here's a list of parallels I've drawn from my life outside of drums.

I ignored understanding Fourier transforms in my 1st year of…

False coloring a video in real time using lookup tables

Image
I've been working on OpenGLES very intensively for video rendering and applying real-time filters for months, and I haven't yet put up a blogpost on one of my favorite filters yet. I love it because it shows how fast and simple things can be in the GPU if you know how to use it well;
What you have to do is given a greyscale image, false color it with a provided mapping from grayscale to RGB values; i.e.,

Grey [0,255] --> R[0,255], G[0,255], B[0,255]
In a traditional CPU world, this is fairly simple. All you have to do is loop the image for each pixel, and use the mapping to convert the grey pixel to an RGB. But this is insanely slow, and cannot be done in real time. In comes the GPU! It's perfect for repetitive tasks, like this simple image processing problem where you have to apply the same function to every pixel of an image. Here's what the output looks like. The center image is the original greyscale image. My favorite color palette is the one on the top right. …

Fully Duplex Multi Client Socket Communication

Image
I’ve written a small, complete and reliable socket communication library. I wanted to avoid using the nio package and use the default ServerSocket and Socket classes provided in the java.net package.

Fully Duplex; Async
This means the server and client can asynchronously send messages to each other whenever they’d like. This is different than most libraries where the server is only listening for messages and responds when a client requests for some information.

Multi-Client
The server can host as many clients as the internal sockets implementation allows. Java’s ServerSocket implementation sets the size to 50 connections.
The server can also restrict the number of simultaneous clients it’d like to host at once.
Any client can send a message to any other client.

Server-Client symmetric API
Effectively, it really doesn’t matter who the server is; each device in the network gets a callback when a device connects or disconnects, and each client is assigned IDs by the server.
The server has s…

WiFi Direct on Android | IT IS TERRIBLE

Wifi Direct; a  wireless communication channel that runs in parallel with your Wifi connection, guaranteeing point-to-point communication at speeds much faster than BLE. It sounds perfect for a local multiplayer game, or a camera peer network. It works up to 20 meters in open space and penetrates a single wall given your antenna is strong.

Sounds perfect? I really wish it was. I spent over a month trying to ensure it works on as many devices as possible. In principle, it works exactly the same as BLE. You make your device visible for a wifi direct connection, and another device upon discovering it connects to the former.

If you're planning to use Wifi Direct, I'd suggest you don't and figure out another mechanism.

Here's a list of problems I faced:
The SDK:The heart of the issue is each phone functions differently.The Android Software Development Kit ensures all devices above API 19 provide the same set of APIs, but the lower layers manufacturer implementation is differe…

Java's Primitive Datatypes are Signed!

Image
It took me a whole 6 hours to write a color conversion from the YUV420P color space to the RGB color space. I thought I had a fairly good understanding of the various color formats (YUV420P, SP, 422, etc etc) and how to access individual Y, U, and V components; but I spent the better half of the 6 hours reading and re-reading the conversions and theory. The converted output's colors were completely messed up, and the images looked something like this:

The colors are all messed up; but notice you don't see the 4 quadrant ghosts as you'd usually see in YUV conversions that go wrong. This hinted that my conversions/element access wasn't wrong.*
Note, you needn't know what each of these really is or the formula of the conversion behind this. The crux of the problem was converting the given byte[] array input, to an int[] array output. I struggled because of a very simple yet hair-pulling gotcha; but it took me hours to realize where I was going wrong. All primitives i…