Posts

Showing posts from 2018

Two years at Tonbo Imaging

This year, compared to last year has been a lot more of software development and making the cameras a finished product - a device that can be sold with the highest quality standards. If you've worked on taking a proof-of-concept stage demo unit to a high-grade production ready, you'd know how time-consuming this is.
Media Libray for Android - Lib Graf
This is probably the company's first Media Library. A lot of the older systems use the same code copy-pasted from one git repository to another for media streaming or creating viewer applications with minor changes for different products. This a step towards stopping that and is specifically for Android devices. Works on Android Handheld and Wear devices.
Read more at rish.space/projects/lib-grafCamera SDK & Camera-Client Architecture
Tonbo has made over 50 different camera devices ranging from 80-pixel tiny cameras to handheld units to ones that go on helicopters. There's a lot of similar code over dozens of repositorie…

Limited by Nature

TLDR; We've been told the sky is the limit to imagination ever since we were kids, but really, all we can think of is limited by what we feel, see or experience from nature.

What makes us human? Is it our ability to feel happy, pleasure, sorrow, those triggered chemicals in our body we call emotions? Or is it our consciousness, our ability to think? We put a great deal of stress on our ability to philosophize, our nature of questioning everything we've been given - the only species to question it's existence - to think of genius or tackle things from a new angle and work collectively towards the betterment of human society.

I'm convinced by my will to not just happily accept that all-we-see-is-all-life-is, that the space we live in, the way we perceive time, the taken-for-granted lifespan given to us - none of it is as simple as we see it, that it probably is a form of a simulation and we'relimited by what we've been programmed into thinking. It's just that…

You say you want to use your phone?

Image
If you have a brother, you know the satisfaction derived from constantly nagging, pinching, purposefully singing loudly when he's asked you to stay shut because he's working on something, saying why to everything on and on and on, asking silly questions, and asking them on repeat, obnoxiously staring at him, etc. etc. Well, now that he's in college and I live in Bangalore, I can't do all that. But I can make his phone unusable!

Paaras: "Bro. I'm really busy. Don't annoy me okay?"
Rish: "Of course not brother dear."
Paaras: "What?"
Rish: "Annoy you I will, and annoy you I shall. B|"
Paaras: "Go away."

...

Another 36 minutes later.

I wrote a Python script that mimics touch commands over adb to your Android device. It called my brother every 10 seconds, making his phone unusable when he was connected to the Internet. Except of course, he blocked me after this.
The script on Github.

An Architecture for a hardware-software system II

This is our solution at Tonbo Imaging to the question I put up here.

Very broadly, I'll break our solution into:

1. The various components and gluing them together
A whole lot of Managers that handle various aspects of the software, the Views, a CommandParser, and MessageResponse Queue, all glued together by an internal RxBus - asynchronously sending messages from various threads to one another.

2. Device Capabilities + Autogen UI
Each camera broadcasts its capabilities, a json file containing a list of commands it can take. Each command contains its parameters, their datatypes and the list/range of values they can take. A lot of the UI is autogenerated from this.

3. Versioning
This was fairly tricky and we're still not 100% sure we're happy with what we have. We have to take care of the versioning between various components:
- camera OS ⇌ camera software
- camera software ⇌ clients
- camera software ⇌ web

4. Camera-Graphics-Display-Streaming 
I'll add the graphics librar…

An Architecture for a hardware-software system

Image
You are to design a single camera software, various client applications, and a communication model for a family of cameras. Each camera runs the Android OS, and will be crafted to make several variants; each of which takes up different forms. Each camera has various channels and interfaces to communicate with.

Variables in the camera variants:
headless or on-device display
ie, the camera may be wireless only; it may have an LED as an indicator, or it may have an OLED or LCDperform on-device analytics
not each camera is allowed to perform on-board analytics; some could do stabilization, some face detection, some motion detection, some may perform all. This is limited by what algorithms the customer buys.camera sensor resolution, fps, camera lens
every variant has a different sensor and lens specificationmultiple camera sensors
some cameras might have multiple sensors; thermal, near IR, or visible lightmotor-driven PTZ
the camera might have a pan-tilt-zoom driven by motors... and many more th…

Lessons from drumming classes

Image
Now, my drums teacher is quite a strict man. He's been teaching for over 10 years and has performed all over the country for another 10 years prior to that. Not everyone approves of his way of teaching, and he ensures he admits only those students fine with his strict ways. I've been attending two-hour classes once every Thursday, for the past 4 months now, and he has not yet let me play simple quarter note exercises. Most drum classes teach this during the first couple of weeks; some even start with these on their first day. He, on the other hand, has forced me to learn how to read notes, practice every exercise from the slowest 60 beats per minute to 240 beats per minute, and engraved in my mind that a great drummer is not one that can play a jazzy riff or perform a fancy dynamic, but one that understands drumming is all about timekeeping.

Here's a list of parallels I've drawn from my life outside of drums.

I ignored understanding Fourier transforms in my 1st year of…

False coloring a video in real time using lookup tables

Image
I've been working on OpenGLES very intensively for video rendering and applying real-time filters for months, and I haven't yet put up a blogpost on one of my favorite filters yet. I love it because it shows how fast and simple things can be in the GPU if you know how to use it well;
What you have to do is given a greyscale image, false color it with a provided mapping from grayscale to RGB values; i.e.,

Grey [0,255] --> R[0,255], G[0,255], B[0,255]
In a traditional CPU world, this is fairly simple. All you have to do is loop the image for each pixel, and use the mapping to convert the grey pixel to an RGB. But this is insanely slow, and cannot be done in real time. In comes the GPU! It's perfect for repetitive tasks, like this simple image processing problem where you have to apply the same function to every pixel of an image. Here's what the output looks like. The center image is the original greyscale image. My favorite color palette is the one on the top right. …

Fully Duplex Multi Client Socket Communication

Image
I’ve written a small, complete and reliable socket communication library. I wanted to avoid using the nio package and use the default ServerSocket and Socket classes provided in the java.net package.

Fully Duplex; Async
This means the server and client can asynchronously send messages to each other whenever they’d like. This is different than most libraries where the server is only listening for messages and responds when a client requests for some information.

Multi-Client
The server can host as many clients as the internal sockets implementation allows. Java’s ServerSocket implementation sets the size to 50 connections.
The server can also restrict the number of simultaneous clients it’d like to host at once.
Any client can send a message to any other client.

Server-Client symmetric API
Effectively, it really doesn’t matter who the server is; each device in the network gets a callback when a device connects or disconnects, and each client is assigned IDs by the server.
The server has s…

WiFi Direct on Android | IT IS TERRIBLE

Wifi Direct; a  wireless communication channel that runs in parallel with your Wifi connection, guaranteeing point-to-point communication at speeds much faster than BLE. It sounds perfect for a local multiplayer game, or a camera peer network. It works up to 20 meters in open space and penetrates a single wall given your antenna is strong.

Sounds perfect? I really wish it was. I spent over a month trying to ensure it works on as many devices as possible. In principle, it works exactly the same as BLE. You make your device visible for a wifi direct connection, and another device upon discovering it connects to the former.

If you're planning to use Wifi Direct, I'd suggest you don't and figure out another mechanism.

Here's a list of problems I faced:
The SDK:The heart of the issue is each phone functions differently.The Android Software Development Kit ensures all devices above API 19 provide the same set of APIs, but the lower layers manufacturer implementation is differe…

Java's Primitive Datatypes are Signed!

Image
It took me a whole 6 hours to write a color conversion from the YUV420P color space to the RGB color space. I thought I had a fairly good understanding of the various color formats (YUV420P, SP, 422, etc etc) and how to access individual Y, U, and V components; but I spent the better half of the 6 hours reading and re-reading the conversions and theory. The converted output's colors were completely messed up, and the images looked something like this:

The colors are all messed up; but notice you don't see the 4 quadrant ghosts as you'd usually see in YUV conversions that go wrong. This hinted that my conversions/element access wasn't wrong.*
Note, you needn't know what each of these really is or the formula of the conversion behind this. The crux of the problem was converting the given byte[] array input, to an int[] array output. I struggled because of a very simple yet hair-pulling gotcha; but it took me hours to realize where I was going wrong. All primitives i…

Bhutan | Fun Facts

I really enjoyed traveling Bhutan, but there were several things that stood out, felt very out of place, or were just surprising and I thought this would make an interesting read.

Naming Things
The Bhutanese aren't very creative while naming things. Everything is Druk-something; they have the Druk Airport, Druk Beer, Druk Bank, Druk Hotel, and the list goes on. The other common name is Tashi. You'll find an equal number of Tashi-somethings there.

Escaping materialism but still expensive?
Being the Buddhist capital of the world, preaching to let go of the material world and all its enticements, ironically it's a very expensive nation to travel. The government sets the price to basic necessities - traveling from place A to B, the price of fruits and vegetables, etc, and it is far more expensive than you'd expect. For a foreigner it's even more so - a fixed rate of $200 USD applies to anyone visiting without an Indian passport.

Killing animals isn't okay, but eating…

Bhutan | Monasteries & Impermanence

Image
I visited a couple monasteries in Paro. The fun part was always getting to know more about these places from the monks living there. As an Indian, you are allowed to roam around where-ever you'd like; but for a foreigner, the Bhutanese government mandates every visitor to have a personal tour guide - this is probably done in an attempt to increase jobs. There are guides who know French, German and Spanish! If you're lucky, you could ask to tag along with one of these foreign visitors explaining things in English. You'll learn a lot more than just walking about on your own.
Inside one of the monasteries in Paro. I asked a tour guide how the monastery decides whom to admit, and how a monk spends his day. The procedure to get into a monastery is like you'd apply to any school - there is a short interview, and if there are vacancies, you get a free pass to daily meals, a room to stay in, and the companionship of other monks studying along with you. Throughout their life, …

Bhutan | Lango & Thimphu

Image
My plan for Bhutan was fairly simple from the time I planned the trip. I wanted to reach there, figure things out as I go, and spend a good 2 weeks - to just be. I wanted to spend some time in a remote village, and some time in a busy city. And I did exactly that. I did nothing more than read my books, sit outside in the sun, and walk for long hours on quiet roads, to justbe. 

I spent 7 days in Lango, a small village 7km ahead of Paro, at a homestay. Lango, just as the rest of Bhutan, is surrounded by mountains. There is only one road that passes through Lango - the one that takes you to Paro, and the entire village is built around this road. There aren't further smaller roads inside. There is a narrow river parallel to the road, a small stretch of which is adopted by a school to clean and protect from further pollution. It is probably one of the cleanest rivers I've seen in a populated locality. The frequency of cars passing by is probably one per a minute, and to my surpris…

Backpacking Varanasi

Banaras. A city confused between the history of its mythical past, present day religion and religious ceremonies, and the influence of modernization and the West. In under ten minutes you can go from feeling as if you’ve stepped into history, to suddenly a dejavu resemblance to yet-another-modern-Indian-city. From sitting in an electric rickshaw to spotting cycle rickshaws and normal autos, Olas and Ubers, pass by a Domino’s surrounded by several small vegetable stores, handicraft stores and pharmacies, see a Pantaloons or a large city mall, and in no time reach super crowded streets queued up by people in hundreds to visit yet another mandir. And if you walk a little further, you’d find yourself by the Ganga at a funeral ghat. All in 10 minutes.
The city is full of color, graffiti-ed with imagery of Shiva, life all around you is alive - busy streets and constant honking, people rushing about but stopping for a second to pray as they pass the dozens of temples on every street. If you b…

Backpacking Rishikesh

The Land of the Bholenath. You may be outside a temple, walking the busy streets across the Lakshman Jhula, or be sitting by the Ganga, you are bound to find a Sadhu, a hippie in the strangest of clothing, or a local in his everyday attire casually slipping a kash of ganja.
The story of how Rishikesh went from becoming an Above of Gods and a land of several myths and legends to the Yoga capital of the nation and a place to let your hippie side out is particularly interesting. The Beatles came in search for answers to life’s larger questions in 1968, and ended up performing transcendental meditation - I’m certain they were merely stoned, and were given the freedom to sit and ponder by the holy Ganga - but it’s here they wrote their most famous album, the White Album. There was then a second wave of foreign artists coming in search for a story of their own, and this is where the locals realized these visits could become a major tourist attraction.
There are now a dozen privately owned hos…

Skiing in Auli

Image
Auli is India’s ski destination. It’s at an altitude of 3300 meters at the base, and touches 3800m at the peaks, it has the right ski infrastructure - ski lifts and chair cars, and even a ropeway certified by a Swiss cable car agency, it has the perfect ski slopes - slopes where India can host International winter games, half of them man-made, the other half natural slopes, and oak forests all around preventing greatly the possibility of avalanches. It really is a perfect ski destination.
EXCEPT, THERE IS NO SNOW!
We humans have messed this world up. It isn’t snowing during the months it should, and when it does the snow contains too much water for the snow to last till Summer begins, causing muddy slush everywhere you walk. During the 1970’s, Auli used to receive an average of 10 feet snow every year. Till 2010, it reduced to about 2-3 feet of snow on the ski slopes. During this period, the Uttarakhand government set up all the ski infrastructure and made international level slopes ho…

Month Long Backpacking | India & Bhutan

It’s been a year working in the hectic life of a startup, and now I need a break. Early October, the monotony of routine and work life started to hit me. I was working far too much for far too many hours a day, and could feel the weariness and lack of productivity set in.
I needed a month off - I was certain a month without thinking of thermal imaging, Android, stabilization and cameras would help me hit the reset button. I was extremely nervous when I went up to my manager. I had practiced asking this for weeks, played out the conversation in my head a hundred times, I knew exactly how I’d build the story and the tone I’d use for each sentence, and how I’d make it sound convincing enough that I really need time off. When I finally asked him I was surprised at how casually he said “4 weeks? Sure - you didn’t need the story, you just had to say you want 4 weeks off.”
And so here it begins! I’m taking a month off to travel the North India, and Bhutan. I’m really looking forward to skiin…

I'm not an Android Dev Anymore

tl;dr: I merely use Android as a platform to write software, and that’s where the subtle difference lies.
Yes, I write code that runs on Android, but this doesn’t mean I’m an Android developer.
I’m glad this realization has struck. My first hand at writing useful code was during my freshman year simply so I could ask my father for an Android phone; I had a Windows Lumia then and it sucked. I started out with simple games and to-do applications, and gradually wrote and contributed to over a dozen applications specifically developed to run on Android and to be published on the Android Playstore - this included making Android variants of websites using their REST APIs, creating replicas of already existing iOS apps. Developing Android-ized experiences of a larger set of software comprises of the work an Android developer does.
In the past year however, I’ve shifted to reading research papers and writing POC implementations, research models on Android, merely because I know the ins and outs …

Caged in the City

Sneezing. Coughing. Always tired. Always checking the time. Rushing. Cursing. Shouting. Moving. Tracking progress. Checking for updates. Complaining. Questioning. Always checking the time. Planning. Postponing. Forgetting. Remembering. Regretting. Remembering. Noting. Never completing. Checking for updates. Discussing. Promising. Leaving. Apologizing. Always checking the time. Everything now. One button gratification. Hacky way. Wrong way. Alas, quick way. Calculating balances. Forever doubting. Checking for updates. Missing food. Delaying sleep. Missing people. Never calling. Always too busy. Promising you’ll improve. Always a tomorrow. Checking for updates. Always checking the time. Cursing the routine. Repeating.
-
This isn’t everyone’s story of their work life in the city, but so far it has been mine. I do not feel I’ve liveda year worth of life in the last 12 months - this doesn’t mean I haven’t had my ups - I’ve partied hard, spent good days with friends and close ones, went on mini vacations, have played…