One Year at Tonbo Imaging

Tonbo Imaging is the best company to have started my career with, for I've had the opportunity to do something I got interested in at the very start of college; developing software for tangible systems. It has been 6 months of internship and 6 months of full time employment, and I'm certain I've learnt more than I ever could have during my four years at college. I've been very fortunate to have a very knowledgeable mentor, and a manager who gives us a lot of space to think and ideate, and spend time on research.

In the past year, I've worked on developing an ecosystem of small-sized thermal cameras running a custom Android OS that perform video analytics, are controlled remotely and support global live streaming. It's taken a year to go from an idea to a proof of concept product - we're in no way scalable yet, and are continuously going through architectural changes while ideating and understanding the potential of this system. Most of my work has been R&D turned demo-ready POC product.
  • Understanding the internals of a camera and video
    The camera is an extremely complex system and I now understand how they work, and the hardware components and camera-lingo that goes along with them.
  • Internals of Android Framework & the AOSP
    I've delved deeper than the application layer in Android, understanding how Android is designed, specifically,
  • Video stabilization using an IMU
    Worked on stabilizing video input and making it appear as if it was captured using a stable tripod.
    This has been by far the most challenging projects this year. I've had to learn 3D geometry, OpenGLES, and work with gyroscopes and smoothen their input using various filters. Specifically, the algorithm
  • Face detection in thermal video
    I worked on detecting faces and bodies in thermal video. I used a HoG feature descriptor and fed its results to a linear SVM. Since this was computationally expensive, I detected once every 10 frames and tracked for the remaining 9 frames using a KLT optical flow tracker, detecting feature points to track using Harris corner detection. I wish I had the time to delve deeper and try more algorithms then, though this is what I shall be working on in 2018.
    View face detection + counting in thermal video results here.
  • GStreamer, a media framework
    I disliked GStreamer for the first couple months, and then I fell in love with it. It's fairly difficult to wrap your head around how things work here, and I still end up spending days to understand/implement new concepts. I now understand how videos are stored, streamed, interact with displays, and a little bit about codecs and muxers, and various video color formats (RGB, YUV and the conversions between them).
    I created a GStreamer Cookbook in C and Python for anyone trying their hand at GStreamer. 
  • Micro libraries for AndroidMost of our code has been divided into small reusable libraries. I've developed from scratch the following :
    • Wifi Direct Peer to Peer Connection Library
      Wifi Direct on Android is difficult. This post summarizes precisely all the challenges I faced - including the time division multiplexing of sharing an antenna, the difficulties in peer discovery and how different hardware implements the Wifi Direct stack differently.
      However, I was able to create an reliable yet insecure P2p library that works in all phones we've tested over 95% of the times. That's a success - I shall be posting a couple of blogposts on this soon, and open sourcing the library on Github midway 2018.
    • Fully duplex, multi-client, secure Sockets Library
      It'll come to you as a surprise that there is no Socket library (I may be wrong, but I could not find any online) that allows fully duplex communication, allows multi-client-single-server, is reliable and secure.
      I shall be posting the challenges I faced here and open sourcing the library around the same time as the Wifi Direct library.
    • Wifi Connector Microlibrary
      Another surprise; post Android Marshmallow, connecting to a Wifi router is quite a challenge! I'll save the details for another post, but in my findings I ended up bricking the Wifi Service on 3 phones before realizing it was an earlier version of my library doing the magic.
      I created a Github Gist for managing Wifi connections that now works on all devices, on all versions.
    • File Logger Microlibrary
      I needed a way to log to a file instead of to the logcat monitor. I created a microlibrary to do exactly this; syntactically the same as traditional Log.d(TAG, "message-here")  except
      logs to a file in the SD card via FileLog.log(file-path, "message-here').
      I created a Github Gist for logging to files.
  • Framework design and architecture
    Late 2017, around the time our production ready hardware arrived and the business team started finding potential customers we could deliver to midway 2018, I started deep diving into architectural design and figuring out how we could version our software, hardware, design capabilities based by product variants that could be sold by the business team. This is WIP and is something I will focus on in 2018.
    So far, we've set up a stable versioning system between our software (web, Android camera, and Android client), and a git workflow that suits us.
  • Hiring & its challenges
    I hired a full time Android Engineer and a Python webdev intern, and boy was it difficult. I've always stressed on a company is only as good as it's people, but it's an insanely difficult challenge to hire good engineers with the right mindset. I hired in exactly the way I'd have to loved to be hired; we gave out difficult projects with tight time constraint to candidates we shortlisted from LinkedIn/Angellist, tracking their progress via a shared Bitbucket repository. I'll put up the details in another blogpost.
All in all, this year has been insanely productive at work, though I've faced a couple of ups and downs on the personal front. I'm gradually getting a hang of completely shutting down and taking breaks over weekends, not spending them wasted with friends and girlfriends, I've joined a co-exercising place (it's literally anything but a gym, though it's great!), and am making time for my personal projects.

Detecting my face in a thermal video. I love this picture!
-

A pat on the back to my one-year-younger-self who made this happen. Read Zoo of Jobs when I almost made the mistake of joining Oracle where none of this would have been possible and A Company I'd Love to Work For for context.

Comments

Popular posts from this blog

[Breaking News] AI takes over universe to calculate Pi

Firebase Auth | The Debug vs Release Signature Problem

Cold Showers