One Year at Tonbo Imaging

Tonbo Imaging is the best company to have started my career with, for I've had the opportunity to do something I got interested in at the very start of college; developing software for tangible systems. It has been 6 months of internship and 6 months of full time employment, and I'm certain I've learnt more than I ever could have during my four years at college. I've been very fortunate to have a very knowledgeable mentor, and a manager who gives us a lot of space to think and ideate, and spend time on research.

In the past year, I've worked on developing an ecosystem of small-sized thermal cameras running a custom Android OS that perform video analytics, are controlled remotely and support global live streaming. It's taken a year to go from an idea to a proof of concept product - we're in no way scalable yet, and are continuously going through architectural changes while ideating and understanding the potential of this system. Most of my work has been R&D turned demo-ready POC product.
  • Understanding the internals of a camera and video
    The camera is an extremely complex system and I now understand how they work, and the hardware components and camera-lingo that goes along with them.
  • Internals of Android Framework & the AOSP
    I've delved deeper than the application layer in Android, understanding how Android is designed, specifically,
  • Video stabilization using an IMU
    Worked on stabilizing video input and making it appear as if it was captured using a stable tripod.
    This has been by far the most challenging projects this year. I've had to learn 3D geometry, OpenGLES, and work with gyroscopes and smoothen their input using various filters. Specifically, the algorithm
  • Face detection in thermal video
    I worked on detecting faces and bodies in thermal video. I used a HoG feature descriptor and fed its results to a linear SVM. Since this was computationally expensive, I detected once every 10 frames and tracked for the remaining 9 frames using a KLT optical flow tracker, detecting feature points to track using Harris corner detection. I wish I had the time to delve deeper and try more algorithms then, though this is what I shall be working on in 2018.
    View face detection + counting in thermal video results here.
  • GStreamer, a media framework
    I disliked GStreamer for the first couple months, and then I fell in love with it. It's fairly difficult to wrap your head around how things work here, and I still end up spending days to understand/implement new concepts. I now understand how videos are stored, streamed, interact with displays, and a little bit about codecs and muxers, and various video color formats (RGB, YUV and the conversions between them).
    I created a GStreamer Cookbook in C and Python for anyone trying their hand at GStreamer. 
  • Micro libraries for AndroidMost of our code has been divided into small reusable libraries. I've developed from scratch the following :
    • Wifi Direct Peer to Peer Connection Library
      Wifi Direct on Android is difficult. This post summarizes precisely all the challenges I faced - including the time division multiplexing of sharing an antenna, the difficulties in peer discovery and how different hardware implements the Wifi Direct stack differently.
      However, I was able to create an reliable yet insecure P2p library that works in all phones we've tested over 95% of the times. That's a success - I shall be posting a couple of blogposts on this soon, and open sourcing the library on Github midway 2018.
    • Fully duplex, multi-client, secure Sockets Library
      It'll come to you as a surprise that there is no Socket library (I may be wrong, but I could not find any online) that allows fully duplex communication, allows multi-client-single-server, is reliable and secure.
      I shall be posting the challenges I faced here and open sourcing the library around the same time as the Wifi Direct library.
    • Wifi Connector Microlibrary
      Another surprise; post Android Marshmallow, connecting to a Wifi router is quite a challenge! I'll save the details for another post, but in my findings I ended up bricking the Wifi Service on 3 phones before realizing it was an earlier version of my library doing the magic.
      I created a Github Gist for managing Wifi connections that now works on all devices, on all versions.
    • File Logger Microlibrary
      I needed a way to log to a file instead of to the logcat monitor. I created a microlibrary to do exactly this; syntactically the same as traditional Log.d(TAG, "message-here")  except
      logs to a file in the SD card via FileLog.log(file-path, "message-here').
      I created a Github Gist for logging to files.
  • Framework design and architecture
    Late 2017, around the time our production ready hardware arrived and the business team started finding potential customers we could deliver to midway 2018, I started deep diving into architectural design and figuring out how we could version our software, hardware, design capabilities based by product variants that could be sold by the business team. This is WIP and is something I will focus on in 2018.
    So far, we've set up a stable versioning system between our software (web, Android camera, and Android client), and a git workflow that suits us.
  • Hiring & its challenges
    I hired a full time Android Engineer and a Python webdev intern, and boy was it difficult. I've always stressed on a company is only as good as it's people, but it's an insanely difficult challenge to hire good engineers with the right mindset. I hired in exactly the way I'd have to loved to be hired; we gave out difficult projects with tight time constraint to candidates we shortlisted from LinkedIn/Angellist, tracking their progress via a shared Bitbucket repository. I'll put up the details in another blogpost.
All in all, this year has been insanely productive at work, though I've faced a couple of ups and downs on the personal front. I'm gradually getting a hang of completely shutting down and taking breaks over weekends, not spending them wasted with friends and girlfriends, I've joined a co-exercising place (it's literally anything but a gym, though it's great!), and am making time for my personal projects.

Detecting my face in a thermal video. I love this picture!
-

A pat on the back to my one-year-younger-self who made this happen. Read Zoo of Jobs when I almost made the mistake of joining Oracle where none of this would have been possible and A Company I'd Love to Work For for context.

Comments

Post a Comment

Popular posts from this blog

Finding My Parter

Finding my partner feels more like finding  my kind . I’m this mix of an Indian boy with Indian familial ties, but European hobbies and a European way of leading life. I've lived in both worlds for 15 years each. There aren't many that have seen and lived in both worlds, and experienced completely different ways of life. Maybe I am asking for too much, but I don’t understand why one wouldn’t. If I have high standards for most things that I do in life, why wouldn’t I hope or want my partner to think from the same lens? Here's my value system. Be generally optimistic. A positive, happy person whose baseline state is to lean towards an optimistic outlook towards everything in life. Be fit. If one is not physically fit, there's low chance they're mentally fit. Plus, being fit shows discipline and consistency. Be hard working. Everything in life is a project. From raising children, to picking up groceries weekly, to building a home. And no one wants to work with a lazy ...
Continue reading »

Two Worlds, Two Lives

I’ve been in India for the past two months now, and for the first time in my life, I can imagine myself live here again. Nothing drastic has changed in my day to day life, nor in either of the nations per se, but maybe I’ve had enough time to experience the flip-side to know what India’s strengths are. I’ve always acknowledged how I get the chance to live in two different worlds. But this is the very first time I’ve realized I also lead two different lives each time I switch countries. For me, Germany is a nation where you put the self first - you learn about yourself as an individual. It’s also a country where you learn to live life in the way they show us in movies - walks in parks, play sports any time of the day, vacations in every season the climate has to offer, meetups at little cafes with chairs out on the street. Work is secondary, rarely one’s primary driver in life. Therefore, it is also rarely one’s identity. The country and the mindset is averse to change. The philosophy h...
Continue reading »

Keeping Fit in India

I have a bold claim to make: As compared to Europe, it's much easier to get and keep fit in India. But it's far less likely. Easier because: Relatability In India, you can relate to the others' bodies. Everyone around you has the same body style, our carb heavy diet that causes so many of us to be skinny fat. And so when you go to a calisthenics gym, it really helps to be able to look around and see that a person who looks like you is fit . That shines hope, telling you that it is possible in the first place. In Europe, the same is counterproductive. Looking at the European's low body fat, tall yet muscular physique makes me think they're just born with it. (Which is not true. I suppose they really do have great genes, but their meat heavy diet and walkable/cycleable cities help with keeping fit). Food is easier in India. You know your food here. You know the spices we use and how it tastes. It's that familiar feeling of eating something you know, as opposed to...
Continue reading »