What 3 Weeks of Deep Learning Have Taught Me

I've almost completed the FastAI course for deep learning, and here's a list of things I've learnt:
  • Deep learning is far from magic. So far I was convinced there was more to deep learning than just matrix multiplications and 11th-grade math. 3 weeks haven't shown me the signs.
  • It's hard, or it feels like magic because humans struggle to visualize beyond 3D space.
  • And so, for my first project, I made a simple classifier that predicts the maximum number from a list of 2 numbers. This can be visualized in 3D. More on why I did this in another blogpost, but in 3 lines here's what I learnt:
    • [activation(input x weights + bias)]many times = output
      loss = how off the output is from what it should have been,
      differentiate loss with the weights,
      and keep fixing the weights,
      until you're happy. 
  • The truth is, you'll never really know if you should be happy with your output. 
  • That's because don't always know if you've reached the most optimal solution to a problem. It'll take your computer way too long to go check every value possible.
  • It's very important that you use big long words that sound impressive otherwise normal people will think they can do it too. ;)
    • And if you want to be really exclusive you shorten the long versions of the names to short acronyms like ReLU to show you're in the exclusive team.
  • Lots of people have made a great business out of the hype. On YouTube for example,
    • Some make silly videos that aren't great to follow/understand from, but have a lot of followers because they're the perfect YouTube vlogger type - Siraj Raval
    • And some make really, really good videos on how they used deep learning for their own projects. Jabrils.
  • Keras is a lot, a lot better than other languages. If you know the theory decently well, nothing in Keras feels odd. It lets you design the network exactly how you want to.
  • FastAI, PyTorch on the other hand, abstract out many things making it hard for you to follow what they're doing inside unless you go check the source code.
  • My previous blog on classifying FIFA images has 5x the number of views my usual posts get. The deep learning hype is real, and I'm all aboard the soon-to-be-deep-learning-engineer train.


Popular posts from this blog

Firebase Auth | The Debug vs Release Signature Problem

A Letter To An Old Love

Things I Could Do (ft. Homer)