Federated Learning

Hi, after a long break, I thought of continuing writing for Geek-Programmer. Things have changed a lot, I got out of grad school, started a job as a software engineer, quit the job, signed up for a PhD in computer science and it’s been more than a year🎉. Time runs pretty fast 🙂

On the way, I learnt a thing or two. My current topic is Federated Learning. Quite an interesting topic though. Federated learning started as a privacy preserved machine learning technique. It has a humble start as a small paper from google. You can start reading it here.

Let’s chat

You have a phone with Google keyboard application. The keyboard will suggest some text while you are typing. But initial suggestions are not that great. However, over time, your suggestions will get better and better. It will know what to suggest even if the word is known by you and your friend.

Well isn’t that a privacy problem? 😀 Does Google know what you type and your secrets? Not really (hope not). They use something called Federated Learning. So basically, what federated learning does is quite simple. Let’s write it in small steps.

1. Google AI servers have a pre-trained “global” machine learning model for the chat application.
2. They send this global model to your phone when you install google keyboard. We now call this “local” model.
3. Now google keyboard reads the text you type and suggest words. You skip the word or select the suggested word. So basically it collects user data and stores them locally in your phone.
4. Google keyboard uses this data to re-train and update the local model.
5. After some training rounds, the keyboard app pushes back the local model to the Google AI server.
6. Server collects many such local models and combines them together to create the next version of the global model.
7. Now repeat from step 2 🙂 until the global/local model reaches a consistent high accuracy.

Hooray! Now you know how federated learning works.

So what’s the big deal?

Have you noticed the difference here compared with old server-based machine learning? You don’t need to send your data to a server. It always stays in your device. So, that’s the big deal. It’s simple. But your privacy is protected. Your chats won’t get uploaded to a Google server so you can chill! 🎉

To learn more about federated learning, I am using this book which I recommend as beginner friendly. I also want to try out this book  if you want to get your hands dirty in federated learning with python!😀 This is also a good read, where we can learn more advanced topics on security and privacy of federated learning.

Hope you enjoyed this post. I’ll plan on adding more content related to AI, python and hope to start a youtube channel soon (really hope!!) If you want to know more about federated learning and their internal working, please comment below 🙂 Share this story with your friends if you find it useful.