What, Why and How of Machine Learning @ Airtel

Two weeks back, I attended the Singtel Group HR Summit at Bangkok. The theme of the summit was "Amp up for the Future". We were about 40 of us delving deep into the "future of work" in the face of disruptive technological advances such as Machine Learning.

Last week, Airtel also announced a unique tie-up with Coursera, a best-in-class online education and learning provider. This strategic partnership will give Airtel employees in India access to Coursera’s learning platform and equip them with relevant emerging skill sets, including Machine Learning. 

A curious employee queried me, “Why would we skill ourselves on Machine Learning ? What does our business have anything to do with Machine Learning ?". I thanked him for asking the question, and embarked on an interesting round of conversations with my Engineering colleagues to get a few answers to these questions. Thanks also to Singtel and Coursera for sparking the curiosity in my mind.

1.    What is Machine Learning (ML)?

When we talk of ML, we usually imagine a robot (like the Terminator). But, it is not sci-fi anymore, it is reality. One of the earliest forms of ML was Optical Character Recognition (OCR), which enabled scanners to read text, numbers and symbols. The e-mail spam filter is another application of ML that has now become part of the routine. Social media sentiment analysis is another huge use case of ML. Alexa and Siri are bringing ML into everyday lives.

So, what is Machine Learning ? It is the science (and maybe even art?) of programming computers so they can learn from data. It is that field of science which gives computers the ability to learn without being explicitly programmed. Recently, I heard a comment, “Machine learning is an artificial intelligence application that makes computers develop intelligence naturally !”.

2.    But then …. why do we need Machine Learning ?

The simplest answer to this question I got from a colleague was, “Machine Learning is a means to use experience, and get better at doing a task, measured by a performance metric". For example, let us think of a computer programme that runs automated credit controls based on specified rules and decision criteria. That was the traditional programming approach. In contrast, a system based on ML techniques automatically learns from the actual experience of credit transactions, re-sets rules and decision criteria and runs automated credit controls to achieve a certain goal, say, delinquency as % of revenues.

What are those typical problems that ML solves much better than traditional approaches? There are typically four types of problems for which ML is best suited – a) complex problems for which we have no algorithmic solution at the start, b) problems for which existing solutions require long lists of hand-tuned rules, c) in fluctuating environments, and d) finally to help humans learn better and get insights about complex problems and large amounts of data, such as in data mining. ML has enabled design and operationalization of neural networks on a wide scale, in contrast to mostly linear / logical applications in the past.

3.    What is a neural network ?

A neural network is a series of algorithms that attempts to identify underlying relationships in a set of data by using a process that mimics the way the human brain operates. Neural networks have the ability to adapt to changing inputs, so that the network produces the best possible results without the need to re-design the output criteria. Increasingly, neural networks are driving decisions which are not just logic driven, but applying human like soft qualities such as intuition and gut.

Let us illustrate with TensorFlow, a powerful, open source software well suited for large-scale Machine Learning. It can train colossal neural networks with millions of parameters on a training set composed of billions of instances with millions of features each. This does not come as a surprise, since TensorFlow was developed by the Google Brain team and it powers many of Google's services such as Google Search, Google Photos, etc. TensorFlow was open sourced by Google in 2015 !

4.    Understanding types of Machine Learning Systems …. for beginners

From a beginner’s perspective, to better appreciate the use cases of ML, it may be a good idea to first understand the different categories of ML systems.   

a)   Whether or not machines are trained with human supervision. In supervised learning, training data with labels is fed to the algorithm. The supervised ML system typically can perform a task of classification (eg: pirated content or legitimate). It can also perform a task of regression (eg: predicting the value of a second hand car). In unsupervised learning, the training data fed to the algorithm is not labelled, but the ML system learns on its own through different techniques such as : clustering (eg; bunching), visualization (eg: heat maps), dimensionality reduction (eg: component analysis, correlation), anomaly detection (eg: fraud detection) or association rule learning (eg: multi-variate analysis). Combinations of supervised and unsupervised learning are also quite prevalent (eg: Google photos).

Another unique approach is Reinforcement Learning. This is a very different creature. This ML system observes, takes decisions and performs actions, and then gets rewards (and penalties) for the outcomes. The system then learns by itself what is the best strategy for it to get the most rewards over time. What a motivated machine !

b)   Whether the Machine is learning in a batch mode or incrementally. In batch learning (also called offline learning), the system is trained, then it is launched into production and runs without learning anymore; it just applies what it has learnt. For the system to adapt to change, one has to update the data and train a new version of the system, as often as needed. This type of learning requires a lot of computing resources. An alternative is : online learning, where one can train the system incrementally by feeding it data instances sequentially, either individually or in mini batches, enabling the system to learn about the new data on the fly.

When designing online learning systems, one needs to keep the learning rate in mind. If the learning rate is set high, the system will quickly adapt to new data, but it will also quickly forget the old data ! Conversely, if the learning rate is set low, it will learn much slower, but it will also be less sensitive to noise from new data.

c)    How does the machine perform generalization - instance-based or model based? In the instance-based learning, the system learns ‘by heart’ and looks for identical / similar features. Whereas in a model-based learning system, the machine builds a model from the data, and applies the model to make predictions.

5.    Pre-requisites of a robust Machine Learning system

For the ML system to be robust, one needs a lot of data, widely representative of the environment. The system will not perform well if the training data set is too small, or if the data is not representative, noisy or ‘polluted’ with irrelevant features. The model should be neither too simple (under-fit) nor too complex (over-fit). The only way to know how well a model will generalize to new cases is to actually try it out.

One way is to put the model into production, and watch how well it performs. Too risky. Is there another option ? Well, you could split the data into two sets : the training set and the test set. If the training error is low but the generalization error is high, it means that the model is over-fitting the training data. Several approaches of validation and cross-validation are therefore emerging in this field.

6.    What is the context of Machine Learning at Airtel ?

Before we come to actual use cases of Machine Learning at Airtel, let us examine whether we have the right conditions. Globally, Airtel in India has the largest in-country mobile customer base, next to top two Chinese operators. As at end Sept 2017, Airtel had 282 million customers in India, 96.5% of whom were attached to the VLR (indicating subscribers hooked on to the mobile switching centers). These customers generated 437.1 billion minutes of voice traffic, and used up 783.8 billion MB of mobile data in one quarter. Airtel’s network covers 95.3% of the 1.3 billion population in India, and is present in 7,896 census towns and 786,032 non-census towns and villages. Airtel’s huge network is housed in 162,954 towers, which helped host 226,132 mobile broadband base stations (on 3G & 4G). Customers are able to recharge their mobiles in more than 1.6 million retail outlets. Why are these stats important ?

At Airtel, we are uniquely privileged to service such a large customer community, the sheer scale. Just consider the huge volume of information that is being handled, with the combination of the active customer base, new customer acquisitions, the retail distribution network, the vast mobile network with thousands of base stations and the optic fibre nodes. Further consider the voice traffic, data consumption, calls generated, recharges made, bills settled, plans configured, bundling plans, pricing plans, customer service provisioning, requests attended to, call centre operations and so on. More exciting plans such as location-based services, IOT, device bundling, home accounts, etc. have been launched. The tons of data that we are handling at Airtel have started unearthing many hidden gems.

7.    What are the Machine Learning use cases in Airtel ?

Imagine use cases such as : customer segmentation, usage and retention analysis, geo-based services, chat bots, network equipment capacity utilization and forecasting, recommendation engines for customers, ARPU optimization, capex optimisation, fraud detection, revenue assurance, payments bank use cases, music, TV, etc. Large data sets, distributed numerical computations and data flow graphs are enabling implementation of Machine Learning algorithms. These are making it possible to train and run neural networks, some of which are likely to become very large.

Machine Learning has the potential of not just disrupting the traditional way of doing telecom business. They are simplifying things and processes hitherto considered quite complex. Most importantly, they have the potential of delivering far superior customer experience at much lower costs. As machines start dealing with fundamental customer issues and solving their problems, it will be fascinating to watch the interactions between machines and humans. Airtel will be a pioneer in applying Machine Learning.

Airtel is expanding its team. So, if you are one of those passionate about Artificial Intelligence, Machine Learning and Data Science, and feel excited about applying technology to solve customer problems, please do reach out to : gayatri.taragi@airtel.com. 

“The more contact I have with humans, the more I learn” (Arnold Schwarzenegger in “Terminator 2 : Judgement Day”)