How can you differentiate reinforcement learning from other machine learning types?

Reinforcement learning (RL) is a machine learning approach where an agent learns by interacting with an environment, receiving rewards or penalties for its actions. Instead of explicit answers, RL focuses on maximizing cumulative rewards over time. Imagine a robot trying to navigate a maze. The robot doesn't know the layout of the maze or the path to the exit. In RL, the robot (agent) explores the maze (environment) and receives a reward when it moves closer to the exit and a penalty when it hits a wall. Over time, the robot learns the best strategy (policy) to reach the exit quickly, all by trial and error. This process mirrors how humans learn from their experiences, trying out various actions, and learning from the outcomes.

Reinforcement learning is unique: - Sequential Decision Making: Focuses on sequences of actions. - Rewards and Exploration: Maximizes cumulative rewards through exploration. - Markov Decision Process: Formal framework for RL problems. Study RL fundamentals and contrast them with supervised and unsupervised learning to grasp distinctions.

I have used supervised learning in healthcare, finance, manufacturing and marketing. For supervised learning the right answer is known before training the model. In a typical supervised learning scenario drawn from my manufacturing experience, an outcome measurement is available, usually quantitative (such as a component lifetime) or categorical (such as component failure/non-failure). The objective of the algorithm is to predict the outcome based on a set of features such as measurements taken from sensors on a production line. A model can be trained using a supervised machine learning algorithm on such an dataset of labelled sensor measurements, each measurement marked as either signifying failure or non-failure.

Using unsupervised learning techniques, the structure of the data can be explored to extract meaningful information without the guidance of a known outcome variable. This distinction requires unsupervised learning to typically involve a large degree of interpretation by a data scientist to make sense and interpret the model results after analysis. Unsupervised learning problems can be further grouped into clustering and association problems. An examle of a clustering problem is using the K-means algorithm to cluster customer segments and an example of an association problem is pre-training of LLMs to predict the next word in a sequence.

For reinforcement learning, the goal is to develop a system that improves its performance based on interactions with the environment. The system typically takes the form of an agent. Since the information about the current state of the environment typically also includes a so-called reward signal, we can think of reinforcement learning as a field related to supervised learning. However, in reinforcement learning this feedback is not the correct ground truth label or value, but a measure of how well the action was measured by a reward function. Through its interaction with the environment, an agent can then use reinforcement learning to learn a series of actions that maximizes this reward via an exploratory trial-and-error approach.

Beyond what is listed, advantages also include the ability for reinforcement learning models to learn the intent and the reward based on data, also known as inverse reinforcement learning. To be more explicit; instead of trying to outline the reward function, which is extraordinarily difficult to do, you have the model watch and learn, and try to come up with a suitable reward function. This has proven to be successful in generating faster traveling routes for transportation, and research is ongoing to implement within autonomous vehicles. Though the disadvantage of this approach is that there may be a lot of implicit policies that do not get captured due to underrepresentation in the training data, which can lead to weird behaviour.

In the case of recommendation systems, it is worth considering simpler non-neural network based algorithms such as Thompson sampling and Upper-Confidence Bound algorithms. These are easy to understand and debug. Implementing neural network-based reinforcement learning models can be difficult if you do not have the resources and correct know-how, as these models have notoriously unstable and unreliable behaviour, especially with regards to the reward shaping process. Therefore, where you are working with highly abstract data and require a neural network, consider non-reinforcement learning neural networks first - and only consider reinforcement learning models if it actually makes a lot of sense to do so.