How can you explain Machine Learning models?

For example, Imagine a machine learning model using medical imaging to identify a particular kind of cancer. After training on hundreds of photos, the model assists radiologists in hospitals with high accuracy. The model labels a patient's scan "highly likely" to be malignant. Given the high stakes, the radiologist must understand why the model predicted such. A tissue pattern, marking, or picture shadow? The doctor's verdict is a black box without explanation. Explainable AI can highlight the area of concern on the picture and show out particular elements that lead to its conclusion. The physician can detect whether the model is mislead by picture artifacts or irrelevant data.

In my experience, when employing machine learning models to forecast energy consumption for two identical buildings, the predictions outcomes didn't turn out as expectedly alike. This disparity in predictions is due to the distinct energy usage patterns of users in each building. This is where explainability steps in, aiding in the understanding of the variations in user behavior and highlighting the factors that contribute to the differences.

Machine learning models have always been treated as blackboxes. We know Garbage in, Garbage out. But many a times it also gives unpredictable outputs for the same inputs. Machine Learning, in general, is highly complex. It would be beneficial to be able to explain what the machine is considering as important features in the input and why, and based on that we could try and tune the model to give more predictable outputs. Thus model explainability matters.

Explainability relates effect to causes. You cannot control the effects if you do not identify the causes. (Without too many words.)

As a System Solutions Architect, I understand that the implications of AI decisions go far beyond technical considerations. They extend to ethical, regulatory, and social dimensions. Machine learning models often operate as "black boxes," which can be problematic when dealing with sensitive data or making high-stakes decisions. Perspective: Explainability is not just a matter of convenience; it's a necessity. It helps in validating, improving, and ensuring the fairness of AI models. Moreover, it enhances transparency, accountability, and ultimately, trust among stakeholders. For organizations aiming to comply with regulations, demonstrating model explainability is a vital step.

Explainability is above all, related to Ethics. Since Ethics is related to the way that people and societies coexist, machine learning models, A.I, on a broader sense, introduce a disruptive coexistence reality and Explainability is the new "Agora" where everybody has to find a common understanding of a certain subject. If a movie or a product is recommended to me, and I understand the reasons why, I improve not just my consuming experience but also the trustness on the recommendation tool, for example.

In efforts to eliminate indirect discrimination against protected categories of people, an important aspect of explainability is to understand the way in which a machine learning model might extract the same information from different variables, groups of variables, and interactions between variables. What does your model think that the color and type of your car says about your gender? And how is your model abusing this information to increase its performance? Knowing what variables and interactions skew our predictions in what directions can also help us find ways of changing the world to be a better place. If we understand the variable interactions that correlate with health risk, we might learn how to reduce health risk!

Measuring explainability in ML models is like checking how well we understand why a computer program makes certain predictions.

Explainability is not a one-size-fits-all concept. It's a contextual property that depends on the specific model, task, and audience. Different situations might require varying levels of explainability. For instance, a medical diagnosis model might need a higher degree of explainability than a movie recommendation system. Insight: In your role, you must recognize that tailoring explainability to the audience's needs is crucial. As a Solutions Architect, you should work closely with data scientists and domain experts to determine the right balance between providing technical details and simplifying explanations for non-technical stakeholders.

I think explainability is making people understand ML models by relating it to things they already know. It will allow humans to leverage existing knowledge to predict how the model will behave, and will make them more accessible. If ML models are explained as something entirely new, it will take greater effort for people to develop an understanding. But to explain these models in a natural/relatable way , one should first know the story behind why these models were conceived, which problem was the inventor trying to solve, and which concept inspired the model. ML & deep learning models were inspired by concepts in biology (like neural networks) , maths, statistics( regression etc) , along with some optimizations from computer science.

Specific to image tasks, you may also use Grad-CAM to aid explainability. Grad-CAM provides the model's attention within the image field where stronger value represents stronger contribution. This may help interpret the result. However, it should be used with caution since it is merely an activation map; not a definitive causality.

We can use various methods for ML explainability like (showing most important data for the predictions, showing decision-making steps, showing charts of model’s computing fairness that doesn’t show biases or over fitting models)

Explainability is not a straightforward metric but involves several principles and criteria such as complexity, transparency, interpretability, justification, and interaction. These principles can be assessed using various methods like visualizations, statistics, rules, examples, and narratives. Perspective: When designing AI solutions, it's essential to factor in how you will measure and communicate explainability. This might involve creating clear visualizations or narratives that help stakeholders grasp the model's decision-making process. Your role as a Solutions Architect involves guiding the team in choosing the most appropriate measurement methods for a specific AI project.

One way to measure explainability is to measure how much of a model's increase in performance from the baseline is explained by a certain variable, a certain group of variables, or a certain interaction between variables. Conditional mutual information between variables, which is to say how much the information in one variable or group of variables accounts for the information in the other, can also be measured in terms of performance increase or lack thereoff when combining variables into the same model.

This is a very difficult question to answer. It is as if you ask which teacher can explain a topic better? You can feel it, but you cannot accurately measure it. In addition, explaiablity also depends on the listener’s potential to understand the topic. Therefore, I think we cannot measure the explainabilty no matter how hard we try, but we can get a sense on which method might be easier to understand than the others.

There are diverse techniques and frameworks available to enhance model explainability, depending on factors like the model type, task, and audience. Techniques like feature engineering, model selection, analysis, comparison, and explanation can all contribute to better understanding a model's output. Perspective: As a Solutions Architect, you'll play a critical role in selecting and implementing these techniques. You must collaborate with data scientists and engineers to ensure that the chosen methods align with the project's objectives. This collaborative approach helps in making the model's behavior more interpretable and justifiable.

What would you do if you cannot understand why your friend makes a certain decision? Obviously, you don’t open their brain and analyse the neuron activity, you simply ask them. I believe we can do the same with machine learning algorithms by exploiting the power of LLMs. I admit that it is very challenging to integrate these algorithms with others to make the AI explain its line of thought, but imagin how wonderful it would be. I say let’s give it a shot.

mproving explainability is like trying to describe the plot of a twisty movie - different friends might give you different summaries! Ever used LIME for a model's explanation and got one story, then tried SHAP and got a mini plot twist? It's like asking two friends about the same movie. Which to trust? It's best to gather views from all your "friends" (techniques) to get the full picture. Complex models can be like onions (or ogres, if you're into Shrek) - they have layers! One explanation might just scratch the surface. Sometimes, to truly "get" a model, you need multiple takes on its story. True understanding? It's like in Quantum Mechanics - one answer may not explain a problem/system.

Doing exploratory and explanatory analysis on ML models and our data can give us new domain knowledge, and domain knowledge can help us make simpler models with fewer features, which again can improve explainability. Achieving higher explainability while maintaining high performance is a process of reducing complexity iteratively through several rounds of explanation.

Effective communication of explainability is as important as achieving it. It involves understanding your audience, simplifying complex concepts, and providing interactive tools for exploration. Ethical considerations should also guide your communication efforts, acknowledging limitations, uncertainties, and biases. Insight: As a Solutions Architect, you must bridge the gap between the technical aspects of model explainability and the practicality of communicating it effectively. This requires not only technical expertise but also a keen understanding of your organization's stakeholders and their expectations.

Interactiveness, scenario simulations combined with story-telling are determinant for an accurate Explainability that engages the audience.

"BLACK-BOX- Myth or Reality" As per my understanding, all ML algorithms can be expressed as mathematical equations. in artificial neural network (ANN), inputs are associated with a weight (bias) in the hidden layer, and it is processed through a mathematical function (sigmoid, ReLU etc) to generate output values. Thereafter, iterations (epoch) are performed to reduce the error associated with the initial ANN model. Thus, nothing is hidden in the hidden layer of the ANN. It is not a "BLACK BOX", as a user has control over the behavior of the ML model.. A user can control and alter the output by tuning hyper parameters. Explainability of ML models can be improved by hyper-parameter tuning.

Understanding two crucial AI concepts, interpretability and explainability, is essential. Interpretability involves comprehending why a decision was made, while explainability delves into AI's decision process. This differentiation is pivotal for transparent and user-friendly AI systems. From a broader perspective, it's vital to emphasize the significance of both interpretability and explainability in AI projects. Interpretability uncovers the reasons behind AI decisions, offering insights into causality. Explainability provides a comprehensive view of the decision-making process, extending beyond mere causes. Combining these factors results in trustworthy AI systems that are both transparent and comprehensible.

One of the most pressing issues with commercial models today is their tendency to provide predictions that are merely "averages". Consider, for instance, the use of "ChatGPT detection" to determine if students have cheated. These models typically deliver a score without any indication of its reliability. Yet, many readily accept these scores at face value, mainly because they originate from "AI". This blind faith is concerning. Just because it's "AI" doesn't mean it's infallible. In fact, these predictions can often be far from reliable! It's crucial that we critically assess and demand transparency in such scores rather than blindly trusting them.

Explanability goes down as model size goes up. A small regression is easy to explain, "Oh yah, 2.5 is between 2 and 3, so the results should be somewhere between" while massive models are harder. When I do my work in data science, I try to determine if this parameter is adding anything to the model. There is no classical way to know for sure, but if we can cut a 30-dimentional problem down to 20, the explanability at least gets easier. Really a problem here is that humans are bad at visualizing more than or so dimensions.

Let's start with statistics. Samples have several features. These features describe every sample explicitly. Now you want to use an algorithm to find a mathematical formula called a model to find the connection between these features to create samples from scratch. So, the limitation in statistics is that features should not correlate or have an effect on each other. Classical supervised learning also suffers from this in another way. If in the real world, the features don't correlate, you are lucky but if they do (in most cases), you need neural networks. They consider correlation in multi-level due to their multiple hidden layers and mathematical induction. These two, used in conjunction, create the masterpieces we see today. Best, Houman