AI-driven spend classification: Turbo-charging spend analytics

Procurement is a data-intensive function whose practitioners are swamped with vast amounts of information from customers, suppliers, and market aggregators. To help digest all these terabytes of information, procurement analytic tools have been augmented with advanced AI capabilities.  

However, even the most sophisticated spend analytics tools struggle to deliver useful insights because most of the time they’re drawing on poor-quality product data. And that’s because the process of collecting and editing product information mostly remains manual, resulting in inconsistent and inaccurate classification. The problem magnifies with the increasing complexity of data sources and IT landscapes. To be fully usable, spend data must be not only complete, unique, and accurate but also standardized and structured.  

Here is where AI can help. AI models can analyze such messy data with human-like understanding and quickly classify them in a standardized and structured way that can then be easily consumed by spend analytic tools. Spend classification forms the most important prerequisite for spend transparency. That makes it a vital function for any organization looking to use AI as a way to streamline its procurement and spend analytics. 

Manual classification – time-consuming and inconsistent

“Spend classification” is the process of grouping spend data into a structured and standardized set of categories called a “spend taxonomy,” against which procurement spending can be mapped. One of the most popular spend taxonomies worldwide is UNSPSC; others used in various regions or industries include CPV, ECLASS, ETIM, GS1, ISIC, and NAICS. 

Most organizations update and classify their spend data on an as-needed basis every few years. Generally, these classification exercises are quite time-consuming, from six months to a year, because they are still mostly manual (plus some rule-based approaches). And they become harder the larger and more diversified the organization becomes. Some common problems organizations encounter with manual spend classification are: 

• Limited to skills and competence of the procurement practitioner(s) 

• Tagging unclear/uncertain items into a catch-all “miscellaneous” category  

• Classifying identical items into different taxonomies (e.g. classifying hydraulic pumps under “industrial components” as well as “mining machinery”) 

• Limiting categorization due to outdated lists of rules (supplier names and keywords)  

• Unable to build five levels of taxonomy to fully describe and differentiate all items (e.g.: Segment, Family, Class, Commodity, Custom) 

• Building redundant or unused categories 

• Maintaining uneven levels of classification 

These issues interfere with getting a clear picture of what organizations are spending on, where, or how. Spend visibility is crucial because it allows organizations to make decisions supporting cost savings, budgeting, sustainability, supplier performance, and other procurement projects.  

Without it, companies will grapple with challenges in making purchasing decisions operationally as well as strategically. Moreover, such inaccurate insights can lead buyers to make incorrect decisions related to sourcing, supplier selection, or vendor consolidation.  

This is where AI can be of great use. AI can accurately cleanse and classify the spend data based on any global or custom taxonomy using several techniques including NLP (natural language processing), supervised, unsupervised, and reinforcement learning.  

Building spend classification with AI

Here is what a successful spend classification program could look like using AI models that have worked time and again for Digitate’s customers: 

• Identify the procurement categories to start gaining precise visibility into, either the most critical or the most common ones (i.e. that cover about 80% of your organization’s spending, as per a common rule of thumb called the Pareto Principle.) 

• Reference an industry-standard framework such as UNSPSC rather than a custom taxonomy, so you have a universally accepted way for organizing spend.  

• Cleanse using NLP techniques to remove basic errors in item descriptions (spelling mistakes, spaces, substitute terms/keywords). This saves many hours of manual editing. 

• Classify using supervised and unsupervised learning models to automatically classify the spend data as per the UNSPSC taxonomy (typically achieving an accuracy of 60-70%). 

• Validate and refine the output with procurement practitioners to bridge the remaining 30% accuracy. This step is essential to capture the tacit and category knowledge of your organization (in other words, the unwritten rules and shortcuts that help guide experienced procurement practitioners). 

• Augment the validation using Generative AI to source additional information that can help to classify items faster. 

• Reinforce the newly labeled and cleaned data to train the model and classify spend data more accurately than before.  

• The model will also continue to stand as a reference to predict and correct incorrect classifications going forward. 

Organizations that follow these practices will have a solid data foundation. In fact, you can think of it as “turbocharging” basic spend reporting into an advanced spend analytics program. The insights will aid the procurement function in making better operational decisions and even influencing the overall organization’s strategic decisions.  

AI-driven spend classification can drastically bring down the speed and accuracy of the entire spend classification program to a couple of months instead of several quarters, which is a clear winner. And instead of making the spend classification an ad hoc activity once in a few years, AI can start fixing all new items on an ongoing basis. This way procurement will look at clean data on a daily basis. 

While there is no solution in the market today that can auto-cleanse and classify with 100% accuracy, the AI algorithms are getting better day by day. With the rise of Generative AI, we are a step closer to mimicking or automating human behavior. Generative AI’s NLP capability will aid in delivering a higher accuracy than regular AI models, thereby eliminating the need for procurement practitioners to validate the output.  

What Digitate recommends

It’s important to also address the spend classification on a recurring basis and not just make it an ad-hoc fix every few years. For this, we advise organizations to follow the below two-step approach for fixing and sustaining spend classification over the long term:  

• Classify historical purchases: Do a one-time activity of cleansing, enriching, and classifying historical spend data. This serves as a bedrock for all the spend analytics and intelligence to follow.  

• Monitor live purchases: Use real-time control mechanisms to fix spend classification issues in near-real time. The key here is consistency. It’s imperative to continue the classification on an ongoing basis so that procurement can continue to maintain trust in the data’s sanctity. 

This cutting-edge approach to classifying spend data is quicker and more efficient, scalable, and sustainable than traditional techniques of spend classification.  

This approach is embodied in ignio Cognitive Procurement, a SaaS-based AI spend monitoring product that uses advanced AI models to classify organizations’ historical spend data and resolve classification errors on transactions on an ongoing basis, thereby delivering greater spend visibility and cost savings of up to 10 percent.  

Learn more about ignio Cognitive Procurement’s capabilities here.

Written by Aravind NS