How do you handle complex and ambiguous queries in your IR system?

The way the query analysis step is implemented has changed massively in the most recent years, but the overall pipeline remains unchanged. First of all, it's fundamental to identify the intent: 1) what are the different classes of intent in your domain? 2) design and implement a system that classifies the intent of your queries (this can be a rule-based lexical, a supervised machine learning system or nowadays a request to a Large Language Model) This step will involve text analysis and tokenization. 3) define different matching and ranking strategies for each of your intent classes. This means you go with lexical queries for some classes of intent and potentially you go with vector-based search for others (or in between with Hybrid search)

Query expansion is facing an extreme revolution with the advent of Large Language Models. In the 'past' (few months ago) we used to: - analyse query logs to find potential query reformulations from users (that was particularly useful when in a session we could find sequences of them to reuse for expansion). - use knowledge bases to extract and link named entities to then expand them with alternative labels - hard coded synonyms, Hyponyms and hypernyms (with weights) - machine learning models (potentially trained on a mix of the above) Nowadays LLM themselves can be a useful tool to expand a query, with a decent prompt you can get many query terms variations and even entire sentences/query reformulations.

Reformulation and expansion are kind of overlapping nowadays. My recommendation is to start simple, with lexical solutions: - spellchecking and phrase suggestions: This is especially useful for 0 results queries or when there are lexically similar alternatives on your corpus of information that may return more results. You should check your term dictionary for typos and can be done with traditional string distance metrics - Log-based reformulations -> captured from user sessions in your specific domain. i.e. how are users reformulating your queries? you can learn patterns and replicate reformulations. One final consideration I have: don't do this transparently, figure out a nice User Experience to show the user how you reformulated

This is still a very interesting problem, especially in multi-term queries on multiple fields. e.g. Query: "Paris Hilton concerts in Rome" for documents that present fields such as 'artist', 'type of event', 'location'. Standard techniques such as Named Entity Recognition, linking to knowledge graphs or mere term/document frequency solutions, proved useful over the years, but I have to admit machine learning and especially Large Language Models are revolutionising this field. We've been experimenting in using LLMs with zero shot or few shots prompts and they work surprisingly well for query segmentation and parsing. My next bet will be on spending some time to study some additional fine tuning specific for the task.

As mentioned in my previous comment, query segmentation and query understanding are getting blended in the realm of Large Language Modelling. We've been working on prototypes that move from natural language queries (or generic keyword-based queries for what matters) to a structured query. And it works surprisingly well. It'd be important to provide the structure (a taxonomy or a dictionary to give additional context to the LLM) and explain the expected task in the prompt. In these regards, I am exploring new libraries which aim to build a more programmatic interface of interaction with LLM to be able to use them for these tasks more reliably.

I and my company have been advocating search quality evaluation for many years and you would be surprised to realise how many companies (also big and famous) don't do any sort of evaluation. My recommendation is to start simple, from an online evaluation platform, that monitors your search system queries and user interactions. This is easier to sell to the business and presents tangible immediate benefits as you will be able to check if your users are interacting with your system in the ways you expect. The following step is to use the online data to synthesise offline ratings and build an offline evaluation tool, that reproduces your online queries offline and gives you the ability to test candidates without affecting production.

When Mr. Kurtzman asks why he can't send a refund check to Mr. Buttle, telling him that the Population Census has got Mr. Buttle down as "dormanted", the Central Collective Storehouse Computer has got him down as "deleted", Information Retrieval has got him down as "inoperative", Security has him down as "excised", and Administration has him down as "completed" doesn't help at all. Without interoperability between departmental communications, he won't know that all these terms are talking about the same thing. Without using plain language, even if he knows that they are all talking about the same thing, he still won't know what that thing is, and his question will go unanswered.