Virtual assistant : Explainability notice

Trustworthy, explainable AI

Trustworthy and explainable AI are key objectives for the EU. The EU is working on a regulatory framework for AI. The proposal of the European Commission of 21 April 2021 for a Regulation laying down harmonized rules on artificial intelligence (hereafter ‘AI Act’) (Appendix 3) puts in place two main types of commitments on the part of providers of AI-based solutions and tools – transparency and the provision of information.

Usersof AI systems have the right to receive information. That right goes together with corresponding obligations, such as the obligation to use the system in line with instructions and to monitor the performance of the AI tool concerned. Compliance with these rules results in accountability of all actors involved.

The proposed AI Act stipulates in Article 52 (Transparency obligations for certain AI systems), that “1. Providers shall ensure that AI systems intended to interact with natural persons are designed and developed in such a way that natural persons are informed that they are interacting with an AI system, unless this is obvious from the circumstances and the context of use. […] “

The proposed AI Act follows a risk-based approach, differentiating between uses of AI that create (i) an unacceptable risk, (ii) a high risk, and (iii) low or minimal risk.

Publio, the intelligent assistant of the OP Portal, interacts with natural persons and performs searches by using keywords relying on information that is publicly available on the OP Portal. That information is published based on the transparency principle underlying all EU policies and legislation. The intelligent assistant does not generate new content, nor does it manipulate, or influence choices made by users, beyond proposing possible filtering options for the users’ search. Publio is therefore to be considered as falling under the “low risk” category (iii) as defined in the draft AI Act.

The draft AI Act stipulates under Title IV (Transparency) that the design and functioning of AI systems need to take account of the specific risks that these might pose of 1) manipulation through subliminal techniques, i.e techniques beyond users’ consciousness, or 2) exploitation of vulnerable groups likely causing psychological or physical harm. Stricter transparency obligations apply for systems that (i) interact with humans, (ii) are used to detect emotions or determine association with (social) categories based on biometric data, or (iii) generate or manipulate content (‘deep fakes’). While Publio does interact with natural persons, the intelligent assistant is not designed to detect emotions with an aim to manipulate nor does the tool contain any elements that could inadvertently lead to such an outcome. Publio therefore does not match the abovementioned criteria.

AI and liability

The European Commission’s proposal for the AI Act was complemented on 28 September 2022 by a proposal for a civil liability regime for AI – the Artificial Intelligence Liability Directive (Appendix 4) (hereafter ‘AI Liability Directive), following the resolution of the European Parliament adopted under Article 225 TFEU (Appendix 5).

As set out in the proposed AI Liability Directive, the existing liability rules, based on fault, are not suited to handle liability claims for damage caused by AI-enabled products and services. Under the existing liability rules, victims need to prove that a wrongful action or omission took place, identify the person who caused the damage, and establish a causal link. The specific characteristics of AI, including its complexity, autonomy and opacity (the so-called “black box” effect), make it difficult to identify the person liable for damage resulting from the use of AI. Victims suffering such damage may therefore be deterred from claiming compensation. Given the nature of the burden-of-proof issue, the proposed AI Liability Directive provides innovative solutions by easing the above-described requirements through the use of disclosures and rebuttable presumptions of non-compliance.

The proposed AI Liability Directive supplies effective means to identify potentially liable persons and relevant evidence. It states for instance that a national court may order the disclosure of relevant evidence (and its preservation) about specific high-risk AI systems that are suspected of having caused damage (Appendix 6). Moreover, subject to the requirements laid down in Article 4 of the proposed Directive, national courts shall presume […] the causal link between the fault of the defendant and the output produced by the AI system (or the failure of the AI system to produce an output).

As concluded in the Explanatory Memorandum to the proposed AI Liability Directive, “such effective civil liability rules have the additional advantage that they give all those involved in activities related to AI systems an additional incentive to respect their obligations regarding their expected conduct”.

About Publio

Terms and definitions

What is Publio?

Publio, the intelligent assistant of the Publications Office’s Portal (OP Portal), is an AI tool that interacts with natural persons and performs searches by using keywords relying on information publicly available on the OP Portal. Publio combines conversational Artificial Intelligence (AI) techniques such as natural language processing (NLP) and machine learning (ML) with interactive voice recognition and traditional search systems to assist users in finding EU publications, EU legislation and contact persons in the European Institutions.

The intelligent assistant specifically enables spoken or typed conversations between end users and the OP Portal. The intelligent assistant is currently available in English, French and Spanish. While being usable by all, the intelligent assistant also provides a solution tailored to the needs of people with reading disabilities, thereby increasing accessibility. It further enables conversational search which is a new way of searching that allows users to speak in complete sentences – just as they would in a normal conversation – to an AI-powered voice assistant that returns answers, with the exchange between user and intelligent assistant taking the form of a conversation.

Which type of data does Publio use?

Publio uses the information publicly available on the OP Portal. That information is published based on the transparency principle applicable to all EU policies and legislation. Based on that corpus, Publio uses classifications and categories available in the OP Portal such as Eurovoc subjects, authors or format to guide and refine the conversation to assist the user to search throughout the three main collections of the OP Portal: EU publications, EU law and the EU official directory (EU Whoiswho).

Publio uses an existing language model to understand questions addressed by users and to guide users throughout their search journey. Based on feedback from users, the Publio language model is constantly further improved and trained to meet user expectations and cope with the diversity of user inputs in the three languages supported.

This training covers multiple components of Publio’s underlying system:

• A predefined series of utterances associated with different possible forms of intent to allow Publio first to grasp the user’s final goal and second to guide the user step by step towards reaching that goal.
• A predefined series of fixed questions that Publio “understands” and for which it can provide a predefined answer to users. Example: How can I order publications?
• Machine learned entities that can be identified by Publio and used in Publio’s background processes to detect search parameters, provide more accurate search results or options to further refine the search. These entities are based on the existing classification and categories of data such as Eurovoc thesauri or other authority tables managed by EuroVoc: authors, formats, languages, EU bodies (organizations), functional roles in the EU public service, etc. On that basis, Publio can for example, grasp from the user’s query that that the user is looking for a publication with a specific subject or from a specific author.

How does Publio work?

Publio uses Microsoft Language Understanding (LUIS) (Appendix 7) combined with a large language model (LLM) based entity extraction system to process user input (a written or spoken utterance) accurately and efficiently. LUIS provides initial insights into the user’s intent and entities, and for more complex queries, the LLM component refines entity recognition, ensuring better results for nuanced phrases or complex requests. Based on the recognized intent and entities, Publio initiates a tailor-made conversational flow aimed at supporting users in getting to the targeted result in a few simple steps.

Currently, Publio implements 4 main conversational flows:

• Document search flow to assist users searching for EU Publications or EU Legal documents
• Person search flow to assist users in searching for public servants employed by EU Institutions
• Organization search flow to assist users in searching for EU bodies (organizations)
• Question and Answers (QnA) flow to answer frequent questions with a predefined answer.

If a “search” flow is recognized, the virtual assistant will guide the user in researching a relevant document, person or organization by asking simple questions to the user and, based on the user’s response, propose filtering options.

For the QnA flow, the staff in the Publications Office maintains a list of pairs with frequently asked questions and their predefined answers. In case the QnA intent is recognized and there is a high level of similarity between the specific question from the user and one of the questions stored in the system, an appropriate answer will be directly returned to the user. For example,

- if the QnA intent is recognized with a query like “Where is my order?”, the assistant will reply with “You can check your order status in My order section in your profile.”

- if the QnA intent is recognized with a query like “Where can I find documents for kids”, the assistant will reply with “Publications for kids are available in Kids’ corner”.

Users can ask questions in writing or by speaking. The chatbot captures the audio input (the user’s spoken questions) and sends it to a speech recognition engine which converts the speech to text. The audio input is processed by Microsoft Azure Speech to text service (Appendix 8). Publio then displays the audio input, in the words interpreted by the speech recognition engine, before displaying the reply both in writing and in spoken words using synthesized voice. This exchange is instantaneous (it happens instantly) and ephemeral (it does not leave any trace).

How does Publio analyse the question and propose results?

Publio does not use LLM or any other generative AI technology to generate the answers provided to users. All Publio messages are based on predefined templates. Publio uses a machine learning language model that is trained (trained language model) to learn how to recognise and respond to user inputs. This model is responsible for processing the user's message or query, identifying the intent of the message, and providing an appropriate response. The process of training Publio involves providing the machine learning model with a large amount of labeled training data which includes examples of user inputs and their corresponding intents and plausible responses.

Publio uses Natural Language Processing, the trained language model and LLM to understand user intent and to extract from the user question the recognized entities (for example search term, subject, author, document date, document format). Publio is using these entities to execute the search and to display the search result.

If the result of the search is too broad, Publio continues the conversation by asking additional questions to the user ito narrow down the search result by filtering on other entities.

The questions formulated by Publio are limited to the identification of the intent of the user, to capture its purpose and to formulate the right search query to the corpus of the content published on the OP Portal. Results proposed by Publio come entirely from this corpus.

User Feedback

At any time during the conversation the end user can provide positive, neutral or negative feedback on his/her experience using Publio. It can be manual feedback triggered by the user at any time or in response to an automatic popup displayed after a few seconds of inactivity requesting the feedback from the user. Additionally, questions which are not correctly “understood” by Publio are automatically logged and periodically processed by a human team of staff in the Publications Office. Both the user feedback and the analysis of questions not properly “understood” by Publio are used to retrain the language model with the goal of continuously further improving the services provided to end users.

Private data processing

Data available on the OP Portal is processed in compliance with the EU General Data Protection Regulation (GDPR) and the Regulation on the processing of personal data by EU Institutions, agencies and bodies (Appendix 9).

Publio does not store, keep or archive any elements, neither the user’s input nor that provided through Publio’s replies.

The collected feedback is anonymized in such a way that it does not make it possible to link the feedback back to the user from whom it was collected.

No personal data is used for automated decision-making, tracing or profiling in any possible way. When using Large Language Models (LLM) for entity extraction, the only data exchanged between Publio and the LLM is the user's question and the preliminary entity extraction result generated by LUIS. No personal user data is shared or transmitted. All services used by Publio are hosted in European data centers, and all data processing complies with EU data privacy regulations. .

Limitations at the current point in time

• Name transcription using voice: while most names are “understood” and transcribed correctly, the name extraction is based on Machine Learning and is not 100% accurate. Due to LUIS limitations some names are not properly extracted even after training the model with the specific examples from the dataset.
• Dates in French and Spanish: date “understanding” is less accurate in Spanish and French in more complex contexts. For example, the year will be recognized in Spanish using “entre 2016 y 2017” without additional context whereas it will not be recognized with “entre el año 2019 y 2020”. In English “2 years ago” is recognized whereas in French “il y a 2 ans” is not.
• Speech recognition only works in one language at a time.
• Speech recognition does not have a spelling mode, meaning that it does not have a feature that would allow users to spell out words character by character in order to ensure accurate transcription.
• The name entity recognition does not work well in larger phrases when combined with "Mrs"/"Miss".

Publications Office of the European Union, 16 May 2023