Federico Marengo’s Post

Data poisoning and chatbots A short and very informative video of IBM Technology where it is explained  • how chatbots work • how a security threat can operate (data poisoning) AI #chatbots (like #ChatGPT) are trained with a knowledge base (corpus) and this knowledge base is used to provide the answer to the user. The knowledge base can be prone to attacks, one of which is data poisoning. In #datapoisoning attacks, adversaries try to manipulate training data in an attempt • to decrease the overall performance (i.e., accuracy) of an ML model, • to induce misclassification to a specific test sample or a subset of the test sample, or • to increase training time. This could potentially happen with any AI system, including ChatGPT. At minute 6:45, he provided an example of a chatbot that was released into the internet after interacting with people, within a day, it started spouting all kind of offensive messages. He is, I assume, referring to “Tay”, a chatbot developed by Microsoft via Twitter on March 23, 2016. After posting racist, misogynist, and negationist comments, it was quickly shut down. The link to the video in comments