AI: Artificial Intelligence or Aggregated Ignorance?

A series of tests did not give me confidence in the results from even a simple query of an AI chatbot. Use them at your own risk.

After a recent Windows 10 update, I noticed that Microsoft Copilot was now pinned to my PC’s taskbar without my asking for it. I haven’t had much interest in exploring the many AI chat tools available, but there it was. So I did some tests with Copilot and several other AI chatbots with help from my friend and colleague, software expert Meilir Page-Jones.

We were less than impressed. Our results serve as a cautionary tale for anyone who thinks today’s AIs will solve their problems.

The Experiment

The question we posed to each chatbot was simple: “Who are the most important contributors to the field of software requirements engineering?” Meilir and I know something about this topic. I’ve been working in that field for nearly 30 years and have written several books and many articles on software requirements, as well as giving hundreds of presentations. The answer to that question is obviously somewhat subjective, but Meilir and I could each come up with a list of, say, ten people who we consider to be important contributors to the discipline. We would consider their books and other publications, for both practitioners and academic researchers, as well as their reputations, visibility, and impact established through presentations and training courses.

To begin our test, I posed the same question to some traditional search engines, Google and DuckDuckGo. As expected, they each returned links to articles or pages that are pertinent to requirements engineering (RE), but they didn’t answer the question, which is why I tried it with the AI tools. This seems like a typical sort of (very simple) query to an AI chatbot.

The Results

When I did an ordinary search using that question on Google, an “AI Overview” appeared above the usual list of search hits. This overview listed three people, with a sentence or two about each. The first name would appear on most lists of requirements engineering leaders. I was not familiar with the second person, who apparently was an early leader in the broad field of software engineering but is not particularly known for their contributions to RE. The third name was a low-profile co-author of just one book on requirements. A human is unlikely to suggest that person as one of the top three when asked the same question.

This “AI Overview” from Google was not an encouraging start. Let’s see if we can do any better.

Microsoft Copilot

Meilir and I separately posed the identical query to Copilot four times, obtaining four different sets of five names along with a largely content-free statement about each person’s work. Two names appeared in all four of the queries. One of them was fully appropriate. The other was someone who has written several important software books but hasn’t written, spoken, or researched a great deal about requirements. We do not consider him to be among the top five important contributors to RE.

The remaining three names presented from each Copilot query varied, with some overlap. Many of them were good choices. However, three of the nine people are best known for their work in areas other than RE, such as cost estimation and software design. Several far more appropriate people were omitted from the results of all four queries. That’s a puzzle.

The fact that Meilir and I obtained such different responses in different query sessions using the same prompt surprised me. Which response should the user believe as being most accurate? What were the criteria for selecting certain names one time and different names the other times? I have no clue. If there are more than five people who are good answers to the query, then why limit the responses to only five?

The brief write-up describing each individual’s contributions to RE also varied from one response to another for those people who appeared in all four sets of query results. I can see how producing varying responses would be useful for, say, high school students who don’t want to submit identical papers for their American History class. But it’s a little odd for users who are looking for definitive answers. Perhaps that variability arises from the probabilistic nature of AI queries and responses.

ChatGPT-4o mini

ChatGPT-4o mini did a better job. It returned eight names, with a short statement about why each one was on the list. Only one of the eight was questionable, again someone who has made substantial contributions to other aspects of software engineering more than to RE. Otherwise, this was quite a good list of requirements engineering leaders.

Perplexity

Perplexity.ai describes its product as “a free AI-powered answer engine that provides accurate, trusted, and real-time answers to any question.” Let’s just see about that.

Perplexity’s answer to my query was more thorough than those from Copilot or ChatGPT-4o mini. It provided eight names, along with a short paragraph about each one’s contributions to the field of RE. Six of the names were good choices. The other two responses were puzzling.

One said, “Person X has made significant contributions to agile methodologies and their impact on requirements engineering.” Person X is a friend of mine, but I hadn’t heard her name in conjunction with requirements. I asked her about that; she said she hasn’t done any work or publishing on RE. So that was an odd name to include in this group.

I had never heard of the eighth person Perplexity identified. Normal web searches for that person’s name plus “requirements” were no more illuminating. If I have to work hard with multiple searches in an unsuccessful attempt to find that mystery person, I wonder how Perplexity concluded they were one of the top eight contributors to the field. So much for “accurate, trusted…answers to any question.”

Repeating the same query with Perplexity delivered only five names this time, with just one name appearing in both sets of results. It also presented me with a short summary of “Emerging Trends in Requirements Engineering,” which I hadn’t asked for. A third attempt yielded another set of eight names (but no emerging trends summary), of which only two had appeared in the earlier queries. At least they were all real people and mostly appropriate. This variability leaves me perplexed.

Google Gemini

Google Gemini provided a richer response than the other AI tools. It presented the names of, and one sentence about, three pioneers in RE, three influential researchers, and three modern contributors. Two of the pioneers were reasonable, though not excellent, choices; one was borderline, in our opinion.

Of the influential researchers, one was a fine choice, one was okay, and the third doesn’t exist. As far as I could tell, no one by that name has done any work in software at all, let alone RE. Google tells me that people by that name include a long-dead composer, an insurance executive, and a murderer from the 1800s. The RE contributions that Google Gemini attributed to this person actually were made by someone else. How did he get on Gemini’s results list?

One of the three modern contributors was appropriate, although several other prominent names were absent. However, the second person in this group is a well-known figure in software whose work in some ways is actually antithetical to requirements engineering.

The third choice was even stranger. The name given was Brian Basili, but the contributions described are actually those of Victor Basili, who has worked primarily in the field of software metrics, not requirements. A Google search told me that Brian Basili was a cross-country runner at Villanova University. Interestingly, when I asked Gemini “Who is Brian Basili?” it replied, “I do not have enough information about that person to help with your request.” I’m confused.

So my reaction to the results from Google Gemini is: “Huh?” To be fair, the Gemini home page has a disclaimer: “Gemini can make mistakes, so double-check it.” Or in President Ronald Reagan’s words, trust but verify?

Which brings us to our analysis.

The Analysis

When I’m reading a book on a topic I know something about, like military history, I will occasionally encounter a factual error. That makes me nervous. Does the book contain other errors that I just didn’t happen to catch based on my prior knowledge? How would I know?

Our results from these small tests with several AI tools are similarly concerning. The signal-to-noise ratio is low, with some marginal responses and complete junk embedded along with accurate answers. The not-previously-informed researcher who’s relying on these AI chatbots for assistance can’t distinguish the good answers from the bad.

This leads us to a cautionary conclusion. Meilir summarizes it thusly:

A. Research, the old way:

1. Look something up.

2. Get the answer.

B. The new way for smart researchers:

1. Ask an AI chatbot.

2. Get a long detailed answer.

3. Peruse long detailed answer.

4. Notice some fishy details.

5. Check each fishy detail.

6. Discover some glaring errors.

7. Go back and check the non-fishy details, just in case.

8. Find one or two more dubious “facts” that need to be checked and verified.

9. Continue asymptotically to weary boredom.

C. The new way for very wary researchers:

1. Perform Step B three times with the same AI chatbot.

2. Again, perform Step B three times, but now with a different AI chatbot each time.

3. Compare the six results for consistency.

4. Throw up hands in exasperation and try Wikipedia.

D. The new way for less smart researchers:

1. Ask an AI chatbot.

2. Get a long detailed answer.

3. Accept long detailed answer verbatim.

Our conclusion is that we should be cautious about accepting the results from AI chatbots as accurate. Responses like those we received could perhaps point a researcher in the right direction as a starting point. However, you’ll still need to do your own exploration to verify the results, lest you merely propagate erroneous information. For instance, in some other queries I posed to Copilot, it intertwined the activities of multiple people who have the same name, thereby yielding garbage. Use these tools at your own risk.

Our results — sometimes inaccurate, inconsistent, and simply baffling — are amusing in the context of idle inquiries. But consider AI tools being used to analyze legal cases (like the ones that cited nonexistent case law) and to assist in police work by attempting to predict crimes and identify likely offenders in advance.

Use at society’s risk.