Comparing AI Search Engines: ChatGPT with Search vs Perplexity vs Google AI Overviews

The competition in search technology has intensified with the emergence of AI-driven platforms like ChatGPT and Perplexity AI, challenging Google's longstanding dominance in the search engine market. In this article, we explore how these AI search engines—ChatGPT with Search, Perplexity, and Google AI Overview—perform across different query types. The analysis examines their strengths and weaknesses using three key metrics: Relevance, Accuracy, and Response Time.

Comparing AI Search Engines: ChatGPT with Search vs Perplexity vs Google AI Overviews - NotebookLM Podcast

A Bit of History

Google's Dominance Since its inception in 1998, Google has been the leading search engine, commanding over 90% of the global market share. Its sophisticated algorithms and extensive indexing have set the standard for web search.

Rise of ChatGPT In November 2022, OpenAI launched ChatGPT, an AI language model capable of generating human-like text based on user prompts. Initially, ChatGPT's knowledge was limited to information up to 2021, restricting its utility for real-time search queries.

Introduction of Perplexity AI Founded in 2022 by former Google intern Aravind Srinivas, Perplexity AI emerged as an AI-powered search engine offering concise, real-time answers with source citations. Unlike ChatGPT, Perplexity integrates live web data, providing up-to-date information. By mid-2024, Perplexity had significantly increased its user base and revenue, positioning itself as a notable competitor in the search industry.

ChatGPT's Search Integration In October 2024, OpenAI enhanced ChatGPT by integrating a web search feature, enabling it to provide real-time information with source links. This development positioned ChatGPT as a direct competitor to traditional search engines like Google and emerging platforms like Perplexity.

Google's Response In response to these advancements, Google has been incorporating AI features into its search engine, such as AI-generated summaries and interactive elements, to maintain its competitive edge. Despite these efforts, Google's dominance faces challenges from AI-driven platforms offering innovative search experiences.

Methodology used in this research

The comparison involved five categories of search queries:

1. Informational (e.g., "What is generative AI?")
2. Navigational (e.g., "LinkedIn login")
3. Transactional (e.g., "Buy organic skincare products")
4. Local (e.g., "Coffee shops near me")
5. Commercial Investigation (e.g., "Best CRM software for small businesses")

For each category, 5r epresentative queries were selected, resulting in 25 queries in total. These queries were input into each search engine under consistent testing conditions. We then evaluated their responses based on three main metrics: Relevance (scored 1-5), Accuracy (1-5), and Response Time (in seconds). Note:  It's clear that this is not significant data to make detailed insight, but its enough to catch some top level trends, which is the goal of this research.

Results

To make the findings more accessible, the following sections are accompanied by visual graphs that represent key performance metrics—Relevance, Accuracy, and Response Time—comparing results across Google AI Overview, ChatGPT with Search, and Perplexity.

AI Search Engine Comparison

Average Response Time by AI Search Engine

Average Relevance by Category and AI Search Engine

Average Accuracy by Category and AI Search Engine

Average Response Time by Category and AI Search Engine

Key Insights

Overall Performance

• Relevance: On average, the relevance score across all queries was 4.03, indicating a strong alignment between user intent and the provided responses.
• Accuracy: With an average accuracy score of 4.30, the AI engines generally performed well in delivering correct information.
• Response Time: The average response time was 3.59 seconds, though there were noticeable differences among the machines.

Comparison by AI Search Engine

Google AI Overview

• Relevance: Scored an average of 3.85, with a clear focus on informational and navigational queries.
• Accuracy: Achieved a solid accuracy score of 4.23, ideal for delivering reliable answers to basic questions.
• Response Time: Google AI Overview was the fastest among the three, with an impressive average response time of 0.52 seconds.
• Observation: Google AI Overview primarily appeared in informational and navigational queries, suggesting optimization for general, quick-reference questions.

ChatGPT with Search

• Relevance: Scored 3.80, with particular strength in providing locally relevant results.
• Accuracy: Maintained an accuracy score of 4.20, aligning well with user needs across different query types.
• Response Time: Response time was slightly slower at 4.08 seconds, potentially due to the varied content formats it supports.
• Observation: ChatGPT with Search adapts its responses visually based on the query type—maps for local searches, images for commercial investigations, and links for navigational queries. However, links often lead to a homepage rather than a specific internal page.

Perplexity

• Relevance: Perplexity scored the highest in relevance, with an average of 4.36, demonstrating strong alignment with user intent.
• Accuracy: Delivered the highest accuracy score of 4.44.
• Response Time: However, Perplexity had the longest response time at 6.29 seconds.
• Observation: Perplexity offered comprehensive and detailed information, often including contact numbers, addresses, and prices, but lacked direct links to external web pages, which could enhance the user experience.

Category-Specific Trends

• Informational and Navigational Queries: Google AI Overview and ChatGPT with Search both performed efficiently in these categories. Google AI Overview excelled in response time, while ChatGPT enriched its responses with relevant visual details.
• Commercial Investigation Queries: Both Perplexity and ChatGPT with Search did well here. Perplexity provided in-depth information, although it lacked direct links. ChatGPT with Search used images to enhance the user experience, which was valuable for visually-oriented queries.

Final Thoughts

This analysis shows that each AI search engine has distinct strengths. Google AI Overview stands out for its quick response times, making it ideal for users needing fast, reliable answers. Perplexity excels in relevance and accuracy but takes longer to respond, which may be suitable for those who prioritize detailed information. ChatGPT with Search balances performance across metrics but has room for improvement in response time. For users spending significant time within ChatGPT, it offers a versatile solution.

Each of these AI search tools serves unique user needs, and their comparative strengths mean that the best choice depends on the specific task at hand.