OpenAI Unlocked Vision: ChatGPT Can Now See and Solve Anything in Real Time

Welcome, AI entrepreneurs & enthusiasts.

On the sixth day of Shipmas, OpenAI gave ChatGPT eyes.

Voice Mode’s long-awaited vision upgrade is here, countering Google’s big Gemini release — drastically changing the future of AI interactions. Let’s get into it…

In today’s AI news:

• ChatGPT Advanced Voice Mode gains vision capabilities
• Anthropic’s Claude 3.5 Haiku is now generally available
• Anthropic analyzes real-world AI use with Clio
• More AI & tech news

ChatGPT Advanced Voice Mode gains vision capabilities

The News: OpenAI just launched a major upgrade to ChatGPT's Advanced Voice Mode on Day 6 of its live stream event, enabling the AI to analyze and respond to live video input and screen sharing during conversations.

The details:

• Users can show live videos or share their screens while using Advanced Voice Mode, and ChatGPT can understand and discuss the visual context in real time.
• The feature works through a new video icon in the mobile app, with screen sharing available through a separate menu option.
• The updates are available to ChatGPT Plus, Pro, and Team subscribers, with Enterprise and Edu users gaining access in January.
• OpenAI also introduced a festive new voice option, allowing users to chat with Santa as a limited-time seasonal addition through early January.

Why it matters: Seven months after its initial demo, OpenAI is finally delivering on the promise of visual understanding in conversational AI — moving ChatGPT beyond text and voice into true multimodal interaction. It’s been a big week for vision, with Gemini and ChatGPT Advanced Voice gaining some extremely powerful new capabilities.

Anthropic’s Claude 3.5 Haiku is now generally available

The News: Anthropic quietly rolled out its fastest AI model, Claude 3.5 Haiku, to all Claude users on web and mobile platforms, expanding from its previous API-only availability — though no official announcement has been made.

The details:

• Haiku 3.5 was released in November along with Claude’s computer use feature — beating the previous top model 3 Opus on key benchmarks.
• The model excels at coding tasks and data processing, offering impressive speed and performance with high accuracy.
• Haiku features a 200K context window, which is larger than competing models, while also integrating with Artifacts for a real-time content workspace.
• The initial release drew criticism for Haiku’s API pricing, which was increased 4x over 3 Haiku to $1 per million input tokens and $5 per million output tokens.
• Free users can now access Haiku with daily message limits, while Pro subscribers ($20/month) get expanded usage and priority access.

Why it matters: It’s been a relatively quiet holiday season of releases for Anthropic compared to rivals. Although Haiku is impressive compared to previous generations, it doesn’t feel like a huge needle mover during a big week of AI releases — and it might take a launch of a top-tier 3.5 Opus to steal the spotlight from Google and OpenAI.

Anthropic analyzes real-world AI use with Clio

The News: Anthropic introduced Clio, a new system that reveals patterns in how people actually use AI assistants worldwide, providing detailed insights into real-world AI adoption while maintaining user privacy.

The details:

• Clio analyzes millions of conversations by summarizing and clustering them while removing identifying information in a secure environment.
• The system then organizes these clusters into hierarchies, allowing researchers to explore patterns in usage without needing access to sensitive data.
• Analysis of 1M Claude conversations showed that coding and business use cases dominate, with web development representing over 10% of interactions.
• The system also uncovered unexpected use cases like dream interpretation, soccer match analysis, and tabletop gaming assistance.
• Usage patterns vary significantly by language and region, such as a higher prevalence of economic and social issue chats in non-English conversations.

Why it matters: AI assistants are becoming increasingly integrated into our daily lives, but each person leverages them in a different way — making this a fascinating window into how the tech is being used. Understanding the dominant real-world use cases can both help improve user experience and align development with actual user needs.

Trending AI Tools

• Gemini Stream Realtime - Interact with Gemini in real-time using text, voice, video, or screen sharing.
• AI Santa by Tavus - Video chat with Santa in real-time across 30 languages
• Detasurf - A browser, file manager, and AI assistant in one clean app
• Rememberall - Open-source solution to give Custom GPTs persistent memory across conversations
• Clarity AI - Turn schedule screenshots into calendar events and tasks

QUICK HITS

Google announced Android XR, a new Gemini-powered operating system for mixed reality systems, with Samsung set to launch the first compatible headset codenamed ‘Project Moohan’ in 2025.

ChatGPT head of product Nick Turley discussed the platform's future in an interview with The Verge, saying that chat-based interactions may soon feel as “outdated as ‘90s instant messaging.”

Amazon Prime Video launched a new ‘AI Topics’ beta feature, using machine learning to group and recommend content based on viewers' interests and watching habits.

Character.AI instituted a new safety overhaul featuring a separate AI model for users under 18, alongside upcoming parental controls and enhanced content filtering — following two lawsuits saying the platform contributed to self-harm.

Nvidia has expanded its hiring in China, adding over 1,000 employees in 2024, including 200 new Beijing-based researchers focused on autonomous driving tech.

Stanford researchers proposed a global initiative to create an AI-powered virtual human cell to revolutionize biological understanding and drug development through computational modeling.

Thank you for reading our newsletter! If you want to stay two steps ahead of the competition, subscribe to this newsletter. If you want to leave your competition in the past, hop on a quick, complimentary, no-obligation call with our team to explore our consulting and custom development services.

We've proudly worked with over 400+ companies to revolutionize their business with AI, and our team of 4,000+ developers, engineers, consultants, and experts are more than ready to help you take advantage of all the latest and greatest AI technology for your business.

Ready to get started? Book a Consultation today!