GenAI Weekly — Edition 15

Your Weekly Dose of Gen AI: News, Trends, and Breakthroughs

Stay at the forefront of the Gen AI revolution with Gen AI Weekly! Each week, we curate the most noteworthy news, insights, and breakthroughs in the field, equipping you with the knowledge you need to stay ahead of the curve.

➕ Click subscribe to be notified of future editions

PDF Hell and Practical RAG Applications

From the Unstract blog:

Struggling with PDF files is something we can all relate to. Extracting text from them might seem straightforward, but if you've tried to do it programmatically, you know how challenging it can be.

In his latest blog, Unstract's co-founder Arun Venkataswamy delves into these challenges, from extracting data and managing large documents to optimizing workflows. He shares how Unstract tackles these issues head-on.

This post explores common PDF frustrations and offers practical solutions Unstract has discovered. Whether you're dealing with complex forms, managing a large volume of documents, or simply looking for tips to ease your workflow, this post is for you.

What We’ve Learned From A Year of Building with LLMs

Eugene Yan et al:

We’ve spent the past year building, and have discovered many sharp edges along the way. While we don’t claim to speak for the entire industry, we’d like to share what we’ve learned to help you avoid our mistakes and iterate faster. These are organized into three sections:

• Tactical: Some practices for prompting, RAG, flow engineering, evals, and monitoring. Whether you’re a practitioner building with LLMs, or hacking on weekend projects, this section was written for you.
• Operational: The organizational, day-to-day concerns of shipping products, and how to build an effective team. For product/technical leaders looking to deploy sustainably and reliably.
• Strategic: The long-term, big-picture view, with opinionated takes such as “no GPU before PMF” and “focus on the system not the model”, and how to iterate. Written with founders and executives in mind.

Our intent is to make this a practical guide to building successful products with LLMs, drawing from our own experiences and pointing to examples from around the industry.

Must read. Nothing else to say.

Mistral releases Codestral, an open weights code model “fluent in 80+ programming languages”

From the Mistral blog:

Mistral AI

We introduce Codestral, our first-ever code model. Codestral is an open-weight generative AI model explicitly designed for code generation tasks. It helps developers write and interact with code through a shared instruction and completion API endpoint. As it masters code and English, it can be used to design advanced AI applications for software developers.

A model fluent in 80+ programming languages

Codestral is trained on a diverse dataset of 80+ programming languages, including the most popular ones, such as Python, Java, C, C++, JavaScript, and Bash. It also performs well on more specific ones like Swift and Fortran. This broad language base ensures Codestral can assist developers in various coding environments and projects.

Codestral saves developers time and effort: it can complete coding functions, write tests, and complete any partial code using a fill-in-the-middle mechanism. Interacting with Codestral will help level up the developer’s coding game and reduce the risk of errors and bugs.

AI darling Nvidias market value surges closer to Apple

Zaheer Kachwala writing for Reuters:

Reuters Zaheer Kachwala

May 28 (Reuters) - Nvidia's (NVDA.O), opens new tab shares rallied around 6% to hit a record high on Tuesday, leaving the AI chipmaker's stock market value about $100 billion away from overtaking Apple (AAPL.O), opens new tab in a major reshuffle of Wall street's biggest players.

Last trading at $1,128, Nvidia's market capitalization reached $2.8 trillion, compared to a market value of $2.9 trillion for Apple, which is Wall Street's second-most valuable company after Microsoft.

Its stock surged as much as 8% to $1,149.39 during the session, an intra-day record high. Apple's stock was down 0.2% in afternoon trading.

Nvidia's shares have surged nearly 13% since it forecast second-quarter revenue above Wall Street expectations last week and announced a stock split, which excited investors as they continue to bet on the AI poster child.

Company #1, Microsoft and Company #2, Apple by market capitalization are household names. Company #3, Nvidia (which might overtake Apple to grab the #2 spot) is a strange aberration and is virtually unknown outside tech circles. Of course, bubbles are only apparent in hindsight. But, Nvidia’s valuation goes to show the kind of impact AI is having on our perception of the future.

1-Bit LLMs are smaller, speedier—and nearly as accurate

Matthew Hutson writing for IEEE Spectrum:

IEEE Spectrum Matthew Hutson

Large language models, the AI systems that power chatbots like ChatGPT, are getting better and better—but they’re also getting bigger and bigger, demanding more energy and computational power. For LLMs that are cheap, fast, and environmentally friendly, they’ll need to shrink, ideally small enough to run directly on devices like cellphones. Researchers are finding ways to do just that by drastically rounding off the many high-precision numbers that store their memories to equal just 1 or -1.

LLMs, like all neural networks, are trained by altering the strengths of connections between their artificial neurons. These strengths are stored as mathematical parameters. Researchers have long compressed networks by reducing the precision of these parameters—a process called quantization—so that instead of taking up 16 bits each, they might take up 8 or 4. Now researchers are pushing the envelope to a single bit.

Fascinating.

Elon Musk’s xAI raises $6B at $24B valuation to build new AI services

Maria Deutscher writing for Silicon Angle:

Artificial intelligence startup xAI Corp. has raised $6 billion in Series B funding round to support its product development and commercialization efforts.

The company, which was founded last year by Elon Musk, detailed on Sunday that the round saw the participation of than a half-dozen investors. The participants included Valor Equity Partners, Vy Capital, Andreessen Horowitz, Sequoia Capital, Fidelity Management & Research, Prince Alwaleed Bin Talal and Kingdom Holding, as well as others. The investment values xAI at $24 billion, up from $18 billion before.

The company said that it will use the proceeds from the round to bring its first commercial products to market. In parallel, x.AI will use a portion of the capital to build “advanced infrastructure.” The disclosure comes less than a week after The Information reported that the company plans to assemble a supercomputer with 100,000 H100 graphics processing units from Nvidia Corp. to support its AI development efforts.

The H100 was succeeded as the chipmaker’s flagship GPU in March by the newer Blackwell B200. When demand for the former processor was at its peak last year, retail prices reportedly reached $40,000. Even if the launch of the Blackwell B200 will cut the H100’s price by two-thirds, the 100,000-GPU supercomputer xAI reportedly plans to build would still cost more than $1 billion.

See also: X.ai’s blog post on the Series-B

The Wall Street Journal ranks ChatGPT, Copilot, Gemini, Perplexity and Claude

Dalvin Brown, Kara Dapena and Joanna Stern:

Dalvin Brown Kara Dapena Joanna Stern

What did these Olympian challenges tell us? Each chatbot has unique strengths and weaknesses, making them all worth exploring. We saw few outright errors and “hallucinations,” where bots go off on unexpected tangents and completely make things up. The bots provided mostly helpful answers and avoided controversy.

The biggest surprise? ChatGPT, despite its big update and massive fame, didn’t lead the pack. Instead, lesser-known Perplexity was our champ. “We optimize for conciseness,” says Dmitry Shevelenko, chief business officer at Perplexity AI. “We tuned our model for conciseness, which forces it to identify the most essential components.”

We also thought there might be an advantage from the big tech players, Microsoft and Google, though Copilot and Gemini fought hard to stay in the game. Google declined to comment. Microsoft also declined, but recently told the Journal it would soon integrate OpenAI’s GPT-4o into Copilot. That could improve its performance.

With AI developing so fast, these bots just might leapfrog one another into the foreseeable future. Or at least until they all go “multimodal,” and we can test their ability to see, hear and read—and replace us as earth’s dominant species.

Training is not the same as chatting: ChatGPT and other LLMs don’t remember everything you say

From Simon Willison’s blog:

I’m beginning to suspect that one of the most common misconceptions about LLMs such as ChatGPT involves how “training” works.

A common complaint I see about these tools is that people don’t want to even try them out because they don’t want to contribute to their training data.

This is by no means an irrational position to take, but it does often correspond to an incorrect mental model about how these tools work.

Short version: ChatGPT and other similar tools do not directly learn from and memorize everything that you say to them.

This can be quite unintuitive: these tools imitate a human conversational partner, and humans constantly update their knowledge based on what you say to to them. Computers have much better memory than humans, so surely ChatGPT would remember every detail of everything you ever say to it. Isn’t that what “training” means?

That’s not how these tools work.

Important to know this stuff.

The future of financial analysis: How GPT-4 is disrupting the industry, according to new research

Michael Nuñez writing for Venture Beat:

Michael Nuñez

Researchers from the University of Chicago have demonstrated that large language models (LLMs) can conduct financial statement analysis with accuracy rivaling and even surpassing that of professional analysts. The findings, published in a working paper titled “Financial Statement Analysis with Large Language Models,” could have major implications for the future of financial analysis and decision-making.

The researchers tested the performance of GPT-4, a state-of-the-art LLM developed by OpenAI, on the task of analyzing corporate financial statements to predict future earnings growth. Remarkably, even when provided only with standardized, anonymized balance sheets, and income statements devoid of any textual context, GPT-4 was able to outperform human analysts.

“We find that the prediction accuracy of the LLM is on par with the performance of a narrowly trained state-of-the-art ML model,” the authors write. “LLM prediction does not stem from its training memory. Instead, we find that the LLM generates useful narrative insights about a company’s future performance.”

Ex-OpenAI board member reveals what led to Sam Altmans brief ousting

Jyoti Mann writing for Business Insider:

The former OpenAI board member Helen Toner has shared explosive new details about what led to CEO Sam Altman's brief ousting in November. In an interview with Bilawal Sidhu on "The TED AI Show" that aired Tuesday, Toner said Altman lied to the board "multiple" times. One example Toner cited was that OpenAI's board learned about the release of ChatGPT on Twitter.

In an interview with Bilawal Sidhu on "The TED AI Show" that aired Tuesday, Toner said Altman lied to the board "multiple" times. One example Toner cited was that OpenAI's board learned about the release of ChatGPT on Twitter.

She said Altman was "withholding information" and "misrepresenting things that were happening in the company" for years.

See also: Paul Graham on X: I got tired of hearing that YC fired Sam, so here's what actually happened:

If you've made it this far and follow my newsletter, please consider exploring the platform we're currently building: Unstract—a no-code LLM platform that automates unstructured data workflows.

For the extra curious

• Notes on the theory behind models like Stable Diffusion and their applications.
• Transformers Can Do Arithmetic with the Right Embeddings
• Reproducing GPT-2 (124M) in llm.c in 90 minutes for $20
• Indexing all of Wikipedia, on a laptop
• Kolkata’s Pen Hospital
• Continuing last week’s coverage of issues in Google’s use of AI in search: Google promised a better search experience — now it’s telling us to put glue on our pizza
• GPT-4o’s Chinese token-training data is polluted by spam and porn websites
• Gemini 1.5 technical paper