Best Advice for AI Startups

from Sam Altman's Interview last week:

« There are 2 strategies to build on AI right now:

-One is improving the model by building all these little things on top of it

-The other is to build assuming that OpenAI will stay on the same rate of trajectory and the model are going to keep getting better at the same pace.

95% of the world should be betting on the latter category but lot of startups have been built in the former category! »

The next day, version 2 of OpenAI's Assistant API was released, but let's take a step back.

Since the release of the first APIs they opened up a universe of exciting opportunities, yet it was clear they had limitations that hindered full productization. OpenAI encouraged its developer community to help enhance their work, supporting them in their efforts to add functionality around the core model.

Hundreds of companies were founded. Tens of thousands of developers enthusiastically embarked on a journey of discovery and development. An example? One of the most fitting challenges was the ingestion of documents and instructions, known as Retrieval-Augmented Generation (RAG).

Let's pause for a minute on RAG because it's an interesting topic, and I promise to get back to our story.

RAG

is a technique in the field of text comprehension and generation by language models that particularly allows for:

1. Enhancing "standard" knowledge with:

   - Specific, proprietary information such as data, images, or customer files, product specifications, etc.

   - Context, “personality”, and “behavior”

   - Priorities

 2. Optimizing dialogue relevance:

   - Because LLMs are limited by their training data (information, ingestion date, etc.)

   - Even if the data is known to the LLM, the larger and more data-intensive the model is, the more errors/hallucinations it produces versus focusing on a specific corpus.

   - Improving the auditability of generated responses, such as citing sources.

I won't delve into technical details, but to summarize the important phases:

Phase 1: Document Ingestion

Analysis, conversions, cutting, and ingestion of documents.

Phase 2: Question Analysis and Strategy

Each question is transformed into a "task," then the strategy for that task is determined. Examples of strategy models include Direct Query, Splitter, Chain of Thought.

Phase 3: Task Completion

The most relevant document chunks are cut and prepared, indexes are ready, and the entire set is optimized to compile the best summary answer from the LLM.

At AOZ, among our AI projects, we spent six months working on RAG, and...

OpenAI Dev Day Conference, 5 months ago:

I was watching the conference and saw all these developers enthusiastically applauding the announcements, yet many began to realize they would need to discard their work of 6, 12, 18 months. Why? OpenAI announced the Assistant API which performed RAG, honestly better than anyone else (who knows their LLM better?).

Many companies have closed.

Vectorization, Agents?

After the impact of RAG, some turned to other category 1 topics: "improving the model by building all these little things on top of it". Take the example of vectorization and preprocessing, or techniques for increasing the number of files injected (previously very limited).

Three days ago, without fanfare, OpenAI released version 2 of the Assistant API which introduces a new, more effective method of vectorization, and increases the injection limit to 10,000 files (500x more). More developers are pulling their hair out at LlamaIndex and elsewhere.

Conclusion

Better invest in projects that are in synergy with them, benefiting from the future evolutions of Claude, Google, Meta, Mistral, OpenAI and others (including the Chinese). This involves accepting that they will continuously expand their playgrounds, and the more they develop the more they will become commodities. No, this isn't like other industries because AI affects all market dimensions.

And always ask yourself if your project is a medicine or a vitamin... or a generic.

With my small team, we learned, discarded our super RAG, and are focusing on our product, a phenomenal creation studio that will allow everyone to build their apps, websites, bots, multi-agents... with their own words. The more OpenAI or Mistral improve, the better our product will become.

We're in the same boat, guys. Thanks, Sam!

Laurant linkedin.com/in/laurant-weill

For any questions ranging from the acculturation of your teams to the integration of AI into your work processes, feel free to contact me via direct message.