I think there are overlooked implications of auto model selection in ChatGPT, which seems to be rolling out slowly. First, this can massively impact OpenAI’s profitability if it can direct more queries to smaller models without adversely affecting quality. Second, this can prevent people leaving ChatGPT for other apps—like those running local models—if it’s just easier to stay in one app and get a faster model or smarter model automatically, rather than use small local models for some things and switch to ChatGPT for others. It’s even possible that ChatGPT could in future run local models on properly equipped devices for the easiest queries, which would completely avoid hitting OpenAI’s servers for those queries. So, it’s a small feature, but I think one with big implications.
Eric Lortie
1mo
Good point. The future of these systems is something like an autonomous AI ecosystem, and so far ChatGPTs auto model select is the most efficient way there is for someone to interact with one. Future versions will have the ability for models to work with other models in the ecosystem before returning your result to you. Future versions will be able to hit API endpoints and do things in remote systems. OpenAIs price increase makes sense if you're paying for a system that can surreptitiously do a lot of the technical work of your job from a single website or app with limited instructions or interactions.
James Kaye
1mo
Another benefit is environmental, most queries won't need a huge model. You only need the big 4x4 car when going off-road instead of going half a mile down the street. Some open source alternatives of this system were claiming 70%+ cost savings.
EIR @ Peak • Product / UX Leader • ex-Google & Gradient Ventures
1moOn top of that, I think this is also just good user experience. Even for early adopters who like to tinker with LLMs, I don't think most always know which model is the best option for a specific query. And more importantly, they don't want to spend the time and headspace doing that mental triage. They just want ChatGPT to solve their requests.