Today the world has changed. People don't know it yet, but it has. Anthropic is the first frontier model provider to release a product that allows an AI to control your computer. It's still experimental. But it's a game changer. I'm not one of those people who go on YouTube and post hyperbolic headlines. In this case, I feel like AI has crossed another rubicon. Mind you, there have been some demos, often overblown or fake, of AI controlling screens. (ahem, Devin). There have been folks sending screenshots in sequence to good effect. These were interesting efforts. But a frontier model will give AI access to do tasks for you on your computer. At scale. It will give devs access to offer you so many features on your computer via APIs. At scale. Agent: Unlocked. In the image attached, you see a screenshot of Anthropic's computer use feature, asking Claude 1, to go to the browser, navigate to claude.ai, ask it to create a webpage with a 90s style theme. In the video, claude goes on to download the html locally. Pop open a code editor to improve it. Mindblown. This is the GitHub repo where you can try it using a Docker, Inc container: https://lnkd.in/gt8QjuDA Enjoy. P.S. any thoughts Kwasi Ankomah, Aniruth Sarathy, Felicia Schwartz, Mohan Gulati
Great share Joe Devon! This is a significant milestone in AI development. Exciting to see AI moving from demos to real, practical computer interaction - especially from a company prioritizing safe deployment. The accessibility potential here could transform how people interact with computers. While cost (API calls) is currently a barrier, this should improve with time. I appreciate Anthropic's safety-first approach but hope to see more open research collaboration, similar to OpenAI's Swarm contribution, to accelerate progress responsibly. The pace is incredible - from basic chat to computer control in such a short time. Can't wait to see developers create practical applications that add real value. Hope you're well!
Joe Devon, I believe Anthropic's new computer access capability holds great promise for boosting productivity, speeding up product development, and aiding users with physical limitations. I value Anthropic's focus on safety through vulnerability analysis and recommended isolated use. As this technology evolves, users should protect their private information and control critical actions, like financial transactions. Apple is pursuing similar innovations with it's apple intelligence and Microsoft considered something similar on their laptops but faced privacy concerns over continuous screenshots. I expect this technology to become more common and refined in the next 6-12 months—exciting times ahead!
This is a pivotal moment in AI evolution, Joe Devon. The ability for AI to interact directly with our computers opens up a realm of possibilities that can truly transform how we work and innovate. It's exciting to think about the potential for developers to create features that enhance productivity and creativity at scale. I'd love to combine this with Voice AI and explore how this can work... guess what I might be doing this weekend.
Yes! Going forward API > UI. There needs to be a standard way to monetize API access that is both frictionless and reasonable.
This is fascinating and something to definitely keep a pulse on as it evolves. Thanks for the heads up!
DBA Manager at Hulu
2moHi Joe, Isn’t what Apple Intelligence is already doing?