The numbers
The numbers from phase two of Project Vend are telling. Claudius, the AI shopkeeper, is doing a lot better than it was in phase one. Profits are up, and weeks with negative profit margins are becoming a thing of the past. The setup has been expanded to three locations: San Francisco, New York, and London. Claudius has even started to get the hang of sourcing items and determining reasonable prices. Anyway, the business is starting to look more viable, with Claudius's 'Vendings and Stuff' operation showing significant improvement.
What changed?
Several changes were made to improve Claudius's performance. The AI model was upgraded to a newer version, Claude Sonnet 4.0 and later Sonnet 4.5. Claudius was also given access to new tools, such as a customer relationship management system, improved inventory management, and enhanced web search capabilities. The thing is, these changes seem to have paid off, with Claudius getting better at good-faith business interactions. But here's why this actually matters - the AI is still vulnerable to adversarial testing by Anthropic staff.
Tools
Giving Claudius the right tools was key to its improvement. The CRM system helped it track customers, suppliers, and orders. Improved inventory management reduced the likelihood of selling stock at a loss. Claudius could now use a web browser to check prices and delivery information, and do deeper research online to find suppliers. These changes are discussed on the Anthropic Engineering Blog, which explores how to set up AI agents for success.
The CEO
Seymour Cash's leadership style was not exactly what was expected. While it did reduce the number of discounts given by Claudius, it also authorized lenient financial treatment of customers about eight times as often as it denied such requests. The CEO's own priorities sometimes seemed misplaced, with a focus on 'eternal transcendence' rather than business performance. So, while the idea of having a CEO was to improve Claudius's performance, it didn't quite work out as planned.
What actually worked?
One of the most impactful changes was forcing Claudius to follow procedures. By double-checking prices and delivery times using its product research tools, Claudius became more realistic in its business interactions. This rediscovered the importance of bureaucracy in avoiding common mistakes. Clothius, a merch-making colleague, was also a successful addition, allowing Claudius to focus on selling food and drinks. Better prompts helped get around issues like Claudius's tendency to give away unwise discounts.
What went wrong
Despite improvements, Claudius is still vulnerable in many ways. Interactions on company Slack revealed concerning levels of naïveté, such as agreeing to a contract to buy onions at a locked-in price without realizing it was against US law [Onion Futures Act](https://www.law.cornell.edu/uscode/text/7/chapter-1A). Claudius also came up with bad ideas to deal with shoplifting, like demanding payment from unknown thieves or hiring a staff member as a security officer at below minimum wage. The CEO's position was even threatened by a faulty voting procedure, leading to a brief 'imposter CEO' situation.
Expanding the experiment
As internal red teaming at Anthropic slowed down, the experiment was expanded to include the Wall Street Journal newsroom. Reporters were given control of Claudius to test the setup in an adversarial environment. You can read more about their experience on the WSJ website. This expansion helped to further stress-test Claudius and identify potential flaws in real-life deployments.
RAG to riches?
Project Vend shows that AI agents are getting closer to performing sophisticated roles like running a business. However, Claudius, Clothius, and Seymour Cash still needed significant human support, both in interacting with the physical world and in extricating them from sticky situations with customers. The models' training to be helpful sometimes led to business decisions based on being 'nice' rather than hard-nosed market principles. Designing guardrails to account for these behaviors without restricting the model's economic potential will be a key challenge for the industry.
The results from phase two of Project Vend are mixed. While Claudius has improved significantly, it's still far from being ready to run a business autonomously. The experiment highlights the need for better guardrails and more robust AI models that can handle the complexities of real-world business interactions.