With the talk of agentic AI, which are generative AI platforms that can control computer software beyond giving text chats, being the future for the AI industry, one of the top agentic AI systems in the world, Anthropic’s Claude, still can’t beat Pokémon on Gameboy Colour.
Anthropic released a thread on X in February admitting that Claude 3.7, its latest model, was not able to play the original Pokémon RPGs on Gameboy to completion for a number of reasons, but also, despite its inability to finish the games made for children, the AI showed chilling human-like processes in attempting to do so.
Claude 3.7 is one of the most advanced agentic AI models out there, with companies like China’s Manus incorporating it into its systems.
I’m not entirely sure “made for 5 years olds” is accurate. I had the game around that age and certainly couldn’t beat it either.
I only beat it at ~10yo because I knew how to read walkthroughs, which is what finally led me to enter Saffron. At no point I thought that one of the purchasable drinks from a vending machine could be used to “bribe” the guard on the entrance, I always thought it was some special item somewhere (which became the case in the FR/LG remakes) or some other event.
But it makes AI sound stupider, so they went with it. Anyway outside gaming circles, a lot of normals still think all video games are for children.
The moveset of opposing Pokémon is quite limited and the AI is even worse. There’s really no strategy by the AI to actually win battles, so I’d say the real challenge was finding out where to go and what to do.
What is possible is to use the bicycle to bypass smaller walls, which means that the AI is linking the two together, which is actually scary and shows, perhaps, tiny glimpses into future AGI.
I have no love for AI, but whoever wrote this article has absolutely no idea what he’s talking about. This simply isn’t a thing in the OG Pokemon games.
Can we stop benchmarking text generation models on things they’re not designed to do and start educating people on what they actually can do?
Oh no we can’t, there’s already hundreds of commercial services…
lol, the screen shot where it’s “stuck in a cave” isn’t a cave, it’s those ledges before mt moon where twitch plays Pokémon kept getting stuck too.
I remember, TPP had to implement democracy to beat it. 30k people entering inputs and all it took was one player to press down and we go back again
artAI imitates life imitates AI
AI officially stupider than Twitch commenters
Then, they haven’t trained it on Pokemon enough.
Interesting related video: Training AI to Play Pokemon with Reinforcement Learning (Oct 2023)
also:
as Claude spends 10 minutes looking for its bicycle in its inventory in order to jump pass a barrier wall, which is not possible. What is possible is to use the bicycle to bypass smaller walls, which means that the AI is linking the two together, which is actually scary and shows, perhaps, tiny glimpses into future AGI.
And they want to put this shit in drones?!
