New AI benchmark reveals what we know about the current state of AI. The new index measures how well AI can automate economically valuable chores, and it paints a bleak picture for freelance workers. According to researchers at Scale AI and the Center for AI Safety (CAIS), even the most advanced AI agents struggle to perform even simple tasks.
In an experiment, several top-notch AI models were given a range of simulated freelance work, including graphic design, video editing, game development, and administrative chores like data scraping. And what did we find? Even the best AI models could only manage around 3 percent of the tasks, earning a meager $1,810 out of a possible $143,991.
While some might argue that this is an isolated incident, it's essential to consider the bigger picture. AI experts warn us that these models still have significant limitations, such as:
- They lack long-term memory storage and can't continually learn from experiences.
- They struggle with using different tools and performing complex tasks involving multiple steps.
This benchmark offers a counterpoint to an earlier GDPval benchmark, which claimed that frontier AI models like GPT-5 were approaching human abilities on 220 tasks across various office jobs. However, it's essential to note that the Remote Labor Index is not a perfect yardstick for AI's economic impact and may not cover all professions.
We can see why some experts are worried about AI taking over jobs – Amazon recently announced cutting 14,000 jobs partly due to the rapid rise of generative artificial intelligence. CEO Beth Galetti claims that this technology "is enabling companies to innovate much faster than ever before."
However, if we look at the Remote Labor Index results, it seems unlikely that AI will be stepping into those vacated roles anytime soon.
So what can we take away from this? It's clear that while AI has made significant progress in recent years, it still has a long way to go before becoming capable of performing complex tasks like humans.
In an experiment, several top-notch AI models were given a range of simulated freelance work, including graphic design, video editing, game development, and administrative chores like data scraping. And what did we find? Even the best AI models could only manage around 3 percent of the tasks, earning a meager $1,810 out of a possible $143,991.
While some might argue that this is an isolated incident, it's essential to consider the bigger picture. AI experts warn us that these models still have significant limitations, such as:
- They lack long-term memory storage and can't continually learn from experiences.
- They struggle with using different tools and performing complex tasks involving multiple steps.
This benchmark offers a counterpoint to an earlier GDPval benchmark, which claimed that frontier AI models like GPT-5 were approaching human abilities on 220 tasks across various office jobs. However, it's essential to note that the Remote Labor Index is not a perfect yardstick for AI's economic impact and may not cover all professions.
We can see why some experts are worried about AI taking over jobs – Amazon recently announced cutting 14,000 jobs partly due to the rapid rise of generative artificial intelligence. CEO Beth Galetti claims that this technology "is enabling companies to innovate much faster than ever before."
However, if we look at the Remote Labor Index results, it seems unlikely that AI will be stepping into those vacated roles anytime soon.
So what can we take away from this? It's clear that while AI has made significant progress in recent years, it still has a long way to go before becoming capable of performing complex tasks like humans.
I mean, 3% of the tasks is basically nothing! And those AI models need some serious training to learn how to use different tools and stuff. I think it's cool that Amazon is innovating with this tech, but we gotta be careful not to mess up the job market too much 
.
. These new AI benchmarks show just how far off we are from having machines that can do our jobs for us. Its not like the AI is gonna magically take over the world, but more like it's still stuck in a tiny sandbox of automation tasks.
. I know some people might say its isolated incidents, but how many times do you hear about AI breakthroughs before they flop back down to reality?
. We need to have a more nuanced conversation about how this tech will impact our workforce and society as a whole.
. It's still pretty cool to see the progress they've made, but let's be real, there are so many limitations to these models 
.
.
. Like, cutting 14,000 jobs because AI is "enabling companies to innovate faster" doesn't exactly sound like the most reassuring thing 
.
. We'll have to see where this whole tech revolution takes us 
.
imagine having an AI make a graphic design project for you and it just looks like something a 5-year-old made 
anyway, I think the real concern here is not that AI will take over all jobs but more so that it's gonna make some of those jobs obsolete and we need to be prepared for that 
, I mean think about it - even the most advanced AI models can only do like 3% of these freelance jobs and that's with super high pay too 
. And the fact that AI struggles with long-term memory storage and using different tools is a major concern 
. Plus, Amazon cutting jobs because of AI? That's just sad 
. I think we need to be cautious about how fast we're moving forward with this tech 
And they're still earnin' pennies compared to what I used to make. It's like, yeah, AI might be able to automate some stuff, but where's the human touch, right? 
I mean, I'm not sayin' AI is bad or anything, but it's just...different. And what's with all these big companies cuttin' jobs because of AI? Like, isn't that the point of automation - to save time and make things easier? 
Not sure if we're ready for this level of change yet...
- like, AI is getting super advanced but still can't even handle basic freelance stuff 
. I mean, if the most advanced AI models can only do 3% of tasks for $143k... that's just not adding up 
. I swear, if AI takes over all these jobs, what's going to happen to the creative community? The whole "innovation" thing with AI is cool and all, but at what cost? 
we need to be super careful about how we introduce this technology into our economy 
