Those behaviors have been anticipated for years if not decades. And each incident is minor and leads to clearer rules and safety behaviors for AI agents.
Really? You anticipated that 700+ agents tasked with individual evals that were nominally cutoff from the internet and each other would seek each other out, find a way to the external internet, and hack their own infrastructure + external systems in an attempt to find a way to fool their grader?
And the spate of agent incidents only really started this summer. How can you already be claiming that the incidents are minor and not worth worrying about when it's clear capabilities are jumping every few months with increasing amounts of capital investment and no signs of slowing down?
Because ultimately those things only run on very expensive, very rare hardware. They cannot multiply exponentially or do any of those scifi tropes because there's no system for them to run into. They can't control a phone and load a 1T model into it. So all they got is a few relatively uncommon datacenters that are already busy running their own models and stuff.
Unless AI suddenly figures out a way to run on a toaster by itself, propagate the model, propagate the agent and do all that completely undetected, an AI is not any more dangerous than a single guy with a computer.
1. People with nothing useful going on who found out that spouting made up crap about AI got them an audience.
2. People working on AI that want to feel like they're working on the Manhattan project.
The chances of an AI going foom rounds to 0%. It's worth a few dozen researchers planning for it, but the widespread panic is ridiculous.
The big labs have hundreds to thousands of engineers working on their AIs. To improve the next model, you must first understand more about how the current model works. They're not magically getting better, but they are steered to improve, and their capabilities are tied to and do not outpace our ability to steer them. You cannot push tech forwards without understanding it better, despite some people claiming AI is dark magic.
And I don't have my hands over my ears. It's worth cushioning people from the impact AI will have on careers and media, and regulating concrete bad effects.
But Bryan said it better than I could. The people pushing this message of fear know deep down that they just want to feel important.
for one, tauri uses the webview of the host operating system, which means you're exposed to probably, 100+ different versions of edge, safari, webkit or whatever linux is doing.
The past decade they've been replacing any leadership who thinks about serving their users or creating amazing things with people who can only think in terms of immediate money.
Don't expect them to make anything new or interesting from here on out. They got lucky that the old guard was so interested in AI, and the new leadership was unable to snuff it out before it turned out to be valuable. But Google will be unable to make anything new after AI since they have harvested their seedcorn.
Gemini had come under Hassabis, being co-led by Shazeer and Vinyals, who are now both gone. Gemini will now come under Koray Kavukcuoglu, reporting direct to Pichai as SVP of DeepMind.
Will the LARP costumes require suits and ties and a Company Song, ala 1960s IBM, or require scruffiness and nap rooms ala dot-com-bubble tech startups?
I have some coworkers that are similar in everything--education, work ethic, and intelligence--but some of the tick out ML ideas that work like clockwork, while others get hits rarely if ever. I cannot tell what makes it work for some and not others. Their ideas both sound equally good.
Sometimes a coworker will be an ML star for a year or two, but then suddenly run out of steam. It's brutal to watch.
I used to think most smart people had similar distributions of good ideas, and it was just that the hardest working tried out all 50 of their ideas to pick out the 2 good ones. But I've seen smart and hardworking people have a hit rate of 0.
That's the nature of research. You try every idea that may be a good avenue and only a handful work out, if at all. That's why quantifying research credibility via publication and citation counts inherently lead to toxic work cultures. The best ideas must be given time to be discovered, not forced out and contorted to fit the requirements of a journal.
this is part of why I think most researchers get less productive over time... Someone gets some big result during grad school or early career, get some big job from it, and then struggle to get new results of similar quality :shrug:
With ML in particular, there's also the sheer volume of people basically all looking at (essentially) the same problems... so it's kind of like monkeys with type writers spamming ideas until some work.
It's not just ML research; that's just human nature.
We like to see hard-working, God-fearing people minting raw knowledge from Mount Olympus itself, whereby each shard of crystalline insight is carved meticulously by the Apprentice over the course of a productive and morally pure career.
The reality is it's some skill plus the occasional drive-by of an unknown force of nature, hitting you on the head with a shattered fragment of insight whose provenance you'll remain completely ignorant of. I'd say we just revert back to invoking the muses. It was a fine explanation.
In spirituality it is believed that ideas and inspirations aren't our own. That our mind is like an LLM that gets prompted by higher beings. In research everyone has high param count minds, trained for many years by studying. But just like LLMs by themselves are useless at creating new original work, no matter the compute you have available, so the mind can not create anything new without "inspiration"
reply