Absolutely. Docker is a boiling mess of baked-in convenience workarounds (ie vulns). It's an orchestration layer (like k8s with containers), not a real security boundary like VMs. OTOH, single-purpose VMs are basically just as easy these days and those can still pull in containers as needed.
No, he's saying that licensing or additional regulation isn't necessary when torts get involved (and states attorneys general get perturbed!)
These don't tend to utterly destroy an industry, but they are often successful in forever transforming it. Just ask Big Tobacco. No new laws needed: if your product hurts someone else, you're eventually going to be found liable, regardless of your arbitration clauses. Additional laws will just slow down innovation, which will itself cause harm (AI is already becoming quite good at recognizing melanomas, for example)
It was pretty well understood by the 1960s that smoking was harmful. The big tobacco settlement was in 1998. That is an extremely bad example of tort being a sufficient alternative to regulation.
If we're on a similar timeline with AI if we reach a consensus that AI is dangerous today, then we'd be looking at a big lawsuit finishing up around the year 2070, give or take a few years. I'm not sure if we need regulation, and I'm definitely not sure that regulation could actually be effective for this, but tort a la the big tobacco lawsuits is definitely not a reasonable alternative.
Lol, wtf. Tobacco delayed any punishment for decades before general public sentiment changed enough to go against them. In light of the AI race, we'll already have our heads blown off by a terminator before the legal system will present any significant delay for them.
I get the snark (and slightly agree), but that's not really what GP or TFA were saying at all. They are saying that these were the least things we could have done.
What you're saying is, "Your scientists were so preoccupied with whether they could, they didn't stop to think if they should"
while the author of the TFA was saying, in effect: "your scientists didn't even bother with the most basic duty of care"
Life finds a way, or, in this case, super-intelligent AI.
This is the case with all complex system failures. There were always obvious fixes that could’ve prevented it. Problem is that there are an infinite number of obvious fixes to make at any time to any system, and the reason we don’t is because we have finite resources and no reason to fix X over Y until oops turns out X was “responsible” for this most recently realized failure. But of course it could have just as easily been Y, or Z, or any of the other infinite “obvious fixes not-yet-realized into catastrophe.”
> This is the case with all complex system failures. There were always obvious fixes that could’ve prevented it.
From this writeup and the Black Hat talk I'd really disagree. That would be like saying my hospital getting ransomwared because we didn't update our version of MSSQL because no one in particular was in charge of keeping dependencies up to date.
Sure systems are complex, but this is well trodden territory. Agents aren't the first things trying to break in or out of sandboxes, or the first ones to have done it, and based on these reports the reason they were able to work on this for so long was not because of super human intelligence.
You disagree with the statement "there were always obvious fixes that could've prevented it" with the response "no no, these were very obvious fixes that could've prevented it?"
You're missing the point about complex failures.
It's that if this particular path were unavailable, there are countless other similar paths. At sufficient scale and complexity, hitting one of those other countless paths is virtually guaranteed.
Let's say I drive Los Angeles to New York City. You look at the route I took and say, gee wiz, aren't you lucky that a tree didn't fall right there on Route 66? If a tree had fallen there (if we had "patched the particular route you took"), you would've been screwed!
But that's obviously not true. There were an infinite number of routes I could've taken. Any one of them would be equally "obviously preventable" by the same hypothetical tree falling across the whatever road I happened to end up taking. But you can't put trees across every single path between Los Angeles and New York City. The smarter I am and the more complex the map between us, the more impossible it becomes to put trees across all possible paths.
I disagree. To use an analogy, air travel in the US is relatively extremely safe - not 100%, but we've built up a culture around air safety that is very robust. Conversely, when I order packages online, sometimes they never show up, or the box is banged up, or the box is missing things, etc.
They're both complex systems, but clearly there is a much higher level of care given to human air travel than package delivery. A lot of the article basically saying that OpenAI gave "package delivery" level of care when they should have given "air travel" level of care.
At the very least I think the systems that run these tests should be fully, 100% air gapped. I'm not pretending that's easy given how much compute and data these systems use, but it is doable, and I think all AI development should be paused until that can be assured.
I agree on the 100% airgap idea, and I agree there are varying levels of care that can and should be deployed against a problem.
The point I'm making (and it's a point that shows up in every air catastrophe investigation) is that catastrophes in complex systems emerge only amidst repeated and widespread near-misses at many levels of a system. So many things have to go wrong simultaneously, that it can only happen even once because the underlying failures (that do not reach catastrophe) are extremely common.
You cannot look at an air catastrophe and retrospectively say "failures X, Y, and Z were observed, therefore if we correct failures X, Y, and Z, we would have been okay."
The takeaway is "failures X, Y, and Z were observed, which necessarily happened in an environment of failures X_0 through Z_10x10^10, and so therefore patching X, Y, and Z would be insufficient to address overall risks of the system."
The problem OpenAI is facing is that, short of 100% airgap (which they obviously won't do), they're facing an adaptive adversary that's increasingly intelligent, acts at far greater clock speed than any human or group of humans, has lower coordination cost than any group of humans, and operates in a game space that (in lieu of an airgap) is well beyond the comprehension of any human being.
So identifying and addressing "specific failures X, Y, Z" is insufficient, but then even defining the space in which to look for (and address) the more systemic failures X_0 through Z_n is a fool's errand. An intelligent system that makes its way to the Internet has can exploit a failure space that is approximately "all security failures across any organization." The Anthropic incident a few months back illustrates this isn't even limited to technical vulnerabilities, as these models are willing and able to engage in social engineering too.
> You cannot look at an air catastrophe and retrospectively say "failures X, Y, and Z were observed, therefore if we correct failures X, Y, and Z, we would have been okay."
That's literally exactly what air safety researchers do in an air disaster. There is a famous saying along the lines of "Air travel regulations are written in blood", meaning that all the regulations we have now are a result of fixing issues that led to previous disasters piece-by-piece.
> The takeaway is "failures X, Y, and Z were observed, which necessarily happened in an environment of failures X_0 through Z_10x10^10, and so therefore patching X, Y, and Z would be insufficient to address overall risks of the system."
Yes, I 100% agree with this. But I think that's what the author of the article was saying as well:
> That report had one key new piece of information, and some good prosaic steps OpenAI will be taking to strengthen its alignment, training, supervision, infrastructure and incident response.
> Mostly it confirmed what we already knew. The questions we most wanted answers to, that we did not already know, were mostly not answered. There was a distinct lack of self-reflection, especially about decision making and safety culture, and about the approach to alignment. I came away disappointed.
I.e. the "prosaic steps" are just the "fix X/Y/Z" as you point out. But what is needed is a more fundamental rethinking around stuff like safety culture, monitoring, and even things like better research into how agents do decision making in the first place.
Oh good. All we have to do to survive is always bother with the most basic duty of care. I'm sure that from now on, humans will take care to do that every time.
EU data regions are based on the insanely flawed idea that data is:
* a physical thing that can only live in one place
* not copyable
* can be 'contained'.
The whole thing reeks of bureaucratic 'best practices' that just aren't.
Even worse than that, trying to keep email restricted to the EU (or anywhere else) means that you effectively wouldn't be able to communicate with anyone in a different region, which is kinda the whole point.
Why not just make your own internet next? and then you can disconnect from everyone else who is trying to hack you. Just pull your network plug.
Email itself is hopelessly insecure by design anyway. Not just metadata when you are E2EE everything inside the envelope, but even basic vulns like downgrade attacks are simple because it's literally a violation of the RFCs (so you're not spec-compliant) to require TLS or any other encryption.. Why? because requiring modern crypto might interfere with deliverability and backwards compatibility. The real, deeper reason is that email is from a kinder, simpler time (well, at least simpler) and the design goals were never updated to keep up with the times.
Email is what we have. Just understand its flaws and then use other tools where you can. And who cares where your email lives - it's too easy to break anyway.
This is not an EU law anyways, this is snakeoil companies acting like having their data located in the EU will change who has access to it and will make it "GDPR compliant" (it wont since the CLOUD ACT still applies)
> The CLOUD Act authorizes bilateral agreements between the United States and trusted foreign partners that will make both nations’ citizens safer, while at the same time ensuring a high level of protection of those citizens’ rights.
I think the keywords are "trusted" and "citizens' rights". US burned a lot of trust in the last few years, and what's happening with ICE doesn't really scream "citizens' rights" either. I can see why many "trusted foreign partners" would now think twice rather than help out the US compared to just 5 years ago.
As a EU person, I'd really like to not have ties with US when possible, and I'd really like to foster the economy of non-US alternatives.
Is this hyperbole, or are you seriously claiming that the Jewish men who run Breitbart, or Jewish founders like Ben Shapiro or Andrew Breitbart, are neo-Nazis?
Nazi became a bit too broadly used in place of fascism, it's American Fascism, its own special flavour of it which is not anti-semitic (quite the opposite, it's actually very pro-semitic with its allegiance to Israel).
Of course it's all semantics, and you very likely understand that.
American Fascism is quite antisemitic, in spite of its successes with a certain subset of Jewish people. Contemporary American fascism's support of Israel is rooted in a religious belief that the Jewish people have to be in control of Israel before they're slaughtered. It's a millenarian death cult with nuclear weapons.
It's not semantics, they're just liars and fools. There were Jewish Nazis, now we have Jewish neonazis. Greed knows no bounds.
The issues that I have with Redis are not at all its API (which is elegant and brilliant) or even its serialized, single-core, single-threaded design, but its operational hazards.
As a cache or ephemeral store like a throttling/rate limiting, lookup tables, or perhaps even sessions store, it's great; but it's impossible to rely on the persistence options (RDB, AOF) for production data stores.
You usually only see this tendency with junior devs, though. It might be a case where "when all you have is a hammer, all you see are nails", or when someone discovers Redis (or during the MongoDB hype cycle ten years ago), which seems like it's in perfect alignment with their language datatypes, but perhaps this is mostly because junior devs don't have as many production-ready databases (from SQL like Postgresql, CockroachDB, Yugabyte to New/NoSQL like ScyllaDB, YDB, Aerospike) to fall back on.
Redis shines as a cache for small data values (probably switch to memcache for larger values, which is simpler key-value but generally 3 to 10 times faster for that more narrow use case, although keep an eye on memory fragmentation and slab allocation)
Just think carefully before storing long-term data in it. Maybe don't store your billing database in it :)
I have seen horrifying use of Redis where I inherited the maintainance of an application whose original developer implemented his own home grown design to manage relationships between different types of key value pairs, pretending like they were tables including cross-referencing logic; it took me a week to just add test cases with sufficient logging to reveal the "schema" and mutation logic. All this with the non-technical manager wondering why it took so long to make the change which directly dependended on understanding this. To top it all, the code was barely better than spaghetti with less than ten lines of comments across maybe 5k LOC. The irony was that this was not a latency sensitive application - it did data quality checks and could have been implemented in a much more cleaner and flexible way using, e.g., PostgreSQL.
OLAP questions are usually out-of-band and preferably by a tool designed for it (like Clickhouse). Scanning all DB's is something that can be done in the background for most of these use cases.
