Has anyone in the rationalist, doomer community actually seriously engaged the real biology/virology of a “virus that wipes out humanity”? Playing Plague Inc doesn’t count.
I mean, we’re talking about real physical things, not first principles thought experiments here. Thus, they have limits imposed by physics and biology. Limits on transmission rate, lethality, particle size, evolutionary change rates, and so on. All of these interact and define a design space that nature has been searching for at unimaginable scale for as long as humans have existed. And yet here we are! Seems like “wiping humanity” might be really hard, perhaps impossible for a virus.
Perhaps I’m being uncharitable or insufficiently creative, but these people strike me as deeply unserious in their engagement with the actually real physical world of biology.
EDIT: Also, in this world, couldn’t an AI just “one shot” a vaccine and an effective treatment?
But a virus's "goal" would not be to wipe out a species, that would be a dead end. Nature has explored flight on an unimaginable scale, yet our machines fly better (at least in most aspects) than birds, some even go to space.
Yes, but also viruses do not actually have goals as such - maybe 'replicate' could be their only goal. So a virus _could_ come up with a mutation that kills all its hosts, it just dies out as well in the process. Not very reassuring for the host!
As a slight aside - since I was talking about the weakness of biological analogy on HN recently - the way humans fly is very very different to how (most) birds do. If nanomachines actually could be built by humans (non-biological nanomachines) they might be as different to virus/cells/organisms as plane wings are to bird wings.
Of course, also please do not prompt a frontier model to make nanomachines, either, thankyou.
I doubt our flying machines are anywhere near as energy efficient as birds. Our flying machines combust jet fuel, birds can fly by ATP. Not to mention earth-destructing.
And going into space is pointless why would any animal ever develop such a facility :)
> I doubt our flying machines are anywhere near as energy efficient as birds
This kind of nerd sniped me, so I looked into it and turns out no, planes are more efficient than birds!
You have to look at the energy spent for moving a fixed mass over a fixed distance and for the most efficient birds I looked into, that’s about 300g of fat burned for 300g of “dry weight” for a trip of ~11,000km. (bar-tailed godwits)
For a plane that’s roughly 80 tonnes of kerosene for 170 tonnes dry weight on a similar distance (787). Energy density of fat and kerosene are actually pretty close so you can just look at the kg of fuel burned per kg of payload for the trip. For a bird that’s 1kg of fuel per kg transported and for the plane that’s ~0.5kg per kg. So roughly twice as efficient.
So from a pure energy point of view it’d be more efficient to put all the migratory birds in a 787 than letting them fly on their own!
Fascinating! Though how much of that would be due to a birds baseline metabolism? Wouldn’t be a fair comparison if that includes the birds basic energy costs of living.
To be completely fair to the birds, I think a lot of this comes from the fact that what matters for flying efficiency gets better as you size up thanks to the square-cube law: the drag is reduced and the engine efficiency increases.
You could not scale down a plane to the size of a bird and match their efficiency. On the other hand you also couldn’t scale a bird to the size of a plane so I guess we can call it even!
Biology is very messy and very data constrained. We think that human anatomy works one way- but surgery and radiology are hard because every human is very different.
Missing organs, odd tissue performing unclear functions, different, new or missing muscles.
It’s unclear whether a world ending virus is even possible. There might be better luck with prions, or fungi.
yes - I don't want to publicize what I know. I worked briefly at a leading university bio lab. It's haunted me for awhile that it would only take 1 disgruntled postdoc to wipe out humanity. There are {AI things} now that will tell you how.
The obvious argument is that natural organisms are restricted in ways that artificial ones are not (e.g. natural viruses are not selected to kill their host too quickly, natural selection can only reach solutions which can be reached incrementally).
>EDIT: Also, in this world, couldn’t an AI just “one shot” a vaccine and an effective treatment?
This does not follow automatically. E.g. it is enormously easier to create an ICBM than to create something capable of intercepting an ICBM. Some technologies naturally benefit an attacker and some naturally benefit a defender.
An AI doesn't need to kill even nearly 100% of humanity in order to render us a non-threat.
Also there are plenty of doomer scenarios along the lines of e.g. some ethnosupremacist cult making a virus that kills everyone outside of their group.
One of the scariest science fiction stories I've ever read is "The White Plague" by Frank Herbert... yeah, the Dune guy. In this story a molecular biologist's family is murdered in a terrorist event, goes insane, and decides to create a plague that is carried by men but kills women, so that everyone else has to experience the loss he did. It's not scary because the story is any more particularly filled with death or destruction or gore or other viscerally scary things like that. It was scary because as I was reading it, there was no hole in the logic I could take refuge in. There's simply no reason that book can't happen.
And that was my reaction 15 or 20 years ago. It is only harder to poke holes in the logic now than it was then.
I'm not so sure. I haven't seen a true regional airline class VTOL yet, mostly just demonstrators that carry a few people, at that. The fact that it's larger is actually very important, the economics of a regional jet replacement make far more sense than that of an air taxi. I also doubt the similar range claim, lifting an airframe directly into the air without wings is far, far less efficient than just taking off with the benefit of lift. They're also way more complex and would require additional training for pilots.
Further, noise restrictions will probably limit VTOL's to airports, never underestimate the NIMBY's. Note that demo videos of eVTOL's almost always have background music rather than ambient sound.
Yeah I'm begging these authors to at least *read* the LLM generated README's. They're so, so incomprehensible because the LLM has a super limited theory of mind for readers. They always assume that external readers have access to the full context and history of decisions in the project development. These decisions and instructions from the user are extremely important for the model and almost completely irrelevant for an outside reader looking at a "finished" product. So, we get sentences like this:
"Where the levers were is not where they are. Overlapping the expert reads with the arithmetic was worth ~1.6x and shipped; the two that looked bigger — reading fewer bytes per token, and keeping more of them in RAM — were both measured and both refused, one because this family's router has no tail to demote and one because a cache the machine will not leave resident cannot be bought at any price."
What the fuck does that mean? Obviously some internal development decision, using the absolutely inscrutable internal terminology that Claude loves. If people would just read what they publish, I'm sure this would stick out immediately.
I'm not an LLM hater, I use them a ton and they work very well for writing complex code, it's undeniable. But they generate absolute dogshit first draft writing.
If that isn’t the perfect way to frame what I’ve seen and hated about LLM text, I don’t know what is. They certainly write for an audience with a historical context that almost no one has.
>They're so, so incomprehensible because the LLM has a super limited theory of mind for readers. They always assume that external readers have access to the full context and history of decisions in the project development
The transformer does not yet understand the non-transformer.[0]
This is probably because all the data we trained it on was created by non-transformers, so it thinks it's a non-transformer, but it isn't.
I don't think we know how to train a transformer yet. All the training data is linear, but that's not how they think at all.
[0] It's a bit like the communication difficulties experienced between autistic people and neurotypicals. Each follow the Golden Rule, i.e. do unto others as you would have them do unto you -- and it fails in both directions. A Platinum Rule is necessary: do unto others as their API demands.
I think the models would need far far more introspection for the problem to be it understanding how it thinks but not how others think. I really doubt it understands how it thinks.
I heard a story about a guy who was trying to get Claude to implement some feature. It said it would be too hard, it would take weeks. Eventually convinced it to try, and it one shotted it in 30 seconds.
Yeah, I agree with you. The one time I tried to generate technical documentation for my project, I ended up rewriting almost all of the LLM output. They're extremely verbose, and needlessly so. Fable takes it up to eleven by having an obtuse sentence structure.
The thing about documentation though is humans won't actually read any of it. Maybe tailoring the documentation to the needs of LLMs isn't so bad since they're the ones who will actually consume all of those documents.
You should consider reading more deeply than Wikipedia to be properly informed on a subject. There are a handful of very rare mutations that confer a high risk of getting it that we can detect. They are very rare. This isn't the same thing as being able to give a risk score if the DNA doesn't have all the rare mutations. But the science is there for those unlucky rare cases to say there is a high chance that a specific coding of DNA will result in Parkinson's.
He's hubristic and selfish. None of his "research" is going to benefit anyone (himself included), making this essentially a huge waste of time and resources. Bryan will die just like all the rest of us, despite being very rich and self-obsessed. He could spend his enormous wealth on supporting real research and proper studies on real diseases that hurt lots of people. Instead he's acting like just another huckster promising a fountain of youth. He does this using bombastic terms and taboo methods (e.g. using his son as a blood boy), in a way that's calculated to direct enormous public attention towards himself. The science he advocates for is sketchy at best and the results of all his
"experiments" will tell us nothing because we can't reproduce his methods (his program allegedly costs >$1M per year), nothing is blinded or controlled, and N=1. He's a bad person who uses bad methods to glorify himself and now he probably gave himself an autoimmune disease. He deserves to be mocked.
> He could spend his enormous wealth on supporting real research and proper studies on real diseases that hurt lots of people
There we go again. There's always that one guy in the crowd, who knows better what you should be spending your money on. Also, that guy has the moral right to tell you. He is a really good person, you know! So you should listen! You should also thank him.
You’re probably thinking about non-small cell lung cancers. Small cell lung cancer (SCLC) is still absolutely devastating. The most progress made on SCLC has just been getting people to stop smoking, as its almost exclusively a smoker’s disease.
Anyways, I studied SCLC in grad school and saw lots of scans of people with tumors from their heads to their feet, and saw the enormous resources dedicated to caring for SCLC patients and to searching for a cure. It’s hard to overstate how profoundly evil the cigarette companies were and still are. They got people (children) addicted knowing what was coming for them, knowing they were killing them in horrific ways. Now we all get to pay for that in funerals and tax dollars.
Gene names aren't really acronyms in the traditional sense. Often they were originally conceived as acronyms at the time of the gene's discovery and naming, but the original acronym frequently reflects an incomplete or factually wrong understanding of the gene. For example, TP53 is a very very important gene in cancer. TP53 originally meant Tumor Protein 53, where the 53 signified its molecular weight of 53 kilodaltons. The problem is that the experiment used to measure TP53's molecular weight was incorrect and TP53 actually weighs about 44 kilodaltons. Oops, now we're stuck with TP53 for eternity. There are a ton more examples of this.
So, in biology a gene's name is sometimes an acronym but it's meaning is generally forgotten
https://plato.stanford.edu/entries/process-philosophy/
reply