upvote
I find this line of argument unconvincing because it ignores the alignment problem. We only administered mRNA vaccines after a large and diverse group of experts with different backgrounds and experiences came together and said that we should go ahead. We only administered them after several rounds of careful testing to ensure that they wouldn’t have damaging effects. The decision to administer mRNA vaccines for to large numbers of people for the first time was arguable risky and certainly controversial, and yet that’s what happened.

I have no issue with LLMs aiding human scientists, but imo we absolutely would need to understand your hypothetical advancements completely and thoroughly before surrendering any agency to them - the LLM’s risk analysis will not reliably reach the same conclusions as a democratic human society

reply
An LLM could also in theory fake alignment and create technology that seems really useful but secretly harms humanity in the long run. If humans don't understand what it does, that possibility becomes more likely.
reply
mRNA vaccines were created by humans who understood them. You exist within a society (a networked distributed information system) where you accept the judgement of the humans who understand and create this things, which is how you end up benefiting without personally understanding.

If we imagine Super Intelligence, where NO human is capable of understanding, then how would it ever be possible for any human to identify what is actually beneficial or not?

This resolves in a paradox, common to all magical thinking. You can certainly wish that some all powerful benevolent entity will solve all of your problems for you, but it is not likely to work out well.

None of this is new. It is the same delusions as alchemy and the same thing that tales about genies warn of.

reply
> mRNA vaccines were created by humans who understood them

If authors understood them, why most of their predictions were wrong? Time to decompose spike proteins, side effects, distribution through brrast milk....

Event the most basic promise 100% protection against infection was wrong! They lied and we allowed infected people into hospitals!

reply
It's incorrect to assume that the information you heard is 100% an accurate representation of the scientific understanding and not information filtered through "public health" which balances that with other interests. I'm not aware specifically what was known at the time about the initial vaccine formulation, but there are several examples where there was a disconnect:

- ~Feb 2020: COVID is known to spread through and linger in the air, not just spread via droplets or fomites; public health messaging fixated on simultaneous 6ft distancing and surface cleaning rather than air filtration and masking

- Spring 2020: it's known that masks reduce the risk of acquiring and spreading infection; in the US, public health messaging stated the opposite, in order to keep as many masks in supply to healthcare workers despite a major shortage of masks

- Jan 2022: it's know that an infected person can remain contagious for over two weeks; in the US, public health isolation guidelines are reduced to 5 days due to noncompliance and in response to pressure from the airline industry, which was experiencing a significant shortage of flight staff due to COVID infection

reply
> in the US, public health messaging stated the opposite, in order to keep as many masks in supply to healthcare workers despite a major shortage of masks

So experts also lied about masks!!!

So many people died because of those lies!!!!

They spread misinformations!!!

reply
I'm saying that "experts" here is not one group. The politicians and public figures choosing what information to publicize and disseminate are not the same people as those who worked on developing the vaccines.
reply
There isnstrong scientific consensus around masks and vaccinees! They work 100%, are 99.999999999999999% effective and there are no measurable side effects (bellow 70%)!

Anyone who says otherwise is not scientist!

reply
> why most of their predictions were wrong?

Citation needed.

reply