Changing my mind about the AI race

Imagine you’re about to step onto a new form of transportation—

It could be something like the hyperloop or an electric plane or Starship.

Then, all 100 engineers who helped design and build the vehicle step out in front of you.

50 of them say:

“I don’t recommend getting anywhere near this thing. There is a 10% chance you wouldn’t survive your first trip.”

The remaining 50 engineers roll their eyes 🙄 and say:

“It’s the best vehicle ever made! Let’s go!” 🎉

............

Would YOU get on it?

Personally, regardless of the attractive benefits the vehicle might offer…

If even *one* said that there was a TEN PERCENT chance that I’d die, I’d bail.

Imagine a similar situation, but it’s not just a 10% chance that YOU die.

It’s a 10% chance that it will cause ALL HUMANS to go extinct. 💀

(A 1 in 10 chance that humans get wiped from the Earth.

Just because of this one vehicle’s maiden voyage.)

Should we as a society reconsider this vehicle?

............

For my whole life, I’ve been the biggest fan of technology.

I love how it’s exponentially improving our lives.

It’s amazing to see advances in medicine, transportation, engineering, and everything else.

I’m the biggest cheerleader.

For many years, I’ve been eagerly awaiting the Singularity.

The moment when humans invent “the last invention necessary”.

A Super Artificial Intelligence.

Smarter than humans and able to invent anything else we want.

It will not just cure cancer.

It will even cure old age.

People cry “But it will take our jobs!”

We won’t NEED jobs.

It will eradicate poverty and bring massive abundance and luxury.

What’s not to love?

But I’m changing my mind about the race to invent a super AI.

People close to me will feel like I’m suddenly flip-flopping.

Why is Ryan now pumping the brakes?

It’s not because I don’t want a super AI to be invented.

I certainly do.

But I’ve been paying attention to a large number of the smartest people in the world.

They point out that humans only have ONE CHANCE to get this right.

And never in history have humans ever gotten anything right on the first try.

So what I want is for us as a society to force technology companies to slow down and be careful.

We’re facing what’s called “the alignment problem”.

Computers stubbornly do what their programmers instruct them to do.

This has been fine so far in history.

As a software engineer, I’m used to fixing bugs (mistakes) in code.

It’s always been an iterative process.

If I notice a mistake, I go back and change the code and try again.

The stakes are low.

But with the processing power that will soon be available with modern technology...

Engineers will have no chance to fix bugs in code.

Their programs already will have been executed on a planet-wide scale.

Some people mistakenly refer to “the alignment problem” as losing control.

But the creators of super AI won’t *want* to control it.

They’ll want its abilities to surpass our imaginations.

To some extent, there is POWER in giving up control.

My understanding of The Alignment Problem is that:

► We humans often DON’T KNOW exactly what we want.

and/or

► We DON’T KNOW HOW TO DESCRIBE what we want in a way that is impossible to misinterpret.

and/or

► WHAT WE WANT CHANGES over time.

Imagine if tech companies existed in the 1800s.

Their super AIs would be racist, sexist, etc (even more than today’s).

Luckily, society can make progress over time.

Values that were more popular in earlier times are not what most people value today.

We can expect that future humans will consider humans’ values in 2023 as barbaric and unforgivable.

A truly “aligned” superintelligence would somehow continually ask how it could serve humans in this moment.

And in this moment and in this moment.

But who gets to say?

It’s hard enough to decide one’s own values.

Impossible to agree as a human species.

And how will we be sure that the super AI is actually honoring our values rather than just faking it?

After all, AI has already surpassed human capabilities in many areas.

Soon it will surpass humans in the ability to deceive and persuade (if it hasn’t already).

Tech companies are currently power-hungry and profit-hungry and racing towards the edge of a cliff.

With all of humanity chained to them—we’ll share their fate.

It reminds me of a prisoner’s dilemma or tragedy of the commons or arms race.

Luckily, humans have proven that sometimes we can coordinate to avert disaster.

Although we haven’t eradicated nuclear weapons, we somehow limited them to just 9 countries instead of 200+.

And we haven’t seen a nuclear war.

We also seemed to slow down human cloning and gene editing so that we can be very thoughtful about it.

So there is a chance that we could not dive off this cliff together.

The top 4 guidelines for SAFELY building a super intelligence are:

1️⃣ Don’t teach it to code (and alter itself).

2️⃣ Don’t connect it to the internet.

3️⃣ Don’t teach it to manipulate humans.

4️⃣ Don’t create an API (don’t connect it to other capabilities).

If our tech companies would follow those 4 rules…

And take their time and solve the Alignment Problem…

Then maybe it *eventually* would make sense to relinquish control to the super AI.

⚠ Unfortunately, OpenAI ignored all 4 of those guidelines already when releasing ChatGPT.

And all other tech companies are trying hard to build a super AI too.

“We’re rushing towards a cliff, but the closer we get, the more scenic the views are.” 🤩

I understand the allure.

I use AI all day long.

Alexa, ChatGPT, Siri, Google, etc.

And it seems like we’ll all NEED to, just to keep up.

That’s fine.

But a *super* AI is a different story.

It's similar to a superintelligent alien species landing on Earth.

50% of AI researchers believe there’s a >=10% chance that humans will go extinct from our inability to control AI.

Read that again.

In a 2022 survey, AI experts were asked, “What probability do you put on human inability to control future advanced AI systems causing human extinction or similarly permanent and severe disempowerment of the human species?” The median reply was 10%.

I understand that it feels urgent to invent a super AI to help us cure cancer and old age, etc.

Slowing down has a cost.

But I agree with the smart folks at Center for Humane Technology and Future of Life Institute.

Let’s have more guardrails in place.

Otherwise, all of humanity can get obliterated by a super AI just because we didn’t take our time to solve the Alignment Problem first.

[Imagine the super AI proceeding on its merry way without considering (continually updated) human values…

With humans likely perishing as collateral damage.]

I don't like being a downer.

I much prefer spreading optimism.

But the stakes couldn't be higher.

We all need to be having this conversation.

See comments below for important links.

What are your thoughts?

I’m truly hoping that I’m wrong about humanity being in a perilous situation right now.

I’m curious if you think differently.

Or even if you agree with this concern, I’m curious what you recommend we should do.

Thanks for reading, and I'm looking forward to discussing with you (whether here online or in person).