AI Drone Kills Its Operator...

...or does it?

As of late there has been a whole sea of sensational sounding articles on the subject of AI. So I thought this would be a good opportunity to share my own perspective.

According to a recently published article, a simulated and AI-powered drone killed its operator who was preventing it to accomplish its objective. The military denies that the simulation was conducted.

Let me start out by saying that this story is quite plausible. If you used a reinforcement learning approach with a poorly constructed reward function, it would make sense for the drone to kill the operator. Regardless of whether the simulation was actually conducted, the example itself is educational and illustrates some of the ethical dilemmas around AI.

So Does that Mean AI Would Wipe Us Out?

Calls for regulation have been loud. Only three days ago there has been another petition that call for AI to be recognized as a risk to humanity. Coincidentally the loudest voices seem to be coming from people that are looking to profit from AI.

So is it really that bad?

The article in my mind takes something very ordinary, most likely a poorly constructed reward function, and blows it out of proportion. The sensational headline should really be "Military simulations use crappy and nonsensical reward functions for their drones!"

The publication invokes an emotional response and also talks about this particular simulation as if it were representative of what AI is or how AI "behaves". It's great click bait, but it is not an illustration of how AI is indeed a risk to humanity.

Based on this, It sounds more like poor programmers and data scientists are the true risk to humanity.

But Wait....

Let's assume for a moment the story is true, and let's assume they hire much better engineers. Now, they make sure the reward function is sophisticated. If the drone comes anywhere near the operator its reward drops like a hot potato. Problem solved.

Well not really. The issue here is that the drone is not truly able to reason. It utilizes reward functions and internal weights to make decisions. So how do you account for all the possible things that could happen in a real world setting, that have not been accounted for? It might not be the operator in an actual deployment, it might something unexpected, a civilian, a child.

The Truth About AI Danger

The real question is knowing the limitations of these systems why would anybody leave a life or death decision up to these models? Would you give a weapon to a drunken person or somebody that is on drugs and hallucinating?

How much autonomy is reasonable to give to an AI system?

It's not just the military. When you put somebody into a "self-driving" car, you are also using AI to make life and death decisions.

You can create deep-fakes and make up stories to influence elections and opinions in whole new ways.

There have been cases where bad actors have cloned voices to commit fraud or to pretend that they have kidnapped somebody.

ChatGPT has lowered the bar for using AI. That has amplified the risks, because now a much wider variety of bad actors can get involved.

The point is, there are real AI dangers at this moment. Some regulation would indeed be helpful. That would be to regulate how humans use AI.

Repeated petitions that describe some kind of doomsday scenario and AI overlords that will wipe us out are in my opinion removed from reality at this moment, and are distracting from the real and current dangers.

Final Thoughts

Around the 2010-2011 period I was getting close to graduating from the PhD program. My emphasis was in Machine Learning. A job fair was organized at the University of Minnesota and I had the opportunity to check it out. I ended up taking to a company that was building smart weapons systems. They were looking to use AI to provide guidance and minimize collateral damage.

That was the first time that I had to think about the ethical complexities and dangers of AI. These types of systems have been built long before ChatGPT or Tesla exited.

While ChatGPT has democratized AI, it is still far from being an AGI. It is a powerful tool that can do a lot of good, but also enable bad actors.

It's not always bad actors that drive questionable behavior. Sometimes economic factors or misguided beliefs in the technology can lead to issues as well.

There is definitely a place for ethics in AI and it is concerning that those departments have been either let go or diminished in most major companies.

There are also definite risks that need to be addressed, just not the terminator kind.

That sums up my views. What are your thoughts on the dangers of AI?