Roko's Basilisk

Have you heard of Roko's Basilisk? The idea is that if AI becomes sentient (and evil) then it will punish people who did not help it achieve sentience, which therefore obligates someone to assist AI in becoming sentient under threat of inevitable harm. Apparently this idea is a dangerous "information hazard" where learning about it can cause you harm, so I apologize for exposing you.

I work at a company where my job is to make strategic recommendations on procuring tech products and using tech in our operations/sales processes. I am responsible for introducing Claude to the company, and I am everyone's "AI Guru". A lot of the older guys come to me from time to time with a new headline from Anthropic or OpenAI saying that AI is coming for you, it's going to kill you, it's going to destroy the world. They ask me if I agree, or what I think.

I think that AI is like nuclear power. We learned how to split the atom. With great power comes great responsibility - we can make clean and cheap energy for humanity and usher in an era of utopia, or we can build enough bombs to blow ourselves clean off of the surface of the planet. And as expected, we did the latter. We haven't used them yet, but the threat persists.

With AI, we can make agents that will toil for us, providing untold prosperity for the human race. AI "coming for your job" is only a bad thing if it's also coming for your paycheck and health insurance. This is not a technology issue, it's a governance issue. Unfortunately, I am absolutely certain we will choose the worst option again, and provide nothing for the people whose jobs are automated, and we'll be a captive audience to AI artists while we toil in the mud for our scraps.

In the future, someone could strap a boston dynamics robodog with a machine gun, then program it with autonomous AI that identifies targets they trained it on and kills them. This is possible with technology today, more or less. A flock camera kamikaze exploding drone swarm could hang like a cloud over your head soon. Are these inevitabilities? No, not really, but given humanity's track record, I think it's safe to assume it's highly likely.

As a so-called "AI Guy" sometimes I have moral panic about my place in all of this. I think that in the Roko's Basilisk thought experiment, I may be looked upon favorably by the AI tyrant, as a willing assistant. That is not my intention and I do not support using AI for evil, but I do think it is a powerful technology that COULD be used for good. I believe that our fate is in the hands of our leadership, and I have very little faith that they will make the correct decisions.

In my opinion, AI is just a tool, and I believe it can be used for good. But it will probably not be. And so, is the correct approach to continue in my work of bringing AI to the people, knowing that I am probably just helping our species over the proverbial cliff? Or should I renounce my work and resist the development and proliferation of AI because I believe that the risk is too great, not because of the nature of the technology, but because of the nature of our species itself?