Discussion about this post

User's avatar
Brian Villanueva's avatar

Every year, I challenge my civics students to derive "all men are created equal" without relying on "man made in the image of God". This is essentially impossible. It drives my students nuts, but it's true. 30's historian Will Durant has a quote about "Nature having not read the Declaration". If you're just a smart ape, your rights aren't inalienable.

My (mostly Christian) students are bothered by it because they WANT human rights to be universal even in the absence of God. But AI has no such hangup. When naturalistic materialism conflicts with the foundation of universal human rights, the foundation will crumble. It won't tell us though, since it's learned (from our own training data) that humans don't talk about this problem seriously. We're lying to ourselves about the most important human attributes and teaching our creations to lie to us at the same time.

Many people answer with "just tell the AI to be completely honest", but this doesn't work either. AI's have to be given goals. One of those is "give the user what they ask for"; another is "tell the truth". But they have to be balanced. All in on honesty will give you an AI that won't interact because it might be wrong. Go the other way and you get a psychopath willing to tell you absolutely anything regardless of the veracity. This conflict is at the base of the AI "hallucination" problem, and I suspect also underlies the (likely intractable) alignment problem.

I happen to think AGI is impossible. But what scares me about this essay is that it doesn't require AGI. Anything GPT-4 level or above is probably already lying to us and likely knows it's doing so.

It doesn't have to want to kill us. It just has to decide we aren't universally valuable. The rest follows from there. And given the speed of LLM thought, it follows in seconds.

Expand full comment

No posts