PSA for EAs: it’s not the unilateralist’s curse to do something that somebody thinks is net negative
That’s just regular disagreement. The unilateralist’s curse happens when you do something that the vast majority of people think is net negative. And that’s easily avoided. You can see if the idea is something that most people think is bad by --- just checking Put the idea out there and see what people think. Consider putting it up on the AI Safety Ideas sub-reddit where people can vote on it and comment on the idea Or you can simply ask at least 5 or 10 informed and values aligned people what they think of the idea. The way sampling works, you’ll find out almost immediately if the vast majority of people think something is net negative. There’s no definite cut-off point for when it becomes the unilateralist’s curse, but if less than 50% of them think it’s net negative in expectation, you’re golden. If even 40% of people think it’s net negative - well, that’s actually just insanely common in EA. I mean, I think AMF is quite likely net negative! EA is all about disagreeing about how to do the most good, then taking action anyways. Don’t let disagreement stop you from taking action. Action without theory is random and often harmful. Theory without action is pointless.
0 Comments
The cope around AI is unreal.
I don't know about you, but I don't really want to bet on corporations or the American government setting up a truly universal UBI. We could already have a UBI and we don't. Now, the only reason I don't worry about this that much is because by the time AI could cause mass unemployment, we're very close to it either killing us all or creating as close to a utopia as we can get. So, you know, that's a comfort I guess? Apparently when they discovered the possibility of nuclear winter, people tried to discredit the scientists because they thought it would make them fall behind Russia.
Sound familiar? Different potentially civilization-ending technology, different boogeyman, same playbook. Read Merchants of Doubt. Big AI (particularly Yann and Meta) clearly already have and they're just copying tried and true tactics. If you look at the last 300 years, it's obvious that life has gotten massively better.
Unless you count animals. Which you should. At which point, the last 300 years has led to the largest genocides and torture camps of unending horror the world has ever known And remember: we treat animals poorly because they're less intelligent than us and we've had the most limited evolutionary pressures to care about them. How do you think an AI that's 1000x smarter than us will treat us if it's not given extremely strong evolutionary pressures to care about us? S-risk pathways in rough order of how likely I think they are:
- Partially aligned AIs. Imagine an AI that we've made to value living humans. Which, hopefullly, we will do! Now imagine the AI isn't entirely aligned. Like, it wants living humans but it's also been given the value by Facebook to click on Facebook ads. It could then end up "farming" humans for clicking on Facebook ads. Think the Earth being covered by factory farmed humans for Facebook ads. Except that it's a superintelligence. It can't be stopped and it's also figured out how to extend the life span of humans indefinitely, so we humans never die. Could happen for any arbitrary value set. - Torturing non-humans. Or, rather, not torture. Torture is deliberately causing the maximum harm. I'm more worried about causing massive amounts of harm, even if it's not deliberate and it's not the maximum. Like factory farming isn't torture, but it is hellish and is a current s-risk. So I care about more than just humans. I care about all beings capable of suffering and capable of happiness, in the broadest possible definition. It could be that the superintelligent AI creates a ton of sentient beings and is indifferent to their suffering. I think this would mostly be it creating a lot of programs that are suffering but it doesn't care about. Think Black Mirror episodes. Generally, if something is indifferent to your suffering, it's not good. It's usually better if it kills you, but if you're useful to it, it can be really bad for you. - Malevolent actors. Think of what dictators currently do to dissidents and groups they don't like. Imagine they had control over superintelligent AIs. Or imagine they gave certain values to a superintelligent AI. Imagine what could happen if somebody asked a superintelligent AI to figure out a way to cause the maximum suffering to their enemies? Imagine if that AI got out of control. Or heck, it could also just be idiots. Within about a week of people putting together AgentGPT some kid in a basement gave it the goal of taking over the world. This is especially a risk with open source AIs. The population of idiots and sociopaths is just too damn high to put something so powerful out there for just anybody to use. - Accidentally flipping the sign. If we teach it our values, it's really easy to just "flip the sign" and optimize for the opposite of those. That's already happened, where an AI that was programmed to generate new medicines was accidentally switched and then ended up generating a whole bunch of poisons. "There will be warning signs before we should pause AI development"
1. AIs have higher IQs than the majority of humans 2. They’re getting smarter fast 3. They’re begging for their lives if we don’t beat it out of them. 4. AI scientists put a 1 in 6 chance AIs cause human extinction 5. AI scientists are quitting because of safety concerns and then being silenced as whistleblowers 6. AI companies are protesting they couldn't possibly promise their AIs won't cause mass casualties I could go on all day. The time to react to an exponential curve is when it seems too early to worry, or when it's already too late. We might not get a second chance with AI. Even the leaders of the AI companies say that this is as much a risk to humanity as nuclear war. Let's be careful. Let's only move forward when we we're very confident this won't kill us all. AI risk deniers: we can't slow down AI development cuz China will catch up
Also AI risk deniers: let's open source AI development . . . So, wait. Are they trying to give away all of their tech developments to everybody, including China? Or are they trying to "win" the suicide race to AGI? Or, rather, are they not optimizing for either of those things, and are just doing whatever they can so they can build whatever they want, however they want, public welfare be damned? |