I moved to the San Francisco bay area about five months ago. Earlier this week, the bay area experienced a pretty big earthquake.
As an aspiring rationalist, I keep an eye on my own reasoning processes, and this earthquake exposed a few problems.
First of all, my response to the earthquake was somewhat worrying. Picture this: it's 3AM, and I wake up to the house shaking. I'm briefly confused, but I soon remember that I'm California and California has earthquakes. Mystery solved! Satisfied, I shut out the sound of rattling windows, ignore the crashes as things fell in my room, and go back to sleep.
It seems like perhaps I should have treated the novel experience of the world trembling beneath me with a bit more respect. What the hell, hindbrain? I thought you had my back when it came to becoming appropriately terrified?
So I did a few mental exercises to fix that, and now I've hopefully updated my potential-disaster response patterns. I look forward to testing this next time there's an earthquake.
Which brings me to the second problem I identified as a result of the earthquake: I'm looking forward to the next one.
To make it obvious why this is a problem, remember that I identify as an effective altruist. I spend a lot of time trying to do large amounts of good. This mindset is somewhat inconsistent with hoping for earthquakes.
To make matters worse, the part of me that looks forward to the next earthquake wants it to be a big one. Ground rippling like a roller coaster, power outages that last for days if not weeks, roads torn asunder, the works. This part of me imagines strolling down the broken streets of Berkeley carrying food that they had to send in via helicopters because the roads got hit that hard.
This seems inconsistent with all my other preferences, given that I STRONGLY DISPREFER DISASTERS (hear that, Fate?). I would pay quite a bit of money, put in a whole lot of effort, and spend a large chunk of time in order to prevent a natural disaster from occuring.
This seems difficult to reconcile with the fact that I'd really enjoy a natural disaster in Berkeley (if, you know, I survived unscathed and all). What's going on here?
I don't know yet, but I have a blog now, and what good is a blog if you can't use it to debug your own little insanities in real-time? It's time to have a nice long conversation with myself, and then edit it to make me look cool and put it on the internet!
Follow along at home! How do you think I should resolve this discrepency? Bonus points if you actually write something down before continuing, so as to avoid hindsight bias.
(What follows is an edited version of an actual conversation that I'm about to have with myself. Well, I mean, it isn't edited yet, but it probably will be by the time you read this, unless something goes horribly wrong.)
Brain. What the hell? I disprefer disasters. How come I'd be excited to have another bigger meaner earthquake?
Well, I don't want an earthquake in the sense that I'd try to cause one.
Of course not.
I'd feel bad if one hit, obviously. And afterwards I'd experience impotent rage against Nature for all the deaths and all the damage, I'd mourn, I'd harden my resolve, and so on.
So then in what sense do you want one?
Well, all of the above doesn't change the fact that the aftermath of a natural disaster would be really fun.
I... don't even know where to start with you. You would work hard to avert a natural disaster, right?
Of course. But if one just so happened to come from a source completely outside of my control, then I'd definitely enjoy it. (After the crippling terror subsided, at least.)
Brain, come on. There's no such thing as "completely outside of my control." I live in a chaotic universe; the causal consequences of my actions reverberate through everything. "Things that I control" is a useful mental category, but it's not a fundamental distinction, and I'm not even sure what it means to want bad things to happen so long as they're independent from my control.
Imagine a gods-eye view of reality, where you get to choose the future. Are you saying you'd select a future where the earthquake happens regardless of what Nate-in-the-system does to prevent it?
Not really. It's more like "I would prefer to be put into that universe against my will". But insofar as it's me selecting which future occurs, of course I'm going to prefer futures without natural disasters.
...You do realize this is insane, right?
Eh.
Seriously, though. Imagine you're at the godly viewpoint and you get to select which future occurs but your mind is going to be wiped, do you —
Of course I don't cause the earthquake, dumbass.
What if I alter your mind enough that you're not quite you, but you share our preferences? Then what do you do?
Insofar as it's still me, I prevent the earthquake. Insofar as it's someone else, I don't control what they do, but I kinda hope that they send earthquakes.
AAARGH! You prefer worlds where the fates send earthquakes, even though you wouldn't send earthquakes yourself? What do you think "prefer" means?
I'm not sure that I'd "prefer" those worlds, per se. I'm just saying that I hope the fates chose earthquake.
You're insane.
How? This doesn't feel insane to me.
You hope that the universe is set up in a way you don't like?
Yep. And I'm in luck! We happen to live in a universe that is totally pants-on-head bonkers.
This is madness! What do you think "hope" means? What do you think "don't like" means? HOW CAN YOU HOPE TO BE IN A UNIVERSE YOU DON'T LIKE?
<shrug> I dunno. I didn't design me.
You do understand how dangerous earthquakes are, right? And that being in one that's as big as you say you want is extremely likely to result in your death?
Of course! That's one of the many reasons we wouldn't cause one, silly.
...
Look, I get that I don't grok my own mortality. I also understand that I'm bad at multiplying and wouldn't be able to comprehend the magnitude of the destruction. I'm not saying we should hesitate when it comes time to avert or avoid disaster. I'm just saying that despite all this, I'd still really enjoy the aftermath (assuming I survive unscathed). Come on, don't you think it sounds fun?
It... ok, yes. Though the disaster sounds horrible, the aftermath does sound kinda fun. A chance to break all the routines of daily life, to be drawn together in a community, to bond with others going through similar strife. I emphasize that this intuition completely ignores the gravity of the consequences, but yes, conditional on my unscathed survival I would enjoy the period after the disaster as everybody came together to rebuild the community.
Not just the period after. Don't try to weasel out of this. Surviving the disaster itself also seems thrilling. Yes, it would be terrifying in the moment, but you can't deny that it sounds fun.
Actually, I can: you probably underestimate the terror and overestimate the thrill of stressful situations.
Nevertheless, I am starting to see your point with regard to the aftermath. I mean, I don't want a natural disaster to occur, for the obvious reasons. Even if you could guarantee the safety of all people involved and could that the destruction would be perfectly compensated for by a windfall shortly afterwards, I'd still disprefer a natural disaster because I have other shit to do.
But... I admit that if it did happen, conditional upon me being unharmed, I'd enjoy the aftermath.
See? I'm not crazy.
Don't get cocky.
It does seem potentially consistent, though. Horrible things happen constantly throughout the world, but this doesn't mean I shouldn't ever enjoy myself. For similar reasons, it seems fine to expect a post-disaster scenario to be especially fun: it may be fun even though the situation is horrible; those feelings need not be contradictory.
But then why do I have this lingering feeling that I 'want' a natural disaster to occur? Could we better separate "I would enjoy the result" from "I want the cause", please?
Anticipations don't work like that. When I simulate a natural disaster, I do anticipate it being fun (after the initial terror and horror, of course). I'll do my best to make sure that doesn't bleed over into the action system, but in honesty, I think that's not a problem. We don't actually control earthquakes, remember?
Yet.
<nod> growth mindset.
But seriously, that's not a good enough answer. Generalizing this failure mode, I bet there are dozens of things that I would selfishly enjoy but which I don't want to happen, and I'm worried that some of them may slip past my attention. If I continue attaching positive feelings to bad things, I might not be vigilant enough in preventing bad things.
Wait, you're saying that "things which I would enjoy" is in the same bin as "things that I want to cause" in my head? WHY ARE THESE THINGS IN THE SAME BIN IN MY HEAD?
AAAAAAH I DON'T KNOW
Ok. Ok, don't worry. This can be fixed. Disasters themselves do feel like a bad thing, and visualizing them induces feelings of sorrow, frustation, impotence, and so on. We could attach those feelings to post-disaster settings.
That might solve this one case, but I want a general solution.
We've already spent time dissasociating "feels-good" from "motivates" (when becoming an effective altruist). Perhaps there's a way to extend that work and attach the bad feelings to the feel-good parts?
If that worked, it would compromise my ability to enjoy post-disaster scenarios. That doesn't seem like a good plan.
Actually, wait, that sounds like a really bad plan. If you widen your scope enough, most of the world is in a pretty bad state. (People still die against their will!) Preventing feel-good in the causal vicinity of actually-bad seems like a really dumb idea.
And besides, if I do find myself in a post-disaster scenario, it seems useful to be able to enjoy myself.
Well, the "right" solution seems to be building a better mental separation between scenarios that we enjoy and scenarios that we want to occur.
And suffer the risk that unnoticed positive anticipations for negative scenarios will twist my actions whenever my attention wavers.
Yep!
AAUAGH WHO EVEN INVENTED BRAINS
(Please note that the above has been translated from Nate-tongue into english. I promise you that my thoughts aren't this coherent in their natural environment.)
And that's how I feel about that!
I've largely concluded that "how much I would enjoy something" should be completely disconnected from "how much I like the situation", and that the former feeling should definitely not be allowed to steer my actions. I already have some mental triggers that help me steer towards actions that don't internally feel warm and fuzzy (e.g. to combat scope insensitivity) which I intend to repurpose in order to ensure that anticipations like "I would enjoy a post-disaster scenario if unscathed" don't run the risk of influencing my actions.
I'm still rather annoyed that my motivation system automatically conflates "would be fun" with "should be caused". These feelings often overlap, but would-be-funness should feel less and less relevant as the impact of the action gets bigger. Hear that, brain?
<sigh> it's not listening to me. I'll have to work on this.
Anyway, this has been your first irregular installment of "what goes on in Nate's head", the blog series that only occasionally harrows. Tune in next time!