The Incredibles 2, and How the Universe is Allowed to Just Kill You Anyway

The Incredibles 2 is a movie about superheroes, which is the sequel to another movie about superheroes. Both are centrally themed around the idea that “no man is an island” – as in, you aren’t alone, you don’t need to be alone, and in fact, you do better when you let others help you – to the point that “Nomanisan Island” is an actual location in the films.

I watched the first Incredibles movie when I was a child. It was good, but it didn’t leave any lasting impression in my young brain beyond “Elastigirl cool”. I thought this might be because I’d seen it before I was sentient, so I watched it again later. I liked it more than I had when I was young, but it still didn’t hammer its central theme into my brain nearly as effectively as its sequel.

There are three main things that made the sequel much better than the original; at least, three that are particularly poignant to me. First, the stakes are meaningful. At the end of the first movie, if the good guys didn’t win, Syndrome would have “made everybody super, so no one is”, whatever that means. At the end of the sequel, if the good guys didn’t win, a gigantic cruise ship would have crashed into a coastal city, killing hundreds or thousands of people. You can imagine which of these is more emotionally moving to me.

Second: partway through the second act, Helen meets a number of other superheroes who have been in hiding. This introduces an important element the first movie lacks: a supporting cast. It fleshes out the group “superheroes” to see more than six, and it shows us the sheer number of people whose lives have been negatively impacted by the outlawing of supers.

This, and a few other scenes, make it clear that we, the audience, are expected to care about people besides the main cast. A lot of movies just take collateral damage in stride, telling the audience not to think about the fact that the good guys let hundreds of unnamed pedestrians die when they crash a bad guy’s helicopter into a building. In Incredibles movies, this comes from a plot focus on minimization of collateral damage from superheroes, but it resonates nicely with the humanist in me.

Third: Because the Incredibles movies feature a family, there have been some critical parts of the plot featuring the children as central characters. But while in the first movie they mostly just held their own, in the sequel they were able to independently progress the plot. In fact, there is a point in the climax where all the adults have been hypnotized by the villain, and it’s up to the children – who have so far been bickering, uncoordinated, and generally unqualified to accomplish this necessary task – to save the day.

They succeed, of course, because this is a family-friendly movie. But that sequence of events produced a genuine feeling of uncertainty about the outcome that most movies struggle for. It captured, for me at least, the precise feeling I have when thinking about global catastrophic risks.

The definition of a global catastrophic risk (GCR) is “a hypothetical future event which could damage human well-being on a global scale”. A sub-type of GCR is existential risks, which include things like non-value-aligned artificial superintelligences. Existential risks are the ones which would cause human extinction.

There are tangible, obvious, salient stakes when talking about GCRs. We discuss them because we care about our fellow humans, and we want them not to suffer or die. But, at the same time, we are in a horribly unfortunate position in terms of actually preventing any of these risks, because we’re all bickering, uncoordinated, and generally unqualified to accomplish this necessary task.

At the same time that people are still working on developing ASI, still pumping massive amounts of greenhouse gases into the atmosphere, still stockpiling nuclear weapons, the rest of us are all wandering about our petty lives, not realizing the actions of these few people might imminently kill us all. We can’t affect the actions of these groups, just like we can’t affect the orbit of the sun – as in, it’s strictly-speaking possible, but extraordinarily difficult.

So we’re stuck between extinction and impossibility. Either accept the greater-than-50% likelihood of a universe tiled with paperclips, or move the sun.

Unlike in The Incredibles movies, real life is not family-friendly. There is no plot armor protecting us from extinction, no reason that the squabbling children should be able to defeat the villain. If we are going to survive, we need to become much better.

What “better” looks like depends on the risk, and there are a lot of them. We all should educate ourselves on what GCRs are, what they look like, how bad each one could be, and what preventative measures could be taken to make us safer. In instances where the powers that be are likely to listen to us, we should rally, and scream loudly enough that they’re forced to listen. And, lastly, in the specific situations where we ourselves are the powers that be – if we are AI programmers or molecular nanotechnology developers or biotechnologists – we need to think long and hard about our decisions.

Do not underestimate the likelihood of a future where someone says “oops”, and seven billion bodies hit the ground.

My Problem with “Women”

I have no trouble with being a woman, any more than I have trouble with being a human in general. I do dislike some parts of that, to be fair: continuous sensory experiences I can’t turn off, a body that needs ongoing maintenance, and the nonexistence of any owner’s manual for the whole system. But these would be the same if I were a man instead, and so short of being uploaded, there’s not a lot I can do about it.

However, being called a “woman” – being referred to as such in any social context – is uncomfortable. I’m aware that the biological definition of the word “woman” does apply to me: so far as I know, I can bear children. But if all people meant by the word “woman” was “an adult human of the type which ovulates and can bear children”, I wouldn’t be uncomfortable with being called that. (I also have no clue why anyone would bother, outside the context of asking to have children with me.)

Humans love to sneak things into definitions that are not actually present in the best definition. We basically never use “woman” to mean what it should actually mean: a set of commonly co-occurring biological traits. Instead, it conveys a whole host of mostly inaccurate, often outdated, sometimes demeaning, stereotypes that have been thrust upon us childbearing-capable humans by various cultures throughout history.

What about ovulating produces poor driving skills? What about a uterus produces a lack of self-confidence? Does a vagina induce a desire to cook, clean, and take care of children? Does chromosome set XX create gracefulness, thinness, or a preference for dresses?

But, of course, these are only stereotypes. While they are annoying, it shouldn’t make me uncomfortable with being called a “woman” just because some idiot thinks women can’t drive.

The problem is, it isn’t only the stereotypes. As social psychologists learn to better isolate relevant variables, studies have been published about the differences between the sexes. Evidently, women are more likely to prioritize work flexibility and job stability over earnings growth. They prefer socially interesting, rather than mechanically interesting, jobs. They’re physically weaker than men.

The inherent trouble with this is segmenting an entire population into two groups, and having the definitions of those groups be any more complex than a single trait. If you assume all women are graceful housewives with poor driving skills and self-confidence issues, or if you assume all women are family-focused social butterflies with a chronic avoidance of lifting anything heavy, then you’ll be wrong no matter whether your sources are stereotypes or science.

Nobody is an average. There is no such thing as “the average woman” or “the average man”. These aren’t people; they don’t exist. When a social psychologist says “the average woman prefers work flexibility to higher salary, prefers socially interesting over mechanically interesting work, and is weaker than the average man”, they aren’t saying that every woman is like this, or that there even is a single woman who is all these things. They are merely pointing out a pattern.

The trouble pointing out patterns is that, in any sufficiently large population, there are always patterns. It is always possible to find traits that are more likely to occur in one group than another, even with statistically significance. If I analyzed a large enough population, I’m sure I could find statistically significant correlations between someone’s personality and their hair color, skin tone, or preference for window coverings.

And even if the patterns do reflect real differences, they still don’t tell you anything about any specific individual. There are statistical differences between age of mortality, for example – women tend to live longer than men – but that doesn’t mean that any given woman will certainly outlive any same-aged man.

The implications of all this pattern-finding, though, is that people who are outside the pattern often don’t get to do outside-the-pattern things they enjoy. For example, I like helping people by doing physically-demanding tasks for them (lifting or carrying heavy objects, opening tight jar lids, etc), but nobody ever asks me to do this because they don’t think of me as the sort of person who would enjoy that type of thing.

And then everyone – especially intellectual and “progressive” folks – decides to bring my womanhood into every conversation. I can’t just be strong, I have to be a “strong woman”. I can’t just be an engineer, I have to be a “woman engineer”. I can’t just be primarily attracted to women, I have to be a “lesbian”.

Keep in mind: I wouldn’t feel any less strongly about any of this if I had been born a man. I could write a whole list of arguments rebutting the stereotypes and tendencies associated with men that I don’t follow, if those stereotypes and tendencies had been the ones to shape my social life.

None of this has anything to do with womanhood; it has to do with gender.

I can understand why gender would be a useful thing to consider, in the primitive societies where language was being developed. I can understand why it would continue to be useful throughout the majority of human history to have gendered words like “woman” and “man” used more frequently than “person”: when your gender determines everything from your clothing choices to your career options, differentiating between genders while talking is sensible, perhaps even necessary. But in modern society, where we’re trying to do away with gender roles, I cannot imagine a reason to continue using gendered words.

It isn’t only the fact that everyone’s gender doesn’t necessarily match their birth sex, although this is true. It isn’t only the fact that not everyone’s gender fits into the Western binary system, although this is true also. It isn’t even about the fact that, if you assume someone’s gender and you’re wrong, it can be anywhere from a minor inconvenience to an immensely traumatizing experience, and you have no idea which.

It’s about the fact that assuming peoples’ gender based on their gender presentation actively entrenches gender roles and stereotypes.

If, in trying to accept transgender people, we go from an attitude of “anyone with breasts is a woman and should be called by she/her pronouns” to one of “anyone wearing stereotypically feminine clothing is a woman and should be called by she/her pronouns”, that isn’t progress. It’s moving backwards. Because, in the act of moving from the one to the other, we get the worst possible mix of both. In order to “pass” as a woman, a trans woman needs to embrace every bit of stereotypical femininity she can. She puts on makeup. She shaves her legs. She paints her nails. Even her actions aren’t free from scrutiny: if she doesn’t keep her knees together while sitting down, speak in a high-pitched quiet vocal register, and apologize constantly, how will people know she’s a woman?

The appropriate thing to do here is definitely not to get upset with transgender people. Passing is necessary for their survival in a cisgender-dominated world: it isn’t their fault that this necessity perpetuates gender roles. The best thing to do, instead, is to abolish gendered words altogether.

This might seem daunting, but you can implement it yourself easily. Just refer to anybody whose gender you don’t know – including hypothetical people and strangers – with they/them pronouns. When describing someone, say “person” instead of “woman” or “man”.

Of course, in the modern world, some conversations are explicitly about gender, in which case, gendered words are inevitable. You can’t restate phrases like “women tend to make less money than men” into gender-neutral words and have them still make sense. But the majority of conversations, containing phrases like “I saw a lady at the bus stop today, she was wearing this gorgeous purple suit”, or “when I was driving home yesterday this old guy was tailgating me the whole time”, can be restated with gender-neutral words easily.

In conclusion: I don’t want to be called a woman because gendered words carry uncomfortable stereotypes and then perpetuate them by their continued use. Instead, I prefer to be referred to with they/them pronouns and gender-neutral words.

In the interest of moving society past the pervasiveness of gender, I invite you to consider the same.

Rationality is Generalizable

In order to make some extra money during this pandemic, I’ve been doing a bit of work fixing up one of my mom’s rental houses. All the work is within my preexisting skillset, and it’s pretty nice to have a physical job to give me a break from my mental work.

When fixing up a house between tenants, the first item on the to-do list is to create the to-do list, called a “punch list”. In order to create it, I walked through the house and noted down the problems: this railing is loose, this stair tread is broken, these tiles are cracked, etc. Once I was finished, I’d made a list that would probably take me about a week to complete.

And then I tripled that time, and sent it to my mom as my time estimate.

The reason I did this was in order to combat something known as the planning fallacy, one of many endemic flaws in the human brain.

When humans make plans, they envision a scenario where nothing goes unexpectedly. But reality doesn’t work the way human-brains-making-plans do. Fixing those cracked tiles ends up requiring ripping out eight layers of rotted wood underneath filling in the resulting two-inch-deep gap with concrete, then leveling it out with plywood before laying the new tiles. Repairing the crack in the living room wall ends up requiring replacing the whole gutter system, which was causing water to run through the bricks on the outside and into the drywall. When we just see some cracked tiles or some chipping paint, we don’t imagine the root problem that might need to be fixed: we just consider replacing a few tiles or repairing a bit of cracked wall.

This generalizes far beyond fixing houses. When a group of students were asked for estimates for when they thought they would complete their personal academic projects,

  • 13% of subjects finished their project by the time they had assigned a 50% probability level;
  • 19% finished by the time assigned a 75% probability level;
  • and only 45% finished by the time of their 99% probability level.

As Buehler et al. wrote, “The results for the 99% probability level are especially striking: Even when asked to make a highly conservative forecast, a prediction that they felt virtually certain that they would fulfill, students’ confidence in their time estimates far exceeded their accomplishments.”

Humans planning things envision everything going according to their plan, with no unforeseen delays: exactly the same as if you ask them for a best-case estimate, in fact. In real life, what happens is somewhat worse than the worst-case estimate.

There are some useful debiasing techniques for combating the planning fallacy. The most useful of these is to utilize the outside view instead of the inside view: to consider how similar projects have gone in the past, instead of considering all the specific details of this particular project. (Considering specific details drives up inaccuracy.)

I’ve used this technique often, but in this particular circumstance, I couldn’t. While I’d done every individual thing that would need to be done to finish this house before, I had never done all of them in sequence.

Considering this problem, you might advise me to ask someone who had done whole houses before. I have easy access to such a person, in fact. The issue with this solution is that this person always makes overly optimistic estimates for how long it’s going to take to complete projects.

You would think her experience would make it easier for her to take the outside view, to consider this house in the context of all the other houses she had fixed. This doesn’t happen, so what gives?

Roy et al. propose a reason: “People base predictions of future duration on their memories of how long past events have taken, but these memories are systematic underestimates of past duration. People appear to underestimate future event duration because they underestimate past event duration.”

In light of all this, my best course of action, whenever I cannot take an outside view myself, is to take my normal (= optimistic) estimate and triple it.

I made that three-week time estimate around the first of June. Today is the 16th, and I’m finishing the last of the work today.

This whole situation, like the planning fallacy in particular, is generalizable. Learning about cognitive psychology, and the persistent flaws in all human brains, is oftentimes more useful for coming up with correct answers than experience.

You might not think that the tactics for “how to not be stupid” would be as generalizable to every field as they are. Each field has its own tips and tricks, its own separate toolbox, carpentry is not like neurosurgery is not like musical composition… But we are all human brains, us carpenters and neurosurgeons and composers, and we are all using the same flawed circuitry generated by the same optimization process. Here, as in planning, the special reasons why each field is different detract from an accurate conclusion.

Knowing about one cognitive bias produced a better time estimate than thirty years of experience in the field. Rationality does not always produce such a large improvement, but it does prevent humans from making the types of stupid mistakes we are prone to. In my personal opinion, if a decision is worth making, it is worth making rationally.

The Problem With Statements of the Form “X is Y”

Cats are mammals.

Gravity is the curvature of spacetime.

Both of these sentences are true. And they both use conjugations of the verb “to be”. But the usage of that verb is very different.

The first sentence is true in the sense that we as humans have defined a category called “mammals”, members of which have certain characteristics, and cats have the required characteristics to put them into that category.

The second sentence is true in the sense that “the curvature of spacetime” describes a thing that exists in the universe, to which we have given the name “gravity”. It’s a literal definition, not a taxonomical categorization.

The problem with these two different uses of “to be” is that people mistake the one for the other. They think that when scientists reclassified Pluto as a dwarf planet, it was because they’d learned something new about Pluto, as opposed to having simply revised the definition of a “planet”. But this is incorrect. A planet is whatever humans say a planet is, because the entire purpose of the category is to allow us to talk about “planets” without having to say “celestial bodies that have assumed a roughly spherical shape, orbit a star, and have cleared the space around their orbits” every time.

This isn’t to say that categories are meaningless, or can be defined in any way we please. Because a category is a convenient way of referring to a set of traits, those traits have to be common enough as a complete set to be worth referring to. We shouldn’t define a category to mean “a person with black hair and green eyes”, because those two traits aren’t any more likely to occur together than any other hair/eye color combination, and that set of traits doesn’t imply anything else useful about the person.

It is to say, however, that a category is not a fact about the universe, and that a category can and should be changed to better suit our purposes. Take, for example, the categories “male” and “female”. Most of us are accustomed to using these categories to imply a common combination of gender presentation, chromosomes, and sex characteristics. However, a lot of people have recently been changing these categories. In doing so, they aren’t violating any universal law that says “all masculine-presenting humans must have XY chromosomes”, because there is no such universal law. They are redefining a category that was defined in the first place by English-speaking humans and can be redefined by those same humans.

(Another recent change to categories: birds are reptiles.)

This same problem occurs the other way around: people mistake observations about the universe for human-made categories. I learned to say the phrase “gravity is the curvature of spacetime” when I was 9. It successfully impressed a lot of grownups, but the image in my head was a stretched-out tarp with a ball in the middle. It took nearly a decade for me to actually understand enough physics and math to grasp its real meaning. And when I did, my thought was basically “oh shit, gravity literally is the curvature of spacetime”.

When we talk about facts we understand, we’re not saying riddles or passwords. If somebody who doesn’t have the relevant background knowledge to understand us overhears, they might not realize that what we say is a literal observable fact, which they too can observe if they know where to look, but that doesn’t change it. The unfortunate reality is simply that the conversion from thought to language is not lossless. And until we invent telepathy, it probably never will be. Still, simply knowing the distinction between these two definitions of “X is Y” type sentences was helpful to me, and so I’m passing it on.

Ditch Pros and Cons: Use a Utility Function

If you’ve ever met me in person, you know I talk a lot about utility. My close friends are used to answering questions like “does this have net positive utility to you?” and “is that a strongly weighted component of your utility function?”. Every time I make a decision – what to do with my evening, or what to do with my life – I think about it in terms of utility.

I didn’t always do this, but I’ve adopted this way of thinking because it forces me to clarify everything that’s going on in my head and weight all my thoughts appropriately before making a decision. When I decide things this way, I genuinely feel like I’ve made the best possible choice I could, given everything I knew at the time.

What on earth is utility?

Utility is just a fancy word for “value”. If you enjoy chocolate ice cream, then eating chocolate ice cream has positive utility. If you don’t enjoy carrot cake, then eating carrot cake has negative utility.

One action can have multiple kinds of utility. You can add together all the utility types to get the action’s net utility. For example, if I assign positive utility to eating ice cream but a negative utility to gaining weight, there will be a certain optimal point where I eat as much ice cream as I can without gaining weight. Maybe, if I assign eating ice cream +5 utility, not gaining weight +5 utility, and exercising -5 utility, then it would make sense for me to hit the gym more often so that I can eat more ice cream without gaining weight.

The set of utilities I assign to all outcomes also tells me the optimal possible outcomes, with the highest net utility. In this example, that would be either modifying ice cream so that it doesn’t make me gain weight, or modifying my body’s energy processing system to get it to process ice cream without storing any of it as fat.

Having numbers that are consistent is helpful sometimes, but isn’t strictly necessary. When I need to make quick, relatively straightforward decisions, I typically just make up some utility numbers. Utility calculations in a small isolated system are basically matters of ratios: it doesn’t matter exactly how much utility I assign to something, but if outcome X has 5 more utility points than outcome Y, X is preferable.

Forcing yourself to make up numbers and compare them to each other reveals what you care about. If you initially thought you didn’t care much about something, but then realize that if you calculated net utility with a low number assigned to that thing, you’d be unsatisfied with the result, then you care more than you thought you did.

It might be somewhat unclear, with my super-simple examples so far, what you can assign utility to. So, here are some examples of things that I assign positive utility to:

  • Reading good books
  • Doing new things
  • Increasing utility according to the utility functions of people I care about
  • Building neat software
  • Drawing and painting
  • Writing stories and blog posts
  • Improving/maintaining my mental and physical health
  • Having interesting conversations
  • Improving the quality of life of all sentient beings
  • Running
  • Riding my bike
  • Taking walks with my girlfriend
  • Eating ice cream

If you enjoy doing it, if you think you should do it, if it makes you happy, if it’s a goal you have for some reason, or anything else like that, you assign it some amount of positive utility.

If you’d like to figure out how much relative utility you assign to different options, compare them: if I had to either give up on improving the quality of life for all sentient beings, or give up on ice cream, the ice cream has gotta go.

You can even assign positive utility to things you don’t end up doing. That’s because the net utility, after accounting for circumstances or mutually exclusive alternate actions. Knowing that you would do something, barring XYZ condition, is a useful thing to know in order to dissect your own thoughts, feelings, goals, and motivations. The converse is true, too: you can assign negative utility to things you end up doing anyway, because the net utility is positive.

So if that’s utility, what’s a utility function?

A utility function is a comprehensive set of everything that an agent assigns any utility to. “An agent” is anything capable of making decisions: a human, an AI, a sentient nonhuman alien, etc. Your utility function is the set of everything you care about.

The inputs to a utility function are quantities of certain outcomes, each of which are multiplied by their respectively-assigned utility value and then added together to get the total expected utility of a given course of action. In an equation, this is:

Ax + By + Cz + ...

Where A, B, C, and so on are individual facets of outcomes, and x, y, z, and so on are utilities.

Say I’m with my sister and we’re going to get food. I personally assign a strong net positive to getting burgers and a weak net negative for anything else. I also assign a positive utility to making my sister happy, regardless of where we go for food. If she has a strong net negative for getting burgers, and a weak net positive for sushi, I can evaluate that situation in my utility function and decide that my desire to make her happy overpowers the weak negative I have for anything besides burgers, so we go get sushi.

When evaluating more complex situations (such as moving to a job across the country, where positives include career advancement and increased income, and negatives include having to leave your home and make new friends), modeling your own utility function is an excellent way to parse out all the feelings that come from a choice like that. It’s better than a simple list of pros and cons because you have (numeric, if you like) weights for all the relevant actions.

How to use your utility function

I don’t keep my entire utility function in my head at one time. I’ve never even written it down. But I make sure I understand large swaths of it, compartmentalized to situations I often find myself in. However, if you decide to actually write down your utility values, proceed to make them all consistent, and actually calculate utility when you make decisions, there’s nothing stopping you.

In terms of the optimal way to think about utility calculations, I have one piece of advice. If you come out of a utility calculation thinking “gotcha, I can do this”, “alright, this seems reasonable”, or even “ugh, okay, I don’t like it but this is the best option”, then that’s good. That’s the utility function doing its job. But, if you come out of one thinking “hmmm… I guess, but what about XYZ contingency? I really don’t want to do ABC…”, or otherwise lingering on the point of decision, then you’ve forgotten something.

Go back and ask “what’s wrong with the ‘optimal’ outcome?”. It might be something you don’t want to admit to yourself, but you don’t gain anything by having an inaccurate perception of your own utility function. Remember that, in absence of a verbal reason, “I don’t wanna” is still a perfectly valid justification for assigning negative utility to an action or outcome. In order for this process to work, you need to parse out your desires/feelings/goals from your actions, without beating yourself up for it. Your utility function already is what it is, and owning up to it doesn’t make it worse.

Once you have a pretty good handle on your own utility function, you can go ahead and mentally model other peoples’. Humans are calculating utility all the time in the form of preferences and vague intuitions, so even if other people don’t know their utility functions, you can learn them by a combination of watching their actions and listening to their words.

The discrepancy between those two, by the way, is indicative of one of two things: either the person is choosing an action with suboptimal utility, or they don’t actually assign utility to the things they say they do aloud (perhaps for social reasons). You can point out this discrepancy politely, and perhaps help them to make better decisions in the future.

Once you begin to use utility functions for both yourself and others, you might be surprised at how much easier it is to make decisions. When considering possible courses of action for yourself, you’ll be able to choose the best option and know it was the best. And, in a group, having an accurate model of other peoples’ utility functions can let you account for their preferences, perhaps even better than they themselves do.

How to Be Happy

Growing up, my mom used to tell my siblings and I that when we were upset and didn’t want to be, we could choose to be happy instead. The whole concept seemed ridiculous to me. “I can’t just flip my emotions on and off like a light switch,” I remember telling her.

But the problem was, she was right. It’s entirely possible to “flip your emotions on and off like a light switch”. There’s a lot of research backing up that statement—not surprising, my mom graduated with a Master’s in psychology, I should have known she didn’t pull this idea out of nowhere. Further, though it took me longer than I would care to admit, I did personally realize the wisdom in her two-word advice, “choose happy”.

Many experiments show that if you smile, you’ll feel happier. It’s not even entirely about the conscious decision to feel happy – merely moving your facial muscles or even forcing a smile using chopsticks can do the trick. Your brain just has an association between smiles and happiness and so smiling can make you happy.

So the research says. But I doubted it. For years, until I realized the truth of it independently. Today, I’m going to dissect the reasons I doubted it, because I feel many people probably have the same doubts when reading articles like this one.

I had two reasons to doubt “choose happy”. The first was that I was afraid people would look at me weird if I went from crying to laughing in the span of less than two minutes. The unaltered procession of human emotions is a slow ebb and flow, and a drastic change would make people ask uncomfortable questions.

They probably would have done that. But I wish someone had told me that there are things much more important in life than seeming strange. Spending a majority of my time feeling depressed and anxious for no reason was dramatically worse than it would have been to have some people think I was odd. I should have weighed the pros and cons of feeling the emotion versus letting it go.

The second reason I doubted the wisdom of “choose happy” was that I thought all emotions were important. I thought that they were always there for a reason, even if I couldn’t find what that reason was. It was a gradual realization that led me to the simple fact that some emotions don’t make sense – they’re the result of hormonal imbalances, meaningless stressors, mental overstimulation, and many other things which don’t need to be dwelled on.

Nowadays, I think about emotions in the context of net utility. Is feeling this emotion useful to me? If I’m feeling embarrassed about a stupid mistake, that feeling can be useful, to prompt me to fix the mistake immediately. But after I’ve done everything I can to fix the mistake, including making the appropriate social reparations, I can let the emotion go, because it’s served its purpose. Continuing to feel embarrassed even when I can no longer do anything about the mistake, including learn from it, is pointless.

And if the emotion didn’t have any purpose to begin with – say, if I’m feeling angry because I’ve had a long difficult day at work, which is not even slightly connected to any particular problem that can be solved – I can analyze the cause, decide it’s pointless, and let go of the emotion.

How do you let go of emotions? After your brain stops intuitively holding on because it thinks they’re important, or that it would be weird to let go, it’s typically as simple as focusing on something else. If just passively thinking about something else doesn’t completely fix it, try smiling, putting on a fun or silly song, deliberately focusing on happy thoughts, or even closing your eyes and imagining a pleasant location to hang out for a while. (I’m deliberately giving advice that doesn’t require getting up, because I personally don’t like advice that says “get up! stretch! jog! sweat!” – it does genuinely work, but it’s always delivered in a very pushy way. That being said, if you haven’t already heard this advice from a hundred thousand people, being outside and/or exercising does in fact make you healthier and happier, so try it if you feel inclined.)

So the list of question to ask when you feel any emotion is:

  1. What emotion is it? Is that really what I’m feeling? Emotions are frequently very transparent, but they can become tangled. Further, some emotions can mask others: a lot of men have a tendency to express anger when they’re truly sad, for example. If your emotions are unclear, sort them out.
  2. What probably caused this emotion? Go over salient events in your mind and find the proximate cause. It doesn’t have to be anything major and it frequently isn’t. You’re looking for a cause, not a good reason.
  3. Does this emotion have net positive utility? Feeling negative emotions has inherent negative utility, but that may be outweighed by the positive utility of the action it makes you take: learning from a mistake, apologizing for a misdeed, fixing an internal or external problem, etc. Figure out if the emotion is prompting you to do anything useful, and if it isn’t, if you really need to keep it. Compute the net utility.
  4. An important note about these utility evaluations: A common trap I’ve seen many people fall into is where they keep a negative emotion around because they believe it prompts them to do something good which, in fact, they would do anyway. In particular, a lot of high-achievers end up with the misconception that being miserable is what prompts them to achieve things, when in fact, they would achieve more if they were happier. Therefore, strongly doubt any utility evaluation that leads you to the belief that you need to be miserable all the time in order to get things done.
  5. If you determined that the emotion has net positive utility, keep it around, but only as long as it continues to be useful. As soon as you’ve done everything useful that the emotion was prompting you to do, throw it away. There is no reason to be miserable longer than necessary.
  6. If you determined that the emotion has net negative utility, toss it immediately, using any of the tricks described above.

A final note about the utility of positive emotions: feeling good is a good thing. I’ve seen people be happy but wonder whether they really should be feeling happy. You can dissect the emotion and what actions it makes you take to figure this out, but don’t decide you need to be unhappy because it’s uncommon to see sane adults who visibly care about anything. Emotions are good to keep if they’re useful, and being happy uniformly makes your life better, so ceteris paribus, happiness is useful, and therefore, happiness is almost always good to keep.

In conclusion:
Choose Happy.

Why Science?

I think that, if I hadn’t grown up hearing casually about their existence, I would be utterly floored by a lot of miscellaneous facts about the modern world. Humans have set foot on the moon, for one example. Y’know, that little white dot you see at night, that’s much larger than all the other white dots but otherwise still seems pasted onto that blue-black dome of the sky? That’s a place. Humans have been there. Ascended into the heavens on towers of fire, to go where no man has gone before. If you’d waited to tell me about it until I was sentient enough to understand its magnitude, I would have screamed.

I’m not advocating that we stop teaching young children about the moon landing, but I do find it curious that we grow up jaded to all of this.

What you grow up with feels like the way the world has always been. Unless you make a deliberate effort to remember history, you will not realize the smallness of your exact state of partial knowledge about the universe, wherein people have walked on the moon but you don’t yet know how consciousness works. This is a tiny, tiny, infinitesimal slice of history that you are living in right now.

But people don’t realize this, and so it seems like their tiny slice of history is the way the world has always been, instead of being the product of hundreds and thousands of years of human ingenuity, building and building upon itself, increasing exponentially with the advent of the scientific method.

If walking on the moon is a product of Science, and the last thousand discoveries have been the products of Science, it follows that the next thousand discoveries will be likewise, assuming we haven’t found something better by then. This isn’t anything in particular about Science: if the first people to walk on the moon had been, say, Orthodox Jews, who had arrived at the moon by faith in God, and if the past thousand discoveries had likewise been products of faith, I would be saying the exact same thing about Judaism. (Actually, there’s a book with roughly this premise.) It’s not the process, but the results.

But if these results are based in Science, then it seems that we should go about using that, since it’s the process that works. How, exactly, does Science work? By, in essence, knowing Reality so well that we can manipulate it to do what we want. This manipulation can’t be done the way you might manipulate a human: Reality, unlike humans, is both shockingly stubborn and astonishingly consistent. So, we have to play by Reality’s rules, but once we know those rules, we can play to win.

When manipulating a human, being able to come up with clever arguments is infinitely superior to having empirically correct answers. With Reality, this is not so. If you don’t know Reality’s rules, you can do nothing. You cannot persuade gravity; if you step off a cliff you’ll just fall. You cannot persuade gravity; if you try to build a rocket and you don’t take gravity into account, it will simply not fly. Even when you get to something as uncertain as uncertainty—probability theory, to be specific—Reality’s stringent rules don’t cease to apply. If you judge incorrectly under uncertainty, you will get a wrong answer. Unlike with gravity, you may not know it right away, but that doesn’t change the actual correctness of the answer, only your assessment thereof.

People often don’t realize most of this, because they’ve grown up with the products of Science, and so it doesn’t feel like a big deal that we have cool houses and warm clothes and literate populations. But these things have, for the overwhelming majority of history, not existed. It has only been since the Enlightenment that they have become a possibility, and since even more recently that they have actually come into common use.

The difference between Science and faith is not a question of personal preference. It’s a test, where choosing the right answer gets you to the moon, and choosing the wrong answer gets you poisoned by mercury.

Why Rationality?

I’ve identified as a rationalist for about five years now. The dictionary definitions are a bit off from what I mean, so here’s my definition.

Epistemic rationality: believing, and updating on evidence, so as to systematically improve the correspondence between your map and the territory.  The art of obtaining beliefs that correspond to reality as closely as possible.  This correspondence is commonly termed “truth” or “accuracy”, and we’re happy to call it that.

Instrumental rationality: achieving your values.  Not necessarily “your values” in the sense of being selfish values or unshared values: “your values” means anything you care about.  The art of choosing actions that steer the future toward outcomes ranked higher in your preferences.  On LW we sometimes refer to this as “winning”.

Eliezer Yudkowsky, “What Do We Mean By ‘Rationality’?”, LessWrong

Of course, these two definitions are really subsets of the same general concept, and they intertwine considerably. It’s somewhat difficult to achieve your values without believing true things, and similarly, it’s difficult (for a human, at least) to search for truth in absence of wanting to actually do anything with it. Still, it’s useful to distinguish the two subsets, since it helps to distinguish the clusters in concept-space.

So if that’s what I mean by rationality, then why am I a rationalist? Because I like believing true things and achieving my values. The better question here would be “why is everyone not a rationalist?”, and the answer is that, if it was both easy to do and widely known about, I think everyone would be.

Answering why it isn’t well-known is more complicated than answering why it isn’t easy, so, here are a handful of the reasons for the latter. (Written in the first person, because identifying as a rationalist doesn’t make me magically exempt from any of these things, it just means I know what they are and I do my best to fix them.)

  • I’m running on corrupted hardware. Looking at any list of cognitive biases will confirm this. And since I’m not a self-improving agent—I can’t reach into my brain and rearrange my neurons; I can’t rewrite my source code—I can only really make surface-level fixes to these extremely fundamental bugs. This is both difficult and frustrating, and to some extent scary, because it’s incredibly easy to break things irreparably if you go messing around without knowing what you’re doing, and you would be the thing you’re breaking.
  • I’m running on severely limited computing power. “One of the single greatest puzzles about the human brain,” Eliezer Yudkowsky wrote, “is how the damn thing works at all when most neurons fire 10-20 times per second, or 200Hz tops. […] Can you imagine having to program using 100Hz CPUs, no matter how many of them you had?  You’d also need a hundred billion processors just to get anything done in realtime. If you did need to write realtime programs for a hundred billion 100Hz processors, one trick you’d use as heavily as possible is caching. That’s when you store the results of previous operations and look them up next time, instead of recomputing them from scratch. […] It’s a good guess that the actual majority of human cognition consists of cache lookups.” Since most of my thoughts are cached, when I get new information, I need to resist my brain’s tendency to rely on those cached thoughts (which can end up in my head by accident and come from anywhere), and actually recompute my beliefs from scratch. Else, I end up with a lot of junk.
  • I can’t see the consequences of the things I believe. Now, on some level being able to do this (with infinite computing power) would be a superpower: in that circumstance all you’d need is a solid grasp of quantum physics and the rest would just follow from there. But humans don’t just lack the computing power; we can believe, or at least feel like we believe, two inherently contradictory things. This concept is, in psychology, called “cognitive dissonance”.
  • As a smart human starting from irrationality, knowing more information can easily hurt me. Smart humans naturally become very good at clever arguing—arguing for a predetermined position with propositions convoluted enough to confuse and confound any human arguer, even one who is right—and can thus use their intelligence to defeat itself with great efficiency. They argue against the truth convincingly, and can still feel like they’re winning while running away from the goal at top speed. Therefore, in any argument, I have to dissect my own position just as carefully, if not more carefully, than I dissect those of my opponents. Otherwise, I come away more secure in my potentially-faulty beliefs, and more able to argue those beliefs against the truth.

This is a short and incomplete list, of some of the problems that are easiest to explain. It’s by no means the entire list, or the list which would lend the most emotional weight to the statement “it’s incredibly difficult to believe true things”. But I do hope that it shed at least a little light on the problem.

If rationality is really so difficult, then, why bother?

In my case, I say “because my goal is important enough to be worth the hassle”. In general, I think that if you have a goal that’s worth spending thirty years on, that goal is also worth trying to be as rational as humanly possible about. However, I’d go a step further. Even if the goal is worth spending a few years or even months on, it’s still worth being rational about, because not being rational about it won’t just waste those years or months; it may waste your whole career.

Why? Because the universe rarely arrives at your doorstep to speak in grave tones, “this is an Important Decision, make it Wisely”. Instead, small decisions build to larger ones, and if those small decisions are made irrationally, you may never get the chance to make a big mistake; the small ones may have already sealed your doom. Here’s a personal example.

From a very young age, I wanted to go to Stanford. I learned that my parents had met there when I was about six, and I decided that I was going to go too. Like most decisions made by six-year-olds, this wasn’t based on any meaningful intelligence, let alone the full cost-benefit analysis that such a major life decision should have required. But I was young, and I let myself believe the very convenient thought that following the standard path would work for me. This was not, itself, the problem. The problem was that I kept on thinking this simplified six-year-old thought well into my young adulthood.

As I grew up, I piled all sorts of convincing arguments around that immature thought, rationalizing reasons I didn’t actually have to do anything difficult and change my beliefs. I would make all sorts of great connections with smart interesting people at Stanford, I thought, as if I couldn’t do the same in the workforce. I would get a prestigious degree that would open up many doors, I thought, as if working for Google isn’t just as prestigious but will pay you for the trouble. It will be worth the investment, the cached thoughts of society thought for me, and I didn’t question them.

I continued to fail at questioning them every year after, until the beginning of my senior year. At that point, I was pretty sick of school, so this wasn’t rationality, but a motivated search. But it was a search nonetheless, and I did reject the cached thoughts which I’d built up in my head for so long, and as I took the first step outside my bubble of predetermined cognition, I instantly saw a good number of arguments against attending Stanford. I realized that it had a huge opportunity cost, in both time and money. Four years and hundreds of thousands of dollars should not have been parted with that lightly.

And yet, even after I realized this, I was not done. It would have been incredibly easy to reject the conclusion I’d made because I didn’t want all that work to have been a waste. I was so close: I had a high SAT, I’d gotten good scores on 6 AP tests, including the only two computer science APs (the area I’d been intending to major in), and I’d gotten National Merit Commended Scholar status. All that would have been left was to complete my application, which I’m moderately confident I would have done well on, since I’m a good writer.

That bitterness could have cost me my life. Not in the sense that I would die for it immediately, but in the sense that everyone is dying for anything they spend significant time on, because everyone is dying. And it was here that rationality was my saving grace. I knew about the sunk cost fallacy. I knew that at this point I should scream “OOPS” and give up. I knew that at this point I should lose.

I bit my tongue, and lost.

I don’t know where I would end up if I hadn’t been able to lose here. The optimistic estimate is that I would have wasted four years, but gotten some form of financial aid or scholarship such that the financial cost was lower, and further, that in the process of attending college, I wouldn’t gain any more bad habits, I wouldn’t go stir-crazy from the practical inapplicability of the material (this was most of what had frustrated me about school before), and I would come out the other end with a degree but not too much debt and a non-zero number of gained skills and connections. That’s a very optimistic estimate, though, as you can probably tell given the way I wrote out the details. (Writing out all the details that make the optimistic scenario implausible is one of my favorite ways of combatting the planning fallacy.) There are a lot more pessimistic estimates, and it’s much more likely that one of those would happen.

Just by looking at the decision itself, you wouldn’t think of it as a particularly major one. Go to college, don’t go to college. How bad could it be, you may be tempted to ask. And my answer is, very bad. The universe is not fair. It’s not necessarily going to create a big cause for a big event: World War I was caused by some dude having a pity sandwich. Just because you feel like you’re making a minor life choice doesn’t mean you are, and just because you feel like you should be allowed to make an irrational choice just this once doesn’t mean the universe isn’t allowed to kill you anyway.

I don’t mean to make this excessively dramatic. It’s possible that being irrational here wouldn’t have messed me up. I don’t know, I didn’t live that outcome. But I highly doubt that this was the only opportunity I’ll get to be stupid. Actually, given my goals, I think it’s likely I’ll get a lot more, and that the next ones will have much higher stakes. In the near future, I can see people—possibly including me—making decisions where being stupid sounds like “oops” followed by the dull thuds of seven billion bodies hitting the floor.

This is genuinely the direction the future is headed. We are becoming more and more able to craft our destines, but we are flawed architects, and we must double and triple check our work, else the whole world collapses around us like a house on a poor foundation. If that scares you, irrationality should scare you. It sure terrifies the fuck out of me.

Language: A Cluster Analysis of Reality

Cluster analysis is the process of quantitatively grouping data in such a way that observations in the same group are more similar to each other than to those in other groups. This image should clear it up.

Whenever you do a cluster analysis, you do it on a specific set of variables: for example, I could cluster a set of customers against the two variables of satisfaction and brand loyalty. In that analysis, I might identify four clusters: (loyalty:high, satisfaction:low), (loyalty:low, satisfaction:low), (loyalty:high, satisfaction:high), and (loyalty:low, satisfaction:high). I might then label these four clusters to identify their characteristics for easy reference: “supporters”, “alienated”, “fans” and “roamers”, respectively.

What does that have to do with language?

Let’s take a word, “human”. If I define “human” as “featherless biped”, I’m effectively doing three things. One, I’m clustering an n-dimensional “reality-space”, which contains all the things in the universe graphed according to their properties, against the two variables ‘feathered’ and ‘bipedal’. Two, I’m pointing to the cluster of things which are (feathered:false, bipedal:true). Three, I’m labeling that cluster “human”.

This, the Aristotelian definition of “human”, isn’t very specific. It’s only clustering reality-space on two variables, so it ends up including some things that shouldn’t actually belong in the cluster, like apes and plucked chickens. Still, it’s good enough for most practical purposes, and assuming there aren’t any apes or plucked chickens around, it’ll help you to identify humans as separate from other things, like houses, vases, sandwiches, cats, colors, and mathematical theorems.

If we wanted to be more specific with our “human” definition, we could add a few more dimensions to our cluster analysis—add a few more attributes to our definition—and remove those outliers. For example, we might define “human” as “featherless bipedal mammals with red blood and 23 pairs of chromosomes, who reproduce sexually and use syntactical combinatorial language”. Now, we’re clustering reality-space against seven dimensions, instead of just two, and we get a more accurate analysis.

Despite this, we really can’t create a complete list of all the things that most real categories have in common. Our generalizations are leaky in some way, around the edges: our analyses aren’t perfect. (This is absolutely the case with every other cluster analysis, too.) There are always observations at the edges that might be in any number of clusters. Take a look at the graph above in this post. Those blue points at the top left edge, should they really be blue, or red or green instead? Are there really three clusters, or would it be more useful to say there are two, or four, or seven?

We make these decisions when we define words, too. Deciding which cluster to place an observation happens all the time with colors: is it red or orange, blue or green? Splitting one cluster into many happens when we need to split a word in order to convey more specific meaning: for example, “person” trisects into “human”, “alien”, and “AI”. Maybe you could split the “person” cluster even further than that. On the other end, you combine two categories into one when sub-cluster distinctions don’t matter for a certain purpose. The base-level category “table” substitutes more specific terms like “dining table” and “kotatsu” when the specifics don’t matter.

You can do a cluster analysis objectively wrong. There is math, and if the math says you’re wrong, you’re wrong. If your WCSS is so high that you have a cluster that you can’t label more distinctly than “everything else”, or if it’s so low you’ve segregated your clusters beyond the point of usefulness, then you’ve done it wrong.

Many people think “you can define a word any way you like”, but this doesn’t make sense. Words are cluster analyses of reality-space, and if cluster analyses can be wrong, words can also be wrong.


This post is a summary of / is based on Eliezer Yudkowsky’s essay sequence, “A Human’s Guide to Words“.

The Limits of the Argument from Incredulity

Of late, I’ve heard a lot of arguments of a general form “X is immoral, unacceptable, unreasonable, unpleasant, and otherwise should really just not be the way it is, so therefore, we must make it stop being this way”. Inherently, there is absolutely no problem with this sort of ethical/moral argument: it has the ability to highlight areas in which the world could be fixed. But of late, I’ve seen this argument used in ways that make very little logical sense, and it occurred to me that people who make this argument may not realize its limitations.

Let me borrow Eliezer Yudkowsky’s example, and argue that I should be able to run my car without needing to put gas in it. “It would be so much more fun, and so much less expensive, if we just decided to repeal the law that cars need fuel.” Owning a car would become more accessible to lower-income households, if you remove the gas expense. There is less pollution if nobody is burning fossil fuels anymore. “Isn’t it just obviously better for everyone?”

Well, that would be very nice if it were possible, but given that cars, like all things in the universe, must obey the Law of Conservation of Energy, it isn’t. Being angry about it will not change that.

When people use these, as I’m calling them, “arguments from incredulity”, the biggest problem is that they are so caught up in being angry at how bad the thing is, they fail to realize that any possible solution is dramatically more complicated than just “abolish the thing”. It’s obvious with something simple, like putting fuel in cars, but when you get to something more complicated, like the minimum wage or housing the poor, it’s less so.

I’ve seen arguments about the minimum wage. They start with something about how the current minimum wage is insufficient to cover what the arguer considers a minimally adequate cost of living, then there’s some tear-jerking personal anecdote tossed in, and it ends with the conclusion that the minimum wage needs to be raised (to 12 or 15 or whatever dollars an hour). Do they take into account the problem that raising the minimum wage, without doing anything about low-income housing/student loans/etc, would simply end up increasing inflation, without doing jack shit about the actual problem they’re so incredulous about? No.

I’ve seen arguments about housing the poor. They start with some statistics about how many houses in America are currently vacant and some other statistics about how many Americans are homeless, then they go on some diatribe about how if rich people weren’t such assholes we could just fill the vacant houses with homeless people and be done with it. Do they bother to consider the fact that what counts as a vacant house for the purposes of those statistics includes a rental house that so happens to not currently be tenanted, but will become tenanted within the next month, as well as many other circumstances that would not fit well with the five-word “stick homeless people in them” solution? No.

Complex problems rarely have simple solutions, but getting angry tends to make peoples’ brains think in simple terms: therein lies the weakness of the argument from incredulity.

So how do we fix this problem? How do we individually stop ourselves from making angry arguments which aren’t useful?

The obvious solution is to think about the problem (preferably for five whole minutes by a physical clock) without getting angry about any aspect of it. Consider the problem’s complexity, consider the possible reasons why this problem might exist, and consider the way this problem came to be in the first place. Remember that the problem is most likely caused and perpetuated by normal, not abnormal, psychology: homelessness does not exist because all landlords are psychopaths. And remember that the solution to the problem will take into account the fact that human beings are not always perfect all the time, not make a clever argument for how humans would all live in perfect harmony if only we would implement communism.

A problem cannot be solved by getting angry with it, like one might persuade a human to do something by getting angry at them. Complex problems are solved by solutions which encompass their every aspect, break them down into manageable pieces, and tackle each piece as thoroughly as necessary. A good solution does not leave the hearer with a lingering sense of doubt; instead, it should make the problem feel solvable. If your solution doesn’t do that, it’s probably a good idea to keep looking.