Composite image with a drawing of a face on the left and the names of colors on the right

Unit 3: When seeing shouldn’t be believing

Guided Learning

Introduction

Those of us who are sighted rely immensely on our visual system to know and understand our physical and social world.

But the visual world is not perceived directly – our perception is only as good as the inferences we make based on past experiences. And sometimes these inferences just fail us. Was the job candidate you hired actually qualified, or did you make an incorrect assumption based on a visual cue that you were unaware of, like the beauty of their face or the style of their suit?

Here, we will explore different visual biases that may impact our decisions. The good news is that once we truly understand that our vision is fallible, we can act to outsmart it.

Module 1 of 5

Strooped!

Video

Most of us believe we can control what pieces of information influence our decisions. But can we? The Stroop Test suggests no.

Watch the video

Transcript

NARRATOR: There’s a simple test that’s been making fools of us since the 1930s. Can you beat it?

You’ll see words printed in different colors. Just name the color of each word you see, as fast as you can. Don’t read the word, just name the color of the letters. For example: yellow, blue, green, red.

Got it? Try it for yourself. Ready? Go!

[The words “yellow,” “green,” “blue,” and “red” appear one at a time, with the color of the text matching the name of the color: for instance, the word “yellow” appears in yellow text; “green” in green text, etc. Each word appears several times in random order.]

Pretty easy, right?

Now that you know what you’re doing, try it again with one small change. Remember: name the ink color, not the word. Ready? Go!

[The color words (“yellow,” “green,” “blue,” and “red”) appear again, except the color of the text now does not match the name of the color: for instance, the word “blue” appears in green text; “green” in blue text, “yellow” in red text, etc. Several word/color combinations appear in random order.]

Was that just as easy? No. But why?

It’s because naming colors may be easy, but reading is easier. Reading is automatic. It’s effortless. We can’t NOT read – and we do it all the time. So when the word BLUE matches the color BLUE, it’s easy to do the task perfectly. But when the two don’t match, suddenly, we take longer. We make more mistakes. It’s hard.

What you just experienced is the Stroop Effect, named for psychologist J. Ridley Stroop. But its effects aren’t limited to the psychology laboratory. In fact, it’s rumored that this test was used by Americans during the Cold War to catch Russian spies. Here’s how. The agents were asked to do the Stroop Task… but in Russian. The Cyrillic characters meant nothing to the American agents, so nothing stopped them from simply rattling off the right answers. But the spies, who could read Russian — they couldn’t ignore the words, so they slowed down, and they stumbled. And this revealed their true identities.

This is an interesting story, but it also teaches us something important about decision-making… even if we’re not Russian spies. Once information is in front of us, we process it — even if it’s irrelevant, misleading, or even wrong. So think about this: if having even two variables makes our job harder, what happens when there are 5, 10, even hundreds of variables in play, for instance when we evaluate people?

It’s commonly argued that without television, John F. Kennedy may not have become the 35th President of the United States. People who watched his debate with Richard Nixon tended to vote for the younger, traditionally attractive Kennedy. But the people who listened to the debate tended to vote for Nixon, who sounded more traditionally Presidential.

RICHARD NIXON: We know the way to progress, and I think first of all, our own record proves that we know the way.

NARRATOR: Our decisions shouldn’t hinge on whether information is seen or heard. But this debate showed that seemingly irrelevant features do matter — even in key decisions like voting.

So what can we do to make our decisions better?

Let’s start with the Stroop Test itself. When people try to beat the test, the first thing they often do is squint. This blurs the word and puts the focus on the color. Smart.

We can do this metaphorically, too. In the 1970s and 80s, American symphony orchestras began using blind auditions to filter out the information they didn’t need. So the musicians played. The judges judged. But between them now was a curtain that kept the focus on the music instead of the musician. Over the next 25 years, the number of women musicians in major orchestras skyrocketed. Their chances of being hired had increased by 25%.

This is a template for how we can outsmart our own minds.

One: acknowledge that you too can be “Strooped.” Our world is fast-paced and rich with information, so we can’t just say, “Oh, I’ll ignore that.”

Two: don’t assume that more information is better. Strange as it may sound, consider saying “please send me the data without x, y, or z.” Remove names from resumes or graded assignments. Make evaluations before knowing anyone else’s opinion.

Each of these strategies takes work, but sometimes you need to squint to see more clearly. Whether you’re looking for spies or your next employee, you’ll do it better if you focus on just what matters.

Expand

Highlights

Key takeaways from this module

Test your knowledge

  • What is the Stroop Effect?
  • From the Stroop Task (saying the color of the word, not the word), the right lessons to take away are: (select all that apply)
Module 2 of 5

Should You Trust Your Vision?

Interactive

Experience a powerful illusion in this demo based on experiments by Professors Dan Levin and Mahzarin Banaji.

Module 3 of 5

Illusions at work

Video

We say that “seeing is believing,” but visual illusions can trick us, whether we’re looking at checkerboards or human faces. How can we outsmart them?

Watch the video

Transcript

NARRATOR: Want to catch your brain making a mistake in real-time?

Take a look at this checkerboard, created by vision scientist Edward Adelson.

The squares labeled “A” and “B” are exactly the same shade of gray.

Don’t believe me? I don’t blame you. I see them as different too.

But here’s the proof: simply connect the two with this gray bar, and voilà! Suddenly, it’s obvious: A and B are identical.

Here’s the strange thing: remove the bar and the illusion comes back immediately, just as strong as it was before. We can do this again for the skeptics: different shades of grey; same shades of grey; different; same. Our eyes can see that A and B are identical. But our brains fall for it every time, no matter how often you debunk the illusion. Why?

Professor Adelson points to two specific details of his illusion:

First: contrast. “A” looks darker because it’s surrounded by lighter-colored squares.The opposite is true for “B.”

Second: Take a look at the soft edges of this darker section. Our brains see this and think “Ah, it’s a shadow.” We know that objects in shadows look darker than they actually are, so our brains compensate for that by making us actually see “B” as lighter.

Something else to consider? We all know what a checkerboard pattern looks like. So we come into this illusion already expectingA and B to be different, and that could influence what we see.

What about our expectations about people?

Which face is darker? The one on the left? Or the one on the right?

Psychologists Daniel Levin and Mahzarin Banaji found that most people (including Mahzarin herself!) see the Black face as darker than the White face, even though the two faces are the exact same shade of gray. Again, debunking the illusion is simple; here’s an idea from Dan:

DANIEL LEVIN: Just cover up the lower parts of the faces. You’ll be left with two foreheads, and they’ll look exactly the same.

NARRATOR: Clearly, the features of the face matter. But in a second study, the researchers showed that you could create this illusion using the same face. People saw a racially ambiguous face labeled “Black” as darker than the same face labeled “White.”

Did you?

Think about what this means: We see two faces. They are identical. But we’re so used to associating different races with different skin-tones that we misperceive a simple physical reality. All because of a single word. That’s how powerful our expectations are.

Now, when it comes to judging the brightness of objects, the errors seem pretty harmless. But research shows that we also see the exact same action as being less “bad” if the perpetrator is attractive; we rate the same musician as more masterful if she’s described as a “natural” rather than a “hard-worker.” These illusions are more costly.

So how can we base our decisions on fact rather than fiction?

Think back to how we debunked illusions before: we needed to have the two tiles, the two faces in front of us at the same time to really prove the truth. So when you have multiple options in front of you (whether it’s job candidates or health insurance plans), consider them side by side. Not only does this let you test your intuitions, research shows that you’ll be more likely to. When people evaluate things together, they’re more likely to use the same specific benchmarks to compare them.

Want something stronger? If you want to avoid illusions altogether, try blinding. We’ve talked about this technique before: when orchestras put a curtain between musicians and judges during auditions and let the music do the talking, a new crop of players rose to the top.

Sometimes we perceive more than what meets the eye. But a few tweaks to the process can help outsmart your mind.

Expand

Highlights

Key takeaways from this module

Test your knowledge

  • In the checkerboard illusion designed by vision scientist Ted Adelson, the shadowed “white” square and the unshadowed “black” square are actually
Module 4 of 5

How well can you read a face?

Interactive

Do you know what “competence” looks like? Try this quiz, based on Professor Alexander Todorov’s 2005 experiment, to learn what your impressions predict.

Module 5 of 5

About Face: How First Impressions Fool Us

Video

Our faces broadcast information about us: whether we’re smart, warm, trustworthy. How do these signals influence decision-making – and are they accurate?

Watch the video

Transcript

NARRATOR: Psychologist Alexander Todorov has discovered the mathematics underlying our first impressions.

Using data he’s collected from thousands of people at Princeton University, Dr. Todorov has built a computer program that can pinpoint the very parts of a face that trigger our gut instincts: that is, whether someone looks aggressive, introvertedattractive, and so on.

See for yourself. As quickly as you can, answer this this question:

Which of these two people is more trustworthy? A or B?

And what about from this pair?

And this one?

This probably wasn’t hard. And in fact, you probably could have done it even faster. Here’s Dr. Todorov:

ALEXANDER TODOROV: You don’t need more than 160-200ms to get pretty much the same first impression that you get from unlimited time.

NARRATOR: This was Dr. Todorov’s first finding: that first impressions are fast.

The second finding is perhaps even more surprising: almost everyone sees the same thing.

Consider the faces you just saw. If you’re like the majority of people in Dr. Todorov’s studies, these are the ones you chose as being more trustworthy.

And what about competence?  Just look at this face, created by Dr. Todorov’s software.

The changes to this face are small: the tilt of the mouth, the size of the brow, the distance between the eyes. And yet taken together, these features trigger a judgment. If we’re honest, we’ll admit that this face seems to be gaining and losing IQ points right in front of our very eyes!

We’re all familiar with sayings like “don’t judge a book by its cover” — but we do.  And Dr. Todorov’s studies show this, too.

ALEXANDER TODOROV: We did some studies presenting participants faces of politicians, and it turns out that using judgments of naive participants, we can predict about 70% of the electoral outcomes. In a nutshell, politicians who happened to look more competent than their rivals were more likely to win in their elections.

NARRATOR: These findings—which included almost every major congressional candidate for the 2004-2008 elections—are incredible.  Campaign promises, advertisements, and experience aside, what really matters is how you look.

And faces don’t just sway who we elect to lead our governments. They also affect who we hire; who we give opportunities to; who we promote.

Now, if everyone sees the same thing when they look at a face, it raises the question: are these impressions accurate? Do people look more competent because they actually are more competent?  Do they look more trustworthy because they are more trustworthy?

The short answer is no.

ALEXANDER TODOROV: Often the natural assumption is there must be some kernel of truth in these impressions. But if you look closely at the data, there is very, very little evidence that these judgments are accurate. In many situations, you can ignore completely the face and your judgment will be much better.

NARRATOR: If faces are so misleading, the question, then, of course, is why? Why do our brains do this?

Here’s a popular theory for the case of trustworthiness: We look more trustworthy the more we look… like babies! Large eyes, small chins, round faces – these are the features we love about infants, the most trustworthy creatures there are.  So when adults possess these features, we view them as more trustworthy, kind, and honest.

Research tells us that we’re more likely to agree with baby-faced adults; to accept their offers in negotiations. We’re even less likely to convict baby-faced defendants for premeditated crimes… and more likely to give them more lenient sentences when we do.

And what does this mean for the workplace? Are baby-faced employees more likely to get the benefit of the doubt in a workplace dispute? Does the width of your jaw influence your hire-ability?

Clearly, our first impressions can get us into trouble. So let’s talk about how to get around them, and make better decisions. It’s hard: first impressions are fast and they’re automatic.  But if we can admit that these judgments happen, we can find simple ways to outsmart our minds.

How many times have you said or heard something like this:

MALE VOICE: I felt like I could trust her as soon as I met her.

NARRATOR: Or…

FEMALE VOICE: I can really see him in this role.

NARRATOR: When you catch these moments, ask a simple question: Based on what

Want something stronger? Here’s one senior leader in education’s strategy: “As soon as I meet someone, I acknowledge my first impression in the first five seconds, and spend the rest of the interview trying to prove myself wrong.” Be your own devil’s advocate by challenging your first impressions.

Finally: when applicable, try putting off the face-to-face early on in the hiring process.  Focus on hard evidence of skill, talent, and experience that’s in a resume or other objective forms of data as a first step.

We can all be blinded by first impressions, but with regular practice, we can learn to look past the surface.

Expand

Highlights

Key takeaways from this module

Test your knowledge

  • Which of these facts about face perception are true? (select all that apply)

What You've Learned

Our vision is fallible. We inadvertently process irrelevant information, and our expectations, not reality, shape our perception.

Think back to the Levin and Banaji study, where pictures of faces with Afrocentric features were seen as darker than faces with Eurocentric features even though both were the same shade of gray. This should surprise us – our vision wasn’t accurate! What we saw was contrary to the facts. What does this mean for how we treat others?

We can hold ourselves to higher standards. We can ask ourselves to prove our initial assessments about a person with data, not gut feelings. Be your own devil’s advocate. Ask yourself questions like: what has this person done to justify this impression?

Vision plays a large role in making decisions. Identify and outsmart the sources of bias from visual information that can lead you astray – by deviating from both accuracy and your own values.

Reflections

Question 1

In the workplace, we often hire or assign tasks based on our evaluations of trust and competence. Of course, we want to hire people who are both trustworthy and competent, but our eyes and what we ‘see’ in a face can mislead us.

Faces are among the most potent sources of bias because we often fail to recognize that we are making deeper assumptions of a person’s character from the physical aspects of their face. What are ways we can outsmart face-based bias?

Think of concrete, actionable ways, not just vague promises like “I’ll try not to form impressions from the face”. These don’t work effectively – our brains automatically process the information in front of us, even if we don’t want them to!

What you write in this box is just for you. You will have a chance to download your response, but it will not be stored on our servers.

Your response:

Our thoughts:

The next time you can see a whole bunch of new faces, look at each face and ask yourself whether you believe the person “looks” trustworthy or not. Then do the same thing but ask yourself if the person “looks” competent or not. See if you can surprise yourself by how easily you seem to be making these decisions based on just the face, even though you know nothing about that person!

Are there patterns to your decisions? Do you think women’s faces exude greater trust than men’s faces? Or, that men’s faces signal more competence than women’s?

Now that you know the data, work to become your own best critic.

Congratulations! You have completed Unit 3: When seeing shouldn’t be believing .

Subscribe to Outsmarting Implicit Bias

Dive deeper

Extra materials if you want to learn more

Frank Stanton, the then-president of CBS, said of the Kennedy-Nixon debate: “Kennedy was bronzed beautifully… Nixon looked like death.” The legendary debate changed the face of political media strategy, but was it warranted? A 2003 study by political scientist James Druckman suggests, yes. People were significantly more likely to think Kennedy “won” the debate when they watched it versus listened to it.

Colors are one thing, but our decisions can also be influenced by something as insignificant as a single letter – whether we’re aware of it or not. Watch this video about how José Zamora dropped a single letter to gain a title, and read more about the research that shows how names can Stroop us.

There are illusions we all fall prey to, like Ted Adelson’s Checker-Shadow illusion or Roger Shepard’s table tops. But some perceptions vary from person to person. Remember this dress? Was it black and blue or white and gold? This simple question set fire to the internet in 2015. The real dress was black and blue… so why did some people see something so different? It turns out that the colors we “see” change depending on how our minds interpret the source of light in the photograph. Read more about the science behind the illusion at Wired.

Just as a face labeled “Black” or “White” looks darker or lighter, or how a musician labeled a “natural” or a “hard-worker” sounds different, the name at the top of a resume can change how qualified we think a person is. To learn more, read our article Can women be biased against other women? and listen to our podcast Race bias in hiring: When both applicant and employer lose.

“Some 51% of the employers who considered candidates individually chose an employee who had underperformed relative to the group. By contrast, only 8% of the employers who considered candidates side-by-side chose underperformers.” How does simultaneous evaluation help us see talent more clearly? Read more about the research at The Wall Street Journal.

“Most companies say they want to attract a diverse workforce, but few deliver. The only solution may be a radical one: anonymity.” Find out how Silicon Valley is taking steps towards a more diverse workforce (New York Times).

See more demonstrations of how faces influence our first impressions on the Perception and Judgment Lab’s website. Dr. Todorov’s book Face Value: The Irresistible Influence of First Impressions was published in 2017 by Princeton University Press.

“People are convinced that more competent-looking business people are more valuable, and they get higher salaries,’ [Professor Christopher] Olivola explained, even though the companies don’t perform any better under their leadership”. From James Hamblin’s article in The Atlantic: The Introverted Face.

“After his mugshot sent the Internet into a frenzy, the 32-year-old received modeling offers. Fundraising efforts and Facebook groups were created for Meeks”. From the Los Angeles Times’ Veronica Rocha: “‘Hot Felon’ Jeremy Meeks released from federal prison and gets job offers”.

A person’s face can even influence the medical care they receive: when clinicians watched videos of women undergoing painful examinations, Amanda Williams and colleagues found that they were less likely to recommend appropriate pain treatment for those who “looked” less trustworthy. Learn more at “The Conversation”.

Next unit:

Unit 3: When seeing shouldn’t be believing