I usually avoid reproducing articles from publications but this was a
well-written piece that I wanted to etch it in my memory.
From,
http://www.wired.com/magazine/2009/12/fail_accept_defeat/all/1
Accept Defeat: The Neuroscience of Screwing
Up
By Jonah Lehrer | December 21, 2009 | 10:00 am | Wired Jan
2010
It all started with the sound of static. In May 1964, two astronomers at Bell
Labs, Arno Penzias and Robert Wilson, were using a radio telescope in suburban
New Jersey to search the far reaches of space. Their aim was to make a detailed
survey of radiation in the Milky Way, which would allow them to map those vast
tracts of the universe devoid of bright stars. This meant that Penzias and
Wilson needed a receiver that was exquisitely sensitive, able to eavesdrop on
all the emptiness. And so they had retrofitted an old radio telescope,
installing amplifiers and a calibration system to make the signals coming from
space just a little bit louder.
But they made the scope too sensitive. Whenever Penzias and Wilson aimed their
dish at the sky, they picked up a persistent background noise, a static that
interfered with all of their observations. It was an incredibly annoying
technical problem, like listening to a radio station that keeps cutting
out.
At first, they assumed the noise was man-made, an emanation from nearby New
York City. But when they pointed their telescope straight at Manhattan, the
static didn’t increase. Another possibility was that the sound was due to
fallout from recent nuclear bomb tests in the upper atmosphere. But that didn’t
make sense either, since the level of interference remained constant, even as
the fallout dissipated. And then there were the pigeons: A pair of birds were
roosting in the narrow part of the receiver, leaving a trail of what they
later described as “white dielectric material.” The scientists evicted
the pigeons and scrubbed away their mess, but the static remained, as loud as
ever.
For the next year, Penzias and Wilson tried to ignore the noise, concentrating
on observations that didn’t require cosmic silence or perfect precision. They
put aluminum tape over the metal joints, kept the receiver as clean as
possible, and hoped that a shift in the weather might clear up the
interference. They waited for the seasons to change, and then change again, but
the noise always remained, making it impossible to find the faint radio echoes
they were looking for. Their telescope was a failure.
Kevin Dunbar is a researcher who studies how scientists study things — how they
fail and succeed. In the early 1990s, he began an unprecedented research
project: observing four biochemistry labs at Stanford University. Philosophers
have long theorized about how science happens, but Dunbar wanted to get beyond
theory. He wasn’t satisfied with abstract models of the scientific method —
that seven-step process we teach schoolkids before the science fair — or the
dogmatic faith scientists place in logic and objectivity. Dunbar knew that
scientists often don’t think the way the textbooks say they are supposed to. He
suspected that all those philosophers of science — from Aristotle to Karl
Popper — had missed something important about what goes on in the lab. (As
Richard Feynman famously quipped, “Philosophy of science is about as
useful to scientists as ornithology is to birds.”) So Dunbar decided to launch
an “in vivo” investigation, attempting to learn from the messiness of
real experiments.
He ended up spending the next year staring at postdocs and test tubes: The
researchers were his flock, and he was the ornithologist. Dunbar brought tape
recorders into meeting rooms and loitered in the hallway; he read grant
proposals and the rough drafts of papers; he peeked at notebooks, attended lab
meetings, and videotaped interview after interview. He spent four years
analyzing the data. “I’m not sure I appreciated what I was getting myself
into,” Dunbar says. “I asked for complete access, and I got it. But there was
just so much to keep track of.”
Dunbar came away from his in vivo studies with an unsettling insight: Science
is a deeply frustrating pursuit. Although the researchers were mostly using
established techniques, more than 50 percent of their data was unexpected. (In
some labs, the figure exceeded 75 percent.) “The scientists had these elaborate
theories about what was supposed to happen,” Dunbar says. “But the results kept
contradicting their theories. It wasn’t uncommon for someone to spend a month
on a project and then just discard all their data because the data didn’t make
sense.” Perhaps they hoped to see a specific protein but it wasn’t there. Or
maybe their DNA sample showed the presence of an aberrant gene. The details
always changed, but the story remained the same: The scientists were looking
for X, but they found Y.
Dunbar was fascinated by these statistics. The scientific process, after all,
is supposed to be an orderly pursuit of the truth, full of elegant hypotheses
and control variables. (Twentieth-century science philosopher Thomas Kuhn, for
instance, defined normal science as the kind of research in which “everything
but the most esoteric detail of the result is known in advance.”) However, when
experiments were observed up close — and Dunbar interviewed the scientists
about even the most trifling details — this idealized version of the lab fell
apart, replaced by an endless supply of disappointing surprises. There were
models that didn’t work and data that couldn’t be replicated and simple studies
riddled with anomalies. “These weren’t sloppy people,” Dunbar says. “They were
working in some of the finest labs in the world. But experiments rarely tell us
what we think they’re going to tell us. That’s the dirty secret of
science.”
How did the researchers cope with all this unexpected data? How did they deal
with so much failure? Dunbar realized that the vast majority of people in the
lab followed the same basic strategy. First, they would blame the method. The
surprising finding was classified as a mere mistake; perhaps a machine
malfunctioned or an enzyme had gone stale. “The scientists were trying to
explain away what they didn’t understand,” Dunbar says. “It’s as if they didn’t
want to believe it.”
The experiment would then be carefully repeated. Sometimes, the weird blip
would disappear, in which case the problem was solved. But the weirdness
usually remained, an anomaly that wouldn’t go away. This is when things get
interesting. According to Dunbar, even after scientists had generated their
“error” multiple times — it was a consistent inconsistency — they might fail to
follow it up. “Given the amount of unexpected data in science, it’s just not
feasible to pursue everything,” Dunbar says. “People have to pick and choose
what’s interesting and what’s not, but they often choose badly.” And so the
result was tossed aside, filed in a quickly forgotten notebook. The scientists
had discovered a new fact, but they called it a failure.
The reason we’re so resistant to anomalous information — the real reason
researchers automatically assume that every unexpected result is a stupid
mistake — is rooted in the way the human brain works. Over the past few
decades, psychologists have dismantled the myth of objectivity. The fact is, we
carefully edit our reality, searching for evidence that confirms what we
already believe. Although we pretend we’re empiricists — our views dictated by
nothing but the facts — we’re actually blinkered, especially when it comes to
information that contradicts our theories. The problem with science, then,
isn’t that most experiments fail — it’s that most failures are ignored. As he
tried to further understand how people deal with dissonant data, Dunbar
conducted some experiments of his own. In one 2003 study, he had undergraduates
at Dartmouth College watch a couple of short videos of two different-size balls
falling. The first clip showed the two balls falling at the same rate. The
second clip showed the larger ball falling at a faster rate. The footage was a
reconstruction of the famous (and probably apocryphal) experiment performed by
Galileo, in which he dropped cannonballs of different sizes from the Tower of
Pisa. Galileo’s metal balls all landed at the exact same time — a refutation of
Aristotle, who claimed that heavier objects fell faster.
While the students were watching the footage, Dunbar asked them to select the
more accurate representation of gravity. Not surprisingly, undergraduates
without a physics background disagreed with Galileo. (Intuitively, we’re all
Aristotelians.) They found the two balls falling at the same rate to be deeply
unrealistic, despite the fact that it’s how objects actually behave.
Furthermore, when Dunbar monitored the subjects in an fMRI machine, he found
that showing non-physics majors the correct video triggered a particular
pattern of brain activation: There was a squirt of blood to the anterior
cingulate cortex, a collar of tissue located in the center of the brain. The
ACC is typically associated with the perception of errors and contradictions —
neuroscientists often refer to it as part of the “Oh shit!” circuit — so it
makes sense that it would be turned on when we watch a video of something that
seems wrong.
So far, so obvious: Most undergrads are scientifically illiterate. But Dunbar
also conducted the experiment with physics majors. As expected, their education
enabled them to see the error, and for them it was the inaccurate video that
triggered the ACC. But there’s another region of the brain that can be
activated as we go about editing reality. It’s called the dorsolateral
prefrontal cortex, or DLPFC. It’s located just behind the forehead and is one
of the last brain areas to develop in young adults. It plays a crucial role in
suppressing so-called unwanted representations, getting rid of those thoughts
that don’t square with our preconceptions. For scientists, it’s a
problem.
When physics students saw the Aristotelian video with the aberrant balls, their
DLPFCs kicked into gear and they quickly deleted the image from their
consciousness. In most contexts, this act of editing is an essential cognitive
skill. (When the DLPFC is damaged, people often struggle to pay attention,
since they can’t filter out irrelevant stimuli.) However, when it comes to
noticing anomalies, an efficient prefrontal cortex can actually be a serious
liability. The DLPFC is constantly censoring the world, erasing facts from our
experience. If the ACC is the “Oh shit!” circuit, the DLPFC is the Delete key.
When the ACC and DLPFC “turn on together, people aren’t just noticing that
something doesn’t look right,” Dunbar says. “They’re also inhibiting that
information.”
The lesson is that not all data is created equal in our mind’s eye: When it
comes to interpreting our experiments, we see what we want to
see and disregard the rest. The physics students, for instance, didn’t watch
the video and wonder whether Galileo might be wrong. Instead, they put their
trust in theory, tuning out whatever it couldn’t explain. Belief, in other
words, is a kind of blindness.
How to Learn From Failure
Too often, we assume that a failed experiment is a wasted effort. But not all
anomalies are useless. Here’s how to make the most of them.—J.L.
1. Check Your Assumptions : Ask yourself why this result feels like a failure.
What theory does it contradict? Maybe the hypothesis failed, not the
experiment.
2. Seek Out the Ignorant: Talk to people who are unfamiliar with your
experiment. Explaining your work in simple terms may help you see it in a new
light.
3. Encourage Diversity: If everyone working on a problem speaks the same
language, then everyone has the same set of assumptions.
4. Beware of Failure-Blindness: It’s normal to filter out information that
contradicts our preconceptions. The only way to avoid that bias is to be aware
of it.
But this research raises an obvious question: If humans — scientists included —
are apt to cling to their beliefs, why is science so successful? How do our
theories ever change? How do we learn to reinterpret a failure so we can see
the answer?
This was the challenge facing Penzias and Wilson as they tinkered with their
radio telescope. Their background noise was still inexplicable, but it was
getting harder to ignore, if only because it was always there. After a year of
trying to erase the static, after assuming it was just a mechanical
malfunction, an irrelevant artifact, or pigeon guano, Penzias and Wilson began
exploring the possibility that it was real. Perhaps it was everywhere for a
reason.
In 1918, sociologist Thorstein Veblen was commissioned by a popular magazine
devoted to American Jewry to write an essay on how Jewish “intellectual
productivity” would be changed if Jews were given a homeland. At the time,
Zionism was becoming a potent political movement, and the magazine editor
assumed that Veblen would make the obvious argument: A Jewish state would lead
to an intellectual boom, as Jews would no longer be held back by institutional
anti-Semitism. But Veblen, always the provocateur, turned the premise on its
head. He argued instead that the scientific achievements of Jews — at the time,
Albert Einstein was about to win the Nobel Prize and Sigmund Freud was a
best-selling author — were due largely to their marginal status. In other
words, persecution wasn’t holding the Jewish community back — it was pushing it
forward.
The reason, according to Veblen, was that Jews were perpetual outsiders, which
filled them with a “skeptical animus.” Because they had no vested interest in
“the alien lines of gentile inquiry,” they were able to question everything,
even the most cherished of assumptions. Just look at Einstein, who did much of
his most radical work as a lowly patent clerk in Bern, Switzerland. According
to Veblen’s logic, if Einstein had gotten tenure at an elite German university,
he would have become just another physics professor with a vested interest in
the space-time status quo. He would never have noticed the anomalies that led
him to develop the theory of relativity.
Predictably, Veblen’s essay was potentially controversial, and not just because
he was a Lutheran from Wisconsin. The magazine editor evidently was not
pleased; Veblen could be seen as an apologist for anti-Semitism. But his larger
point is crucial: There are advantages to thinking on the margin. When we look
at a problem from the outside, we’re more likely to notice what doesn’t work.
Instead of suppressing the unexpected, shunting it aside with our “Oh shit!”
circuit and Delete key, we can take the mistake seriously. A new theory emerges
from the ashes of our surprise.
Modern science is populated by expert insiders, schooled in narrow disciplines.
Researchers have all studied the same thick textbooks, which make the world of
fact seem settled. This led Kuhn, the philosopher of science, to argue that the
only scientists capable of acknowledging the anomalies — and thus shifting
paradigms and starting revolutions — are “either very young or very new to the
field.” In other words, they are classic outsiders, naive and untenured. They
aren’t inhibited from noticing the failures that point toward new
possibilities.
But Dunbar, who had spent all those years watching Stanford scientists struggle
and fail, realized that the romantic narrative of the brilliant and perceptive
newcomer left something out. After all, most scientific change isn’t abrupt and
dramatic; revolutions are rare. Instead, the epiphanies of modern science tend
to be subtle and obscure and often come from researchers safely ensconced on
the inside. “These aren’t Einstein figures, working from the outside,” Dunbar
says. “These are the guys with big NIH grants.” How do they overcome
failure-blindness?
While the scientific process is typically seen as a lonely pursuit —
researchers solve problems by themselves — Dunbar found that most new
scientific ideas emerged from lab meetings, those weekly sessions in which
people publicly present their data. Interestingly, the most important element
of the lab meeting wasn’t the presentation — it was the debate that followed.
Dunbar observed that the skeptical (and sometimes heated) questions asked
during a group session frequently triggered breakthroughs, as the scientists
were forced to reconsider data they’d previously ignored. The new theory was a
product of spontaneous conversation, not solitude; a single bracing query was
enough to turn scientists into temporary outsiders, able to look anew at their
own work.
But not every lab meeting was equally effective. Dunbar tells the story of two
labs that both ran into the same experimental problem: The proteins they were
trying to measure were sticking to a filter, making it impossible to analyze
the data. “One of the labs was full of people from different backgrounds,”
Dunbar says. “They had biochemists and molecular biologists and geneticists and
students in medical school.” The other lab, in contrast, was made up of E. coli
experts. “They knew more about E. coli than anyone else, but that was what they
knew,” he says. Dunbar watched how each of these labs dealt with their protein
problem. The E. coli group took a brute-force approach, spending several weeks
methodically testing various fixes. “It was extremely inefficient,” Dunbar
says. “They eventually solved it, but they wasted a lot of valuable
time.”
The diverse lab, in contrast, mulled the problem at a group meeting. None of
the scientists were protein experts, so they began a wide-ranging discussion of
possible solutions. At first, the conversation seemed rather useless. But then,
as the chemists traded ideas with the biologists and the biologists bounced
ideas off the med students, potential answers began to emerge. “After another
10 minutes of talking, the protein problem was solved,” Dunbar says. “They made
it look easy.”
When Dunbar reviewed the transcripts of the meeting, he found that the
intellectual mix generated a distinct type of interaction in which the
scientists were forced to rely on metaphors and analogies to express
themselves. (That’s because, unlike the E. coli group, the second lab lacked a
specialized language that everyone could understand.) These abstractions proved
essential for problem-solving, as they encouraged the scientists to
reconsider their assumptions. Having to explain the problem to someone else
forced them to think, if only for a moment, like an intellectual on the
margins, filled with self-skepticism.
This is why other people are so helpful: They shock us out of our cognitive
box. “I saw this happen all the time,” Dunbar says. “A scientist would be
trying to describe their approach, and they’d be getting a little defensive,
and then they’d get this quizzical look on their face. It was like they’d
finally understood what was important.” What turned out to be so important, of
course, was the unexpected result, the experimental error that felt like a
failure. The answer had been there all along — it was just obscured by the
imperfect theory, rendered invisible by our small-minded brain. It’s not until
we talk to a colleague or translate our idea into an analogy that we glimpse
the meaning in our mistake. Bob Dylan, in other words, was right: There’s no
success quite like failure.
For the radio astronomers, the breakthrough was the result of a casual
conversation with an outsider. Penzias had been referred by a colleague to
Robert Dicke, a Princeton scientist whose training had been not in astrophysics
but nuclear physics. He was best known for his work on radar systems during
World War II. Dicke had since become interested in applying his radar
technology to astronomy; he was especially drawn to a then-strange theory
called the big bang, which postulated that the cosmos had started with a
primordial explosion. Such a blast would have been so massive, Dicke argued,
that it would have littered the entire universe with cosmic shrapnel, the
radioactive residue of genesis. (This proposal was first made in 1948 by
physicists George Gamow, Ralph Alpher, and Robert Herman, although it had been
largely forgotten by the astronomical community.) The problem for Dicke was
that he couldn’t find this residue using standard telescopes, so he was
planning to build his own dish less than an hour’s drive south of the Bell Labs
one.
Then, in early 1965, Penzias picked up the phone and called Dicke. He wanted to
know if the renowned radar and radio telescope expert could help explain the
persistent noise bedeviling them. Perhaps he knew where it was coming from?
Dicke’s reaction was instantaneous: “Boys, we’ve been scooped!” he said.
Someone else had found what he’d been searching for: the radiation left over
from the beginning of the universe. It had been an incredibly frustrating
process for Penzias and Wilson. They’d been consumed by the technical problem
and had spent way too much time cleaning up pigeon shit — but they had finally
found an explanation for the static. Their failure was the answer to a
different question.
And all that frustration paid off: In 1978, they received the Nobel Prize for
physics.
Contributing editor Jonah Lehrer (jonah.lehrer@gmail.com) wrote about how our
friends affect our health in issue 17.10
