Primarily, the idea was to "problematize" the appeal to intuitions in our readings. Here's how it goes. An author is defending an account of what makes a life good or meaningful. The claim entails that a person in a certain scenario would (or wouldn't) have a good life (or a meaningful life, etc.). The author says "we think" the entailed fact is true. But do we?
We flagged points like this in the literature covered in the course--on the good life and the meaningful life--and tested out whether survey-takers shared the "authorized" (so to speak) intuition -- i.e. the one the author both has and surmises that we all share. Below, I'll be using the phrase "authorized intuition" in just this artificial sense.
The results gave us a basis for discussing the following questions:
- If it's important what we think, should philosophers do experiments to find out what we think? Are they qualified to do these experiments?
- If philosophers should do experiments, what is the appropriate study population? Should subjects have any particular education or training, to make their intuitions "count"?
- Should philosophers appeal less to intuitions about scenarios?
- Should they limit themselves to stating their own intuitions, instead of saying "we think ..."?
- What attributes made a difference to how survey-takers answered the questions? How did philosophy background factor into it? Religion? Gender?
- What does the impact of those various factors tell us about questions 1-4?
Here are some things we weren't trying to do by means of the survey--
- We weren't attempting to answer philosophical questions about the good life or the meaningful life.
- We weren't trying to support or refute the philosophers we had read. It was up for discussion and debate whether the results had any bearing on the truth of their theories.
- We weren't trying to comprehensively diagnose how each individual survey-taker looks at the good life or the meaningful life. For that purpose, we would have needed both more questions and different questions.
Over the course of the semester, scenarios were derived from the literature we read. Often the scenario in the survey was exactly as stated in the literature. Sometimes it was tweaked a bit for various reasons. Questions were kept as short as possible so people would read them in their entirety and complete the survey.
Answer options were limited to two or three, with no option of "not sure" or the like. This was because my teaching experience tells me people do usually lean in one direction or another, but will avoid commitment if given the chance. Since I worried we'd have very few survey-takers (I was wrong about that!), I wanted to avoid this sort of non-result.
We used multiple "collector" links so we could differentiate groups. The first link I used was disseminated here, at Twitter, and on Facebook. It would up (not by request) at the very popular atheist blog Pharyngula. That link yielded 4,912 responses.
Students disseminated another link via Facebook, email, and so on. That link yielded 136 responses. A couple of other links were used to attract philosophers and we had some international students who attracted a few Indian and Chinese responses.
In addition to disseminating the survey, students also interviewed someone of their choice about their answers and reported to the class on their findings. This gave us an additional source of information about various answers and patterns.
Because of the influx from Pharyngula (presumably), our total sample was heavily male (71.3%) and unreligious (93.1%). The link used by students generated the opposite balance--it was heavily female and religious. When we sorted the answers by gender and religion we did see some differences in responses.
Our sample was large enough to be informative, even if the population we're trying to represent is very large. (All people? Everywhere? Just all internet users? I'm not sure how we should think about that.) However, the sampling was by no means random. I wouldn't take the exact statistics too seriously, but there are suggestive patterns. Some claims about "what we think" were strongly confirmed and some were strongly disconfirmed.
Again--the point was to "problematize" the role of intuitions and to raise questions about whether (or not!) philosophers should become experimentalists. The idea was not to settle any philosophical disputes by letting "the people" take a vote.
I'll discuss the results question by question, but if you don't have the patience for that, you can take a look at them all yourself.
Broken down by religiosity
Broken down by gender
Broken down by amount of background in philosophy
QUESTIONS 1-3 -- NECESSITIES
These questions are based on scenarios in chapter 5 of The Weight of Things--my book on what it is to live a good life. I offer what's known as an "objective list" account of the good life, and use these scenarios (among many other bits of evidence and arguments) to support putting autonomy (the Bella scenario), self (the Norbert scenario), and progress (the Constance scenario) on my list. My account says the items on the list are not just contributors of good (so interchangeable), but necessities. So a paucity of even one item on the list would mean someone's life was flawed.
We toyed with various ways of stating the answer options, to capture the notion of a life being flawed for lack of one ingredient. I thought "flawed" sounded too harsh, taken out of the context of the book. So we settled on asking whether the person's life could be "going entirely well."
By a wide margin, survey-takers rejected the "authorized" intuition (recall from above--that's the one the author both has and surmises that most people would have). This is especially so in the Constance scenario. Only 6.2 had the "authorized" intuition!
It's interesting that on the other two questions, there's some correlation between philosophy background and sharing the "authorized" intuition. On the Bella question (about autonomy), 32.1% of philosophers agree with me, whereas 21.3% do, overall. On the Norbert question (about self), 62.5% of philosophers agree with me, and just 30.3% do overall.
QUESTIONS 4-6 - SOURCES OF HAPPINESS
The two Maggie questions are also about scenarios in chapter 5 of The Weight of Things. The issue there is whether it matters whether a person's happiness comes from valuable or non-valuable sources. I maintain that one item on the list of necessities is having some happiness that comes from valuable sources. In the two scenarios, none of Maggie's happiness comes from a source that could be seen as valuable. The "authorized" intuition is that her life is therefore flawed--"couldn't be going entirely well." Overall, 41.4% agreed with the "authorized" view about Maggie on Magic Drug, and 49.3% agreed with the "authorized" view of Maggie the Gambler. Women were more likely than men to have the "authorized" intuition on both of these questions.
The question about the peeping Tom concerned whether or not it's true that happiness adds to life wherever it comes from. This is again from chapter 5 of The Weight of Things. The "authorized" view is that the immorality of Tom's behavior detracts from his life, but the enjoyment he gets from it does add. So there are two separate impacts here that both affect how his life is going.
About half the responders agreed that the enjoyment does add (whether a little or a lot). Here gender made a difference. 44.8% of males said his enjoyment added to his life "not at all" but 53.8% of females said his enjoyment added to his life "not at all".
QUESTION 7 - DISABILITY
The question about Carlos is from chapter 6 of The Weight of Things. Phew, finally some strong support for the "authorized" intuition. 82% agree that Carlos's life could be going entirely well. In that chapter I argue that different objective lists pertain to different people, depending on their capacities. So the absence of morality in his life doesn't create any flaw.
QUESTIONS 8-10 - ACHIEVEMENT, KNOWLEDGE
These questions were derived from the book The Best Things in Life by Thomas Hurka. He also has an objective list view about the good life, but doesn't think of the items on his list as "necessities". They merely add goodness to a life, but can be interchanged with other goods. Question 8 is a scenario the class devised to see whether survey-takers agree with Hurka about the value of achievement, and especially the value of playing games.
Again, the "authorized" intuition doesn't meet with popular support. Only 19.2% thought the better chess player has a life going "a little better" than the mediocre chess player. But philosophers were more impressed with achievement than others. The more background, the more likely that you'd find value in being a better chess player.
The Angela scenario in questions 9 and 10 was inspired by a discussion in Hurka's book (chapter 5, pp. 88-9) about the value of knowing your place. He says
To get the idea across, he describes a scenario in which you are lying in a hospital bed, in pain, nearing the end of life. He asks: Is it worth the pain to acquire some new knowledge of your place (e.g. where you are, what your co-workers think of you, or the like)? And is it worth the pain to overcome some mistake? He says there's an asymmetry: it's worth more to overcome the mistake than to acquire new knowledge. The Angela scenario is meant to assess whether survey-takers agree about this asymmetry.Being positively mistaken about where you are or how you relate to people near you is an important good; the mismatch is a greater evil than the match is a good. Earlier I argued that pain is more evil than pleasure is good, and there's a similar asymmetry here. Being wrong about some truths in our second category [knowing your place] is more evil than knowing them is good; with these truths it's more important to avoid the evil of error than to achieve any good in getting them right. (p. 89)
We changed some of the details. It seemed to stack the decks against knowledge having any value to imagine someone so close to the end of life. Instead, we made Angela someone recovering from surgery and in pain. It also seemed to stack the decks to suppose the topic is what co-workers think of you or whether your spouse is faithful. That sort of knowledge/error has a huge impact on happiness.
The topic in our question is literally knowing your place--knowing where you are. The mistake is Angela thinking she's on Mars. The knowledge she could obtain is that she's in the hospital. We tried to remove the issue of happiness from the equation by saying her sedative keeps her from suffering or being anxious.
If Hurka's asymmetry hypothesis meets with popular approval, what should we find? We should find that, of those who say yes to 9 (yes, it's worth a 5% increase in pain to stop thinking she's on Mars), most go on to say no to 10 (no, it's not worth another 5% increase in pain in order to come to know she's in the hospital). Here's how the yessers on 9 (52.6% overall) answered 10:
89% of those who think it's worth a 5% pain increase to avoid Angela's Mars-error also think it's worth a 5% pain increase to acquire knowledge that she's in the hospital. There's not much agreement with the "authorized" intuition.
QUESTIONS 11-15 - THE MEANING OF LIFE
Now we have a series of questions derived from Richard Taylor's lovely article "The Meaning of Life." These scenarios are taken directly from the article, except the last, which has been fortified with some concepts from Harry Frankfurt's book The Reasons of Love.
Taylor holds that the myth of Sisyphus offers the definitive case of a meaningless life, and holds clues to what "meaningful" and "meaningless" mean. A meaningful life must have some significant and lasting result. Do "the people" agree with Taylor that Sisyphus lives a meaningless life (question 11)? Most do--70.2%.
Question 12 proposes a variant on the Sisyphus myth--one suggested by Taylor. Suppose Sisyphus intensely wants to push those rocks up the hill. Taylor says this doesn't make his life meaningful. His life is still objectively meaningless, but it's subjectively meaningful. "The people" agree, by a narrow margin (52.3% agree).
Question 13 contrasts two ways the gods could help out Sisyphus--these are Taylor's scenarios. In one, the gods inject him with a drug, so he wants to push the rock up a hill. In the other, he pushes rocks up the hill and builds a beautiful temple. (We changed "temple" to "castle" to avoid the religious connotations of "temple".) The "authorized" answer is that the gods are more benevolent if they inject Sisyphus with the drug. Sadly, "the people" disagree. 78.6% say letting him build the castle is more benevolent. Philosophers are a little more inclined to agree with Taylor (69.6% say "castle").
Question 14 adds a Frankfurtian twist. Taylor just affirms the value of doing things we want to do. In The Reasons of Love, Frankfurt has an elaborate account of the mental states involved in living meaningfully. They involve not just wanting to do X, but "higher order" mental states--wanting to want to do X. We live meaningfully when we identify with our desires. 71.7% think those Frankfurtian elements are crucial; without them, doing what you want to do isn't actually good for you.
Question 15 assesses whether "the people" agree with Taylor about the meaninglessness of repetitive, cyclical animal lives. 63.8% disagree and think the glow worms don't have meaningless lives.
QUESTION 16 - PRIORITY OF MORALITY
This question was about being guided in one's career choices by a moral ideal and being guided by a non-moral ideal, like becoming a master chef. It was inspired by Frankfurt, and also by the book Meaning in Life and Why It Matters, by Susan Wolf. They both challenge a view that's dominant among philosophers--that morality has some sort of superlative importance in our lives. The scenario was devised by a student and then collectively revised. The survey shows more support for the view that morality is preeminent.
QUESTIONS 17-20 - THE EXPERIENCE MACHINE
These questions were taken from Robert Nozick's famous discussion of "the experience machine" and a challenge to it from philosopher Felipe DeBrigard.
Nozick claims that people want not merely happiness, but contact with reality, as evidenced by the fact that they wouldn't plug into an experience machine that provided them with a simulated, but fantastically happy, life. Question 17 confirms that most wouldn't plug in--70.8% say No.
DeBrigard hypothesized that not wanting to plug in isn't actually due to a desire for contact with reality. He tried to confirm this experimentally by telling subjects they were already plugged into an experience machine and had the option of staying there or unplugging. He found that most wanted to remain plugged in--if they were told certain things about the real world they'd be returning to. He surmised that people refuse the two "jumps" in the diagram below for the same reason--out of status quo bias. They're averse to losing whatever good they have in their present situation.
Questions 18 and 19 are meant to test out DeBrigard's theory. In question 18, you are told you are already in the experience machine, and asked if you want to unplug, assuming that reality may or may not be as you remember it. This is about the same as the way DeBrigard stated the question. He told participants that the real world would be "different".
The important question is how people answer 18, if they already said No on 17--no they didn't want to plug in. If someone says No to the first jump and No to the second jump, this shows what's driving them is status quo bias, not a desire for contact with reality--or so DeBrigard says. We didn't find the "No, No" pattern though. Those who said No to plugging in on 17 mostly said Yes to unplugging on 18--86.3%. They want to be in both blue zones of the above diagram!
Just in case people did say No to plugging in and No to unplugging, we wanted to explore why. Was it necessarily because of status quo bias in both cases? It seems to me that DeBrigard doesn't do enough to make other things equal, and thus we can't draw that conclusion. The experience machine is very enticing--you get exactly the life you would want (experientially) if you plug in. The reality you have the option of returning to is unenticing--you're just told it's "different." If people prefer reality in the first case, and the experience machine in the second, you can't conclude it's for the same reason. They may be avoiding the unknown in the second case (status quo bias), but not in the first.
The scenario in question 19 is supposed to eliminate this problem. Reality, post experience machine, is made much more controllable and enticing, just like the experience machine in Nozick's original thought experiment. If people had answered 18 as DeBrigard predicted they would, it would have been interesting to see if they answered 19 in the same way. No on 17 and 19 would convince me it really is all about status quo bias.
Of those who said No on 17 (only those are represented in the table below), 93.6% said Yes on 19. They would unplug from the experience machine. Again, folks seem to want to be in both blue zones of the diagram.
Question 20 is one more attempt to find out if people desire contact with reality. Do people want the real trip to Antarctica or the simulated trip? The point here was to offer alternatives that wouldn't trigger status quo bias. It's a short trip, and you'll confront "the unknown" equally, whichever option you choose. It would be hard to construe the preference for the real trip as being due to anything but desire for contact with reality. In fact, most people did choose the real trip--82.2%.
On the experience machine questions, it was interesting to compare religions and non-religious responders. Fewer religious people (19.2%) wanted to plug in to the experience machine than unreligious people (29.9%) (question 17).
QUESTION 21 - RELATIVISM
Finally, a question from chapter 2 of The Weight of Things. My intuition is that Simeon Stylites lived a bad life, and I use that as part of my argument that we can make judgments about different lives, and shouldn't simply defer to cultural standards. 58.9% had the "authorized" intuition about his life, whereas 34.7% said we shouldn't judge. Only 14.3% of the professional philosophers said we shouldn't judge.
This survey showed that "authorized intuitions" are often shared by the public but are also often not. The "not" cases could be because of problems with the way some of the questions were worded. Class discussion generated ideas about how to restate the Constance question (1), the chessplayer question (8), and a few others. But even with rewording, some intuitions would probably still have only minority support. Should a philosopher be seriously concerned?
The more I think about it, the more I think not. An intuition about a scenario seems analogous to a simple, theory-free perception ("the sky looks blue"), but actually isn't. The way you understand and respond to a scenario is a complicated matter having to do with education, training, background assumptions, adaptation to an author's various assumptions, being convinced of prior arguments, etc. You respond to a scenario differently depending on whether you did or didn't read what the author said on the previous 100 pages, did or didn't read other books and articles, etc.
For example, Hurka invites readers to share his intuitions about knowing your place (questions 9 and 10) and achievement (8) only after chapters of his book arguing that happiness is not the only thing that matters, and that contact with reality does matter, and that knowing your place and achievement essentially involve contact. Lifting scenarios out of that context stops readers from being exposed to the arguments that might lead them to share his intuitions about Angela and the chess-players.
Likewise, in chapter 5 of my book, I invite readers to share intuitions about cases (like in questions 1-3), but only in the context of many other arguments about the values in question. Furthermore, chapter 5 comes after chapters that try to convince readers of many relevant things: we can judge other people's lives, happiness is not the only thing that matters, the desire fulfillment account of the good life is wrong, etc. I'm hoping people will share my intuitions after exposure to all of that, not independently of exposure to all of of that. (Or so I am more inclined to say, after the experience of creating this survey!)
Students had various different thoughts about what we gained by doing this survey. I'll end with a few of their reflections--
"I'm not sure I see much to be gained from this survey, except for those responses that came from people who have studied philosophy, preferably extensively....We cannot trust the public's intuitions because they are exactly that: intuitions. They are not necessarily sound judgments that come from critical thought, which is the whole point of philosophy as a field of study. Nevertheless, it is interesting data to have in order to spark further discussion about what we stand to gain from philosophy by comparing the answers of philosophers to those of the rest of the respondents." (JH)
"I believe that the amount of similarity between the 'lay' public's responses and the philosophers' responses casts doubt on the idea that training and expertise are necessary for one's responses to a scenario to be meaningful. I believe that, in many cases, a non-philosopher may use his intuition to arrive at the same conclusion that a philosopher might achieve through deliberate reasoning." (KD)
"I think that this experiment taught us that the public's intuition does follow, for the most part, the beliefs of philosophers. In areas where the intuition differs, I think we should look extra carefully to examine whether these philosophical ideas really hold merit. Other that that this experiment was just a really fun way to see what the public really thinks about certain situations." (TL)
"I think we learned that to survey people's intuitions about philosophical topics, the questions must be extremely clear. To us, the questions made sense, but when giving it to people who had not been studying these topics and discussing them in class, there were people who were confused by the questions." (PF)
"I think we did learn from this survey and its results. The public's intuitions do mostly match with the intuitions of the professional philosopher. However, religion does play a major role in determining the values people place on different matters. For the next survey I'll do, I will have a more precise and well-worded survey. I believe wording also changes the question's interpretation." (NA)For more discussion of experimental philosophy, pro and con, see this forum. I also recommend Joshua Alexander's book Experimental Philosophy: An Introduction.