One of the themes that I’ve seen within the skeptical community over time (and, more specifically, in recent discussions about race (here and here) and gender (here and here and here and here and here)) is the hyper-skepticism about social issues and a hyper-rationality when seeking to understand social issues. It often comes in the form of JAQing, typically from people who “just want data.” What they often (but not always) mean by this is that they want specific, quantitative, peer reviewed, top-tier journal published research on whatever specific topic is being discussed, and if that type of research doesn’t exist, then the problem is moot.

In this post, I’m going to address a few of the common arguments advanced by some members of the skeptic community that over-depend on quantitative data and further seek to minimize the usefulness of qualitative data.

Let’s start with a few arguments that I’ve seen around the skeptical blogosphere.

Argument: “The plural of anecdote is not evidence/data.”

Response: That depends on the question being asked. For example, if your question is “how many people experience racism/sexism?”, then anecdotal data are not helpful because you’re seeking a quantity.

However, if your question is “what are some of the ways that people have experienced racism/sexism?” then anecdotal data is absolutely helpful. The answer to this question could certainly be compiled to put a percentage on the ways, but the question itself isn’t focused on quantity but quality.

“But Will,” you might be thinking, “how can we start to fix the problem of racism/sexism if we don’t know the scope of the problem?”

Argument: “We cannot advance a solution unless we know the size and scope of a problem.”

Response: The only thing we learn from quantitative data on the scope of a problem is how widespread the problem is. It tells us very little (if anything) about why the problem is occurring. Qualitative research, on the other hand, is much more useful in helping us to understand why certain behaviors occur, because it utilizes much more focused data collection methodologies. What I mean by “focused” here is that qualitative research seeks a narrow depth whereas quantitative research seeks a shallow breadth.

What this means is that multiple qualitative studies are important for understanding the underlying causes of problems more broadly. In other words, we need more than one qualitative study to be able to generalize. Conversely, we can start with a quantitative study and then move to qualitative data collection to give some depth to the generalized statistical data.

Argument: “Humans are flawed and thus anecdotes and testimonials are not completely accurate; therefore, anecdotes are invalid and tell us nothing.”

Response: This confuses accuracy with reliability. Anecdotes/testimonials can be quite reliable and informative, even if they are not completely accurate. They can paint a personalized and specific picture of a problem that is otherwise overlooked in quantitative data. This can help to humanize social issues (e.g., look at how much American culture has changed with regards to gay and lesbian people over the last ten years—it’s due mostly to the visibility of queer people sharing their lived experiences, not because some statistical dataset was pushed into the public sphere).

Accuracy and reliability are also issues in quantitative, statistical data (obviously, since statistical data is always accompanied with margins of error). But we don’t dismiss a quantitative study because it had a 95% accuracy rate. That would be pretty damned reliable, despite not being completely accurate.

The problem as I see it is that much of the skeptic community highly values quantitative data over any qualitative data regardless of the research question being asked. This is a fundamentally flawed way of approaching research. It seems to me that this comes from an over emphasis on objectivity such that many skeptics feel that they can escape subjectivity. But this is not possible. We are temporally and materially situated beings living in the world. By necessity, we cannot be completely objective, as this would require what Donna Haraway refers to as the “god trick.” That is, we would have to be all seeing and all knowing in order to achieve true objectivity. Our phenomenological situatedness means that we cannot achieve true objectivity.

I point this out because I often encounter people who identify as skeptics that claim to be objective, as if they were somehow able to jettison their situatedness (subjectivity). It is a pervasive thought in the skeptical community that we are somehow above the cognitive errors so common to the rest of humanity, and the irony of this type of thinking is completely lost on those who subscribe to it.

There’s nothing wrong with subjectivity. It’s an important and vital aspect of human existence. We should pay careful attention to our subjective biases and be up front and open with them as much as possible—this includes having them pointed out by others.

See, the important part of skepticism to me comes in the critical thinking. It comes in the recognition that we are all humans who live in an imperfect world because we are imperfect. Once we recognize this and cease to pretend that we can rise above this imperfection, we can begin to work with and around our subjectivities. Though we will never eliminate it, we can (and should) seek to be critical of and minimize our biases in our data. And we can’t do that if we think we’re already objective and free of bias.

So, my plea to my fellow skeptics is this: don’t be afraid of the anecdotal or the subjective, and be skeptical of the idea of pure objectivity. This doesn’t mean we must jettison objectivity or that we should only rely on subjectivity, it just means we must use the right methodological tools to answer the different types of questions we have about the world.

Will

Will

Will is the admin of Queereka, part of the Skepchick network. They are a cultural/medical anthropologist who works at the intersections of sex/gender, sexuality, health, and education. Their other interests include politics, science studies, popular culture, and public perceptions and understandings of anthropology. Follow them on Twitter at @anthrowill and Facebook at facebook.com/anthrowill.

Previous post

Skepchick Quickies 8.8

Next post

Speaking Out Against Hate Directed At Women: Carlos Alfredo Diaz

81 Comments

  1. Profile photo of Anne S
    August 8, 2012 at 12:14 pm —

    Will, this is great. That we need to “be skeptical of the idea of pure objectivity” is an important idea not brought up enough.

    This seems like a good time to link to Julila Galef’s Straw Vulcan video from Skepticon IV: http://www.youtube.com/watch?v=tLgNZ9aTEwc It isn’t just outsiders that think skeptics are always hyper rational.

    • Profile photo of Will
      August 8, 2012 at 2:06 pm —

      Thanks for pointing to that video! I totally forgot about it and it’s perfectly relevant. :)

    • Profile photo of dr. dr. professor
      August 8, 2012 at 2:49 pm —

      The people who practice selective over-skepticism to claims of sexism generally follow 3 rules

      Rule 1: If I don’t want it to be true, there must be rigorous peer-reviewed proof before anything is done about it

      Rule 2: If people do present proof, I’ll likely just ignore it or challenge it further without even stopping to consider it

      Rule 3: I will ignore being skeptical of things I agree with

      Lame eh?

    • Profile photo of Unnullifier
      August 8, 2012 at 3:25 pm —

      That video outlines the problem quite well. The “I thought we’re supposed to be skeptics here” “where’s the data” hyper-skeptical JAQ offs are in fact behaving in an extremely irrational manner: they are abusing the concepts of skepticism and data to use as a shield against aspects of reality they wish not to be true.

    • Profile photo of blgmnts
      August 9, 2012 at 11:39 am —

      How about actually calling hyper-scepticism out as a straw vulcan just like we call out straw men?

      I always found hyper-sceptic positions obnoxious and intellectually dishonest but difficult to handle; rhetorically they work too well, for now.

  2. Profile photo of Corey Feldman
    August 8, 2012 at 12:35 pm —

    Will, well written and thought out. I am a big believer that to many in the skeptical community seem to be under the illusion that they are above their own biases and subjectivity. In other words no one is truly skeptical about everything.

  3. Profile photo of Amy Roth
    August 8, 2012 at 1:02 pm —

    Excellent post that will be great to point to. Thank you!

  4. Profile photo of cthandhs
    August 8, 2012 at 1:11 pm —

    Love it! This is a great response to people who treat harassment like UFO sightings.

  5. Profile photo of mrmisconception
    August 8, 2012 at 1:12 pm —

    If scientists (who many of these same skeptic pretend to emulate) refused to acknowledge their subjectivity many of them right now would be trying to figure out how neutrinos are able to exceed the speed of light. Instead they realized that there was something about their current subjective state that was most likely wrong and asked for other scientists to help them identify what that might be.

    Subjectivity is not the enemy, it is a condition we must deal with and take into account and at times it can be more productive than objectivity, after all not everything is science.

  6. Profile photo of spokesgay
    August 8, 2012 at 1:34 pm —

    Excellent Will.

    I think it’s just as important to note the *motivation* behind JAQ’ing off as it is to note the JAQ’ing. It’s a very selective thing; it’s applied only to phenomena the JAQ’er very much does not want to be true. Wicknight, for example, would never have acted this way about anything he and his fellow doodbros encounter every day. It wouldn’t even have occurred to him (he’s just the latest example, by no means unique).

    That so many men are, well, pathologically compelled to go down this rabbit hole when the topic of abuse of women comes up should garner as much (maybe more) scrutiny as the JAQ’ing itself. Calling out the motivation to selectively apply this bizarre distortion of skepticism is desperately needed.

    • Profile photo of Will
      August 8, 2012 at 2:05 pm —

      I totally agree. Great point.

    • Profile photo of dr. dr. professor
      August 8, 2012 at 2:20 pm —

      ^^^^
      THIS

      Yes, the “it can’t be true without rigorous verification” crap is applied mostly to things they don’t want to be true.

      Selective skepticism, can that be a new term?

  7. Profile photo of Gene
    August 8, 2012 at 2:08 pm —

    An xcellent argument. I have encountered the attitude you refer to before from some skeptics and rationalists before.

  8. Profile photo of Jason
    August 8, 2012 at 2:28 pm —

    One can certainly fix a problem without knowing its scope or severity, however it is difficult to argue that something is a problem that warrants high prioritization without that specific information.

    That’s the problem that a lot of us have with the data-free anecdote-driven approach that’s being pushed in relation to social issues in skepticism lately.

    • Profile photo of Will
      August 8, 2012 at 2:43 pm —

      Someone didn’t read very well. “Data-free anecdote-driven approach”?? Whether or not something is data depends on the question being asked.

      If your question is “how should we prioritize dealing with x and y?” then sure quantitative data can be useful.

      However, if your question is “do people experience x and y?” then qualitative–I’m sorry, “data-free anecdote-driven approach”–is certainly useful.

      You’re completely ignoring the context in which information is being sought, which was one of my chief complaints in the post (the urge to always prioritize quantitative data no matter what).

      • Profile photo of spokesgay
        August 8, 2012 at 2:48 pm —

        Yeah, not “no matter what” though. Mostly when it comes to anything questioning white male privilege or patriarchy.

        • Profile photo of Will
          August 8, 2012 at 6:27 pm —

          I’m not sure I agree. I see this sort of thing tossed around all the time in the skeptic community about how qualitative research is useless (or mostly useless). You can see this in a comment below from someone who identifies as a feminist. That sort of way of thinking is extremely common.

          • Profile photo of spokesgay
            August 9, 2012 at 2:18 am

            Ugh. Sadly you’re right. There must be some sort of Uber Skeptic disease. Someone should track it down and engineer an antibiotic.

          • Profile photo of Will
            August 9, 2012 at 2:20 am

            Yes, but first, we need to concoct a methodologically perfect quantitative study to determine the scope of the problem.

            ;)

      • Profile photo of Jason
        August 8, 2012 at 10:35 pm —

        You are absolutely right that when you ask questions about anecdotes (what are some instances of sexism people have experienced) then anecdotes are data. The reason why they become data in this instance, is the question itself biases them to be data.

        You make it sound as if the only thing that social-activist skeptics in this arena are concerned with is proving that certain things occur.

        There are claims made about problems that need to be dealt with. Terms like “patriarchy” are thrown around which include claims inherently that speak to prevalence. Issues of prevalence, scope, and generalized claims are made all the time by the people you’re defending with this piece.

        To me, it seems like you’re missing the fact that quantitative claims and questions are being made all the time. And that’s why people are asking for quantitative data.

        If someone asks for quantitative data in response to another person merely discussing an experience of sexism, that’s a douchey thing to do. That’s not right. You can’t use statistics to argue against personal anecdotes of oppression or injustice.

        But you must use statistics to back up far-reaching claims. And that isn’t happening nearly as often as it should be.

        I’m sorry I used the term “data-free,” it was loaded and insensitive. I could’ve been much more polite. And I am a believer in feminism, because I believe the data bears it out. However, I feel like we don’t talk about or emphasize it enough.

        • Profile photo of Will
          August 9, 2012 at 1:55 am —

          You are absolutely right that when you ask questions about anecdotes (what are some instances of sexism people have experienced) then anecdotes are data. The reason why they become data in this instance, is the question itself biases them to be data.

          That’s the point of what I wrote. To say that “anecdotes are not data” is false because it depends on what question you’re asking.

          You make it sound as if the only thing that social-activist skeptics in this arena are concerned with is proving that certain things occur.

          Oh? And how do I make it sound like that? What I’m actually advocating is for people to be open to both quantitative and qualitative approaches to topics such as sexism and racism. That’s all.

          There are claims made about problems that need to be dealt with. Terms like “patriarchy” are thrown around which include claims inherently that speak to prevalence. Issues of prevalence, scope, and generalized claims are made all the time by the people you’re defending with this piece.

          I highly recommend you go watch Christina Rad’s new video where she talks about patriarchy.

          Who are people I’m defending with this piece, by the way? I’m simply pointing out that people who respond to calls for anecdotes or people sharing their experiences with “yeah but where’s the evidence?” are hyper-rational pricks. If someone makes claims about the scope or breadth of a problem, it’s fine to ask for quantitative data. If someone makes a claim about their experience, asking for quantitative data about the problem makes the person a shithead. It really is that simple. And if you don’t think this is happening, go explore some of the links I provided (particularly the recent hubbub with the commenter Emil on Greta Christina and Ian Cromwell’s blogs).

          To me, it seems like you’re missing the fact that quantitative claims and questions are being made all the time. And that’s why people are asking for quantitative data.

          To me, it seems like you’re missing the fact that qualitative claims and questions are being made all the time, but people are still asking for quantitative data in response to those qualitative claims.

          If someone asks for quantitative data in response to another person merely discussing an experience of sexism, that’s a douchey thing to do. That’s not right. You can’t use statistics to argue against personal anecdotes of oppression or injustice.

          Agreed, and that’s my point.

          But you must use statistics to back up far-reaching claims. And that isn’t happening nearly as often as it should be.

          Agree on the first part, don’t agree on the second part. Often what happens is people do provide links to studies and such and then the goal-post-shifting begins. There’s lots of quantitative data out there to back up common claims made about sexism, harassment, and racism. People just don’t like it so they pretend it’s not an important issue in our communities because, like I argued towards the end of my post, we are skeptics so we are by default more rational than the rest of the population. And that sort of thinking is quite unskeptical and can be quite dangerous.

          To be clear, I don’t think you are engaging in that kind of thinking. And honestly I don’t really disagree with much of what you’ve written.

          I’m sorry I used the term “data-free,” it was loaded and insensitive. I could’ve been much more polite. And I am a believer in feminism, because I believe the data bears it out. However, I feel like we don’t talk about or emphasize it enough.

          No worries, and I totally agree. What I’m advocating is really that we use all of the data available, and not just limit ourselves to quantitative data because of some misheld notion of pure objectivity.

          • Profile photo of Jason
            August 9, 2012 at 1:09 pm

            I believe you that people are asking for statistics in response to others merely discussing their personal experiences, and that makes me sad. There are quite a few assholes out there who are upset at the mere fact that sexism exists on any level and is being talked about.

            You make good points about those people, I’m gonna guess I just got some wires crossed when reading your post.

            Anyway, sorry again about the “data-free” comment — it was stupid.

    • Profile photo of spokesgay
      August 8, 2012 at 2:45 pm —

      Bitchez lie. Herp Durr. Anecdotes are extraordinary claims and hard to believe. Herp Durr. Why won’t you give me data that is hard to collect for reasons I won’t acknowledge and that I’d only say was insufficient anyway. Herp Durr.

      • Profile photo of Improbable Joe
        August 9, 2012 at 1:35 pm —

        That’s right! Bitchez fucking LIE!! I know, because I heard about this one time that some guy slept with a woman and then she called it rape when he stopped answering her phone calls. That’s proof right… what? Oh, my cousin was telling me about this guy he knew who saw it happen to a co-worker. Anyways, that’s the proof in a nut- why do you keep interrupting me? One of my cousins, I guess maybe Jim but I’m not sure because it has been a few years.

    • Profile photo of Anne S
      August 8, 2012 at 2:47 pm —

      I would more inclined to agree with you if I saw any evidence that prioritizing efforts to minimize harassment in our community was drawing precious resources from more important things.

      And in case anyone tries to point to the number of blogs posts in the past year that have had to address these issues as an example of resources being wasted–no. This conversation has been as drawn out as it has because there are certain people who reject to any time at all being spent on these issues. If it weren’t for these constant derailments, we could have used far fewer words.

      For many (including people like DJ), growing this movement is a priority. I don’t see how we can do that without simultaneously working to minimize harassment.

      And the fact that you say “data-free” suggest that you didn’t not read Will’s post very carefully. Quantitative data is not always the answer.

    • Profile photo of spokesgay
      August 8, 2012 at 2:47 pm —

      You know how I prioritize issues? I use my ethical compass. My ethical compass tells me that abuse of women, queers, and racial minorities is a problem that needs desperate attention yesterday.

      The question is: Why doesn’t your ethical compass tell you that?

      Failing to react to these issues with concern instead of skepticism is a serious character flaw.

    • Profile photo of Ginger
      August 8, 2012 at 2:49 pm —

      How high a percent of racial minorities in the skeptical movement do you think need to report racist comments or behaviors before it should be a priority to tell people it’s not cool to make racist comments? How about before an org leader tells it’s members it’s not allowed? 15%? 50% Or maybe just 1 out of 20 is ok to ignore and never be discussed? Please. State a number.

    • Profile photo of dr. dr. professor
      August 8, 2012 at 2:56 pm —

      One can certainly fix a problem without knowing its scope or severity, however it is difficult to argue that something is a problem that warrants high prioritization without that specific information.

      Funny, that sounds a lot like “prove you were raped/harassed before we do anything”. The victim of sexual harassment trauma must produce mountains of data to a board (of probably mostly men) before it can be prioritized.

      That’s VERY STUPID.

      Policies to prevent race, sex, or sexuality based harassment should be in place even before the first report of any harassment comes in.

    • Profile photo of WhatPaleBlueDot
      August 8, 2012 at 6:49 pm —

      There is plenty of hard data backing up the social issues addressed here and elsewhere. It’s also easy to find. So stop waaaaahing and bother to do a review yourself.

      • Profile photo of dr. dr. professor
        August 8, 2012 at 9:47 pm —

        Yeah that’s the other thing. Guys who don’t want to believe in harassment issue will jump very quickly to “we need more data” without spending 5 minutes on google to see if there is any data…

        Laaaaaaame.

  9. Profile photo of Unnullifier
    August 8, 2012 at 3:29 pm —

    But Will, I thought biggest data is best data?

  10. Profile photo of mraby
    August 8, 2012 at 3:38 pm —

    Great post! As a historian of science, I have to admit how excited I was to see you citing Donna Haraway. This also makes me want to recommend Sandra Harding’s work on the concept of “strong objectivity.” An oldie but goodie:

  11. Profile photo of John Moeller
    August 8, 2012 at 3:54 pm —

    Thanks for this. It’ll be great to refer back to.

  12. Profile photo of Larkness Monster
    August 8, 2012 at 4:34 pm —

    As a methodological pluralist, I thank you very sincerely for this excellent post.

  13. Profile photo of Jai Withani
    August 8, 2012 at 5:59 pm —

    Dissenting. Quantitative data is extremely important because it’s more resistant to our implicit biases; problems start to arise when people are inconsistent about the importance and pertinence of quantitative information depending on what’s being discussed/whether-they-agree-with-it-or-not.

    Qualitative and anecdotal evidence are still evidence, but should probably be regarded as _much weaker_ evidence, not just in determining scope, but in reliability. And, of course, not all qualitative evidence is equal. I’m more likely to be skeptical of qualitative evidence from a popular news article – where the unusual tends to be selected for – than in something I hear from multiple friend or colleagues. More generally, I’m extremely wary of self-selection effects when interpreting individual stories drawn from a large population.

    JAQing and refusing-to-acknowledge-problems-without-peer-reviewed-evidence-in-a-top-tier-journal are irritating and harmful – because they’re _inconsistent_.

    I’m a feminist largely because of a slight obsession with quantitative data. I take the results of the Implicit Association Test over many subjects as much, much stronger evidence of deeply-embedded sexism and racism than any single story. And the same goes for any number of studies demonstrating that women/minorities/non-heteronormative/queer people face discrimination. Quantitative data tells me that my social circle is likely extremely non-representative about attitudes towards feminism and women and general – that the situation is significantly worse than I would infer from qualitative evidence alone.

    Much of my obsession with quantitative data – and I think this goes for a lot of skeptics – is at least in part a reaction to the fact that I (and people in general) am _intuitively_ much more responsive to a vivid story than a bar graph. I try to counter this by emphasizing quantitative evidence over anecdotes when the two are in conflict, and seeking out quantitative data when it isn’t available.

    The absence of quantitative data does not mean that a phenomenon doesn’t exist – especially when there’s reason to believe that evidence is hard to come by. And I will stand shoulder-to-shoulder with anyone against those who would hold any proposition they disagree with to an absurdly-and-arbitrarily-high standard of evidence. But I will continue to be extremely skeptical of any given piece of qualitative or anecdotal data to be representative or particularly informative about the world.

    Also (as an anecdote!), I notice MRAs frequently using anecdotes to justify their absurdism. And this is part of why I’m so skeptical of anecdotes in general – with 7 billion people experiencing things, filtered through extremely unreliable brains, and filtered again by media outlets targeting interest-to-audience rather than accuracy, you can find a large number vivid stories supporting almost ANY thesis.

    • Profile photo of Will
      August 8, 2012 at 6:21 pm —

      I actually don’t really disagree with your overall point. But, again, you’re overlooking my point which is that the types of methodologies that are most useful depend on the questions that are being asked. Ideally, we should use mixed quantitative and qualitative methods whenever possible.

      Quantitative data is extremely important because it’s more resistant to our implicit biases; problems start to arise when people are inconsistent about the importance and pertinence of quantitative information depending on what’s being discussed/whether-they-agree-with-it-or-not.

      Let’s stick to social science research, because that’s what I’m more familiar with and because we’re talking about social issues.

      How are quantitative methods more resistant to our (who is “our”? the researcher? the informants/subjects?) implicit biases? The ways that questions are framed in surveys are certainly subject to implicit biases. Perhaps you mean the sampling methods? But this gets back to what I was saying about the problem of trying to generalize from qualitative data (which is not usually done unless there is a huge amount of qualitative data).

      Qualitative and anecdotal evidence are still evidence, but should probably be regarded as _much weaker_ evidence, not just in determining scope, but in reliability.

      Depends on the research question.

      And, of course, not all qualitative evidence is equal. I’m more likely to be skeptical of qualitative evidence from a popular news article – where the unusual tends to be selected for – than in something I hear from multiple friend or colleagues.

      Just as I’m more likely to be skeptical of quantitative evidence from a popular news website where they conduct polls. But that’s not really what I’m talking about here, now is it?

      I take the results of the Implicit Association Test over many subjects as much, much stronger evidence of deeply-embedded sexism and racism than any single story. And the same goes for any number of studies demonstrating that women/minorities/non-heteronormative/queer people face discrimination.

      Once again, it depends on the question being asked. Quantitative data certainly is useful when looking at how deeply embedded sexism/racism are in a society or group. And seeing how widespread these issues are is certainly convincing that there’s a problem. But it doesn’t tell us anything about the experiences of those people. If that’s something you don’t find important, then that’s fine! But when people like Greta Christina make a call for people to share their stories of racism and someone comes in and says “anecdotes are useless,” that is a privilging of quantitative data when it is unnecessary. Because the question isn’t “does this problem exist?” or “how widespread is this problem.” The question is “what is your experience with racism?”

      But I will continue to be extremely skeptical of any given piece of qualitative or anecdotal data to be representative or particularly informative about the world.

      As you should because qualitative data should not be seen as representative or generalizable. That’s not what it’s for.

      I notice MRAs frequently using anecdotes to justify their absurdism. And this is part of why I’m so skeptical of anecdotes in general

      That’s kind of a weird reason to be skeptical of anecdotes…MRAs also use English and WordPress to justify their stupidity, and yet here we sit using English and WordPress to communicate. Just because some group of people uses anecdotes doesn’t mean they’re inherently bad. Isn’t that some kind of fallacy?

      with 7 billion people experiencing things, filtered through extremely unreliable brains, and filtered again by media outlets targeting interest-to-audience rather than accuracy, you can find a large number vivid stories supporting almost ANY thesis.

      Sure, and that’s why I favor the methods of the discipline I’m in: anthropology. We use ethnographic methods, which utilizes both talking to people and observing people (participant observation). It’s still qualitative research.

      It just feels like you’re comparing apples and oranges a little bit here. You keep talking about qualitative data as if there is no rigor in the data collection process while there is always rigor in the data collection process of quantitative methods. That’s simply not the case.

    • Profile photo of dr. dr. professor
      August 8, 2012 at 9:52 pm —

      Well all that aside. A harassment policy should be in place and be enforced even if there are 0 pieces of qualitative data.

      The ridiculous thing here is that some people are saying “we won’t put a harassment policy in place unless we get overwhelming quantitative evidence”.

      That’s realllllly stupid and sexist.

  14. Profile photo of davew
    August 8, 2012 at 8:28 pm —

    I read Will’s piece twice and to be honest there are parts I still don’t follow. I agree with the central thesis, but I am still fuzzy on how to apply it. It appears a lot like special pleading. “The problem/question I want to address is not subject to quantitative data therefore by asking for data you are being a troll or derailing or otherwise being disingenuous.” It is true some trolls will come across like this, but so will some people with honest disagreements. I do not get a strong sense of which questions/problems are subject to which set of rules.

    To bring this home let’s try a bizarre hypothetical. “Homeland security should devote a portion of it’s budget to preventing alien abductions and counseling the victims. We don’t have any data to support this plan, but we have lots of testimony. Plus we believe that Homeland Security can address this without distracting from its other functions.”

    I do not believe alien abductions are real and I do believe the problems discussed here are real. Objectively, however, I see no difference. This is just my bias, personal experience, and yes anecdote. On the other hand I have no problem dealing with problems such as discrimination in a purely subjective ways. Skeptical organizations should address issues of discrimination solely because their members want them to. They don’t address alien abductions because their members don’t want them to. This should be good enough. I don’t see any reason to try to justify this by trying to dress up the subjective as logical and by doing so we start to sound a bit like psychics and homeopaths.

    • Profile photo of Will
      August 8, 2012 at 8:46 pm —

      The problem/question I want to address is not subject to quantitative data therefore by asking for data you are being a troll or derailing or otherwise being disingenuous.

      Yeah, I never said that. I’d appreciate it if you’d not put words in my mouth.

      What I said was that whether or not quantitative data or qualitative data are useful depends upon the question being asked.

      So when someone puts out a call for the experiences of people with racism/sexism, it’s completely stupid to respond with “those are anecdotes and don’t prove anything.” Because nothing’s trying to be proved by asking about people’s experiences.

      I do not get a strong sense of which questions/problems are subject to which set of rules.

      It depends on the question. it depends on the question. It depends on the question. How many times do I have to say it?

      There is no official list of which questions/problems are subject to which methodologies. Different questions/problems are better suited towards different methods. I’m not going to make some list as if I know what methods are best for every question. This is not a call to categorize all subjects into either quantitative or qualitative. It’s a call for people to stop pretending that quantitative is always the best way to go.

      To bring this home let’s try a bizarre hypothetical. Blah blah blah alien abduction blah blah blah.

      Are you fucking kidding me with this? Social justice issues are in no way the same thing as alien abduction, nor should they be thought of as required the same standard of proof.

      I do not believe alien abductions are real and I do believe the problems discussed here are real.

      Then why do you bring it up?

      Objectively, however, I see no difference.

      Objectively, my ass. One of the most blindly privileged people around these parts thinks he’s being objective. SHOCKER!

      You’re doing exactly what I’m talking about where you think you’re somehow above the rest of humanity with your biases.

      This is just my bias, personal experience, and yes anecdote.

      Which is it? Are you being objective or sharing your biased anecdote? You’re making no sense at all.

      I don’t see any reason to try to justify this by trying to dress up the subjective as logical and by doing so we start to sound a bit like psychics and homeopaths.

      I see. So by doing qualitative research, I am completely illogical and no different from a psychic and a homeopath.

      And I’m the one being irrational and illogical?? You’re a piece of work.

  15. Profile photo of davew
    August 8, 2012 at 9:16 pm —

    Are you fucking kidding me with this? Social justice issues are in no way the same thing as alien abduction, nor should they be thought of as required the same standard of proof.

    Why not? Each has a victim who claims harm and has a story to back it up. I don’t want to put words in your mouth, but are you saying stories of alien abduction require data, but social injustice does not?

    To be clear I agree that social injustice is a problem and I agree with most of the remedies put forward at Skepchick. I just don’t see how your rationalization applies.

    There is no official list of which questions/problems are subject to which methodologies. Different questions/problems are better suited towards different methods. I’m not going to make some list as if I know what methods are best for every question. This is not a call to categorize all subjects into either quantitative or qualitative. It’s a call for people to stop pretending that quantitative is always the best way to go.

    If you refuse to apply your general outlook to specific issues especially to those addressed here how do I know they apply at all?

    • Profile photo of spokesgay
      August 8, 2012 at 9:53 pm —

      You’re so full of shit it’s coming out of your keyboard. Do not tell me that you don’t understand that:

      A. Claims of alien abduction are extraordinary and bear the highest burden of proof

      B. Claims of sexual harassment are mundane and completely believable

      . . and expect me to believe you don’t see a difference and that you think both require equal skepticism.

      I don’t believe you. So I want to know what your real motivation is.

    • Profile photo of Will
      August 8, 2012 at 10:02 pm —

      In addition to what Josh said…

      I don’t want to put words in your mouth, but are you saying stories of alien abduction require data, but social injustice does not?

      No. The fact that you think that’s what I’m saying either indicates you don’t know how to read or that you have an ulterior motive. Maybe there’s other options, too. I’d hate to box you into a false dichotomy of stupidity vs. malice. But my entire post was saying that there are other types of data besides quantitative data. Meaning that there is both quantitative and qualitative data on sexism/racism, and that people should stop poopooing on the qualitative data because it is important, too.

      If you refuse to apply your general outlook to specific issues especially to those addressed here how do I know they apply at all?

      Honestly, I don’t really understand what you’re saying here. You’re not making any sense.

      • Profile photo of davew
        August 9, 2012 at 12:50 am —

        But my entire post was saying that there are other types of data besides quantitative data. Meaning that there is both quantitative and qualitative data on sexism/racism, and that people should stop poopooing on the qualitative data because it is important, too.

        So you didn’t like my hypothetical. I didn’t like it much either, but it was the best I could do in the time available. I appreciate the difference between quantitative and qualitative and they are both important. The fact that you wrote here, however, and received such a welcoming response must mean that your approach has some bearing on skepticism or feminism. Does the qualitative approach apply to anything we have been discussing here recently? If it does how would you apply it? (I realize that I assumed before that it is directly applicable to current discussions. I should not have jumped to this conclusion.)

        • Profile photo of spokesgay
          August 9, 2012 at 1:53 am —

          More JAQ’ing off from you. And in Word Salad 2010.

          “Does the qualitative approach apply to anything we have been discussing here recently? If it does how would you apply it?”

          Dude, are you not reading what anyone here writes? Seriously. Are you a Turing Test? You respond as if you have no comprehension whatsoever of the people you purport to be responding to. It’s quite literally like talking to an automaton. I have *no* idea what you’re asking or why you’re asking it.

          It’s as if someone said, “Lots of people remark that butter has a more full mouth-feel than margarine.”

          And you respond, “But are they talking about butter? I realize that margarine is not butter, but how would the texture of butter apply here?”

          Literal non sequitur.

        • Profile photo of marilove
          August 10, 2012 at 10:27 am —

          It is official. Your Devil’s Advocate shit is tired.

  16. Profile photo of dr. dr. professor
    August 8, 2012 at 10:02 pm —

    Why not? Each has a victim who claims harm and has a story to back it up. I don’t want to put words in your mouth, but are you saying stories of alien abduction require data, but social injustice does not? To be clear I agree that social injustice is a problem and I agree with most of the remedies put forward at Skepchick. I just don’t see how your rationalization applies.

    Seriously dude? It’s because discrimination on race/sex/sexuality happens daily, and there’s plenty of research and data to back it up. Alien abductions are not proven and there’s no data to back it up.

    Seriously, you win a Herp Derp award for that statement. Does the following patronizing meme point out why what you said is missing the point?

  17. Profile photo of LOLogies
    August 8, 2012 at 10:38 pm —

    You make some very salient points here about the usefulness of qualitative data in helping us get a rounded view of an issue, but I feel like a few places are a little circular and you are throwing up some strawmen.

    “What this means is that multiple qualitative studies are important for understanding the underlying causes of problems more broadly. In other words, we need more than one qualitative study to be able to generalize.”

    …Is this not quantitative data, then? If you release a survey asking for a target group’s feedback, then continue to survey multiple groups chosen at random and collect their feedback, does that not become quantitative?

    “Argument: “Humans are flawed and thus anecdotes and testimonials are not completely accurate; therefore, anecdotes are invalid and tell us nothing.””

    This seems to me a bit of a strawman and a disingenuous portrayal of the potential problems with relying too heavily on anecdotes alone.
    It’s not that the possible lack of accuracy means they’re invalid and “tell us nothing”, it means we need to exercise skepticism towards these claims with human fallibility (and the common lapses in our memory) in mind. “Tells us nothing” would mean there is 0% accuracy, but that’s still not the same as a “lack of accuracy”. 0% is a lack of accuracy, but a lack of accuracy doesn’t mean 0%, if that makes sense.

    Hell, even if there’s 0% accuracy, it may still spring up some interesting ideas about WHY we’re getting such a low percentage, so it could still tell us *something*.

    Finally, just a small little bit that bothered me:
    “However, if your question is “what are some of the ways that people have experienced racism/sexism?” then anecdotal data is absolutely helpful.”

    Well, yes, because essentially this is a redundant situation. Anecdotes are individualized accounts. Asking for individualized accounts of sexism is anecdotal. Therefore, receiving anecdotes when you’re asking for anecdotes is…pretty obviously helpful…?

    • Profile photo of Will
      August 9, 2012 at 1:54 am —

      “What this means is that multiple qualitative studies are important for understanding the underlying causes of problems more broadly. In other words, we need more than one qualitative study to be able to generalize.”
      …Is this not quantitative data, then? If you release a survey asking for a target group’s feedback, then continue to survey multiple groups chosen at random and collect their feedback, does that not become quantitative?

      Yes, then it becomes quantitative. Because, as I’ve said, qualitative data is not meant to be generalized. Which is why I say that if you want to understand the underlying causes more broadly, then you will need to use quantitative data.

      “Argument: “Humans are flawed and thus anecdotes and testimonials are not completely accurate; therefore, anecdotes are invalid and tell us nothing.””
      This seems to me a bit of a strawman and a disingenuous portrayal of the potential problems with relying too heavily on anecdotes alone.

      I never advocated relying on anecdotes alone. I advocated not jettisoning them from our discussions. And that argument is not a straw man, it is an amalgamation of many arguments against anecdotes that you would see if you clicked and read the links I provided (particularly someone named Emil on Greta Christina’s blog). Sure, no one single person said that exactly quote, but it’s a fair representation of a common argument among skeptics.

      It’s not that the possible lack of accuracy means they’re invalid and “tell us nothing”, it means we need to exercise skepticism towards these claims with human fallibility (and the common lapses in our memory) in mind. “Tells us nothing” would mean there is 0% accuracy, but that’s still not the same as a “lack of accuracy”. 0% is a lack of accuracy, but a lack of accuracy doesn’t mean 0%, if that makes sense.

      It looks like a little bit of pedantry to me, but okay I’ll grant you that. If you’d like I will amend my original statement to end with “almost nothing.” Better? We should exercise skepticism toward everything, but that means thinking critically, not just straight up cynicism.

      Finally, just a small little bit that bothered me:?“However, if your question is “what are some of the ways that people have experienced racism/sexism?” then anecdotal data is absolutely helpful.”
      Well, yes, because essentially this is a redundant situation. Anecdotes are individualized accounts. Asking for individualized accounts of sexism is anecdotal. Therefore, receiving anecdotes when you’re asking for anecdotes is…pretty obviously helpful…?

      Sure, and these anecdotes can tell us a lot about how sexism and racism operates in a given setting/group. Is it completely 100% accurate? No. Is it reliable? Maybe. Perhaps. Also, how do you get the quantitative data if not through people reporting anecdotal information on surveys? And this is why in anthropology we do participant observation such that we don’t just rely on people’s stories (though some anthropologists do great narrative work) but we infuse our own observations of their lived experiences.

      It still depends on the question being asked as to whether qualitative or quantitative data are better suited to the task. If you’re more comfortable with quantitative, then by all means frame your questions in such a way that they’re best answered by quantitative data. I’ll go with mixed methods to try to get at both breadth and depth as much as possible.

      • Profile photo of spokesgay
        August 9, 2012 at 2:00 am —

        Not to mention the fact that when we ask how people experience hostility, chilly climates, and sexual harassment, the point is not to say “I’m skeptical that you feel that way,” or “Thank you, but I’d like more quantifiable data.”

        Honestly. Do you “skeptics” even understand what it means to take on board the lived experience of people? If you give the tiniest shit about how minorities or traditionally oppressed people *experience* these issues, then what the fuck else are you asking for?

        Do you want to know how they feel or not? Do you want to address the feeling of hostility or not? Do you care more about correcting the situation or about dissecting and quantifying peoples’ reported experience to see if it lives up to your standards of Reasonable Response to Stimuli?

        Shorter: I’m skeptical that you should feel that way. More data please.

        We’re Martians and Earthlings talking to each other. It’s fucking insane.

        • Profile photo of Will
          August 9, 2012 at 2:03 am —

          ^^ This!!

        • Profile photo of dr. dr. professor
          August 9, 2012 at 2:57 am —

          Awesome Spokesgay.

          And I just have another question.

          Isn’t it just common fucking decency to have anti-harassment measures in place at events and organizations? I don’t get why any respectable organization wouldn’t just have those in place in the first place.

          It has to become such a huge problem that there needs to be a large set of data showing discrimination before anything is done about it? Seriously WTF IS THAT?

          Most US companies have anti-discrimination/harassment policies. But somehow, many of the groups within the skeptic/atheist movement that consider themselves super enlightened don’t implement these policies on account of there’s “not enough data to point to a problem???”. Seems a lot of “enlightened” skeptics are no better in some ways than the fundamentalist Christians they trash talk.

  18. Profile photo of spokesgay
    August 9, 2012 at 2:06 am —

    Will, I’m just damned tired of this bullshit. It’s the very definition of an ill-posed question. The hyper skeptics are asking things totally orthoganal to the issue. It’s bizarre. And it’s stupid.

  19. Profile photo of davew
    August 9, 2012 at 9:43 am —

    Literal non sequitur.

    To recap.

    Is my hypothetical relevant? “No.”

    Can you demonstrate with any real-world examples what is being discussed in generalities above? No.”

    Does this topic have anything to do with feminism or skepticism? “This question makes no sense.”

    Did I get the gist of it?

    I don’t even have a point of view on this yes. I’m just trying to figure how to apply Will’s abstractions to real world scenarios so I can understand his thesis. I have a hard enough time dealing with abstractions in my own field.

    Maybe I misread Will’s piece and it really is pure abstraction with no real-world application. If so then I cop to my poor reading skills and I’ll be on my way. My interest in Skepchick does not extend to philosophical discussions about the social sciences.

    • Profile photo of dr. dr. professor
      August 9, 2012 at 10:03 am —

      Meh, all will is saying is the methodology used to form an appropriate response to an issue or question DEPENDS ON THE ISSUE OR QUESTION.

      When a woman says “I’m being harassed, I would like a policy to deal with anyone who harasses me”, that should be a no-fucking-brainer shouldn’t it? Harassment is unacceptable period and there should be a problem to STOP it before it’s even a problem.

      And to have “Skeptics” (as they’re claiming they are) saying that giant mountains of quantitative data before any policies can be put in place is an indication that sexism is as alive in well in the Skeptic community as it is in Conservative Christian communities.

      So the point is that you’re missing the point in trying to dissect the mechanics of his argument. When the basic point is:

      SEXUAL HARASSMENT = ALWAYS BAAAAAAD
      POLICIES TO DEAL WITH IT = ALWAYS GOOOOOOOD

    • Profile photo of dr. dr. professor
      August 9, 2012 at 10:11 am —

      And also, no offense, but what you’re doing is what Will is rallying against. It’s getting too extreme with trying to determine inquisitive methodologies and asking for data.

      I mean, as an example, if you were a commander in a combat zone, would you wait for another armed group to attack you 1000 times so that you then had a large enough sample space to determine if they were really attacking you? Or would you believe your soldiers when they said “they are trying to fucking KILL US!!!!”?

      You’re digging way too deep and making it an abstraction. Stop doing that, accept that obviously sexism exists, and join the solution man, yeesh :).

  20. Profile photo of davew
    August 9, 2012 at 1:50 pm —

    Meh, all will is saying is the methodology used to form an appropriate response to an issue or question DEPENDS ON THE ISSUE OR QUESTION.

    I’d still like some examples of each, but I’m going to stop asking.

    And also, no offense, but what you’re doing is what Will is rallying against. It’s getting too extreme with trying to determine inquisitive methodologies and asking for data.

    Not in any sense. If I was going to explain the difference between quantitative and qualitative chemistry I could give endless examples without breaking a sweat. I think the reason the same thing isn’t happening here is Will’s thesis breaks down when examined. It is not general. It is special pleading for sexual harassment which doesn’t actually need special pleading.

    Harassment is unacceptable period and there should be a problem to STOP it before it’s even a problem.

    As I said above I agree with this.

    I don’t think this is going anywhere so if anyone posts any new insight I’ll happily read it, but I’m just starting to repeat myself which is boring and annoying even to me.

    Ta.

    • Profile photo of Will
      August 9, 2012 at 2:00 pm —

      Are you seriously asking me to give you a course in social science methodologies?? You can’t fucking google it for yourself and learn something? Here, you lazy schmuck: http://www.xavier.edu/library/help/qualitative_quantitative.pdf

      And I never even mentioned sexual harassment in my original post, so how in the everliving hell can my post be special pleading for sexual harassment??

      • Profile photo of Will
        August 9, 2012 at 2:05 pm —

        And just to show that mixed methods are used outside the social sciences, here’s a link to NIH Best Practices for mixed methods. Particularly of interest to you might be this part:

        The nature of qualitative research and its evidence: A salient strength of qualitative research is its focus on the contexts and meaning of human lives and experiences for the purpose of inductive or theory-development driven research. It is a systematic and rigorous form of inquiry that uses methods of data collection such as in-depth interviews, ethnographic observation, and review of documents. Qualitative data help researchers understand processes, especially those that emerge over time, provide detailed information about setting or context, and emphasize the voices of participants through quotes. Qualitative methods facilitate the collection of data when measures do not exist and provide a depth of understanding of concepts. Typical qualitative approaches used in health research are case studies, grounded theory, ethnography, and phenomenology.

        The nature of quantitative research and its evidence: Quantitative research is a mode of inquiry used often for deductive research, when the goal is to test theories or hypotheses, gather descriptive information, or examine relationships among variables. These variables are measured and yield numeric data that can be analyzed statistically. Quantitative data have the potential to provide measurable evidence, to help to establish (probable) cause and effect, to yield efficient data collection procedures, to create the possibility of replication and generalization to a population, to facilitate the comparison of groups, and to provide insight into a breadth of experiences. Typical quantitative approaches used in the health sciences are descriptive surveys, observational studies, case-control studies, randomized controlled trials, and time-series designs.

  21. Profile photo of davew
    August 9, 2012 at 5:24 pm —

    Will, credit where credit is due. I think you did an excellent job of summarizing this material in your original post.

    • Profile photo of Will
      August 9, 2012 at 5:31 pm —

      …thanks, I think? It’s so hard to tell if people are serious or sarcastic.

      And I want to apologize for calling you a lazy schmuck and being so abrasive. I have to be honest, sometimes your posts come across as extremely privileged and it gets frustrating really quickly. If you do seriously want to know more about mixed methods or social science methodologies, I’d be happy to talk about it more. It just came across as you being dismissive earlier and not as having a genuine interest in how rigorous qualitative data can be useful.

  22. Profile photo of davew
    August 9, 2012 at 5:58 pm —

    I have to be honest, sometimes your posts come across as extremely privileged and it gets frustrating really quickly..

    I agree. I’ll work on that. The only mild defense I have a hard time being clear without also being, yeah, abrasive is a good word for it.I promise to never use irony or sarcasm here. I don’t find it helpful.

    I have a strong feeling this topic will come up again in this forum a different context. Maybe then I’ll get a clearer understanding. My hard-science brain has always had a difficult time with the social sciences.

  23. Profile photo of emilkarlsson
    August 10, 2012 at 2:49 pm —

    Hi Will.

    I wrote a reply to this blog post in a entry at Debunking Denialism called “The Triumph of Modern Quantitative Science”.

    The general conclusions I reached was that:

    1. Will does not provide any evidence for your cult accusation, which was a weak attempt at a guilt by association.

    2. Will incorrectly framed the discussion as “does examples of racism exist in the skeptical community?” versus “how big is the problem of racism in the skeptical community” as “what are some of the ways that people have experienced racism/sexism?” versus “how many people experience racism/sexism?”.

    3. Will misrepresented my position that solutions need to be informed by an appreciation of the size and scope of the problem as the ridiculous claim that solutions are impossible without knowing the size and scope of the problem.

    4. Will attempted a bait-and-switch: my arguments where against the notion that anecdotes and testimonials are valid responses to racists who incorrectly believe that the size of the problem of racism is negligible. You made it appear as if it was an attack on qualitative studies, which it was not. Qualitative studies with a reasonable methodology transforms anecdotes to reliable data when the scientists are critically analyzing, interpreting and publishing their results.

    5. Will incorrectly characterized my criticisms of anecdotes as “Humans are flawed and thus anecdotes and testimonials are not completely accurate; therefore, anecdotes are invalid and tell us nothing.”, when the actual argument was that the major problems with testimonials and anecdotes is that (1) there is no independent evidence for accuracy and (2) they do not tell us how representative the descriptions are (i.e. to what extent they can be generalized to the population) and that these two factors need to be taken into account and that these are irrelevant when it comes to size and scope of a problem. Also, nowhere did I claim that anecdotes are “not completely accurate”. My claim was about epistemology: often there is no independent evidence for anecdotes, even when they are true. Anecdotes can very well be completely accurate, but that does not mean we necessarily have a clear-cut way of telling if the evidence supports that conclusion or not (and therefore if we should accept the anecdote or not). In other words, Will confused epistemology with ontology.

    6. The fact that well-defined margins of error appear in quantitative studies is a strong argument in their favor, while the the lack of independent corroboration and lack of knowledge of representativeness of an anecdote is a strong argument against it. Both qualitative and quantitative studies will have limitations and inaccuracies, we have to understand that there is a huge difference between the level of uncertainty between quantitative results with well-defined error bars and the average quantitative study (present one with 95% accuracy!) and that the latter generally do not, and indeed usually cannot in practice, even attach a margin of error to their raw data to begin with.

    7. When Will attempted to explain why his opponents reached the conclusion they did and advance it as an argument against their position, Will performed the fallacy of argumentum ad hominem circumstantial. The reason people have for holding a position has nothing about the merits of that position.

    8. Will forgot to describe the fact that qualitative studies are usually more subjective than quantitative, because qualitative studies is often based on the subjective opinions of the subjects that may be poorly understood and analyzed because of subjective misrepresentations by the researcher. In quantitative studies, on the other hand, there is operationalization of variables (and therefore an unambiguous way to measure them) and replicates. Quantitative studies are not perfect and have their own set of limitations (so the explanation of why some prefer qualitative studies is another straw man), but there is an important difference in degree of subjectiveness.

    9. In defense of Will, he did make a very good point that being a skeptic does not entail that one is rational. Unfortunately, this does not outweigh the rest of the content in what can only be described as a poorly argued post. Before criticizing a position, make sure you actually understand the argument.

    Thanks.

    • Profile photo of mrmisconception
      August 10, 2012 at 3:50 pm —

      Wow, so Emil disagrees. I guess that’s over, how can we argue with a hyper-rational outlook like that.

      Oh yeah, social justice =/= science. Gee, that was easy.

    • Profile photo of dr. dr. professor
      August 10, 2012 at 4:02 pm —

      You know how your post reads to me?

      “BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA BLA”

      The point is, assholes are stalling doing anything about sexism and using “skepticism” of the issue as a justification for it.

      What’s the problem with addressing sexual discrimination before it even happens? No, rather than spending a few hours writing an anti-harassment policy, people spend days and days writing long neckbearded responses (that make little sense) to anti-sexism writings.

      Your post is one of those.

    • Profile photo of Will
      August 10, 2012 at 4:43 pm —

      Let me start off by saying this: Wow, what an ego on you! That you could think my entire post was all about you! Sorry to disappoint, but it’s not. You’re merely one small piece of what I’m talking about in this post. I’m getting at general trends, not specifically targeting you. If I was doing that, I’d have mentioned you by name and used your exact wording and not said things like “themes I notice in the skeptical community” and “arguments I’ve seen around the skeptical blogosphere.” I also clarified in the comments below the article that these arguments are really amalgamations of arguments I’ve seen/heard over time.

      So, no, this post is not all about you.

      Also, I’m not going to take as much time to respond to any future comments on this thread as I’ve made my point and I will just let respectful disagreements stand and let other commenters respond if they wish. But I felt I should at least respond to you since I did mention you by name (EDIT TO ADD: in the comments section below the article) and point to a couple of blog posts about you.

      On to your points:

      1. Will does not provide any evidence for your cult accusation, which was a weak attempt at a guilt by association.

      It’s a rhetorical device. I am not saying there is a literal cult that worships quantitative research. I was using hyperbole and metaphor.

      2. Will incorrectly framed the discussion as “does examples of racism exist in the skeptical community?” versus “how big is the problem of racism in the skeptical community” as “what are some of the ways that people have experienced racism/sexism?” versus “how many people experience racism/sexism?”.

      Uh, no. I’m pointing out that those are different questions and when people are talking about their experiences of sexism/racism, demands for quantitative evidence are unnecessary at best (and hurtful at worst). And that’s exactly what you did.

      For example, your exact response to Greta Christina’s call for people to share their personal stories about experiencing racism was: “more scientific evidence, less anecdotes.” So it’s not me that’s trying to re-frame the discussion. It’s you.

      Your own words: “I am insisting that the discussion should be framed around the size of the problem, not merely it’s existence.”

      More of your words: “My general point has been that it is much more productive to talk about the size of the problem than if it merely exists or not and that anecdotes tell you nothing about the size.”

      So it’s not that I’m “incorrectly” framing the discussion—it’s that I am not framing it the way you want it framed. And you don’t get to decide how marginalized people frame their discussions of social justice.

      3. Will misrepresented my position that solutions need to be informed by an appreciation of the size and scope of the problem as the ridiculous claim that solutions are impossible without knowing the size and scope of the problem.

      Yeah, this entire post wasn’t written about you. Your exchanges with Greta and Ian were just tiny little pieces of a larger mosaic that makes up this obsession with objectivity and quantitative data. I mentioned you by name once in a comment, and it was in response to the question of whether anecdotes are useful data. It is clear to me that you feel that they are not. When you title your blog post “The Plural of Anecdote is not Scientific Evidence,” that’s a pretty clear indication of your feelings about anecdotes.

      Another time I mentioned Greta’s post about your comment was in this recent exchange where a commenter tried to re-frame the discussion by JAQing off: “To clarify I’m looking for data supporting the notion that women face wide spread abuse and threat from the skeptical community.” Which is similar to what you did to Greta (though you weren’t JAQing off, you were just straight up demanding people talk about the topic of racism in the way you felt it is best framed).

      And speaking of misrepresentation, the only use of the word “impossible” on this page is in your comment. I never used the word impossible. Sure, I could have put a qualifier in my original post: “We cannot advance any real/effective solutions…” Is that not your position?

      4. Will attempted a bait-and-switch: my arguments where against the notion that anecdotes and testimonials are valid responses to racists who incorrectly believe that the size of the problem of racism is negligible.

      Those are arguments you injected into the conversation on Greta’s blog. Her question was related to oppressed peoples being blamed for playing the victim card and how they should proceed. Her post had absolutely nothing to do with racists who believe racism is negligible. It had to do with the double bind put upon marginalized people.

      You made it appear as if it was an attack on qualitative studies, which it was not. Qualitative studies with a reasonable methodology transforms anecdotes to reliable data when the scientists are critically analyzing, interpreting and publishing their results.

      So then your premise that “the plural of anecdote is not scientific evidence” is wrong then? Anecdotes can serve as scientific data? Or are you maintaining that somehow anecdotes magically become something else when they’re collected in qualitative research?

      5. Will incorrectly characterized my criticisms of anecdotes as “Humans are flawed and thus anecdotes and testimonials are not completely accurate; therefore, anecdotes are invalid and tell us nothing.”,

      I certainly did not say: “Emil said this…” in my original post. So I was not “incorrectly characterizing” your criticisms of anecdotes, I was speaking more broadly, as I indicated in my response to LOLogies. I agree, you did not say that anecdotes tell us nothing. But as I commented over on Greta’s blog (which you never replied to), it comes across as if you are demanding some quantitative data such that you can determine the extent of racism (in this case) and until such a time, valuable solutions cannot be enacted. And so while you may not say exactly what I posed as an argument above, it reads as though that’s what you’re doing.

      Now, you can certainly disagree with my interpretation, but I’m not the only person who read your comments this way, so it behooves you to clarify, which you have (somewhat) done in this comment here.

      when the actual argument was that the major problems with testimonials and anecdotes is that (1) there is no independent evidence for accuracy

      I don’t really disagree. I even addressed accuracy vs. reliability in my post. However, I would add that you can get independent evidence for the accuracy of anecdotal evidence through observation. But if it’s something that’s already happened that someone is providing an anecdote about, then yes I agree.

      and (2) they do not tell us how representative the descriptions are (i.e. to what extent they can be generalized to the population)

      I don’t disagree. And I have addressed this, too.

      and that these two factors need to be taken into account and that these are irrelevant when it comes to size and scope of a problem.

      Once again, it depends on the question being asked. If your question is about size and scope, sure. But that’s not the question that was being asked when you responded to Greta Christina. It’s also not the topic when most hyper-rational people start making demands for quantitative evidence to back up people’s experiences of sexism/racism.

      Also, nowhere did I claim that anecdotes are “not completely accurate”.

      Nowhere did I claim that you claimed that. At least, not that I can recall. If I used your name and quoted you as saying that, please point out where I did so and I’ll gladly retract that statement.

      My claim was about epistemology: often there is no independent evidence for anecdotes, even when they are true. Anecdotes can very well be completely accurate, but that does not mean we necessarily have a clear-cut way of telling if the evidence supports that conclusion or not (and therefore if we should accept the anecdote or not).

      I don’t disagree. And my post doesn’t disagree because that’s not really the issue I’m discussing.

      In other words, Will confused epistemology with ontology.

      How’s that, if I’m not disagreeing with you? Please elaborate on this point, because I think it’s bullshit.

      6. The fact that well-defined margins of error appear in quantitative studies is a strong argument in their favor, while the the lack of independent corroboration and lack of knowledge of representativeness of an anecdote is a strong argument against it. Both qualitative and quantitative studies will have limitations and inaccuracies, we have to understand that there is a huge difference between the level of uncertainty between quantitative results with well-defined error bars and the average quantitative study (present one with 95% accuracy!) and that the latter generally do not, and indeed usually cannot in practice, even attach a margin of error to their raw data to begin with.

      I’m not fucking arguing against quantitative research. I’m arguing for people to recognize the value in qualitative research. This is not a mutually exclusive proposition. I’m arguing that people should recognize the strengths and weaknesses of both kinds of research, and that the types of questions people are asking determines what types of methods should be used. That’s it.

      What you’re doing here is exactly the thing I’m complaining about, which is this defensiveness and overpriviliging of quantitative data regardless of the questions or topic.

      7. When Will attempted to explain why his opponents reached the conclusion they did and advance it as an argument against their position, Will performed the fallacy of argumentum ad hominem circumstantial. The reason people have for holding a position has nothing about the merits of that position.

      What opponents? What part of my argument, exactly are you talking about? Be specific.

      8. Will forgot to describe the fact that qualitative studies are usually more subjective than quantitative, because qualitative studies is often based on the subjective opinions of the subjects that may be poorly understood and analyzed because of subjective misrepresentations by the researcher.

      I did? I thought that was the whole fucking point of my conclusion. It is possible that I was unclear, but so far you’re the only person who seems to have missed it.

      In quantitative studies, on the other hand, there is operationalization of variables (and therefore an unambiguous way to measure them) and replicates.

      Oh, I see. No subjective bias in quantitative data collection. Operationalization of variables happens in a vacuum free of any other contexts. All variables are readily apparent and visible. Got it. *eyeroll*

      I will speak directly to what I know best: ethnographic research. Because ethnographic research takes place in natural settings (as opposed to laboratories), situations are not replicable in any exact sense. As LeCompte & Goetz note: “Problems of uniqueness and idiosyncrasy can lead to the claim that no ethnographic study can be replicated. However, generation, refinement, and validation of constructs and postulates may not require replication of situations. Moreover, because human behavior is never static, no study can be replicated exactly, regardless of the methods and designs employed” (p. 35).

      There will always be a subjective bias regardless of methods because human beings are temporally and materially situated in the world and are not omniscient, omnipotent, omnipresent beings. Like I said, we can successfully minimize bias to a great degree, but we are nonetheless always in a position of situated knowledge.

      Quantitative studies are not perfect and have their own set of limitations (so the explanation of why some prefer qualitative studies is another straw man), but there is an important difference in degree of subjectiveness.

      I’m glad you can at least admit that quantitative methods are not a panacea. Honestly, that’s something that the hyper-rationalists rarely do.

      And of course there are different degrees of subjectivity—I would never deny that, and I doubt any qualitative researcher would. But those degrees are an asymptote towards objectivity. What I’m arguing is that we should not be dismissing the subjective as unimportant when discussing racism/sexism. That’s all. And whether or not you meant to, that’s exactly what you did in your response to Greta Christina.

      9. In defense of Will, he did make a very good point that being a skeptic does not entail that one is rational. Unfortunately, this does not outweigh the rest of the content in what can only be described as a poorly argued post. Before criticizing a position, make sure you actually understand the argument.

      Emphasis for irony.

    • Profile photo of marilove
      August 13, 2012 at 11:29 am —

      “Hi Will.”

      First of all, you spelled his name wrong.

      Second of all, is there some sort of MRA Troll Rule List? Because they all start “Hi (nym)”. I’ve mentioned this before. IT IS SO WEIRD.

      • Profile photo of Will
        August 13, 2012 at 11:33 am —

        Nah, he spelled my name right. ;)

        But yeah, it was weird to address the comment to me and then continue to talk about me. He clearly wasn’t really interested in dialogue judging by the comment below. He really just wanted to come on and spout some straw-vulcan-hyper-rational bs.

        • Profile photo of marilove
          August 13, 2012 at 11:55 am —

          I read it as one “L” the first time around… IGNORE. I was on my Fire, lol. :)

  24. Profile photo of dr. dr. professor
    August 10, 2012 at 4:06 pm —

    6. The fact that well-defined margins of error appear in quantitative studies is a strong argument in their favor, while the the lack of independent corroboration and lack of knowledge of representativeness of an anecdote is a strong argument against it. Both qualitative and quantitative studies will have limitations and inaccuracies, we have to understand that there is a huge difference between the level of uncertainty between quantitative results with well-defined error bars and the average quantitative study (present one with 95% accuracy!) and that the latter generally do not, and indeed usually cannot in practice, even attach a margin of error to their raw data to begin with.

    Oh look at you, you’re so smart and correct!!! You feel like a big boy now?

    Dude, if several women say they’re being harassed, you write a policy to prevent it, not write 9 paragraphs of pseudo-intellectual bullshit.

    • Profile photo of emilkarlsson
      August 10, 2012 at 4:43 pm —

      Where did I say I object to anti-harassment policies? If they are enforced, they are probably very successful and therefore make sense.

      You claim that what I write is “pseudo-intellectual bullshit”. Yet you do not back this up with an argument or provide any evidence. In other words, you are committing the fallacy known as argumentum ad lapidem.

      I think you need to take a step back, take a breath, and read what I actually wrote before you attempt to address it.

      Thanks.

      • Profile photo of Will
        August 10, 2012 at 4:46 pm —

        I think you need to take a step back, take a breath, and read what I actually wrote before you attempt to address it.

        This is not acceptable, do not do it again.

        • Profile photo of emilkarlsson
          August 10, 2012 at 4:51 pm —

          You believe it is unacceptable to point out that someone is wildly misrepresenting my position and provide suggestions for how the conversations could proceed in a more productive way?

          Wow.

          At any rate, I think it is clear that you are not concerned with actually having a productive discussion about the actual arguments as you evade my arguments without replying and quote me out of context several times.

          I will probably not bother interacting with you again.

          • Profile photo of Will
            August 10, 2012 at 4:55 pm

            No, it is unacceptable to point out someone’s emotional state as clouding their judgment. You’d understand why if you actually clicked and read the link I provided.

            And if you choose not to respond to me, that’s fine. No skin off my back. But I think it’s funny that you think I’m not interested in having a productive discussion as I just spent the last hour crafting a response to you. And your response is “I guess you don’t want to talk.”

            No, serious, at this point, just go away. I really do not want to talk now.

  25. Profile photo of dr. dr. professor
    August 10, 2012 at 6:42 pm —

    At any rate, I think it is clear that you are not concerned with actually having a productive discussion about the actual arguments as you evade my arguments without replying and quote me out of context several times.

    You know, this is EXACTLY what women feel when they try to bring up sexist actions against them.

    They find that people don’t actually want to have a productive discussion about the actual arguments (i.e. they want to be right, they don’t actually care about what women experience).

    So now you kinda know how it feels.

  26. Profile photo of Otoki
    August 12, 2012 at 5:11 pm —

    Oh my fuck. The ridiculous amounts of privilege.

Add Comment Register



Leave a reply