AI: What Is Scientific Literacy?

AI: What Is Scientific Literacy?

Last week we had some fun with this quiz posted by The Christian Science Monitor, which is meant to test one’s level of scientific literacy.

After braving the slow-load issues of the test’s site, most of you who commented in the Afternoon Inquisition, reported logging very good scores. And of course, we expect nothing less from you all.

However, many commenters pointed out (and rightly so) that the quiz actually tests retention of facts more than it tests scientific literacy. And this raised further interesting questions that we felt might make a good discussion for today.

So let’s take a step back and start at the beginning:

What exactly is scientific literacy? How should it/can it be measured? Does retention and regurgitation of factoids play any part in it? Can we begin to design a tool/method/quiz for measuring it here? Use the comments if you have any ideas.

The Afternoon Inquisition (or AI) is a question posed to you, the Skepchick community. Look for it to appear Tuesdays, Thursdays, Saturdays, and Sundays at 3pm ET.

 

Sam Ogden is a writer, beach bum, and songwriter living in Houston, Texas, but he may be found scratching himself at many points across the globe. Follow him on Twitter @SamOgden

20 Comments

  1. I’m a science librarian at a large university, and part of my job is to teach science literacy. We generally define it as the set of skills that a person uses to understand and process scientific information, not necessarily basic facts like the quiz.

    The Association of College & Research Libraries (ACRL, which is itself a subset of the American Libraries Association) actually has published standards for science literacy education. It’s very detailed, but you can get the gist by reading the 5 big standards.

    http://www.ala.org/acrl/standards/infolitscitech

  2. Commenting again to add: The same ACRL group that put together the standards I linked above has a wiki with lots of science literacy tutorials from different universities.

    http://wikis.ala.org/acrl/index.php/Science_Information_Literacy_Tutorials

    Granted, all of these resources are for university students, which is a pretty specific demographic, and I believe science literacy is important for everyone! These are just some of the things I’m most familiar with.

  3. I think that basic understanding of how the world works is important. Many of the questions on the quiz seemed to be testing knowledge of Latin or Greek more than basic science. I think questions like ‘An astronaut standing on the moon lets go of a screwdriver. What happens?’ or something like ‘Is there gravity on the moon?’ would be a good if very basic question. I have asked people around my campus that question and a it was answered incorrectly almost a quarter of the time. I am sure that coming up with simple questions like that for every branch of science would be very hard to do, but I think that would be at least a start.

  4. Asking specific questions about specific bits of knowledge seems pretty unhelpful. I knew plenty of people in college who could regurgitate facts and made straight As in honors level classes but seemed generally a little clueless.

    In my own experience at least, if you understand the material, the factoids just kind of get absorbed.

    There was an NPR story on similar lines to this recently, where some physicists tried to tackle that very question:

    http://www.npr.org/2012/01/01/144550920/physicists-seek-to-lose-the-lecture-as-teaching-tool

  5. I think Scientific Literacy is one part understanding basic scientific information (Newtonian physics, simple geometry, evolution basics, etc) and two parts knowing how to ask the right questions. The difference between what you know and what you don’t know, if you’re looking, is a factor of time more than anything else. Even things no one knows yet are largely a factor of time (though how much time is the question). Even if someone doesn’t know analogies and metaphors and arguments and evidence for Evolution, for example, if they know how to ask the right questions they can understand http://www.youtube.com/watch?v=mcAq9bmCeR0

    Totally not linking that just because it’s my favorite evolution video right now.

  6. Scientific literacy consists of two parts: understanding the scientific method and a basic level general scientific knowledge. I think many of the questions in the quiz focused too much on minutiae, but they covered most of the topics I would have. To consider a person scientifically literate they should have an understanding of gravity, newtons laws, atoms/molecules/compounds, biology, evolution, geology, and astronomy. To understand these, however, you need to understand at least a little statistics and logic.

    • I agree.

      You should know about the scientific method, hypothesis-driven research and exploratory research, how observational sciences (astronomy, evolution, geology) differ from experimental sciences, why double blind trials measuring clinical outcomes are the gold standard for medical research, and why sometimes you need to satisfy yourself with less.

      Optional extra is how science is organized: industry, universities, research grants, conferences, graduate students, peer review.

      You also need some facts:
      Chemistry: electrons, nuclei, elements, periodic table chemical bonds, chemical reaction equations, acids and bases, metals and non-metals, a few important chemical reactions such as combustion, smelting metals from ores, photosynthesis, synthesis of nitrogen fertilizers.

      Astronomy: The big bang, very basic nucleosynthesis, galaxies, stars, planets (in general), components of our solar system, day/night, seasons, phases of the moon.

      Physics: laws of motion, laws of thermodynamics, electricity and magnetism, optics, radioactivity, special relativity, very basic quantum mechanics (all these to a basic level – very little mathematics required.)

      Geology: volcanism, plate tectonics, uplift, erosion, types of rock, hydrological cycle, dirt, glaciation, history of the earth.

      Biology: what is life, cells, dna, evolution, overview of phylogeny of life, basic genetics, genes and proteins, ecology, the internal plumbing of vertebrates.

      Medicine: what are our organs and what do they do, pathogens, how diseases spread, hygiene and other self-care issues, medicines, evolution of pathogens against medicines, vaccination, cancer and other spontaneous pathologies, diet.

      I’m sure I’ve missed a few, but that is what I come up with at short notice.

      • Yeah, this. I never took physics, so I totally failed most of the questions having to do with that topic. While I think the test was flawed, it did make me realize how ignorant I am about physics-related info.

  7. I’ll admit that I had shit to do and stopped about 20 questions into the quiz but I thought they had done a pretty good job up to that stage testing science literacy. They just focused on understanding of scientific theories (a major part of scientific literacy) and not so much on scientific methods which is often the more problematic and less understood part of scientific literacy. While that may be the problem area that does not negate the fact that you need to understand major scientific theories to be scientifically literate. Like it or not remembering facts is always going to be a major part of scientific literacy.

    If you wanted to design one here basic understanding of current major theories is a good place to start but add in questions about conducting scientific tests, statistics, controls, errors and interpretation.

  8. I was frustrated somewhat by the test and its focus on facts. But I think I’ve changed my tune.

    Understanding the scientific method and how to think critically are skills like reading.

    One can be able to read and not be well-read. Likewise, unless you know something of those giants who have gone before, are you scientifically literate?

    I would like for our society to be scientifically literate. But sometimes I feel like I’d settle for critical thinking to be common.

  9. I created a quiz that, while very basic, shows the types of things that I think comprise scientific literacy. What I find saddest about this is how poorly people did on these very basic questions. BTW – I really hope I didn’t get any of the facts wrong.

    • That was fun, although sort of stressful when I was trying to read all the answers with a timer:P

      You scored: 793 / 1000

      You scored 8 questions correct on your first try.
      You have earned 132 FunTrivia points for this quiz.

  10. The ability to withstand this process: http://electroncafe.wordpress.com/2011/05/04/scientific-process-rage/

  11. Not to get into it much right now, but physics and astronomy education researchers use “concept inventories” (http://en.wikipedia.org/wiki/Concept_inventory) to probe real conceptual understanding of certain parts of their subject, and to measure gain in understanding over time (ie, before and after taking a physics or astronomy course, for example).

    In physics one common one is the “Force Concept Inventory”, and in astronomy, the “Light and Spectra Concept Inventory.” I can’t find good versions for y’all with a quick search right now, but I’ll post links later when I can. Waiting for some email responses from a few people…

    Full disclosure: I work a lot with the people in the Center for Astronomy Education (http://astronomy101.jpl.nasa.gov/), and they have great resources there that I hope will be of use to some people here!

    • From what I understand, a LOT of work goes into those concept inventories. One of the hardest tasks they gave us in a CAE workshop I attended was just coming up with a good multiple choice question to test an astronomy concept. I think it took four of us half an hour and our question still wasn’t quite clear!

      I think there is a “Stars” one now as well? But no general astronomy one… but that would force one to pick and choose among the *many* possible Astro 101 topics.

  12. I think it’s perfectly reasonable to test factoids. Compare with the literature test that asks you about a particular text.

    Of course, anyone can read and write and anyone can think logically/scientifically (with a little practice) about a problem, but you’ll only reach a valid conclusion if you’re basing it on valid facts.

  13. I agree with most of the above, the answer should definitely be inclusive of those points, some factoids are important, but I don’t think you need to be able to calculated forces off the top of your head, but you should probably know what inertia is, and (something many people don’t know), that mass and inertia are independent of gravity. An object far out in space still has the same mass, and the same inertia.

    But I’d like to emphasize two things: First is that good scientific literacy should include an understanding of elementary probability and statistics, and how numbers are used to communicate results to the public (including why relative risk can be problematic). Second is an understanding of the process beyond simple experimental design and into the realm of the broader process: what the scientific literature is, how it develops, how each individual experiment and paper contributes to it, how it contributes to replication of experiments, why that’s important, and what peer review is, how it works, and why it’s important.

  14. Neil deGrasse Tyson seems to agree with us that facts are only a piece of the picture of scientific literacy:

  15. “Literate” implies ability to comprehend, not retention of knowledge. Scientific literacy is about having enough of a grasp of scientific method to critically evaluate what you are told. When I left secondary school, I could probably have answered every question on that test correctly, because I took science subjects at school, but I was not scientifically literate.

    I did not know the difference between a cohort study and a case control study was, for example. I did not have the words to explain why cherry-picking one result to prove an argument on the internet was invalid. I could not have told you why spurious “evolutionary psychology” about why girls like pink was bunk.

    That quiz does not test scientific literacy at all. If you want to know a factoid, you can look it up in a book or on the internet; what enables you to do that is having the practice at researching and thinking critically about what you read.

Add Comment Register



Leave a Reply