- No .com websites
- You must use an article from the New York Times
I still remember seeing these two requirements listed on the same assignment, and wondering how to open a conversation with a teacher about the fact that if she wanted students to use an article from the New York Times, they would have to use a .com website. I don’t think she ever successfully resolved the cognitive dissonance (or revised the assignment requirements).
- If a website ends in .org it’s reliable
- Don’t use Wikipedia – it’s not reliable
These two I’ve heard more times than I can count. From teachers, students, and strangers who find out what I do for a living. I sometimes want to point out that Wikipedia ends in .org. When it’s teachers and students, I try and engage in a more nuanced discussion, but these two beliefs about the reliability of online sources seem to be deeply embedded in peoples’ minds.
- Check the “About Us” section if you want to learn more about the source and whether or not it’s trustworthy
Will someone who’s trying to manipulate me tell me that on their About Me page?
- You can’t trust Wikipedia, but you can trust the sources it cites.
This (from a student) was a new one for me, but it seems to be a reflection of a teacher trying to teach students how to use Wikipedia. And it’s such a fascinating takeaway for students to have! How can the sources be reliable, but the content created from them not reliable? How do we connect that to students’ understanding of why we cite sources?
I’ve heard iterations of all these ideas (and more!), and I’m sure you have, too. I am starting to think of them as the appendix of information literacy – they may have served a purpose at one point, but they’re not really helpful now. And, like an appendix, they can cause problems.
I understand the appeal of these source evaluation shortcuts. The world of information is big, and confusing, and often overwhelming. We want an easy way to decide where to spend our trust and time. But as we all know, there is no real shortcut when it comes to source evaluation.
What’s fascinating to me is the way that these shortcuts get passed down from generation to generation like a form of folk wisdom. Even as I start working with younger and younger students I’m realizing I need to make sure I spend time getting them to talk about their assumptions about sources and how to evaluate them.
One way I’ve been doing that recently is by asking students to start by thinking about how they evaluate gossip for trustworthiness. The metrics they describe are often *exactly* the kinds of things I hope they’re thinking about as they evaluate sources for research. We can then make those connections as we move the conversation to thinking about how to evaluate sources for their research. Here are some slides with the prompts I used and notes from class discussions in the slides below.
This has been really useful for surfacing and clarifying some of the vestigial understandings they have about reliability. I’d love to hear how other folks are engaging teachers and students in updating their understandings of reliability.
Thank you for writing this. You bring an important topic some needed clarity. And I like the idea about how we evaluate gossip being similar to how we might evaluate sources of other information. FYI. I think you have a typo early on “that” should be “than.”
Thank you for catching that!
These are the useless appendix of information literacy instruction and are clearly just as painful to remove! I’m beginning to feel that the CRAAP method is also sliding into this category (I saw it on another school’s LibGuide list and actually thought, “How 2010 of them”).
And thank you for sharing the awesome slides! I love the gossip reliability prompt (and the cat pic) as a way to get to the heart of what constitutes “reliable”.
Oh, CRAAP definitely belongs on that list (in my opinion, at least). It’s interesting – I think non-library folks starting leaning into CRAAP just as our thinking about source evaluating was shifting.
I’ve been using the gossip analogy for a while, but only recently started using it as a prompt and it has been so great.
I will definitely be using the “gossip” method from now on. I think that will really connect with the students. Thank you as well for sharing the slides!
Thanks for this post, Sara. I love the analogy of evaluating gossip from a peer group. I’m filing this away in my toolkit with another analogy you gave me – the difference between personal essays and letters of recommendation when applying for college, illustrating the difference between reading an About page and checking a source’s reputation through lateral reading. I use that all the time, along with an iteration of the activity you shared with us – Trust/Don’t Trust/Proceed with Caution.
One thing I would add to your list of vestigial appendages are the presence of ads as a factor in determining reliability. Along with the others you mentioned, my students often cite this as something they’ve been taught not to trust. I’ve seen kids get hung up on this when looking at sites like the New York Times. I try to teach students that most news organizations need advertising to “keep the lights on.” I saw on your slides that ads and pop ups could indicate untrustworthiness. Using that criteria, a site like CNN would automatically be out. How do you teach students about advertising’s role in the news ecosystem when they are evaluating sources?
Thanks again for sharing. Always grateful to learn from you.
Yes, the ad thing! That comes up a lot. As well as the opposite – if it “looks nice”, it must be reliable. One thing we talked about was differentiating between sites with ads and sites that are so busy (design-wise) that it’s hard to get to the information you’re looking for. If the ads/pop-ups/clicks (“you won’t believe how many times you have to click to see number 7!” make it hard to find the information, it almost doesn’t matter how reliable the site is – you probably want to find your information somewhere else. But yes, site design as a marker of reliability is definitely another vestigial appendage.
I think you are going to have a slew of librarians using this gossip analogy as an opening prompt. I also like the use of the term “folk wisdom.” What may have been helpful to one generation might be less applicable to the next or the technology might have adapted in the meantime. Your two examples are spot on for what we’ve all seen in assignment design.
The “folk wisdom” analogy is one that came to me while I was writing this – I think I’m going to use that when talking about this with teachers more often.
I want to stand up for the CRAAP test. To me the CRAAP test offers students a solid checklist to think about as they look at the information and try to figure out whether it is trustworthy. I’m not sure why CRAAP has become so outdated, because I think so many of us are still asking our students to check their sources with the methods listed in CRAAP. Through the CRAAP test, students are prompted to gather evidence within each criteria, to ultimately make an informed decision. They should consider the authority by noticing whether it is a .org/.com/.edu and wondering how this matters for the topic. They should consider the publisher’s perspective by reading the About Us pages and Wikipedia . They should be looking for corraborating information. Etc. If students can work their way through CRAAP on a particular article, they will find enough “proof” to make an informed decision on the trustworthiness of the source. For learners without any skills in this area, CRAAP offers a clear list of things to explore or a way to start. I think it is a fantastic teaching tool.
With that said, I appreciate this blog post, becasue it does point out how so many teachers and students have taken one checkpoint and made it “a golden rule of reliability” in an attempt to shortcut the evaluation process. Really, the “golden rule” should be evaluate a source in more than one way!
I don’t necessarily think the CRAAP test is bad, but it is very complex and, in my experience, sort of overwhelming for students. The checklist asks a lot of questions, but doesn’t give students a lot of guidance for how to go about determining those things. It also encourages students to dig deep on the source itself to determine its reliability, rather than using the lateral evaluation strategies that expert fact-checkers use. This is one of the many reasons why I prefer using SIFT strategies with students. SIFT is also flexible and adaptable depending on how the information landscape changes, so students aren’t memorizing concepts that will become outdated (.org vs. .com does not reliably tell you anything about the site or its creators). CRAAP was a useful tool, and the ideas in there continue to inform how I talk with students about sources – but we can do better.
This article might also be of interest: https://blogs.slj.com/neverendingsearch/2020/11/01/enough-with-the-craap-were-just-not-doing-it-right/
Thanks so much for this! I’m adding it to the ever growing list of ideas I’ve borrowed and used that you’ve shared over the past few years–the menu of services idea is one that I’m actively trying to work into some curricular committee and PD work that we’re working on now. I love the evaluating gossip analogy as well as the personal essays vs. letters of recommendation analogy that Chris raised! Thank you, again! Yay!!!
I’m so guilty of these! Thanks for lifting them up so that I can level up, Sara.
I’ve been reading this article describing proactive evaluation (meaning information in a network, evaluating a whole system, not just one result) and finding it theoretically useful. I have used “corroboration” and “triangulation” as an element information searchers should look at. Gossip is a great way to get into that topic on a student view level!
I read that article earlier this year and have been thinking about it a lot! I definitely need to think more about how to incorporate it into my practice. I think one of the challenges is that the more I think about all of this the more I realize how nuanced it all is, and I don’t want to overwhelm teachers and students.