Searching with the Sevens

This fall we had the odd experience of orienting the 7th graders to our physical library – giving them the introduction they would have had when they entered the school as 6s had we not been…well, you know. Since we already knew each other somewhat, I had more chance to observe over students’ shoulders as they pursued our “get to know the library” scavenger hunt, which is how I had the opportunity to watch several students search Destiny for [ books about birds ] and come away quite frustrated, telling us we did not have any bird books in the collection.

Very fortunately, the 7th grade dean was able to arrange for an hour for them to learn how search works (which I rarely get to teach anymore) and – when it became clear that the grade-level work they would be doing during our January intersession would revolve around finding “personal narratives” relating to “indigenous peoples and climate change” I was able to get another hour with them. During that class I tried out a lesson plan I’ve been wanting to test drive for more than a decade.

I’m not sure I have ever had two full hours just to teach students about the functioning of search tools and then the functioning of human expression in interaction with the search tools. As usual, I’ll share what I did here in the spirit of asking for feedback or thoughts so that (hoped for) future iterations can be smoother.

The first lesson: How search works

Thanks to the hard work and feedback of my wonderful Research TAs, I was able to pull together a lesson that demonstrates how search tools actually locate information and that involved lots of cat memes. Memes are great for search activities because they have so few words on them – and none of the words is actually “meme.”

The lesson objectives were to:

  • Understand that search tools crawl individual sources and index the words on each page.
  • Use a model index to locate physical sources.
  • Create a search query that will find what you need when you are working with a limited index.
  • Practice rudimentary imagining of sources.

Since I was running this class in one room, while three colleagues (with assistance from my TAs) were running it simultaneously in other rooms, I made both slides and a step-by-step script. There were activities building up to it, but the core of the lesson was pairs of students working together: one was the “searcher,” the other was the “computer.”

A picture of three lesson handouts: A prompt to try to find memes based on a French scientists' paper arguing that cats are a liquid, some samples of student searches for that meme, and an index of the words appearing on a collection of memes about cats being liquids.
Left: The “Searcher” had a secret prompt and search boxes to fill in. Right: The “Computer” had an index of words that appeared in eight different, numbered memes.

The “searcher” got a secret prompt and empty “search boxes” to fill out. The “computer” who – like our real computers – had no earthly idea of the context for the words written in the search box, had this very simple index to work with (but could not show it to the “searcher”). The computer also had small black-and-white printouts of eight cats-are-liquid memes, numbered 1-8 to correspond to the index. They could only see the number assigned each meme, not the meme itself. The “computer” could only return “no results” or a meme identified by the index. Looking at the “searches” on the left and the index on the right, one of the original searches, [french scientist cat memes] must have returned zero results, as three of the search terms do not even appear in the index. However, [cats are liquid] found two memes (numbered 6 and 8) and [cat liquid] found numbers 2, 4, 6, and 8 …  the “searcher” just kept trying until they got a meme result. Then, of course, each pair joyously looked at all eight possible memes and identified why the “computer” had been unable to “find” so many of them (because, of course, they said things like: “Liquid mode activated,” and so could not even be found if the searcher used the word [ cat ] in their query. 

We solidified this understanding by looking at actual search results and highlighting where our search terms showed up – proving that the words we typed in were the ones that search tools were identifying to bring back our results:

Sample search results for [ science cat memes ] – student identified where these search terms appeared in relation to the image of the meme itself.

The second lesson: How language made by humans works

My room got very engaged with the lesson, though I had my usual doubts about if it was all about the memes, or if anything actually stuck. Thus, I was very pleasantly surprised when – two months later – students did retain the big points of the lesson (ok, admittedly, a number of students in the class were individually able to help compile a list of points):

My students recalled these guidelines from the search lesson two months earlier.

For the follow-up lesson I had a harder ask: teach students how to translate the idea of “personal narratives” (a term both students and teachers love) into functional search terms. 

Often, when personal narratives are desired, I teach (older) students to look for [ oral history ] a wonderful context term that has the distinct advantage of describing collections of personal narratives. However, when looking for narratives from individuals from various indigenous communities around the world – particularly on the topic of climate change – we needed a different strategy entirely. I had learned from many years on a project we do with the ninth graders that individuals’ anecdotes that put a human face on “issues” like climate change often appear at the start of newspaper articles and in other, similar formats. My job became teaching the seventh graders to imagine search strategies and search terms to find these types of sources. 

This time, I had the whole 64-person grade in the library at once, and slides were once again in order. We considered the whole range of strategies for finding personal narratives, and trust me that the first two made for a lot of student chatter and example-sharing:

Slide outlining different ways to try and find “personal narratives”

We discussed searching in YouTube for their subject’s name (solution 3A), searching for terms like [ interview ] or [ transcript ] (solution 3B), and then I took a risk and tried a method I had wanted to undertake for years. I handed out excerpts from sources that offered stories from individual’s lives (such as this article or this one or this one), selecting the portion of the source that indicated that such a narrative was about to appear. Once again, I had them read, observe, and highlight. If an article did not use the words “personal narrative,” what words might it use?

Their observations were just phenomenal!

Terms my students identified from sources that contained personal narratives – including several I had never noticed myself! Several new options made my own list for future use.

(BTW: Forgot to say before that my personal favorite way to search for these narratives might look like this: [ farm OR livestock she OR hers OR he OR his OR me OR my OR our OR ours OR them OR they ] and this one when I need life-background information [ farm OR livestock “as a child” ].)

Once again, I am not sure every student got the idea I was going after, but I have rarely seen as many hands in the air and I actually had to cut them off so we could continue with the lesson (which was using Boolean in Google-form and in database-form to try to look for more personal narratives using these terms). We were in the middle of their final project for the year, writing Simple English Wikipedia pages for notable female-identifying individuals. While we would not use personal narratives for writing Wikipedia pages (they are not acceptable by Wikipedia’s source quality standards), the students were pretty excited about their subjects so we used them for search examples. 

I will have to see how their work goes during intersession (which will be virtual and for which I will be assigned to a different grade level), but this series of lessons did appear to offer heartening outcomes.

It was an excellent reminder to me: it is almost never a waste of time to give most of your time to the very, very basic building blocks students need to do research right.

#GlobalFact8: Learning fact-checking from the experts

This past week, I was fortunate enough to spend four days “attending” the International Fact-Checking Network’s eighth annual conference. It was compelling and eye-opening!

Many of the discussions really could have been taking place at a school library conference: questioning how to better teach media literacy, grappling to understand why mistrust in journalism and fact-checking is so high, wrestling with necessary relationships with certain corporations to maintain funding and access without letting those companies set the global fact-checking agenda, and discussing how to do more work with less money. Other topics, like the massive mental toll of both spending your days lurking on lists that are promoting misinformation and possibly worse, and harassment ranging from insulting comments to imprisonment to death threats made or carried out, are elements I am deeply grateful are much less a part of our work lives.

I’ve been noodling on what to share with you all, but also suffering a bit from screen fatigue. Furthermore, I neither want to simply hand out the intellectual property of these individuals who work so hard to find and share what is true, and I want to be thoughtful about naming individuals who are already suffering from harassment.

Due to our very similar fields and goals, however, I am in contact with the IFCN about how our professions might work together — so stay tuned for more (and keep your fingers crossed).

A few, random hot tips, though:

  1. The preponderance of research suggests that educating people to recognize misinformation (“prebunking” or “inoculation”) is much more effective than trying to debunk misinformation in the moment.
  2. TickToc is by far fact checkers’ favorite tool for prebunking education about how algorithms work, since they say (sorry, I have to take their word for it!) it is so very, very clear what the algorithm is doing within that social network.
  3. Fact-checking videos is the hardest, YouTube is not interested in transparency, collaboration or funding fact-checking, and their algorithm very decisively recommends videos that blatantly run afoul of their takedown policy but are up and running and being promoted.
  4. In many countries, the national statistics agencies are run by political appointees. And the resulting statistics may be re-tabulated and/or deleted by subsequent administrations. May be whole different agencies, as well.
  5. The absolute best video for teaching the technicalities of researching if a video is real or a deep fake is this aerobics class. (Reach out if you want more about doing so — don’t want to just share out someone else’s lesson, but here is the actual fact-check on the video.)

In the meantime, at the risk of giving you just a list of resources, I would like to share a slightly annotated … list of resources. The following are some of the most compelling reports, slide decks, and videos that were shared over the course of the conference:

Some context on US situation from the American Press Institute:
*Report: A new way of looking at trust in media: Do Americans share journalism’s core values?
Slides – not from this talk, but much of the same content in shorter form
*Also check out:
–2016 Report: A new understanding: What makes people trust and rely on news
–2018 Report: Americans and the News Media: What they do — and don’t — understand about each other

Sponsors of the conference: International Fact-Checking Network – Poynter
*IFCN Code of Principles
*International Fact-Checking Network YouTube Channel — lots of great content!
–Including: Fact-checking in school: Best practices from around the world (I have not watched this yet, but I did see some of these speakers on other, related topics)
*MediaWise – Poynter – they have a teen fact-checking group

Verification Handbook – a detailed primer by a number of strong professionals in the field.

Inoculation Theory – one researcher argued there is the most evidence that this is the most effective, others have supported the approach in their talks. One paper, as an example. A field that can really guide us in thinking about the most impactful use of our limited time with students.

First Draft News
*A guide to prebunking: a promising way to inoculate against misinformation
*The psychology of misinformation
*Many other useful reports – worth a look!

Global Trends in Fact-Checking: A Data-Driven Analysis of ClaimReview by Thomas Van Damme
*Data visualizations
*Thesis

Why Gendered Disinformation — #SHEPERSISTED – Explains the issue of gendered disinformation that targets female politicians, etc.

Lead Stories – has a “red feed” and a “blue feed” and is clearly run by a beloved and respected member of the fact checking community

Disinformation for Export: how false content generated in the United States reaches Latin America

YouTube Regrets – how YouTube algorithm surfaces and recommends misinformation
*Press release for shorter read

A typology of caveats (from FullFact) – We’re continuing to investigate the type of contextual information that statistics need in order to be meaningful and used correctly.

INFODEMIC COVID-19 IN EUROPE: A VISUAL ANALYSIS OF DISINFORMATION

Is it a Marvel film or a fact-checking newsroom? How Maldita.es uses its readers’ ‘superpowers’
This reminds me of some programs that could be built on preexisting TA programs, etc. in schools

A framework for information incidents – exploring how fact checking organizations should best respond in different incidents that can give rise to misinformation

GlobalFact 8 talk – Why good national statistics are so important for fact checkers – 2021-10-12
Gives insight into why it is so hard to use different country’s statistical data

Fact-Checking – Duke school of journalism

A comparison of reverse image search tools – has a handy summary chart if you scroll down

Weaponizing fact-checking: What Canada needs to know

https://fcl.eun.org/facts4all: The Facts4All – Schools as community hubs against disinformation is a one year project co-funded by the European Commission’s Media Literacy for All Programme project, which aims to increase awareness and critical thinking in relation to online disinformation across generations – in particular young people and their (grand)parents. — There is a MOOC

Headlines Network – Drive conversations towards improving mental health in the media and communications industries.

What do we want students to know about our databases?

It has been just over a year since I emailed the AISL list, looking for a truly representational news database aimed at the K-12 market. The push for such a product is ongoing, and I hope to be able to update you more on that process soon (and please contact me if you would like to get involved in the campaign to drive change).

In the meantime, our library decided to assure that students would see themselves and each other in our digital resources, even if it did require accessing multiple databases to do so.

Having a larger number of databases is a challenge, as we all know, and we need to take a multi-pronged approach to ease adoption. Among our strategies are:

1. Recruiting student “Database Ambassadors.” Our students have grade-wide, homework-related group texts and so the Ambassadors will be on the scene when homework is hard to complete. They will remind classmates to reach out to librarians for help. I set out to have a few ambassadors in each grade, but students started asking if they could join the cadre and fully 10% of our 8th-12th grade students have asked to volunteer for this project.

Training starts today, and I am trying to figure out the few ideas I have to drive home. No use trying to familiarize them with a substantial number of databases during a study hall! My Research TAs tell me that the bare bones of the class I taught our Juniors are a critical part of the lesson….

2. The lesson/lecture for all 11th grade US History classes was an introduction to working with primary sources. The theme was “database business models as systemic injustice.” Some colleagues have asked me to share the slides from that class. Though they are not at all pretty, you can find them here. Honestly, I never thought to get emails with words like “moving” and “fascinating” related to a database lesson!

To be frank: none of the systemic issues with databases were news to students who have aspects of their identity that have been minoritized. It has been painful for me to realize of the substantial number of students who have felt alienated by our collection for so long.

Nonetheless, we move forward with anticipation. We believe that our students can both thrive on our more inclusive collection and simultaneously look at it with a critical eye. It is our job to help them do so.

If such a lesson feels like a bit more than you can bite off, check out Rebecca Hall’s graphic novel, Wake: The Hidden History of Women-Led Slave Revolts. Actually, read it whatever the case. In addition to history I never learned, it talks about silences in sources and how they impact our understanding of historical events.

Have a lovely week!

NOTE: This database work is a group effort that currently involves Sarah Levin (Urban School), Alea Stokes (Thayer Academy), and Sara Kelley-Mudie (Beaver Day School). Also, my awesome Library Director, Jole Seroff, who is supporting this work as an integral part of the program we provide our school. I’m just the only one who cannot stop talking about it. It is how I roll. 😉

Another encouragement to rest….

For those of you who know me, you are about to experience a thing I never thought would happen: I am not in a mood to think, write, or talk about research. I am sure this state is temporary, and I did have a plan for this post, but I am fortunate enough to be someplace where I spent many happy hours in childhood, so I’ll share the images that inspire me at the moment (though it may not be scientifically correct):

Be like these eagles…

Cattle Point Lighthouse, San Juan Island, WA

…when you find a place with a good view….

… stop and enjoy it.

Reuben Tarte County Park, San Juan Island, WA

Promise to have library-related content next time, but for now … wishing you joy however you find it.

Squeezing in a beloved annual project

Honestly, I have infographics on the brain right now. Welcome.

This year has been surprisingly busy and fulfilling on the instructional front, though nothing like past years or on almost any expected topics. I was thrilled, then, when our wonderful 7th grade History teacher reached out to ask me to help her wrap up the year with a lengthy infographics project we usually do in November.

Initially, when this collaboration began nine years ago (and before I came to this school, and shepherded by the wonderful Jole Seroff), the idea was to follow the unit on African empires with a look at positive aspects of life in African countries today. Time has taught us, sadly, that statistics do not give rise to seventh graders focusing on encouraging analysis. The project has endured nonetheless and continues morphing in the way good collaborations can sometimes do.

The way we teach this unit has streamlined over the years to make it easier for students, though it might have a bit of a fools rush in… aspect to it, as well. I’ll admit my desire to refresh it each year makes for a tremendous amount of work for me. It is also a big leap for students, and asks them to embrace some really new ways of thinking. But it is worth the growing pains — while high school students look back on it with some exaggerated shudders, there also exists some great love, and a handful of skills for which they voice deep appreciation in those later years.

Overall, I would argue that this is a process-based project: if you look at the final infographics they don’t tell the actual story. The insights offered in student presentations are much more telling. And the conversations we have over the course of this unit are varied and deep: Can we use data from different years in different graphs? Why doesn’t Rwanda have statistical data on prevalence of certain genetic diseases within its population the way the US does? Why doesn’t the US have an official literacy rate the way other countries do? Can we trust China’s numbers on incarceration? Explain that GDP thing again? What is “primary education”? This year brings: My hypothesis is that Russia has fewer women in prison than the US, but all my data shows that predictors of imprisonment are higher in Russia…. Do I need to change my thesis? (So many interesting conversations there.) And, hearing a roomful of seventh graders recommending that a classmate use “per capita” data in her graph instead of gross numbers, and explaining why… well, it is certainly a high point in my time as an educator.

This project has a lot of quirky methods and rules that have developed over the years, but they seem to help us get the project running smoothly. I’ve outlined the 3-4 week lesson plan below, for those who want to dig in. As always, very open to feedback and suggestions! Would love to hear in the comments if you do infographics or other data projects and what your objectives are.

Library’s data literacy objectives:
1. Develop a sense for the kinds of topics on which official statistics are collected and some of the specialized language used by statisticians.
2. Understand the difference between description and interpretation of a visualization. Be able to identify and produce both.
3. Understand that different visualizations are good for different types of data. Be able to select/produce the right kind.
4. Understand that evidence does not always have your topic mentioned in it. Broaden individual understanding of what can count as evidence.

Day 1:
1. What is an infographic? (Slides 1-2)
2. Virtual gallery walk of good and bad infographics (This year, rest of slides; most years we have a lot of infographics we hang around the room for the gallery walk.)
A. Have students list characteristics they see of different infographics, and then keep track of every time they saw that characteristic used, categorizing them into:
i. thesis, evidence, graphs, and design and
ii. helpful or unhelpful
B. Build class rubric (adapts to project rubric)

Day 2:
1. Look at different data visualizations, discuss the difference between description and interpretation of a visualization. Practice with the Banana Timeline.
A. Students start at their assigned slide and follow directions on slide 10.
i.Interpretation: This graph shows THAT…
ii. When can you use this type of visualization?
iii.Tips
B. Pair and share: introduce your graph to your partner, sharing what you wrote. Ask your partner a question about their graph.
C. As a class, build “Tips for graphing” so you will know how and when to use different data visualizations.
2. Read this quick summary of topic areas and pick one.

Day 3:
1. Childhood obesity practice:
A. Explain “working thesis/hypothesis” – can change
B. Your working thesis should have a comparison (across time, gender, place)
C. Example: “Over the past 20 years, childhood obesity has been a growing
problem in the United States.”
i. Identify the comparison.
ii. Brainstorm possible “subtopics.” Subtopics are related topics which, in this project, may not repeat words/ideas from your thesis (except time, gender, or place names). Students often come up with ideas like:
a. Hours of screen time
b. Percentage of schools with PE classes
c. Percentage of families who eat dinner together
d. Average distance to closest supermarket
e. Hours of homework
f. Participation in athletics
g. Cost of healthy food, like product
h. etc.
2. Students read background portfolios on their topic area:
A. Practice stepping stones as a class, then together look for stepping stones (No
pandemic? Do this in groups.)
C. Look at starting graphs and draft a working thesis.
NOTES:
1. Over the years, we have found it works best for us to provide them with graphs from which they chose one and write an interpretive statement communicating some of the information shown as their hypothesis, which can change as they develop their infographic.
2.This year, I was able to create these background reading documents, which I stuffed with references to indicators related to their topic. They used these background readings as stepping stone sources to plan the next round of research. For example, one of our topics was global female imprisonment, and the background reading included this passage:

…I learned that crime is not what really brought these women to prison. Far from it. It started with a lack of education, whose supply and quality is not equal for all. It starts with a lack of economic opportunities, which pushes these women to the petty survival crimes. The broken health system, the broken criminal system, the broken social-justice system. If any of these poor women fall through any of these cracks, the bottom of that chasm is a prison.

From this passage, they can note that indicators related to imprisonment include: education, economic opportunities, healthcare, etc.

Homework: Write a working thesis, complete noting potential subtopics mentioned in the background reading.

Day 4:
1. Introduce acceptable data sets. We provide all data. This is still hard and messy, to navigate, but just sending them to online databases (go look at UN Data!) is waaay too hard. So we give them this TOC (mostly links to spreadsheets I have downloaded and cleaned up).
2. Take out list of subtopics gleaned from the background reading and look through the TOC (we have them read the *whole* thing, saves arguments later!).
3. Pick a data set that covers a topic that will help contextualize your thesis (we talk quite a bit about how this is not really “evidence” since neither causation nor directionality is assured).

Homework: Draft your first graph.

NOTES:
1. We spend a *lot* of time demonstrating imperfect circles, terrible stick figures, and sloppy handwriting. They hear not to spend more than 10 minutes on the draft. We compliment students on the “draftiness” of their drafts.
2. We ask for screenshots of the data and the citation because the number of ways students find to lose track of their data as this project continues defies the imagination. Also, always better to build your citations as you go!

Next several classes:
Independent work. Each night, for four nights, students sketch a rough draft on paper of a visualization of one data set they planned to use as evidence. Each day, in class, we select five visualizations and discuss them as a class–each student has a chance to have one visualization complemented and critiqued. She tells the class her working thesis, describes her data visualization (projected on the screen), and tells us how it helps explain a systematic factor impacting her thesis. Then, she got three compliments, three questions, and three kinds of constructive feedback. We encourage student to take notes/jot down new ideas based on peer work and/or critiques.

When they do the fourth graph, we also ask them to write four “statistical statements” – a bit more data or other interpretive information to help flesh out the story their infographic tells.

Day 8:
1. Students get a piece of paper and post-its to do a mock up (wireframe) of their infographic.
2. When students turn in their bibliography and show their paper wireframe, they get their code for Pictochart and learn to set it up.

Day 9:
Assemble infographic on Pictochart

Day 10:
Prepare to present your infographic. You will tell us:
Explain your main argument or thesis statement. What do you intend for your infographic to convey? Explain why you were interested in this topic, and the information you were thinking you’d find when you started this project.
Explain 2 of your 4 graphs and how they relate to your thesis.
Explain 2 pieces of your additional evidence, and how they add to your argument
What was one aspect of your infographic that was difficult for you? How did you resolve this difficulty?
What information did you wish you had found that you were unable to find?
If you were to start this project again with the same topic, how would you approach your infographic differently?

Day 11-12:

Presentations (eight per day)

Goals for next year: Visual note-taking and further exploration

I suspect we all have one (at least). Some project we were hot on going into March, 2020, and have left dangling since. Please bear with me while I share my dropped passion project here, to get myself geared up for next year; I encourage you to share yours in the comments below to kick-start your own enthusiasm!

Note-taking has been an interesting animal: not one that interests me a lot, or that I was ever trained to teach, but a skill that definitely falls by the wayside in an age of copy-and-paste-into-a-Google-Doc availability. We are a proud NoodleTools school (which the students love-we literally have a cohort of Seniors pushing to endow NoodleTools to graduates in perpetuity as their Senior Gift, that is how much they love it), but we have needed some additional ways to help students read meaningfully. In particular, our Middle School teachers focus heavily on plagiarism, and have started thinking about its relationship to reading comprehension. So, we began wondering what kind of note-taking lessons we wanted to offer as a strategy for asking students to slow down and prioritize understanding.

Our experimental solution: visual note-taking.

By some bit of luck, I got to work with both our seventh grade science class and our seventh grade history class on a pilot of this endeavor.

In Science, students were preparing to design experiments for our Middle School science fair. Often, when researching and writing their proposals, their teacher noted a tendency to parrot language that they did not understand. So, as part of the lesson when we talked about NoodleTools and writing citations as a form of source evaluation, we also practiced drawing for comprehension:

  • Slides I made for class included samples made by my Research TAs when I was testing the usefulness of the approach, and then took a passage on transpiration for which the class assisted me in collaboratively build a visual note (none of us knew what it meant when we started, but we did when we finished).
  • Drawing together on the whiteboard and then showing my notes let me emphasize the “very drafty” nature of my notes, as opposed to something one might create for an art show.
  • Small groups has short passages on circulation systems and respiratory systems in insects and earthworms. Their prompt was to draw notes that they could understand to help them unpack the meaning of their reading.

Many of the students were vocal about hating the process — they found it so much slower than just writing down random words and sentences from their reading. However, their visual notes made it clear that they understood their reading very well:




Some weeks later, we returned to this strategy in History. Students were doing research for a Renaissance Dinner Party: They each learned about an assigned historical figure, created a class presentation, and then had to seat ten people covered by the presentations around a table at a dinner party in such a way that no fights would break out and all guests would be entertained.

For the research stage, we returned to the idea of visual note taking. This lesson was a longer process, covering several days, and including a number of different strategies for communicating learning in a manner where the language of the articles they read would not suit (such as a fake Twitter where the historical figures chatted and threw shade).

  • At the end of the first day, the students compiled a list of advice for visual note-taking.
  • We once again practiced as a class with drawing “messy stick figures,” and students started comparing and bragging about the messiness they achieved.
  • Students did note that they wanted to demonstrate emotions/interactions and to be able to tell individuals apart. I brought in iconography from the amazing Good Tickle Brain* and looked at how Mya Lixian Gosling’s very simple drawings of Shakespeare characters (for example, Cleopatra and Juliet), which seemed to help a lot.

Some students still felt frustrated (particularly those who felt more successful and comfortable memorizing and repeating), but some really interesting feedback did come my way. It is anecdotal, but impressive:

  • The day that students had to hand in their visual notes on three articles they read for homework, a bunch were waiting for me at the library. They explained that they had found it boring to re-draw the same material, and wanted to check if it was ok that they took all their notes on one set of pictures.
  • Let me rephrase: They synthesized their notes from several sources onto one set of images. Students naturally moved from a linear set of pull quotes, article-by-article, to integrated knowledge.
  • An elated student stopped by to tell me about how her “super-smart, intimidating” uncle had come by for dinner. She often felt nervous with him, because he always wanted to know what she was learning from school: “and I remembered everything, without even *looking* at my notes!”

This pilot felt meaningful to me. Genuinely understanding and remembering content, paired with natural synthesis is a holy grail I will happily continue to pursue. Later, I developed (but have not tested) a theoretical self-grading rubric to use with visual notes, based on a Verbal to Visual post:


I was able to take another brief stab with our Chemistry classes when they were supposed to be looking at how a range of experiments were conducted, but students tended to focus on the outcome because they often could not visualize the experiments themselves. Sixth grade Science took on a drawing project this year to practice understanding relative dating and geology. And, with our History department’s Advanced Topics Research & Writing class, we now how three years of evidence of many students radically relearning note-taking for deep research.

But I really want to develop a much deeper understanding of note-taking, and get our students experimenting with different methods to find the best fit for themselves. I’ve also been dreaming of doing an exhibit of employee’s notes — showing students that the adults on campus have developed a range of methods that work personally for each of us. I worry that the hurry to “get through the workload” makes more work as students develop frictionless paths that feel like less work…but since they tend to sidestep understanding, I suspect they end up taking much more work in the long-term, with less actual learning.

So — that is something I am excited to get back to work on. How about you?

*And don’t forget to check out Good Tickle Brain — I doubt you will regret it!

Agency from anywhere: Why you should learn to edit Wikipedia, and teach your students, too!

Recently, my younger child declared I have a new motto. He even put it on a shirt for me:



“When life gives you lemons, write Wikipedia pages about amazing women”

My child observed that I spent the afternoon of January 6, 2021, watching coverage of the insurrection in DC while editing Wikipedia furiously, and that I used editing to manage my worries during other periods of uncertainty over the last year. He is not wrong, but here is how I see it:

  • From my armchair I tangle with systemic inequities arising from the specific guidelines meant to make Wikipedia “more reliable.” 
  • Even while sheltering in place, I have the ability to broaden the narrative of our nation and our world as it is shaped by a source which is, arguably, a de facto arbiter of truth in our time.

While I have been guiding students in Wikipedia editing lessons since about 2010, I worked primarily with upper elementary and middle school students editing Simple English Wikipedia. When I joined the Castilleja faculty in 2013, I took over a similar project my library director, Jole Seroff, had developed. Along with the project came her notes on gender imbalance among editors and how the skew towards male editors (85% of editors, and something like 91% of all edits) impacted the content we see when we access Wikipedia. A 2011 article in the New York Times noted that who edits impacts the emphasis of the source, comparing a four-paragraph page on friendship bracelets (“A topic generally restricted to teenage girls”) to the much longer page about “something boys might favor” like baseball cards. Setting aside for a moment the gendering of topics, it is notable that today these discrepancies remain:

Sources: “All page views: ‘Baseball Cards'” and “All page views: ‘Friendship Bracelets'”

Similarly, a study released in October 2014, noted that only 15.53% of English Wikipedia’s biographies were about women. A number of groups, including WomenInRed, have focused on adding biographies about females, and they count that number rising to 18.79% as of 15 March 2021. Mountains of evidence point to lower number of pages being written about women and topics to do with women, as well as fewer editors adding information and a significantly higher deletion rate of pages about women/women’s issues because the subjects are apparently “not notable” (especially regarding STEM-focused topics).

If the numbers are so grim for women, imagine what inclusion might look like for other individuals and topics related to minoritized identities.

During the 2019-2020 school year some of my high school students became interested in hosting an edit-a-thon, and I decided it was time to actually learn how to edit for real. I attended my first in-person edit-a-thon at a local library in February, and then everything shut down. In June, my students and I decided to host a virtual edit-a-thon for Upper School students, and the real fun began.

In preparation, a number of our school librarian colleagues kindly joined me in an experimental edit-a-thon, which sufficed to demonstrate that I had picked a terrible way to organize my event. However, that afternoon also demonstrated the value of editing in community, as we each noticed different aspects of systemic prejudices in the structure of this venerated source. For example, one of our number is a classroom teacher in an English department, with a specialty in Southeast Asian American Literature. When she decided to work on the page Asian American Literature, another of our number called our attention to the Talk page, where editors discuss issues and challenges that arise in writing the page itself. In addition, due to the very reasonable desire to keep an eye on coverage in specific fields, and point out what work needs to be done, WikiProjects on various topics rate the importance of specific pages under their purview, like this:

Talk page for Asian American literature

…in which WikiProject Literature (that is, people who are interested in Wikiedia’s coverage in the field of literature) rated the Asian American literature page Low-importance.

Similarly, I was reading up on Patricia Roberts Harris. She was the first Black American female: 1. ambassador, 2. cabinet member (third Black American cabinet member overall), 3. dean of a law school, and 4. director of a Fortune 500 corporation. Here is her talk page:

Talk page for Patricia Roberts Harris

Over time, that original meeting of teachers grew into a weekly editing group. We learn by doing together, and we have learned very well just how hard it is to prove notability for genuinely notable people of color. It was actually in trying to set up a middle school Wikipedia editing project in 2018, covering notable female activists, that I really ran up against to problem of databases containing predominantly-white-perspective sources and the challenges that ensued in finding articles about non-white, non-cis-male individuals. That lesson has held firm as I try to write about women of color and struggle to meet the standard that Wikipedia articles should be “based on reliable, published sources,” meeting Wikipedia’s definition of reliable sources. There is no question that these guidelines are needed so that people do not fill pages with self-promotional material, as often used to happen. However, there is also no question that the guidelines to block self-promotion make it extremely hard to write about many genuinely notable people, as well, especially if they are not media darlings.

As an instructional librarian, I focus on teaching research skills. Therefore, I find joy in digging and in piecing together sources and arguing for their reliability (when necessary), all while avoiding running afoul of Wikipedia’s “No original research” policy. I’ve come to believe strongly in the many benefits of teaching others to edit and editing in community. I now help run three Wikipedia editing groups for: alums from my college, my students and colleagues, and other librarians/teachers.

Editing Wikipedia is a way to:

  1. “Do the work”:
    1. Decolonize your mind – if the only astrophysicist you have spent time thinking about is a LatinX transgender individual, then the picture you have in your head of an astrophysicist will be of a LatinX transgender individual
    2. Make people with minoritized identities discoverable
    3. Create or expand or improve pages that will be seen by millions of people – the least-used page I have worked on has been accessed 14 times since March 1, others have been accessed several thousand times
    4. Give others access to role models – a.k.a.: the perfect gift — I add women to Wikipedia as graduation gifts for young women who might not easily see role models in fields that interest them
  2. Build information literacy skills (for students):
    1. Explore the notion that “authority is constructed and contextual”
    2. Develop a strong sense of what a range of authoritative sources might look like
    3. Synthesize evidence to create a narrative
    4. Practice writing in the register of an encyclopedia
    5. Experience gatekeeping and its impact on knowledge construction
    6. Question why needed systems give rise to systemic prejudice
    7. Encounter systemic racism and other systemic prejudices and begin to understand their prevalence and impact
    8. Construct authority

If you would like to learn more about editing Wikipedia with students or for yourself, please join Corey Baker, Amy Pelman, Linda Swarlis, and myself at the upcoming AISL conference on April 9 for “Equity through Editing: Contributing to Wikipedia for Social Justice Inside and Outside the Classroom,” or reach out to any of us for more information.

Imagining Multiple Perspectives: Experimenting with “Priority-Based Perspectives”

This year, our school’s annual, 40+ hour, 5-month-long, co-curricular project with our ninth graders took place over six hours during the first week of January.

During this project students work in groups to research and ultimately present about a social issue of their choice to the grade-level parents. While this year’s nine-person faculty team determined that students would still make a brief Flipgrid video on their topic as a final product, I volunteered to design the week-long mini-course in which they would research and present their topics. I then had to decide what I really, really wanted our students to learn. Since our first run, eight years ago, students have consistently been asked to investigate multiple points of view with regard to their topic, and I have just as consistently been struck with how challenging it is for them to imagine what those viewpoints might be. So, I decided to focus on skills related to searching for and identifying multiple points of view. Specifically, our one requirement for the week was that they identify at least three perspectives on their topic.

Before winter break, students had studied Early Modern Islamic Empires in World History, and had submitted connections they saw among the political history of the Safavids, Mughals, and Ottomans and issues of global health and/or justice in the world today. I took these contemporary connections and, based on the topics each student highlighted, assigned them to groups. Since our time was so short, I framed broad research questions for each group based on their expressed interests. Almost every student articulated a pressing engagement with how personal identifiers played into political leaders’ use of power to privilege or repress different people under their influence, across a range of topics from voting rights to vaccine development to rights of women in Muslim countries (see the outcome of this last topic below).

They had their topics. I had my learning objective. Now, I just had to figure out how to teach the skill set of searching for and identifying multiple perspectives … in 40 minutes or less.

Through a day-long debate with my 20-year-old offspring, we formulated the following approach to identifying multiple perspectives. Though I taught it to 64 students at once in a Zoom meeting, I would definitively say my thinking is still in draft form. Please help me kick the tires and see if it holds up! I would be deeply grateful for critiques and feedback.


It always feels helpful to start with a framing activity that helps to ease students into the complexities we will be tackling in a lesson. In this case, I wanted to get them thinking about how the “government’s job” is not a unitary or settled notion. By adapting a few questions from the (wonderful!) World Values Survey and a recent budget survey done by the city in which our school is located, I asked them to answer four questions about what governments should prioritize.



Though I did briefly show them results to demonstrate that, even in the “room,” we had lots of varying opinions, I cared less about their answers and more about the students contemplating competing priorities.

A bar chart recording responses from class survey demonstrate that everyone has different opinions about the role or priorities of government.

Ultimately, we framed the lesson around the notion of “Priority-based perspectives,” the idea that points of view vary because individuals and organizations must sort through conflicting needs, and some must ultimately take precedence. I drafted a list of what some of these categories of priorities might be:

Image of a slide from class, with a stamp on it saying: DRAFT: Some categories of priorities: moral, economic, logistical, public relations/political, allegiance

The big epiphany I had (though it seems obvious in retrospect) was that my students were never thinking beyond the moral concepts that drove their own interest in their topics. They simply don’t have the life experience to suggest another path for investigation. So, we spent some time discussing that, for example, no matter how strongly you consider rest a moral right, the need to feed and house your family might still require a higher priority than time to rest. Similarly, you may morally believe that everyone has the right of equal access to a COVID vaccine, but we have been witnessing the real logistical challenges of efficient and equitable distribution.

One element that turned out to be pivotally helpful was that this construct moved students away from the binary pro/con approach for identifying perspectives (“Who agrees with me? Smart people. Who has a different opinion? Mean people.”) and instead started their research in a fundamentally different place: “What are the economic considerations of vaccine development? What different priorities might be competing? What are the logistical considerations of vaccine development? What different priorities might be competing with regard to logistical issues?” and so on. It removed their thinking from themselves and their closely held beliefs and allowed them space to be curious about what issues and priorities might exist.

Of course, we reviewed a few key search tips:

1. Pay attention to search terms, which will yield specific POVs:
— For example: Undocumented workers, illegal aliens, birthright citizenship
2. Search for your answer, not your question:
— Think about what you might expect someone to write about a topic and search for that.
— What words might someone use when they are talking about “morals”? How about “economic”? For example: budget, cost, price tag
3. Consider: how might people with the same goals have different priorities that lead to different POVs?
4. Remember to use stepping stone sources:
— Identify expert vocabulary/pertinent search terms that appear in the sources you have read so far. For example: CRISPR, designer babies, genetic engineering
— Who are some stakeholders mentioned in your sources who have points of view on this topic (people, groups, or organizations who care about the issue)? For example:
1.”According to the Department of Justice…”
2. “Emails leaked by the whistleblower”
3. “Professor Arvind Gupta, a researcher at the University of Oklahoma…”

There was one final issue that required addressing in this lesson: what does one do with points of view that do not meet with standards of evidence-based-reasoning? In the past, I have taught finding various political perspectives and a separate lesson on more diverse news consumption generally (finally this year I pointed out to a colleague that selecting sources by left-leaning and right-leaning just left out too many people, and ethnic, religious, and other community news sources needed to be included in any list of sources for reading political news). Yet, I try to balance understanding various perspectives against including harmful or factually incorrect perspectives in the classroom. For example, we do not accept as factual perspectives disproven medical studies (they may be mentioned for influence, but not presented as informational sources). I’ve never quite figured out how to walk that line effectively.

In this lesson, I drew on the World History course theme of historical empathy, and explained that identifying priority-based perspectives is an interstitial step that assists in identifying a range of perspectives; doing so also helps us understand the mechanisms that create structures in our world today, including the persistence of structural inequity. After identifying different priorities, and (potentially) sources that speak from the perspective of those authorities, it is the researcher’s job to evaluate the source (or, as we teach, identify the context and construction of the source’s authority) and determine whether it passes muster in our evidence-based environment.

The faculty for the ninth grade project all agreed that the students’ work this year was quite strong, despite the various emotional challenges everyone faced during our six hours of project time (including the events of January 6, 2021). I was particularly proud of the group that wanted to focus on women in Islam, as they moved themselves beyond a blind critique of the “other” and ended up delving deeply into a complex set of women’s perspectives about being a hijabi.

So, the idea of priority-based perspectives seems to work. Yet, I suspect I am missing (various) pitfalls or have elements lacking clarity. Thoughts?

News databases: Diversity without equity or inclusion

The Problem

Back in September, 2020, I sent out a call for help across AISL and other school librarian-oriented lists in hopes of finding databases that provide “diverse, inclusive, and equitable access to perspectives mirroring the composition of our country in magazines, historical newspapers, and contemporary news.” Generally, database companies sell “core” collections that are positioned as “high quality sources,” comprised almost entirely of white-perspective news outlets. Then they up-sell from a menu of discrete “ethnic” packages to provide “alternate perspectives.” Students deserve better.

Thank you to the many folks who responded hoping to hear of a good database in which to invest. Sadly, the answer is…so far I’ve found no way to buy this unicorn of the database world. Ultimately, I started doing my own diversity audit of our databases and others on the market to try to better articulate the nature of the problem.

I am currently only part way through this process. First semester ended up (happily) being much more crowded with instruction than I had anticipated. Even the terminology I use to think about this set of issues is still in crude form. Here is an update on what I have learned so far, however. To date I have focused on US news, historical and contemporary, and have only been able to compare offerings from two companies. This work has served — at the very least — as evidence that the problem is real and pressing.

Inclusion

In the fall, I had not yet fully realized the insidious nature of the juxtaposition we often attribute to databases: quality sources vs. alternative perspectives. I’ve been sitting with this formulation increasingly in the intervening months, and contemplating how our professional narrative around databases is driven by the marketing efforts of the database companies themselves. Consider the act of marketing a database as “providing researchers access to essential, often overlooked perspectives” that exists because the perspectives have been intentionally overlooked and isolated to sell us another database. So how much does the title list of an intentionally curated “ethnic” database (which mysteriously includes the LGBTQ+ collection, by the way) overlap with a product intended for high school?

ProQuest: Compared title lists for Research Library Prep and Ethnic Newswatch databases.

Please note that the “Overlap (%)” column conveys how many of the “specialized” Ethnic Newswatch titles also appear in “general” Research Library Prep. It does not convey the percentage of Research Library Prep that are/are not white perspective — those numbers would apparently be vanishingly small. 

An issue that struck me immediately as I got started was that scholarly journals comprise, by far, the largest mass of content in Ethnic Newswatch that is also available in Research Library Prep. These sources differ distinctly from newspapers or popular magazines; academic discourse may well be quite removed from the community it studies. That is, a large percentage of the authorial and editorial work is carried out within a realm of authority modeled on European institutions and constructed in our academic halls of privilege. To put it plainly: the perspectives appearing in the University of Pennsylvania Press’ Hispanic Review may not reflect community voices in the same way that those appearing in La Prensa Texas newspaper do. Both source types provide important points of view; their creation does not serve the same purpose.

Important as it is to have a diversity of voices in our scholarly works, they provide fundamentally different types of evidence from newspapers. Not to mention, they are not accessible to most K-12 students.

Gale: General OneFile, In Context: High School, OneFile: High School Edition, OneFile: News, In Context: US History

It has been challenging to figure out how to do a diversity audit, but since many database companies seem to start monetizing diversity with Black American newspapers, I decided to work from lists of existing and historical Black papers, including: National Newspaper Publishers Association Current Members and Princeton University Library – African-American Newspapers (1829-present). I used titles from these lists to search within Gale’s title lists, and found:

Checking Black American newspaper titles against Gale title lists yielded vanishingly few overlaps.

In the process of looking at the titles that are listed, the Atlanta Daily World and the Chicago Defender — historically both very important publications in the 20th Century United States — only had coverage from 2014-present, with exceptions from 2015/16 to the present. Meaning, in fact, they only have a handful of issues of each paper.

Once again, these databases provide news sources that almost entirely reside within historically white readerships.

Equity

In another sense, it does not functionally matter if a database includes sources from diverse sets of communities. When the algorithm privileges white perspective publications, searchers may never encounter other points of view.

Spot checks of ProQuest’s ranking of newspaper results in Research Library Prep confirmed that their methods for ranking heavily favored specific titles. Specifically, the New York Times dominated results, with a smattering of hits from the Chicago Tribune, Washington Post, US Fed News Service and Targeted News Service

I ran a series of searches, noting how many unique titles were returned for each search, as well as those titles’ spreads across the top 100 results. In essence, how many pages would I need to scroll through to access more than a few titles? I searched for [ the ] — as a word that appears universally in English-language newspapers — and also for words like [ miami ], [ skagit ], and [governor] — each of which strongly suggest local news. In every case, the results looked something like the results for [ the ]:

  • Returned 81 unique titles
  • Top 100 results:
    • 95 results from the NYT
    • Other titles ranked: #37, 41, 71, 91, 95
    • Other publications in the top 100 results: Chicago Tribune, Washington Post, Targeted News Service 

Progress!

However, there is good news. A Gale sales rep who is on one or more school library lists began wondering about this issue themselves, and carried out an independent audit that they then presented to their acquisitions department. As a result, when I last checked in this past November, Gale publisher relations personnel have identified:

  • Licensed periodicals where the issues aren’t current
    • Updates are in various states of progress 
  • Important periodicals with lapsed agreements 
    • Updates are in various states of progress  
  • Over 140 new periodicals from the following communities:  “African Americans, Arab Americans, Hispanic Americans/ Latinx, Native Americans, Ability Diverse, LGBTQ+, Women, & more”
    • Requests have been sent to publisher relations to pursue license agreements

Though not within the scope of my current work, Gale has also taken a look at their reference overviews and biographies and have made efforts to offer more coverage, as well.

The solution?

Does this issue interest you? Would you like to join me in fighting for single databases that are diverse, equitable, and inclusive? Whether you would like to audit a database you have, suggest a consistent method for auditing, share findings at your state conference, or talk to your database companies once we have a clear report — kindly reach out. If the idea is that we are better together, let’s unite and make a difference!

I am deeply grateful to my director, Jole Seroff, for being so invested in and supportive of this exploration, and colleague Sara Kelley-Mudie for helping me focus my thinking.

Distinguishing evidence from analysis: A student’s perspective on the first step in source evaluation

Sara Zoroufy is a junior and the Research Teaching Assistant for the Castilleja School library. Inspired by Nora Murphy’s work on source literacy, Sara chose to spend this year observing research lessons and unpacking how she and other students think about sources. Her work helps inform lesson planning. Here, she shares an idea she has been contemplating recently.

“CNN reports that the Justice Department found the following statistics…”

During a presentation in our tenth grade government class, this phrase caught my attention. Why would a speaker attribute a statistic to two different sources? I have been thinking about this turn of phrase for a long time, trying to understand precisely why it troubled me. Recently, I realized that students struggle to distinguish factual evidence from a source’s analysis of that evidence. In the example above, the student was having trouble determining what type of information she was citing and which source was responsible for the creation of that information. Without separating evidence from analysis, we can neither evaluate nor properly cite a source. I tried to draw a visual to help myself understand how a source breaks down into these components, which culminated in this flowchart:

Mapping these concepts in this way helped me identify a number of key points in the process of evaluating a source. I began to think that the essential questions that students should initially ask when faced with a particular source are: what is the evidence that is presented, what is the corresponding analysis, and what sources shaped each component?

Asking these questions is the first step in unpacking a source, and the answers are not always immediately clear when students encounter unfamiliar genres of writing. This year, my grade was presented with an excerpt from a Pulitzer prize winning piece of investigative journalism about the diagnosis of black lung in coal miners. We were asked to identify the sources of the statistics in the article. No one was able to locate this information because journalistic convention dictates integrating the names of sources into the text, as opposed to employing parenthetical citations that students use in their own writing. For example, just prior to starting a bullet pointed list of statistical evidence, the article said, “The Center [for Public Integrity] recorded key information about these cases, analyzed [the medical expert’s] reports and testimony, consulted medical literature and interviewed leading doctors.”[1]  Since the students weren’t accustomed to this particular form of citation, many of us responded that no sources were given.

Students had been instructed to pinpoint the evidence in the article and label it with an “E” and to identify and label its sources with an “S.” As I sat with Tasha Bergson-Michelson, our instructional librarian, and considered my flowchart in relation to the lesson, we realized that the instruction had skipped over several crucial steps in the process of identifying the evidence. This experience made it clear that identifying the sources of evidence can be confusing, and that simply telling students to exercise that skill was not effective. Rather, the development of this skill requires explicit instruction and opportunities to focus on practicing it.

Once we’ve identified the source’s evidence and where it came from, we are able to further evaluate it. Depending on the type of evidence, we can investigate its quality and veracity in different ways–reading the methodology behind a study or poll, for example, or comparing the details of anecdotal evidence across various sources. Another factor to take into consideration is the original publication venue of the evidence itself. Recognizing the background of the publication adds to our understanding of the ethos of the evidence, as well as the sponsor’s motivation for collecting the evidence.

After examining the evidence, we can begin to consider the analysis of that evidence. The analysis reflects the perspective of the author and the publication in which it appears. Often, students stop their investigation into a source once they have determined its bias or perspective, but that is only the beginning. The real importance lies in the source’s purpose–why and how that perspective is being argued. Our history department uses the acronym SOAPA–Subject, Occasion, Author, Purpose, and Audience–to remind us to critically evaluate each aspect of a source.[2] This strategy has been particularly helpful in reminding us to think about the author’s purpose and how it shapes the analysis of their evidence.

I find it useful to think of every source, be it a journal article or a photograph, as an essay that selects and interprets evidence to support its thesis, but that comparison is not necessarily intuitive. This idea that all sources make an argument is easily overlooked, especially when we students are presented with historical documents which we sometimes subconsciously perceive as pure fact. In our 8th grade science classes, students first encounter the idea that nonfiction can be analyzed like literature. The lesson teaches students to consider the language of a source to determine what argument the author is making and what they want the audience to think, feel, or do.

Differentiating between evidence and analysis is the first step in considering the three essential questions: what is the evidence that is presented, what is the corresponding analysis, and what sources shaped each component? Answering these questions helps us understand:

  • Sources make arguments using evidence and analysis.
  • Evidence tells us what the source is using to make its argument.
  • Evaluating the origin and quality of the evidence contributes to our understanding of the strength of the argument.
  • Critically evaluating the publication venue of the source itself helps us recognize the perspective the analysis will try to validate.
  • Doing a close reading of the analysis in the source gives us insight into the author’s intention in making the argument.

In the case of the quote that started this whole journey, knowing that the evidence came from the Department of Justice and the analysis from CNN allows students to draw on any credibility offered by the DOJ’s statisticians and CNN’s popularity as a source of reporting. The students themselves attain credibility by demonstrating that their thinking is based upon rigorous sources.

________________

  1. Chris Hamby, Brian Ross, and Matthew Mosk, “Breathless and Burdened: Dying from black lung, buried by law and medicine,” The Center for Public Integrity, last modified October 30, 2013, accessed March 2, 2017. https://www.publicintegrity.org/2013/10/30/13637/johns-hopkins-medical-unit-rarely-finds-black-lung-helping-coal-industry-defeat.
  2. The College Board recommends a similar version, SOAPSTone, for its history APs.