What do we want students to know about our databases?

It has been just over a year since I emailed the AISL list, looking for a truly representational news database aimed at the K-12 market. The push for such a product is ongoing, and I hope to be able to update you more on that process soon (and please contact me if you would like to get involved in the campaign to drive change).

In the meantime, our library decided to assure that students would see themselves and each other in our digital resources, even if it did require accessing multiple databases to do so.

Having a larger number of databases is a challenge, as we all know, and we need to take a multi-pronged approach to ease adoption. Among our strategies are:

1. Recruiting student “Database Ambassadors.” Our students have grade-wide, homework-related group texts and so the Ambassadors will be on the scene when homework is hard to complete. They will remind classmates to reach out to librarians for help. I set out to have a few ambassadors in each grade, but students started asking if they could join the cadre and fully 10% of our 8th-12th grade students have asked to volunteer for this project.

Training starts today, and I am trying to figure out the few ideas I have to drive home. No use trying to familiarize them with a substantial number of databases during a study hall! My Research TAs tell me that the bare bones of the class I taught our Juniors are a critical part of the lesson….

2. The lesson/lecture for all 11th grade US History classes was an introduction to working with primary sources. The theme was “database business models as systemic injustice.” Some colleagues have asked me to share the slides from that class. Though they are not at all pretty, you can find them here. Honestly, I never thought to get emails with words like “moving” and “fascinating” related to a database lesson!

To be frank: none of the systemic issues with databases were news to students who have aspects of their identity that have been minoritized. It has been painful for me to realize of the substantial number of students who have felt alienated by our collection for so long.

Nonetheless, we move forward with anticipation. We believe that our students can both thrive on our more inclusive collection and simultaneously look at it with a critical eye. It is our job to help them do so.

If such a lesson feels like a bit more than you can bite off, check out Rebecca Hall’s graphic novel, Wake: The Hidden History of Women-Led Slave Revolts. Actually, read it whatever the case. In addition to history I never learned, it talks about silences in sources and how they impact our understanding of historical events.

Have a lovely week!

NOTE: This database work is a group effort that currently involves Sarah Levin (Urban School), Alea Stokes (Thayer Academy), and Sara Kelley-Mudie (Beaver Day School). Also, my awesome Library Director, Jole Seroff, who is supporting this work as an integral part of the program we provide our school. I’m just the only one who cannot stop talking about it. It is how I roll. 😉

Another encouragement to rest….

For those of you who know me, you are about to experience a thing I never thought would happen: I am not in a mood to think, write, or talk about research. I am sure this state is temporary, and I did have a plan for this post, but I am fortunate enough to be someplace where I spent many happy hours in childhood, so I’ll share the images that inspire me at the moment (though it may not be scientifically correct):

Be like these eagles…

Cattle Point Lighthouse, San Juan Island, WA

…when you find a place with a good view….

… stop and enjoy it.

Reuben Tarte County Park, San Juan Island, WA

Promise to have library-related content next time, but for now … wishing you joy however you find it.

Squeezing in a beloved annual project

Honestly, I have infographics on the brain right now. Welcome.

This year has been surprisingly busy and fulfilling on the instructional front, though nothing like past years or on almost any expected topics. I was thrilled, then, when our wonderful 7th grade History teacher reached out to ask me to help her wrap up the year with a lengthy infographics project we usually do in November.

Initially, when this collaboration began nine years ago (and before I came to this school, and shepherded by the wonderful Jole Seroff), the idea was to follow the unit on African empires with a look at positive aspects of life in African countries today. Time has taught us, sadly, that statistics do not give rise to seventh graders focusing on encouraging analysis. The project has endured nonetheless and continues morphing in the way good collaborations can sometimes do.

The way we teach this unit has streamlined over the years to make it easier for students, though it might have a bit of a fools rush in… aspect to it, as well. I’ll admit my desire to refresh it each year makes for a tremendous amount of work for me. It is also a big leap for students, and asks them to embrace some really new ways of thinking. But it is worth the growing pains — while high school students look back on it with some exaggerated shudders, there also exists some great love, and a handful of skills for which they voice deep appreciation in those later years.

Overall, I would argue that this is a process-based project: if you look at the final infographics they don’t tell the actual story. The insights offered in student presentations are much more telling. And the conversations we have over the course of this unit are varied and deep: Can we use data from different years in different graphs? Why doesn’t Rwanda have statistical data on prevalence of certain genetic diseases within its population the way the US does? Why doesn’t the US have an official literacy rate the way other countries do? Can we trust China’s numbers on incarceration? Explain that GDP thing again? What is “primary education”? This year brings: My hypothesis is that Russia has fewer women in prison than the US, but all my data shows that predictors of imprisonment are higher in Russia…. Do I need to change my thesis? (So many interesting conversations there.) And, hearing a roomful of seventh graders recommending that a classmate use “per capita” data in her graph instead of gross numbers, and explaining why… well, it is certainly a high point in my time as an educator.

This project has a lot of quirky methods and rules that have developed over the years, but they seem to help us get the project running smoothly. I’ve outlined the 3-4 week lesson plan below, for those who want to dig in. As always, very open to feedback and suggestions! Would love to hear in the comments if you do infographics or other data projects and what your objectives are.

Library’s data literacy objectives:
1. Develop a sense for the kinds of topics on which official statistics are collected and some of the specialized language used by statisticians.
2. Understand the difference between description and interpretation of a visualization. Be able to identify and produce both.
3. Understand that different visualizations are good for different types of data. Be able to select/produce the right kind.
4. Understand that evidence does not always have your topic mentioned in it. Broaden individual understanding of what can count as evidence.

Day 1:
1. What is an infographic? (Slides 1-2)
2. Virtual gallery walk of good and bad infographics (This year, rest of slides; most years we have a lot of infographics we hang around the room for the gallery walk.)
A. Have students list characteristics they see of different infographics, and then keep track of every time they saw that characteristic used, categorizing them into:
i. thesis, evidence, graphs, and design and
ii. helpful or unhelpful
B. Build class rubric (adapts to project rubric)

Day 2:
1. Look at different data visualizations, discuss the difference between description and interpretation of a visualization. Practice with the Banana Timeline.
A. Students start at their assigned slide and follow directions on slide 10.
i.Interpretation: This graph shows THAT…
ii. When can you use this type of visualization?
iii.Tips
B. Pair and share: introduce your graph to your partner, sharing what you wrote. Ask your partner a question about their graph.
C. As a class, build “Tips for graphing” so you will know how and when to use different data visualizations.
2. Read this quick summary of topic areas and pick one.

Day 3:
1. Childhood obesity practice:
A. Explain “working thesis/hypothesis” – can change
B. Your working thesis should have a comparison (across time, gender, place)
C. Example: “Over the past 20 years, childhood obesity has been a growing
problem in the United States.”
i. Identify the comparison.
ii. Brainstorm possible “subtopics.” Subtopics are related topics which, in this project, may not repeat words/ideas from your thesis (except time, gender, or place names). Students often come up with ideas like:
a. Hours of screen time
b. Percentage of schools with PE classes
c. Percentage of families who eat dinner together
d. Average distance to closest supermarket
e. Hours of homework
f. Participation in athletics
g. Cost of healthy food, like product
h. etc.
2. Students read background portfolios on their topic area:
A. Practice stepping stones as a class, then together look for stepping stones (No
pandemic? Do this in groups.)
C. Look at starting graphs and draft a working thesis.
NOTES:
1. Over the years, we have found it works best for us to provide them with graphs from which they chose one and write an interpretive statement communicating some of the information shown as their hypothesis, which can change as they develop their infographic.
2.This year, I was able to create these background reading documents, which I stuffed with references to indicators related to their topic. They used these background readings as stepping stone sources to plan the next round of research. For example, one of our topics was global female imprisonment, and the background reading included this passage:

…I learned that crime is not what really brought these women to prison. Far from it. It started with a lack of education, whose supply and quality is not equal for all. It starts with a lack of economic opportunities, which pushes these women to the petty survival crimes. The broken health system, the broken criminal system, the broken social-justice system. If any of these poor women fall through any of these cracks, the bottom of that chasm is a prison.

From this passage, they can note that indicators related to imprisonment include: education, economic opportunities, healthcare, etc.

Homework: Write a working thesis, complete noting potential subtopics mentioned in the background reading.

Day 4:
1. Introduce acceptable data sets. We provide all data. This is still hard and messy, to navigate, but just sending them to online databases (go look at UN Data!) is waaay too hard. So we give them this TOC (mostly links to spreadsheets I have downloaded and cleaned up).
2. Take out list of subtopics gleaned from the background reading and look through the TOC (we have them read the *whole* thing, saves arguments later!).
3. Pick a data set that covers a topic that will help contextualize your thesis (we talk quite a bit about how this is not really “evidence” since neither causation nor directionality is assured).

Homework: Draft your first graph.

NOTES:
1. We spend a *lot* of time demonstrating imperfect circles, terrible stick figures, and sloppy handwriting. They hear not to spend more than 10 minutes on the draft. We compliment students on the “draftiness” of their drafts.
2. We ask for screenshots of the data and the citation because the number of ways students find to lose track of their data as this project continues defies the imagination. Also, always better to build your citations as you go!

Next several classes:
Independent work. Each night, for four nights, students sketch a rough draft on paper of a visualization of one data set they planned to use as evidence. Each day, in class, we select five visualizations and discuss them as a class–each student has a chance to have one visualization complemented and critiqued. She tells the class her working thesis, describes her data visualization (projected on the screen), and tells us how it helps explain a systematic factor impacting her thesis. Then, she got three compliments, three questions, and three kinds of constructive feedback. We encourage student to take notes/jot down new ideas based on peer work and/or critiques.

When they do the fourth graph, we also ask them to write four “statistical statements” – a bit more data or other interpretive information to help flesh out the story their infographic tells.

Day 8:
1. Students get a piece of paper and post-its to do a mock up (wireframe) of their infographic.
2. When students turn in their bibliography and show their paper wireframe, they get their code for Pictochart and learn to set it up.

Day 9:
Assemble infographic on Pictochart

Day 10:
Prepare to present your infographic. You will tell us:
Explain your main argument or thesis statement. What do you intend for your infographic to convey? Explain why you were interested in this topic, and the information you were thinking you’d find when you started this project.
Explain 2 of your 4 graphs and how they relate to your thesis.
Explain 2 pieces of your additional evidence, and how they add to your argument
What was one aspect of your infographic that was difficult for you? How did you resolve this difficulty?
What information did you wish you had found that you were unable to find?
If you were to start this project again with the same topic, how would you approach your infographic differently?

Day 11-12:

Presentations (eight per day)

Goals for next year: Visual note-taking and further exploration

I suspect we all have one (at least). Some project we were hot on going into March, 2020, and have left dangling since. Please bear with me while I share my dropped passion project here, to get myself geared up for next year; I encourage you to share yours in the comments below to kick-start your own enthusiasm!

Note-taking has been an interesting animal: not one that interests me a lot, or that I was ever trained to teach, but a skill that definitely falls by the wayside in an age of copy-and-paste-into-a-Google-Doc availability. We are a proud NoodleTools school (which the students love-we literally have a cohort of Seniors pushing to endow NoodleTools to graduates in perpetuity as their Senior Gift, that is how much they love it), but we have needed some additional ways to help students read meaningfully. In particular, our Middle School teachers focus heavily on plagiarism, and have started thinking about its relationship to reading comprehension. So, we began wondering what kind of note-taking lessons we wanted to offer as a strategy for asking students to slow down and prioritize understanding.

Our experimental solution: visual note-taking.

By some bit of luck, I got to work with both our seventh grade science class and our seventh grade history class on a pilot of this endeavor.

In Science, students were preparing to design experiments for our Middle School science fair. Often, when researching and writing their proposals, their teacher noted a tendency to parrot language that they did not understand. So, as part of the lesson when we talked about NoodleTools and writing citations as a form of source evaluation, we also practiced drawing for comprehension:

  • Slides I made for class included samples made by my Research TAs when I was testing the usefulness of the approach, and then took a passage on transpiration for which the class assisted me in collaboratively build a visual note (none of us knew what it meant when we started, but we did when we finished).
  • Drawing together on the whiteboard and then showing my notes let me emphasize the “very drafty” nature of my notes, as opposed to something one might create for an art show.
  • Small groups has short passages on circulation systems and respiratory systems in insects and earthworms. Their prompt was to draw notes that they could understand to help them unpack the meaning of their reading.

Many of the students were vocal about hating the process — they found it so much slower than just writing down random words and sentences from their reading. However, their visual notes made it clear that they understood their reading very well:




Some weeks later, we returned to this strategy in History. Students were doing research for a Renaissance Dinner Party: They each learned about an assigned historical figure, created a class presentation, and then had to seat ten people covered by the presentations around a table at a dinner party in such a way that no fights would break out and all guests would be entertained.

For the research stage, we returned to the idea of visual note taking. This lesson was a longer process, covering several days, and including a number of different strategies for communicating learning in a manner where the language of the articles they read would not suit (such as a fake Twitter where the historical figures chatted and threw shade).

  • At the end of the first day, the students compiled a list of advice for visual note-taking.
  • We once again practiced as a class with drawing “messy stick figures,” and students started comparing and bragging about the messiness they achieved.
  • Students did note that they wanted to demonstrate emotions/interactions and to be able to tell individuals apart. I brought in iconography from the amazing Good Tickle Brain* and looked at how Mya Lixian Gosling’s very simple drawings of Shakespeare characters (for example, Cleopatra and Juliet), which seemed to help a lot.

Some students still felt frustrated (particularly those who felt more successful and comfortable memorizing and repeating), but some really interesting feedback did come my way. It is anecdotal, but impressive:

  • The day that students had to hand in their visual notes on three articles they read for homework, a bunch were waiting for me at the library. They explained that they had found it boring to re-draw the same material, and wanted to check if it was ok that they took all their notes on one set of pictures.
  • Let me rephrase: They synthesized their notes from several sources onto one set of images. Students naturally moved from a linear set of pull quotes, article-by-article, to integrated knowledge.
  • An elated student stopped by to tell me about how her “super-smart, intimidating” uncle had come by for dinner. She often felt nervous with him, because he always wanted to know what she was learning from school: “and I remembered everything, without even *looking* at my notes!”

This pilot felt meaningful to me. Genuinely understanding and remembering content, paired with natural synthesis is a holy grail I will happily continue to pursue. Later, I developed (but have not tested) a theoretical self-grading rubric to use with visual notes, based on a Verbal to Visual post:


I was able to take another brief stab with our Chemistry classes when they were supposed to be looking at how a range of experiments were conducted, but students tended to focus on the outcome because they often could not visualize the experiments themselves. Sixth grade Science took on a drawing project this year to practice understanding relative dating and geology. And, with our History department’s Advanced Topics Research & Writing class, we now how three years of evidence of many students radically relearning note-taking for deep research.

But I really want to develop a much deeper understanding of note-taking, and get our students experimenting with different methods to find the best fit for themselves. I’ve also been dreaming of doing an exhibit of employee’s notes — showing students that the adults on campus have developed a range of methods that work personally for each of us. I worry that the hurry to “get through the workload” makes more work as students develop frictionless paths that feel like less work…but since they tend to sidestep understanding, I suspect they end up taking much more work in the long-term, with less actual learning.

So — that is something I am excited to get back to work on. How about you?

*And don’t forget to check out Good Tickle Brain — I doubt you will regret it!

Agency from anywhere: Why you should learn to edit Wikipedia, and teach your students, too!

Recently, my younger child declared I have a new motto. He even put it on a shirt for me:



“When life gives you lemons, write Wikipedia pages about amazing women”

My child observed that I spent the afternoon of January 6, 2021, watching coverage of the insurrection in DC while editing Wikipedia furiously, and that I used editing to manage my worries during other periods of uncertainty over the last year. He is not wrong, but here is how I see it:

  • From my armchair I tangle with systemic inequities arising from the specific guidelines meant to make Wikipedia “more reliable.” 
  • Even while sheltering in place, I have the ability to broaden the narrative of our nation and our world as it is shaped by a source which is, arguably, a de facto arbiter of truth in our time.

While I have been guiding students in Wikipedia editing lessons since about 2010, I worked primarily with upper elementary and middle school students editing Simple English Wikipedia. When I joined the Castilleja faculty in 2013, I took over a similar project my library director, Jole Seroff, had developed. Along with the project came her notes on gender imbalance among editors and how the skew towards male editors (85% of editors, and something like 91% of all edits) impacted the content we see when we access Wikipedia. A 2011 article in the New York Times noted that who edits impacts the emphasis of the source, comparing a four-paragraph page on friendship bracelets (“A topic generally restricted to teenage girls”) to the much longer page about “something boys might favor” like baseball cards. Setting aside for a moment the gendering of topics, it is notable that today these discrepancies remain:

Sources: “All page views: ‘Baseball Cards'” and “All page views: ‘Friendship Bracelets'”

Similarly, a study released in October 2014, noted that only 15.53% of English Wikipedia’s biographies were about women. A number of groups, including WomenInRed, have focused on adding biographies about females, and they count that number rising to 18.79% as of 15 March 2021. Mountains of evidence point to lower number of pages being written about women and topics to do with women, as well as fewer editors adding information and a significantly higher deletion rate of pages about women/women’s issues because the subjects are apparently “not notable” (especially regarding STEM-focused topics).

If the numbers are so grim for women, imagine what inclusion might look like for other individuals and topics related to minoritized identities.

During the 2019-2020 school year some of my high school students became interested in hosting an edit-a-thon, and I decided it was time to actually learn how to edit for real. I attended my first in-person edit-a-thon at a local library in February, and then everything shut down. In June, my students and I decided to host a virtual edit-a-thon for Upper School students, and the real fun began.

In preparation, a number of our school librarian colleagues kindly joined me in an experimental edit-a-thon, which sufficed to demonstrate that I had picked a terrible way to organize my event. However, that afternoon also demonstrated the value of editing in community, as we each noticed different aspects of systemic prejudices in the structure of this venerated source. For example, one of our number is a classroom teacher in an English department, with a specialty in Southeast Asian American Literature. When she decided to work on the page Asian American Literature, another of our number called our attention to the Talk page, where editors discuss issues and challenges that arise in writing the page itself. In addition, due to the very reasonable desire to keep an eye on coverage in specific fields, and point out what work needs to be done, WikiProjects on various topics rate the importance of specific pages under their purview, like this:

Talk page for Asian American literature

…in which WikiProject Literature (that is, people who are interested in Wikiedia’s coverage in the field of literature) rated the Asian American literature page Low-importance.

Similarly, I was reading up on Patricia Roberts Harris. She was the first Black American female: 1. ambassador, 2. cabinet member (third Black American cabinet member overall), 3. dean of a law school, and 4. director of a Fortune 500 corporation. Here is her talk page:

Talk page for Patricia Roberts Harris

Over time, that original meeting of teachers grew into a weekly editing group. We learn by doing together, and we have learned very well just how hard it is to prove notability for genuinely notable people of color. It was actually in trying to set up a middle school Wikipedia editing project in 2018, covering notable female activists, that I really ran up against to problem of databases containing predominantly-white-perspective sources and the challenges that ensued in finding articles about non-white, non-cis-male individuals. That lesson has held firm as I try to write about women of color and struggle to meet the standard that Wikipedia articles should be “based on reliable, published sources,” meeting Wikipedia’s definition of reliable sources. There is no question that these guidelines are needed so that people do not fill pages with self-promotional material, as often used to happen. However, there is also no question that the guidelines to block self-promotion make it extremely hard to write about many genuinely notable people, as well, especially if they are not media darlings.

As an instructional librarian, I focus on teaching research skills. Therefore, I find joy in digging and in piecing together sources and arguing for their reliability (when necessary), all while avoiding running afoul of Wikipedia’s “No original research” policy. I’ve come to believe strongly in the many benefits of teaching others to edit and editing in community. I now help run three Wikipedia editing groups for: alums from my college, my students and colleagues, and other librarians/teachers.

Editing Wikipedia is a way to:

  1. “Do the work”:
    1. Decolonize your mind – if the only astrophysicist you have spent time thinking about is a LatinX transgender individual, then the picture you have in your head of an astrophysicist will be of a LatinX transgender individual
    2. Make people with minoritized identities discoverable
    3. Create or expand or improve pages that will be seen by millions of people – the least-used page I have worked on has been accessed 14 times since March 1, others have been accessed several thousand times
    4. Give others access to role models – a.k.a.: the perfect gift — I add women to Wikipedia as graduation gifts for young women who might not easily see role models in fields that interest them
  2. Build information literacy skills (for students):
    1. Explore the notion that “authority is constructed and contextual”
    2. Develop a strong sense of what a range of authoritative sources might look like
    3. Synthesize evidence to create a narrative
    4. Practice writing in the register of an encyclopedia
    5. Experience gatekeeping and its impact on knowledge construction
    6. Question why needed systems give rise to systemic prejudice
    7. Encounter systemic racism and other systemic prejudices and begin to understand their prevalence and impact
    8. Construct authority

If you would like to learn more about editing Wikipedia with students or for yourself, please join Corey Baker, Amy Pelman, Linda Swarlis, and myself at the upcoming AISL conference on April 9 for “Equity through Editing: Contributing to Wikipedia for Social Justice Inside and Outside the Classroom,” or reach out to any of us for more information.

Imagining Multiple Perspectives: Experimenting with “Priority-Based Perspectives”

This year, our school’s annual, 40+ hour, 5-month-long, co-curricular project with our ninth graders took place over six hours during the first week of January.

During this project students work in groups to research and ultimately present about a social issue of their choice to the grade-level parents. While this year’s nine-person faculty team determined that students would still make a brief Flipgrid video on their topic as a final product, I volunteered to design the week-long mini-course in which they would research and present their topics. I then had to decide what I really, really wanted our students to learn. Since our first run, eight years ago, students have consistently been asked to investigate multiple points of view with regard to their topic, and I have just as consistently been struck with how challenging it is for them to imagine what those viewpoints might be. So, I decided to focus on skills related to searching for and identifying multiple points of view. Specifically, our one requirement for the week was that they identify at least three perspectives on their topic.

Before winter break, students had studied Early Modern Islamic Empires in World History, and had submitted connections they saw among the political history of the Safavids, Mughals, and Ottomans and issues of global health and/or justice in the world today. I took these contemporary connections and, based on the topics each student highlighted, assigned them to groups. Since our time was so short, I framed broad research questions for each group based on their expressed interests. Almost every student articulated a pressing engagement with how personal identifiers played into political leaders’ use of power to privilege or repress different people under their influence, across a range of topics from voting rights to vaccine development to rights of women in Muslim countries (see the outcome of this last topic below).

They had their topics. I had my learning objective. Now, I just had to figure out how to teach the skill set of searching for and identifying multiple perspectives … in 40 minutes or less.

Through a day-long debate with my 20-year-old offspring, we formulated the following approach to identifying multiple perspectives. Though I taught it to 64 students at once in a Zoom meeting, I would definitively say my thinking is still in draft form. Please help me kick the tires and see if it holds up! I would be deeply grateful for critiques and feedback.


It always feels helpful to start with a framing activity that helps to ease students into the complexities we will be tackling in a lesson. In this case, I wanted to get them thinking about how the “government’s job” is not a unitary or settled notion. By adapting a few questions from the (wonderful!) World Values Survey and a recent budget survey done by the city in which our school is located, I asked them to answer four questions about what governments should prioritize.



Though I did briefly show them results to demonstrate that, even in the “room,” we had lots of varying opinions, I cared less about their answers and more about the students contemplating competing priorities.

A bar chart recording responses from class survey demonstrate that everyone has different opinions about the role or priorities of government.

Ultimately, we framed the lesson around the notion of “Priority-based perspectives,” the idea that points of view vary because individuals and organizations must sort through conflicting needs, and some must ultimately take precedence. I drafted a list of what some of these categories of priorities might be:

Image of a slide from class, with a stamp on it saying: DRAFT: Some categories of priorities: moral, economic, logistical, public relations/political, allegiance

The big epiphany I had (though it seems obvious in retrospect) was that my students were never thinking beyond the moral concepts that drove their own interest in their topics. They simply don’t have the life experience to suggest another path for investigation. So, we spent some time discussing that, for example, no matter how strongly you consider rest a moral right, the need to feed and house your family might still require a higher priority than time to rest. Similarly, you may morally believe that everyone has the right of equal access to a COVID vaccine, but we have been witnessing the real logistical challenges of efficient and equitable distribution.

One element that turned out to be pivotally helpful was that this construct moved students away from the binary pro/con approach for identifying perspectives (“Who agrees with me? Smart people. Who has a different opinion? Mean people.”) and instead started their research in a fundamentally different place: “What are the economic considerations of vaccine development? What different priorities might be competing? What are the logistical considerations of vaccine development? What different priorities might be competing with regard to logistical issues?” and so on. It removed their thinking from themselves and their closely held beliefs and allowed them space to be curious about what issues and priorities might exist.

Of course, we reviewed a few key search tips:

1. Pay attention to search terms, which will yield specific POVs:
— For example: Undocumented workers, illegal aliens, birthright citizenship
2. Search for your answer, not your question:
— Think about what you might expect someone to write about a topic and search for that.
— What words might someone use when they are talking about “morals”? How about “economic”? For example: budget, cost, price tag
3. Consider: how might people with the same goals have different priorities that lead to different POVs?
4. Remember to use stepping stone sources:
— Identify expert vocabulary/pertinent search terms that appear in the sources you have read so far. For example: CRISPR, designer babies, genetic engineering
— Who are some stakeholders mentioned in your sources who have points of view on this topic (people, groups, or organizations who care about the issue)? For example:
1.”According to the Department of Justice…”
2. “Emails leaked by the whistleblower”
3. “Professor Arvind Gupta, a researcher at the University of Oklahoma…”

There was one final issue that required addressing in this lesson: what does one do with points of view that do not meet with standards of evidence-based-reasoning? In the past, I have taught finding various political perspectives and a separate lesson on more diverse news consumption generally (finally this year I pointed out to a colleague that selecting sources by left-leaning and right-leaning just left out too many people, and ethnic, religious, and other community news sources needed to be included in any list of sources for reading political news). Yet, I try to balance understanding various perspectives against including harmful or factually incorrect perspectives in the classroom. For example, we do not accept as factual perspectives disproven medical studies (they may be mentioned for influence, but not presented as informational sources). I’ve never quite figured out how to walk that line effectively.

In this lesson, I drew on the World History course theme of historical empathy, and explained that identifying priority-based perspectives is an interstitial step that assists in identifying a range of perspectives; doing so also helps us understand the mechanisms that create structures in our world today, including the persistence of structural inequity. After identifying different priorities, and (potentially) sources that speak from the perspective of those authorities, it is the researcher’s job to evaluate the source (or, as we teach, identify the context and construction of the source’s authority) and determine whether it passes muster in our evidence-based environment.

The faculty for the ninth grade project all agreed that the students’ work this year was quite strong, despite the various emotional challenges everyone faced during our six hours of project time (including the events of January 6, 2021). I was particularly proud of the group that wanted to focus on women in Islam, as they moved themselves beyond a blind critique of the “other” and ended up delving deeply into a complex set of women’s perspectives about being a hijabi.

So, the idea of priority-based perspectives seems to work. Yet, I suspect I am missing (various) pitfalls or have elements lacking clarity. Thoughts?

News databases: Diversity without equity or inclusion

The Problem

Back in September, 2020, I sent out a call for help across AISL and other school librarian-oriented lists in hopes of finding databases that provide “diverse, inclusive, and equitable access to perspectives mirroring the composition of our country in magazines, historical newspapers, and contemporary news.” Generally, database companies sell “core” collections that are positioned as “high quality sources,” comprised almost entirely of white-perspective news outlets. Then they up-sell from a menu of discrete “ethnic” packages to provide “alternate perspectives.” Students deserve better.

Thank you to the many folks who responded hoping to hear of a good database in which to invest. Sadly, the answer is…so far I’ve found no way to buy this unicorn of the database world. Ultimately, I started doing my own diversity audit of our databases and others on the market to try to better articulate the nature of the problem.

I am currently only part way through this process. First semester ended up (happily) being much more crowded with instruction than I had anticipated. Even the terminology I use to think about this set of issues is still in crude form. Here is an update on what I have learned so far, however. To date I have focused on US news, historical and contemporary, and have only been able to compare offerings from two companies. This work has served — at the very least — as evidence that the problem is real and pressing.

Inclusion

In the fall, I had not yet fully realized the insidious nature of the juxtaposition we often attribute to databases: quality sources vs. alternative perspectives. I’ve been sitting with this formulation increasingly in the intervening months, and contemplating how our professional narrative around databases is driven by the marketing efforts of the database companies themselves. Consider the act of marketing a database as “providing researchers access to essential, often overlooked perspectives” that exists because the perspectives have been intentionally overlooked and isolated to sell us another database. So how much does the title list of an intentionally curated “ethnic” database (which mysteriously includes the LGBTQ+ collection, by the way) overlap with a product intended for high school?

ProQuest: Compared title lists for Research Library Prep and Ethnic Newswatch databases.

Please note that the “Overlap (%)” column conveys how many of the “specialized” Ethnic Newswatch titles also appear in “general” Research Library Prep. It does not convey the percentage of Research Library Prep that are/are not white perspective — those numbers would apparently be vanishingly small. 

An issue that struck me immediately as I got started was that scholarly journals comprise, by far, the largest mass of content in Ethnic Newswatch that is also available in Research Library Prep. These sources differ distinctly from newspapers or popular magazines; academic discourse may well be quite removed from the community it studies. That is, a large percentage of the authorial and editorial work is carried out within a realm of authority modeled on European institutions and constructed in our academic halls of privilege. To put it plainly: the perspectives appearing in the University of Pennsylvania Press’ Hispanic Review may not reflect community voices in the same way that those appearing in La Prensa Texas newspaper do. Both source types provide important points of view; their creation does not serve the same purpose.

Important as it is to have a diversity of voices in our scholarly works, they provide fundamentally different types of evidence from newspapers. Not to mention, they are not accessible to most K-12 students.

Gale: General OneFile, In Context: High School, OneFile: High School Edition, OneFile: News, In Context: US History

It has been challenging to figure out how to do a diversity audit, but since many database companies seem to start monetizing diversity with Black American newspapers, I decided to work from lists of existing and historical Black papers, including: National Newspaper Publishers Association Current Members and Princeton University Library – African-American Newspapers (1829-present). I used titles from these lists to search within Gale’s title lists, and found:

Checking Black American newspaper titles against Gale title lists yielded vanishingly few overlaps.

In the process of looking at the titles that are listed, the Atlanta Daily World and the Chicago Defender — historically both very important publications in the 20th Century United States — only had coverage from 2014-present, with exceptions from 2015/16 to the present. Meaning, in fact, they only have a handful of issues of each paper.

Once again, these databases provide news sources that almost entirely reside within historically white readerships.

Equity

In another sense, it does not functionally matter if a database includes sources from diverse sets of communities. When the algorithm privileges white perspective publications, searchers may never encounter other points of view.

Spot checks of ProQuest’s ranking of newspaper results in Research Library Prep confirmed that their methods for ranking heavily favored specific titles. Specifically, the New York Times dominated results, with a smattering of hits from the Chicago Tribune, Washington Post, US Fed News Service and Targeted News Service

I ran a series of searches, noting how many unique titles were returned for each search, as well as those titles’ spreads across the top 100 results. In essence, how many pages would I need to scroll through to access more than a few titles? I searched for [ the ] — as a word that appears universally in English-language newspapers — and also for words like [ miami ], [ skagit ], and [governor] — each of which strongly suggest local news. In every case, the results looked something like the results for [ the ]:

  • Returned 81 unique titles
  • Top 100 results:
    • 95 results from the NYT
    • Other titles ranked: #37, 41, 71, 91, 95
    • Other publications in the top 100 results: Chicago Tribune, Washington Post, Targeted News Service 

Progress!

However, there is good news. A Gale sales rep who is on one or more school library lists began wondering about this issue themselves, and carried out an independent audit that they then presented to their acquisitions department. As a result, when I last checked in this past November, Gale publisher relations personnel have identified:

  • Licensed periodicals where the issues aren’t current
    • Updates are in various states of progress 
  • Important periodicals with lapsed agreements 
    • Updates are in various states of progress  
  • Over 140 new periodicals from the following communities:  “African Americans, Arab Americans, Hispanic Americans/ Latinx, Native Americans, Ability Diverse, LGBTQ+, Women, & more”
    • Requests have been sent to publisher relations to pursue license agreements

Though not within the scope of my current work, Gale has also taken a look at their reference overviews and biographies and have made efforts to offer more coverage, as well.

The solution?

Does this issue interest you? Would you like to join me in fighting for single databases that are diverse, equitable, and inclusive? Whether you would like to audit a database you have, suggest a consistent method for auditing, share findings at your state conference, or talk to your database companies once we have a clear report — kindly reach out. If the idea is that we are better together, let’s unite and make a difference!

I am deeply grateful to my director, Jole Seroff, for being so invested in and supportive of this exploration, and colleague Sara Kelley-Mudie for helping me focus my thinking.

Distinguishing evidence from analysis: A student’s perspective on the first step in source evaluation

Sara Zoroufy is a junior and the Research Teaching Assistant for the Castilleja School library. Inspired by Nora Murphy’s work on source literacy, Sara chose to spend this year observing research lessons and unpacking how she and other students think about sources. Her work helps inform lesson planning. Here, she shares an idea she has been contemplating recently.

“CNN reports that the Justice Department found the following statistics…”

During a presentation in our tenth grade government class, this phrase caught my attention. Why would a speaker attribute a statistic to two different sources? I have been thinking about this turn of phrase for a long time, trying to understand precisely why it troubled me. Recently, I realized that students struggle to distinguish factual evidence from a source’s analysis of that evidence. In the example above, the student was having trouble determining what type of information she was citing and which source was responsible for the creation of that information. Without separating evidence from analysis, we can neither evaluate nor properly cite a source. I tried to draw a visual to help myself understand how a source breaks down into these components, which culminated in this flowchart:

Mapping these concepts in this way helped me identify a number of key points in the process of evaluating a source. I began to think that the essential questions that students should initially ask when faced with a particular source are: what is the evidence that is presented, what is the corresponding analysis, and what sources shaped each component?

Asking these questions is the first step in unpacking a source, and the answers are not always immediately clear when students encounter unfamiliar genres of writing. This year, my grade was presented with an excerpt from a Pulitzer prize winning piece of investigative journalism about the diagnosis of black lung in coal miners. We were asked to identify the sources of the statistics in the article. No one was able to locate this information because journalistic convention dictates integrating the names of sources into the text, as opposed to employing parenthetical citations that students use in their own writing. For example, just prior to starting a bullet pointed list of statistical evidence, the article said, “The Center [for Public Integrity] recorded key information about these cases, analyzed [the medical expert’s] reports and testimony, consulted medical literature and interviewed leading doctors.”[1]  Since the students weren’t accustomed to this particular form of citation, many of us responded that no sources were given.

Students had been instructed to pinpoint the evidence in the article and label it with an “E” and to identify and label its sources with an “S.” As I sat with Tasha Bergson-Michelson, our instructional librarian, and considered my flowchart in relation to the lesson, we realized that the instruction had skipped over several crucial steps in the process of identifying the evidence. This experience made it clear that identifying the sources of evidence can be confusing, and that simply telling students to exercise that skill was not effective. Rather, the development of this skill requires explicit instruction and opportunities to focus on practicing it.

Once we’ve identified the source’s evidence and where it came from, we are able to further evaluate it. Depending on the type of evidence, we can investigate its quality and veracity in different ways–reading the methodology behind a study or poll, for example, or comparing the details of anecdotal evidence across various sources. Another factor to take into consideration is the original publication venue of the evidence itself. Recognizing the background of the publication adds to our understanding of the ethos of the evidence, as well as the sponsor’s motivation for collecting the evidence.

After examining the evidence, we can begin to consider the analysis of that evidence. The analysis reflects the perspective of the author and the publication in which it appears. Often, students stop their investigation into a source once they have determined its bias or perspective, but that is only the beginning. The real importance lies in the source’s purpose–why and how that perspective is being argued. Our history department uses the acronym SOAPA–Subject, Occasion, Author, Purpose, and Audience–to remind us to critically evaluate each aspect of a source.[2] This strategy has been particularly helpful in reminding us to think about the author’s purpose and how it shapes the analysis of their evidence.

I find it useful to think of every source, be it a journal article or a photograph, as an essay that selects and interprets evidence to support its thesis, but that comparison is not necessarily intuitive. This idea that all sources make an argument is easily overlooked, especially when we students are presented with historical documents which we sometimes subconsciously perceive as pure fact. In our 8th grade science classes, students first encounter the idea that nonfiction can be analyzed like literature. The lesson teaches students to consider the language of a source to determine what argument the author is making and what they want the audience to think, feel, or do.

Differentiating between evidence and analysis is the first step in considering the three essential questions: what is the evidence that is presented, what is the corresponding analysis, and what sources shaped each component? Answering these questions helps us understand:

  • Sources make arguments using evidence and analysis.
  • Evidence tells us what the source is using to make its argument.
  • Evaluating the origin and quality of the evidence contributes to our understanding of the strength of the argument.
  • Critically evaluating the publication venue of the source itself helps us recognize the perspective the analysis will try to validate.
  • Doing a close reading of the analysis in the source gives us insight into the author’s intention in making the argument.

In the case of the quote that started this whole journey, knowing that the evidence came from the Department of Justice and the analysis from CNN allows students to draw on any credibility offered by the DOJ’s statisticians and CNN’s popularity as a source of reporting. The students themselves attain credibility by demonstrating that their thinking is based upon rigorous sources.

________________

  1. Chris Hamby, Brian Ross, and Matthew Mosk, “Breathless and Burdened: Dying from black lung, buried by law and medicine,” The Center for Public Integrity, last modified October 30, 2013, accessed March 2, 2017. https://www.publicintegrity.org/2013/10/30/13637/johns-hopkins-medical-unit-rarely-finds-black-lung-helping-coal-industry-defeat.
  2. The College Board recommends a similar version, SOAPSTone, for its history APs.

 

Reading News across the Political Spectrum

Last spring, I had to confront a gaping hole in my professional knowledge. But that is jumping ahead. Let’s start at the beginning, with our students.

It began with a project in the 10th grade American Political Systems class. Working in pairs, students were to select two articles with differing viewpoints about a contemporary issue, then lead a current events discussion. Last spring was the first time I had the opportunity to meet with students about their article choices. Inspired by Nora Murphy’s work on source literacy, I asked students to talk to me about what types of source they were looking at, and why they felt each article was sufficiently authoritative.

Repeatedly, I found myself facing students’ inability to distinguish between what felt good and what was good quality. Generally falling along a political spectrum, articles that aligned with what students already believed earned a rating of “reliable” … and the other sources were all considered equally foreign and indigestible. Certainly, they were applying no particular standards to finding authoritative expressions of the opposite viewpoint, beyond maybe a shrugged: “Well, it was in the library databases,” as if that assured authority, or “It was ranked high in my search results.” In these discussions, students generally knew how to think about the authority of authors, but it was clear that it did not occur to most of them to consider how the identity of the publisher shaped a piece. As their research skills educator, I found myself demanding, again and again, that there were standards of rigor to which we must hold every publication, which provided a way to identify quality sources at many points along the political spectrum.

The first step we covered was simply learning a bit more about what possible points along the political spectrum were, and identifying where a particular publication situated itself. Wikipedia was invaluable in these investigations. In many cases, the first few sentences of entries on a specific media outlet provided a lot of vocabulary (progressive, libertarian, neoconservative, paleoconservative) that gave us good talking points for getting started. Sections on editorial staff, board members, funding, and past controversies were also helpful, if not taken in isolation. “About” pages could sometimes be useful, but were often so full of obfuscating language as to be impossible to parse (we first addressed this difference in readability in 9th grade, when students tried to decode what Sputnik News was). However, a publication’s media kit for advertisers and their submission guidelines for writers were often much clearer indicators. These tools helped us build a crucial understanding about the nature of the source we were encountering. It did less to help us understand rigor.

One trouble is that the standards of rigor remain, well, elusive. One high school student and I spent the entire subsequent summer debating how to define those qualities in a manner that would be findable and useful: If we cannot find out about editorial process, what is a reasonable way to measure the same thing? What role does the size and demographic of readership play in our understanding of a publication? How does it matter if a publication was “born digital?” If most people view media by clicking through from social networks, what weight do we give to how clickbait-y their headlines feel? What does the publication articulate as its own claim to authority? But we barely scratched the surface.

Blur: How to Know What’s True in the Age of Information Overload better helped me frame my thinking. It identified four forms that “news” reporting might take. Two, in particular, caught my eye. “Journalism of verification” requires journalists to examine evidence relating to a question to determine what conclusions to draw, whereas “journalism of affirmation” is “a new political media that builds loyalty less on accuracy, completeness, or verification than on affirming the beliefs of its audiences, and so tends to cherry-pick information that serves that purpose” (34). While I wanted my students to be selecting sources that followed the model of verification, they tended to encounter and be attracted to sources that were rooted in affirmation. Reading journalism of affirmation is rewarding; it articulates the reader’s feelings clearly, makes her feel smart, and is widely available. So not making a practice of identifying a source’s model — verification, affirmation, or other — is how we end up with a gulf between what feels good and good quality.

As a result of these conversations, this fall I had the opportunity to have an hour with the now-11th graders to help them set up news feeds. My assignment was to make sure every student was regularly following some kind of news. After thinking long and hard, I turned back to Blur and decided to break the lesson into two parts: 1. Practicing how to identify sources and evidence in journalistic writing, and 2. Giving every student an opportunity to identify a method for following news on a regular basis. It was a first time through this topic; the lesson plans linked to above have a great deal of room for growth. The lesson culminated in a plea that students commit to reading a variety of good quality sources that could help them experience a range of perspectives.

Which brings us back to the gaping hole in my professional knowledge that I mentioned at the outset. Because the truth was that last spring, while I was urging students to pick sources based on rigor rather than emotions, I found myself limited in the rigorous sources I knew to recommend. Let me be more honest: because I had not cultivated my own knowledge of good quality conservative sources, I knew of only three. Three sources that I kept pointing to, again and again. Frankly, I had to struggle continually with my own snap judgments. My knowledge proved completely insufficient, and I barely sounded convincing, even to myself. How could I stand for a rule of rigor across the spectrum if I was unable to point students to acceptable options?

So, I decided I had to educate myself, which is what I was trying to do when I attended my first AISL conference in Los Angeles last year. Many of you kindly allowed me to question you about your collections, and your thoughts on a variety of news sources.

Slowly, my list of sources from around the spectrum began to grow. It is still smaller than I would like. But I am fortunate to have three learners in my life who are particularly committed to notions of source literacy, reading broadly, and engaging with multiple narratives. One student and I had conversations about how floods of articles from two or three favorite news sources started to feel like seeing the same stories over and over again; it was taking 90 minutes a day to weed through and find new ideas or events. We determined that it was better to select a small number of media outlets representing differing perspectives, so that each article would offer a new point of view, even if the topic repeated. Another argued that there were more varieties of narratives than just international and political; sources from various US geographic regions and affinity groups (news from ethnic, religious, and other groups) went on my list. Work with each of those students — including my Research TA, who chose to focus on source literacy — is messy and ongoing, and would be a post unto itself. Each of them did, however, respond to the questions I was asking of myself with a passion and commitment that transformed my curiosity into something I actually needed to live up to. They made me do more than just make lists — they made me expand my reading.

The final element of the 11th graders’ lesson this fall looked at what it might mean to access multiple perspectives through news. Given the fractious environment at the time, I decided simply to share a selected list from the sources I was following (at the time, I was testing close to 150 different news sources) and let students explore for themselves. I am grateful to colleague Connie Williams for making me realize that the list I shared with students is not founded on consistent standards. For example, my list of international sources includes many options that fall under what Murphy would term “necessary bias,” aimed at assuring exposure to various narratives being promoted around the world. My affinity group media options similarly look to expand my own ability to access and hear the multiplicity of narratives experienced by the United States’ diverse population, while my regional newspapers attempt to balance that same need with a sense of editorial rigor. My political spectrum is where I look most critically at the use of sources, evidence, and careful argumentation. It seems that, in the process of writing these words, I have discovered the next stage of work I have to do on my own thinking.

Nevertheless, I am gaining something deeply important from both investigating potential sources and reading those I selected on a somewhat regular basis. It is a time-consuming process, and it can be quite unsettling to encounter religious, political, or ethnic viewpoints to which I have not really been exposed before. Yet I also find it immensely enlightening to read about an issue in an expressly paleoconservative, African-American, or Catholic leaning publication, or any publication outside my regular media diet. Seeing a reasoned argument proceeding from a different set of values or experiences often provides me with crucial details that are lost when filtered through more familiar media. Often, I encounter statements that make a lightbulb go off for me — I’ve found myself reading and re-reading a portion of the Constitution, until I can finally figure out what someone else is seeing in the words.

Ultimately, it comes down to a real question of what my professional values — and my human values — really are. Just as journalistic ethics require not neutrality but that conclusions should arise from fact-based evidence, every time I work with students I drill into them that research is not about starting with something you believe to be true and cherry-picking evidence to prove it. Rather, we look at a range of rigorously supported viewpoints and draw evidence-based conclusions. Our library teaches students to use close reading and a knowledge of logical fallacies to unpack how word choice and argumentation impact readers’ emotional responses to nonfiction. In our program, those are becoming core source evaluation skills. If I actually stand by all parts of this curriculum, then it seems reasonable to me to expect that I should be accessing multiple, rigorous, and diverse viewpoints in my own reading of news, before drawing conclusions. When I hold myself to those standards, I can do more than just telling students to avoid fake news; I can offer them a positive range of news options on which to draw. When I do that, I model for students that I live by what I teach.

NOTE: I should have asked this initially, but I would love for anyone to share ideas for sources with me. I hope to build a much more rigorous list over time.