![[Critical Mass]](/archives/cmlogo.gif)
![[dot]](/archives/cmdot.gif){kind=small}
March 19, 2009
![[feather]](/archives/feather.gif){kind=small}
Humanizing assessment
In 1991, when I was a first year grad student at Michigan, I took a wonderful course on American culture co-taught by June Howard (who is still on the English department faculty at Michigan) and David Scobey (who is now a historian at Bates). Part of what was great about the course was the sheer volume of amazing reading we did--from Howells and Jewett and Dreiser, to Veblen and Taylor and Gilman, to secondary writing on everything from the Chicago World's Fair to the Pullman company's model town. And part of what was great about the course was its tone--Howard and Scobey were both aware that they were continually crossing out of their disciplinary comfort zones, the one into literary analysis, the other into historiography. They foregrounded this, made their own uncertainties about how to think about certain events, patterns, and kinds of writing a focal point of the course. This allowed methodology--and, vitally, intellectual modesty and curiosity--to become a major theme of the course. It was formative for me, and I have never forgotten it.
I came to the course as a lit student with only the barest idea how to use the library, and no clue about what historians do, or how much fun they have doing it. But that class opened up a lot for me. I'll always be stuck on history, stuck on the real joy of the moment when straightforward analysis gives way to genuine constructive confusion--and then, if you stick with it long enough, to creative if tentative synthesis. That was one hell of a course, with two wonderful teachers who were, together, greater than the sum of their parts.
I was reminded of all that this morning as I scanned my daily email from the people at Inside Higher Ed. Who should be writing about higher ed assessment but David Scobey? And what he has to say about it is characteristically wise:
Many self-styled reformers have called on (and called out) colleges and universities to systematically assess how well we educate our students. Margaret Spellings' Commission on the Future of Higher Education decried our lack of clear, comparative measures of academic success. Some administrators have responded with initiatives like the Collegiate Learning Assessment, designed to quantify and rank the success with which institutions achieve liberal learning outcomes. "Assessment," in short, has become a word to conjure with.
In the face of this rising sentiment, many humanities faculty respond like King Canute, taking arms against the tide. We humanists are notoriously hostile to systems of assessment. We tend to believe that the most important effects of a humanities education resist measurement: nuanced communication skills, reflective dialogue between theory and interpretation, attention to context and complexity. Conversely the outcomes that can be most readily measured seem to us the least salient: informational content in a sub-discipline, performance of competent analyses according to check-listed rubrics.Humanists tend also to look askance at the abbreviated time-frame of many assessment tools, whether these tools test student performance at a single moment or mark change in a "formative-summative" sequence. To the contrary (our experience tells us), the most powerful learning in the humanities takes place in ways that are meandering, iterative, self-reflexive, and unpredictable. "We murder to dissect," Wordsworth famously wrote. The machinery of evaluation, we worry, threatens to kill the soul in the machine.
And so we become assessment Luddites. We sabotage efforts to systematically evaluate how well we are educating students via the time-honored faculty practices of filibustering critique, committee inertia, or sheer disengagement. My own experience in teaching American Studies and leading civic-engagement programs makes me sympathetic to the skepticism. I have rarely seen evaluative tools that do justice to my experience or that of my students. And yet, I want to argue, it is time for humanists to move beyond Luddism and constructively engage the advocates for student assessment.
There are two overriding reasons: one strategic, the other educational.
First, the current calls for assessment are part of a larger crisis of legitimacy in U.S. higher education--a crisis that faculty ignore at our peril. The crisis has many causes: tuition increases that have long outstripped the growth in the cost of living, the erosion of educational access and attainment, culture wars over perceived political bias in the academy. In the context of these discontents, calls for "accountability" have sometimes masked efforts to police campus politics; and too often they have oriented higher education toward instrumental goals of job training and economic competitiveness.
Yet (especially in a time of scarcity and crisis) it is a fair challenge to the academy that we be accountable for the vast resources and autonomy to which we lay claim--that we offer a compelling argument about our value to the larger society. Precisely because others have their own reductionist agendas of how to measure success in higher education, we need to offer our own vision of means and ends. The most self-damaging response we can make is to build a defensive bulwark of guild privileges around ourselves.
More substantively, it is not simply in our interest but in the best traditions of the humanities to pose the questions that underlie the calls for assessment. What constitutes a good liberal education, one that is emancipatory and transformative for students? What is the distinctive role of the humanities in that education? How do we know whether our educational practices embody these values? It is hard to find assessment tools that advance rich answers to these questions; all the more reason for skeptical humanists to enter the conversation.
Scobey goes on to answer his own questions, using Barack Obama's well-known and instructive educational trajectory as a test case. Worth a read. And so, by the way, are Dreiser and Jewett and Howells, if you have some time.
Trackback Pings:
TrackBack URL for this entry:
http://www.erinoconnor.org/movabletype/mt-tb.cgi/1612
Comments:
Apologies in advance for the length of this comment:
I'm reluctant to generalize about "teachers of the humanities" not liking assessment. In my own experience, I don't think many professors *like* assessment. The jackass physics prof who gives tests with an average score of 45 may seem "hardcore" but is really abusing assessment to motivate students through humiliation.
However, Scobey is completely correct that English teachers overuse the "higher skills" excuse to avoid testing. In my own (high school, 10th and 11th grade) classroom, I love quizzes and tests. If students spend a quarter reading *Les Miserables*, they should be able to recite the entire plot, they should be able to discuss the causes and effects, they should be able to name the major characters, their motivations, their conflicts, etc. My students perform extremely well on tests, so I give that to them to play to their advantages. This gives me room to have high expectations on their essays and to be a very strict grader of writing. I tell the students at the beginning of the year: grab the easy points, do well on tests and complete the homework on time, and that way a C on a paper won't kill a chance at an A for the quarter.
Another of my personal hobbyhorses is a campaign against the whole "multiple choice tests can only evaluate basic skills and knowledge" complaint. I design what I consider to be very thoughtful multiple choice tests. It's all about creating a sequence of questions that build on each other. Begin by asking the birth and death dates of an author. Ask the publication date of the novel. Ask what historical events were going on in that time. Ask what the dominant philosophies were. Ask students to pick out a theme statement about the novel or a conflict statement about a character that expresses that philosophy or connects to a larger historical conflict. Such sequences require higher level thinking, but they also show the teacher exactly where the student is having problems.
As far as making "the case for the humanities," I'll simply reprint what I wrote in a comment over on the Chronicle blog about this topic:
The world is all that is the case. The study of anything in the world (including imaginary things, which are as much a part of the world as anything else, as Salman Rushdie reminds us in Haroun and the Sea of Stories) should be a respectable thing to do, whether for a brief while (college) or for a living (teacher/professor).
It’s sad when humans have to justify wanting to know about their worlds. It’s sad when utility (which ultimately comes down to $$$) is the only measure of right action.
Last week, I was teaching Book 4 of The Odyssey to my high school sophomores. We read aloud together Menelaos’s speech to Telemakhos, as he explains that all the bling in his megaron means nothing compared to the cost he paid to get it: loss of brother, loss of friends, loss of essential aspects of his humanity.
My students visibly felt the truth, for Menelaos, of this speech. They aren’t going to change their lives just yet; and the point of teaching literature isn’t to preach to a bunch of 16 year olds. But for a moment, it was clear that a bunch of 16 year olds were in the shoes of an aging warrior filled with regret. There’s nothing useful about that, but if you don’t value such an experience, I cannot begin to convince you.
Another example:
Yesterday, I was teaching Book 8 of The Odyssey. Same sort of moment. Odysseus, anonymous, requests a song of the Trojan Horse from the minstrel Demodokus, who sings of the Achaian rape of Troy and of how Troy’s fall was more about the gods’ wrath than the Greeks’ skill or power. Odysseus weeps at the song, and Homer compares him to a woman weeping, a war-widow, about to be led into slavery.
The students protested: that simile is wrong. Odysseus is a victor; the woman is a victim. I asked them what Odysseus has lost up to this point: wife, son, father, mother, nineteen years, friends, treasure, name, reputation, self-possession. Again, they entered the life of a warrior reduced to a nobody (who used to be able to joke about being Nobody). They felt what it might be like at once to have had everything and lose everything (even as we all know he’s about to regain everything).
It’s not the usefulness of useless knowledge. It’s the value of experience. We should send our children to college so that they can have these experiences, so that they can have four years away from the stupid, brutal, and base pressures of the world to accumulate and reflect on new experiences. They won’t be made into better or happier people, but at least they might have an inkling of why they are neither happier nor better.
Luther:
Beautifully said. I am now fortified for the day ahead. Thank you.
I like your response, Luther, as far as it goes. But the controversy before us is not about simple grading. It's not about the assessment of individual students by the instructor. It's about program-wide assessment and institution-wide assessment. Things get a lot stickier on these larger levels. Consider, to take just one example, the question of just what skills and knowledge one is assessing for. That's far easier to answer on the individual level than the program or institution level.
I can easily articulate what I want my students in any given class to understand and be able to do, and design exams and writing assignments accordingly. I can read a student essay and tell a lot about the student's ability to put a verb to a noun; to understand various kinds of texts; to accurately summarize someone's argument, identify its underlying assumptions, and evaluate its strengths and weaknesses; to conduct library research; to integrate the results of that research into a coherent argument of one's own; to use literary-critical terms and concepts; to understand a text's relationships to its historical and cultural contexts, etc., etc.
But my colleagues might have decidedly different pedagogical goals for the exact same course. In order to create a program-wide assessment we have to meet and debate and compromise and finally approve a set of core goals narrow enough to satisfy the majority. (In the currently fashionable jargon imposed on us by our latest accreditors, they're not "goals" but "Student Learning Outcomes" or SLO's. Whatever.)
My department went through the laborious process of settling on goals and creating instruments some years ago. We (my college's faculty) set up a system whereby each program assesses itself, meets annually as a program to discuss the assessment results, and sends (every five years) a detailed report to the campuswide gen-ed committee for review. This five-year program review includes not just assessment results but also detailed data about program costs and efficiencies, changes in major enrollments, and so on--these reports are lousy with tables and graphs. The process culminates in a meeting where the department chair or program director has to defend the efficacy of the assessment as well as the program's responses to any revealed weaknesses.
Given the amount of work and expense put into it, I'd like to think the whole process works, but I really can't say. It's impossible to say because we keep getting forced by our various overlords to revise the process. Only a few years after we'd first set up the system we went through the whole process of meeting-discussing-compromising again, and changed both the assessment criteria and the instruments in order to satisfy our accreditors. (Actually, we bow and scrape before two accrediting agencies.) Then we changed things again in response to some conservative fad sweeping our state higher-ed board. With our new Democratic governor and legislative majority, perhaps we'll soon be changing things all over again. And perhaps someday after that we'll do it again because the feds want us to.
These revisions not only eat up time; they not only reduce faculty autonomy by shoehorning the considerable variety of legitimate goals of humanities instruction into a set of SLO's narrowed down solely to those that can satisfy a variety of off-campus constituencies; they not only breed faculty cynicism--worst of all, they make it very difficult to actually evaluate our program's effectiveness over time. The perpetual changes results in apples and oranges. Too many cooks and all that.
Anyway, I really just want to debunk the canard that we're opposed to assessment per se. At least some of us do value assessment, but want to keep it local. And we'd like the busybodies to leave us alone long enough that we could do it right.
And, FWIW, it astounds me that so many of these busybodies call themselves conservatives.
