Measuring value added--or not
RIchard Vedder's College Affordability blog has a number of sharp and informative posts on the Spellings Commission's final report on higher education--just keep scrolling to get his take on where the report undermined itself by pulling punches, and on where Vedder sees the whole thing going.
Note, also, his post on a recent study issued by the Intercollegiate Studies Institute. Entitled The Coming Crisis in Citizenship: Higher Education's Failure to Teach America's History and Institutions, the ISI report actually attempted to do something that the Spellings Commission has--quite controversially--recommended: It attempted to measure how much knowledge of American history and civics college students gain over the course of their college educations. The results are telling:
The report says:
1. America's college students are woefully inadequate in their knowledge of American history, the operations of our political institutions and foreign relations, and economics.
2. The amount of learning about these core subjects occurring during the college years on average is appallingly little, approaching zero.
3. The "value added" (increase in learning) about these topics is typically much greater in the less well known, less expensive private institutions than in the elite yuppie universities like Harvard, Yale and Princeton.
Many will discredit the report by proclaiming that the Intercollegiate Studies Institute is a conservative organization. True, but that is irrelevant in this case. ISI hired the well-regarded polling group at the University of Connecticut to do the testing. A non-partisan, fact-based test of 60 multiple choice questions were designed to test knowledge of key events, persons, and ideas that relate to our civic institutions and our heritage. Several of the questions were taken directly from the highly regarded National Assessment of Economic Progress exams administered to 17 year old Americans. More than 14,000 students were surveyed on 50 campuses, roughly one-half of them freshman and the other half seniors (about 140 of each on average on each campus).
At 16 of the 50 schools, the senior scores averaged lower than the freshman ones, implying that little or no learning of these key topics was occurring at these schools. Some of those schools: Yale, Duke, Cornell, the University of Virginia, and the U. of California at Berkeley. The five schools showing the largest gains in knowledge were Rhodes College, Colorado State University, Calvin College, Grove City College, and the University of Colorado at Boulder --hardly viewed at top schools by the more popular magazine rankings.
The Spellings Commission and the Department of Education have been crying for measures of student performance, of the value added by attending colleges. ISI has actually done something to measure value added. Their findings are scary, exciting, and particularly humiliating for expensive elite private schools. Seniors at little known Grove City College did as good or better than students at more prestigious Williams College or Washington and Lee. George Mason students outperformed those at Michigan or Berkeley. This is the type of information parents and students need in evaluating schools, and this kind of test result is precisely why many schools, including the private elite ones, will likely fight this sort of testing tooth and nail.
Needed: Someone to give ISI (or CCAP) $1 or $2 million to replicate and expand this study -- perhaps to new subjects, to more universities, to bigger student samples. Let the light shine in.
Let it shine, indeed. Then maybe we can talk more honestly about why the academic establishment is so hostile to testing--even no-stakes testing that does not affect students' progress toward degree but that does allow schools' overall effectiveness to be measured. As Mark Bauerlein put it at Phi Beta Cons, the ISI study
provides precisely the kind of data that is essential in understanding the performance of higher education--the intellectual progress of students from first-year to graduation. Unfortunately, it is just the kind of assessment that universities will resist. Humanities folks don't like the idea of selecting certain knowledges as central.
Administrators will say that schools are too diverse in their missions to warrant comparisons. Top-tier schools don't want any new measurements that might compromise their standing. And lesser schools worry about findings that might reflect poorly on their benefits, angering legislators and parents who may be footing the bill.
But what could be more fair than to ask for some information on the acquisition of learning?
For a different sort of meditation on standardized testing for college students, see Tim Burke's post on the issue, as well as his commenters' responses.
September 13, 2006
Offensive math prof wrongfully suspended
Defenders of the academic status quo frequently dismiss FIRE as a right-wing organization that is not as concerned with individual rights on campus as it is with promoting conservative causes. Those folks are dead wrong--and they should take note of FIRE's most recent case:
SEATTLE, September 13, 2006--Bellevue Community College (BCC) near Seattle has decided to suspend a professor for composing a math exam question that involved a person named "Condoleezza" dropping a watermelon from the top of a building--a question that originally featured the comedian "Gallagher." After months of public outcry, BCC informed Professor Peter Ratener that he would be suspended for a week without pay for his "offensive" question. Ratener then contacted the Foundation for Individual Rights in Education (FIRE) for help.
"Given the reaction of the community and the college, one might think that Ratener was guilty of committing a serious crime, rather than writing an accidentally offensive math problem," stated FIRE President Greg Lukianoff. "Everyone involved has acknowledged that Ratener intended no offense, and Ratener even apologized for the question, so what exactly is BCC trying to prove by suspending him? This punishment is not only unfair and a violation of the First Amendment, but also totally unnecessary."
In 2004, Ratener composed a question for a math exam that read in part, "Condoleezza holds a watermelon just over the edge of the roof of the 300-foot Federal Building, and tosses it up with a velocity of 20 feet per second." Ratener states that he had planned to use the comedian "Gallagher" in the question. But realizing that many current students would be unfamiliar with the comedian, who was well-known for smashing watermelons on stage, he later substituted the more recognizable name "Condoleezza." The exam with this question was administered to students in 2004 and elicited no complaints.
In March 2006, another professor distributed the exam featuring Ratener's question to his class as a practice exam. This time, one student approached the math department chair to express that she was offended by the question. Within days, the math question had turned into a controversy and appeared on Seattle's local KOMO-4 news, in The Seattle Times, and was soon picked up by media outlets from across the country and activists from across the political spectrum. The Urban League of Metropolitan Seattle, which is chaired by BCC Trustee Paul Chiles, even issued a press release branding Ratener's exam question "another example of hate and bigotry" and calling for BCC to take action.
On April 15, BCC's Board of Trustees held a special meeting to discuss how to react to Ratener's question. The Board issued a public statement, available on the BCC website for months, which read, "We ... are deeply offended by the conduct of a math instructor ... and strongly condemn this offensive behavior," calling it "a gross violation of BCC's mission and core value of respect for diversity." The Board then asked BCC President Jean Floten to take appropriate disciplinary action and "examine the curriculum and practices of all its departments."
On April 19, Ratener himself issued a public apology, admitting that he had made a mistake but stating that the invocation of a negative racial stereotype was completely unintentional. Even BCC's Executive Dean for Instructional Services Ron Leatherbarrow characterized the question's offensiveness as "unintentional" in a letter to Ratener in early May. Leatherbarrow nonetheless stated that the question "interfered with the educational process" for the student who complained, "and, possibly, for others as well." Concluding that Ratener had not met BCC's standards "regarding choosing appropriate test materials and treating students with respect," Leatherbarrow suspended Ratener for one week without pay. Ratener has filed a formal grievance through his union and currently awaits arbitration on this matter.
FIRE wrote to BCC's president and Board of Trustees on August 28 to protest the fact that the college "has raced to vilify Ratener and punish him for what he admits was a mistake," instead of giving Ratener--a 26-year veteran of BCC with a spotless record--the benefit of the doubt. On September 8, Washington State Assistant Attorney General Alan Smith responded to FIRE's letter but refused to comment on the situation before the arbitration that is scheduled for January.
FIRE's Lukianoff concluded, "Liberty on campus cannot and will not long survive if professors can be disciplined merely because some interpret their speech as offensive. While BCC and others are free to criticize Ratener for his choice of words, the answer to speech one dislikes is more speech, not official punishment."
I have a bit of trouble swallowing the premise that the professor really had no clue that the casual alignment of blacks and watermelons is a common racial stereotype in this country. You'd have to be exceptionally tone-deaf to miss the base caricature of Condoleezza Rice that emerges from the math question. But FIRE is right. Suspending the professor without pay is not the proper course of action here.
UPDATE 9/14: More at InsideHigherEd.com.
September 12, 2006
The growing gender gap in higher education has attracted the attention of Penn's student paper, The Daily Pennsylvanian, with disappointing results. While noting that 58% of undergraduates nationwide are now women, and while observing that Penn's undergraduate population mirrors that trend with 55% of arts and sciences undergrads being women and 95% of nursing undergrads being women, the article misses the point by a mile. There is no reflection whatsoever in the article about where the men who might be going to college are going instead, no concern about what this might mean for young men's future professional and economic prospects. Instead, the article thoughtlessly promotes the outlandish proposition that women's educational displacement of men should not be read as a sign that women are not still terribly oppressed by the patriarchy:
At 58 percent nationally, women now make up a majority of college students. Only a generation before, they lagged significantly behind men.
As of 2003, 30 percent of women in the 25- to 29- year old age group had college degrees. Only 25 percent of men in the same group could say the same.
At Penn, 55 percent of the College of Arts and Sciences' Class of 2010 is female, as are 95 percent of School of Nursing freshmen.
Even for fields where women are in the minority, many are working to ensure they're noticed when they graduate.
Last year, 55 percent of the University's Career Services' counseling appointments were taken by women, according to Career Services director Patricia Rose.
"Women are more likely to come in to seek advice and have their resumes checked," Rose said.
But when it comes to deciphering what these statistics mean, many say that even having a slight majority of women on campus won't mean these women are landing top-notch jobs.
"As long as you have a culture wherein women are predominantly seen as responsible for childbearing and homemaking, you aren't going to achieve workplace equity," said Donna Phillips of the American Council on Education, an advocacy group.
Phillips added that "just because there are more women getting degrees doesn't necessarily mean that there are fewer men" doing the same thing.
My concern here is less with the errors of an inexperienced student journalist than with the institutional and intellectual climate that does nothing to mitigate them. While the journalist is missing the point--or at least missing a point--Donna Phillips should know better, and should be taking a more rounded, nonpartisan view of demographic patterns in higher education. She should be as concerned about men in higher education as she is with women--but she just isn't. If she were, her conclusions about what the issues are and what these numbers mean would most likely be substantially different. Meanwhile, the student reporter simply imbibes the "reasoning" of the academic experts--without questioning whether that reasoning is actually reasonable.
September 7, 2006
Fun with logical fallacies
This short post from Timothy Burke contains multiple logical fallacies--and does more to damage his own credibility as a temperate commentator on academic politics than anything else.
Readers are invited to parse his post, and to entertain the broader question of how to discuss academic culture in a way that does not simply further polarize an already wildly polarized debate.
UPDATE 9/12: SCSU Scholars weighs in.
September 1, 2006
On grade inflation
As part of University of Colorado president Hank Brown's decision to tackle the tough issue of grade inflation, CU regent Tom Lucero is inviting members of the public to contribute their thoughts on the subject:
Even cum laude graduates sometimes lack the skills needed to succeed in today's workplace. This can prove to be an expensive and frustrating problem for new employers who must allocate the time and resources to adequately train new-hires.
I would like to invite you to participate in a discussion about grade inflation and its impact on the quality of our college graduates.
--What influence does grade inflation have on individuals, society and the economy?
--What are your experiences with the caliber of work from recent college graduates?
--What measures can be taken to better prepare students for life in the real world?
We are beginning a debate at the University of Colorado about the important issue of grade inflation. Please send your comments and thoughts to email@example.com.
Additionally, there will likely be opportunities to make a formal, two minute presentation to the CU Board of Regents at our meeting on October 4, 2006. In many cases, a college degree is merely a filtering mechanism for future employers, proving that graduates had the ability to finish something that they started. Every degree should be worth more. Students should graduate with certain fundamental skills, including the ability to write clearly and think critically. All graduates should be able to immediately add value to the workplace and society.
This is an important--and potentially transformative--moment, not only for the University of Colorado, but also for higher education's treatment of grade inflation (which has largely amounted to turning a blind eye) and for higher education's willingness to engage the public in discussions about governance. Do take a moment to frame your thoughts for the Colorado regents--and please do post them here as well.
The American Council of Trustees and Alumni took up grade inflation in its 2003 report, Degraded Currency: The Problem of Grade Inflation. It's a good starting point for anyone interested in thinking about the issue.