Surely the authenticity of the test is a "real" issue, and I'm not sure that it is settled.

I seem to remember a whole series of posts, not so long ago, on Climategate. It is thus ironic, to say the least, to encounter the implicit suggestion (explicit in some of the responses to the previous post) that questions of fact are not that important: real or not, the evidence "might as well" be true.

Peter, I never said it "might as well be true." Don't put words in my mouth, please. It matters very much whether the test was real, as my efforts to get it authenticated should show you. If you still question the authenticity of the exam, what would satisfy you? What is your standard of evidence? And while we are at it, how do I know you are Peter Shoemaker? How do you know I am Erin O'Connor?

If I--or Minding the Campus--posted a copy of the exam, would you accept that? Or would you say, "You faked it!" If the anonymous professor came forward--still anonymously--and vouched for finding the exam, would you believe that? Or would you say, "That's a fake too!" When the New York Times, or the AP, or any number of other news sources uses an anonymous source, do you reject all that they say instantly? If not, what do you do to ensure that they have fact-checked to your satisfaction?

Long ago, C S Lewis critiqued a silly book for high school students (which dealt with the whole realm of beauty/aesthetics by simplistic debunking.) In wondering why someone would write such a book, Lewis observed that "literary criticism is difficult, and what they actually do is very much easier."

I suspect the same factor is important in creating phenomena such as this test and the "teaching" behind it: Actually teaching history might be difficult, and it's much easier to simply mouth current political catch-phrases.

I also suspect that the vast expansion of academia in recent decades has swept in a lot of people who really don't have an academic vocation, but pursued academic careers because they couldn't think of anything else they wanted to do. (This also drives many people who become lawyers.) I personally don't think a person who really loved the study & teaching of history would write a test such as the one linked, whatever his political opinions might be.

Erin,

If you read my post carefully, you will note that I did not put words in your mouth; rather I pointed to I still think is an *implicit* inconsistency on your part in your treatment of climate science, on the one hand, and "convenient" anecdotal evidence such as this, on the other.

As for the question of fact, the document probably *is* authentic, but all the evidence that we really have for its authenticity is one anonymous source and two named sources who are dependent upon the anonymous source to vouch for the provenance of the document that they have seen. (It's not like the quiz has hologram on it that proves that it's legit.) And yes, I know that journalists use anonymous sources all of the time, but journalistic "best practices" dictate *two* independent sources (or at least they used to). And there's a difference, frankly, between journalists and editors, on the one hand, and columnists and commentators, on the other.

So by all means, let's have a conversation about accountability in higher education. But let's not pretend that this is some smoking gun, either.

Hi David -- Interesting thoughts. I would love to know more about the person who administered this exam, as I think it pertains to how we read it. My guess would be that Soc 101 is a large service class -- and that there are decent odds the professor is either utterly disengaged (perhaps delivering canned lectures but using grad students to do the grading and run any discussion sections) or quite young and inexperienced (there is a "cookbook" feel to the questions as well as the answers, which speaks poorly not only for the exam but for the quality of instruction and suggests a lack of discussion and debate). I also suspect that a culprit here might be the use of an egregiously bad textbook--but not being up on sociology textbooks, I'm not in a position to pursue that line of thought.

Peter -- Does this mean that you now agree that there are some very troubling issues with how climate science has been generated, documented, reported, and used to underwrite massive, expensive policy efforts?

Also, I'd be very curious to know your thoughts on Jonathan Gruber. What's your sense of his responsibility--not only to the public, but to his profession and his institution?

Oh...I should have read more carefully...I thought this was a *history* exam. It certainly doesn't seem to have any more to do with the serious study of sociology than it does with the serious study of history.

A friend who is a sociology professor tells me that many new graduate students in that field are astonished how much work is involved (at least the way he teaches it), because they thought all they would have to do is sit around & spout political platitudes like they apparently did as undergraduates.

Isn't there another issue, in addition to whether the document is authentic? Assuming it is authentic, isn't it the case that refusing to name the institution, class and professor casts an ugly cloud of bias and incompetence over all Sociology departments in all public universities in the East?
As to authenticity, speaking only for myself, I would never be completely convinced that a document is authentic without names, dates and places. I believe the Soc. exam in question is authentic because the people vouching for it have credibility. However I would not let the document be entered as evidence if I was a judge of a debate, or a trial.
When the NY Times, AP, etc. uses anonymous sources, I don't automatically reject their articles, but I do take what the anonymous sources say with a grain of salt. If you don't know the source, you have no way to judge the source's credibility, nor any way to figure out what his or her agenda might be.

Hi David, I think there is much in what you say. I really do wish we could at least have the name of the institution. My hunch is that the reason we don't is that if we had that, we could reconstruct who gave the test -- and could also use the course register to reconstruct who found it. And then the whistleblower comes in for no end of professional grief.

So let's be qualified in our assessment. One exam can't be read as a synecdoche for the whole discipline. Still, it's evidence of some quite poor teaching. And not having the names can be useful in this regard. I would hope that professors and administrators who see the exam stop for a moment to ask, "Is this the sort of thing that is going on at my school? If so, how would I even know?" There is a perennial problem on campus with ensuring that teachers (professors, adjuncts, grad students) are doing their jobs responsibly. There is an awful lot of don't ask, don't tell; an awful lot of I won't scrutinize you and in exchange you don't scrutinize me; and an awful lot of justifying such no-nothingism with the academic freedom mantra. Faculties need to do better.

There is a perennial problem on campus with ensuring that teachers (professors, adjuncts, grad students) are doing their jobs responsibly. There is an awful lot of don't ask, don't tell; an awful lot of I won't scrutinize you and in exchange you don't scrutinize me;

Alan Wolfe offered just this assessment in his book Marginalized in the Middle.

I seem to recall about a decade ago a piece of survey research which asked professors in various disciplines to select a term which best described their social and political perspective. Soi-disant 'radicals' outnumbered soi-disant 'conservatives' by a margin of thirty to one. Sociology can be a rigorous and engaging discipline, so the situation described is most distressing. It is hard to figure a way to correct the problem sytematically, other than for administrators to cut the number of hires and to shut down the most egregious graduate and undergraduate programs.

Erin,

When did I ever suggest that there weren't reasons to be skeptical about climate science? Quite to the contrary, I wrote, that the Climategate episode "*should* make people more skeptical about anthropocentric [sic] global warming. But I think there's a difference between healthy skepticism and dogmatism masquerading as skepticism." And I'm still happy to stand by that statement. As for Jonathan Gruber, I don't know that much about the details of the case, but it certainly seems that his relationship with the administration should have been disclosed to the public, the NY Times, etc. Does that mean that we can throw out any and all economic analysis in support of the administration's health care plan? That's another story.

By the way, I entirely agree with the last three sentences in your most recent post in this thread. It's an issue that transcends politics, so perhaps we can have a discussion about what to do about it.

A correction to the above: the 30:1 ratio was among sociology professors, not professors-in-general.

It's an issue that transcends politics, so perhaps we can have a discussion about what to do about it.

Your problem is the insularity of the faculty. Two observations, one from Barry Shain and one from Mike Adams: 1. The professoriate is made up of those were not as youths exceptionally affluent or socially adept; and 2. that the properties of academic employment attract certain character and personality types. The type has a signature way of viewing their social world and the place they occupy within it, and faculty governance and autonomy renders each cohort of faculty more resemblant to the type.

Bushwacking one's way out of the woods is going to require ending the insularity of the faculty: replacing faculty governance with governance by trustees, replacing tenure with renewable contracts, and implementing hiring policies which enhance the quantum of migration into and out of academic employment.

Art Deco -- I tend to agree with you on the macro-level, but also am trying to think through pragmatics that offer the possibility of faculties taking on the responsibility of repairing their own broken peer review processes. I don't feel hopeful that they will, and I think they are likely to continue to flush their professional wellbeing down the john out of stubbornness and lack of perspective--but I still think the ideal of a self-regulating faculty is one I want to try to maintain and argue for. If we could have that, I think it would be better for both research and for teaching--but I acknowledge that it's a pipedream much the way socialism is (looks so good in theory, cannot ever actually work in practice).

Looking at things this way, I find myself wondering about mechanisms of accountability and grievance within institutions. For example, did the professor who found the problem test have any avenue for bringing it to the attention of administrators in-house? I rather doubt s/he did. But let's say there was such a mechanism. What would it look like? Would the adjunct status of the professor effectively prevent him/her from being able to use it?

Turning the lens another way -- I wonder what procedures this institution has, at the department and college level (if any) for collecting, publishing, and reviewing syllabi and for spot-checking things like course materials and exams. What would the sociology department chair say about this exam? What would other soc professors say about it? What would colleagues in neighboring fields, or admins within the college say about it? Would they defend it--or would they agree that it's inexcusable?

We can't know the answers, of course. But we can at least begin to think about how "academic freedom" has become an awfully strong cover for lack of transparency, discussion, imagination, and accountability of the kinds I sketch above.

Peter -- Thanks for your thoughts. Quick final question on ClimateGate: Do you think Penn State is conducting its investigation appropriately (no external evaluators, etc.)? And given the enormous amount of money and public policy hinging on Ye Olde Hockey Stick, do you think Penn State should make the results of its investigation public?

On Gruber: In principle I agree with you. But it's also worth noting that the more people press on this one, the more it's looking like Gruber cooked his facts to serve the White House's ends. Check out http://news.firedoglake.com/2010/01/16/gruber-and-selective-disclosure/

Well, from the point of view of someone not American, it doesn't seem so *horribly* biased -- it's about colonialism and its results, which much of the world deals with daily.

I give ID questions on 1492, the Black Legend, the Middle Passage, the mines of Potosi, Anaconda Copper, etc. in Latin American history courses all the time, and the simple _facts_ of this history do not make the U.S., or U.S. based industry, or the ruling elites (usually white) of the modern world look very good; it doesn't make patriarchy look too great, either.

But the quiz is not very well constructed at all, and the answers aren't complete.

Professor Zero wrote:

I give ID questions on 1492, the Black Legend, the Middle Passage, the mines of Potosi, Anaconda Copper, etc. in Latin American history courses all the time, and the simple _facts_ of this history do not make the U.S., or U.S. based industry, or the ruling elites (usually white) of the modern world look very good; it doesn't make patriarchy look too great, either.

Well, you've conflated the US with the "modern world."

While I can't speak for the "modern world" — which would include the likes of such persons not of color like Ahmadinejad, Putin, Chavez, Assad, the Castro bros, and Kim Jong-il — I believe that the US suffers by comparison only to the perfect of a utopian world, not when compared to the realities of, well, the real world.

If you think I'm wrong, then you're welcome to condemn the US's relief efforts in Haiti, and I'd welcome you to advocate that the US withdraw completely and immediately, and let the UN fund and administer the relief efforts there without the help of the US.

You might well remember who did what in Indonesia, too. Oh. And then, there's the Balkins, as well. You know — the place where Muslims were being slaughtered and only intervention by the US brought it to an end.

Forgive me, but to drag out past US imperfections without dragging out the past and present imperfections of other nation states seems . . . unfair.

it's about colonialism and its results, which much of the world deals with daily.

No, it is about the elites of various and sundry third world countries refusing to assume responsibility for their policy failures.

Erin,

I will try to answer your questions. Regarding Mann, presumably Penn State has procedures for this kind of thing and should follow them. I don't know whether that includes external evaluators. Given what is at stake in terms of public policy, obviously *someone* should conduct a external review.

As for Gruber, the link that you provide doesn't exactly suggest that he "cooked" his facts, but rather that he overreached in some of his conclusions, which does suggest that he was eager to get results that would support a specific policy recommendation. I'm not sure that makes him that different from most economists, but it is one more reason why he should have disclosed his conflict of interest, and one more reason to be skeptical of the economic "facts" that are tossed around by all sides in policy debates..

Peter Shoemaker wrote:

Given what is at stake in terms of public policy, obviously *someone* should conduct a external review.

Agreed. But I'd go further: I'd say that the external review should be made public, for just the reason you cite: the huge worldwide impact Mann's work has had on framing public policy (this is taxpayer money, after all).

About Gruber: I see him having made 2 or 3 dubious decisions, the total impact of which is far greater than the sum of the parts.

I think "overreached" is a most charitable way to frame part of what he did, but for the sake of discussion, I'll accept that characterization.

But here as with the Mann investigation, the enormous potential impact of his "study" on public policy cannot be ignored: When you're dealing with matters of such import (1/6 of the economy, after all), you and your data must be squeaky-clean.

And if you're going to err, you don't err on the side of "overreach" when such "overreach" serves the interests of those who surreptitiously funneled nearly $400k into one of your accounts.

By not disclosing his relationship with the administration early on — say, when he released his "study" — he committed a second faux pas. And he committed a third transgression by keeping mum every time the administration or its supporters represented his work as if it were an independent, objective study.

Now, perhaps Gruber meant no harm, and every little indiscretion was simply an honest oversight.

But . . . if he were deliberately interested in removing his scientific hat and don a political one as unobtrusively as possible for the specific purpose of representing a biased ("overreached") study as an honest, sober, independent and objective one, what, pray, would he have done differently?

And what does it say about the administration that it didn't disclose its relationship to Gruber, and presented his "study" as if it were an entirely independent piece of work?

Just my way of looking at things. Feel free to disagree.