Saturday, September 13, 2008

Eeny, Meeny, Miny, Moe*: Or, How to Evaluate the Candidate Pool

One of the things everyone is talking about in the presidential race is the capacity for good decision-making, who has it, and what relationship that bears to previous experience making hard decisions. OK, so you are not running for national office, nor are you a pit bull with lipstick (or was that a hockey Mom who is a pig? I can never remember.) But you are on a search committee. And you have never been on one before. And there is a large drawer of files to evaluate. You have decisions to make. So today's topic is:

How do you evaluate a candidate pool and decide which 10-12 people you want to invite to a preliminary interview for a tenure-track job?

There are a number of criteria that will be in play, depending on what kind of slot your optimum candidate is expected to fill and how broadly you advertised in terms of field. But one of the things I think is important is to have some sense of what you are looking for, prior to reading the files; a sense of who the ideal candidate is that is well-defined, but also flexible. Your approach to reading files needs to simultaneously respect the ad that was approved by your colleagues and the administration and be responsive to the individual strengths and weaknesses of the actual candidate pool you have before you.

Thus, your evaluation of a file starts with the ad criteria, but then extends to the role that this candidate will fill in the department more generally. The basic questions that need to be asked of each candidate are:

1) How well does this candidate fit the job for which we advertised, in terms of rank, field and any stated subfields?

One of the reasons I advocate a carefully written ad that addresses department needs (and dreams) is that defining, or redefining, the job in relation to the candidate pool that you have can sometimes be necessary and desirable, but it can also create a lot of chaos on the committee, and later in department discussions. A committee member is most likely to be seen by others as "pushing an agenda" when s/he tries to persuade colleagues that a candidate who is inappropriate for the job actually fits the ad. Be honest: how well is each candidate responding to the ad? Does s/he actually do research in the field the department needs, or is that field really secondary to another focus? And, while all of us are likely to shift focus somewhat in our scholarly careers, does that candidate seem to want to sustain research in the field and -- this is the sign of someone to take more seriously, in my view -- seem to have ideas that are pushing the field in a new direction?

2) Does the candidate offer something special, in addition to the field in play, that would be an advantage to your department?

This is a tricky one. In the latter days of the Dark Ages, when I was interviewing for jobs in political history, at some point I would usually be asked: "And could you teach the women's history survey?" This question came up because I was a woman, of course, since I had not yet done any research or published in the field. Because historians of women were (except for Tom Dublin) women, it stood to reason that the opposite was true as well, so there were any number of departments who saw me as a "plus one." In other words, they could hire a woman to teach political history, and never have to worry about hiring a historian of women. Ta-da! I always said yes, of course (who wouldn't? I needed to eat too), and one day, someone actually gave me a job where that was part of what I was hired to do because my assertion that I could was never undermined by their curiosity about what kind of training might be necessary to teach the history of women.

This is a cautionary tale that actually has a happy ending. I learned to teach the course, and soon began to do research in the fields of gender and sexuality within political history. Better yet, that job led to another job, which led to a career in academic blogging. But be careful: people are not necessarily interested in studying or teaching about people "like them" as a favor to you. Unless she offers to, the Chicana woman who works on French colonies in North America probably doesn't have an ethnic history survey in her back pocket that she would love to develop for you so you can get the dean off your back, and she will resent being asked about it at in the preliminary interview.

However, when you see something on a vita that suggests the candidate brings something else to the table, that is evidence (as opposed to veiled bigotry), and the only way to pursue it is to interview the person. But the ancillary field should be raised as a question at the interview stage, not a new requirement for the job, and in a way that doesn't divert from the issue at hand: how well might the candidate address the needs your advertisement expressed?

To summarize: noticing that a candidate really does offer something special could be a reason to bounce hir to the next stage if only to find out whether the additional expertise is for real, and whether the person actually is a "plus one." If you are a small department, you might not be able to afford to devote a whole position to Italian history, but a historian of the French Renaissance whose research touches on Italy, and who has taught a section of an Italian history survey, is worth taking a longer look at.

3) What has the candidate published, and where? If the candidate is still a graduate student, how close is the dissertation to being done?

I think if one article on the curriculum vita has been seen through to publication, or has been accepted for publication, this should be counted as a real plus for a candidate. It shows maturity, and it shows that the candidate has been willing to accept the scrutiny of anonymous peer review and learn from it. Because publishing is long, hard, and often tedious work, I like to know that a job candidate has mastered the rudimentary discipline of submitting, revising, meeting deadlines and actually finishing something, an act that will have to be performed repeatedly if tenure is to be achieved. The only exception to this rule, in my book, is that there are some programs that are trying to move graduate students through more rapidly than the conventional seven years, and in that case, it may be that the bulk of the labor has gone into completing the dissertation, and the lack of a published article means nothing.

Encyclopedia articles, by the way, are nice, but don't count. They are a great thing to do for the community, but they aren't the same achievement as the sustained re-writing and re-thinking that go into a refereed article. Sorry. Book reviews: same.

As to the dissertation, if the job letter says that there are fewer than three chapters completed by mid-fall, I have concerns about whether the dissertation will be done by the time the candidate hits the ground in September, concerns that need to be addressed by the referees. The dissertation director, in particular, will be very specific about the timing of a defense if the defense is on the horizon. What is worse than that is mentors pushing students out on the market and rushing them through a quick and dirty defense on a first draft of a dissertation. This can mean that the process of turning the dissertation into a book can be a nightmare, both because the candidate has never successfully "finished" the dissertation (and thus has no clear sense of what it means to complete a major project) and because s/he may have a manuscript that s/he is embarrassed about and won't show any of hir new colleagues to get help for the next stage.

4) Does the candidate have teaching experience?

This is something the committee needs to discuss in advance. Do you need someone who can hit the ground running, in a field where there is enormous demand? Is it a new field where classes will likely be smaller in the candidate's first few years, and s/he has time to develop? Is teaching the most important thing at your institution, and candidates without experience cannot, in all fairness, be considered? (Hint: if this is so, I hope you put it in the ad.) Teaching a few classes doesn't mean the individual is an experienced teacher; not having taught at all doesn't mean the person won't be a quick study and do well in the classroom. But hiring very inexperienced teachers means that there should be institutional support for helping that person get through the first year or so until s/he develops confidence. It also means that members of the department need to be prepared to do observations early on, go over student evaluations, and look at syllabi prior to the beginning of classes to make sure that the novice teacher is on target. There is nothing sadder than a young teacher floundering in the classroom, working desperately to succeed, and not knowing how; unless it is a teacher who has floundered for three years and no one knows until reappointment. Nobody's teaching evaluations should be allowed to play God in a person's career.

Hence, teaching experience is important to consider in relation to the institutional context, not necessarily as a relative factor in the candidate pool. Teaching experience, in my view, might bounce someone onto a semi-finalist list, but lack of it should not exclude someone who seems to be an innovative scholar from your interview list because such a person can be taught to teach. Great teachers are not born: they are made, and helping a terrific young scholar become the teacher s/he can be is an interesting way to spend part of the following year.

5) Has the candidate been active in institutional work and in professional organizations?

This may seem obvious, but you are hiring a scholar, a teacher and a colleague. People who have been involved with their graduate student organizations, TA unions, campus organizing and whatnot are people who "get it" that a university or college is a living thing that needs care and feeding. The lack of such experience should not be a negative, since sometimes a graduate school is not congenial, or a person's community activities may be based in a neighborhood, church or other sphere. But institutional work is learned, and being responsible to others is the mark of a scholar who will keep up hir end of the bargain when it comes to departmental work.

As for professional organizations, the only advantage this gives a candidate from my perspective is that again, it communicates maturity. This person is starting to network, make connections in hir field, and is making the transition from graduate student to colleague already.

6) Does this candidate add diversity to our department?

There are federal laws when it comes to diversity hiring. As important, your institution has regulations. Be familiar with them, with the categories of human being who need to be identified, and with practices appropriate to the full consideration of such candidates. If you are a really "with it" search committee, you will have met with your university's affirmative action officer to discuss what kind of diversity needs to be encouraged. And there may be forms of diversity that do not count under federal law that you may value all the same: an international scholar, a scholar who speaks an extraordinary number of languages, a military veteran, or a GLBT candidate. In my view, it is wise to discuss this in advance, and keep discussing diversity in your hire as long as it is relevant.

In conclusion: I have a couple pieces of advice to keep the committee functioning as it should. First, the committee needs to meet at least once prior to reviewing the files to talk about evaluating the pool, not just so that everyone is on the same page more or less, but so that when it recommends the final three to four candidates to the department it can say why. Second, keep good notes. I create a xeroxed sheet that I fill out as I am reading, so that I have the same information for each candidate. Otherwise, how can you compare them? But also, as the committee discusses the candidates, that discussion should be pinned to actual evidence of who the candidate is and what s/he has accomplished, evidence that can be easily referenced in the file.

Your optimum state is consensus, and try not to quarrel, particularly over the petty things that carry over from other moments in departmental and university life. Interview an extra candidate if you must to prevent antagonism or division among committee members. I actually think one viable strategy is to give every committee member their top pick as a semi finalist, and then fill out the rest of the list during the ensuing discussion. As a gesture, it shows respect for the expertise every member of the committee brings to this enterprise. As a practical matter, it also means no member of the committee goes away feeling so frustrated or disappointed that s/he starts to lose commitment to the process.

And finally, no candidate should be perceived as fitting into -- or frustrating -- the ambitions of a particular faction in the department. Not only is this unfair to the candidate pool more generally, but the man who looks like a conservative on paper may be exactly the colleague that you and your feminist friends would enjoy sparring and collaborating with. Furthermore, if you are doing that, you are not seeing who they are: you are judging them in relation to who they might be to you. And the search isn't about you. It's about the department, and the scholar, deep in the pile, whose career is about to take a happy turn for the better.

*The genealogy of this title which dates it to, among other things, an early Celtic sheep counting rhyme, can be found here.


davidjhemmer said...

Conventional seven years? Holy cow, that is ridiculous!

Sisyphus said...

That Hockney painting is sadly lacking in beautiful naked men, for shame!

And you might look for TA union experience, but I would advise all applicants to leave it off. I have through personal experience learned not to let any of that type of stuff get on my cv --- and applicants will never know if the prof who told me he'd make sure I never worked again is reading their applications. This is a plus that will just have to come out after hiring.

Anonymous said...

one question about pubs, TR: how would you evaluate book chapters? These are often not deemed "peer-reviewed" in that they don't undergo double-blind review. But in my limited experience, book chapters are subject to more reviewers' eyes than articles and are often done so via single-blind review, in that the reviewer might know who the author is but the author might not know the reviewer. And many grad students/early asst. profs get pubs in edited books. Thoughts??

Tenured Radical said...

I like book chapters just as much as articles.


Bardiac said...

I think what you say about treating other members of the committee with respect and working for consensus is right on the mark, and very important. I really learned a lot from a couple of my colleagues on my first search committee because they really made their respect apparent. It taught me a new level of working with people, and made me hope I can be like them when I'm looking forward to retirement.

Anonymous said...

earlier in this series, you said you would be writing on 'how to evaluate a candidate pool that contains scholars with different levels of experience' and I would be interested to hear more about your approach to that. For any beginning T-T position, some applicants are finishing the diss this year, and some are 5+ years out of the PhD, having been post-doc, VAP or adjuncting since then. How do you compare vitas in these situations?

Jarrod Hayes said...

Wonderful post as usual on the subject, and very helpful for me as an aspiring T-T assistant professor. One question though...typically, how important are diversity issues? I take it from your post and the diversity questionnaires I've filled out that diversity does not generally mean the intellectual a white male, I'm interested in how much effect this has. (Disclaimer: I'm generally for hiring/admission systems that address structural inequities, so I'm not gunning for proof that the white man is being discriminated against)

Profane said...

Thanks for the wonderful post! As the youngest member of my department by over a decade, this could end up consuming a great deal of my life. . .

davidjhemmer said...

In my short career I have already been in one situation where the department selected a candidate (a man not a white man) and the office of affirmative action (which was essentially a one-man fiefdom) ordered us to make an offer to a woman who was third on the list. As it turns out she does not appear headed for tenure.

Tenured Radical said...

Dear Jarrod,

The intelligent department selects someone who is tenurable. Affirmative action does not mean (in academia -- it's different in the civil service) bouncing people ahead on the list because of race or gender. It means going the extra yard to find out if they are viable candidates, and why.

And the wise administration does not force a candidate on a department that does not respect that person. It's not good for anyone.

In my field affirmative action seems to play virtually no role. At latest count, fewer than 10% of Ph.D.'s were people of color, so that tells you something right there. Over half the Ph.D's in history go to women; over half the jobs go to men. You can go to the AHA website and look up the Lunbeck report for a fuller report.


Jarrod Hayes said...

Thanks to both right-wing prof and TR!

davidjhemmer said...

As I described, sometimes affirmative action in academic means exactly bouncing people up on the list. In our case we had several hundred applicants and the person was not bounced very far, one could argue the difference between the two candidates was epsilon where epsilon is very small. Nevertheless, and perhaps because of this, the department was indeed ordered to move this candidate up the list.

Another personal anecdote. I applied for a job at a small department(-12 people) that had THREE openings in one year. This department already had a couple people in my general area. When I interviewed I was told one offer had already been made, the other two would wait until everyone interviewed. To make a long story short here is what happened;

1. The dept had no woman, they were under orders to hire one if at all possible. Thus once a woman interviewed and was acceptable, she was made an offer, before the interviews were complete.

2. The woman they offered the job to was in my area. There was no way they would make two hires in this area. Thus, I interviewed for a job which had already been offered to someone else!

3. A couple weeks later she turned them down and they made me an offer, which I ended up refusing due to $$.

Needless to say this left a bad taste in my mouth. The inside information I seem to have was passed along to me by one of the faculty members who also had a bad taste in his mouth!

Anonymous said...

I was such a candidate that added "diversity" to the department. Once I started my job, it was very obvious that my purpose was to make the department feel good about itself for having hired a woman of color. My job was to be seen and not heard, and the department would have felt comfortable with me only if I didn't challenge prevailing modes of thinking. Needless to say, I left that job in a hurry, and I'm now in a much more hospitable place.

For those committees out there who are looking to hire for reasons of diversity, you need to ask yourself hard questions not only about strategies for recruiting this person but also for retaining this person once s/he is hired. You need to ask yourself if you are prepared to have someone like this in your department. If you expect this person to be "yes ma'aming" and "yes sirring" all the time, suffice it to say that you will be searching again for their replacement before long.

Anonymous said...

I wrote something and then I decided to shut up. guess I've learned my lessons well.

ditto to your last two comment makers.

Anonymous said...

So,according to TR, the following pieces of conventional wisdom are not really true at all:

You must be finished with the dissertation or you will not get a job.
You must have numerous articles published, not just one, or you will not get a job.
You must be a god of teaching or you will not get a job.

Actually, I know people who have been hired, for very good positions, who had none of these things to their credit, but it's good to hear it from the other side. If there are any rational principles governing the market, I have yet to figure them out.

Tenured Radical said...

Dear anonymous,

Spot on. It's hard to believe, given how horrible the market is, but many search committees still want to see you for who you are and what you think about. And yes, there are achievements that help lift you out of the crowd. But -- obviously speaking from the privileged positions of reading the apps -- there is little more exciting than starting to read a candidate's letter and realizing that this person is doing terrific research and that you want to meet him or her. Maybe the only thing more exciting is actually meeting that person in a preliminary interview and realizing that s/he is just as good as you thought (interesting, smart, mature, confident, witty) ....and really, there is no formula of counting things on a vitae that can sway you once you realize that this is the candidate you *must* have.


Anonymous said...

Thanks for the feedback, TR. One other question about conventional wisdom - what is your take on this one?

If you don't get a TT job in X number of years, you should give up and leave the profession.

What does X represent?

Tenured Radical said...

X represents your tolerance for uncertainty.