Eeny, Meeny, Miny, Moe*: Or, How to Evaluate the Candidate Pool

One of the things everyone is talking about in the presidential race is the capacity for good decision-making, who has it, and what relationship that bears to previous experience making hard decisions. OK, so you are not running for national office, nor are you a pit bull with lipstick (or was that a hockey Mom who is a pig? I can never remember.) But you are on a search committee. And you have never been on one before. And there is a large drawer of files to evaluate. You have decisions to make. So today's topic is:

How do you evaluate a candidate pool and decide which 10-12 people you want to invite to a preliminary interview for a tenure-track job?

There are a number of criteria that will be in play, depending on what kind of slot your optimum candidate is expected to fill and how broadly you advertised in terms of field. But one of the things I think is important is to have some sense of what you are looking for, prior to reading the files; a sense of who the ideal candidate is that is well-defined, but also flexible. Your approach to reading files needs to simultaneously respect the ad that was approved by your colleagues and the administration and be responsive to the individual strengths and weaknesses of the actual candidate pool you have before you.

Thus, your evaluation of a file starts with the ad criteria, but then extends to the role that this candidate will fill in the department more generally. The basic questions that need to be asked of each candidate are:

1) How well does this candidate fit the job for which we advertised, in terms of rank, field and any stated subfields?

One of the reasons I advocate a carefully written ad that addresses department needs (and dreams) is that defining, or redefining, the job in relation to the candidate pool that you have can sometimes be necessary and desirable, but it can also create a lot of chaos on the committee, and later in department discussions. A committee member is most likely to be seen by others as "pushing an agenda" when s/he tries to persuade colleagues that a candidate who is inappropriate for the job actually fits the ad. Be honest: how well is each candidate responding to the ad? Does s/he actually do research in the field the department needs, or is that field really secondary to another focus? And, while all of us are likely to shift focus somewhat in our scholarly careers, does that candidate seem to want to sustain research in the field and -- this is the sign of someone to take more seriously, in my view -- seem to have ideas that are pushing the field in a new direction?

2) Does the candidate offer something special, in addition to the field in play, that would be an advantage to your department?

This is a tricky one. In the latter days of the Dark Ages, when I was interviewing for jobs in political history, at some point I would usually be asked: "And could you teach the women's history survey?" This question came up because I was a woman, of course, since I had not yet done any research or published in the field. Because historians of women were (except for Tom Dublin) women, it stood to reason that the opposite was true as well, so there were any number of departments who saw me as a "plus one." In other words, they could hire a woman to teach political history, and never have to worry about hiring a historian of women. Ta-da! I always said yes, of course (who wouldn't? I needed to eat too), and one day, someone actually gave me a job where that was part of what I was hired to do because my assertion that I could was never undermined by their curiosity about what kind of training might be necessary to teach the history of women.

This is a cautionary tale that actually has a happy ending. I learned to teach the course, and soon began to do research in the fields of gender and sexuality within political history. Better yet, that job led to another job, which led to a career in academic blogging. But be careful: people are not necessarily interested in studying or teaching about people "like them" as a favor to you. Unless she offers to, the Chicana woman who works on French colonies in North America probably doesn't have an ethnic history survey in her back pocket that she would love to develop for you so you can get the dean off your back, and she will resent being asked about it at in the preliminary interview.

However, when you see something on a vita that suggests the candidate brings something else to the table, that is evidence (as opposed to veiled bigotry), and the only way to pursue it is to interview the person. But the ancillary field should be raised as a question at the interview stage, not a new requirement for the job, and in a way that doesn't divert from the issue at hand: how well might the candidate address the needs your advertisement expressed?

To summarize: noticing that a candidate really does offer something special could be a reason to bounce hir to the next stage if only to find out whether the additional expertise is for real, and whether the person actually is a "plus one." If you are a small department, you might not be able to afford to devote a whole position to Italian history, but a historian of the French Renaissance whose research touches on Italy, and who has taught a section of an Italian history survey, is worth taking a longer look at.

3) What has the candidate published, and where? If the candidate is still a graduate student, how close is the dissertation to being done?

I think if one article on the curriculum vita has been seen through to publication, or has been accepted for publication, this should be counted as a real plus for a candidate. It shows maturity, and it shows that the candidate has been willing to accept the scrutiny of anonymous peer review and learn from it. Because publishing is long, hard, and often tedious work, I like to know that a job candidate has mastered the rudimentary discipline of submitting, revising, meeting deadlines and actually finishing something, an act that will have to be performed repeatedly if tenure is to be achieved. The only exception to this rule, in my book, is that there are some programs that are trying to move graduate students through more rapidly than the conventional seven years, and in that case, it may be that the bulk of the labor has gone into completing the dissertation, and the lack of a published article means nothing.

Encyclopedia articles, by the way, are nice, but don't count. They are a great thing to do for the community, but they aren't the same achievement as the sustained re-writing and re-thinking that go into a refereed article. Sorry. Book reviews: same.

As to the dissertation, if the job letter says that there are fewer than three chapters completed by mid-fall, I have concerns about whether the dissertation will be done by the time the candidate hits the ground in September, concerns that need to be addressed by the referees. The dissertation director, in particular, will be very specific about the timing of a defense if the defense is on the horizon. What is worse than that is mentors pushing students out on the market and rushing them through a quick and dirty defense on a first draft of a dissertation. This can mean that the process of turning the dissertation into a book can be a nightmare, both because the candidate has never successfully "finished" the dissertation (and thus has no clear sense of what it means to complete a major project) and because s/he may have a manuscript that s/he is embarrassed about and won't show any of hir new colleagues to get help for the next stage.

4) Does the candidate have teaching experience?

This is something the committee needs to discuss in advance. Do you need someone who can hit the ground running, in a field where there is enormous demand? Is it a new field where classes will likely be smaller in the candidate's first few years, and s/he has time to develop? Is teaching the most important thing at your institution, and candidates without experience cannot, in all fairness, be considered? (Hint: if this is so, I hope you put it in the ad.) Teaching a few classes doesn't mean the individual is an experienced teacher; not having taught at all doesn't mean the person won't be a quick study and do well in the classroom. But hiring very inexperienced teachers means that there should be institutional support for helping that person get through the first year or so until s/he develops confidence. It also means that members of the department need to be prepared to do observations early on, go over student evaluations, and look at syllabi prior to the beginning of classes to make sure that the novice teacher is on target. There is nothing sadder than a young teacher floundering in the classroom, working desperately to succeed, and not knowing how; unless it is a teacher who has floundered for three years and no one knows until reappointment. Nobody's teaching evaluations should be allowed to play God in a person's career.

Hence, teaching experience is important to consider in relation to the institutional context, not necessarily as a relative factor in the candidate pool. Teaching experience, in my view, might bounce someone onto a semi-finalist list, but lack of it should not exclude someone who seems to be an innovative scholar from your interview list because such a person can be taught to teach. Great teachers are not born: they are made, and helping a terrific young scholar become the teacher s/he can be is an interesting way to spend part of the following year.

5) Has the candidate been active in institutional work and in professional organizations?

This may seem obvious, but you are hiring a scholar, a teacher and a colleague. People who have been involved with their graduate student organizations, TA unions, campus organizing and whatnot are people who "get it" that a university or college is a living thing that needs care and feeding. The lack of such experience should not be a negative, since sometimes a graduate school is not congenial, or a person's community activities may be based in a neighborhood, church or other sphere. But institutional work is learned, and being responsible to others is the mark of a scholar who will keep up hir end of the bargain when it comes to departmental work.

As for professional organizations, the only advantage this gives a candidate from my perspective is that again, it communicates maturity. This person is starting to network, make connections in hir field, and is making the transition from graduate student to colleague already.

6) Does this candidate add diversity to our department?

There are federal laws when it comes to diversity hiring. As important, your institution has regulations. Be familiar with them, with the categories of human being who need to be identified, and with practices appropriate to the full consideration of such candidates. If you are a really "with it" search committee, you will have met with your university's affirmative action officer to discuss what kind of diversity needs to be encouraged. And there may be forms of diversity that do not count under federal law that you may value all the same: an international scholar, a scholar who speaks an extraordinary number of languages, a military veteran, or a GLBT candidate. In my view, it is wise to discuss this in advance, and keep discussing diversity in your hire as long as it is relevant.

In conclusion: I have a couple pieces of advice to keep the committee functioning as it should. First, the committee needs to meet at least once prior to reviewing the files to talk about evaluating the pool, not just so that everyone is on the same page more or less, but so that when it recommends the final three to four candidates to the department it can say why. Second, keep good notes. I create a xeroxed sheet that I fill out as I am reading, so that I have the same information for each candidate. Otherwise, how can you compare them? But also, as the committee discusses the candidates, that discussion should be pinned to actual evidence of who the candidate is and what s/he has accomplished, evidence that can be easily referenced in the file.

Your optimum state is consensus, and try not to quarrel, particularly over the petty things that carry over from other moments in departmental and university life. Interview an extra candidate if you must to prevent antagonism or division among committee members. I actually think one viable strategy is to give every committee member their top pick as a semi finalist, and then fill out the rest of the list during the ensuing discussion. As a gesture, it shows respect for the expertise every member of the committee brings to this enterprise. As a practical matter, it also means no member of the committee goes away feeling so frustrated or disappointed that s/he starts to lose commitment to the process.

And finally, no candidate should be perceived as fitting into -- or frustrating -- the ambitions of a particular faction in the department. Not only is this unfair to the candidate pool more generally, but the man who looks like a conservative on paper may be exactly the colleague that you and your feminist friends would enjoy sparring and collaborating with. Furthermore, if you are doing that, you are not seeing who they are: you are judging them in relation to who they might be to you. And the search isn't about you. It's about the department, and the scholar, deep in the pile, whose career is about to take a happy turn for the better.

*The genealogy of this title which dates it to, among other things, an early Celtic sheep counting rhyme, can be found here.