In 2011, researchers at Ben Gurion University in Israel and Columbia University found Judges’ rulings are affected by lunch breaks. The judges granted 65% of parole requests at the beginning of the day. The rate declined steadily until a lunch break, after which the rate jumped back to 65%. They granted almost no requests at the end of the day.

One of the authors, Jonathan Levav, says the judges may have been grumpy from hunger, but they could also be experiencing mental fatigue.

This study got me thinking about a similar situation where I judge others. As an engineer who is part of the interview rotation at The New York Times (check out our careers), I must constantly judge others’ skills and experience.

Am I more likely to fail an interviewee at the end of my day? Am I more likely to pass an interview shortly after my lunch break?

Here are a few ideas to reduce bias, focused on things I can control at my level: an individual not in charge of the hiring process.

Consistent Time

A straightforward solution is to conduct interviews at consistent times. Generally, I do interviews between 1-4 pm eastern time, and I don't offer availability outside this window. Sure, 3 pm is a couple hours after my lunch break, but I have to be at least somewhat flexible for candidates.

This isn't a perfect solution because some candidates aren't available during this window. But it has worked for most interview schedules I've been a part of recently.

Consistent Questions

Another way to avoid bias is to have a consistent set of questions I judge candidates against. These are not the questions that I ask in the interview. These are the questions I use to evaluate the interview.

These questions are not absolute, where 5 out of 5 equals a hire recommendation, and 2 out of 5 results in a rejection recommendation. Instead, I judge interview scores relative to each other. If most interviews score 2 out of 5, I should recommend someone who gets 3 or 4 out of 5.

One word of caution, you should avoid a race to the bottom, where you accept candidates who don't meet your standards just because it's taking a while to find the right candidate.

Example Questions

For coding interviews, I might use a set of questions like this:

  • Did they ask clarifying questions about the problem?
  • Does the code compile?
  • Did they add or describe tests that consider edge cases?
  • Are they able to describe the algorithmic complexity of the solution?
  • Were they able to use language concepts such as structs, maps, or arrays?

For a full stack system design interview, the questions might look like this:

  • Did they ask clarifying questions about the problem?
  • Did they describe the API according to a consistent standard such as REST or GraphQL?
  • Were they able to justify choices with a technical reason? (Not: it's what I always use)
  • Did they consider SEO, security, and accessibility in the design?
  • Were they able to describe a clear component structure?

These questions aren't perfect. However, they should be better than going by my “gut” feeling, which will be affected by hunger, how difficult my day has been, or even how nice the candidate's zoom background is.

The important part is that you use the same questions for every interview. The consistency here is key, because it ensures each candidate is judged against the same criteria.

Resources