We’re Bad at Interviewing Developers (and how to fix it)
We typically don’t get trained how to interview, and we’ve all experienced the haphazard approaches of those new to it — poor organization, repeated questions, fizz-buzz… In this interview, , Lead Software Engineer at LivingSocial, tells us how to run interview days, the types of questions to ask, how else we can evaluate technical candidates and what to do after the interview.
Read more interviews with some of the world’s best developers on * or follow us on or follow us on .*
“I like to think of my software teams as little ecosystems, or tiny arcologies that exist in a bottle. They’re not entirely a closed environment, and like any ecosystem anytime you introduce anything new to that realm there’s going to be changes… Any time you hire somebody, you’re changing that ecosystem,” says Miller. “A lot of teams and companies don’t do a really great job of understanding that”.
As a result, one mistake all too many of us make according to Miller, is just “hiring from our friend networks. The friend network is such an important part of how we get jobs, but it also tends to reinforce our monocultures. We tend to be friends with people who are mostly like us, and so those are the people that we’re going to be recommending, and so those are the ones that get hired.” But, “it’s important to have some diversity… and not just the diversity we talk about in terms of gender or ethnicity or race, but age, class, looking at people’s technical backgrounds, do they come out of CS programs versus being self-taught or a boot camp? Industry backgrounds… Were they at startups versus large enterprise companies, or somewhere in between? All those pieces of diversity are going to be influential and improve the health of the ecosystem of your team.”
It’s important to consider each team in this wider context. As Miller explains, “in soccer, they say, ‘run to where the ball will be, rather than where the ball is’” and we all need to hire like that as well. You can start by having “early conversations about who you need to hire, and what you want to look for, what sort of energy and person do you want to add to your team, to influence it into a good direction and then go to those people, find them, whether it be through meetups, or user-groups… and not just your immediate friend network.”
When it comes to interviewing technical candidates, Miller suggests that you start by “splitting up the interview into topics” and then working out “the questions you’re going to ask” about that topic. However, it’s important that “you’re going to consistently ask these to all of your candidates”. It may feel “a little bit like reading a script, but it really lets you compare apples to apples as much as possible,” says Miller.
Another thing she recommends is that, if there are multiple people involved in interviewing a candidate and let’s says “you’re hiring for a front-end developer, you should have one person ask about JavaScript. You should have one person ask about browser interaction, or working with designers. Just split up the interview so that you’re not asking the same questions over and over again”. This helps because it means you’re able “to get really solid signal on a person’s skill sets, what they’re comfortable with, and what their concerns are,” explains Miller.
So what are good types of questions that we should be asking? Miller suggests that we “focus on questions about decisions that they’ve made, what choices have they made, and what choices would they make again in the future?” You want to understand whether “they’re reflective about mistakes that they’ve made. Is a candidate looking for opportunities to improve, and how do they actually go about it?”
Then you also want to get a feel for whether “they make plans for themselves, like how they would improve a certain skill set, whether that be a technical skill set or a more soft-skill set, for example, management, or project shepherding for example. Those are the kinds of questions that I think really get you at the heart of, not necessarily what somebody knows, but what they’re capable of.”
“I’m not a big fan of whiteboarding,” says Miller. “I think that’s something that we just automatically do, and we don’t think about what questions we are trying to answer by asking a candidate to solve a problem”. Instead, she recommends “pairing on projects, like actually working with somebody. It doesn’t have to be a formal or traditional pair-programming situation with one computer and two people, talking through the technical choices that they would be making as they programmed on something”.
Miller explains that “at LivingSocial, we do a code challenge… but as a launching pad to having a discussion with a candidate.” Then as they’re undertaking that exercise you can ask questions, like “why did you choose this technique? How would you do it better? What if we sat down and refactored? That’s one really good way to get to the heart of why they are making the decisions they’ve made.”
Another approach Miller suggests is “asking the employee to explain something to me, or teach something to me”. This is great for understanding “how well they communicate about something that they’re a local expert in but their intended audience is not. Could they then go off and learn a new framework, or go have a meeting with, perhaps, a stakeholder, or a client, and come back and explain what the actual requirements are to me, to distil down what I need to know?” After all, Miller says that “communication is such a big part of what we do in this job, testing for that essential skill in a really clear and explicit way can be really useful and get you a really good signal about who that candidate is and how they’re going to fit into your organization.”
“Once you’re done with your little section of the interview, you should immediately go back to your desk and not get back to work, but write down what your impressions were. What were the pros and cons, the bullet points, and find something good about the candidate and something not-so-good about the candidate, something that you wish they did have. Don’t pass this feedback back to the group but pass it back to a central person, like the hiring manager, so you’re not coloring the impressions of other people.”
For Miller, this immediate but individual feedback is important to capture so that “when you get back into that room with everybody else, whether it’s virtual or real, to really discuss your opinions… you can’t be swayed by the impressions of somebody else. For example, if you were supposed to interview them about JavaScript, and the senior JavaScript person, who’s got twenty years of experience in JavaScript, just really did not like that person, how would that color your opinion if you had to give it in that moment? If you wrote it down previously, no, this person really is good at JavaScript, then you’ve captured that honestly and you can really give honest feedback.”
When it comes to measuring and improving your hiring process, Miller says that “it’s really hard to look at who you hire and decide whether you have a good or bad process”. Instead, she recommends taking a “look at who you don’t hire. You can look at that in terms of what were the false negatives? Did we bounce this person out of the process for a specific reason and then it turns out that that reason wasn’t good, based on where they ended up going to work?” For developers in particular, “it’s really easy to LinkedIn stalk people, and peek into their GitHub profiles… to see what they’re doing a few months later. It can be really useful to four, five, or six months down the road, go back and look at the candidates that you passed over and see what they’re doing.”
Another aspect that Miller thinks you should consider is “understanding what your pipeline of candidates consists of. At each step, you have a certain amount of leakage, because people just simply don’t make it through the process or they abandon the process. How many people are you losing at each step, and is there one step that you’re losing a lot of people at? Maybe you need to refine that step, remove it, or move it earlier or later in the process based on what your organizational needs are. I think it’s also important to look at who you’re losing as well. Are you losing junior developers at a step that you really don’t want to be losing them at? Are you losing more diverse candidates? Are more women abandoning your process at a certain step than men are, and understanding, or questioning at least, your process to see, is that a problem? Can we fix it? How do we fix it?”
Read more interviews with some of the world’s best developers on * or follow us on or follow us on .*