Interview Strategies at Flurry

April 3, 2013
|
Jon Miller
Facebook Twitter Google+ LinkedIn Email Email

To meet the demands of the market, Flurry has grown rapidly. Our numbers have roughly doubled each year, and we plan to continue this trajectory for the foreseeable future. Because of this pattern, we spend a significant amount of time interviewing candidates. Here is our approach.

Synopsis

Untitled

Our process is designed to help us discover as much as we can about each candidate as efficiently as possible. We begin with a phone screen, which is a low effort, low cost handshake between us and the candidate. This gives us a basic sense of their programming knowledge and their compatibility with us. Then, we follow up with a code test that evaluates their ability to design and implement a solution to the proposed challenge. Finally, we conduct two onsite interviews, one of which covers the design decisions made in their code test and their technical abilities, and another which covers their creative problem solving skills and delves into their past experiences.

During the two interviews, we assess the following:

  • Communication skills
  • Problem solving skills
  • Design ability
  • Team fit

Giving Hints

Most candidates will get stuck solving a problem at least once during the interviews. This is our opportunity to direct the conversation with a hint. A good hint can sometimes be very difficult to give. If an interviewee has chosen to answer a problem solving question in a way that is unusual but not necessarily incorrect, and then gets stuck, it is up to us to think ahead and supply a hint that helps them get to where they want to go. This can be especially challenging if the interviewee knows more about the subject than we do. The alternative is to give a hint that sets the candidate onto a path that leads to a known good solution, but this has several drawbacks. First, it wastes time because the interviewee must start again. Second, the candidate often still has their original idea in mind, which distracts them. Finally, pointing someone in a completely different direction is jarring and throws them off, especially if they start analyzing whether they have jeopardized their chances at an offer by getting the question wrong. Ideally, a good hint will provoke a thoughtful discussion about how to arrive at a solution. This also introduces a collaborative element to the interview and gives the interviewee a chance to teach us something. By digging in like this, we gain a wealth of information about their ability to communicate and how they approach problems.

Reviewing the Code Test

The code test tells us several things about the candidate:

  • How they write code in a natural setting
  • How they structure blocks of code – is it organized into logical units of work?
  • How they test their code – are their tests concise and relevant? What do the tests cover? Is their code designed to be testable?
  • How they design their algorithms – what approach does the candidate take?

Reviewing the test with the candidate mimics an actual code review. We discuss the compromises they made, such as performance, readability, and testability. We also explore the design decisions the candidate made to gain insight into how they prioritize their development practices. This also gives us a sense of how they respond to feedback: do they give justifications for their design decisions? Do they readily acknowledge mistakes?

Digging Deeper

Any candidate can memorize the answers to common interview questions, but eliciting such canned responses will not give us useful information about their abilities as a programmer. Therefore, we choose questions that lead to interesting discussion. For a hypothetical example, take the simple question of “What is a tree set?” If the candidate talks about the O(log n) find/insert/remove operations, we could follow up with “Since a tree is faster than a linked list for common operations, what are some reasons to use a linked list instead of a tree set?” This forces the candidate to explain their thought process, which is much more valuable than knowing whether the candidate can regurgitate runtime efficiencies of common structures.

Listening Well

At Flurry, we care about a candidate’s communication skills and fit just as much as their technical abilities. We pick up that information in a few ways. A lot of this type of information is gleaned during the technical portions. If a technical question is too hard or unexpected, we can look at an interviewee’s coping mechanisms – do they panic? What is their thought process – does it jump around or does it logically build from a set of premises? If a question is too easy, how does the candidate respond? Is he arrogant or dismissive? These are the keys which will tell us if an interviewee is confident about what they know and don’t know and is able to think for themselves. This kind of information can come from anywhere at any time, so we make sure to stay and take notes regardless of what it happening. By the end of the process, several different interviewers must come to a single conclusion, and if anybody thinks the candidate is not qualified, we pass on them.

Describing Flurry to the Interviewee

Anyone familiar with the process knows that interviews are just as much about helping the candidate learn about the company as they are evaluating the candidate. Since Flurry values intelligence and honesty, we want to attract the same qualities in our candidates. To that end, during the interview process, we are truthful about Flurry’s strengths and weaknesses and present a clear picture of what it will be like working here. The people interviewing the candidate are their future colleagues, and interviews take place in the same location as their future work environment. By the end of it, a candidate should have a good idea what it is like to work here, should they accept an offer.