How to be a Great Technical Interviewer

The path to becoming a great technical interviewer is full of doubts, tough decisions, and self-discovery. But I believe every engineer should try to walk it.

In my career, I’ve interviewed close to 150 people by phone, on-site, and recently — via zoom. I’ve criticized the existing on-site interview and took ownership to reshape it. I’ve inspired others to reshape the phone interview. I am mentoring and onboarding new interviewers to the pool. And I’ve realized that the technical interview is not perfect. Before we jump to explore the problematic points, let’s first learn how we came to it in the first place.

It all started with a light bulb

Or to be precise, with the person who invented the light bulb — Thomas Edison.

Before Thomas Edison and the Industrial Revolution, there was no concept of Job or Workplace. If you were born to a blacksmith — you’d become a blacksmith. Professions were inherited from father to son, and occasionally via an apprenticeship.

Then came the Industrial Revolution, which brought means of mass production and a Human Workforce. People started to become educated and were able to choose careers.

And Thomas Alva Edison decided that he needs people to help him with his work. He was self-educated and, rumors say, was not impressed with the knowledge that people acquired through formal education. Thus, he required a way to screen potential candidates.

Edison decided to create a test to evaluate candidates¹. His test included questions related to the field Edison was working in, and general knowledge questions on topics such as history and cosmology. The test then leaked in the New York Times², and thus more companies started to practice a similar process. Fun fact: Albert, yes Einstein, allegedly couldn’t pass this test.

Modern Days

Back in the modern days, not much has changed. Few studies concluded that the technical interview is useless at predicting job performance³. While I don’t believe that we should abandon the technical interview altogether, I think I’m in a position to outline a few of the problems with the technical interview and propose alternative solutions.

Problem #1: 🧠 Brain Teasers

It’s worth noting that brain-teasers slowly die, and yet some companies continue to practice them. They include questions such as How many ping pong balls you can fit in Boeing 747 or How many cows are in Canada? — and similar.

Unless your company is trying to ship ping pong balls via airplanes, or do a count of the cow population in Canada - the primary reason to ask those questions is …to make the interviewer feel smart - Former Google SVP Laszlo Bock⁴.

Problem #2: 👩‍💻 On-site coding / Home assignment

This one is still common. Companies are afraid to hire candidates whose coding abilities are mediocre, so they ask them to perform an on-site coding task or submit a coding assessment done at home. There are two main problems with those.

On-site coding does not simulate a real environment

Let’s say you ask the candidate to implement a To-Do app. You provide him with basic scaffolding of an API Server and DAL and give him 1.5 to 2 hours to implement 3 endpoints: add a to-do, complete a to-do and get a list of all the non completed to-dos.

Now, let’s ask a question — what are you trying to test? What’s the Definition of Done for such an assignment? Successful completion in the allocated time frame? A clean code and maintainable architecture?

I want to see how he writes code under stressful situations! — you say. And I ask Why? Is your company writing a lot of code under stressful situations? Then I’d rather move on to the next company.

I want to test his code qualities! — you tell me. And I ask — By putting the candidate in a constrained environment and limited time frame? That’s hardly an environment that produces quality code. In a timed environment, such as a production incident, my number one priority is to resolve the problem ASAP and get my product back online so that customers continue to be happy, not to write nice code, whatever the hell this means. I’d refactor the solution tomorrow when I have time, and I’ve done my postmortem.

Home assignment requires a time investment

As with on-site coding, home assignments goals are vague. What are you trying to test? That the candidate can write a code that adheres to your coding standard? Your coding standards are something that can be learned, not something that the candidate should guess.

And moreover, home assignments require a time investment. I only want candidates that are willing to show their dedication towards the position. Of course you do! And candidates want a fair process in which an employer shows dedication towards them, not a home assignment that will take them 2 to 3 hours of unpaid work time, while you will throw a 7 minute glimpse at the code and conclude whether it adheres to your coding standards.

Do coding question instead

Instead of performing an on-site coding that takes 1.5 to 2 hours, or throwing a home assignment — do a coding question.

The combination I’ve come to love is to design an algorithm and then translate it to code. Let’s say your algorithm is finding whether a given word is a palindrome. You can start by providing a definition:

A palindrome is a word that reads the same backward as forward. Examples of palindromes are madam or racecar. Now try to describe an algorithm that will identify if the given word is a palindrome.

The candidate then tries to come up with a solution, while you can guide her if needed. In the end, you ask her to implement the algorithm by providing a basic scaffolding of the function signature such as: function is_palindrome(word: string): boolean and a set of tests that should all pass once she completed the implementation.

This combination achieves multiple goals.

It tests the candidate’s thinking abilities

By giving a problem and asking to find a solution, you simulate a real-life scenario in which the candidate requires to think in order to find the right solution. Palindrome is not the best example, but you can come up with any other algorithm (or use Google if you lack imagination). You can understand how the candidate thinks, how he approaches unfamiliar problems.

You can complicate it with Computer Science

Algorithms can be solved in multiple ways. Some are more performant than others. You can complicate this question by asking the candidate to try to come up with a better complexity. You can ask her how she would implement it in a real-life scenario rather than laboratory conditions. The latter gives you more understanding of tools and technologies she’s familiar with and how she evaluates them for a given problem.

You see how she translates words to code

Coding is, essentially, translating algorithms and requirements to code. This skill is way more valuable than simply writing clean code. But please use an actual computer and an IDE. Nobody ever writes code on Whiteboard during their career (unless they are University Professors or writing pseudocode).

It’s simple and adaptable

Unlike a 1.5-2 hours of on-site coding or home assignment, algorithm implementation has an obvious goal — make the tests pass by implementing the missing function. There is no ambiguity in terms of should I add log calls or how performant my code should be. You’ve already discussed the solution, and now you need to implement it. On top of all that — function implementation (which should be limited to a couple of lines of code) is adaptable. It can be done in any language, and in case you are not looking for a “native speaker” of your chosen language, giving him or her option to use a more familiar language — will make the process more smooth.

Problem #3: 👑 Manifestation of your superiority

I’ve mentioned this briefly in the Brain Teasers section. Let me do a quick overview of the path from a Trainee to an Interviewer.

The newly onboarded technical interviewer will be shadowed for a couple of interviews. She starts by being completely quiet and observing her senior counterpart leading the interview. In the next interview, she gains some more confidence and asks a question or two. By third or fourth interview - I, as a Senior Interviewer, take a leap of faith and let her lead the interview by making myself completely quiet (interrupting only if I see that there is a misunderstanding between her and the interviewee).

At every step, it’s important to have a feedback call right after the interview. I want to understand how she was feeling during the interview, are there any questions. I want to listen to her decision on the interviewee. I want to give her the tools to navigate the interview better. And I want to understand whether she is ready for her first solo interview.

The transition between being a Trainee to becoming an Interviewer

At that point, the well-shadowed trainee interviewer is on his own. It’s now his path and his decisions (I’m always here for him to support and mentor should he need that).

His first interviews will be a hit or a miss. Occasionally, he will come to you with questions such as I’ve interviewed this engineer, she looks great but lacks X and Y. What do you think? I’ve been like this at the beginning of my path. But like with anything new, we adapt and start to make decisions on our own. And there comes a time when each of us is going to have to make a choice.

Either bring balance to the Force or leave it in darkness⁵

Everything about the technical interview can be summarized by one word — Control. The interview is about having control over the outcome of the hiring process. We want to have the control that we won’t hire a candidate that has bad coding standards, is unable to perform architecture design, or is not proactive enough. This is the reason some companies have 3, 4, or even 5 interviews, each focusing on different aspects starting from coding and going to architecture, proactivity, and even personal skills such as conflict resolution, or working in a team.

And while having control is important, control can also be abused. And the easiest way to abuse control — is to feel superiority. Consider the following, somewhat hypothetical, somewhat real, scenario: An interviewee is coming to your company for a technical interview. He is greeted by 2 senior engineers with grumpy faces who are tired of this process because the last 5 people they interviewed were trying to impose a schema on a by-design schema-less database (Sorry, it’s a personal pain point for me).

Constant failure brings frustration and builds resentment towards the process. Don’t be like that. Don’t be the person who signals that the potential candidate is a failure just because your last interviews were not successful. Yes, smile. Yes, be nice. The interview is not about you and your frustration with the process. Keep it professional. Remember, behind every candidate might hide a great engineer who can bring value to your company and your product, as well as your team and even yourself.

When you start asking questions, please remember the following — You’ve asked them hundreds of times and know to navigate yourself from every possible answer and situation. The same is not true about the candidate, avoid making yourself the smartest man in the room. Remember — it’s not about you.

The fix is in framing

Recently, I’ve read an article by Mayank Verma about how you should treat the Technical Interview as more of a discussion rather than an interview. The change is not radical — it’s in framing. Instead of treating the interview as a process in which you judge the interviewee by his answers, we can turn the interview into a discussion where both the Interviewee and the Interviewer are working towards a joint solution with the purpose of understanding whether the Interviewee is suitable for the position.

This can take form in different representations. For example: it can be a discussion about the interviewee’s experience. An exercise I like to do is to ask the candidate to pick a feature or an architecture that she considers to be interesting, challenging, or comfortable to discuss. This achieves 2 things:

1. It gives the candidate freedom and control over the interview. Instead of throwing her into the water and asking her to implement some sort of system I’ve designed, I let her talk about something that she did — thus transferring the control from myself to her. This makes the relationship between us equal, instead of Interviewer / Interviewee.
2. This gives me a glimpse at what is the most challenging or interesting thing the candidate worked on.

I then proceed to a discussion. We discuss who was responsible for this feature, who decided on the architecture, what was her role in that. What would she change now, if she had the time and resources? This gives me way more insight about her as a candidate, instead of a generic design question such as Let’s now design {Some System}. The former is familiar to the candidate, and she probably knows the in-s and out-s of the architecture; while the latter is… you can’t expect a candidate to do a system design in 1.5 hours and have a great design. Moreover, the latter might be a hit for someone who worked with systems in that particular domain, and a big miss for someone who did not.

🗒 The Manifest of a Great Technical Interviewer

As I’ve mentioned earlier, I’m getting close to the mark of 150 interviewed people. I have an unexplainable passion for interviewing. I have a great track record for post-interview feedback. Furthermore, I learn a lot from the interview as an interviewer, and I think every engineer should try interviewing. Below is a summary of traits that I consider mandatory to become a Great Technical Interviewer.

1. Be curious. By being curious, you arrive at the interview with the purpose to discover. Curiosity allows you to evaluate the candidate without prior biases.
2. Evaluate instead of judging. By judging, we search for imperfections; by evaluation — we try to find a match. Approach the interview from an evaluation standpoint rather than from a judging one.
3. Let go of the ego. The interview is not about you or your knowledge. You are not here to show the candidate how awesome you are. Let go of the ego and focus on evaluating the candidate’s skills for the position.
4. Listen first, only then speak. In an ideal interview, the candidate should talk most of the time. Your role is to listen and guide her towards the skills you are looking for.
5. Understand his motivation. Many candidates submit their CVs to multiple jobs in different domains. Remember that your role is not only to find the best engineer for your team, but to also make sure that your team is the best fit for the candidate. Listen to what he is looking for and be honest with whether you can provide that.
6. Be an expert in your field (or admit your lack of knowledge). If you are evaluating a candidate for great RDBMS knowledge, then you should be close to an expert in that field. Finding yourself arguing with a candidate on a specific matter, only to later find out that she was right, and you were wrong, due to lack of knowledge, — is an unpleasant situation to be in. If in doubt, you can always say I don’t know, but I’ll definitely check that after the interview (and obviously go and check).
7. Seek to learn. Every person I’ve interviewed — taught me something new. It might be filling a technical gap I’ve had or acquiring new communication skills. Always seek to learn, even from candidates that do not pass your interview.
8. Have a structure, but know to be flexible. You must lead the interview. It should have structure. It should evaluate the candidate based on the position you are interviewing for. But you should also remember that everyone is unique, and occasionally, you will encounter candidates with a unique experience that will break your interview structure. You must know how to navigate yourself back towards the qualities you are looking for, without making the process feel robotic or scripted.
9. Accept the fact that you are going to fail. No matter how great you are as an interviewer, eventually you will miss a great engineer or hire an unsuitable one. Your goal should be to minimize those mistakes, but you should accept the fact that they will happen.
10. Mentor others. When you feel comfortable with the interview, mentor other developers so they, too, will become Great Technical Interviewers.

This path is hard, but it is rewarding

Interviewing is a hard process. There are times when the hiring is on fire, and you will have 2-3 interviews a week, each taking 1.5 to 2 hours. Don’t overload yourself. Pushback if you feel you’ve had too many interviews and need a rest. Always continue to improve yourself and the process. Questions the existing status quo — always remember that the status quo became the status quo because someone questioned it. And enjoy the process — it’s way more fulfilling once you do. Good luck, young Padawan.

Bonus — Tips for candidates from an Interviewer’s perspective

Once I became an interviewer, I’ve realized some things that I wish I knew when I was an interviewee. Below is a summary of them.

1. Embrace rejections. Rejection is not necessarily a red mark that says you are not a good engineer. Rejection simply means the qualities you possess do not suit our position. I’ve interviewed talented engineers that I’ve had to reject because they were not suitable for the position.
2. Always ask for feedback. I was surprised how few engineers ask for post-rejection feedback. If you are interviewing for a company and get rejected — always ask for honest feedback. Don’t just settle on We’ve decided to proceed with other candidates. Ask what you can improve so that the next time you apply — you will succeed. (Not all companies are willing to provide such feedback. If you request one and get a generic response or no response, consider yourself lucky — you’ve avoided a workplace that isn’t concerned with your development).
3. Avoid biased information. Your CV is not your Facebook page. You think that the fact that you play the guitar is cute — but we are all humans, and we can be biased. You want to avoid blowing your interview just because you fell on an interviewer who prefers drums to guitar. Interviewers and HR does a lot of work to eliminate bias from our decision-making, but we are all humans. Avoid including any personal information in your CV.
4. Interview the company. Remember, the interview is not only about them interviewing you — but it’s also about you interviewing the company. Make a list of things that are important to you in a company, and make sure to ask questions and understand whether this particular company can provide what you are looking for.
5. Don’t be afraid to say I don’t know. I’d have more respect towards a candidate that says “I don’t know, I never worked with X. I’d take some time and read about it”, instead of a candidate that will try to confuse me with buzzwords.
6. Have fun. Treat interviews as games. See how well you can pass them and what you can learn from them.

Footnotes

1. Business Insider - The Evolution of the Job Interview ↩

2. The New York Times 1921-05-11 - Edison Questions Stir up a Storm of Victims ↩

3. The New York Times - The Utter Uselessness of Job Interviews ↩

4. The Atlantic - Google Finally Admits That Its Infamous Brainteasers Were Completely Useless for Hiring ↩

5. Obi-Wan to Darth Vader ↩