Interviewing for an engineering job in the tech world can mean you’ll be asked all sorts of questions. Sometimes, the job interview questions can be pretty straightforward: “Tell me about a time that you have implemented an effective monitoring solution for a production system.” Other times, the questions are impossible to answer and designed to spark your creativity: “How many windows are in New York City?” After passing the initial interview, the applicant or candidate graduates to the next tier of interviewing: The often-dreaded technical interview.
What is a technical interview?
A technical interview is one that is conducted to gauge a candidate’s skill level for positions in the information technology, engineering, and science fields. It may also determine how much a candidate knows in more niche areas of a company, such as marketing, sales, and HR.
Preparing for technical Interviews
Prospective engineers often face a challenge when it comes to preparing for the technical interview, largely because there is no playbook for how companies set them up technical. It’s unclear whether to prepare by memorizing many different topics, or focusing on specific projects. Is it better to practice with a computer or a peer engineer? There are an overwhelming number of resources available online, but with little clarity as to what the standard is for a technical interview and little guidance from the company on what to expect, most of the time engineers start technical interviews in the dark.
Inconsistencies in the technical interview process isn’t just a job candidate problem. In fact, many companies struggle to set up a technical interview process that is effective, equitable, and allows the hiring manager to compare candidates. The problem with technical interviewing compounds when a company is experiencing rapid growth.
The challenge with conducting technical interviews at a growing company
"Imagine you had a hiring target of doubling your team size and all your interviews are conducted remotely. Welcome to GitLab," says Clement Ho, frontend engineering manager on the Monitor: Health team at GitLab.
GitLab more than doubled the number of hires from around 400 in 2019 to roughly 1300 by end of 2020.
We identifed three core challenges with orchestrating technical interviews as GitLab grows.
- We didn't have enough interviewers for the pipeline of candidates.
- Our technical interviewing process was inconsistent and even a little biased.
- It was difficult to measure whether or not we were raising the bar.
"And by raising the bar, I mean making sure each candidate that joins the team makes the team better," says Clement.
These problems are by no means unique to GitLab. Any engineering company that is scaling rapidly will encounter some growing pains when it comes to hiring, and many will end up falling back on some of the typical models for conducting technical interviews.
The typical technical interview methods
During his talk, "Using GitLab to Power our Frontend Technical Interviews" at GitLab Commit San Francisco, Clement explained the four different techniques that are often employed in technical interviews. Each method comes with advantages and disadvantages from the perspective of the hiring manager.
Good technical interview questions
A good technical interview needs to be about more than practical skills – it’s about the whole package.A candidate should possess the ideal coding skills but also be a team and culture fit and be able to discuss developer topics efficiently. A technical interview should include both situational interview questions and a skills assessment to discern a candidate’s potential.
The types of questions to ask can concern a candidate’s technical abilities and background, their career journey so far, and queries specific to the team or company.
Types of questions asked during a technical interview and their purpose
Even though employers have already reviewed your resume and cover letter, they will want you to flesh that out during the interview to learn more about how you attained those skills. In order to assess your level of experience, they will likely also ask you to provide concrete examples from prior jobs. Make sure you are prepared—do your research on the company and the type of questions you may be potentially asked. This will help build your confidence level and reduce any nervousness you might feel. It’s also an opportunity for you to set yourself apart from other candidates by showcasing your knowledge and additional skills you can bring to the job.
It is important to be honest about your skill set because that is something employers value. You may find the company will be willing to hire someone who is transparent about the areas where they need to improve and where they’d like to gain more skills.
Examples of common questions to expect in a technical interview:
- What coding languages are you most familiar with?
- What is your experience with Kubernetes with a specific example?
- What’s the purpose of continuous integration in an automated build?
- How have your previous technical roles prepared you for this job?
- Tell me about a time when you received an unexpected assignment: how did you react, and what did the experience teach you?
- Please provide more details about your educational background and how it prepared you for this position.
- How did you go about teaching yourself a necessary technical skill while you were working on a project?
- What are your strengths, and where do you think you need to improve your skills?
- Do you have any technical certifications?
- Please detail the work you did on the project you are most proud of.
- What are your favorite and least favorite tech tools, and why?
- What are the pros/cons of working in an agile environment?
Sample technical interview questions and answers
- How do you stay current with your technical knowledge and skills? It’s a good idea to list online content you use to educate yourself, as well as tutorials and conferences you have attended to gain more knowledge. Perhaps you have also worked closely with vendors or attended sessions to learn about new product features.
- How do you troubleshoot technical problems? Discuss the steps you take when you are answering a question. This will give employers a sense of how you problem-solve, and it provides a good overview of how well you understand the relevant concepts. Even if you don’t answer a question correctly, it will show the interviewer your process and reasoning, which are also important. You can mention resources you use, such as GitLab and Stack Exchange, as well as the developer community and any publications you read for advice.
- What is your level of experience with the software programs mentioned on your resume? Describe how many years you have used the tools, your impressions of them, and bring up the companies you used them at, with specific examples.
- What programming language are you most proficient in? You should discuss how you have become proficient in this language and why it is the one you are most comfortable using. You can also cite other languages you are familiar with.
- Describe a time you made an error and how you resolved it. Don’t use an example of an egregious error since that may put you in a negative light. Be sure to emphasize that you took responsibility and acted with integrity, and did whatever it took to resolve the issue.
Soft skills and coding skills to highlight in a technical interview
A technical interview assesses your technical expertise, coding skills, and ability to fit into a team. However, soft skills are just as important and often aid in the development of more technical skills – particularly in a team setting.
As the technical interview progresses, be prepared to tackle some questions about soft skills like:
- Communication skills: How does the candidate contribute to group discussions, confront problems, or give and receive feedback?
- Organizational skills: What are the ways in which the candidate provides visibility into their work processes and their methods of staying on task?
- Collaboration skills: Are they interested in helping their teammates? What do they think are the keys to successfully navigating a team project? How have they collaborated on past projects?
- Creative problem solving: How do they work through a problem in a project? Do they use both analytical and creative thinking to come up with solutions?
Verbal technical questions
There are countless articles online that try to prepare job candidates for a verbal technical interview, but whether this method truly effective for evaluating the technical competency of a software engineer is debatable.
In the typical scenario, the interviewer asks the candidate to describe a technical concept and tries to measure their fluency in said concept based on the quality of the conversation.
The advantage of this method is that the interviewer can understand how the candidate communicates, which is of particular importance when the engineering team is all-remote, as is the case at GitLab. The drawback? Being a good communicator does not necessarily mean the candidate knows how to code effectively.
"So I've interviewed candidates that could talk the talk, but they couldn't really write the code,” says Clement. "And that's not a great situation for an engineer to join GitLab." Clement’s team has moved away from using verbal technical questions as a method for evaluating candidates.
Live coding exercises
One of the more popular methods for evaluating engineers is through live coding. While it allows the evaluator to see how engineering candidates answer data structure questions, it also has its disadvantages.
A key advantage of live coding data structures is that it offers a fairly consistent measurement and evaluation.
"I can talk to another manager or another interviewer and be able to communicate, 'Hey, this person wasn't able to do a linked list, they got stuck here. They weren't able to understand a runtime efficiency here.' So it's pretty consistent," says Clement.
But the ability to create data structures is not always the best indicator of ability. Oftentimes engineers with a very traditional background or recent graduates will shine here, but someone who is more senior and able to do a lot of great things, but is perhaps not as brushed up on data structures, may struggle.
Live coding interviews probably aren’t going anywhere fast, but the pitfalls of this method are well documented by engineers and hiring managers. Brennan Moore, a product engineer in New York City, explains why he does not conduct live coding interviews when evaluating a prospective candidate:
"Much like the SAT when applying for college, live coding is a structured test. I didn’t go to a school that trained me to do live coding, and so will probably fail the test. As I’ve experienced it, live coding isn’t the meritocratic space that it pretends to be. Live coding interviews weed out the people who are good at live coding interviews," says Brennan in his blog post.
At GitLab, we found that live coding exercises don't accurately represent engineering capability. Oftentimes, a recent computer science graduate will outperform a more senior candidate with a lot of valuable experience. In summary, live coding exercises will often disadvantage more senior candidates, people who are nervous in high-pressure situations (read: everyone), and advantages more junior engineers or people who have practiced live coding.
Digital prompt
A third common method for evaluating candidates is to ask the engineer to code a UI using an online editor while on screen share with the evaluator.
The advantage of this method is that it allows the evaluator to observe how a candidate builds. The drawbacks here are similar to those with live coding. First, the engineer is under pressure to build while the evaluator watches on, making it a nerve-wracking situation. The other drawbacks come from an evaluation perspective: It is challenging to measure the effectiveness of this method and it is hard to compare between candidates.
Take-home project
Any engineer (or writer, for that matter) can tell you, the supplemental take-home project is a very common ask when going through an interview process. The advantage here for us is that this assignment closely mimics the reality of building environments while working remotely at GitLab.
But this task comes with major drawbacks, mainly that it disadvantages candidates who may not have the time or capacity to complete the project.
"… imagine a scenario where you're a single parent and you have kids; you may not have as much opportunity to take dedicated time, a couple of hours after work to really focus on a take-home project compared to someone from a more privileged background," says Clement. "They might be able to dedicate and output something better."
Diversity and inclusion is a core value for GitLab, and anything that disadvantages candidates from underrepresented groups is not inclusive, and therefore suboptimal for evaluating candidates based on their engineering abilities.
What are they looking for during a technical interview?
Companies want candidates who can discuss the industry in the context of the job they are applying for. Be prepared to discuss examples of your work. Many will want to hear about soft skills, too—your ability to communicate and collaborate and work with others to problem-solve issues.
They will also want to see how passionate and enthusiastic you are and whether you have the self-motivation to not only do the job but take the initiative to do more than what you’re tasked with.
Also, interviewers will want to see whether candidates have the desire to increase their technical knowledge.
Online preparation tools and resources for technical interviews
- Indeed offers a career guide to help prepare for a technical interview.
- Interview Kickstart has several webinars to help prepare engineers for interviews.
- Udemy offers a course in Technical Interview Skills.
- Codecademy also offers a course called - Technical Interview Practice with Python.
- Here are some more general interview tips that are applicable to all candidates.
Meaningful questions to ask the interviewer
Candidates will also be given a chance to ask questions they might have to learn more about the company. This is a great opportunity to gain more insight into how the company operates, what its philosophy is, and its vision for the long term.
It’s also a good way to glean how the company views its IT team. If you don’t ask questions, that could give the impression you are unprepared or not terribly interested in the job.
Questions to ask can include:
- What does a typical day looks like in this role?
- Are there opportunities for training and further advancement?
- What software development methodology do you use?
- What are your code review practices?
- Do you have on-call rotations? If so, how long is one rotation?
- What are the responsibilities of the person on call?
- Please provide more details about the team I will be working with, such as how many people are there, what their roles are, what the hierarchy is, and what areas of improvement you would like to see on the team.
The new way
While each method for conducting a technical interview comes with advantages, there are also numerous disadvantages when it comes to conducting an effective and measurable evaluation and creating an equitable interview process. Under the guidance of Clement, the Monitor:Health team decided to interview frontend engineers in an entirely new way using GitLab.
Now let's take a deep dive into the nuts and bolts of reinventing the technical interview for frontend engineers at GitLab. Just wondering about the key takeaways? Skip ahead. As we continue to iterate on a more effective and measurable technical interview process, we hope this inspires other engineering organizations to rethink theirs and share learnings with us.
Our first step: Standardize the interview process.
Fixing an MR on a test project
The team standardized the interview process by creating an open source test project, called project-seeder, which seeds projects to different candidates using a GitLab Bot. Candidates are assigned a merge request to troubleshoot in the project created for the technical interview. The project-seeder is powered by the GitLab Bot so the interviewer doesn't have to worry about API keys, and works in four steps:
- Exports the template project
- Imports template project
- Adds users with expiration
- Triggers pipeline for candidate to review MR
The candidate is sent an email with a link to the MR the candidate is assigned to fix as part of the technical interview.
Standardize the evaluation rubric
The team also created a standardized rubric for how the candidate's performance on a technical interview is evaluated.
"We don't want to be in a situation where unconscious bias or bias of one candidate over another plays a part because of our preconceived notions," says Clement.
Creating a rubric that looks at multiple categories allows the evaluator to look at the performance of the candidate from a more holistic perspective, as opposed to looking at a candidate's performance on one technology.
The team created a Periscope dashboard to create a feedback loop between the candidates and evaluators to identify opportunities for improvement in the technical interviewing process.
The frontend engineering team used Periscope to collect feedback from candidates who participate in technical interviews.
Demoing the technical interview
Inside the technical interview project
Clement created a sample project to demonstrate how we use GitLab to power our technical interviews.
In the gl-commit-example group, there is a subgroup with all the interview projects we are seeding to the imaginary candidates, a template, and a project seeder.
The interview project's subgroup, template, and project seeder application lives inside the sample project for the technical interview.
Inside the template, there are GitLab pages and the interview test merge request.
The assignment here is pretty simple. The candidate needs to update the website to say "Hello GitLab Commit SF," but in order to accomplish this, the candidate will need to fix the failing pipeline.
Powering project-seeder
We use variables from GitLab CI to configure the project-seeder application.
Inside the project-seeder application which seeds the interview projects to job candidates.
"I'm creating new-project-example-two, and I'm adding this bot user that I created and the expiration, so I can just easily run this pipeline and it'll seed this project," says Clement.
The next step is to run the setup pipeline, which will create the project, import the project, export the project, and share it with the job candidate.
A look inside the pipeline that will create the test project.
Looking inside example-one, we can see there is a project and broken MR.
"And an example for a candidate – they would probably look at the CI and see, 'Oh there's a failing test. Let's see what that's about. Oh, it looks like it's checking for "hello world". So since we changed the message earlier, we can just change this and get this test passing and then pass this interview,'" says Clement.
Why this new model for technical interviews is better
The new model surpasses the old model because we created realistic scenarios that reflect what it's like to actually work for GitLab, and we established a more consistent method of measurement.
"So we're able to get better candidates overall. Candidates that pass through this technical interview, we're sure that they're going to be successful at GitLab," says Clement.
By designing our technical interviews this way, we can ensure that the interview project matches our actual product architecture at GitLab, which in this case is Ruby on Rails for Vue JS.
We also struggled in the past with finding a good way to check that the candidate knows how to use Git, and can navigate pipelines and testing. By using GitLab for interviews, we're able to confirm a candidate's competency with Git implicitly by evaluating their performance on the technical interviews.
We wanted to mirror the actual experience of troubleshooting a broken MR while working at GitLab, so we allow our candidates to use the internet during their technical interview. This allows the evaluator to see how the candidate solves problems and see their resourcefulness.
"If you're already using GitLab for your tooling, you're just exposing them to what it's like to work at GitLab; it's a more accurate representation," says Clement. "And you can also make sure you're measuring testing proficiency and you make sure they understand how that works before they join your company."
Four key takeaways from our technical interview update
Whether or not a company uses GitLab, there are a few key lessons that we learned by iterating on how we conduct technical interviews for engineers.
-
Make technical interviews as much like real work as possible: Nine times out of ten, an engineering manager isn't going to sit back and watch an engineer break a sweat in a live coding exercise, any more than they will watch on as an engineer builds in UI. Create realistic scenarios based on the actual work and evaluate based on the candidate's performance.
-
Make any technical interview process "open-book": Engineering doesn't involve much rote memorization. Instead, allow the engineering candidate to use the internet (and in our case, the GitLab Handbook) to look up their questions. It's better to see how a candidate applies their knowledge and troubleshoots the inevitable problems that may arise. This change will likely improve your candidate experience too.
-
Standardize your rubric: However the technical interview is done, make sure that the rubric is as objective as possible and that the candidate is evaluated based on various criteria, not on their familiarity with a particular technology. A strong rubric means a stronger, more valid method for evaluating candidate performance.
-
Create an inclusive process: Think critically about how the technical interviewing process and evaluation is structured so a diverse group of candidates can be recruited and evaluated based on their merits. When in doubt, ask a diversity, inclusion and belonging expert or turn to your human resources team for advice. Still coming up empty? Hire a diversity consultant; it will be worth it.
Interviewing at GitLab? We encourage you to use the resources GitLab creates during your technical interview. We don't publish our evaluation criteria publicly, but we do have the Periscope dashboard which can provide some insight.
Watch Clement's talk from GitLab Commit San Francisco to learn more about how we used GitLab to power our technical interviewing process.
“Technical interviewing is broken. Here are some tips on how to fix it @GitLab” – Sara Kassabian
Click to tweet
Guide to the Cloud
Harness the power of the cloud with microservices, cloud-agnostic DevOps, and workflow portability.
Learn more