How to Make Technical Interviews More Inclusive - HackerRank Blog

Every metric we have indicates that diversity, equity, and inclusion initiatives in the tech industry are in a state of crisis.

Women, people of color, and LGBTQ+ employees have had the hardest time adapting to the new remote-first workplace. And the workforce participation rate of women in November 2021 hit its lowest point in over three decades.

If organizations don’t act quickly, decades of work and progress toward a more equitable workforce will be lost.

In this post, we break down how organizations can advance their DEI goals and build hiring processes that are fair, equitable, and inclusive.

DEI & Talent Pipelines in Tech Hiring

The differences between the participation rates of men and women in STEM careers are well-documented. Between 1970 and 1984, the percentage of computer science degrees earned by women rose from 14 to 37 percent. Currently, however, women make up only 18 percent of computer science graduates. 

By looking at demographic data for technical roles, we can see this disparity in education translating into a disparity in workforce participation. In the United States, women account for only 22 percent of software engineers, 21.8 percent of web developers, and 20.4 percent of data scientists. 

The state of affairs is similar along racial lines. Despite making up 13.9 percent of the U.S. population, only 4.9 percent of software engineers identify as Black or African American. The same is true for Hispanic and Latino engineers, who hold just 6.9 percent of software engineering roles in the U.S., despite making up 18.9 percent of the U.S. population.

The reasons behind these STEM participation rates are complex and far-reaching – and far more than one blog post can cover. But one way innovative organizations are making progress with their DEI initiatives is by reassessing how their hiring processes support or hinder workforce participation of underrepresented groups.

Technical Interviews Are the Standard. But Are They Broken?

The technical interview, sometimes called the coding interview, is the process of evaluating candidates through interviews that place an emphasis on technical skills. 

During the process, hiring managers and recruiters rely on individual assignments, often referred to as coding questions or challenges, to score a candidate’s proficiency in key skills. 

In theory, the purpose of technical interviews is to evaluate whether an applicant can code, work in a team environment, and communicate their ideas effectively. However, the way that technical interviews are often conducted significantly limits their ability to measure these skills. 

Gaps in the process may also open the door for factors other than one’s ability to code, cooperate, and communicate to influence an applicant’s performance in the interview.

The processes companies use to interview developers have huge implications for their ability to build effective teams and attract great talent. And the careers of developers rely on the opportunity to demonstrate their skills in a fair, effective interview process. 

Technical Interview Flaws

In a survey we conducted of over 5,000 developers, 75% agreed that technical interviews are broken. Their main complaint? The disconnect between the competencies tested in a technical interview and the actual skills a developer needs to perform the job at hand.

There are two key features of typical technical interviews that create a disconnect with the actual work of software developers.

First, tasks in whiteboard-style interviews tend to focus on algorithms, data structures, and other topics taught in university computer science courses. A frequent criticism by applicants is that technical interview questions like these seem to have little relevance to the jobs for which they are applying. 

Second, most technical interviews are intimidating and stressful. They require applicants to simultaneously understand and solve a problem, write code on a whiteboard, and describe their approach and logic to an interviewer. As a result, the quality of an applicant’s performance may be more of a reflection of their ability to withstand stress and anxiety than their ability to code, communicate, and work effectively with others. And the type of stress tolerance needed to succeed in the interview may not actually be the type of resilience required on the job. 

How Technical Interviews Affect Women and Underrepresented Groups

The dynamics of technical interviews that we covered don’t just undermine the ability of companies to evaluate desired skills. They can also undermine efforts to create a more diverse technical workforce. Women and traditionally underrepresented groups may be more disadvantaged by the shortcomings of technical interviews described above. This can have the effect of disproportionately eliminating women and minority group members from your pipeline after the technical interview.

A recent study by researchers at North Carolina State University and Microsoft demonstrated just how derailing “think aloud” technical interviews can be. They randomly assigned computer science students to one of two conditions. In the “public” condition, students were asked to “think aloud” while solving a coding problem (using a whiteboard) in the presence of an interviewer. In the “private” condition, students were asked to complete the same challenge, but the interviewer left the room after introducing the problem, allowing students to solve it on the whiteboard in private. 

The study provided a powerful demonstration of the impact of the requirement to “think aloud” while solving a challenging problem:

• Students in the public, “think-aloud”condition were more nervous and experienced greater difficulty when solving the problem. They reported difficulty concentrating as well as “thinking and writing” on the whiteboard at the same time. They earned significantly lower scores on their solutions, and 61% of them failed the task.
• In the private condition, students reported less stress and generally felt comfortable solving the challenge. Only 36% failed to successfully complete the task
• In the public setting, no women successfully solved the problem, while in the private setting all women solved it successfully.

This study demonstrates that the current interview format that dominates the tech industry privileges candidates with a traditional skill set and education while doing a disservice to developers from underrepresented groups. 

If we take a step back, we can see that technical interviews have a compounding effect on the challenges women, people of color, and other minority groups face:

• Women and people of color are much more likely to suffer from imposter syndrome, making technical whiteboard interviews even more intimidating and stressful, and negatively impacting their performance.
• Technical interviews that emphasize concepts taught in universities tend to create hiring processes that favor candidates who learned computer science in an academic setting – a setting in which women and other minority groups are underrepresented.
• Although bootcamps and coding academies are a key source of diverse talent, they are much less likely to spend time on academic topics like algorithms and data structures. Most of the instructional time is spent on learning how to do the work. Moreover, bootcamps are much less likely to provide students with opportunities to practice technical interviews, which has been shown to be a major contributor to interview success. Both of these dynamics put diverse applicants at a further disadvantage in technical interviews.

How to Make Technical Interviews More Inclusive

Creating a more diverse, equitable, and inclusive tech industry is a long-term project. One that will require the time and talents of countless developers, managers, recruiters, and HR professionals. 

However, that change has to start somewhere. Often, the easiest first step is to build a more equitable and inclusive technical interview process. Here are 14 changes that employers can make today to begin or continue their DEI journeys.

• Focus on problems that reflect tasks candidate’s will have to perform on the job and are similar in terms of difficulty, required skills, and time constraints. 
• Provide a detailed description of the interview process to applicants.
• Give an applicant the opportunity to work on a problem or complete a project before being joined by an interviewer to discuss their approach to the problem.
• Promote a collaborative, problem-solving discussion about a question rather than requiring the applicant to solve the question while thinking aloud. 
• Provide a warm-up exercise that gives the applicant a chance to become familiar with the format, setting, and tools available.
• Allow an applicant to “reset” or start over with a new problem if they are flustered.
• Provide an applicant with an initial shell or outline of a solution to the problem.
• Consider interview formats such as a code review of a structured take-home assignment or a discussion of code the applicant created previously.
• Train interviewers to conduct more inclusive interviews.
• Use standardized problems from a question library.
• Use a standardized rating scale.
• Identify atypical interviewers and provide training to recalibrate them and reinforce interviewer expectations.
• Set a collaborative, two-way tone by encouraging applicants to ask questions or for help. Knowing when to ask for help is actually a good characteristic of an employee.
• Give candidates the benefit of the doubt by conducting multiple interviews.

Some of the ideas we’ve listed above are quick adjustments that any team can make. Other changes, however, will require the building of new hiring processes with modern interviewing best practices and technologies. 

Fortunately, it’s never been easier to access the tools required to do so. Virtual interviewing platforms. Collaborative IDEs. Objective assessments. Question libraries. The evolution of the technical interview is already transforming the tech industry for the better.