How many golf balls can you fit into a school bus? Think carefully — your answer could determine whether you get a tech job or not.
It’s a common question used by companies hiring engineers, but what if you don’t play golf or have never seen the inside of an American school bus? Then there are the questions about your projects outside work, which need to resonate with the hiring manager while also being unique and industrious. Or the notorious whiteboarding treatment, where candidates must conjure code out of thin air.
While usually well-intended, these common interview challenges often make candidates feel isolated, and they can also introduce bias into the evaluation process. Unless you think really hard about what the experience is like for people with a range of backgrounds, you may alienate talented candidates and reduce diversity on your team.
Four years ago, Chicago-based Braintree decided to rethink its engineering interview strategy to improve its diversity. The recruitment team partnered closely with the diversity and inclusion team to clean up potential biases and create a better evaluation process. They revamped job postings to include gender-neutral language, acknowledged impostor syndrome to encourage more candidates to apply, conducted hiring trainings and nixed brain teaser questions.
It’s not an exact science, but senior tech recruiter Kyle Barbato believes those steps have helped boost the company’s quality of talent and diversity.
“It’s been a jaw-dropping improvement on what we’ve been able to do as far as increasing folks from underrepresented groups and genders,” Barbato said.
Six Inclusive Hiring Practices From Braintree
- Look for unconscious bias at each step of your hiring process.
- Working side by side with the D&I team has helped Braintree’s recruitment team to create a formal interview process that reduces the opportunity for bias to creep in.
- Formalize the interview process by training your team on best practices. Every employee involved in the interview process at Braintree goes through a training program to create a standardized and objective evaluation process.
- Keep the focus on work. Braintree abstains from asking engineers brainteaser questions, and no longer hosts an informal lunch to see if an engineer is a culture fit to ensure they are objectively evaluating the candidate.
- Be conscientious of people’s time commitment. Braintree removed deadlines on its coding exercise and provides flexible on-site times to prevent excluding candidates with pre-existing time commitments, like caregivers and those entering the field from a non-traditional background.
- Submit official feedback first before reviewing a candidate in person. Braintree prevents discussing a candidate until after everyone’s feedback has been submitted on Greenhouse. This prevents people from influencing each other’s opinions.
Laying the groundwork for an inclusive interview process
Before anyone at Braintree can participate in the interview process, they must go through the company’s diversity and inclusion training. Employees receive training on unconscious bias, conscious inclusion and Braintree’s broader diversity and inclusion efforts. They must also shadow certified interviewers to learn which questions to ask, and how Braintree conducts interviews.
“Diversity and inclusion often falls on recruiting,” Barbato said. “So it’s great to have a counterpart that focuses on this every single day and can think about things that we should be thinking about.”
While interviews are far from an exact science, the training helps the recruitment team to standardize the interview process as much as possible to give every candidate a fair chance, Barbato said. They’re also cognizant not to include a panel of all-white males during the interview process.
“Diversity and inclusion often falls on recruiting. So it’s great to have a counterpart that focuses on this every single day and can think about things that we should be thinking about.”
That focus on inclusion extends to job postings, too. Braintree has been careful not to include gender pronouns and masculine language that can trigger biases and prevent a person from applying. The end of every post includes an acknowledgment of the confidence gap and impostor syndrome, along with related reading links for candidates to learn more. The paragraph is designed to reassure candidates and encourage them to apply.
“It’s our way of acknowledging that we know things like that can get in the way of applying,” Barbato said.
Keeping the screening process tech-focused
Once a candidate applies, Braintree kicks off its assessment process with a take-home coding assignment.
At this stage, Braintree’s only goal is to evaluate an engineer on their ability to write high-quality code, Barbato said. They simply want to see whether a candidate knows how to model a solution and can follow programming language conventions.
While other companies may like to ask brain teasers to trick candidates at this stage, Braintree prefers to keep their coding assignment related to the work the person will be asked to do at the company. Asking how many golf balls fits into a school bus isn’t only pointless — it runs the risk of alienating an applicant who doesn’t play golf, Barbato said.
“Silicon Valley's kind of notorious for those brain teasing type questions,” Barbato said. “We don't believe in that approach. We're going to be writing code day to day, so that’s what we focus on testing.”
On average, the test takes anywhere from an hour up to four hours to complete, but Braintree has learned not to put a deadline on it. Not every candidate is able to immediately devote that time to a coding challenge, Barbato said.
“Someone who went through our interview process told us, ‘Hey, one of my friends had a deadline and she was a nursing mother. She didn’t have time to complete the assignment in the time the company gave her,’” Barbato said. “We took that feedback and removed our time limit because we didn’t want that to be a problem for someone applying.
Candidates are also encouraged to use any programming language of their choice. This ensures they present their best work possible, Barbato said.
When it comes to evaluating the assignment, Braintree conducts a blind review of the code. They don’t know who wrote it, what their background is, or their gender. All they care about is the quality of code, Barbato said.
“We're going to be writing code day to day, so that’s what we focus on testing.”
If a candidate passes the code review, the next step is the technical phone call, where anything on the candidate’s resume is fair game. This is where the interview training kicks in. The interviewer is trained to ask every candidate similar questions specific to their work experience. Questions might include: How did you build that product? Why did you make that decision? Which roadblocks did you experience, and how did you work around them?
The goal is to push the candidate to the edge of their knowledge of the tools they worked with and the work they did to evaluate their technical ability. Still, it can be tricky to evaluate a person on the phone, Barbato said.
For example, just because a person isn’t confident in their responses, doesn’t mean they don’t know what they’re talking about. While the recruitment team is always looking to improve its evaluation process, the training helps interviewers catch potential biases and standardize the process, Barbato said.
Removing bias from the on-site experience
Once they pass the coding assignment and technical phone interview, candidates are invited for an on-site interview.
The day usually starts at 9 a.m. and lasts about four-and-a-half hours, Barbato said. The length is designed to give a candidate enough time with each interviewer to answer questions and complete technical tasks. However, the company is flexible both with the start and finish time to adapt to an interviewee’s schedule.
The day kicks off with another technical interview, which will go deeper into the employee’s coding knowledge and work experience. From there, the candidate will break for lunch, where an interviewer will lead them through a core values presentation and ask questions related to Braintree's culture.
The presentation, which has replaced a more informal lunch, is designed to more accurately determine whether a person matches Braintree’s culture, Barbato said. It removed the potential for bias introduced by conversations about shared interests like sports, other hobbies and popular culture — which ultimately measure how similar the candidate is to current members of the team.
“The core values session was our attempt to eliminate those types of questions that don’t offer anything in evaluating a candidate,” Barbato said.
Next the candidate will pair program with another engineer, who is also a Braintree-certified interviewer, to return to the coding assignment they completed at the beginning. While Barbato didn’t want to give away what the problem will entail, he said they will continue to build on the work the candidate established. This allows them to further evaluate their coding ability and process in an atmosphere that suits their strengths.
This phase gives the company another chance to evaluate how the candidate approaches and solves problems. And since Braintree engineers spend about 90 percent of their time pair programming, candidates get a taste of what it’s like to work at the company.
“The core values session was our attempt to eliminate those types of questions that don’t offer anything in evaluating a candidate.”
The on-site wraps up with a product session, where a candidate will collaborate with the interviewer to build a product from scratch. There’s no code involved in the process, but it is an opportunity to see how the candidate approaches a product, what steps they take to build it and how it relates to their past experiences.
“We hope to hit on areas where a candidate has a ton of experience and can talk about in great detail about how they built out a step and how it relates to their work experience,” Barbato said. “Whether it’s on the back end or front end, there’s an opportunity for every candidate to be successful.”
Finally, the candidate meets with the hiring manager, which operates more like a meet-and-greet to end the day, Barbato said.
“It's really, as an end of the day check-in,” Barbato said. “It’s one last opportunity for the candidate to ask questions and for us to give more context on Braintree.”
Eliminating groupthink from candidate discussions
Before any Braintree interviewer can discuss their impressions of the candidate, they must submit their assessments online. This is crucial to creating an unbiased evaluation process, Barbato said.
While sharing opinions informally may seem harmless, those conversations can sway a person’s opinion. Someone who feels strongly about a candidate may influence another person who was on the fence. That can lead to groupthink and misleading results.
“We’ll always be asking ourselves how we can build a more diverse workforce and have the most inclusive workspace.”
Once the feedback is submitted, all of the interviewers will get together and host a roundtable discussion about the candidate. The hiring manager plays the role of facilitator, Barbato said, so that everyone feels empowered to speak. They then go session by session assessing the candidate.
“They monitor the conversations to ensure that they're productive and that we're evaluating the candidate on the actual core criteria that matters in making a hiring decision,” Barbato said.
But while a lot of thought has gone into the candidate interview process, Barbato said the team is always looking for ways to improve. Building a diverse workforce requires constant tinkering and improvement.
“We’ll always be asking ourselves how we can build a more diverse workforce and have the most inclusive workspace,” Barbato said. “It’s not something that can just happen overnight.”
