Should you outsource your technical interviews? And should you use us or Karat?

We’ve recently entered the outsourced interview business (which has historically been dominated by the good people at Karat).

The pieces had been there for a while. Meta had used our tooling to conduct their own interviews, several top-tier VCS had used us to help them vet their talent pool, companies routinely use us to help their engineers prepare for an acquisition, and, most importantly, hundreds of thousands of engineers have used us to practice for their own interviews over the last decade.

So, we're finally doing it! Our big differentiator is interviewer background and experience. Our interviewers are senior, staff, and principal engineers from FAANG, FAANG+, and the frontier AI labs. Only the top 5% of interviewers from our mock interview pool qualify to conduct interviews on behalf of customers. (More on how we vet interviewers and the metrics we track to make sure quality stays high is below.)

So, should you outsource interviewing? Here are three things to think about:

• Eng team bandwidth & appetite vs. cost
• Attrition and candidate experience
• Signal quality

And if you do decide to outsource interviews, should you use us or Karat? I’ll talk about that at the end of the post. (If you'd like to use us, please sign up, and we'll be in touch soon!)

Eng team bandwidth & appetite vs. cost

The thinking here is probably familiar. Engineering time is expensive, and conducting interviews isn't a straight one-hour-for-one-hour cost. You have to factor in the switching costs: the prep before, writing feedback after, and the focus time you lose on either side of a context switch. A 60-minute interview eats up 2+ hours of an engineer's day.

Moreover, conducting interviews is a polarizing activity. Some engineers love it, but many hate it, and view it as a necessary disruption between shipping features. Moreover, most employers don’t explicitly reward engineers for conducting interviews or for being great interviewers, so the incentive structure to focus on interviewing rather than building isn’t really there.

So, to figure out if it makes sense to outsource interviews, take how much you pay your average interviewer, and multiply it by 2 to account for switching costs. If outsourced interviews cost less than that (ours do!), you’re already in the black.

Then you have to account for engineers’ on-the-job happiness. This is a bit harder to quantify, but if a non-trivial portion of your engineers hate interviewing, every hour you make them spend on something they resent is an hour of degraded output and a growing retention risk.

And of course, if a portion of your interviewers don’t want to be there, candidate experience will suffer, which brings me to my next point.

Attrition and candidate experience

The biggest concern around outsourcing interviews – and the biggest initial pushback we get from customers – is candidate attrition. If you pawn your candidates off on a rando who doesn’t work at your company, how will that make them feel? And maybe more importantly: are you giving up the chance to use the interview as a selling vehicle, either directly (an engineer on your team sharing what it's actually like to work there) or indirectly (a problem rooted in the real work you do, the kind a strong candidate finds genuinely interesting and remembers a week later)?

Yes, it’s true, an outsourced interviewer will never sell your company as well as an excellent in-house interviewer who’s engaged and wants to be there. And it’s true that, out of the gate, a candidate might not feel great about speaking to a third party. But, we’d argue that these are not the trade-offs you should be making. We don’t recommend replacing your best interviewers or your superforecasters, nor do we recommend outsourcing all of your interviews.

Instead, we recommend freeing up enough interviewer bandwidth so that you can deploy your best interviewers strategically. Make it so they can swoop in at the most impactful moments, with the candidates you want to sell the hardest, without burning them out.

What we can do is give back your team some of their time and free up the people who don’t want to do it.

Obviously, we think we’re good at this, but fortunately, we have some data to share as well.

We’ve been conducting interviews for over a decade, and over time, we came up with two metrics that we track for each of our interviewers: candidate experience and calibration. (More on calibration in the next section). After each interview, our candidates review their interviewer on a series of dimensions, like so:

We've found that an interviewer's history of candidate ratings is highly predictive of what their next candidate will say about them. So we constantly aggregate this data, and only the top-rated interviewers qualify to do outsourced work on behalf of customers.

So, how does that translate to attrition? We’ve been doing outsourced interviews for startups for the last 4 months and have been tracking candidate attrition. On average, across our customers, we’re seeing about 10% drop-off, which is comparable to what they’re seeing in their own processes.

That number drops further when employers reframe the outsourced interview to: even if it doesn't lead to an offer, they're walking away with a realistic mock interview from a highly experienced engineer at FAANG+, plus detailed feedback on what to work on. Candidates routinely pay hundreds of dollars for exactly that.

Signal quality

Even if you’re convinced so far, agree that it makes sense to offload some of your interviews, and aren’t super worried about attrition, you’re probably still wondering how much signal these interviews will get you, as compared to your own interviewers.

I mentioned that for every interviewer, we track two metrics: candidate engagement and calibration. Calibration is very important, and because we have so much historical and redundant data for each candidate, it’s quite meaningful as well.

That's where the second metric comes in: calibration. **For all the candidates that a given interviewer interviews, we compare how they rated their candidates to how those candidates performed later, in real interviews. We also compare how the same candidate was rated by other interviewers, across multiple interviews. We put these data points together to figure out for each interviewer, exactly how strict or lenient they are. Interviewers who are too lenient don’t qualify for this program, nor do interviewers who are too strict (though we have a slight preference for too-strict over too-lenient, probably the same preference you have in your own process). ** Metrics are always abstract, though. For each candidate we interview on your behalf, you get scores, a detailed writeup, and full replay of their interview, so you can always check the receipts.

Once eng leaders have had a chance to sanity-check our calls against what they would have decided themselves, the feedback we hear is consistent: our recommendations are functionally indistinguishable from theirs.

Should you use Karat or interviewing.io?

Karat and (now) interviewing.io both do outsourced interviews. Which one to go with depends on what you’re optimizing for.

Karat is built for throughput and consistency. Organizations that work with Karat hire hundreds or thousands of engineers a year, often with university recruiting cycles, contractor pipelines, and global hubs to staff. When you're operating at that volume, the hard problem isn't whether each interview is brilliant; rather, you need every candidate to get the exact same experience and you need defensible records for legal and compliance reasons. Karat solves that with a large interviewer workforce (many outside the US, mirroring the global hubs of their customers), a fixed question bank, and tight scripts. It's a throughput machine, and it's a good one.

interviewing.io is built for high-stakes (and high bar) hiring. Most of our customers are top-tier startups, where the problem isn't consistency; it’s quality. For our customers, every hire matters disproportionately, and the interview itself is part of how you sell the company.

Our entire product (starting with mock interviews) has always been built around interviewer caliber. Many of the engineers who use us for practice are already at FAANG+ and are senior themselves, so, out of necessity, our interviewers have to be senior, staff, and principal engineers from FAANG, FAANG+, and the frontier labs. And from within that population of interviewers, we only hire the ones who actually like interviewing and have done a lot of it.

As a result, we don't need to train interviewers on a question bank; we hire people who love interviewing and who've done it a lot and let them do what they do best. And because our interviewers have done this a lot, the interviews themselves are bespoke rather than scripted. They have the experience and autonomy to ask follow-ups, dig into how a candidate thinks, and adjust when someone is strong in unexpected ways.

Finally, when a candidate doesn’t pass our interviews, they still walk away with something useful: a realistic mock interview from an engineer at one of the world’s best companies, plus detailed feedback on what to work on. Candidates pay top dollar for mock interviews with FAANG engineers, and with us, in the worst case, they’re getting that for free.

In summary, we're a great Karat alternative for companies who don't need a "one size fits all" solution and value interviewer caliber over other things. Here's a handy table.

Want to use us for some of your interviews? Sign up, and we'll be in touch soon!