Why engineers don’t like takehome assignments – and how companies can fix them

[My experiences with take-homes] drive home the idea that this employer doesn't care if you are a carbon-based life form, as long as code comes out of one or more of your orifices.

Take-home assignments could, in theory, be great for both companies and candidates. What better, fairer way to evaluate someone’s ability to do the work… than to have them do the work?

Unfortunately, in practice, take-homes typically suck for engineers.

We surveyed almost 700 of our users about their experiences with take-homes and interviewed a handful more for deeper insights. We learned a lot—mostly about candidates' poor experiences and negative feelings toward take-homes. They take a lot of time. They don’t respect candidates’ time. Candidates often get no feedback. And candidates are almost never compensated.

The good news? Turns out there are some pretty simple things companies can do to vastly improve their take-home assignments. But before we dive into that…

Why do companies use take-home assignments?

Take-homes vary a ton by role and company in terms of the types of questions, subject matter, length, and intensity. At their simplest, take-homes can be the same questions as in an algorithmic interview, except done asynchronously. The other extreme is asking candidates to build an entire app and deploy it.

We were surprised to see how often companies use take-homes. About 85% of our users got one at some point in their career, independent of their experience level. Of the users who encountered them, they tended to see them as part of the process about 20% of the time, again, independent of their experience level¹.

Why are take-homes relatively popular among employers? They mostly use them to save time in the hiring process. There are, however, some more noble reasons a company might use a take-home assignment:

• Get better signal during the interview process, as a take-home can be more indicative of actual work
• Get a candidate's best work out of them in a lower-stress environment than a live algorithmic interview and/or attract candidates who don’t like algorithmic interviews (of which there are plenty)
• Broaden the candidate pool by offering a way in for candidates from non-traditional backgrounds—in lieu of a resume screen, which they’d likely fail, candidates can do choose to do an assignment

Accordingly, here’s the relevant part of a great conversation between Vincent Woo of CoderPad and Patrick McKenzie (known to Hacker News folks as patio11) of Stripe, formerly of Starfighter.

Vincent: What general sort of high level change do you think that recruiters at tech companies that are roughly Stripe’s size or bigger ought to make?

Patrick: If I could wave a magic wand and sell the world on one concept, it would be selling the world on the desirability of work sample testing… where the way to tell if someone is good at something is to have them do the something.

Despite enthusiasm for the theory of take-homes and some very well-intentioned reasons, candidates overwhelmingly don’t like take-homes. Here’s why.

Why don’t candidates like take-homes? It’s about value asymmetry.

Though users expressed a lot of frustration with take-homes, we were surprised to see very few take a hard-line stance and refuse to do them. Only 6% outright refuse, and 66% of people complete take-homes all or most of the time. Surprisingly, these stats didn’t really change when we looked only at senior engineers. I was expecting that experienced engineers would do them almost never, if at all, but that’s not what the data shows. It’s possible that seniors are just louder in their disapproval.

Nevertheless, the more desirable a company, the more likely candidates will do the take-home and feel OK about it—70% told us they completed them because they “Really wanted to work at the company and were willing to do what it took.”

I found Weedmaps [to be] a very interesting company. They were the first marijuana related company to IPO. So you'd be on the frontier working for them. I found that exciting. So I applied and they had [a] take-home. I was like, sure, of course I’ll do this.

Other reasons our users gave for completion included: “Because the take-home would be discussed at the onsite” (38%) and “Interesting/cool assignment” (37%). However, many of those who did finish them had such a poor experience that they said they’d never apply to certain companies ever again. We’ll talk about what makes the experience poor in a little bit.

Of the people who refused to do at least one take-home at some point in their career, here were their reasons.

The common thread among all of these reasons is value asymmetry. The worst take-homes feel unrewarding to candidates. Even exploitative. Take-home assignments ask a lot of candidates: a significant investment of their time, with an often unclear scope, no guarantee of progressing to the next round, often without feedback, and almost always without compensation. Meanwhile, the company has basically invested nothing, except to send the task. We heard this a lot.

When I'm interviewing, I look for things that are proxies for valuing team members… or not. If they want me to do a take-home test, and they haven't even spent 30 minutes on a phone screen, I begin to sense an asymmetry in our relationship, with their time and resources being very valuable, and mine not being valuable at all.

A divergence between how much effort they want me to put in, and how much they want to put in themselves. It signals that they are more worried about their time than mine, their costs than mine. It also means they underestimate how much effort it takes to write code, so that if I go to work for them, I am likely to face demands to work uncompensated overtime to meet their optimistic estimates.

Spending five hours on their one-hour test for nothing leaves hard feelings. And if they ghost me after the test, I will happily tell every developer who asks what cheapskates they are.

Is there any relationship between who candidates are and their willingness to do take-homes?

Perhaps surprisingly, our data says no. We ran a regression to compare our survey respondents’ interview performance on our platform to how likely they were to do take-homes. The relationship was so weak as to be negligible.

Similarly, we ran a regression to see if people who look good on paper are more or less likely to do take-homes. The relationship there was negligible as well.

In other words, contrary to some popular opinions, you’re not necessarily weeding out your best candidates by doing take-homes, whether you define “best” in terms of how their resume looks or how they perform in interviews.

How companies can make take-homes better (and why they probably should)

After reading the stats above, you might think that, despite their grumblings, candidates generally do take-homes, and the best candidates won’t be weeded out. So, if you’re one of the many companies that uses them, it may not make sense to invest your limited time into making them better. It’s not that simple. For some companies, like the FAANGs, who have extremely strong brands and are known to pay well, changes are probably not worth it, especially in this market, where junior and senior engineers are willing to jump through more hoops than ever before.

If you’re not a FAANG, though, listen up. Here are some questions you can ask yourself to figure out if you should indeed make some changes to your take-homes.

First, take an honest look in the mirror and ask yourself about your brand strength. Are you a household name? Does having you on candidates’ resumes give them automatic prestige? Are you known to pay above market? If the answer to all of these questions isn’t a resounding yes, your brand strength is probably not strong enough to make people jump through hoops.

Here’s a sketch to drive that point home. Unless you have a ton of brand strength, candidates’ willingness to jump through hoops drops off sharply.

If you aren’t maxing out on brand strength, there are two questions you should ask yourself:

1. What is my take-home completion rate?
2. What is my offer acceptance rate?

If your completion rate is below, say, 60% (our data shows that candidates complete take-homes around 62% of the time), then it’s DEFINITELY time to make a change. Honestly, regardless of what our data says, if people aren’t completing them at least 85% of the time, it’s probably time to make a change—losing more than 15% of your candidates to attrition at any given stage in the funnel is bad.

What about your offer acceptance rate? If it’s less than 50% and you’re using a take-home already, there’s an opportunity to make some improvements. You might be wondering what this has to do with take-homes in the first place. To answer that, let’s change how we think about different parts of the interview process. At face value, every part of the process is there to vet candidates, to determine if they’re the right fit for your organization. However, when used correctly, every part of your process should become a selling vehicle as well. This is especially important for companies who do not have a strong, well-known brand. The FAANGs can get away with using their interview processes primarily as vetting exercises because candidates are already sold on the pay or prestige or sometimes on the work and the product. When you don’t have an established brand, the candidates who come to you are, at best, open to learning more, and the interview process is the instrument that teaches them. Wield it accordingly.

Though we strongly advocate coming up with great, unique interview questions² and making sure you have great interviewers, if your process does have a take-home component, it is one of the more overlooked parts of the process when it comes to selling. You have the opportunity to have someone do the actual work that you do! This is your chance to pick the coolest stuff you’ve worked on and serve it up to someone on a platter and make it stick in their brains and make them imagine what it’d be like to work on these kinds of problems every day! Why wouldn’t you jump at this opportunity?

You may think you don’t need to sell in this market. But just because employers have all the power right now, it doesn’t mean that will always be the case. And great senior engineers still have a lot of leverage.

If, after considering your take-home completion rate and your offer acceptance rate, it looks like you do need to make some changes, here are some practical tips, based on what we’ve learned from talking to our users (overwhelmingly senior engineers who are targeting top-tier companies—probably the people you want). Let’s start with what we just talked about: using the take-home as a selling vehicle.

Make them interesting and relevant to the actual work

You're getting a chance to spend a couple of hours with somebody in a take-home (metaphorically). Why wouldn't you do everything you can to get them excited? Pick a problem that you've worked on, and get them hooked on it. Pick the kernel of an interesting problem that you've solved, and build something around it that will challenge candidates. Something that gets them thinking, “I could have done that better” or “This is a different or more efficient way to do it.” That's going to be more effective than the standard perks many companies offer.

[Best take-home I’ve seen was an] open-ended system design question on the type of system I would work on, was meant to simulate a team discussion on the system we needed to build, and was a great way for me to start thinking about what I'd be working on there.

One way you could do this, is to have your engineering team keep a shared doc of ‘cool’ solutions they've found, or new things they've tried. These can serve as jumping-off points for creating your take-homes.

The… challenge was for an internal tooling team that specialized in incident response tools; their challenge was to create a scaled down version of a tool already in use at the company. The focus was more on understanding the domain and customer than wiring up a bunch of complicated stuff, and it was a delight. It being a greenfield also gave you an opportunity to showcase some software design skills. I did not get this job, but enjoyed the experience and still feel connected to the team.

It was conceptually related to the sort of work the team was performing, but it was simplified and stand-alone enough to clearly not be unpaid labor for their product.

Just be sure that when you come up with a practical problem that you strip out the annoying parts and focus only on the juicy kernel of the problem, the part that’s actually cool and lets the candidate be smart and creative. Don’t make them do grunt work or wrestle with their dev environment!

[This] was for a tooling team in Support, where they didn't have a lot of experience creating challenges or interviewing. One of the engineers took a difficult task that he had accomplished recently and just made that the take home challenge. It involved a lot of Ruby version conflict debugging. It was completely demoralizing and felt like hazing.

Keep them short

Candidates overwhelmingly favor take-homes that respect their time, i.e., short ones.

The best were short and brief, took no more than 2 hours and were directly related to what I would be doing on the job.

Best are realistic and time bound, i.e., low time investment required.

Short and quick take-homes are great.

Over 80% of survey respondents said that take-homes should take 4 hours or less, and a plurality thought that they should take 2 hours.

Take-homes being short in theory is one thing… but we got a lot of feedback that take-homes often go far beyond how long companies tell candidates that take-homes should take.

Have someone on your team QA the actual length of the assignment

This disconnect between actual time and expected time is another instance of value asymmetry: companies didn't even bother to have someone on their team do the take-home themselves and time it.

I've done a couple of algorithm questions as take-homes that the interviewer said should take an hour. They routinely take longer than that.

Companies will say, ‘don’t spend more than X hours on it,’ but then it actually takes 5X that amount. It's just too much time. And most of the time they didn’t give feedback at all even though I spent so much time on it.

Companies should clearly communicate the expected time commitment for a take-home. And they should be realistic about how much time it should take.

Best was an interesting problem with a realistic time frame. They estimated 2 hours and it took me exactly that, which was refreshing and made the process seem fair.

I've done a couple of exercises where they were truly respectful of my time. The tasks were small, clearly defined, and they gave me a realistic timeframe to complete them.

For the best take-home I did, the company told me: ‘Tell us when you're going to start, and you'll have 4 hours. We'll be available over email that whole time, and we'll answer your questions within minutes.’ That felt respectful of my time, and was a more reliable signal of my capabilities than a 20-minute algorithms exercise.

To get a realistic estimate, companies could simply QA their own take-homes—just by having someone on your team take it, and time them.

Still, some candidates will spend more time anyway, because they really want to work for a certain company. And it can be easy for candidates to go down a rabbit hole and get most of a take-home done in the first few hours, then spend another day or more, perfecting it to give themselves a better chance.

Do you want me to literally only work two hours on something that's clearly going to take 10 to really be able to have a conversation with you about how I think about things? Because two hours of decisions isn't as good as 10 hours of decisions.

It’s okay if candidates want to put in extra time, but it should be made clear that that's not the expectation.

Set a clear scope, related to the role

The scope was poorly defined, which made the assignment not only tedious, but also seemed like a waste of time.

Having clear, straightforward instructions can significantly enhance the take-home experience for candidates, making them feel purposeful and relevant. The best take-homes are those that directly align with the actual job. When candidates are asked to solve a simplified version of a real problem your company is facing, it not only tests relevant skills, but also gives a candidate insight into what work at your company would be like, which is really what selling is. (This is, of course, assuming that you believe what you’re working on is cool… If you don’t, how will you ever convince others?)

One was a great learning experience because I had to solve a problem similar to what I would face in the job. It was concise and focused.

Being able to ask questions before or during the take-home, e.g., having a direct contact point who’s an engineer at the company, can also be a big plus.

Be thoughtful about where in your process you insert the take-home, and give candidates a choice between a take-home and something else

For take-homes early in the process, before speaking to a human

We expect that many of the companies reading this piece have take-homes as the first step in their process, before ever talking to a human. This decision isn’t very popular.

If a company calls me for a screening interview, it means they've read my resume and cover letter, and not immediately dismissed my application as inappropriate. It also means they're putting a person on a phone call with me for the duration of the screening interview. This is symmetric, my effort matches theirs.

[I] did a take home that was automatically sent to me after applying, before even speaking to a human. The feedback was 'you are too expensive,' no feedback on the coding.

Company needs to build an investment with the candidate first—they shouldn't ask for it before meeting the candidate at least once.

That said, take-homes can be a great way to let candidates who don’t look good on paper show what they can do, and those candidates are more likely to complete them. To prevent candidate unhappiness/attrition, give them a choice. Either submit a resume or do the take-home assignment or both. If you go this route, though, design a take-home that you trust. We’ve seen companies take this approach and then throw out perfect-scoring take-homes when they didn’t like the resume. If you use a take-home, then respect your candidates enough to follow up with the ones who’ve done well, even if they don’t look good on paper.

For take-homes later in the process

Another way to make take-homes purposeful is to give your candidates an explicit choice about whether they’d rather do a technical interview or a take-home. Engineers are, in fact, split on which they’d prefer. Giving them a choice allows them to showcase their skills in the format they prefer and feel best prepared for. If you have good questions, you should be able to get good signal from either.

Only 10% of respondents told us that they were given the choice of take-home versus technical interview. So there is an opportunity for more companies to do this. It's a candidate-friendly gesture that shows empathy and can help candidates shine.

Many candidates have spent months preparing for standard technical interviews. So it can feel anticlimactic when they find out that a company they're excited about doesn't do them. Others get so nervous in a live interview that they don’t perform.

I rarely don't pass take-home assessments, but I often fail to pass live interviews.

Interestingly, one user we interviewed told us that they spend far less time on a take-home compared to preparing for a live technical interview. And because of that they prefer take-homes.

I prefer take-homes over all other assessments because I find I spend far less time on take-homes then I do preparing for live interviews. It's hard to overestimate the amount of extra time I spend preparing for a technical interview, compared to doing a take-home—for me it’s maybe 10 times as much. I've spent hundreds of hours, maybe 500 hours, over the course of my career preparing for technical interviews. And if I have one pop up, I can't just drop everything and do it right away. I have to spend a lot of extra time just ramping up for a particular interview, in addition to the hundreds of hours that I've done.

Give candidates a good, rational reason to do the take-home

Clear communication about the purpose of a take-home in the hiring process, as well as why it’s rational to spend time on it, is important for candidates. It’s a way to make sure it feels purposeful and not like a random task.

One way to make the take-home feel deliberate is to replace some parts of your process with it. A standard process without a take-home has a recruiter call followed by a technical phone screen followed by an onsite (virtual or otherwise). The technical phone screen usually lasts about an hour. The onsite usually lasts 6 hours.

Let’s say your take-home takes 2 hours to do. You can make it replace the phone screen and one of the onsite rounds, which nets out to the same number of hours spent. If you go this route, we recommend doing the math explicitly for candidates and showing them that the time they spend on the take-home is equivalent to the time they’d be spending on a process without it.

Another way to make the take-home feel deliberate is to incorporate it into the onsite, where at least one of the rounds, if not more, will include a code review and/or thoughtful discussion about tradeoffs and choices made. This should be standard practice, but isn’t always. 32% of our users said companies had told them this, and it was the reason they decided to do a take-home.

The best take-homes were ones that we discussed in the first rounds of interviews. The worst ones were ones that I submitted and we never talked about them again.

Ideally, you do both of these things together, and very clearly explain to candidates both the math and how the take-home informs the content of the onsite.

Compensate candidates

Probably the most striking result of our survey was that 58% of candidates think that they deserve compensation for completing take-homes. Yet only 4% reported ever receiving it. Compensation can shift candidate perceptions of the hiring process and of the company:

They compensated me for my time, which made the process feel very professional and respectful.

It was a completely open source codebase and so their process was the exact same as someone that was an employee: here's the ticket with the information to do it, set up the environment, download all the code, get everything running. They gave me a few different tasks I could choose from, I could pick two, and if I completed them I would be compensated a fixed rate, which was $100 for each task. Which in terms of the time I spent on it, is still really cheap for them.

If candidates know they’re going to be paid for their work on a take-home, they’ll be more likely to complete it as well. Compensating candidates is a clear gesture that shows you value their time and effort, that there’s more symmetry in the relationship. It also goes hand-in-hand with time: paying also forces a company to scope the take-home to a reasonable number of hours. So it's a forcing function for good behavior—if you can’t afford much, then don't make the take-home too long!

How much to pay?

Responses about how much companies should pay for take-home assignments were split.

1. Fixed amounts: Just over half of respondents (52%) suggested specific and reasonable fixed amounts, ranging from $50 to $500.

2. Hourly rates: The other almost half (47%) favored an hourly rate, with suggestions ranging from $50 an hour and upward (average of $217 an hour). Some suggested that the rate should correspond to the salary of the position being applied for, or be comparable to what an employee at that level and company would earn. As one user put it:

   Maybe just pay market?

   One antipattern when it comes to comp is NOT having a set rate, asking the candidate to name their price, and thereby putting the candidate in a position where they feel like they need to negotiate. In this scenario, the candidate has to negotiate twice: once on the take-home and once on their actual comp, with the worry that negotiating too aggressively on the take-home might count them out… or not aggressively enough anchoring the company to lower compensation down the line. No one needs these mind-games in an already stressful process. Just have a set rate, for god’s sake.

3. Symbolic compensation: This is probably not the best option, but a handful of survey respondents (1%) did mention that a minimal symbolic compensation would do. While 1% is small, we found this interesting to include because, when we interviewed people, two of them mentioned this.

   I think any compensation at all has symbolic value. A $100 Amazon card would impress me. A $50 Amazon card and a company t-shirt would at least not insult me. It’s kind of a consolation prize to say, ‘No hard feelings.’

   I feel like a couple meal vouchers would do it these days.

Of these options, we’d recommend a reasonable fixed amount based on the task, and the actual time it’s supposed to take. And of course you know how long it takes because you had one of your engineers do it themselves, right? Right??

Give feedback

Lack of feedback was the primary reason our survey respondents said their experience with a given company was bad. Regardless of interview type, we’re always pro feedback, but feedback is especially important for take-homes, because in a way they ask more of a candidate. Offering constructive feedback, regardless of the hiring decision, respects the candidate's effort on the take-home.

Despite the time and effort they invested in completing take-homes, many of our survey respondents said they received no feedback at all. This was seen as demoralizing, and it deterred candidates from applying to future opportunities at those companies.

Getting rejected without having a chance to discuss the code with anyone is a terrible experience.

It is really discouraging spending a large amount of time to find out you are rejected without explanations.

They provided no feedback after submission, which made the whole effort feel unappreciated and one-sided.

Incidentally, the main reason companies don’t give feedback is fear of getting sued. As it turns out, literally ZERO companies (at least in the US) have ever been sued by an engineer who received constructive post-interview feedback.

Thanks to Dan Fennessy for all the behind-the-scenes work on this post.

Footnotes: