Having been involved in few coding tests in my career, I have come to conclusion that it has been mostly a waste of time.
Warning, this will be a ranty post!
Also FYI, I have been on both ends, as someone being tested and as someone who reviews code submission.
Andy took a coding test one day
Let me tell you a story.
Once upon time, there was a developer, let’s call him Andy, approached by an in-house recruiter for company A. 99% of the time, Andy would ignore a cold call approach like that, however at that time Andy felt he could do with a new situation.
Furthermore Andy has a friend working in that company plus company A also has good reputation, so Andy thought why not? The recruiter just asked for an informal chat with the manager anyway.
And so Andy and the manager met, the meeting went well, Andy can see the fit of working in company A under this manager. The manager asked the developer if he would like to apply for a role in the company, Andy said yes.
The next step was a take home coding test. The test was not that hard, the Andy thought to himself. Andy asked what programming language should he code the solution in.
Manager said whatever language that Andy is most comfortable with, but given that the manager was a big proponent of Golang language, he said why not try do it in Golang.
Andy is a Ruby developer by trade, but he’s always up for learning new things - he has heard of this Golang the new hotness, so he is keen to give it go. He thought surely he can do the challenge in Golang that would make a good impression. Surely company A would reward his ability to pick up a new language quickly (so he thought).
Andy spent a week frantically reading and learning all he can about Golang and wrote the solution in Golang.
Just to be doubly impressive, he also done it in Ruby and submitted the solution in both Ruby and Golang.
Hours spent for the coding test: 15.
And so the code got “reviewed” and he was asked to come in for a technical interview. When he arrived, the interviewers asked: should we look at your Golang or Ruby code? Which one you are more confident in, they asked. That question as well as other subtle hints, gave an impression that they actually haven’t reviewed Andy’s code much.
The interviewer decided to look the Ruby solution given that he was a Ruby dev. The interviewee didn’t mean to dismiss his effort in submitting 2 solutions - but clearly in the interviewee’s eyes, it has not given Andy advantage that he had hoped for.
Andy was crushed, all the time he spent learning Golang was, in a way, wasted for this application.
Andy didn’t get the job in the end.
Andy’s story is totally not related to my experience at all - pure fiction :D
My problems with coding tests
Anyway let’s talk about my problems with coding test.
Let me firstly define what I mean by coding tests that I have objection with. It is the take home / homework coding test, test that will take more than one hour to do.
Don’t spend too much time on the test
The things that companies always say is, candidate should not spend more that x hours to complete this task.
OK, fair - but candidates will spend more time than that. It is human nature when someone is not in the position of power to compensate for it.
Furthermore, some candidates might apply for more than one position - if these positions require take home test too, then it really puts a lot of pressure on the candidate. Assuming most of these candidates are still holding their current job, then they would do these tests after work and on the weekends - at the cost of their personal and family time.
The benefit is highly skewed towards the company not the candidate and this doesn’t sit well with me.
Unpaid homework
Related to the point above, coding test is an upfront investment in time from candidate that brings little return to the candidates (unless they got the position).
I don’t have the numbers to back it up, but I suspect the majority of coding tests are just unpaid homework. This article from qz, articulates this point: Job interviews for programmers now often come with days of unpaid homework
Here is a quote from the article:
Developers all over the U.S. had encountered the practice, spending anywhere from a few hours to over three days working on their unpaid interview assignments.
So you want to talk to people in the company? Do this coding test first.
Coding test sometimes is used as a gatekeeper, this is one of my pet hate.
I have experience it the first hand, when a company asked me to do coding test first before I can talk to other developers or the manager.
I told the recruiter, I don’t even know if I can have an indication of a fit with the people and the company and you already asked me for an upfront expensive investment?
Thankfully, she relented and allowed me to talk to the manager and the the tech lead before doing the test.
I hope this is not the practice in your company, talk to the candidates first please.
You don’t know what’s important for the company / reviewers
I am involved in reviewing code submissions along with other engineers and I’ve seen candidates penalised for the following:
- Not dockerising the component (the coding test mentioned nothing about Docker).
- Using the wrong database.
Coding is just a percentage that a developer do
Seriously, it might be counterintuitive to you - but good developer or engineer aren’t just a code monkey.
And this is even more true for senior engineer.
Some of the things that a (senior) engineer might do:
- Looking at metrics / alerts
- Pull request reviews
- Requirement analysis
- Writing documentations
- Leading technical meetings
Lots and lots of communication - written and verbally. A lot of negotations.
Writing on the keyboard is just a percentage of what an engineer do, so it should not in my opinion be the only way to assess a candidate.
I would argue communication skills, attitude should carry equal an equal weight when assessing a candidate.
OK, I see your point, but what can else can we do?
Paying for the candidate’s time for doing the take home project
For senior role, I think this is especially appropriate.
It shows commitment both from the company and the candidate.
You’d probably need to have some strong indication that both parties are interested in each other though - expensive exercise otherwise.
Candidate picks the code / project to discuss
This is how my current employer does tech interview and this is how I got the job. I must say it was one of the best interviews I have ever had.
I went one step forward, instead of just showing them the code, I shown them pull requests, this way they can see how I give and receive feedbacks.
Re-using candidate’s take home projects for other applications
This is a variation of the previous point.
Chances are the candidate have applied to other companies that require take home projects, assess the candidate on those projects instead.
Ask candidate to contribute to an Open Source project
I admit this is probably not realistic, but hear my out.
Instead of spending 4 hours on the company’s made-up test project, that is most of the time will be a throw away, why not spend that time on an open source project?
This benefit the users of the project as well as the canditate herself (as she can showcase this another potential employees).
A good project to contribute to: Exercism
TL;DR - message to the candidates
If you think take home test is a time waster, just be honest and tell the companies / recruiters that you won’t do it.
But offer an alternative, see some of the suggestions above.