“A surprisingly large fraction of applicants, even those with masters’ degrees and PhDs in computer science, fail during interviews when asked to carry out basic programming tasks. “
When I first heard this years ago I initially thought it had to be inaccurate. I had seen a large number of developers flunk interview coding tests and write really bad code, including some of this. But how could that many career developers out there pass for great programmers, yet when it comes down to actually producing code couldn’t do it well.
In my previous company I saw developers there using FizzBuzz with candidates during the interview process and I asked myself why are we testing for this? Who could not pass it?
“Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.”
And yet a large percentage of people couldn’t do it. It’s not like they can’t code their way through very complex algorithms and abstract theoretical problems. These are not Google-style computer science interview problems. They can’t program something as simple as FizzBuzz.
Enlightenment through Real Data
We recently started investigating using an online coding test system from a company. It uses automated tests which run automated assessment tests verifying correctness, completeness, scalability and code complexity. This company is able to capture metrics on results and pass rates from a much wider audience of developers across different companies. And this has given us some enlightening information.
Here are just two tests with metrics on the number of developers who took it and pass rates for the test.
Test #1 – Count Div
“You have a,b and k integer numbers, count all integers which are between a and b and they are commensurable with k.”
This seems relatively straightforward and here’s a working solution in Java:
It was taken by over 5,000 developers. Over half flunked it. 8% of developers get to a complete working solution. That’s 8 out every hundred.
Test #2 – Find the Equilibrium Index
“Find an index in an array such that its prefix sum equals its suffix sum.”
It’s a relatively well known problem which basically says in a sequence A:
A0 = − 7
A1 = 1
A2 = 5
A3 = 2
A4 = − 4
A5 = 3
A6 = 0
3 is an equilibrium index, because:
A0 + A1 + A2 = A4 + A5 + A6
The test was taken by over a quarter of a million of developers. Almost 30% completely flunked it. And only one in ten got to a completely working solution.
What is going on?
There are dozens of reasons why we have this problem, other than the very obvious that a large number of people somehow become programmers who probably shouldn’t.
As Jeff Atwood, the author of the original link said, “this is the equivalent of attempting to hire a truck driver and finding out that 90 percent of the job applicants can’t find the gas pedal or the gear shift.”
There is an unfortunate perspective in this industry that asking a developer to take a test makes them uncomfortable, puts them under pressure and questions their abilities as a engineer. But in reality it’s just checking you can literally do the job.
Like most technology companies we need developers who can program and write code that works. We have a great team of engineers who pair program, code review each others work and enjoy bouncing ideas and problems off each other during the day. The people we bring on board need to be able to do the same and it’s why we test that programmers can program.