Code tests in a world with AI
Generative AI is changing software engineering hiring. Since generative AI can be used to complete a coding test in record time, we need to update our hiring process and ensure that our approach to coding tests still focuses on evaluating what is important for the people that we want to join our teams. Some folks will try to ban AI in code tests, but I say embrace it. It is changing Software Engineering so our hiring process needs to change as well. The best engineers today are those who collaborate deeply, take on feedback and strengthen the team around them. That’s what our hiring needs to measure for, more than ever has in the past.
I still see the value in a take home code test. The code test should be relevant to your industry or the job they will be doing as much as possible and use a real world example. I avoid putting silly puzzles or tricks in there. I will post a more in detailed post about a good code test design as some point. What is important in the context of hiring today, is that having the candidate spend some time on the code test without having someone look over their shoulder is really important and they should also know the code they wrote quite well. Then there is the second most important part, the interview after they have submitted their solution.
As collaboration and working in a team is what I value, the interview after they submit their code test is so important. In this session have two Software Engineers from the team review the code beforehand and provide some feedback on the submission. This is to emulate a pull request or code review. What I am looking for here is how the candidate takes on the feedback and how they respond.
The second part of the interview is an extension of their existing solution. This part is the critical part in the age of AI. The candidate should know their code submission quite well and be able to work with the two Engineers to add a new feature to the project. They are encouraged to think out loud and explain how they are thinking about solving the problem.
At this point in the interview you get a really good idea in how the candidate thinks about problem solving and communicates this process with others. The key here is we also see if they understand the code they wrote and this can help us see if they used AI and didn’t read the code that it wrote. We can also look at how they approach the solution and consider alternative options. Its not perfect as someone could try to be very covert about it but I think that overall this human approach is a very effective way to do a code test in a world with AI.
Your goal isnt to catch people out but its to find the people that make your team better. When AI can generate code in seconds; curiosity, teamwork, communication and the ability to build on ideas together is what matters. Thats what great Software Engineering is about and the hiring process should reflect that.
