Hiring in a Time of AI

For those in the same position as me, you will probably recognize the pattern:

Create a hiring notice -> get flooded by applicants -> filter for the qualified applicants -> send out a small coding test -> filter again -> interview with a leetcode test -> potential hire.

But the pattern itself has changed.

Use of AI is also a double-edged sword, as hiring managers use AI to filter resumes, the applicants are forced to create resumes using AI. This spills over into the hiring process itself, where both sides are actively using AI during tests and interviews. Comparing hiring before and after the AI boom, there is an increased challenge when assessing candidates.

How AI Changed Each Stage

To get closer to a solution, we need to first assess the changes in the landscape and the problem they cause.

Applications

There has been a massive increase in what I call “spam” applications. With this I mean that a bot, most likely powered by AI, is sending out applications in bulk where the applicant is hoping to beat the numbers game without having evaluated if their profile actually fits the position and if they are even interested. The resumes in these applications are usually filled with invisible keywords that are optimized for a resume checker also powered by AI. This only results in inboxes being flooded, and more time needed to shovel through the applications.

Coding Tests

Take-home tests have also experienced change. These days anyone can copy the instructions into an AI, and have it solve the test while the applicant is making coffee. Yes there are workarounds such as including invisible instructions, but these tests are widely no longer indicative of a candidate’s skillset.

Interviews

With certain AIs now able to create an avatar based on the applicant’s voice and face, it is increasingly difficult to know if you are talking with a person or an AI. Add to this that many candidates use tools that listen to the interview and give suggestions and answers in real time to the candidate, getting an understanding of the candidate’s real knowledge and experience is a skill in itself.

The Core Issue

To summarize, all these changes have caused an increase in false positives, and good candidates are being drowned out by the sheer number of applicants.

A Better Approach

Now let’s focus on the solution.

The coding test

Although the take home test is mostly broken, we can build upon its traditional structure. Ask the candidate to show you a project they have created themselves to solve an issue they had.

Key focus points:

• what was their problem, and why did they choose to solve it like this
• why did they make the design/tech decisions they did (here you can dig deep to get an understanding of their way of thinking)
• name things they learned during the process
• if they could start over, what would they do differently

In an interview I like to spend 15-20 minutes on the above. Remember that the candidate is also evaluating you to decide if this is the place for them.

Assessing experience

Asking questions that allow them to express their understanding rather than memorized or generated answers are key.

You can ask questions such as these:

• what was a problem you had where you could not find a solution, and what was your process
• tell me about a time you used AI to solve a problem, but the answers it would give proved to be wrong, what did you do to solve it?
• whether you use AI or not, walk me through your steps when working on a task

These questions are open-ended, and highlight the way a candidate uses their tools and how they handle the challenges that come with.

Using pair programming

Another approach is to do live coding with the candidate. I like to ask the applicant to share their screen, and give them a task. It does not have to be something very challenging, but something that we can discuss. I let the candidate use whatever tool they are comfortable with including searching the web, but tell them that they should avoid asking the AI directly for the solution, but that they can use it to ask questions or ask for documentation.

During this session, I ask the candidate to speak out their thought process, and ask about why they are doing things in a certain way. For pair programming it is important to keep the conversation going.

Adaptability with the unknown

This one is more demanding, and I generally only use it when I am really unsure if the candidate’s skills are really their own. I ask them to create a simple hello world or echo CLI tool in a language they have no experience with. Zig or C are usually great for this.

Again, asking the candidate to share their screen and to walk you through their process, will give a sign of learning vs simply using the information. Have them reason about what they read of documentation, and have them ask you questions as much as possible.

Conclusion

In an age where solutions are easy to come by, the new skill to focus on, is how the candidate approaches the issue and their way of problem solving. No matter how advanced the tools someone uses, it will always be lacking if not used right. Skills can be learned, but the curiosity for understanding is different from person to person.

I hope this post has given you some helpful tips on how to hire in a world of AI. Good luck interviewing your next candidate!