Written by Emily Goldsack, R&D Consultant
Generative artificial intelligence (GenAI) is a major technological advancement which is disrupting the job market in many ways. Research from Cibyl for the ISE found that half of students were already using AI to help them with completing the selection process. As the first generation to grow up fully immersed in technology, it is crucial to assess how Gen Z candidates are leveraging GenAI and reshaping the job market. At Talogy, we conducted global research to gather insights from hiring managers and early career professionals on the use of GenAI during the hiring process. In this blog, we will explore some of our findings and offer suggestions for how organisations can prepare for the continuing rise of AI in assessment.
What is GenAI and why should we care about it?
GenAI tools such as ChatGPT, DALL-E, and Gemini use machine learning algorithms to generate new text, video, and audio content almost instantly. They have rapidly gained popularity in recent years, and although they have the potential to enhance talent assessment and development, there are also concerns about their impact on the hiring process.
One such worry is their potential to help candidates gain an unfair advantage in the selection process. For instance, 70% of job seekers report a higher response rate from companies when using ChatGPT to help them write their CV or cover letter. Such figures suggest that GenAI tools may already be influencing recruitment dynamics and creating an unlevel playing field for candidates.
Additionally, cheating with AI has the potential to undermine the psychometric validity of assessments, leading to more unsuitable candidates advancing to later stages of the selection process. But what does this mean for employers and how can they strengthen their hiring processes to mitigate the risks associated with GenAI?
Navigating AI and cheating in early talent recruitment
First, it is helpful to gauge current attitudes and behaviours around the use of GenAI. As part of our research, we surveyed 560 hiring managers, 564 early career professionals, and 138 job seekers in 26 countries. We explored their perspectives on GenAI and the extent to which candidates are likely to use these tools when applying for early career roles.
When we asked managers, we found that 65% are somewhat or very concerned about candidates using GenAI to cheat on recruitment assessments. This echoes what we’ve heard in conversations, suggesting that the use of these tools is a pressing concern for employers today. Although, it is worth noting that when we asked managers more broadly about their challenges for entry-level hiring, this issue was not frequently mentioned. They seemed to be more focused on perceived skill gaps, ensuring a good fit, alignment in salary expectations, and the competition for top talent.
Next, we wanted to explore the question “Will early career professionals use these tools to cheat on assessments?”
Of those we surveyed, 58% said they would be unlikely or not at all likely to use GenAI tools when completing recruitment assessments. It appears that across all regions, a notable proportion of the early talent population is hesitant to use these tools. Among job seekers specifically, only 22% reported using GenAI when completing assessments during their job search.
Note: These results are based on the combined sample of early career professionals and job seekers (n = 702).
What’s more, only 15% of early career professionals and job seekers said they would be either very or extremely likely to use it. These figures don’t indicate that cheating is widespread at the moment, although they do suggest that the topic merits ongoing attention and investigation.
This is something we have been doing at Talogy by continually monitoring scores on existing assessments that are used in high-volume hiring. We have found no notable differences in average scores before and after the release of ChatGPT with large samples of candidates, providing evidence that GenAI isn’t having a real impact on assessment scores to date. However, this is something we will continue to monitor and report on as the use of AI grows and evolves into the future.
For further insights, we asked job seekers about their motivations for using GenAI. Our results showed that early career candidates use AI as a tool to improve their ideas and written responses, rather than relying on it as a method to simply cheat by providing all the answers. Many said they were reluctant to use it because they want to show their true selves. They felt using it would be dishonest or could potentially result in them being disqualified from the process. Others said they don’t trust it to be reliable or accurate enough to support them in high-stakes assessment contexts.
Best practices for managing the use of GenAI in assessments
Given that GenAI may unfairly provide advantages to some candidates and reduce psychometric validity, employers must regulate its use within their assessment processes. To understand what steps should be taken, we asked job seekers how likely they would be to use GenAI if it was stated that it was forbidden. In our research, 85% said they would be unlikely or not at all likely to use it, suggesting that candidates are not motivated to cheat, especially when instructed not to.
This shows that small steps – such as clearly stating that the use of AI is forbidden – can effectively deter candidates from cheating. We recommend that organisations make it clear to candidates at the start of the recruitment process about what is and isn’t an appropriate use of GenAI tools within their hiring process. A practical approach to implementing this is to include warning statements or ‘honesty contracts’ where participants agree upfront not to use GenAI during assessments. They can also include the possibility of later re-testing in person to replicate the results and validate their skills and abilities for the role.
This would also address some of the confusion around how the use of GenAI is viewed in the hiring process. Our research found that some candidates think GenAI tools will be allowed (42%), some think they won’t (41%), and others are unsure (17%). Most think it will be allowed for CVs and cover letters but not for ability testing, interviews, and assessment centres. This highlights the importance of organisations communicating clear guidelines to candidates about the use of GenAI at each stage of the process.
Employers can also do the following to effectively evaluate their current processes and mitigate the risks associated with AI:
- Score monitoring – Monitor score trends on your assessments over time to check for evidence of cheating, indicated by any notable differences in scores. Start by focusing on the most high-risk assessments, such as traditional cognitive ability assessments or forced choice Situational Judgement Tests (SJTs). This can also help you keep track of whether your assessments maintain validity. While cheating has always been a concern that will continue into the future, what ultimately matters most is ensuring assessments predict high performance.
- Evaluate assessment formats – Investigate how susceptible different test formats are to cheating with GenAI. Interactive tests or those that involve more complex response formats – as opposed to having a clear right or wrong answer – tend to be more resistant (i.e., ideal point SJTs). Seek to implement these designs where possible. For example, you could replace a traditional cognitive ability test with a more interactive format such as Talogy’s Mindgage assessment which assesses how candidates approach the exercise, rather than just successful completion.
- Technology features – Consider implementing technology features to deter cheating, such as blocking copy and paste functionality, making it more difficult to transfer content into tools like ChatGPT. This can be an effective deterrent, particularly in timed assessments, as it means the use of these tools becomes a hindrance to a candidate’s performance and their ability to complete the assessment. Remote proctoring of online assessments is also an option for the highest security, although the cost to implement this and the potential negative impact on the candidate experience may outweigh the benefits.
Adapting recruitment to the world of GenAI
Given that early career candidates are arguably the most comfortable generation to date when it comes to leveraging technology, it’s no surprise that there are questions about how they will use it as they enter the workforce. While cheating in online assessment processes has always been a concern, this is a new method that requires innovative approaches to manage its impact on assessments.
Our results suggest that widespread cheating is not taking place with GenAI, but there is still a need to safeguard your hiring process from potential risks. At the same time, organisations need to focus holistically on the recruitment process, keeping other important factors in mind such as job and cultural fit, addressing skills gaps, and attracting and retaining top talent to ensure the hiring process maintains its integrity and results in a quality hire.
