Written by our Science team.
You and I can both agree that the biggest unsolved puzzle in recruitment is probably how to hire the best talent out there. So without a doubt, over the past decades, countless hiring tools and methods have been created and used as means of helping companies make better hiring decisions. Methods, such as different types of assessments, job tryouts, resumes, motivation letters, references, and so on, are often combined with interviews to assess the fit of candidates.
However, one question still remains: do they actually work?
A Glimpse into Gamified Assessments
Traditional assessments are tests used to evaluate applicants’ ability, ultimately using those results to predict their future job performance. Some examples are personality tests, cognitive ability tests (CAT), skill-specific tests, situational judgement tests (SJT), and more.
Although cognitive ability tests and personality tests are proven to be strong predictors for future performance (Schmidt & Hunter, 1998; Borghans et al., 2016; Kuncel & Hezlett, 2010), they still have disadvantages that might limit your hiring success.
- Firstly, cognitive ability tests usually produce high cognitive load for test-takers, which can result in high dropout rates during the hiring process.
- Secondly, when using traditional personality tests, candidates can often tend to pick the most socially desirable answer or one that merely reflects how they want to be seen. That can cause an increase in scores, especially in high stake situations, leading to an incurracy for predictive power (Niessen et al., 2017). Even for perfectly honest candidates, a self-assessment may not be very predictive of actual behaviour given research showing how inaccurate self-assessments can be (Dunning et al., 2004; Stajkovic & Luthans, 1998).
- Additionally, traditional assessments normally contain repetitive questions or elements that might provoke boredomness in applicants, leading to an early withdrawal in the hiring process.
Luckily for you, advancements in technology have led to a game-changing method to improve potential negative impacts of traditional assessments, which is gamification. Within various forms of gamification, we can broadly divide them into two: gamified assessment and standalone game-based assessment (Landers et al., 2021; Armstrong et al., 2016).
Simply put, gamified assessment refers to modifying the original form of traditional assessment by adding game elements to increase motivation.
For example, a gamified assessment can be letting an avatar ask the items within a personality test or adding other animation and/or sound effects (Collmus & Landers, 2015). On the other hand, game-based assessments are deliberately-built games that capture data of players’ behaviour, measure desired aspects, and try to make sense out of the gathered information. For instance, the game-based assessment includes a mini-game purposely designed to measure players’ attention to detail.
TLDR: Watch a video about Gamification in Recruitment
3 reasons why you should pick Gamified Assessments over Traditional Assessments
Reduced Self-presentation and (un)intentional faking
Traditional assessments, especially personality questionnaires, have the tendency to induce faking behaviour. In other words, a phenomenon called “Social Desirability Bias” (also known as “self-presentation”) was introduced for (un)intentional faking, whereby candidates (involuntarily) choose the most socially desirable answer instead of the options that represent them the most (Ganster et al., 1983). Their responses would then be inflated and ultimately induce inaccuracy for the score interpretation.
This phenomenon occurs more often in high stake situations (e.g., hiring process or other personnel selection process) compared to low stake situations, which might cause undesired consequences in decision making (Niessen et al., 2017). In addition, there might be dishonest test-takers who only choose the extreme answers in a deliberate attempt to boost their scores (Landers, Sackett & Tuzinski, 2011).
Gamified assessments can help with these issues in two ways. Firstly, a gamified assessment usually has less obvious socially desirable answers. Often gamified assessments are better at concealing what exactly the games measure (Tountopoulou et al., 2021; Nikolaou et al., 2019; Armstrong et al., 2016). For example, while playing games during an application procedure, if the candidates are not fully aware of the purpose of the game, they will play striving to win, not to appear in a particular light. This can lead to a more accurate evaluation of candidates’ ability, rather than “sugar-coated” assessment scores.
Secondly, traditional assessments often use self-reports as the primary instrument to collect applicants’ responses. There’s a high potential for faking because it is solely based on personal judgement. Engaging in games and using contexts that diagnose the ability of candidates in more specific situations could lead to more solid inferences about future performance than assessments that depend on self-reporting (Fetzer et al., 2017; Fiore, 2016).
Similar predictive power but with greater organizational attractiveness.
Are you worried about the predictive power and validity of gamified assessments? Let me reassure you – in fact, several studies have shown that the gamified version of assessments have the same predictive power and validity (Landers et al., 2021; Georgiou et al., 2019; Nikolaou et al., 2019; Levy et al., 2016).
For example, Gorgeous and colleagues (2019) have proven that gamified SJTs have the same construct validity as the traditional SJTs. Furthermore, Levy and colleagues (2016) demonstrated that a game-based assessment showed consistent responses with traditional personality tests and behaviours measured in the game could be used as predictors of certain personality traits.
Additional benefits of gamified assessments have been pointed out by Chow & Chapman (2013, p.93), who state that:
“an organization that employs the use of gamification in its recruitment process may lead individuals to infer that the organization is technologically advanced, trendy and innovative”.
Innovative technologies have repeatedly been proven to have positive effects on applicant perception of an organizational image (Bartram & Hambleton, 2006; Landers et al., 2021; Sinar et al, 2016). For instance, Gkorezis and colleagues (2021) have demonstrated that gamified SJTs have a positive impact on applicants’ perceptions toward organizational attractiveness, which indirectly increase the recommendation intention to their peers.
Higher candidate engagement and more positive reactions.
To win the ongoing talent war, as a company it’s about time to prioritise candidate experience when determining your recruiting methods. If a selection procedure is perceived favourably by candidates, it can lead to less early hiring-process withdrawal, higher possibility of job acceptance, less potential propagation of rumours that hinders company reputation, and underlying legal issues (Ababneh et al., 2014; Derous et al., 2004; Rynes & Connerley, 1993; Smither et al., 1993).
Applicant reactions towards gamified assessments can be categorised into two groups, motivational and attitudinal reactions.
Let’s first look into motivational reactions. This category covers autonomous motivation (in other words, candidate engagement), test motivation, test anxiety and more. Gamified assessments lead to increased candidate engagement, test motivation, satisfaction, and reduce test anxiety by allowing candidates to complete assessments in a more lively, less repetitive way (Landers et al., 2021; Guin et al., 2012; Collmus & Landers, 2019).
Researchers had shown that applicants reported a higher level of process satisfaction and more enjoyment while participating in gamified assessments, compared to their traditional counterparts (Georgiou & Nikolaou, 2020; Guin et al., 2012). In other words, gamified assessment improves test-takers’ engagement through fulfilling the psychological needs for satisfaction (i.e., autonomy, competence, and relatedness) in the game dynamics (Suh et al., 2016). Even the simplest game element, like a progress bar, can reduce the dropout rate, as it serves as a communication tool with the candidates about how many more questions there are left to answer (Yan et al., 2011).
Attitudinal reactions of applicants refer to the perceived fairness of candidates within the recruitment process. Examples of these are distributive justice (the accuracy of how the scores reflect actual performances), procedural justice (chance to perform), job-relatedness (the perceived relevance to the job nature), test propriety, and more. Researchers have proven that all of those aforementioned examples were perceived as more positive when participants were in the gamified assessments, compared to the traditional cognitive ability tests (Landers et al., 2021).
To conclude
Over the past century, researchers have been putting effort into investigating the efficiency and effectiveness of countless personnel selection methods. The science behind gamified assessment methods has proven them to be valid and useful. They have the potential to provide more objective results than relying solely on human judgements. While no assessment method is flawless, gamified assessment can tackle several known issues and help to promote organizational attractiveness, candidates experience and reduce inaccuracy due to social desirability bias.
Cheers, Jiaying
A psychologist majoring in science-based talent assessment.
Mainly focusing on research regarding stakeholders’ perception and reaction towards the recruitment process.
References
Ababneh, K. I., Hackett, R. D., & Schat, A. C. (2013). The Role of Attributions and Fairness in Understanding Job Applicant Reactions to Selection Procedures and Decisions. Journal of Business and Psychology, 29, 111–129. https://doi.org/10.1007/s10869-013-9304-y
Armstrong, M. B., Landers, R. N., & Collmus, A. B. (2016). Gamifying recruitment, selection, training, and performance management: Game-thinking in human resource management. In Emerging research and trends in gamification (pp. 140-165). IGI Global.
Armstrong, M. B., Landers, R. N., & Collmus, A. B. (2016). Gamifying recruitment, selection, training, and performance management: Game-thinking in human resource management. In Emerging research and trends in gamification (pp. 140-165). IGI Global
Bartram, D., & Hambleton, R. (Eds.). (2005). Computer-based testing and the internet: Issues and advances. John Wiley & Sons.
Borghans, L., Golsteyn, B. H., Heckman, J. J., & Humphries, J. E. (2016). What grades and achievement tests measure. PNAS Proceedings of The National Academy of Sciences of The United States Of America, 113, 13354-13359. doi:10.1073/pnas.1601135113
Collmus, A. B. & Landers, R. N. (April 2015). Game narrative in personality assessment: The development of a scale. Presented at Virginia Psychological Association’s Spring Convention and Education Conference, Virginia Beach, VA.
Collmus, A. B., & Landers, R. N. (2019). Game-framing to improve applicant perceptions of cognitive assessments. Journal of Personnel Psychology.
Derous, E., Born, M. P., & Witte, K. D. (2004). How applicants want and expect to be treated: Applicants’ selection treatment beliefs and the development of the social process questionnaire on selection. International Journal of Selection and Assessment, 12, 99-119. https://doi.org/10.1111/j.0965-075X.2004.00267.x
Dunning, D., Heath, C., & Suls, J. M. (2004). Flawed self-assessment: Implications for health, education, and the workplace. Psychological Science in the Public Interest, 5(3), 69-106.
Fetzer, M., McNamara, J., & Geimer, J. L. (2017). Gamification, serious games and personnel selection. Pulakos, J. Passmore, & C. Semedo (Eds.), The Wiley Blackwell handbook of the psychology of recruitment, selection and employee retention, 293-309.
Fiore, C. A. (2016). Gamification of Individual Differences.
Ganster, D. C., Hennessey, H. W., & Luthans, F. (1983). Social desirability response effects: Three alternative models. Academy of Management Journal, 26(2), 321-331.
Georgiou, K., & Nikolaou, I. (2020). Are applicants in favor of traditional or gamified assessment methods? Exploring applicant reactions towards a gamified selection method. Computers in Human Behavior, 109, 106356.
Georgiou, K., & Nikolaou, I. (2020). Are applicants in favor of traditional or gamified assessment methods? Exploring applicant reactions towards a gamified selection method. Computers in Human Behavior, 109, 106356.
Georgiou, K., Gouras, A., & Nikolaou, I. (2019). Gamification in employee selection: The development of a gamified assessment. International journal of selection and assessment, 27(2), 91-103.
Gkorezis, P., Georgiou, K., Nikolaou, I., & Kyriazati, A. (2021). Gamified or traditional situational judgement test? A moderated mediation model of recommendation intentions via organizational attractiveness. European Journal of Work and Organizational Psychology, 30(2), 240-250.
Guin, T. D. L., Baker, R., Mechling, J., & Ruyle, E. (2012). Myths and realities of respondent engagement in online surveys. International Journal of Market Research, 54(5), 613-633.
Kuncel, N. R., & Hezlett, S. A. (2010). Fact and fiction in cognitive ability testing for admissions and hiring decisions. Current Directions in Psychological Science, 19, 339-345. doi:10.1177/0963721410389459
Landers, R. N., Armstrong, M. B., Collmus, A. B., Mujcic, S., & Blaik, J. (2021). Theory-driven game-based assessment of general cognitive ability: Design theory, measurement, prediction of performance, and test fairness. Journal of Applied Psychology.
Levy, L. M., Solomon, R., Johnson, J., Wilson, J., Lambeth, A. J., Gandy, M., … & Liu, R. (2016). Grouches, Extraverts, and Jellyfish: Assessment Validity and Game Mechanics in a Gamified Assessment. In DiGRA/FDG.
Niessen, A. S. M., Meijer, R. R., & Tendeiro (2017). Measuring non-cognitive predictors in high-stakes contexts: The effect of self-presentation on self-report instruments used in admission procedures. Personality and Individual Differences, 106, 183-189. doi: 10.1016/j.paid.2016.11.014
Nikolaou, I., Georgiou, K. & Kotsasarlidou, V. (2019). Exploring the Relationship of a Gamified Assessment with Performance, The Spanish Journal of Psychology, 22. https://doi.org/10.1017/sjp.2019.5.
Rynes, S. L., & Connerley, M. L. (1993). Applicant reactions to alternative selection procedures. Journal of Business and Psychology, 7, 261-277. https://doi.org/10.1007/BF01015754
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262-274. doi:10.1037/0033-2909.124.2.262
Sinar, E. F., Reynolds, D. H., & Paquet, S. L. (2003). Nothing but ‘net? Corporate image and web-based testing. International Journal of Selection and Assessment, 11, 150-157.
Smither, J. W., Reilly, R. R., Millsap, R. E., AT&T, K. P., & Stoffey, R. W. (1993). Applicant reactions to selection procedures. Personnel psychology, 46, 49-76. https://doi.org/10.1111/j.1744-6570.1993.tb00867.x
Stajkovic, A. D., & Luthans, F. (1998). Self-efficacy and work-related performance: A meta-analysis. Psychological Bulletin, 124(2), 240.
Suh, A., Wagner, C., & Liu, L. (2018). Enhancing user engagement through gamification. Journal of Computer Information Systems, 58(3), 204-213.
Yan, T., Conrad, F. G., Tourangeau, R., & Couper, M. P. (2011). Should I stay or should I go: The effects of progress feedback, promised task duration, and length of questionnaire on completing web surveys. International Journal of Public Opinion Research, 23(2), 131-147.
