Game based assessments (GBAs) are a hot topic. But how do you separate the science from the spin? Ben Williams, MD of Sten10 and Board Member at the Association for Business Psychology, provides a practical guide to working out whether they are right for your organisation.

 1)      Build: Is the GBA built well?

What qualities does the GBA claim to measure – ability, personality, motivation, values?

• Is there evidence that the different parts of the game measure the same quality? On a 0-1 correlation scale, this should be at the level of 0.7 or higher. You should ask your potential GBA provider for this figure (as well as the others in this article). Most GBAs currently range between 0.51 and 0.96.
• What evidence is there that the GBA replicates the results of more established measures? For example, if a GBA claims to measure verbal reasoning, the publisher should be able to compare how well people perform on the GBA versus a ‘traditional’ verbal reasoning test. If the correlation is over 0.7, it can be considered a good alternative measure. Research shows this figure typically ranges between 0.2 to 0.4 – suggesting the two test types are not interchangeable.

Are there relevant comparison groups in place? You will need to benchmark how people perform against others who have previously taken the GBA. When a GBA is new, the comparison groups can be restricted to a certain industry sector or level, so consider creating your own groups.

Is the technology robust? Candidate perceptions of GBAs are severely hampered when they encounter technology issues, so ask about compatibility tests and if it’s been used for similar volumes to what you need.

2)      Consistency: Does it give the same results, time after time?

If it does perform consistently, the test is judged reliable. If not, then the test is unduly affected by things like time of day, motivation level, background noise, type of device etc. Alternatively, it could be that the quality that you are trying to measure changes over time (for example, work motivation). 

Look for a correlation figure of 0.7 or higher to prove consistency. Research to date from GBA publishers shows this is typically in the range of between 0.57 and 0.82.

3)      Predictivity: Does it tell you something meaningful about how people will do in the job?

Do those who score ‘best’ in the assessment go on to be the best in the assessment centre or in the job itself? If so, the test is judged valid. If not, then there is no clear rationale for using the test.

Current research tends to focus on the ability to predict assessment centre performance, with correlations in the range of 0.14 to 0.35. We typically look for 0.3 or higher. You may need to conduct a small-scale validity study in your organisation to prove the tests will be predictive in your context.

4)      Fairness and Legality: Does it discriminate fairly and legally?

 Ensure there are no statistically significant differences between groups who take the GBA. Group characteristics could include those that are legally protected such as age and gender, but also others such as handedness, screen size or gaming experience. Research so far from publishers is positive, with few significant differences in performance found for any of the above categories.

5)      Engagement: Does it increase the likelihood a candidate will accept a job offer or promote your brand in general?

One of the key reasons often quoted for using GBAs is that they will help to improve candidates’ perceptions of the hiring process and the organisation; as a result, they will be more likely to accept a job offer.

Candidates generally see GBAs equally or more fair than traditional psychometrics. However, one study found that millennials and those with lots of gaming experience tended to have a lower perception of game-based assessments – perhaps due to high expectations. Another study found that those who were more ‘agreeable and open to the experience’ were more likely to enjoy them. The danger is that using GBAs will influence the profile of people you are attracting.

Ultimately, when deciding whether to use GBAs, bear in mind a study from Laumer (2012) that showed although candidates enjoyed a GBA, they valued fairness, ease of use and usefulness of the assessment more than enjoyment. Therefore, these criteria should perhaps be paramount in your decision about which type of assessment to use.

Visit our Knowledge section for more insights on the use of technology in recruitment