Does coefficient size really matter? The impact of social desirability and common method variance on the construct validity of game-based assessments

Some critics of Game‐Based Assessment (GBA) have focused on the small sized coefficients that tend to emerge during construct validations, and this casts unnecessary doubts on the overall validity of
the approach. This paper will combine three pieces of evidence that suggest other factors may affect
coefficient sizes when estimating the construct validity of GBAs, and why this should not impact the
overall validity of the GBA. Specifically, evidence comparing two groups of people differing only by
their tendency to respond desirably in self‐reported survey will be presented to demonstrate that
the weakness of self‐reported measures typically used for construct validations are often the very reason behind the lack ‐ or smaller size – of correlations between GBAs and other tests measuring the same construct. Furthermore, Multi‐Trait‐Multi‐Method (MTMM) matrices will be discussed to
show the impact that common method variance has on self‐report measures, and how this compares to GBA. Finally, recent developments of a GBAs scoring keys will be used to Illustrate how
well GBAs can capture the full breadth of a given construct, maintaining great content coverage
especially when dealing with facets that exceed the test takers’ capacity for self‐insight. These three
sources of evidence will equip the audience with a good understanding of how construct validity of
GBAs differ from self-report measures, and how this does not constitute a weakness.