Adaptive testing, in vocational psychology dictionary

home > library > dictionary > adaptive testing
Dictionary of Vocational Psychology

Adaptive Testing

A concept related to methods through which to efficiently learn an examinee's level on a particular trait. When administering test items to an examinee, one learns the most when one administers an item close to the presumed true trait level of the examinee. If the item is too difficult or extreme in one direction, and the examinee fails it, one learns little. Likewise, if the item is too easy or extreme in the other direction, and the examinee passes it, one again learns little. The information value of an item is greatest when it is near the examinee's true level on a trait, and testing is most efficient--that is, makes use of the fewest possible items in a test--when items congregate around the examinee's true level on the trait. Adaptive testing--also known as tailored or individualized testing--takes advantage of this principle through test administration procedures in which different examinees received different sets of items.

Various types of tests incorporate principles of adaptive testing. Some individually administered clinical assessments, such as the Stanford-Binet Intelligence Scales, Fifth Edition (SB5), use various methods to achieve this. For example, the SB5 includes a routing test (specifically, either or both of the subtests Verbal Knowledge and Nonverbal Reasoning) to estimate the examinee's general ability level. Then the examinee is routed to start administration of the remaining subtests at that point. Further administration is tailored to individual ability levels in the various specific abilities tested based on the questions that the examinee passes and fails. The idea is to gain maximum information about the examinee's various abilities (general, broad, and specific) as efficiently as possible. This particular form of adaptive testing--which makes use of an initial routing test--is also referred to as two-stage testing. The routing test is sometimes referred to as a locator test.

It is also possible to allow the examinee him or herself to decide which items to take. In such a scenario, the items of the test are ordered from easiest to hardest. The examinee is encouraged to start with items of moderate difficulty, and then to complete a range of items, moving gradually in the direction of items that are easier and harder, until items in both directions are perceived as being either too difficult or too hard. Allen and Yen (1979, p. 233) describe this method, noting that its relative effectiveness remains to be evaluated.

Finally, adaptive testing may be implemented through computer administered items, also known as computerized adaptive testing (CAT). The general use of computers to administer a test is known as computer-assisted or computerized testing, so CAT represents a subset of possible computerized tests. Various algorithms have been proposed to maximize the efficiency of CAT (see Allen & Yen, 1979, p. 234). One of the difficulties in developing such an algorithm is to avoid making the CAT-based test discouraging and demotivating for the test taker. It is all to easy in such tests to create a test in which the test taker experiences repeated failure and frustration. In addition, it is important in such tests to ensure that the bank of items from which the algorithm may select items is sufficiently deep and extensive. It is also important that the model guiding the CAT-based test accurately reflect the structure of the trait being assessed; for example, if in some examinees the skills being assessed reflect a bimodal distribution in the domain being tested, it may be impossible for the algorithm to zero-in on a specific level of true ability for the examinee. In such cases, testing may continue indefinitely in a less-sophisticated CAT-based test.

References