Title: Optimal item calibration for computerized achievement tests
Authors: Mahmood UL-Hassan - Stockholm University (Sweden) [presenting]
Frank Miller - Stockholm University (Sweden)
Abstract: Item calibration is a technique to estimate characteristics of questions (called items) for achievement tests. In computerized adaptive tests (CAT), item calibration is an important tool for maintaining, updating and developing new items for an item bank. To efficiently sample examinees with specific ability levels for this calibration, we use optimal design theory where we assume that the probability to answer correctly to an item follows a two-parameter logistic model. A locally D-optimal unrestricted design for each item has two design points for ability. In practice, it is hard to sample examinees from a population with these specific ability levels due to unavailability or limited availability of examinees. To counter this problem, we use the concept of optimal restricted designs and show that this concept naturally fits to item calibration. Locally D-optimal restricted designs provide us two intervals of ability levels for optimal calibration of an item. Several scenarios with optimal restricted designs are presented here when one or two items have to be calibrated. These scenarios recommend us that the naive way to sample examinees around unrestricted design points is not the optimal way to calibrate an item.