A Bayesian Student Model without Hidden Nodes and its Comparison with Item Response Theory
In IJAIED
15 (1)
Publication information
Abstract
The Bayesian framework offers a number of techniques for inferring an individual's
knowledge state from evidence of mastery of concepts or skills. A typical application
where such a technique can be useful is Computer Adaptive Testing (CAT). A Bayesian
modeling scheme, POKS, is proposed and compared to the traditional Item Response Theory
(IRT), which has been the prevalent CAT approach for the last three decades. POKS
is based on the theory of knowledge spaces and constructs item-to-item graph structures
without hidden nodes. It aims to offer an effective knowledge assessment method with
an efficient algorithm for learning the graph structure from data. We review the different
Bayesian approaches to modeling student ability assessment and discuss how POKS relates
to them. The performance of POKS is compared to the IRT two parameter logistic
model. Experimental results over a 34 item Unix test and a 160 item French language
test show that both approaches can classify examinees as master or non-master effectively
and efficiently, with relatively comparable performance. However, more significant differences
are found in favor of POKS for a second task that consists in predicting individual
question item outcome. Implications of these results for adaptive testing and student modeling
are discussed, as well as the limitations and advantages of POKS, namely the issue
of integrating concepts into its structure.
Keywords. Bayesian inference, adaptive testing, student models, CAT, IRT, POKS
