PewePro

Adaptive Assessment

A method for adaptive assessment of structured domains.

Overview
Technologies used: Microsoft .NET, MySQL Server, XML
Inputs: Assessment items described in XML format
Output: Measure of student's ability

Addressed Problems

Web-based learning assessments are carried out using mostly by linear and, more recently, adaptive computerized tests. However, they do not provide sufficient freedom of expression required to assess student's progress in solution paths of the problems and thus are not a viable option for deep assessment.

The proposed method selects problems according to the student's demonstrated ability, structured task description schemes allow for a detailed analysis of student's errors, and on-demand generation of task instances facilitates independent student work.

Description

The method we had developed incorporates four major aspects that are important in any robust assessment system:

For an overview see the figure below. Tasks are processed into a parametric task description schemes described using a high-level object library by which they are parametrized to provide sufficient abstraction for the generator module to create new task instances on-demand.

During the assessment phase, the method selects the task scheme that provides the most information for the current student's ability estimate. Using the selected task scheme, an unused task instance is generated and displayed to the student for answering.

method overview

Assessment method overview.

Students' answers are semi-automatically assigned to predefined categories. Having determined the category, the system either (1) asks the student a deeper question regarding her solution path, or (2) finishes the instance administration providing the task outcome which is subsequently used to update the ability estimate. Depending on the amount of error in the updated estimate, the selection module either (i) selects another task scheme at an appropriate level of ability to continue with, or (ii) finishes the assessment process providing the final ability estimate together with the amount of error.

Summary

We presented an adaptive assessment method proposed in an effort to empower online assessment processes with computerized methods. It adaptively selects tasks according to the student's ability. Higher achieving students receive harder and lower achieving students easier questions, giving each the opportunity to demonstrate her ability level.

Publications

  1. Tvarožek, J., Kravčík, M., Bieliková, M. (2008): Towards Computerized Adaptive Assessment Based on Structured Tasks. In: Proc. of AH 2008, Nejdl, W. et al. (Eds.), LNCS 5149, Springer-Verlag, pp. 224–234.