A Formative Classroom Evaluation of a Tutorial Dialog System that Supports Self-Explanation.pdf

A Formative Classroom Evaluation of a Tutorial Dialogue System that Supports Self- Explanation Vincent ALEVEN Octav POPESCU Amy OGAN Kenneth R. KOEDINGER Human Computer Interaction Institute Carnegie Mellon University 5000 Forbes Ave, Pittsburgh, PA 15213, USA aleven@, octav@, aeo@, koedinger@ Abstract. Previous research has shown that self-explanation can be supported effectively in an intelligent tutoring system by simple means, namely, menu selection (Aleven Koedinger, 2002). We are exploring whether self-explanation can be supported even more effectively by means of a tutorial dialogue system. When students explain in their own words, they may pay more attention to the crucial features of the problem and thus acquire more transferable knowledge. To test this hypothesis, we are developing the Geometry Explanation Tutor, a tutor which supports students as they solve problems and engages them in a restricted form of dialogue to help them state general explanations of their problem-solving steps. We conduced a formative evaluation study in a local junior high school, comparing two tutor versions, one in which students explain in their own words, one in which they explain by means of a simple menu. We found that there was little difference between the learning results of the two conditions. We also found that the pre-test scores in both conditions were unexpectedly high. Thus, the hypothesized advantages for explaining in one’s own words do not seem to materialize for students who start out with considerable knowledge of the subject matter, yet may still materialize for students more typical of the target population. Introduction A number of cognitive science studies have shown that self-explanation is an effective metacognitive strategy that helps students learn with greater understanding(Aleven Koedinger, 2002; Bielaczyc, Pirolli, Brown, 1995; Chi, 2000; Renkl, 1999). Self- explanation can be supported effectively in intelligent tutoring systems, using fairly s