The Geoscience Concept Test: A New Assessment Tool Based on Student Misconceptions

We developed and began pilot testing of an earth science assessment tool called the geoscience concept test (GCT). The GCT uses student misconceptions as distractors in a 30 item multiple-choice instrument. Student misconceptions were first assessed through the analysis of nearly 300 questionnaires administered in introductory geology courses at three institutions. Results from the questionnaires guided the development of an interview protocol that was used by four interviewers at four different institutions. Over 100 in-depth student interviews lasting from 0.5 to 1 hour probed topics related to the Earth's interior, geologic time, and the formation of Earth surface features such as mountains and volcanoes to better define misconceptions. Thematic content analysis of the interviews identified a number of widely held misconceptions, which were then incorporated into the GCT as multiple-choice distractors (wrong answers). For content validity, the initial GCT was reviewed by seven experts (3 geoscientists and 4 science educators) and revised before pilot testing. Approximately 100 introductory and non-science major college students from four institutions were assessed with the GCT pilot in the spring of 2002. Rasch model analysis of this data showed that students found the pilot test difficult, and the level of difficulty was consistent between the four institutions. Analysis of individual items showed that students had fewer misconceptions regarding the locations of earthquakes, and many misconceptions regarding the locations of volcanoes on the Earth's surface, suggesting a disconnect in their understanding of the role of plate tectonics in these phenomena. Analysis of the misfit statistic for each item showed that none of the questions misfit, although we dropped one question and modified the wording of another for clarity in the next round of piloting. A second round of piloting scheduled for the fall of 2002 includes nearly 3000 students from 34 institutions in 19 states.