Assessing Student Understanding of Physical Hydrology

Our objective is to characterize and assess upper division and graduate student thinking by developing and testing an assessment tool for a physical hydrology class. The class' learning goals are: (1) Quantitative process-based understanding of hydrologic processes, (2) Experience with different methods in hydrology, (3) Learning, problem solving, communication skills. These goals were translated into two measurable tasks asked of students in a questionnaire: (1) Describe the significant processes in the hydrological cycle and (2) Describe laws governing these processes. A third question below assessed the students' ability to apply their knowledge: You have been hired as a consultant by __ to (1) assess how urbanization and the current drought have affected a local spring and (2) predict what the effects will be in the future if the drought continues. What information would you need to gather? What measurements would you make? What analyses would you perform? Student and expert responses to the questions were then used to develop a rubric to score responses. Using the rubric, 3 researchers independently blind-coded the full set of pre and post artifacts, resulting in 89% inter-rater agreement on the pre-tests and 83% agreement on the post-tests. We present student scores to illustrate the use of the rubric and to characterize student thinking prior to and following a traditional course. Most students interpreted Q1 in terms of physical processes affecting the water cycle, the primary organizing framework for hydrology, as intended. On the pre-test, one student scored 0, indicating no response, on this question. Twenty students scored 1, indicating rudimentary understanding, 2 students scored a 2, indicating a basic understanding, and no student scored a 3. Student scores on this question improved on the post-test. On the 22 post-tests that were blind scored, 11 students demonstrated some recognition of concepts, 9 students showed a basic understanding, and 2 students had a full understanding of the processes linked to hydrology. Half the students had provided evidence of the desired understanding; however, half still demonstrated only a rudimentary understanding. Results on Q2 were similar. On the pre-test, 2 students scored 0, 21 students scored 1, indicating rudimentary understanding, 2 students scored a 2, and no student scored a 3. On the post-test, again approximately half the students achieved the desired understanding: 9 students showed some recognition of concepts, 12 students demonstrated a basic understanding; only one student exhibited full understanding. On Q3, no student scored 0, 9 scored 1, 15 scored 2 and 1 student scored 3. On the post-test, one student scored 1, 16 students scored 2, and 5 students scored 3. Students were significantly better at responding to Q3 (the application) as opposed to Q1 and Q2, which were more abstract. Research has shown that students are often better able to solve contextualized problems when they are unable to deal with more abstract tasks. This result has limitations including the small number of participants, all from one institution, and the fact that the rubric was still under development. Nevertheless, the high inter-rater agreement by a group of experts is significant; the rubric we developed is a potentially useful tool for assessment of learning and understanding physical hydrology. Supported by NSF CAREER grant (EAR-0955750).