Evaluating Changes in Climate Literacy among Middle and High School Students who Participate in Climate Change Education Modules

Middle school (MS) and high school (HS) teachers have developed and taught instructional modules that were created through their participation in Clarkson University's NASA-funded Project-Based Global Climate Change Education project. A quantitative survey was developed to help evaluate the project's impact on students' climate literacy, which includes content knowledge as well as affective and behavioral attributes. Content objectives were guided primarily by the 2009 document, Climate Literacy: The Essential Principles of Climate Sciences. The survey was developed according to established psychometric principles and methodologies in the sociological and educational sciences which involved developing and evaluating a pool of survey items, adapted primarily from existing climate surveys and questionnaires; preparing, administering, and evaluating two rounds of pilot tests; and preparing a final instrument with revisions informed by both pilot assessments. The resulting survey contains three separate subscales: cognitive, affective, and behavioral, with five self-efficacy items embedded within the affective subscale. Cognitive items use a multiple choice format with one correct response; non-cognitive items use a 5-point Likert-type scale with options generally ranging from "strongly agree" to "strongly disagree" (affective), or "almost always" to "hardly ever" (behavioral). Three versions of the survey were developed and administered using an on-line Zoomerang™ platform to college students/adults; HS students; and MS students, respectively. Instrument validity was supported by using items drawn from existing surveys, by reviewing/applying prior research in climate literacy, and through comparative age-group analysis. The internal consistency reliability of each subscale, as measured by Cronbach's alpha, ranges from 0.78-0.86 (cognitive), 0.87-0.89 (affective) and 0.84-0.85 (behavioral), all satisfying generally accepted criteria for internal reliability of educational surveys. MS and HS students completed the on-line survey prior to and at least 3 weeks following participation in one of the newly developed project-based climate change modules. Surveys were completed anonymously. In all, 9 HS and 3 MS teachers successfully completed the educational programming and assessment protocol in AY2012, yielding 200 HS and 227 MS matched pre/post climate literacy surveys. Both groups of students demonstrated significant gains in climate-related content knowledge (p<<0.001) and affect (p<0.01). MS students also experienced significant gains in their climate-related self-efficacy (p=0.03), with no significant change in self-efficacy for HS students and no change in either group on the behavioral subscale. Post-scores were remarkably similar for the two groups of students; reported as percent of maximum attainable score for HS/MS students: 59%/58%, knowledge; 65%/64%, affect; 71%/72%, self-efficacy, and 63%/62%, behavior. The presentation will include a description of the development and content of the climate literacy survey used in this research, as well the interpretation of specific pre/post changes in participating MS and HS students relative to the content of and approach used in the project-based modules.