The Assessment of Partial Knowledge in Multiple-Choice Chemistry Examinations

In this thesis the literature on the use of partial knowledge techniques in objective testing is reviewed with particular reference to the work done in the area of multiple-choice testing. Four methods for assessing partial knowledge in multiple-choice Chemistry examinations are investigated - (1) The Free-Choice Method, (2) The Elimination Method, (3) A Confidence Testing Method and (4) The 'Three-Decision' Multiple-Choice Test. The Elimination and Free-Choice Methods were compared by administering a forty item multiple-choice Chemistry test to 180 fifteen year old pupils, in the Paisley area, who had followed the course for the Scottish Certificate of Education Examination in Ordinary Grade Chemistry. The items in the test were arranged in the 'easy' to 'difficult' order and consisted of twelve items measuring Knowledge, twelve items measuring Comprehension and sixteen items measuring the Higher Skills of Application, Analysis and Evaluation. It was found that pupils made greater use of the partial knowledge technique in the Free-Choice Method. No improvement was obtained in Facility Value, Reliability and Discrimination when the results obtained were compared with those obtained from the control test. The methods available for the item analysis of tests which allowed for partial knowledge were also reviewed and a computer program produced which could be used to carry out an item analysis of the results obtained in the Free-Choice and Elimination Methods. The Confidence test was administered to 100 fifteen year old pupils who had also been following the course for the Ordinary Grade Examinations in Chemistry. The test used was the same one that was used for the comparison of the Free-Choice and Elimination Methods. Again no improvement was obtained in item statistics, but valuable information was obtained concerning the pupils' state of Knowledge at the time of the test, particularly concerning their willingness to guess and the degree of accuracy of their guesses. The investigation into the use of the 'Three-Decision' Multiple- Choice Test consisted of two parts (a) its use with school pupils and (b) its use with university students. In investigation (a) a forty item multiple-choice test was administered to 66 fourteen year old pupils who were following the course for the Ordinary Grade Chemistry Examination and in investigation (b) a twenty item multiple-choice Chemistry test was administered to 285 first year university students. Again no improvement was obtained in item statistics, but valuable diagnostic information was obtained concerning the examinees' knowledge of Chemistry at the time of the test particularly with reference to the nature of the 'Second-Choice' correct answer. University students were found to gain more marks than school pupils and were more successful in their use of the 'Second-Choice' correct answer. The diagnostic use of partial knowledge tests was investigated by a detailed analysis of several representative items. An attempt was made to measure risk taking in multiple-choice tests, but the method chosen did not prove to be very reliable.