Evaluation of mathematical questioning strategies using data collected through weak supervision
Abstract
A large body of research demonstrates how teachers' questioning strategies can improve student learning outcomes. However, developing new scenarios is challenging because of the lack of training data for a specific scenario and the costs associated with labeling. This paper presents a high-fidelity, AI-based classroom simulator to help teachers rehearse research-based mathematical questioning skills. Using a human-in-the-loop approach, we collected a high-quality training dataset for a mathematical questioning scenario. Using recent advances in uncertainty quantification, we evaluated our conversational agent for usability and analyzed the practicality of incorporating a human-in-the-loop approach for data collection and system evaluation for a mathematical questioning scenario.
- Publication:
-
arXiv e-prints
- Pub Date:
- December 2021
- DOI:
- 10.48550/arXiv.2112.00985
- arXiv:
- arXiv:2112.00985
- Bibcode:
- 2021arXiv211200985D
- Keywords:
-
- Computer Science - Artificial Intelligence;
- Computer Science - Human-Computer Interaction;
- Computer Science - Machine Learning
- E-Print:
- Accepted to appear at the NeurIPS 2021 Workshop on Math AI for Education (MATHAI4ED)