A neural network solves, explains, and generates university math problems by program synthesis and few-shot learning at human level
Abstract
We demonstrate that a neural network automatically solves, explains, and generates university-level problems from the largest Massachusetts Institute of Technology (MIT) mathematics courses at a human level. Our methods combine three innovations: 1) using recent neural networks pretrained on text and fine-tuned on code rather than pretrained on text; 2) few-shot learning synthesizing programs that correctly solve course problems automatically; and 3) a pipeline to solve questions, explain solutions, and generate new questions indistinguishable by students from course questions. Our work solves university-level mathematics courses and improves upon state-of-the-art, increasing automatic accuracy on randomly sampled questions on a benchmark by order of magnitude. Implications for higher education include roles of artificial intelligence (AI) in automated course evaluation and content generation.
- Publication:
-
Proceedings of the National Academy of Science
- Pub Date:
- August 2022
- DOI:
- 10.1073/pnas.2123433119
- arXiv:
- arXiv:2112.15594
- Bibcode:
- 2022PNAS..11923433D
- Keywords:
-
- Computer Science - Machine Learning;
- Computer Science - Artificial Intelligence
- E-Print:
- 181 pages, 8 figures, 280 tables