Testing GPT-4 with Wolfram Alpha and Code Interpreter plug-ins on math and science problems
Abstract
This report describes a test of the large language model GPT-4 with the Wolfram Alpha and the Code Interpreter plug-ins on 105 original problems in science and math, at the high school and college levels, carried out in June-August 2023. Our tests suggest that the plug-ins significantly enhance GPT's ability to solve these problems. Having said that, there are still often "interface" failures; that is, GPT often has trouble formulating problems in a way that elicits useful answers from the plug-ins. Fixing these interface failures seems like a central challenge in making GPT a reliable tool for college-level calculation problems.
- Publication:
-
arXiv e-prints
- Pub Date:
- August 2023
- DOI:
- 10.48550/arXiv.2308.05713
- arXiv:
- arXiv:2308.05713
- Bibcode:
- 2023arXiv230805713D
- Keywords:
-
- Computer Science - Artificial Intelligence;
- Mathematics - History and Overview;
- Physics - Popular Physics