Skill Check: Some Considerations on the Evaluation of Gamemastering Models for Role-playing Games
Abstract
In role-playing games a Game Master (GM) is the player in charge of the game, who must design the challenges the players face and narrate the outcomes of their actions. In this work we discuss some challenges to model GMs from an Interactive Storytelling and Natural Language Processing perspective. Following those challenges we propose three test categories to evaluate such dialogue systems, and we use them to test ChatGPT, Bard and OpenAssistant as out-of-the-box GMs.
- Publication:
-
arXiv e-prints
- Pub Date:
- September 2023
- DOI:
- 10.48550/arXiv.2309.13702
- arXiv:
- arXiv:2309.13702
- Bibcode:
- 2023arXiv230913702G
- Keywords:
-
- Computer Science - Computation and Language;
- Computer Science - Artificial Intelligence
- E-Print:
- 11 pages. Accepted at GALA 2023 (Games and Learning Alliance 12th International Conference)