AutoMentor: Virtual Mentoring and Assessment in Computer Games for STEM Learning
Goal. This Full Research and Development Project will address the STEM Challenge: “How can all students be assured the opportunity to learn significant science, technology, engineering, and mathematics (STEM) content?” This project will develop a system for producing automated professional mentoring, as a critical piece of technological infrastructure for a new, more motivating, and more inclusive approach to STEM education a decade or more in the future, where students are motivated to learn STEM concepts because they play computer games based on STEM professions.
Technology. The project will add two important components to prior work on NSF-funded STEM computer games. We will develop automated mentoring technology, AutoMentor, building on previous research on automated tutoring systems (specifically on AutoTutor, a computer tutor that helps students learn about science and technology topics by holding a conversation in natural language with the learner) and Evidence Centered Assessment Design (specifically, Epistemic Network Analysis, a methodology developed with NSF funding to assess students’ ability to think and act like STEM professionals through
game play).
Hypothesis. In so doing, the project explores a specific hypothesis about STEM mentoring: A sociocultural model as the basis of an automated tutoring system can provide a computational model of participation in a community of practice, which will produce effective professional feedback from nonplayer-characters in a STEM learning game.
Method. The project will use a Wizard of Oz methodology, in which data will be collected about player/mentor interactions over multiple instances of game play, and the resulting database used to develop and validate a system for automatically coding interactions. The coded database will then be used to generate automated responses to player actions in the game, and the resulting system will be tested to see whether players’ STEM learning with automated mentoring are comparable to outcomes with live mentors.
Team. The project team includes leading researchers in intelligent tutoring systems (Graesser),
assessment (Mislevy), and game-based learning (Shaffer). The team also includes a computer scientist (Gleicher), STEM content expert (Asligul Gocmen, Assistant Professor of Urban and Regional Planning,
University of Wisconsin-Madison), measurement expert (Andre A. Rupp, Assistant Professor of Measurement, Statistics, and Evaluation, University of Maryland) and a collaborating institution with expertise in STEM educational programming (Massachusetts Audubon Society). The combination of these areas of expertise is, we believe, unique and novel, and has the potential to transform work in each of the core areas of the proposal: intelligent tutoring, assessment, and game-based learning.
Intellectual Merit
The development of AutoMentor will represent a significant contribution to our knowledge about game-based learning and the science of learning more generally. The development of a computational
model of participation in a community of practice will provide an important link between traditional cognitive science and situated views of learning. It will also potentially contribute to research in artificial intelligence and intelligent agents.
Broader impact
This work will provide a powerful technology for incorporating professional STEM expertise in STEM education activities. The project enhances the infrastructure for joint research by forming a collaborative partnership among three research institutions (the University of Wisconsin-Madison, the University of Maryland, and the University of Memphis) and an educational delivery organization (The
Massachusetts Audubon Society). Results will be disseminated through scientific papers and conferences, but also through the work of the Massachusetts Audubon Society. The game incorporating AutoMentor will be available for use by schools and non-profit organizations.
