Nephrotex is a new epistemic game in which undergraduate engineering students role-play as professional engineers-in-training in order to develop the skills, knowledge, identity and values of engineers.

In Nephrotex, students are welcomed as early career hires into the fictitious company Nephrotex, whose core technology is the ultrafiltration unit, or dialyzer, of a hemodialysis machine. The students’ assigned task is to design a next-generation dialyzer that incorporates carbon nanotubes and chemical surfactants into the hollow fibers of the dialyzer unit. Since design is a foundational discipline for engineers, we focus on bringing engineering design to first-year engineering students.

Our design project is offered as part of a simulated workplace environment for established professionals in practice Thus, the learning develops in context, and the experience has the potential to more realistically mimic the engineering experience.

Working as a mentor in an epistemic game is also very difficult.

We work with a limited amount of information about what the players are doing: we chat with the players via instant message, but we can’t see the players screens, can’t read their body language, and can’t hear the live conversations that they have offline.

So we are thrilled when we get information from our partners who are on the ground with the players. Here are two, from two versions of Urban Science we are running right now:

1. “Just thought you should know that I found out today that Amanda [not her real name] goes home and talks about this program every week non-stop to her parents over dinner. So, she’s pretty excited about it and enjoying herself.”

2. “That was so cool today. The changes you have made since last year have really made a difference. The access to chatting with mentors and the ability to track the targets as the changes are being made in the maps make all the difference. On the way to lunch, I overheard kids talking about the changes they made and getting close. The child with Aspergers was really engaged and wanting to make the target changes.”

http://epistemicgames.org/eg/wp-content/uploads/engineering-workshop.pdf

We propose, then, the development of a new method of STEM assessment—called epistemic network analysis (ENA) that focuses not on whether students master specific scientific facts, math skills, or engineering concepts, but on whether and how students link the skills, knowledge, identity, values, and epistemology of a STEM practice into a coherent way of thinking about complex STEM problems.
We describe ENA as “potentially transformational” because it is in its early stages, and involves a radically different and interdisciplinary approach to the problems of STEM assessment. In this proposal we link prior work on an innovative theory of STEM thinking with the mathematical and conceptual tools of social network analysis to create a new conceptual and statistical approach to the measurement of STEM thinking and STEM learning.

We have been developing ENA as an assessment tool in the context of a particular theory of learning (the epistemic frame hypothesis) that applies to a specific kind of STEM learning computer game (epistemic games). However, we want to emphasize ENA is an approach to assessment that could be used in any situation of complex STEM thinking where the connections between things being learned are more important than isolated pieces themselves.

Intellectual Merit
In this proposal, we link work across several domains—including learning theory, psychometrics, and sociology—to develop a new assessment technique designed to measure the impact of technology-based STEM learning environments. This emerging research is, by its nature, uncertain. However, the collective work of the project team—including joint work on the pilot phases of this project—suggests that this research will produce conceptual, theoretical, and methodological results with the potential for far-reaching and long-term impacts on the theory and practice of network analysis, visualization, and their applications to STEM learning.

Broader Impact
The development of the cognitive model proposed here will help educators in a wide variety of fields analyze and improve STEM education by providing a means to dynamically assess the development of complex STEM thinking. The tools we develop will contribute to the field of network analysis, and our development process will enhance the skills and career trajectories of at least five young investigators.

Technology. The project will add two important components to prior work on NSF-funded STEM computer games. We will develop automated mentoring technology, AutoMentor, building on previous research on automated tutoring systems (specifically on AutoTutor, a computer tutor that helps students learn about science and technology topics by holding a conversation in natural language with the learner) and Evidence Centered Assessment Design (specifically, Epistemic Network Analysis, a methodology developed with NSF funding to assess students’ ability to think and act like STEM professionals through
game play).

Hypothesis. In so doing, the project explores a specific hypothesis about STEM mentoring: A sociocultural model as the basis of an automated tutoring system can provide a computational model of participation in a community of practice, which will produce effective professional feedback from nonplayer-characters in a STEM learning game.

Method. The project will use a Wizard of Oz methodology, in which data will be collected about player/mentor interactions over multiple instances of game play, and the resulting database used to develop and validate a system for automatically coding interactions. The coded database will then be used to generate automated responses to player actions in the game, and the resulting system will be tested to see whether players’ STEM learning with automated mentoring are comparable to outcomes with live mentors.

Team. The project team includes leading researchers in intelligent tutoring systems (Graesser),
assessment (Mislevy), and game-based learning (Shaffer). The team also includes a computer scientist (Gleicher), STEM content expert (Asligul Gocmen, Assistant Professor of Urban and Regional Planning,
University of Wisconsin-Madison), measurement expert (Andre A. Rupp, Assistant Professor of Measurement, Statistics, and Evaluation, University of Maryland) and a collaborating institution with expertise in STEM educational programming (Massachusetts Audubon Society). The combination of these areas of expertise is, we believe, unique and novel, and has the potential to transform work in each of the core areas of the proposal: intelligent tutoring, assessment, and game-based learning.

Intellectual Merit
The development of AutoMentor will represent a significant contribution to our knowledge about game-based learning and the science of learning more generally. The development of a computational
model of participation in a community of practice will provide an important link between traditional cognitive science and situated views of learning. It will also potentially contribute to research in artificial intelligence and intelligent agents.

Broader impact
This work will provide a powerful technology for incorporating professional STEM expertise in STEM education activities. The project enhances the infrastructure for joint research by forming a collaborative partnership among three research institutions (the University of Wisconsin-Madison, the University of Maryland, and the University of Memphis) and an educational delivery organization (The
Massachusetts Audubon Society). Results will be disseminated through scientific papers and conferences, but also through the work of the Massachusetts Audubon Society. The game incorporating AutoMentor will be available for use by schools and non-profit organizations.

While we were talking, one of her students walked into the room. The teacher enthusiastically told her that the class would be playing Urban Science again this spring. The student looked at us and wordlessly unzipped her coat to reveal the Epistemic Games t-shirt that all of the players got the previous year.

While I don’t want to go too far in interpreting the synchronicity of this encounter, I couldn’t help but think that 5th graders do not make sartorial choices lightly. It can sometimes be hard to know the inner transformations that happen as kids are learning and growing. But every once in a while, if you are lucky, you can get an unzipped glimpse of what kids take with them.

Engineering institutions nationwide are pursuing first-year engineering design courses to attract and retain nontraditional students. However, these courses often have high enrollment rates and can be resource intensive. Virtual design projects offer a potential solution to the physical resources requirements but often result in an overly constrained design space, creating uninteresting or non-challenging design problems. We are developing a design problem within a novel virtual environment (i.e.a game) that provides first-year engineering undergraduates with a more authentic engineering design experience and a more complete and accurate understanding of the engineering profession. The design problem presented challenges students to incorporate carbon nanotubes and chemical surfactants into a hemodialysis ultrafiltration unit. our approach seeks to provide students with experience in the skills, knowledge, values, identity, and epistemology of the engineering profession, which is the epistemic frame of the profession. The virtual environment also provides a uniquely comprehensive platform for assessing the students’ epistemic frame development over time. We anticipate that this approach will be highly engaging to first-year undergraduate engineering students and will help engineering instructors understand how engineers-in-training learn to become engineers.

Computer Games in Education

Oct. 22, 2009

by Emily Mawer

A research team at the University of Wisconsin-Madison will study computer games and learning with federal research grants.

The principal investigator on several of the grants, David Williamson Shaffer, a professor of educational psychology at UW-Madison, said computers games allow students to live in a simulated world where they can face real life problems.

“What computer games do is give young people an opportunity to prepare for the kind of innovative and creative real world problem solving that they need to deal with in a global economy,” Shaffer said.

The largest grant, from the National Science Foundation, is devoted to research surrounding the Urban Science computer game, previously created at UW-Madison.  In the game, middle school and high school students become urban planners and solve problems that planners typically face, including going on site visits, talking to stake holders and using feedback to create a design proposal.  The research will focus on creating mentors for the game to coach students through their questions.

Shaffer explained that students studying to be urban planners are given the opportunity to try out parts of the planning process and then talk about their work with mentors.

“It is those conversations that turn the action that they are doing into understanding about the way the profession works,” Shaffer said.  “So in the game we recreated that.”

As the game currently exists, students can seek help from adult mentors through online chats.  Shaffer’s team, in partnership with the Massachusetts Audubon Society, will collect a database of commonly asked questions and answers.  Then they will analyze the information and create a system within the game that can respond appropriately to students.

Shaffer hopes that this part of the research will be applicable beyond this one game.

“The real payoff is in being able to use this same approach to make high quality professional mentoring available in all kinds of fields and all kinds of games,” Shaffer said.

The Urban Science research will take place over the next five years, with the first trials over the winter.

In addition to the Urban Science project, research grants are funding the creation of a computer game to make the engineering school at the University more diverse.

Shaffer is working with Naomi Chesler, an associate professor in the School of Engineering, to create a game that will increase retention in the engineering school, especially retention of woman and minorities.

“The idea is that a diverse workforce makes it possible to communicate with other countries and other places more effectively, because you have people from a variety of backgrounds,” Shaffer said.  “It brings other ideas and other perspectives.”

In the game, Nephrotex, engineering students will build a component of a dialysis machine using nanotechnology.  Shaffer said the idea is to give students a realistic experience of engineering design and to allow them to follow the process from beginning to end.

“Learning to become engineers is not just about math and science, but also about learning to see the world through a certain viewpoint,” Chesler said.

It is the math and science coursework that may cause so many students to decide against an engineering major, according to Shaffer.

”It is this kind of humanistic perspective on engineering that is often missing from the early parts of the curriculum,” Shaffer said.  “So, you get people who want to solve problems for other people, instead they end up solving a bunch of math equations and they get discouraged and drop out.”

The Nephrotex game will provide the kind of real world experience students need to stay motivated in their other courses, Shaffer said.

Nephrotex will be ready within the next two years, according to Chesler.