ICLS2010-submission-173-revised-final

Preparing citizens to address the complex problems inherent in cities requires our changing society to embrace a new kind of education. One way to train people to think about complex problems is to identify and study how professionals who think in those ways develop their epistemic frame. In this paper, we examine one of the ways urban planners master and appropriate relevant expertise through an ethnographic study of an urban planning practicum. Specifically, we use a new method called epistemic network analysis to look at presentation feedback sessions during two weeks of the practicum to explore emergent relationships between the teacher’s planning expertise and the students’ expertise. The results of this study indicate that epistemic network analysis offers a technique for analyzing the kinds of situated understanding that result from sociocultural learning and for observing the translation of pedagogy into practice in various types of learning environments.

ICLS2010-submission-173-revised-final

Epistemic games have been developed to help players develop domain-specific expertise that characterizes how professionals in a particular domain reason, communicate, and act (Shaffer, 2006; Shaffer & Bagley, 2009). Grounded in a sociocultural and sociocognitive approach to learning, epistemic games are designed to foster situated learning that leads to data structures with high levels of dependencies. As one might expect, traditional measurement models struggle to accommodate such contextual dependencies, especially when data are collected at smaller scales and epistemic network analysis (ENA) has been developed to provide a practically feasible modeling alternative (e.g., Rupp et al., 2009; Shaffer et al., in press). In this paper, we describe a research program that addresses key statistical considerations for modeling data from epistemic games using ENA with an eye toward representing different learning progressions of players within such games. Current approaches for representing learning progressions using ENA are juxtaposed with approaches for simulating such data using particular statistical constraints.

He also quotes epistemic games researcher Elizabeth Bagley, “There are tradeoffs and consequences, and these are things they need to deal with as a planner. They are learning how to really facilitate compromises because there are stakeholders whose goals don’t overlap.”

While the game facilitates collaboration and critical thinking, it also connects young people to the environment and to their neighborhoods. View the full article on the Journal Sentinel website, or a PDF here.

‘Well, Maria thought that she did the assignment wrong,’ I replied.

‘I can’t believe that. Her map was amazing. She had the entire city zoned for specific uses,’ responded the student.

‘Maria said that all of the other kids drew houses with smoke coming out of the chimneys and trees in the lawn.’ 

‘They did, and they were totally lame,’ the student said decisively. 

But that’s not what Maria thought. Since her work looked nothing like her peers’, she assumed that she had done the assignment wrong.  She even threw it away shortly after receiving a grade (an A) on it, not recognizing her own innovation since her teacher hadn’t.

That’s why I was excited that the student remembered Maria’s work a year after she had observed her class, and I followed up by talking with the teachers-in-training about the messages we too often send our children in school, the message that their work is little more than a means to a grade. I hope that Maria’s story helped some of the students think outside of the traditional ‘assessment box’ and inspired them to reflect on how they measure success in their classrooms.  

In this paper we look at educational assessment in the 21st Century. Digital learning environments emphasize learning in action. In such environments, assessments need to focus on performance in context rather than on tests of abstracted and isolated skills and knowledge. Digital learning environments also provide the potential to assess performance in context, because digital tools make it possible to record rich streams of data about learning in progress. But what assessment methods will use this data to measure mastery of complex problem solving, the kind of thinking in action that takes place in digital learning environments?

Here we argue that one way to address this challenge is through evidence-centered design: a framework for developing assessments by systematically linking models of understanding, observable actions, and evaluation rubrics to provide evidence of learning. We examine how evidence-centered design can address the challenge of assessment in new media learning environments by presenting one specific theory-based approach to digital learning, known as epistemic games, and describing a method, epistemic network analysis, to assess learner performance based on this theory. We use the theory and its related assessment method to illustrate the concept of a digital learning system: a system composed of a theory of learning and its accompanying method of assessment, linked into an evidence-based, digital intervention. And we argue that whatever tools of learning and assessment digital environments use, they need to be integrated into a coherent digital learning system linking learning and assessment through evidence-centered design.

This short video gives an overview of our work on Urban Science and other epistemic games as part of the Macarthur Digital Media and Learning Project and the National Science Foundation.

In these games, players have a chance to learn 21st century skills by playing as urban planners, engineers, journalists, and other professionals in the knowledge economy.

I suppose next we’ll need to make an epistemic game about making epistemic games….

http://epistemicgames.org/eg/wp-content/uploads/ijgcms-bagley-shaffer.pdf

Trying on different identities via new media was hot at this year’s AERA conference. Elisabeth Soep spoke about her work with Youth Radio. Participants learn the basics of broadcasting, and in the process, they explore identity through authorship of new media stories for local and national outlets. What is especially powerful about Youth Radio is that young people have a specific role from which they can complete consequential tasks and explore new identities. Instead of merely writing a news piece for a grade in social studies class, the Youth Radio journalists write pieces that are broadcast on local and national media outlets. Their writing becomes consequential to a larger community, affecting people outside of their school and reinforcing a new way of being for the journalists themselves.

Erica Halverson and Damiana Gibbons discussed their collaboration with Reel Works Teen Filmmaking. In the class Reel Impact, teen filmmakers work with mentors and learn how to bring their stories to new audiences through broadcast, web-cast, film festivals and community screenings.

Designing such consequential identities for young people to try on is also an important part of my own work on the epistemic game Urban Science. In Urban Science, players role play as urban planners redesigning their city. Through the process of creating land-use plans and justifying their decisions, players explore what it’s like to enact a professional identity with responsibilities to represent a diverse community, while also gaining a new perspective on their own interests in that community. As in Youth Radio, these identity explorations then become consequential when players present revised plans to local planning officials and the mayor. Through projects like Urban Science, Youth Radio, and Reel Works, we can take up the challenge of providing pro-social roles for kids that let them use digital tools to produce meaningful work for an exterior audience-to let kids try on realistic, positive and consequential digital identities.

‘Our system is, more than anything, an artifact of our Colonial past. For the religious dissenters who came to the New World, literacy was essential to religious freedom, enabling them to teach their own beliefs.’

I came across the above quote in a recent article by Matt Miller in The Atlantic. While volunteering in a 6th grade classroom, I got a close look at what passes for ‘essential’ literacy in schools these days.

The students were asked to read pages 79-84 in their Social Studies textbooks and answer questions 3, 4, and 5 at the end of the section. Sound familiar? Here’s the literacy strategy most of the students followed:

1. Read pages 79-84.
2. Read the questions at the end of the section.
3. Search for the answers by using the glossary, skimming for the keywords in the section, or by looking up the keywords in the index.
4. Write the answer, directly as it appears in the text, in a notebook.
5. Read the answer aloud if called on by the teacher.

The questions were mildly interesting (What contributions to medicine and astronomy did the Egyptians make?), but I was confused about what the students were supposed to do with the disconnected facts they found.

So what if the Egyptians correlated the movement of the star formation Sirius with the flooding of the Nile to create the 365 day year? Why should the students care, and how will the information be meaningful to them beyond the test on Friday?

Let’s compare this to a similar literacy activity in the game Urban Science, which was similarly played by middle school students this summer. The students were asked to read a request for proposals that called for redesigning a neighborhood in Madison, Wisconsin. Here’s the literacy strategy most of the students followed:

1. Collect background information on the site and write a memo to their fellow planners
2. Conduct a physical site visit and write a neighborhood assessment
3. Listen to stakeholders’ opinions in the virtual neighborhood and complete a stakeholder assessment
4. Design preference surveys to elicit stakeholder feedback
5. Compile stakeholder feedback into a comprehensive stakeholder assessment
6. Use the information previously gathered to complete a final plan and a final report
7. Present and defend planning solutions to the mayor of Madison, professional planners, and other city officials

What’s striking about the differences between the literacy strategies used in Urban Science and in the Social Studies classroom is not just that these are better literacy strategies. The reason the Urban Science strategies seem like better literacy practices is that the tasks are actually meaningful. In this epistemic game, the search for information is directly related to the problems players are solving. As David Williamson Shaffer talks about in How Computer Games Help Children Learn, the kind of learning that takes place during epistemic gameplay ‘requires that players care about what they are doing. They have to care enough to persist in doing it in the face of obstacles significant enough that overcoming them leads to real learning’ (126). In the epistemic game Urban Science, players perform tasks that are consequential; that is, the tasks have ramifications for future tasks and goals.

In other words, literacy is a set of skills that matter in the world. Traditional print literacy mattered in Colonial times because it was essential to things that people cared about, like religious freedom and the ability to teach their own beliefs. In our digital world literacy is no less essential to freedom. But there are different literacy skills–even different kinds of literacies–that young people need to secure economic, cultural, social, and intellectual freedom.

And the literacy skills that are meaningful in this new century seem to be more a part of a game than they are a part of at least one classroom that I visit on a regular basis.

In How Computer Games Help Children Learn, David Williamson Shaffer describes how epistemic games are designed to do something very different than schools today. Today’s schools were designed, he points out, at the turn of the century to avert social strife in rapidly expanding cities by socializing young people to a new industrial order: essentially, they were designed to prepare factory workers. But we don’t need to train children to work in an industrial society anymore. Instead, we need children to be able to think in creative and innovative ways. And epistemic games are designed to do just that.

Maria was one of twelve students who played the game Urban Science this summer. As part of our study, I interviewed all of the players last month, after they had finished the game and were back in school for a few months.

In her interview, I asked Maria if she had thought about Urban Science during the last three months. She told me a really interesting story about an assignment she completed in school’ how she had thought like a planner, and what happened (or didn’t happen) as a result.

Maria said:

‘In history we had to do like a city planning thing. Everyone else drew little houses and grocery stores, and I did my zoning thing. I did it on grid paper and then had little roads and stuff and had different boxes and they represented different things. There wasn’t enough housing for people, so then I put higher density housing in. And then I put a variety of market rate and affordable [housing]. And then it was funny because I looked at other people’s and they drew houses with chimneys and smoke coming out.’

I was delighted, of course, that Maria’s creative response to this social studies project suggested that Urban Science had helped Maria think about complex urban issues the way planners do. She was considering the needs of the stakeholders and representing changes the way a real planner would.

I couldn’t help but imagine (and hope) that her project might have caused a rich discussion in her class about cities and how they work. So I asked Maria what her teacher said about her project.

Maria replied:

‘Um. She gave me an ‘A’. I don’t know. I just turned it in.’

There were no comments. No reflection. No discussion.

And then the heartbreaker: Hoping to give Maria the feedback she didn’t get from her teacher, I asked her if I could see her project. She told me she had thrown it away.

Maria’s experience made me think again about how different epistemic games are from school. It was a stark reminder that too often the message we are sending our children in school is that their work is little more than a means to a grade.