Role Selection: Max Seidman and Dr. Gili Freedman – Tiltfactor

Role Selection is an interview series in which we chat up folks who work, live, and play board games in a variety of ways to learn about the roles in the hobby they’ve chosen.

The Interview:

Dr. Gili Freedman: The one we’re doing tonight is about climate change. We have one game that we’re researching that’s a digital game looking at climate change, and now we’re doing the non-digital one, which is a party game that is a – what’s its relationship to “Cards Against Humanity”?

Max Seidman: Unofficial expansion.

Gili: Thank you. And some of the cards involve climate change ideas or phrases or things like that. Some people are going to be playing that version, and some people are going to be playing a neutral version, and we’re going to look to see if that actually changes their attitudes and behaviors toward climate change.

For example, we’ll give them a questionnaire about the environment and whether they should recycle. We’ll also covertly measure their recycling behavior at the study, so we’re giving them water bottles to see what they do with them.

 

Matt Golec: So that won’t be explicitly explained, but you’ll be observing their water bottle behavior.

Gili: Yes. And we have IRB (institutional review board – a committee that oversees behavioral and other types of research projects) approval and all that to do these things. People consent to the process. It’s like any other study; it just happens to take place in a pub, and it’s a game instead of filling out a questionnaire.

 

MG: Tell me of a couple Tiltfactor projects that you’ve really enjoyed or think worked well.

Max: To me, one of the most interesting things about the research we do…there are two parts to it. The first part is, ‘is the game doing what we want it to be doing’? And the second part is, ‘how can we study this game to generalize for all people who want to make impactful games’? We really think that every game is having an impact on its players, right? So better to know what it’s doing than not know what it’s doing.

A great example of this is Awkward Moment. Awkward Moment plays a little bit like Cards Against Humanity or Apples to Apples – it was actually developed at the same time as Cards Against Humanity, so we both ripped off Apples to Apples. It’s part of our Bias Project to combat gender bias in science, technology, engineering and math (STEM).

The way the game works is, most of the moments are awkward things like, you see your friend coming down the street toward you so you wave like crazy and call her name, and it turns out it’s not her. Everyone has done things like this.

MG: Sure

Max: Everyone submits a reaction card from their hand, and the decider chooses the funniest or most appropriate. I really like ‘fix your hair’ for that particular one.

(Max then demonstrates a wave turning into patting the side of his hair as Matt and Gili laugh)

Max: A minority of the cards – somewhere between 20 and 40% – have something to do with gender bias, or gender bias in STEM, in some way. They’re usually pretty light, they’re things like, ‘you’re in a mall and you see a store selling T-shirts for girls that say ‘Math is Hard.’’ This actually happened and is ripped from the headlines, right?

So we wanted to know if this game was combatting gender bias in STEM. We did some studies with middle schoolers where they played the game or a neutral version of the game. Afterwards, we said, ‘Here are six characters in an upcoming game.’ Then we gave them six faces. They’re neutral faces, three men, three women, but we don’t know which professions should be matched to what character, and one of the professions was a scientist. We wanted to know how often they matched the scientist to a woman.

If they didn’t play Awkward Moment, they matched 20 percent of the time. If they did play Awkward Moment, it was 60 percent. That’s a big jump. That’s precisely what we were looking for. So we knew the game was working, but what can we generalize from that?

So we did a follow-up study on a hunch where we overloaded the deck with biased moment cards, so instead of 30% it was 90%. And as you might expect, they had less fun with the game because they could figure out what it was supposed to be doing. But as you might not expect, it was also way less effective. I think all the effects were wiped away. So this is kind of a coup in the games-for-change space, which is really ‘hit you over the head with what the game’s supposed to be doing and more content equals better for them’. It turns out that in this case, more impactful content was actually worse.

MG: Because people figured it out and then shut off?

Max: That’s what we think is going on. They were putting up barriers when they know we’re trying to change them. And we’ve seen very similar things in a bunch of other games including Buffalo and Pox. This is one of the most interesting things we’ve found so far: That you can make impactful games without players knowing, and it’s more effective.

 

A Day In The Life Of A Scientist

MG: That’s very interesting. Gili, tell me what a typical research day looks like at Titltfactor.

Gili: It varies a lot. Today is a strange day where we’re doing a lot of prep for the study, so really mundane things like getting the cards sorted and figuring out if the water bottles are labeled. But usually it’s a lot of brainstorming, looking through previous research, figuring out where the gaps are in the research and what we can fill with our ideas. A lot of reading, as you might imagine!

Also, Max and I talk a lot about what games should look like and how we can get them to meet both of our goals, so they’re fun and impactful. I talk with the team – we have students who work with us; we make sure they all have something to do and that they’re involved in the projects in a meaningful way.

 

MG: How much of your time is spent writing up the research or doing the IRB applications? Do you do research grants, or does Dartmouth cover that?

Gili: Correct me if I’m wrong, Max, but the lab is funded by grants. I haven’t yet written a grant because I’ve only been here a month, but we are starting a new grant very soon. So that will take up a lot of time.

We are hoping to submit the card game we were just talking about to conferences, so we’ll be writing up those. Before you got here, I was writing up an IRB amendment. So definitely a lot of writing once we get things going.

 

MG: You’re not just sitting around, making up games. There’s a lot of work to be done to keep it on the straight and narrow…

Gili: Yes, a lot of background stuff to do.

(Max then brings over a water bottle.)

Max: This is one of the things we’re preparing for [tonight’s] study. Somebody recycles this water bottle. How do we know who recycled it?

(Max shines a black light on the bottom of the bottle, revealing the number 4.)

 

MG: Wow! That’s cool. So what does the ‘4’ mean?

Gili: It’s their ID number. We don’t want to collect any identifiable data, because we don’t want to have anybody’s names that would violate privacy and confidentiality. So everyone is assigned a number that’s not connected to their name in any way. Later, when I have all the bits of data, I can connect them together.

(Gili points to the black-light number)

That’s brilliant!

Max: We’re going to do this on the tokens, too! (laughter)

 

 

Failures, Data, And Societal Change

MG: Max, one of the research projects I saw on the website that looked interesting to me was about three failures – games that didn’t work out. I’m curious to hear what about those games didn’t work out, and what did you learn from that?

Max: Game design is all about failure, right? It’s something you have to be comfortable with. One of the best things about being grant funded at Tiltfactor is we do have a little bit of freedom. If we make something that’s not working, we can say it’s not working and abandon it. I feel like some places in the games-for-change space that are more commercially minded, if they make a game that’s fun but isn’t working, they might be tempted to publish it and say it’s working anyway.

So those particular games [from the study] were all part of the bias project, which Awkward Moment and Buffalo were out of as well. For every game that we finished, there are at least two prototypes we abandoned, and at least 5 or 10 concepts. Those failed games were ones we brought to the prototype stage and were either not fun, not effective, or not efficient enough. We’re a small team, so we can’t make games that are huge.

There was one game we developed, Skyline, because there’s a lot of research that shows one of the barriers for retaining women in engineering is low spatial-reasoning skills coming into college.  And one of the really interesting things about that is it’s easily trainable. We thought we could do a spatial reasoning game that helps with mental rotations. So we made this fun game. People were having a good time with it – we brought in a bunch of middle schoolers who it was targeted at, and they had a really good time. We brought it to a bunch of festivals where people enjoyed it. Then we did a bunch of studies to see if it was actually training people in spatial reasoning, and it wasn’t.

So we’ve got that game. We’re going to do something with it at some point, but it’s not going to be attached to spatial reasoning.

 

MG: That’s the thing you guys do. You have the studies to back up your games to see if they’re really working.

Max: There’s lots of people out there who say, ‘Scrabble – it’s teaching spelling.’ Probably it is, but who knows? Somebody may know, but I haven’t see the study.

 

MG: Tell me about why is this all so important. What does this work mean to society?

Gili: That’s a big question. We’re trying to foster social change. We’re trying to see the problems going on that can be addressed by games…and games are a really great medium for getting people to engage with an issue.

The reason I think the work going on in this lab is so important is because a lot of the interventions that are out there are really obvious, and the people they’re going to work on are the people who don’t need it. You’re preaching to the choir a lot of times, right? Who’s going to go to a thing where they’re talking about climate change? People for whom that’s already a concern.

But a lot of people already play games. This is something that ties people together, ties families together, and communities. So you can really get in there and change people who wouldn’t normally be changed by these things because they wouldn’t go looking for them. I think it’s a really nice way to get at problems that are difficult to get at, and where we don’t know what to do.

If you look at the news right now, bias is huge. Every article you’re seeing on the front page is about bias in one way or another. Everyone’s going to try to come up with methods to fix this problem that is tearing the country apart in some ways.

The goal of this lab is we can do it in this really nice way and they’re going to have fun while they’re doing it, and they’re not going to think we’re trying to target them or change who they are. That’s not the point. We’re trying to make people engage with each other in a better way.

A night out the pub won’t slow down Tiltfactor’s upcoming efforts. They’re working on the Bystander Project to combat campus sexual assault; one of their climate change games will be releasing soon; and they’re studying the game Cards Against Humanity to see if its extremely adult-themed cards cause racism or other social harm, as some people have suggested. They’re still keeping a lid on the results, but the word is: “We found some really interesting stuff.” 

For more information about Tiltfactor and a list of their games, visit their webpage. There you can also check out their social media links and mailing list for up-to-the-minute updates, as well as opportunities to participate in playtests.

Matt Golec is game designer with a background in print journalism. Combining these skills, he aims to explore and give voice to the many different jobs within the hobby industry that don’t frequently get reported on. He can be best reached via Twitter.

You can discuss this article and more on our social media!

 

Photo Credits: Multiple Photos from the Tiltfactor site and their Facebook page.