I finally broke down and spent the money to get the audio from this presentation, and thought I'd give a summary plus some additional info that I dug up after listening.: "Analyzing Virtual Worlds: Next Step in the Evolution of Social Science Research". American Association for the Advancement of Science (http://www.aaas.org/). 2009 AAAS Annual Meeting February 12-16, 2009, Chicago Illinois. The research team analyzed three years of data (over 60 terabytes) from the complete server logs and click-streams of SOE's MMORPG game, EverQuest 2, tracking every action performed in-game. EQ2 has more than 300,000 players who average 26 hours per week playing the game. Because of the intense level of involvement and multi-player environment of the game, the researchers were able to study human behavioral dynamics using the game as a proxy. Dmitri Williams -- University of Southern California Noshir Contractor -- Northwestern University Jaideep Srivastava -- University of Minnesota Scott Poole -- University of Illinois, Urbana-Champaign Sponsors: National Science Foundation, United States Army, Sonic Lab Northwestern University, Sony Online Entertainment, and others. The research team was quite varied. Aside from the primary investigators, specialists in other disciplines such as economists and anthropologists were also involved. Database analysis was handled at the National Center for Supercomputing Applications at the University of Illinois, Urbana-Champaign. Past studies have relied on surveys and self-selected samples. This project involved actual server data from EverQuest 2. It's estimated that in the Western world (ignoring Asia), there are about 40-50 million MMORPG player accounts, mostly fantasy RPGs. Some areas of interest for study are how (or if) the virtual world mirrors real world human interactions. For example, networking behavior, economic behavior, how groups work, how people talk, the way people get into conflicts, the way people learn, etc. Mapping Principle: when do behaviors in a virtual space act as a proxy for behaviors in real life interactions? In some cases, the results from virtual actions map closely to observed real world results, while in others the results do not correlate well. It's not scientifically possible to say with certainty how accurate the mapping is without further study. Prior work in this space was primarily ethnographic and survey-based research. This approach has problems due to self-selection and also difficulty in generalizing results. The current project allowed the researchers to work with a huge body of data. This was also correlated with in-game surveys of random players, allowing some interesting depth of insight. Who plays how much and why? It's not kids, not pasty white teenage boys in their mother's basements. This study found the average players were adults in their thirties, there are more players in their 30s than in their 20s. The age range gravitates to older audiences. EQ2 players tend to play a lot of hours per week. Older players tend to play more hours per week than younger players. Hours spent playing were less than average hours of TV watching per week, and gaming tends to displace consumption of other media such as TV. Players don't watch less news, but they do watch less entertainment television and go out to movies less, replacing their entertainment consumption with gaming. An unexpected finding was that players tended to be healthier than the general population, as measured by their Body Mass Index (BMI). They tended to be slightly more depressed, but less anxious. The healthiest participants in this sample were the older women. This led to interesting thinking about gender differences in gaming. Surveys revealed that while women were in the minority of players, they tended to be the most intense players, which is counter-stereotypical. The women were more likely to be bisexual, more likely to be disabled, and were a little bit older on average than the men. Researchers have fairly strong notions of gendered behavior in a society, which is to say that they acknowledge both nature and nurture, and realize that people's behaviors are shaped by gender roles into which they are socialized. Scientists realize and recognize that there are roles and expectations of types of behaviors that people feel they must stay within, and therefore they tend to expect women to behave certain ways and men to behave differently. The research team in this study made hypotheses about these behaviors and found that, in fact, men are indeed the majority of players in this space, which was not surprising, because frankly, these games are mostly made by men, for men. However, when the researchers looked at who the "hard-core" players are, as defined by the intensity of play and the attitude towards play, the hard-core players were the women. Women play more hours per week (four more hours per week), and are much less likely to quit a game. Women were also much higher on measures of satisfaction and interest in the game than male players. When researchers looked at the players playing at the very high end, the players playing the largest amounts, the women are much more highly represented than they are in the overall player population. This led to an interesting side-finding with some implications for social science research and this area in general, which is the difference between how much time a player self-reported that they played, versus how much they actually played as reflected in the server logs. When people were asked how much they play, they estimated between 24 and 26 hours, but they actually played 25 to 29 hours per week. This is interesting and notable for two reasons: (1) It suggests that the past 30 years of scientific estimates of game play in prior studies relying on self-reported estimates are incorrect, and may be off by as much as 10% if these findings are generalizable, and that marginal findings in this research are suspect. (2) The women clearly have a bigger gap in these two forms of data than men do, which suggests that women are either lying to themselves or lying to researchers moreso than the men are, which may confirm that these are gendered behaviors and gendered expectations. If video games are, in fact, "for boys", then you would expect women to underreport more, as there might be a social stigma or shame value attached. The observed behavior seems to fit this explanation. Men and Women Playing Together Another side finding is what happens when men and women or boys and girls play together. When men and women play together, the women are happy and the men are less happy. When they play apart, the men are happier and the women are less happy. Thus, what's good for the goose is not good for the gander. The women like to be playing in relationships, the men are less happy when doing so. Roleplaying Another interesting finding came when looking at roleplayers. These are "Massively Multiplayer Online Role Playing Games", which suggests that people are pretending to be something they are not, taking on the "role" of a character they are playing. In fact, the study found that relatively few people actually participate in roleplaying. A "moderate" level of roleplaying might be 15% of the players. When people are asked who is an "intense" roleplayer, that tended to be about 5% of the players, and these people are very, very different than the rest of the players. The roleplayers play for different reasons, and seem to have different outcomes. Why roleplayers roleplay, what do they get out of roleplaying, the kinds of identity issues, these are all rich avenues for future investigation. The roleplayers are psychologically much worse off than non-roleplayers. They tend to be more depressed, they tend to have been diagnosed with more ailments and illnesses, they are more likely to be physically disabled. Gameplay does not make them this way: interviews with these players tends to reveal use of gaming and roleplaying as coping mechanisms for dealing with pre-existing problems. It is hypothesized that roleplaying is an adaptation for people who are less accepted in real life spaces, and who go into gameworlds to be "who they truly are" or who they want to be. Investigating the demographics of roleplayers, intense roleplayers were more likely to be non-heterosexual, non-mainstream-religion, and minorities. The further out of the mainstream population they were, the more likely a player is to engage in roleplay. This suggests that the game space may be a little less judgemental and more accepting. Economics Economist Ted Castronova of Indiana ("Virtual Worlds: A First-Hand Account of Market and Society on the Cyberian Frontier"). Looking at the EQ2 data, Castronova investigated whether people behave "rationally" in the in-game economy, as they would be expected to do in real life. They investigated whether they could at least make predictions about basic economic aggregates, based on data culled out of the logs. The first step was calculating the GDP of one of these worlds, and also calculated price levels. They discovered that these were, over time, fairly steady, but could be a bit more volatile than normal non-economic-crisis real world trends. The Quantity Theory of Money (the theory that money supply has a direct, positive relationship with the price level) does apply in Norrath as it does in real life. Networks in Virtual Worlds Noshir Contractor investigated networks in virtual worlds. During the session, an article from Science magazine was passed out discussing how, with the advent of virtual worlds, computational social science can be performed on an unprecedented scale (David Lazer, Alex Pentland, Lada Adamic, Sinan Aral, Albert-Laszlo Barabasi, Devon Brewer, Nicholas Christakis, Noshir Contractor, James Fowler, Myron Gutmann, Tony Jebara, Gary King, Michael Macy, Deb Roy, and Marshall Van Alstyne. "Computational Social Science". Science 6 February 2009: 721-723.). EverQuest 2 is a multi-dimensional network, in which some of the nodes are people, some are documents, etc. There are different kinds of relationships, chatting, trading, buying, selling. It's of interest how these various nodes and structures are co-evolving and what kind of structures they lead to. To perform an analysis of networks in EQ2, the team investigated motivations. Why did people create links with someone else? Why send a chat message? Why would a player want to group with others for quests or other excursions? Why trade with someone? Why send someone in-game mail? Some of this is based on self-interest: I need something you have and I am going to economically maximize my own goal. Another could be based on social exchange: I want something from you, you want something from me, we have complimentary expertsie that brings us together. A third is that neither player wants anything from the other, but that together they have a better shot at beating or killing a certain monster, the collective action or mutual interest. A fourth is contagion, in which a new player wants to connect with those who are already really well connected, those who are rich in network links are going to get richer in such links. Fifth is balance, in which I want to play with someone not because they're particularly good at it, but because they're a friend, or a friend-of-a-friend, and I want to be able to keep balances between my friendships. Sixth is homophily, in which a player seeks others that are the same age as themselves, or the same gender, or other attributes that are common to me and that provides the networking motivation. Seventh is proximity, in which a player wants to play with others in their geographical area -- although you'd think that geographic proximity wouldn't matter in a virtual world, they discovered that it does have an impact on who plays together; or electronic proximity, in which players who have both been in the game a long time interact; temporal proximity or cohort effect in which players started playing at the same time and as a result have more connections with each other. From a network analysis point of view, each of these motivations has a unique structural signature. In a real world, messy network, researchers can use statistical tools to extract the extent to which each of these structural signatures is present in the network. For example, one could determine that a given network's exchanges are based upon 10% self interest, 15% social exchange, 20% by homophily, etc. This allows the researcher to decompose the network and understand the motivations that are driving the network. People playing EQ2 may be interacting both virtually and also within a shared physical space, and things that happen in the online world may spill over into being upset with the other in the real world also, or vice versa. It's been theorized that virtual worlds would expand our ability to interact with people widely, but we also see that it acts in some ways as an extension of the regular offline world and its interactions. Types of Relationships in EverQuest 2 Interpersonal interactions/Partnership interactions, in which two players come together to play together in a combat activity. What motivates two players to interact with one another? Why do two players use instant messaging within the system? Transactional interactions. What motived two players to trade with one another, which requires a "face to face" meeting in the game world? Why does one player send in-game mail, which may be just a message or may include an item? Network analysis was performed by looking at one week of activity on the Antonia Bayle server: Player age range from 5.6 to 71 years of age. IM was a very small network, trade was the largest. The overall structure of the network was quite similar to offline networks. Hypotheses: Individuals are not likely to engage in interactions randomly in the virtual world. The study confirmed this hypothesis. There individuals that are "hubs" in the network, individuals who are more likely to engage in interactions. The study determined that people who were "hubs" engaged much more in partnerships and chat, but less in trade interactions or in-game mail. Friend-of-a-friend, in which individuals who both interact with the same third party are more likely to interact with one another. The study determined this was the case in EQ2, especially for partnership and chat interactions. Those who have greater real world geographical proximity are more likely to engage with others from their real-world geographical area online as well. Counterintuitively, people who are close by (10km) are 5x more likely to interact in game than those far apart (100km or more). This may be due to the fact that these people have offline networks, such as being real-world friends, which induces them to play together more in-game. Those with greater real world temporal proximity, that is, those in the same time zones, will interact more. Players in the study were 1.5 times more likely to play with others in their own time zone. Researchers also examined homophily interactions, seeing whether people were more likely to play together based on gender, age, or in-game experience. There are not good age cues in-game to let people know the real life ages of other players, so age was not a factor in the formation of relationships. People of the same gender also were not more likely to engage in interaction. Females have a strong tendency to interact with male partners, but a negative female to female motivation, even when you control with the gender disparity overall within the game. Network structure data may be able to help SOE to identify illegal plat sales, and also to boost player retention. Large Dataset Data Mining Jaideep Srivastava is an engineer and computer scientist interested in data mining and machine learning. Large volumes of data provides challenges in analysis. From an algorithmic and computational basis, large dataset data mining is an interesting challenge. In trying to understand behavior using the EQ2 dataset, focus was placed on how items interest to the marketing world, such as customer churn, could be applied to virtual worlds. There is an opportunity to observe the player behavior down to every click, and from that there's a chance to understand behavior from that at a higher level with great accuracy. The new environment creates changes in behavior. For example, in the real world people do not often abandon a full shopping cart, but this is not at all uncommon at virtual retailers such as Amazon.com. Behavior online changes based on the technology and new modes of the new environment. Customer churn has been extensively studied in real world interactions by ISPs, phone networks, subscription-based services etc. because the cost of retaining an existing customer is far less than acquiring a new one. Churn analysis is focused on identifying customers who are likely to churn, and determining how to retain those customers. For SOE, it's of interest to determine who are "key" players, because a person who is an "influencer" or "hub" who leaves to go to another game may take all their friends with them. For example, a popular game such as World of Warcraft can suck population from smaller games such as Lineage and Lineage 2. The study looked at determining whether a given player was likely to churn or not likely to churn. Traditionally, marketing research into churn has looked only at each individual in isolation, because they don't have access to their networks to understand well the social influence of one person on another, although cell phone companies have started to be able to model such data. It's quite valuable to understand how interactions between players affect subscription and retention decisions. In examining the EQ2 dataset, researchers were able to look not only how much a player played in the previous week or quarter, but also who that person played with. The amount of time a person plays correlates with their engagement with the game. With this large a data set, multiple analysis passes through the data are not feasible. And most networks are not greatly scaleable.
Some other interesting articles that came out of this research: Kyong Jin Shim, Richa Sharan, and Jaideep Srivastava. "Player Performance Prediction in Massively Multiplayer Online Role-Playing Games (MMORPGs)". February 04, 2010. "This study uses performance data of game players in EverQuest II to build performance prediction models for game players. The prediction models provide a projection of player’s future performance based on his past performance, which is expected to be a useful addition to existing player performance monitoring tools. First, we show that variations of PECOTA and MARCEL, two most popular baseball home run prediction methods, can be used for game player performance prediction. Second, we evaluate the effects of varying lengths of past performance and show that past performance can be a good predictor of future performance up to a certain degree. Third, we show that game players do not regress towards the mean and that prediction models built on buckets using discretization based on binning and histograms lead to higher prediction coverage." Kyong Jin Shim, Muhammad Aurangzeb Ahmad, Nis_hith Pathak, Jaideep Srivastava, "Inferring Player Rating from Performance Data in Massively Multiplayer Online Role-Playing Games (MMORPGs)," Computational Science and Engineering, IEEE International Conference on, vol. 4, pp. 1199-1204, 2009 International Conference on Computational Science and Engineering, 2009. "This paper examines online player performance in EverQuest II. The study uses the game's player performance data to devise performance metrics for online players. We report three major findings. First, we show that the game's point-scaling system overestimates performances of lower level players and underestimates performances of higher level players. We present a novel point-scaling system based on the game's player performance data that addresses the underestimation and overestimation problems. Second, we present a highly accurate predictive model for player performance as a function of past behavior. Third, we show that playing in groups impacts individual performance and that player-level characteristics alone are insufficient in explaining an individual's performance, which calls for a different set of performance metrics methods." B. Keegan, M. Ahmad, J. Srivastava, D. Williams, N. Contractor (2010). Dark Gold: Statistical Properties of Clandestine Networks in Massively Multiplayer Online Games. Proceedings of IEEE, SocialComm-10. "Gold farming is a set of illicit practices for gathering and distributing virtual goods in online games for real money. Using anonymized data from a popular online game to construct networks of characters involved in gold farming, we examine the trade networks of gold farmers, their trading affiliates, and uninvolved characters at large. Our analysis of these complex networks’ connectivity, assortativity, and attack tolerance demonstrate farmers exhibit distinctive behavioral signatures which are masked by brokering affiliates. Our findings are compared against a real world drug trafficking network and suggest similarities in both organizations’ network structures reflect similar effects of secrecy, resilience, and efficiency."Ratan, R., Chung, J., Shen, C., Poole, M. & Williams, D. (2010, in press). Schmoozing and Smiting: Trust, Social Institutions and Communication Patterns in an MMOG. Journal of Computer Mediated Communication. "This study examines how trust is related to online social institutions, self-disclosure, mode of communication, and message privacy in Everquest II. The findings, based on survey and behavioral data from over 3,500 players, illustrate how MMOGs may support trust-development. Trust was higher within closer social circles: trust was highest in teammates, followed other players across the game, followed by others online. Self-disclosure was positively related to trust of teammates and others in the game, while voice chat was only related to teammate trust. These findings indicate that social structures and communication processes contribute to trust development in MMOGs, supporting the claim that these online spaces provide social support that is unavailable in other realms of our society." Williams, D., T. Kennedy & R. Moore (2010, in press). Behind the Avatar: The Patterns, Practices and Functions of Role Playing in MMOs. Games & Culture. "The data here—a combination of quantitative and qualitative—come together to paint a picture of the role playing population as a small, vibrant, and unique class of game players. Indeed, these players are often playing their own game, largely independent from the other players and the larger world they populate. Knowing that this population exists and is different, future work should consider them separate from other players. Studies of virtual worlds would do well to consider controlling for role playing, or engaging in sub-analysis. Folding them into the general population would skew many findings, starting at least with the demographic and psychosocial differences shown here, but possibly extending into other measures."Williams, D., M. Consalvo, S. Caplan & N. Yee. (2009). Looking for gender (LFG): Gender roles and behaviors among online gamers. Journal of Communication. 59, p. 700-725. "Several hypotheses regarding the importance of gender and relationships were tested by combining a large survey dataset with unobtrusive behavioral data from 1 year of play. Consistent with expectations,males played for achievement-oriented reasons and weremore aggressive, especially within romantic relationships where both partners played. Female players in such relationships had higher general happiness than their male counterparts. Contrary to stereotypes and current hypotheses, it was the female players who played the most. Female players were also healthier than male players or females in the general population. The findings have implications for gender theory and communication-oriented methods in games and online research—most notably for the use of self-reported time spent, which was systematically incorrect and different by gender."Williams, D., N. Yee & S. Caplan (200. Who Plays, How Much, and Why? A Behavioral Player Census of Virtual World. Journal of Computer Mediated Communication. "Online games have exploded in popularity, but for many researchers access to players has been difficult. The study reported here is the first to collect a combination of survey and behavioral data with the cooperation of a major virtual world operator. In the current study, 7,000 players of the massively multiplayer online game (MMO) EverQuest 2 were surveyed about their offline characteristics, their motivations and their physical and mental health. These self-report data were then combined with data on participants’ actual in-game play behaviors, as collected by the game operator. Most of the results defy common stereotypes in surprising and interesting ways and have implications for communication theory and for future investigations of games."
Amazing write-up! I will poke through it a little later when I'm off the clock. (Oh, who am I kidding? I'm never off the clock, hehe.) I worked with Mr. Williams from USC when I was in Marketing (just by means of helping him with accounts; our research specialist actually organized our partnership with him), and he's always been very eager to include SOE games in his studies. Very nice man.
I used to play paper and pen D&D and I roleplayed properly when playing that, however I dont have all the ailments that are described in the article. When I moved onto MMO's I roleplayed for about one minute before I realised I looked like a complete idiot and learnt the lingo in chat channels. The very few occasions I've met an RPer in EQ2 I've generally backed away slowly making soothing noises. It creeps me out like online marriages or the Temple of Psychosis that is second life... *shudder*
Guy De Alsace wrote: RPing in an MMORPG is no different from RPing pen-and-paper. I'm sick of bigoted statements like yours that seek to further marginalize a perfectly acceptable pastime. Powers &8^]
Powers wrote: Just going by my stay on AB. Sorry, but thats the experience I've had. I got a Bruiser to 73 on there before I went back to Runnyeye. Not saying all RPers are weird but damn, theres RPing and then trying to get off with my toon continually - following me around and being thoroughly annoying...and thats just one example.
Guy De Alsace wrote: That's when you turn around and get nose-to-nose with them and /shout "Forsooth! Who art thine sire? Doth thou wanteth some more? Bringeth it oneth! Thou shalt get mine smackethdown!" They will assume you are completely insane and slowly back away making soothing noises Seriously tho, that's a very interesting study -- definately gonna have to take the time to read up on it!
Aesome@Kithicor wrote: umm .... yes.... quite a few and why that is in any way a question boggles me.
The section on roleplayers doesn't surprise me, to be honest. There are plenty of people with serious, serious issues out there, and they lurk in the dark corners of the community. I've been on AB for five years now, and have met plenty of people who have serious, serious issues. I don't roleplay much (though I do try to communicate in a way that's appropriate for Norrath, so as not to ruin anyone else's immersion), because it seems like there's a subset of the RP community that are 'touched' as my father would say. The idea of roleplaying, and being able to basically write a story with other people, is certainly interesting. Having to put up with the agoraphobic, bipolar, borderline personality 400 lb person who makes up lies about people just for attention sort of puts a damper on that interest, though. No, I'm not making that up. That's a personal experience.
umm .... yes.... quite a few and why that is in any way a question boggles me. - I guess you had to be there....Much love to Ryoko - Yes I was aware of the fact that the fairer gender plays mmo's. The reference was kinda obscure - Once upon a time in a land far far away there lived a troll who loved spark heated debates about whether women played mmo's or not, he went by the name Ryoko...Normally a person trolling with Ryoko's bait would quickly be dismissed and added to the ignore list....The problem though was that he was very good, and was highly sought after for both his expertise and companionship in virtually any raid....So he was allowed to troll- Good story? No of course not, but reading the portions of the report discussing women and their being more intense players struck a chord of reminscience and while it was an impulsive post I had fun.
May I humbly suggest, for those interested "What Video Games Have to Teach Us About Learning and Literacy" by James Paul Gee. I'm an instructional technologist in the public schools, and one of my primary objectives is the integration of gaming and simulation into curriculum and pedagogy to enhance and improve student learning. I recently taught a unit on the US Government Executive and Legislative branches using SimCity 4 as the simulation engine, for example. There is a LOT of potential for this field, and as such it is one of my primary research areas of interest in my work. A lot of profound implications about interpersonal communication, human sociology and behavior, and the mind to be found in the study of gaming.
