This past week I was hot on the Meetup circuit here in Silicon Beach and I decided to take in a presentation “Data Science @ Activision“. Activision is the publisher of the famously popular video game “Call of Duty.” The company has multiple analytic teams. The talk included a detailed overview of data science at Activision and provided some additional detail on two of their analytic groups: the Game Analytics Team and the Marketing & Advanced Analytics Team.
The event took place at Cross Campus, a co-working, office space, and events venue in the heart of LA’s start-up community. Cross Campus has adopted a couple of the local meetup groups including the host of the Activision event – LA Machine Learning, and also the LA Big Data User Group. Cross Campus is the perfect place for tech Meetups, being a hip, tech-centric destination with a really great vibe. My friend Dan Dato, one of the Cross Campus founders, has done an excellent job creating this resource. Some really tasty hot food was catered by Activision, as well as a bottomless keg of cold beer. I had a tough time concentrating 100% on the talk because Cross Campus had a wide screen TV tuned into the Kings v. Blackhawks game 5. I put on my multitasking hat and enjoyed both attractions (the Kings lost in 2OT). So let me see – data science, cold beer and hockey? I was in heaven.
- Analyzes the behavior of the game and the players to improve game design, in-game algorithms and the player experience.
- Builds out analytic services to support the game. One example is a cheating detection system which is very similar to fraud detection systems used for credit cards and insurance.
This part of the presentation was by Josh Hemann, Principal Statistician in Activision’s Game Analytics Team, where he builds analytic services that support video game development studios. His industry experience in analytics spans diverse settings such as oil and gas exploration, aerospace, retail loyalty programs, recommendation systems for grocers, and massively multi-player on-line games. Josh has an MS in Applied Mathematics from the University of Colorado at Boulder where he maintains involvement in air pollution research.
Hemann described what his department does by providing a couple of vignettes. “Vignette 1: Algorithm Detection of Assholes.” Some players found a shortcut to raising their rank in Call of Duty known as “boosting.” Two players enter a game on opposite teams and take turns killing each other. This quickly improves their rank. At first, most of the “boosting” detection had to be done manually with Activision players surveying random games. However, the team now uses algorithms and data patterns to determine how quickly the player jumped in rank, the number of times they killed the same player, the number of games with the same player, the player positions on the map, etc. Historically, the number of cheaters that the company has been able to identify was a couple of hundred per day. Now it is several thousand.
The team’s second improvement came from customer feedback and was called “Vignette 2: Natural Language Processing.” This involves running algorithms against social networking sites such as Reddit to learn about their audience’s feedback.
We get more than a terabyte per day of data, and data volumes are often heaviest at launch,” says Hemann. “But, it can be hard to confidently see trends in the data over the first few weeks when people play these games for multiple years. The idea of using data from sites like Reddit is to complement the telemetry data we get which can be especially useful in those first few weeks after launch. We of course conduct a lot of player surveys and traditional consumer research as well, and we are using statistical methods to make social media one more instrument to help us design better games.”
The Marketing & Advanced Analytics Team:
- Uses gamers’ play data to optimize marketing communications across channels.
- Builds Personalization Engine Rules for 1:1 communications with individual gamers.
- Predicts gamers’ likelihood to churn or to respond to up-sell offers.
The second part of the presentation was by Mae Coughlin. After her PhD in marketing at Stern School of Business, Mae Coughlin has spent over 10 years working in database marketing. Her experience spans across financial services, banking, technology, consumer electronics, software and most recently interactive entertainment. Prior to Activision she built and led analytical teams from both the marketing agency and client side in the US and in China.
Coughlin started the presentation with a song and some visuals. She showed how their marketing data affects sales decisions. For example, graphs included time played vs. retention or churn vs. amount of DLC (downloadable content) purchased. The data was so specific that Activision could estimate the monetary value for one hour of play for one person.
The team is also working on other projects such as a personalized 1:1 targeting email strategy to reduce churn. Novice players receive encouragements, while expert players learn about new DLC or “Double XP” weekends. Then, the team can see if those who received emails decided to continue or increase playing Call of Duty. Currently, the number of unique email templates ranges in the 6-figure mark.
One thing that was pretty clear from the Activision presentation is that they are HIRING. Both presenters repeated that theme over and over again, sometimes to the chagrin of the audience. Yours truly, at one time earlier this year, was interviewed with Dr. Coughlin, alas, I took another position, but Activision seems like a swell place to work. How bad can it be, they “require” you to play Call of Duty at least 10 hours per week as part of the job!
Sign up for the free insideBIGDATA newsletter.