Valve vs. DOTABUFF: A War of Statistics

There is a war brewing. A war fueled by hate for a community and an obsessive need to control it. Well, that’s one way of looking at it. You could also say that it’s a dumb squabble being overblown by a group who put all their eggs in a basket that just gave out. There are also a hundred interpretations in between that you could argue. It’s a grey area the size of Texas, yet a very interesting story to follow.

Dota 2, the “sequel” to the Warcraft III mod turned international phenomenon, that Valve has been developing over the past few years, has grown into a pretty massive community. Valve recently releases a few statistics saying that the game recently had it’s one hundred millionth match (100,000,000!) and gets over three million unique players each month, while still being technically in beta. Last September they had their second million dollar tournament, The International, which had millions of viewers. So yeah, it’s getting big. Like any big online game, the community is constantly making new websites and programs for the community, to serve different purposes.

Dotabuff is one of those services, and probably the biggest, recently boasting how they get millions page views a day with hundreds of thousand of unique visitors every month. The site provides a detailed collection of the matches you have played. This included kills, deaths and assists for everyone in the match, the items they built, as well as personal records, such as highest gold per minute, most kills, etc. Many other sites also provided this information, such as Dotaholic, but Dotabuff is the most refined and professional looking of those sites. It has come a long way from its simple beginnings with a basic design and search.

The problems started with an announcement of new features that Dotabuff was planning to roll out in the coming days. Some of these features were improved statistic tracking, hero build support and more in depth match details taken from the replays. Some of these features would only be available to Dotabuff Plus members, a paid membership program. There was also a feature called DBR or Dotabuff Rating. This rating uses your personal statistics and places you in a percentile/league, ranging from bronze to diamond. Now, if you played Starcraft 2 at all, this may sound familiar to its ladder system which ranges from bronze to grandmaster. You could also compare it to the Elo system that League of Legends uses, or the Public Skill Rating (PSR) that Heroes of Newerth uses. There is an important distinction; DBR is a third party made rating that is calculated with statistics but has no bearing on the actually matchmaking used in Dota 2, unlike the others I stated. Instead, what determines matchmaking is the Matchmaking Rating (MMR) that Value uses. There is a very big different between the Dota 2’s MMR and everything else I noted, which is that the MMR is a private statistics that not even the player themselves can see.

When Dotabuff announced there was an outcry. The debate of a public rating has been burning for a long time. Over 10 months ago, I remember reading a thread on the Dota 2 development forum about public stats. It was raging on both sides, and 1,410 posts later, nothing was agreed on. On one side, players feel that without a detailed list of statistics they can’t track if they are getting better, or what they should improve. Some also feel that the absence of career statistics are a detriment to the games competitive community.  Heroes of Newerth has statistics like these: total kills, total deaths, average gold and XP per minute. All public, anyone can view it. On the other side, players say it is a privacy issues, that they don’t want people snooping around their history or statistics, since they can be misleading and not really determine the players skill. This is especially true in a game like Dota 2, where it’s more about situational awareness, gameplay knowledge and reaction then it is about your kills to deaths ratio and averages. Heroes of Newerth was plagued by players snooping on their teammates before the match begun, and then trying to tell them what they can and can’t play based on these stats. This player could be the best player in the world, but because they mainly play a support role, which gets little gold and few kills, their stats are misleading others into believing they are unskilled.

So, Dotabuff put up a simple poll. Do you think there should be public stats? Should these stats only be private? Or do you not care either way? The results came in, and public stats was winning having over 50% of the vote, out of ~100,000 votes. The community was still divided though. Many said that it was too small of a response. Other said that it was only taking into account the people who already use Dotabuff and would know about the vote, not the other millions of players who’s statistics would be taken and complied for the public eye without even knowing. There was one important figure that had not weighed in on the issue yet. Valve. They were silent about the issue at this point. This isn’t to say they have never said anything about it. Several testers, moderators and employees read and responded to that thread I referenced earlier. While nothing could be considered an official statement, the general consensus was that Valve didn’t like the idea of public stats.

It wasn’t until Wednesday, January 23rd, that Valve responded in their weekly content update for Dota 2.

– Added a privacy setting in the UI so that players can specify whether they want to allow external 3rd party websites to be able to access their match history (defaulted to private).

A pretty to the point response. Valve is not happy with DBR, and they are willing to shut down third party access if it means keeping players privacy in place.  The next day, Dotabuff finally released DBR. They kept it private, having to log into the site via Steam to view only your DBR. The next day, January 25th, Dotabuff released a statement on their website. You are redirected to this statement if you try to access any feature of Dotabuff. At the top, it says “Valve shuts down Dotabuff“. In the statement, they note that because of two changes to Dota 2, they are no longer able keep updating any features on the site. Those changes are the privacy change that I posted above, and a change to the URL that is used to gather replays. Dotabuff was using said replays to put in a number of new features that dug into the code of the replay to gather information that could be used to create more interactive features, such as heat maps, features that would be impossible with the current WebAPI. Speaking of the WebAPI, they go on to complain that it does not have the features they would need to continue the site, and that it has been inaccessible for 6 months (note: it is currently back up and running according to Cyborgmatt). Then, they posts the email address of IceFrog, the lead developer of Dota 2, a link to Valve employee emails and to the development forum.

Finally, they thank their Plus members for their support and to wait while they try to resolve this issue. I found this part to be the most telling about this entire announcement. Valve didn’t shut down Dotabuff. They didn’t specifically tell them to close their doors. Valve reinforced their privacy policy. Dotabuff was also charging for statistics that Valve obviously doesn’t want to be public, while reverse engineering their replays to gather them. This is what turns this murky grey area squabble into a full blown kerfuffle.

Cyborgmatt, known for his weekly content analysis posts on his website, recently posted an image demonstrating what can be gathered from the current WebAPI that Valve provides.

Compare that to matches on Dotabuff before the changes that they said killed their methods of gathering data.

Names have been cropped out.

At the time of writing, Valve hasn’t officially responded to Dotabuff, but since their development forum has been flooded with threads about it, moderators have started redirecting everyone to a specific thread created by a JoinDOTA administrator, another popular Dota 2 website. In the thread, they stated that with the WebAPI, third parties can still gather data from users with private profiles, but it is done anonymously. Cyborgmatt demonstrated this with a quick program that gathers data in a very similar way

I will continue to update this post as this issue unfolds.

Edit 1 (Jan, 25th): Dotabuff removed the redirect to the statement and the site itself is now accessible, but claim they are not able to get any new data. The statement remains unchanged.

About Addison

Addison is a bit of a jerk. His favorite game is The Legend of Zelda: Majora's Mask. This is really all there is to him. Very hollow, very boring.