Introduction
Reprising all the relevant facts and accusations involved in what has become known as "the Driv3r review-buying scandal" would, I feel, take up too much time and space for this blog. If you are totally new to the situation, I encourage you to start with this (surprisingly moderate) Spong article to get an overview, then click some of the links above if you want more source material. The rest of this article assumes knowledge from those sources.
Given the huge implications this "scandal" could have on the video game journalism industry, I've been doing a lot of reading and thinking on the matter over the past weeks. That being said, I'd like to preface my remarks by saying that this post is entirely my opinion and is based on the same publicly available information everyone else has access to. It is not legally or morally binding in any way, and other opinions on the matter are certainly possible.
Why I'm Not Convinced
Based on the above-linked sources and others, I've come to the conclusion that allegations of outright review score buying or inducement on the part of Atari and Future Publishing are circumstantial at best and unsubstantiated at worst.
First of all, while the original message board accusations claim to be from video game journalism industry insiders who are in a position to know about the "scheme," there is absolutely no way of proving this. This is the internet, and the remarkable ease of lying and/or misrepresenting yourself on a message board can not be overstated. This is a subject I gained first-hand experience with when a person claimed to be me on the SMBHQ chat room and managed to do some damage to my online character. Anyone could have made these accusations for any number of reasons, the truth quite possibly not among them.
Secondly, the message board posters that broke this scandal offer absolutely no definitive evidence -- outside of their statements -- to support their claims. There are no scanned contracts, no recordings of illicit conversations, no (legit) photos of envelopes full of money, or anything of the kind. Not that the lack of such evidence automatically exonerates the parties, but it does make the accusations seem a little hollow when they have no documented support.
The flimsy "evidence" that does sometimes get cited -- such as the high review score itself, the omission of gameplay bugs from the review, and a sticker advertising Future's scores on the game packaging -- is highly circumstantial and a long way from a standard of proof that would satisfy me.
But the most important reason I lean away from any foul-play interpretation of the facts is that there are other sources that agree, or come close to agreeing, with the 90% rankings of PSM2 and XBW. Smaller sites like XGR, AceGamez, MS Xbox World, larger sites like 1up and even one of the reviewers in EGM (Kevin Gifford) all gave the game between 80 and 90 percent, well within a reasonable margin of error to be considered comparable to the suspect Future scores. Unless you claim that Atari gave all of these sites exclusives, or paid them all loads of money, or otherwise coerced and misled them all, then arguments that Future's conduct must have been unethical doesn't hold water.
Now, there were plenty of sites on those GameRankings lists that did not give Driv3r such glowing marks. An overwhelming number, in fact, did not give the game anything close to a 90 percent. The other two EGM reviewers besides Gifford were less forgiving (Giving it 7.5 and 6.5 out of 10, respectively). But the important thing to remember here is that a difference of opinion -- even a large one -- does not prove unethical intent. It doesn't even necessarily imply it.
There are those who have said that the high scores given by the Future Magazines are so outrageous, so unbelievably high in light of the enormous glitches in the game, that they go beyond a difference of opinion and into the realm of ethically questionable. This argument doesn't really hold water to me because of the amazingly subjective nature of game appreciation.
In my experience, people have a wide range of tolerances for glitches in games. I've had weeks-long arguments with my friends about games I consider unplayable that they think are masterpieces and vice versa. These arguments usually hinge on how much we're able to get over a game's flaws to see what might be an eminently playable game underneath. What one person considers a game-sinking flaw might be a forgivable annoyance to another. There is no objective standard for what makes a game a 3/10 or a 9/10. How else can you explain such a wide range of opinions on the game, even among non-Future publications?
This observation, combined with the anonymous, unsubstantiated nature of the accusations, leads me away from holding Future accountable.
How It Could Have Been Prevented
While I don't think there was any outright illicit relationship between Atari and Future, there is one important element of these reviews that was handled in a way I feel is sloppy at best and misleading at worst. Consider this quote from Nick Ellis, deputy Editor of Xbox World, originally posted in the (now-deleted) GamesRadar forum thread (still accessible in the Spong article):
"Because of the long lead times for magazines and the fact that it was an exclusive review, the code we reviewed from was not final. We were made aware of some bugs in the game and were promised that these would be sorted by the time of release. I cannot comment on whether these were fixed or not..."
This quote brings up an interesting dilemma for any game reviewer. In reviewing an early copy of the game you may, in fact, be playing a version significantly worse than the one your readers will get to play. Given the often end-heavy development cycles today's games go through, large changes can be made to a game between a press deadline (often months before release) and the day the game ships. Talking about flaws that may not appear in the final game could is potentially embarrassing and a disservice to the readers.
On the other hand, it's quite easy for a developer, publisher, or hired PR firm to use the "it'll be fixed in the final version" line to smooth over any potential problems with a preview copy. The mythical "final version" of a game that develops from these promises -- in which all current problems and bugs are corrected and the game is the best ever made -- will probably differ wildly from what the consumer finally gets. An over-reliance on the word of those with a vested interest in the game's success is obviously also against the reader's interest.
How does a reviewer handle this tug-of-war between preview copy roughness and "It's getting better all the time" optimism? Simple. Just include this disclaimer in any review that is not final code:
"This review is based on a preliminary build that may not accurately represent the final, retail copy of the game."
By including this line, or something like it, you can accurately report your impressions of the code in front of you without being pressured by outside parties to forgive errors that may or may not be fixed later. When the retail version does come out, a follow-up piece can note any significant changes between the reviewed copy and the final copy and even change the score accordingly.
This solution benefits all parties. The publication is saved from any potential embarrassment stemming from differences in the preview and final copies without sacrificing that coveted "first to print" edge. The observant reader gets a good idea of where a game stands without forming unbendable expectations about a game that could change significantly before it's released. And the game company gets a chance to fix reviewer-noted bugs without being overly penalized for roughness in a pre-retail build.
The Damage
It's impossible for any outside party to know exactly what happened between Atari and Future, but that is immaterial to the court of internet opinion that has already declared both parties guilty. What's almost worse is the total lack of surprise among many vocal members of internet message boards.
People seem to take it for granted that video game magazines and web sites are corrupt and willing to sell out their editorial ideals for higher circulation, free stuff, or even direct pay-offs. Every accusation of such conduct, no matter how specific or unsubstantiated, hurts the credibility of the entire video game journalism industry.
How can the video game press defend itself from such attacks? Here are a few ideas:
- Declare, in no uncertain terms, your publication's determination to be unbiased and free from coercion. Put it in a prominent place in every issue, or on every page of a web site. It doesn't have to be long or pretentious; just a simple statement saying that you won't let any publisher, developer, or advertiser to dictate your editorial policy. Then... follow that policy.
- Appoint an ombudsman. Not just an outsider who whines on a blog, like me, but a full time staff member who's there to represent the readers. Give him a regular space to address reader complaints -- not in a cute, humorous way like in many magazine letter columns, but with a more serious tone. It can be someone already on your staff or someone new (BTW, I'm available). Make sure his ombudsman duties are separate from his editorial duties and not subject to overbroad editing or oversight by the powers-that-be. He should be independent enough to operate without fear of losing his job but connected enough to get answers from the editorial team when he needs to.
- Have an in-depth staff policy on acceptance of gifts and attendance of industry events and parties. The policy should be focused on preventing even the appearance of a conflict of interest between the editorial team and any outside company. This doesn't have to be featured as prominently as the other two suggestions, but it should be available to the readers in some way. Need some help getting started? Use the Ombudsman Gift Guide.
Even if the particular charges against Future and Atari are false, it's obvious that sweeping changes are needed in video game journalism, if only to prevent such charges from being so easily accepted by a highly skeptical readership. Video game publications should have a clearly defined policy on conflicts of interest -- and follow that policy -- to diffuse such criticism before it starts. At the same time, they must make themselves more accessible to their readers and give them a way to address any concerns they may have with the publication. These steps are necessary if we want readers to trust video game publications as an unbiased arbiter of game quality.
No comments:
Post a Comment