A Review, As A Matter Of Fact

Thanks to Ombudsman reader Sally Jacques for writing in to ask "what your stance on what mags should do when the person who is responsible for an article is proved totally wrong."



Sally pointed me to a thread on the Official XBox UK Magazine (OXUKM) forums as way of example. Seems that the magazine recently ran a review for Headhunter: Redemption developed by Amuze and published by Sega. The thread, purportedly by two Amuze developers, points out some factual errors with the review:



In your review of Headhunter Redemption (Issue 34, October 2004), you claim that cut scenes "can't be skipped... Grrr!" This is incorrect: all cut scenes in the game are skippable. Grrr, indeed.



More mysteriously, you conclude your review by stating that "the odd bike chase succeeds in breaking up the on-foot action." There are no biking sequences in Headhunter Redemption: a fact that would be apparent to anyone who had actually played the game.



First of all, let me say that I have not verified that (a) these statements were printed in the review, (b) the statements in the review were false or (c) the original posters are actually Amuze employees. That being said, I have no reason to doubt any of these things given the corroborating statements in the rest of the thread and a similarly worded posting on Amuze's web site. While these facts are central to this specific dispute, they are not relevant to the more general question of how to handle apparent factual errors in reviews.



Later in the OXUKM thread, the author (or someone claiming to be him) responds to the charges:



As for factual inaccuracies, things seem to have been taken out of context.

I stated that cut-scenes couldn't be skipped because, in the review code we were sent, no amount of Start / A / any button pushing could skip the cutscenes.



And as for mentioning the bike sections, again I was referring to the cutscenes.



This defense highlights the two (or three?) main ways that I can envision factual errors getting into a review: versioning discrepancies and author sloppiness/duplicity.



A versioning discrepancy (i.e. a difference between review code and final, released code) is an unavoidable side effect of top-heavy game development schedules and bottom-heavy magazine publishing schedules. As I have said before, I feel that videogame reviews based on unreleased code should clearly state that they are based on an early version of the game. If, when the game comes out, the early review turns out to be inconsistent with the final product, a correction should be published at the earliest possible convenience.



Sometimes, as in this case, the correction would be relatively minor ("It turns out the cutscenes are skippable... whoops"). Other times, the changes to the game and the review may be substantial, even necessitating a change in overall opinion or scoring ("What was a buggy, badly conceived game three months ago turned out to be sufficiently playable upon release.") Either way, the correction should be prompt and easy for readers to find. (On a small side note, I specifically remember Game Players magazine handling minor corrections like these quite amusingly in their letters section with threats of violence to the offending author.)



In the case of author sloppiness (i.e. an author writing things about a game that are just wrong) there might be bigger issues at stake. In this case, I find it hard to believe that the bike chases which the author allegedly said, "succeed in breaking up the on-foot action," were actually in cut scenes. First of all, the language clearly implies that the bike chases are a part of gameplay the breaks up other parts of the gameplay. Secondly, Amuze claims that there are no bike chases in any of the game's cut scenes (again, this is all unconfirmed, but I have been given no reason to doubt it).



Sometimes, a factual error like this, especially a small one, could be chalked up to simple confusion on the part of the reviewer. No two gameplay experiences are exactly alike, and it's possible that a specific set of circumstances in a reviewers playing of a game might lead him or her to perceive something that isn't really true. It's hard to come up with an example of this that doesn't sound too contrived, but it is a possibility.



The other possibility, that the reviewer didn't actually play the game, is much more serious. With the amount of easy information available from press releases and previews, it's all too easy to completely fabricate a review of a game based on no play time. Factual errors like these could be a major red flag that such fabrication is occurring, especially if the language can be traced back to earlier material that was also proven false. In this case, a May 12, 2003, press release for the Headhunter: Redemption lists "lightning-fast motorcycle thrills on futuristic highways" as a feature, making the motorbike comment especially suspect.



In cases like this, it's up to the editor to talk to the author and further investigate whether the review represents sufficient time spent with the game (how much time is "sufficient" to review a game is a much larger topic for another time). If it's found that an author misrepresented their work, then a formal apology to the readers and a second review of the game are probably in order. Disciplinary action against the author should be left to the editor's discretion.



But be careful here. It's very easy to accuse an author of not having played a game, especially if they write a review you don't agree with. Such accusations fly around internet message boards all the time. But a simple difference of opinion is rarely enough to prove that someone wrote a review without playing a game. Factual errors and suspiciously similar language are among much more reliable methods for determining misrepresentation.



So, to sum up, how to deal with factual errors depends on their cause and their size. If the case of small, honest mistakes, all that's necessary is a correction. In the case of larger, suspicious errors, further investigation and action by an editor might be necessary. Hope that answers your question, Sally.