Democratic Entertainment Expo

I don't know how many of you have been following the coverage of this year's Democratic National Convention, but a running theme in both the print and broadcast media this year seems to be the convention's utter lack of news importance. The ironic point of these stories is that the DNC is nothing more than a well-coreographed, four-day informercial that is all about surface impressions and offers very few surprises.



A similar charge could, and has, been levied against the relevance of our own Electronic Entertainment Expo.



It's not hard to find similarities between the gatherings. Among them:

• Both often seem more conerned with sound bites and flashy presentation than with serious debate of new ideas. At the DNC, issues like gay rights, corporate crime and election reform are pushed to the side for messages more palatable the mainsteam voters. At E3, experimental games like Katamari Damacy, Ribbit King and X-Tango are pushed to the dark corners of the convention floor in favor of blockbuster games with mass-market appeal.



• Both give attendees a very limited opportunity to really get to know the candidates/games that are being displayed. At the DNC, 45 minutes in front of an audience is just barely enough for a candidate to begin to outline their vision for the future. At E3, five minutes in front of a game console is just barely enough to get an idea of the play mechanics before writing up a "from-the-floor" report.



• Both offer very little that isn't alreasy announced or expected. Today's party conventions feature candidates that have been picked months before in nationwide primaries, a far cry from the often shocking battles for nomination in years past. Today's E3 offers very few games or hardware announcements that aren't leaked to the press at least a few days in advance, a far cry from the shocking price-cut and new game announcements that came during the show in years past.



• Both conventions shower the attendees with free stuff. This one is self-explanatory.

Given these similarities, one has to wonder if E3 is that important for the press anymore. Would the role of the Expo be better served just by sending out a bunch of demo disks and press releases to the media and saving everyone the plane fare?



I would say so, except for one key benefit of gathering everyone together at a centralized Expo -- the social aspect. The landscape of video game journalism is very fractured, both physically and content-wise. The myriad staffs of the various web sites and magazines occupy every different state of the union and state of mind you could care to list. E3 is a chance for these journalists to come together and share stories, ideas, and time with one another.



Besides meeting much of the staff of Game Critics at this year's Expo (my first), I managed to bump in to Brandon Sheffield of Insert Credit, Davis Thomas of the Denver Post and Chris Morris of CNN/Money (who I literally almost bumped in to). I also chatted with many other journalists -- from publications big and small -- to swap tales and opinions with during the rare idle moments of the show.



This casual meeting of the minds is essential in unifying what is an incredibly varied video game journalism landscape. I know many disparate groups of journalists often organize meet-ups at these events, but it'd be nice if there was some place that game journalists of all stripes could just hang out and meet one another (outside of the mad dashes for food that are the media breakfast and lunches). I know Dave Thomas tried to do one through the IGJA this year, but it laregly feel through. I hope we can be more organized about such a meet-up next year, because this year I learned that the real value of E3 is as much about the people as it is about the games.

This Article Has At Least Two Minutes of "Read-Time"

First off, welcome to all the new visitors finding this blog through the recent mention in Electronic Gaming Monthly (Did I ever mention how much I like that magazine ^_^). Now that you're here, I hope you'll continue to read and enjoy the site. Please skim through the archives, comment on some stories, and by all means let me know what you think of any and all video game journalism.



Thanks to _render_ for (1) linking to my site and (2) pointing me to this TotalVideoGames (TVG) article saying that Grand Theft Auto: San Andreas will have "150 hours of gameplay." This is a perfect springboard for me to rail against how utterly useless and misleading the "gameplay time" statistic is.



Now, I understand that video game critiquing is a subjective craft. What is a great game to one person could be the worst game ever made to another. The main problem with the "gameplay time" statistic is that it sounds very absolute and scientific when, in actuality, it's just as subjective as any other part of a review.



First off, the amount of time it takes for a person to beat a game is largely a function of general game playing skill and style. I was able to beat "Paper Mario" in about 24 hours, according to the in-game clock, it took my girlfriend -- who has less experience with RPGs and tends to spend more time exploring -- nearly 34 hours. Even my roommate -- whom I consider an equal in skill and style -- took two fewer hours than me to beat the game.



But even if you can assume a base level of skill for your entire audience (hardly a comfortable assumption), the "gameplay hours" statistic, as it's usually used, is incredibly meaningless for a large segment of games. By, "as it's usually used," I mean defined as: "the time it will take an average player to thoroughly beat the game on the first playthrough."



Using this metric, what kind of "hours played" statistic would you give Tetris, a game that is technically unbeatable. The Sims, which has no goal except that defined by the player, is equally uncategorizable. It took over 20 years for Billy Mitchell to fully "beat" Pac-man by getting past the 256th screen without losing a life or missing a ghost. Would his thousands of hours of game time be an accurate measure of anything?



Even for games that do have an easily defined "end" the statistic is rife with problems. Technically, it probably took me only about 40 hours to beat Super Mario 64 with all 120 stars. But since I reached that point, I've come back to the game constantly, easily quintupling that base play time just running around the castle, finding glitches and repeating challenges. For a game that gave me over 200 hours of time in front of the screen, 40 hours seems pretty inadequate.



This doesn't even get into the issue of when today's games -- often full of unlockable levels and secrets -- really end. Do you cut Super Mario 64's time off at the minimum 70 stars required to beat it or the maximum 120 stars? The answer can severely skew the statistics.



The only "gameplay time" statistic that really matters is how much time you're willing to keep playing the game, and therein lies the problem. Despite the seemingly straightforward and absolute definition given above, most "gameplay time" statistics end up reflecting how much time the reviewer was willing to spend with the game -- an inherent value judgement that is largely invisible to the reader. If one reviewer blazes through a game he hates in five hours to get the review done, and another spends 20 hours engrossed in what he considers a deeply moving experience, who's to say which number is the correct measure of "gameplay time?" And who's to say which one will be closer to the amount of time a player actually spends on the game?



Outside of all this, I have to say that taking a "gameplay time" number from the producer of the game, as TVG did, is absolutely ludicrous. Of course he's going to say that the game has lots of play time. It's his game... he wants to sell it. It's like printing a quote from a developer who says his game will have "good graphics." It's entirely expected and entirely meaningless to the reader.



To sum up: I suggest reviewers spend more time playing the game and less time worrying about how much time they played the game.

Th3 Driv3r D3bacl3

Replacing the "E"s with "3"s in this headline is breathtakingly unoriginal, I know. But after reading what seems like a never-ending flood of thoughts and opinions about this issue for the past few weeks, I've been left seeing "3"s everywhere.



Introduction



Reprising all the relevant facts and accusations involved in what has become known as "the Driv3r review-buying scandal" would, I feel, take up too much time and space for this blog. If you are totally new to the situation, I encourage you to start with this (surprisingly moderate) Spong article to get an overview, then click some of the links above if you want more source material. The rest of this article assumes knowledge from those sources.



Given the huge implications this "scandal" could have on the video game journalism industry, I've been doing a lot of reading and thinking on the matter over the past weeks. That being said, I'd like to preface my remarks by saying that this post is entirely my opinion and is based on the same publicly available information everyone else has access to. It is not legally or morally binding in any way, and other opinions on the matter are certainly possible.



Why I'm Not Convinced



Based on the above-linked sources and others, I've come to the conclusion that allegations of outright review score buying or inducement on the part of Atari and Future Publishing are circumstantial at best and unsubstantiated at worst.



First of all, while the original message board accusations claim to be from video game journalism industry insiders who are in a position to know about the "scheme," there is absolutely no way of proving this. This is the internet, and the remarkable ease of lying and/or misrepresenting yourself on a message board can not be overstated. This is a subject I gained first-hand experience with when a person claimed to be me on the SMBHQ chat room and managed to do some damage to my online character. Anyone could have made these accusations for any number of reasons, the truth quite possibly not among them.



Secondly, the message board posters that broke this scandal offer absolutely no definitive evidence -- outside of their statements -- to support their claims. There are no scanned contracts, no recordings of illicit conversations, no (legit) photos of envelopes full of money, or anything of the kind. Not that the lack of such evidence automatically exonerates the parties, but it does make the accusations seem a little hollow when they have no documented support.



The flimsy "evidence" that does sometimes get cited -- such as the high review score itself, the omission of gameplay bugs from the review, and a sticker advertising Future's scores on the game packaging -- is highly circumstantial and a long way from a standard of proof that would satisfy me.



But the most important reason I lean away from any foul-play interpretation of the facts is that there are other sources that agree, or come close to agreeing, with the 90% rankings of PSM2 and XBW. Smaller sites like XGR, AceGamez, MS Xbox World, larger sites like 1up and even one of the reviewers in EGM (Kevin Gifford) all gave the game between 80 and 90 percent, well within a reasonable margin of error to be considered comparable to the suspect Future scores. Unless you claim that Atari gave all of these sites exclusives, or paid them all loads of money, or otherwise coerced and misled them all, then arguments that Future's conduct must have been unethical doesn't hold water.



Now, there were plenty of sites on those GameRankings lists that did not give Driv3r such glowing marks. An overwhelming number, in fact, did not give the game anything close to a 90 percent. The other two EGM reviewers besides Gifford were less forgiving (Giving it 7.5 and 6.5 out of 10, respectively). But the important thing to remember here is that a difference of opinion -- even a large one -- does not prove unethical intent. It doesn't even necessarily imply it.



There are those who have said that the high scores given by the Future Magazines are so outrageous, so unbelievably high in light of the enormous glitches in the game, that they go beyond a difference of opinion and into the realm of ethically questionable. This argument doesn't really hold water to me because of the amazingly subjective nature of game appreciation.



In my experience, people have a wide range of tolerances for glitches in games. I've had weeks-long arguments with my friends about games I consider unplayable that they think are masterpieces and vice versa. These arguments usually hinge on how much we're able to get over a game's flaws to see what might be an eminently playable game underneath. What one person considers a game-sinking flaw might be a forgivable annoyance to another. There is no objective standard for what makes a game a 3/10 or a 9/10. How else can you explain such a wide range of opinions on the game, even among non-Future publications?



This observation, combined with the anonymous, unsubstantiated nature of the accusations, leads me away from holding Future accountable.



How It Could Have Been Prevented



While I don't think there was any outright illicit relationship between Atari and Future, there is one important element of these reviews that was handled in a way I feel is sloppy at best and misleading at worst. Consider this quote from Nick Ellis, deputy Editor of Xbox World, originally posted in the (now-deleted) GamesRadar forum thread (still accessible in the Spong article):



"Because of the long lead times for magazines and the fact that it was an exclusive review, the code we reviewed from was not final. We were made aware of some bugs in the game and were promised that these would be sorted by the time of release. I cannot comment on whether these were fixed or not..."



This quote brings up an interesting dilemma for any game reviewer. In reviewing an early copy of the game you may, in fact, be playing a version significantly worse than the one your readers will get to play. Given the often end-heavy development cycles today's games go through, large changes can be made to a game between a press deadline (often months before release) and the day the game ships. Talking about flaws that may not appear in the final game could is potentially embarrassing and a disservice to the readers.



On the other hand, it's quite easy for a developer, publisher, or hired PR firm to use the "it'll be fixed in the final version" line to smooth over any potential problems with a preview copy. The mythical "final version" of a game that develops from these promises -- in which all current problems and bugs are corrected and the game is the best ever made -- will probably differ wildly from what the consumer finally gets. An over-reliance on the word of those with a vested interest in the game's success is obviously also against the reader's interest.



How does a reviewer handle this tug-of-war between preview copy roughness and "It's getting better all the time" optimism? Simple. Just include this disclaimer in any review that is not final code:



"This review is based on a preliminary build that may not accurately represent the final, retail copy of the game."



By including this line, or something like it, you can accurately report your impressions of the code in front of you without being pressured by outside parties to forgive errors that may or may not be fixed later. When the retail version does come out, a follow-up piece can note any significant changes between the reviewed copy and the final copy and even change the score accordingly.



This solution benefits all parties. The publication is saved from any potential embarrassment stemming from differences in the preview and final copies without sacrificing that coveted "first to print" edge. The observant reader gets a good idea of where a game stands without forming unbendable expectations about a game that could change significantly before it's released. And the game company gets a chance to fix reviewer-noted bugs without being overly penalized for roughness in a pre-retail build.



The Damage



It's impossible for any outside party to know exactly what happened between Atari and Future, but that is immaterial to the court of internet opinion that has already declared both parties guilty. What's almost worse is the total lack of surprise among many vocal members of internet message boards.



People seem to take it for granted that video game magazines and web sites are corrupt and willing to sell out their editorial ideals for higher circulation, free stuff, or even direct pay-offs. Every accusation of such conduct, no matter how specific or unsubstantiated, hurts the credibility of the entire video game journalism industry.



How can the video game press defend itself from such attacks? Here are a few ideas:

• Declare, in no uncertain terms, your publication's determination to be unbiased and free from coercion. Put it in a prominent place in every issue, or on every page of a web site. It doesn't have to be long or pretentious; just a simple statement saying that you won't let any publisher, developer, or advertiser to dictate your editorial policy. Then... follow that policy.



• Appoint an ombudsman. Not just an outsider who whines on a blog, like me, but a full time staff member who's there to represent the readers. Give him a regular space to address reader complaints -- not in a cute, humorous way like in many magazine letter columns, but with a more serious tone. It can be someone already on your staff or someone new (BTW, I'm available). Make sure his ombudsman duties are separate from his editorial duties and not subject to overbroad editing or oversight by the powers-that-be. He should be independent enough to operate without fear of losing his job but connected enough to get answers from the editorial team when he needs to.



• Have an in-depth staff policy on acceptance of gifts and attendance of industry events and parties. The policy should be focused on preventing even the appearance of a conflict of interest between the editorial team and any outside company. This doesn't have to be featured as prominently as the other two suggestions, but it should be available to the readers in some way. Need some help getting started? Use the Ombudsman Gift Guide.

Conclusion



Even if the particular charges against Future and Atari are false, it's obvious that sweeping changes are needed in video game journalism, if only to prevent such charges from being so easily accepted by a highly skeptical readership. Video game publications should have a clearly defined policy on conflicts of interest -- and follow that policy -- to diffuse such criticism before it starts. At the same time, they must make themselves more accessible to their readers and give them a way to address any concerns they may have with the publication. These steps are necessary if we want readers to trust video game publications as an unbiased arbiter of game quality.

Ends and odds

A few small things. First off, Jeremy Parish, the author of the 1up articles cited in my controversial post, "The Sincerest Form of Flattery" has responded more fully to the situation on his blog (look at the last post on the linked page). He's probably right when he says I went too far in hinting at plagiarism when there were merely just surface similarities. I thank him for his response and his encouraging words. Also thanks for the comparison of this blog to "sniffing the cork and critiquing the bouqet of a $3 jug of cherry wine." I'm going to have to use that one at parties.



Secondly, I somehow forgot an important point I wanted to mention in my post about confusing reviews, "Thy Audience, Thyself." For a few months now, Game Informer magazine has included a small breakout at the beginning of their reviews section describing many technical terms "for our newer readers." From acronyms (like GBA) to technical terms (like motion-capture) to industry terms (like third-party), the breakout provides a good reference in a space of less than half a page. A great idea, and one that I hope other magazines pick up on.



One thing I would improve in Game Informer's list: Better genre descriptions. Describing "fighting" as, "A term we use for games like Street Fighter or Dead or Alive" does nothing for the total gaming neophyte. How about, "A term we use for games that involves two (or occasionally more) characters beating up on each other," or some such similar.



Also, a small correction to the "Thy Audience, Thyself" post. Radiant Silvergun was identified in that post as a Dreamcast game. The game actually came out for the Saturn. The Ombudsman regrets the mistake.



All right, now that all that's out of the way... back to business starting tomorrow. Scroll down a bit and post your thoughts on my "yearly review" down below.

The Ombudsman -- One year later

According to the ol' Blogger counter here, this is my 100th post to the Ombudsman since it started a little over a year ago (That's almost a post every three days, for those of you that like to complain about me never updating). Back then, I started this site up to "put a check on the often sloppy, lazy, biased, and unethical world of video game reporting," and said that I would "mix original reporting in the form of research and interviews with my unapologetic opinions about the industry." I also said that this space would be "your forum to speak out about what you love and hate about the video game journalism business."



In that spirit, I'd like to listen to your thoughts on where this blog has been, where it is, and where it should be going. Consider this my yearly performance review. Have I lived up to the goals I set out in that first post? Have I become just another whiny voice in the internet ether? What have I done well? What could I do better? What issues would you like to see me tackle in the coming year? I, of course, have my own thoughts on these matters, but my opinion on this subject doesn't mean too much.



Hit up that comment link below and share your thoughts.

Thy audience, thy self.

In cruising around the NPR site during work the other day, I noticed a new column by NPR Ombudsman Jeffrey Dvorkin on the grating nature of some of NPR's music reviews:



The problem, according to some listeners, is that NPR's reviews are too hip to be good journalism. In short, some musical commentary, especially on All Things Considered, is incomprehensible to some listeners, and I confess, to me.



A similar charge could be made against some video game reviews, although the specific "too hip" complaint might not apply as often. Instead, game reviews are often too technical or too coarse for the average reader to appreciate fully.



Not that this is always a bad thing. The hardcore gamer audience is definitely a large segment of the video game press' readership, and there is no lack of sites (and, to a lesser extent, magazines) catering to this crowd. What I fear is that the hardcore audience is driving the games press so much that other, less knowledgable readers are left scrambling for comprehensible information on a subject they may be interested in becoming interested in.



What scares off the casual reader? Here's a few likely suspects, off the top of my head:

• Jargon and technical terms - These are probably the biggest point of exit for the non-gamer. Terms like bump mapping, frames per second, anti-aliasing, and others can get thrown around with startling frequency sometimes. Video game writers, who should be familiar with these terms, have to make a concious effort to notice and limit their use of these terms.



• Comparisons to niche/esoteric games - Specifically, I'm talking about comparisons that don't explain themselves further (i.e. "Ikaruga should be immediately familiar to fans of Radiant Silvergun") If you are using the other game to explicitly illustrate and explain a trait of the reviewed game, then it is more acceptable (i.e. "Ikaruga's frentic action and non-stop stream of enemies is reminiscint of Radiant Silvergun, a largely overlooked Treasure shoot-em-up for the Dreamcast.")



• Point-by-point breakdows - Maybe this is just a pet peeve of mine, but I don't think the general reader appreciates a review that has sections titled "Graphics", "Sound", "Controls" and "Fun Factor," etc. These nitty-gritty details about a game are important, but just listing them out seperately is a good way to lose readers. Instead, try to weave these elements together into a greater narrative about why the game is good or not and why the reader should care.

Rolling Stone, in its heyday, did a great job of making music criticism and discussion feel relevant to the everyday listener (recently, I feel that focus has fallen away). Newspapers and general interest magazines that cover games usually do a good job of avoiding jargon and describing why regular people should care about a game. I encourage sites and magazines of all stripes to try and find a happy medium between keeping the hardcore happy and keepnig the non-gamers interested if only to increase their appeal and readership to a broader audience.