Lies, damn lies, and rental statistics

Imagine if a respected newspaper decided that, instead of listing the top 10 grossing movies for all theatres nationwide each week, it would start listing the top grossers only at Cineplex Odeon Theatres. Such an idea is ludicrous, of course. Why list the statistics from one company when the more relevant listings for the whole industry are readily available?



We should be asking many members the video game media that very question right about now.



There were a number of sites that decided to publish the top 10 rented games list provided by online rental house GameFly for the week ending Sept. 28. Here is that list, copied from Gamespot:



Rank / Title / Platform/ Publisher

1 / Dungeons and Dragons Heroes / Xbox / Atari

2 / Simpsons: Hit & Run / Xbox / Vivendi Universal

3 / WWE Raw 2 / Xbox / THQ

4 / Tiger Woods PGA Tour 2004 / Xbox / EA Sports

5 / Simpsons: Hit & Run / PS2 / Vivendi Universal

6 / Soul Calibur II / Xbox / Namco

7 / Tiger Woods PGA Tour 2004 / PS2 / EA Sports

8 / Otogi: Myth of Demons / Xbox / Sega

9 / Soul Calibur II / PS2 / Namco

10 / Disgaea: Hour of Darkness / PS2 / Atlus Co.





As you can see, XBox has 6 of the top 10 spots on the list, including the top 4. Also notice that when a game is listed for the PS2 and XBox, the XBox version has the higher placement.



The news sites were quick to pick up on this supposed trend. "XBox rentals show their strength" said GameSpot. "It looks like more and more XBOX titles are becoming the top rentals," said OnlyOnXBox.net. But is this really the case? Can we really extrapolate anything about the industry from the report of this one retailer?



A quick look at a report from the Video Software Dealer Assocation trade group says otherwise. Here's their rental rating list for the week ending Sept. 27. copied from GameMarketWatch.com:





1. (PS2) Madden NFL 2004 -- Electronic Arts

2. (PS2) Soul Calibur II -- Namco Hometek

3. (PS2) NCAA Football 2004 -- Electronic Arts

4. (PS2) Enter the Matrix -- Atari

5. (PS2) Midnight Club II -- Take-Two Interactive

6. (Xbox) Madden NFL 2004 -- Electronic Arts

7. (PS2) ESPN NFL Football 2K4 -- Sega of America

8. (Xbox) Soul Calibur II -- Namco Hometek

9. (PS2) Cebela's Deer Hunt 2004 Season -- Activision

10. (XBox) Star Wars: Knights of Old Republic -- LucasArts





Notice any difference? There are only three XBox games on the VSDA's list, and games listed for both systems are ranked higher for the PS2.



How do you decide which list to use? Well, GameFly's list gives the top rentals among people who rent their games from one specific online retailer, while the VSDA's list "collects point of sale information from video retailers nationwide" according to ConvergentData.com. Gamefly's list skews towards hardcore gamers willing to pay a monthly fee to get unlimited length rentals through the internet, while the VSDA's list encompasses the undoubtedly larger number of gamers that rent games for a set period of time from brick-and-mortar establishments.



It should be obvious that the VSDA's list paints a more accurate picture of the game rental landscape.



This doesn't mean GameFly's list is useless. If you're looking at trends among the niche group of gamers that uses online rental houses, the list is a valuable resource. But none of the sources I saw reporting on the list mentioned this important distinction in their stories (with the exception of EvilAvatar, which said that the prevalence of XBox games "might have something to do with [GameFly] being an online rental service.")



It's pretty obvious that most of these sites used GameFly's list simply because it was made easily available to them. In fact, it is easily available weekly to anyone who e-mails pr@gamefly.com with a request, according to PCVsConsole.com (which seems to have copied GameFly's press release verbatim, right down to the About GameFly, Inc. kicker).



But just because something's in your inbox doesn't mean it's the best information for your readers. The VSDA's weekly numbers are also available every week on their web site, free to use for anyone interested (with registration). The VSDA numbers are undeniably a more accurate representation of the industry and a more relevant resource for most readers. I highly recommend them over GameFly's numbers.



I'm going away for the weekend, but when I get back I'll try to make some sense of this whole "the PS2 is a computer"/"The PS2 is not a computer" mess.