What I Learned From James Petruzzi

Long before I asked James Petruzzi, developer of Take Arms and 48 Chambers, to do his excellent Tales from the Dev Side editorial for my site, I sought out his help for a planned article that never really panned out.  Although that didn’t come to pass, the hours Brian and I spent talking with James completely altered my perception of how certain Xbox Live Indie Games should be judged.  Before that conversation, I didn’t appreciate the absurd difficulty and almost unbelievable sounding limitations that Xbox Live Indie Game developers are saddled with.  As someone who has never developed games, I couldn’t grasp just how hard it was.  Mind you, I was (and really still am) new to the XBLIG scene.  I was told that XNA was one of the simplest development tools many long-time indie developers had worked with.  So it was like “well if that’s the case, why is putting online in your game such a big deal?”

Well, obviously I was wrong.  I quietly backed away from my “games should have online functions” policy.  Sure, I will still say that games can benefit from online play, or having online leaderboards, but I’m not going to let that be the focus of any review, which I had done in the past.

I’ve reviewed multiple games with online functions, and about two months ago, Brian and myself came to a realization: not a single online XBLIG we’ve played has ever been without some really serious glitches.  That is without exception.  It is universally true.  Most of these games I review shortly after their release, and it’s not unusual for me to have to accept a review code to give to someone else to test the online feature because of the lack of other active players.  This is  only time I do accept review tokens.  The code is given to someone else, while my copy is purchased by me.

Bug Ball was the game that created a change in my online review policy. It’s a good game, but networking issues greatly hampered its online playability.

I have a reputation as being the harshest critic on the XBLIG scene, and I’ve certainly earned it.  I’ve been told I’m overly brutal, too nit-picky, and sometimes even mean.  That might all be true, but there is one thing you can’t deny: I’m fair.  Every game I review starts with a clean slate.

Back in February, a developer requested that I play their latest game, Bug Ball.  A review code was provided, which I gave to Brain and his roommate.  We really enjoyed the game, but unfortunately, it was riddled with multiple glitches related to online play.  Characters would disappear from one player’s screen, the ball would disappear from one player’s screen, or sometimes the game would just stop working on one of our sides.  I believe this was the first online game I reviewed following my conversation with James, and thus it was the first time I was aware that the developer had no way of knowing that these kind of glitches were happening.  After all, they could not truly test the game over Xbox Live.

Brian and I talked about it, and we both decided that if I was to publish a review noting the glitches and how it ruined the experience for us, it would eliminate my right to claim that, no matter what I’m accused of, I’m always fair.  Because slamming a game for issues a developer could not possibly have been aware of would not have been fair.  Thus, we decided it was time for a change in Indie Gamer Chick policy.  I contacted the developer and told them what issues we had, and that I would hold off on my review until they had a chance to fix the problems.  Shortly there after, I added this policy to my FAQ.

I am often asked if I could help playtest games, or join the AppHub.  I’ve had more than a dozen people generously offer to stake my XNA membership fee.  But it’s not something I’m interested in, nor is it something I think I should be doing.  As a critic, I feel it’s important that I stay separate from the development process.  Although I understand that developers do want honest feedback in their games before they reach the marketplace, and I really do sympathize for them when they can’t get that, it shouldn’t come from me.  Doing so would compromise the entire point of my site.

I had a lot of fun playing Spectrangle360, but multiple issues with online play has caused my review of it to be delayed while its developer works to figure out what is going wrong.

But, I am willing to help once the game reaches the marketplace.  I am aware that, for many games, I’m the first person that will play it once it goes on sale.  Since I’ve never talked about this policy outside my FAQ, I want to lay it out here.  It goes as follows.

What I will do.

• I will contact the developer and list all glitches related to the networking parts of their game, explaining as clearly as I can what happened, both on my end and on the end of whoever my playing partner was.
• I can take any follow-up questions asking for clarification if necessary.
• I will leave it up to the developer whether they want me to go forward with writing the review immediately or if they would like me to hold off on it until they have a chance to fix the game.
• If the developer asks for me to hold off on the review, I will not count that as their Second Chance with the Chick, and they retain the right to request a second review once the original review is published and further patches are added to the game.

What I won’t do.

• I’m not willing to try an re-create any issues I come across for the developer.  Besides, I usually play the game long enough to see the same glitch happen multiple times.  Once the game returns to development, it’s up to them to figure out how to test it.
• I’m not willing to test the game with the developer to try to set off the issues.  Again, once I’ve sent the information back to the developer, I consider the game to be back in development, which I should have no part of.
• I’m not willing to continue to play the game some more to try to find even more issues.  Once the game is in the market and thus playable by the developer on the network it was designed for, they should be busy themselves looking for issues.  Asking me to do your work for you takes time away from me being able to play games from other developers who are eager to get their games reviewed here.
• Once the developer tells me they’ve fixed the problems and are ready for the game to be replayed for its review, I will not inform them of any further glitches that come up.  The game will be reviewed as is, and any further fixes will have to use up your Second Chance with the Chick.  So make sure that when you tell me the game is ready, you’ve tested it thoroughly and are sure it’s as ready as it can be.

By the way, I certainly hope nothing here or in James’ Tales from the Dev Side discourages developers from trying to add online components to their games.  Yes, doing so is extremely challenging, and maybe even not worth the effort.  However, if you came to the scene looking to challenge yourself, why sell yourself short?  It’s almost like what John F. Kennedy said of going to the moon.  You choose to put online in your games.  Not because it is easy, but because it is hard.  A developer who can make a game with online play on Xbox Live Indie Games is a skilled developer indeed.

I have nothing but respect for the Xbox Live Indie Game community, and I’m always willing to offer advice when someone wants it.  I know a lot of you wish I was willing to help more in the development process, and given how crappy the playing testing and peer review system you guys have to deal with is, I can’t blame you.  Because I feel that doing so is a conflict of interest, I regretfully have to turn you down.  But, when it comes to online play, I am willing to lend you a teeny tiny hand.  I’m still the same Indie Gamer Chick I’ve always been.  I call it like I see it.  I’ve absolutely demolished games here.  I show no mercy.  But with online XBLIGs, I’m willing to cut you some slack and give you a chance to make things better.  Why?  Because it’s the right thing to do.