izziebytes.net

The Legend of Beta Part I: WTF Happened?

Part 1Part 2Part 3

There are certain key words that have become natural triggers for people who are gamers: free, epic, loot, newb. But none have the lasting power or influence over our minds that that one, magical four letter word: Beta. Say it at convention and 20 heads will turn your way. Mention it online and your traffic boosts by 10%.

At PAX, I must have gotten 30 different invites and codes to sign up for them. But there is a huge loss that is hidden behind those enticing strings of digits. What has come lately is a huge shifting mindset regarding beta-testing from gamers, and more importantly, developers. Times have changed as gaming becomes more popular and mainstream. Infiltrated with twelve year old boys with tender egos and attention-whoring house wives, this sub-culture of the hardcore gaming comes with it’s own set of social rules, heirarchy, and bragging rights. Just like high-school all over again, everybody’s trying to be the cool-guy and forget the point of school is to learn. Well, once upon a time the point of betas was to test a game and improve on it. But that has been completely ruined: now betas are feared by developers and savagely sought by gamers, and for all the wrong, unfortunate reasons.

The issue is that Beta invitations have become a status symbol in recent years. As the popularity of PC gaming grows, so does the desire to be “in the know” and on the inside of exclusivity. Everybody’s blogging about it, posting about it, tweeting about it, and they are driven to be the one of the firsts to get their hands on it. And no matter who you are, if you’re an online gamer, you have definitely experienced that uncanny urge to Beta. I know I have. I am part of this pandemic. And so are you.

“So why is this a problem?” you ask. Because when status among peers becomes a driving force behind something, that something loses it’s “something”. See: teenage sex. See: iphones. They do it just to do it, they buy it just to buy it, because they want to be the cool guy and the object of envy from their peers. The real meaning and merit is lost in this struggle for status. We forget sex is supposed to be meaningful. We forget iphones are supposed to have utility in day-to-day life. That’s what betas have lost. The “testing” got dropped and hasn’t been able to find it’s way back, and it’s killing this important part of online gaming.

“So what’s this about developers being scared?” you ask.

The first issue is that gamers don’t respect NDAs. NDA Stands for Non-Disclosure Agreement: that’s a binding pact that during certain phases of testing, no information or media regarding said game is supposed to be coming out. You will always inevitably have that guy, or those guys, who ignore this and post screenshots, videos, and bits of information.

In small amounts this isn’t too big a deal. In fact, it’s even a good thing. Small leaks often keep the masses craving more. In droves, however, it becomes dangerous. It’s kind of like the density of water. It shrinks a bit before expanding. Likewise, for a small moment in time, leaked info is ok but then it becomes a problem that leads to the next big issue:

Gamers prematurely base their opinions on betas, and developers have become all too aware of this. How many times have I heard “Ugh, is that armor clipping?”, “How come these animations are so glitchy?” “WTF? LAG ON MY BETA SEVER? I’m not buying this game.” and I cringe every time. One of the biggest reasons Betas are postponed closer and closer to release is because people decide before it comes out whether or not it’s worth time based on glitchy, laggy, incomplete versions of the game. Sometimes they haven’t even tested it: they’ll make conclusions based on a screenshot and some angry guy’s blog post. Soon enough, if not already, having an open beta early in development will be marketing suicide and companies won’t be able to do it even if they wanted to. The suits will forbid it.

Gamers don’t fucking test in beta testing. Seriously. While I don’t have any official numbers, just from experience alone I can safety bet no less than 75% of players who get into betas haven’t contributed an ounce of useful information to the developers. All they do is cry in public chat. So basically there’s these droves of people posting screenshots and videos and running around wasting data and sever space just to say they can, will, and have.

That’s like having painters come over to work on your house and all they do is sit around, eat your food, order porn on your TV, and kick your dog. Wouldn’t you be pissed? Can you understand why developers are fed up?

Once upon a time, the point of a beta was to break a game. Testers purposefully pushed the client to it’s limits, and the developers wanted this. They wanted you to overload an area, they wanted you to walk through walls and fall through the world. And once upon a time, the bragging rights wasn’t about who was -in- the beta but rather who was able to find and report the weirdest bug and best exploit. It’s called beta testing after all, not beta leeching.

The only company that can pretty much put out a beta whenever they want is Blizzard because if someone complains about something, they can basically say “Fuck you, it’s beta, if you don’t like it, there’s 3,000 other people willing to take your place.” Blizzard has a large enough following that by sheer numbers, there are enough reliable testers around to make it worth the 80% of others who do nothing but spout bullshit about low-res textures and nerfs to their class.

It’s a crying shame, too, because with technology and social networking being what it is, there has never been a better time to test out games and help developers dig up the sorts of bugs only players have the magic ability to find.

We’re likely to find in the next few years that game companies no longer release open betas at all. Instead, fans are likely to get a “demo” a few weeks, maybe a month or two, prior to release when the game is essentially done and simply needs the sort of en-masse testing for stress, servers, and population issues. And we only have ourselves to blame. Yes, our number-crunching, epeen evny-ing, status obsessed selves.

Anti-Lurk Q&A: Have you ever beta’d without testing (DO YOU FEEL GUILTY? J/K)? Do you think companies are right to worry? How do you feel about the impending Death of the Beta?

  • In part I think some companies may as well do without beta on the basis that they release games filled to the brim with bugs anyway. So why bother?

    In any case, unfortunately, most companies just do not have the fans and number of people required to do closed in house testing. Like arenanet does.

    I think betas will go on, they’re probably just that important, but I think certainly there needs to be a change, in the way testers are chosen, in the way beta is often used by companies as promotion instead of testing, and as you’ve said, in the attitudes of gamers.

    • “In part I think some companies may as well do without beta on the basis that they release games filled to the brim with bugs anyway. So why bother?”

      That’s a very interesting point you bring up, actually. I wonder if a lot of that has to do with the lack of input during betas and what-not. But there’s a numbers game here. Even with closed and open betas, it’s nearly impossible to anticipate the amount of bugs and stress load a game will have once it goes lives. There will always be something that was missed.
      There are some games, though, that have just been horrendous on release. I can’t remember which one but I do recall a title within the past couple of years that was so terrible the servers had to be shut down and stuff.

      Betas will go on, no doubt, but like you said, there has to be a change. My idea is more of a split, that I’ll get to in the next part.
      And yes. Gamers need to grow up.

  • kaae

    I think I can say I agree on many of your points on how some people who get into beta don’t take what they’re doing seriously. However, I do not think its quite at that large a scale yet. Perhaps I am a bit bias and optimistic because I’m in a couple myself, but from personal observations, for every one person I may see rising up to proclaim hate and frustration, I see 2-3 other voices rising up with suggestion to report the issue and give feedback.

    I run by the assumption too that the blogger and youtube-ish video makers are not a majority, but a very vocal minority. They are (in my opinion) just looking to cause a stir and be ‘sensational’ rather than be objective and fair. And while you do have those negative nancys running around writing those blogs and making those videos… you also have a more quiet group who are doing positive spins and creating positive videos.

    I don’t think it would be a wise decision for a company to stop running betas for their games simply because a few people were horrid reviewers. If I were the company, I would probably consider a few things first. I would look at what kind of beta I am presenting to the testers and to not take offense if a couple complain about an element not yet worked on or is known to be incomplete. And I would probably look at how I’m releasing information to the public as well — I would be sure to keep a steady flow and be fairly open about the progress of the work. I might also consider creating an open developer’s blog – with images and perhaps discussions on the current projects and images of work in progress, etc…

    Do I feel the death of the beta is coming? I can’t see it happening to be honest. I think its marketing suicide to close off so much information about the game. I see a Beta as two sided. Betas are a fantastic way to stress test the game, get initial reactions, and to work out as many bugs as possible and perhaps adding content not considered thanks to those who contribute information. Additionally, the amount of word of mouth, and internet information exchange makes the game familiar to the crowds and keeps it from being released as some unknown game that someone might not really glance at in the walmart isle.

    Just my two cents. I’m looking forward to your next posts!!!

    • Thanks Kaaebies! You’ve got some really good points in here.

      You’re biased because you are a good beta-tester, I think, and that’s not a bad kind of biased at all. Because you are productive and likely hang with like-minded playrs in game, you see that side of the deal, which is awesome.

      I still feel, at least in my experience, that the majority of people who get beta invites are wasting it, but the way you put it definitely sheds light on a few things. Namely, not everybody scrambling to get into a beta just to try it are just trying to be cool. Many of them legitimately want to try it before they drop the 50 / 60 bucks.

      I think that’s a great counter to my rant, kudos.
      You should write something on it in your blog, I’d love to hear a full post on your thoughts.
      <3 Ya

      • kaae

        Not sure if it counts as a full post, but its there now. Took a few hours to try to write something beyond my normal winding walls of text. argh haha

        Looking forward to your next posting!! <3

  • Winterbay

    Good points there. I feel however that part of the problem is the fact that fewer and fewer of the gaming companies actually release demos of their games and as such the only “reliable” way of getting any hands-on experience with the game without buying it is to be part of the Beta which means that you get more people trying to test the feel of the game rather than the actual game and its engine.

    If more companies put out official demos I believe that the betas might get back to being a testing ground. But I may also be completely wrong 🙂

    • I actually think you are completely right, and that’s what my part 2 is gonna be about (great minds, as they say XD).
      What devs need to do is stop using betas as demos but instead have a clear-cut separation of the two and not be lazy about preview weekends, like Arenanet did with GW1.
      That way gamers have a chance to play a final, or close to final, version of the game and not feel the need to be in the beta and base their opinions on that alone.

      Very good thinking, Winter!

      • Winterbay

        Thank you 🙂

        I also believe, albeit this has less actual experience behind it, that quite a lot of today’s torrenting of games is done in order to be able to trial the game before you buy it. Unfortunately once you have a fully functioning game on your computer it is rather hard to remove it and buy the real thing instead and thus sales dwindle.

        Good demos would let these persons try out the game without getting the full deal and can then decide to buy it later on. A game which I felt had this well done is “The Chronicles of Spellborn” which however failed spectacularly, most likely (in my mind) due to their horrendous copy-protection scheme (II tried out the demo and after about 30 minutes my spyware program started to complain about one of the game files…)

  • I stumbled upon your blog last night. I’m an ex-GW1 player (it’s getting boring now, but who knows, I may pick it back up — I always seem to).

    I have never been in any Beta testings and, of course, would love to me in be in one. However assuming developers like ArenaNet want to put out a Beta version, they must be prepared for “negative nancys” (read all the comments too, lol).

    I think you hit the nail on the head saying “technology and social networking being what it is, there has never been a better time to test out games and help developers dig up the sorts of bugs only players have the magic ability to find.”

    Social media can be used by developers too. I’m sure ArenaNet community relations team has been following the Guild Wars 2 brand via tools like Radian6 and can select a handful of bloggers and Twitterers that would probably give them real good feedback as well as publicity. The DNAs need to go (if they are still here at all). Share it all and share it again. Don’t hand over the Beta to just about anyone. Using your example of painters, those painters need to qualified to paint your house in the first place — not just some guy off the street.

    Will there be bad feedback? Yes, of course but developers should be prepared for it and use it. Address the bad feedback and the blogger and his readers will see that they are fixing the bug or whatever.