July 22, 2016

Pokémon Go bugs: Gotta catch 'em all?


Unless you've been hiding under a Geodude for the past couple of weeks, you probably have at least a passing familiarity with the Pokémon Go craze. I was slightly too old to fall under its spell in the 90s, but am happily jumping on the nostalgia bandwagon this time around!

The game has had widespread media coverage, with pretty much every angle covered. We've got:

...and that's barely scratched the surface.

In amongst all of that is the game itself. Ironically, popular consensus among fans is that it's one of the worst Pokémon games to date; feature-wise it's bare-bones to say the least, and is heavily stripped-down from the AR game that developers Niantic made prior to it (Ingress). Then there are the constant crashes, server downtime and infuriating glitches (thought you just caught that rare critter? Better hope that your phone manages to sync to the server in time!)

There's no denying that the game has had its problems, to such an extent that SmartBear published a very interesting article, 4 Software Quality Lesssons from Pokémon Go's Wild First Week. But, as a counterpoint to that, here are a few other interesting things that we've learned:

MVP works, if you get the V right.

You might remember from my last presentation about quality in videogames that I was bemoaning companies who forgot the V in Minimum Viable Product: there's too much effort spent getting the first thing possible out of the door, without focusing on whether that product is actually going to serve a purpose for the audience. Pokémon Go got this spot-on, by delivering core gameplay which - whilst admittedly rather thin around the edges - gives Niantic a platform on which to build future features.

Many have complained that the game should have been labeled as a beta at best, but these are changing times that we're living in. With models such as Steam's "Early Access" program, continuous deployment and always-on internet connections, consumers' appetite for still-under-development applications is stronger than ever. Ask any Pokémon fan whether they'd rather wait another six months for the full game, or get the game now with future enhancement drip-fed over time; I bet very few would choose the former.

Subtle microtransactions can be the most profitable.

Of all the statistics you can pull from the success of Pokémon Go, none are more insane than the impact on Nintendo's share price. The company's market value rose eleven billion pounds during the week of the game's release, and is reportedly raking-in over $1million per day from in-game microtransactions (the game currently accounts for 50% of all mobile in-app purchases). If you've tried other free-to-play games, you probably have a mental model of how these in-app purchases typically work: constant pop-ups, locking away the game's best content behind a paywall, or limiting the amount of progress you can make in a single day.

What's most impressive about Pokémon Go is how it's gone against the grain with its purchase options. The store is buried deep inside a menu which you wouldn't notice if you didn't go looking for it, and the game doesn't ever push you in its direction. Almost all of the store items can be earned through playing the game normally (there are Pokéstops liberally scattered at landmarks across the world, which will grant you a handful of free items every few minutes), and there's no sense of a "pay-to-win" mentality. If anything, purchasing store items (such as lures, which increase the rate at which Pokémon spawn at your current location) are geared towards people who want to interact with other players in the real-world; buying a store item is almost like gifting a party to friends and strangers alike!

Hopefully the success of this subtle approach to in-app purchases will create a sea-change in the market, because lord knows the market needs it.

We still haven't taught users enough about app permissions.

The game had a staggered worldwide release, presumably designed to help manage server load/demand. This meant that UK players faced an agonising and potentially indefinite wait while their American counterparts were already engrossed in the game. (Even after the UK release, it was another week before the title launched in its home territory of Japan.)

Because of this wait, impatient players took to the dark corners of the internet, in an attempt to get their hands on the game before its official launch. There were even guides from major media outlets, such as Eurogamer's How to get Pokémon Go now, even in the UK. Their suggestions basically amounted to:

  • Configure your device to allow applications to be installed from untrusted sources;
  • Download an .apk of unverifiable quality (actual quote: "you may be asked about installing files not from your app store, but this is fine.");
  • Install the game and grant it full access to your Google account (the first release of the game required this access, apparently by accident).

People did this in their droves, at least in part due to the fact that the media reassured them it was OK to do so. Although this shows just how keen people were to start catching Pokémon, it also highlights that the average consumer (and, apparently, the average journalist) will still put their privacy at risk for surprisingly small potatoes.

It's time to refine our definition of quality.

To summarise, we can't just think of quality in terms of bugs, and the success of Pokémon Go illustrates just why that is. Despite its wealth of issues, Pokémon Go isn't just a game - it's a bona-fide social phenomenon, taking over the world in an incredibly short space of time, from a franchise which hasn't had mainstream recognition since the late 90s. By all normal measures, it should have been a disaster. Yet here we are: it's Saturday, and I'm wondering if I should pack a spare battery pack on my travels as I'm going to be visiting a new town tomorrow and I don't know whether my creatures are going to be strong enough to beat their current gym leader and OH GOD MAKE IT STOP

It's a subject we'll be discussing in this Sunday's Weekend Testing Europe: WTEU-70 - Measurable Quality, where myself and Amy Phillips will be presenting live on-camera for two hours from a secret London location. Why not join us and have your own say!

For a TLDR version, this week's episode of The Jimquistion sums-up why, in Jim Sterling's NSFW words/pictures, Pokémon Go is "the best worst Pokémon game ever" (skip to 2min35sec for the start of the review):

  • LinkedIn
  • Tumblr
  • Reddit
  • Google+
  • Pinterest
  • Pocket
Comments powered by Disqus