In this penultimate article, game developer Mike Mullins looks at how to take the feedback from your play tests, and process the development of your game into a smoother, more streamline gaming experience.
When I started writing articles about games for public consumption, they were most often about things I was playtesting. I consider myself objective, but I’d always delineate my relationship with the game and designer to be as transparent as possible. Here, I’m just laying out my own thoughts and processes pertaining to game development, so this entire disclaimer is somewhat superfluous. I’ve just done it this way for a long time; it’s kind of my thing now. Sorry for subjecting you to my compulsions! However, it is worth reminding you that game development will vary wildly for different types of people and projects, so take from what follows anything you might find useful.
What constitutes “advanced development?”
or: “What is this article actually about?”
I want to talk about the stage of a game’s creation where the foundation is solid. The game is fully-functioning and has been played by enough testers that you’ve gotten a fair amount of feedback. Let’s call this Beta version 1.5, which assumes you’ve already made a few rounds of changes. You are not only comfortable but also confident in the mechanics. How do we get to Beta v2.0?
The most important task
I’d be treading well-worn paths to tell you to ask your testers why they made a particular comment. However, it’s important that you take that answer and go a step further. For the purpose of our continued discussion, let’s say that you are working on a 54-card game with a target play time of about 30 minutes. In the feedback from a blind playtest, a tester expresses frustration because her opponent could play cards with impunity while she struggled to get going.
While every piece of feedback is important, placing it in the proper context is crucial to properly responding as a designer. Without being able to observe the game and consider factors such as player aptitude or luck, you need to compare other factors across the instances of similar feedback. As you scan for these comments, take note of the other “stats,” and look for any confluences of the listed factors.
- Was this a first and/or only play? – possible learning curve
- What were the scores? – could indicate a skill or luck disparity
- What was the play time? – also hints at skill-luck gap
- What was the player count? – mechanisms might not scale properly
- What was the overall rating? – may indicate player understands the game so the issue could be an anomaly
How About An Example?
After combing through feedback (and your own testing notes!), you find several comments having to do with a difficulty playing cards. I’ve named them for easy reference later.
Ann: “I struggled to get going,” 15 min game, 2 players, 20-7 loss, 8/10 rating
Lou: “I didn’t really know what to play,” 30 min game, 2 players, 25-9 loss, 5/10 rating
Tim: “Nothing seemed to click,” 20 min 3 player game, 18-15-10 loss, 7/10 rating
and from one of your notes: “12 min landslide playing both sides. 8 cards in hand?”
To borrow a line from a Christmas song, “Do you see what I see?”
Three of the four had significantly shorter play times, and (assuming you like your own design) gave the game a decent rating. Lou was on the short end of a lopsided score, gave the game a mediocre rating, and used the somewhat unique verbiage of “didn’t know” as opposed to “couldn’t.” This could very well indicate he wouldn’t enjoy your game even if some adjustments were made, so lets focus on the other three.
Given comments, playtime, and ratings that are roughly equivalent, the scores will hopefully provide us our next clue. In the three player game, even though Tim couldn’t get a strategy to work, the scores were relatively close. In your mirror match, unless you were trying out a radical strategy for one side, luck likely played a big factor. Finally, though there is no definitive way to conclude this, you might venture that Ann was cold-decked (couldn’t draw the right cards) because she loved the game even after getting crushed.
Ok, awesome. Um, now what?
You noted that you had 8 cards in your hand, which is significantly above the 5-6 card average, and remember that Ann’s opponent “could play cards with impunity.” Is there a hand limit issue? Would a restriction mitigate the apparent luck factor and add significant decisions? You can take this idea and scan the comments again, or forge ahead with this question in mind for future tests.
While much of this stage of development may be balancing, that can usually be achieved simply with volume of plays and noting which cards/abilities lead to victory more often than not. Although game balance is an important part of this process at this stage, the main reason you need to address it now is so the game itself can be fairly evaluated. Let’s say a small percentage of cards in your game increase the draw rate, while others provide the energy to play multiple cards per turn. Although a single card, played commonly enough during playtests, can drastically affect a player’s perception of gameplay, don’t be afraid to restrict something globally or drop one or more elements altogether to diagnose a problem. In this case, you may decide to add a hard hand limit of five, superseding the effects on other cards. Once you observe the results from implementing this global change, you can begin to fine tune your rules, card effects, and card counts to fit your purpose.
Ask yourself “Why?” too.
Whenever I’m testing for someone, I’m constantly questioning their choices, forcing them to defend their design. This is more than just indulging my narcissism or playing devil’s advocate; if you can’t remember why you put something into your game, it probably doesn’t have to stay in there. These mystery elements are often the product of your initial filling of design whitespace. Sometimes these artefacts of creation fit perfectly into your developing whole, but they can often be cut to streamline your gameplay.
Since I’ve decreed that you’ve designed a card game with a potential hand limit problem, let’s ask some questions. Why were the “draw extra cards” effects added? Was it to fill out the deck because it’s a common feature of card games? Did you add them to provide necessary strategy, or to address a gameplay concern? Why should they be kept in the game at the cost of adding a hand limit rule to mitigate their impact? If you can’t come up with a convincing justification for keeping these imaginary cards in this hypothetical game I’m attributing to you, they should be removed.
Handling aberrations, or, Why bother fixing that?
I’ve hopefully impressed upon you the importance of identifying trends in your feedback, but what about the outliers? For the most part, unless a comment can be properly contextualized and applied to a currently established trend or design goal, it should be noted and addressed on the occasion it comes up again. Alternatively, if you have the opportunity to replicate or otherwise validate the issue on your own, you can go into the tank and address it then and there. There are times when an unexpected comment or an odd game report really captures your attention. It may come from someone whose opinion you value highly, or touch on something you’d considered once before and set aside. Ask yourself, “is this an issue I can tolerate happening again, or does it negatively impact the game experience enough that it warrants correction?” Two specific instances that likely require your intervention are “broken” elements, and subversive strategies.
I consider a game to be “broken” when, given a certain situation, one course of action leads to a near-guaranteed victory. This often comes in the form of “whoever gets X first always wins.” The issue, outside of simple balance, is that the “game” element has been removed; all players will be on a similar trajectory to achieve X, in a near-vacuum of decision making. If you can convince yourself that the game leading to condition X is sufficiently compelling, X should be the win condition, not the clarion call to assemble and await the winner ‘s coronation. More often, that decisive element will require removal or restructuring, allowing your game’s true vision to be realized.
What about subversive strategies? First, I want to emphasize that I’m not referring to unique strategies. In fact, you are far more likely to attract attention if your design accommodates multiple viable strategies, even (especially?) if some of them are considerably more difficult to execute well. A “subversive” strategy intentionally manipulates game elements in a way that isn’t consistent with your desired game experience. While you can’t dictate the way players enjoy your game, these types of play styles often ruin the game for the other players at the table, therefore warranting your attention. A simple example is a player completely paralyzing a resource management game by relentlessly hoarding, which is easily addressed by adding a maximum storage capacity.
One more “Why?” – Why add something new?
As difficult as it may be, you should be resisting unrestrained ideation at this stage (a rule I’m notorious for breaking. Bad developer! Bad!). That is not to say your game cannot benefit from additions as you move into late beta; the key word here is “unrestrained.” With your vision largely realized, creativity now needs to be married to purpose. Too often I’ve seen solid progress delayed, or worse, derailed, by the inclusion of otherwise great ideas.
So, when can we add stuff? Ideally additions are only in response to the observed feedback trends described above. Even then, addition should only follow attempts to streamline gameplay by refining or removing. Careless addition of new parts can obfuscate the underlying issues your already-functional game has. Don’t add rockets to your new car, but maybe consider air conditioning if the passengers are too hot?
Wait, rocket cars? I’m listening…
Here is your loophole creative people: sometimes an idea is too good to pass up. At the end of the day, you need to make the game you want to make. Inspiration is rarely considerate of timing, and you may very well have a brilliant idea that you cannot ignore. This is ok as long as you realize that a substantial change doesn’t necessarily get you closer to your final product. If you are willing to change the foundation of your game rather than save new material for a variant or expansion, you should also be willing to commit to more rounds of rigorous testing.
You are an expert now!
(After misspelling “expert” above, autocorrect replaced it with “escort,” making for a much different subject heading)
Odds are there are some assertions made above that you disagree with, and hopefully some that you hadn’t considered. I encourage you to seek out more advice, try new development techniques, and be open to change your philosophy and processes. While you will undoubtedly have a style that is all your own, challenging established patterns is how growth is achieved.
Thanks for reading, and please feel encouraged to ping me @bluedevilduke with questions or comments.
Mike is the co-designer of Bottom Of the 9th and has worked on the development of many games for companies as diverse as Dice Hate Me Games and Three Hares Games, and has specialised in creating official solo variants for Castle Dice, Lagoon, and the Compounded Geiger Expansion.