Design: Playtest, playtest, playtest.

Over the next few weeks, Whose Turn Is It Anyway is running a series of guest articles about board game design.
In the second article, designer Matt Wolfe, talks about what to do once you’ve designed your game. His focus is on what is possibly THE most important phase of design, playtesting. Matt talkes us through the three stages of playtesting games.


 
Today I’m here to talk to you about playtesting your designs. But before we get into all of that, I want to go over the golden rule of playtesting your designs: Do not get defensive about feedback. Ever. Playtesters are donating their time to help your design. If they have negative feedback that feedback is about your design, not you. It can be difficult to separate your feelings for your creative work from your feelings about yourself. But know that you are not your design and your self worth and value are not defined by the designs you put in front of playtesters. It can feel like criticism of your design is aimed at you, but it’s not. The best designers can compartmentalize critical feedback about their design and not let it affect them personally. It takes time and practice, but you’ll need to get to the point where you can not let critical feedback affect you personally.On to playtesting! Playtesting is a catch-all term used to describe the process of playing an unpublished design for the purposes of improving or testing the design. There are several stages of playtesting a design, and almost every design will go through these stages:

  1. Early “initial” playtesting
  2. Later “alpha” playtesting
  3. Final “beta” playtesting

Let’s talk about these stages in order.

 

Early “initial” playtesting.

You have a game idea and you made a quick and dirty paper prototype to test out that idea. Congratulations! You’re doing initial playtesting. In initial playtesting you’re testing ideas to see how (and if) they work. Your goal should be to rapidly iterate your prototype and try out lots of ideas to see what works, what’s fun, and what needs to go. Playtesting is generally just you playing multiple players at once in order to test the design. Some designers will also enlist their spouse or a trusted partner for testing at this stage. What you’re not worried about at this point is any kind of numerical balance in your game. Some people refer to this stage as “finding the fun” in the design. I don’t like that verbiage because not all games are “fun” for all people. I like to think of this stage as finding a compelling core to your game. You’re trying to find what makes the game compelling enough to want to play more than once.

Keep careful notes during this type of playtesting. Use a design journal or a Google doc or anything that works for you and will help you keep notes. Take notes on what worked, what rules you were using, what the fun or compelling part of the game was, and what didn’t seem to work. For subsequent playtests you can note what you changed from a previous iteration. Sometimes that change will be something small like a rule change (although you’ll be surprised how often a small change will make a big difference in a game) or it could be an entirely new mechanism. Noting the changes and how those changes impacted the game is very helpful for down the road when someone suggests something that you tried before and you can go back to see what happened when you tried it and what the result was. (And for when you want to write a design diary on BGG about your game!)

I’ll give you an example from my own playtesting experience. For my game, Wombat Rescue, the very first playtest was with my wife and myself. I had a rule that you could move your lead wombat in any direction and in any manner you wanted on your turn. We quickly found that it wasn’t very fun to do that. So I made a change about midway through the first game that the lead wombat could only move in a straight line on the hex grid. That small change made a world of difference and all of a sudden movement became very interesting. You had to do a lot more planning and thinking ahead when you were figuring out how and where to move.

And that example illustrates a point: don’t be afraid to make a change mid-playtest or even end a playtest early if you can see the initial design isn’t working. An initial playtest is a success not if it makes you feel good about your design but if it reveals an issue you need to resolve. I could clearly see (and my wife agreed) that the movement wasn’t working and it didn’t make any sense to keep playing under the same rule. A quick rules tweak for the remainder of the playtest made the remaining time worth it.

Avoid the urge to involve outside playtesters at this stage. Outside playtesters will want to offer feedback and may suggest things that are contrary to where you want to take the game. Outside playtesters may not be willing to play a design that is most likely broken because it is so early in the development cycle. Or they may give feedback that is too harsh because they were expecting a fully working game. Try to wait until the alpha stage to open up your design to outside play testers.

Similarly, designs in this stage should not be shared as a print and play (PnP)! It’s a waste of playtesters time to take the time to assemble a PnP and provide feedback when the state of the game is still in flux. Playtesters are a precious commodity. Do not abuse their generosity by soliciting feedback via PnP prematurely.

Don’t worry about quantitative data about this point. Because the rules are changing so often any quantitative data will most likely be out of date and no longer applicable when you move on to later playtesting. You’re looking solely for qualitative feedback at this point.

After you have playtested your game many, many times and have started having fewer and fewer rapid changes between playtests, it’s time to move on to “alpha” playtesting. Be aware that some designs never make it out of the initial stage and that’s OK! Not every idea was meant to be.

Goals of initial play testing: Rapid iteration, finding what works and what doesn’t, finding what makes the game fun and compelling.

What not to do: Balance the game, engage PnP play testers, solicit quantitative feedback.

 

Later “alpha” playtesting.

When do you know when to move on to alpha play testing? When you stop having a rule change after every single playtest, you’re probably ready to move onto alpha playtesting. Alpha playtesting is where you’ll work on further refinement and balance of the game and can start to open up the game to outside play testers.

For this stage of playtesting, you want to stop participating in the actual playtest and will want to observe (if possible). Watch carefully at the reactions of players to events in the game. Are players invested in the playtest? Or are they sitting there playing on their phone? How did they react when something negative happened to them? Are players enjoying themselves? Any “stand up” moments in the playtest? Any clever moments in the play test? How much chatter is going on during the playtest? How much are players concentrating on the game and not interacting otherwise?

You’ll notice that these types of questions are all “soft” questions that won’t have an absolute, correct answer. They are also questions that you’ll will have to be able to answer the best you can based on your interpretations of how the players are reacting. Being a careful observer of humanity will help here! You can also ask playtesters to fill out a feedback form after the playtest. You should also debrief the playtesters after the playtest.

Try not to interject during the game unless someone asks you a question. Also try not to pause the playtest to ask a player about a specific decision they made. Instead write down notes while you’re observing the playtest, including specific points you want to ask players about afterwards. You can also provide paper and pens for playtesters to write down any thoughts they have during the playtest instead of pausing to mention their thoughts. Sometimes playtesters will have a thought during the middle of the playtest that they will then erase when more of the design is revealed and they realize their thought is no longer relevant.

If players consistently make a mistake while they are playing, don’t correct them! You’ll need to analyze why players are making that mistake. It could be that what you think is a mistake is actually something that you should allow the players to do. Or it could be something that is confusing in the rules of the game or the prototype UI. Mistakes that players make during the playtest can be very illuminating on problem areas in your design. Don’t assume that the issue is with the player. It’s certainly possible that the player is the issue, but it’s more likely that something in your design is the issue. If players, especially multiple players, consistently want to do a thing in the game, pay attention to that impulse! It might be something you should incorporate into the design.

I’ll give you an example from my design experience for Wombat Rescue. Early on in the design there was a food track. Each time a lead wombat ate a piece of food, players were supposed to move the marker up on the food track 1 space. When the food track marker reached 15, the food spaces were refilled on the board at the end of that player’s turn. The problem was most players never remembered to move the food track marker up when they ate food. This was a consistent issue with that mechanism. I then tried putting food discs on the food track when the food discs were digested and moved off the player boards. And while that was easier to remember it was a little too fiddly for the game. I eventually ended up throwing away the food track altogether and incorporating refilling food in another way that was much more integrated into the existing mechanisms in the game. And that suggestion was thanks to a playtester (and a fellow designer).

Alpha testing is also the time to work on numerically balancing your design. Balancing a game is beyond the scope of this article, but now that your rules are largely solidified you can focus on balance. Trying to balance the game before the rules are largely pinned down is a waste of time as any balance you do can easily be undone by rules changes. Resist the urge to balance too early!

During alpha testing you’ll want to have players fill out feedback forms so you can start to gather quantitative data. You’ll want to track which player in player order won (if your design is a turn-based game), the final scores of all players, the delta between the winner’s score and the player in last place, and lots of other data. You’ll want to make extensive notes about the strategies the players used and how those strategies worked out. You’ll want to have your playtesters record their qualitative thoughts about the game. Remember to not take any critical feedback personally!

If your playtesters have the time, you can debrief them after the playtest is finished. The types of questions you ask depend largely on the style of game you’re designing. For example, “Did you feel like you were making friends or enemies with the other players?” is a great question to ask if your game has social elements that you want to emphasize. “Can you explain why the victorious player won?” is a great question for a strategy game.

Be sure to ask probing questions about feedback your receive. This is your opportunity to ensure you get an accurate understanding of the feedback. If a playtester has a concern, acknowledge that concern and try to ask “why” questions to get to the root of the issue. “Why do you feel that way?” “Why do you feel that is an issue?” “Why do you dislike that action?” Questions like that.

Be wary of suggestions made from playtesters. Thank playtesters for suggestions, write the suggestion down, and honestly consider the suggestion. But at the end of the day this is your design and you are the only person who knows what the game should be like. Sometimes playtesters will have their own ideas of what the game could be and those ideas can be at odds with what you want the game to be. Sometimes what a playtester is suggesting is actually covering up what the real issue is. It’s up to you to critically evaluate what the playtester is saying and what issue they are attempting to resolve.

Alpha playtesting is the time to work on the UI of your prototype. Do not spend a ton of time trying to make a “pretty” prototype. Do not pay for art. What you want to do is make a functionally effective prototype. That means cards are consistently laid out, text is clear and readable, the board is arranged to make logical sense, etc. The goal is to start to figure out what makes sense and helps players interact with the game. For example, if your design has cards and players need to hold a hand of cards, think about how to lay out the information on the cards so a player can see the important information when the cards are fanned in hand. If you have a board, think about how to segment different sections of the board so players can tell different areas apart.

Some designers will put together a PnP for their design at this stage. I recommend limiting the PnP just to trusted playtesters who cannot playtest in person. Save an open PnP for the beta stage of playtesting.

It’s possible that you may uncover a major issue with your design that would cause you to take it back to the initial playtesting stage. That’s OK! That’s what playtesting is for. Better to find the issue now than when you posted a PnP or sent the game to a publisher to evaluate.

Goals of alpha play testing: Locking down rules, initial balancing, gathering quantitative and qualitative data, improving the UI.

What not to do: Make rapid changes to the design, conduct blind playtests.

 

Final “beta” playtesting.

Beta playtesting is the final step for testing a design. Beta is when the rules are final, the design has been balanced to the best of your ability, and the UI of your design is refined and understandable. The purpose of beta playtesting is to attempt to break the design and test the rulebook. Beta playtests are usually done blind, meaning the players will learn the design from the rulebook. Blind playtesting is critical for ensuring your rulebook makes sense and people can actually learn the game without you there to teach them. If you cannot watch the playtest yourself, ask the players to record the playtest so you can watch it later. Most modern mobile phones have an excellent video camera and playtesters can upload the video to YouTube and mark it private if you do not want the playtests in the public eye.

Beta stage is the best time to have a PnP for your design. Unless you’re lucky enough to have new playtesters locally, you will have to produce a PnP for blind playtesting.

Try to seek playtesters who are generally very savvy players and are used to trying to break designs. Feedback at this stage should be gathered in extensive spreadsheets to track what players did during the playtest. Listen carefully to your playtesters and make sure you’re not making any changes based on just a single data point.

Goals of beta playtesting: Break the design, break the rulebook, fine-tune the balance of the design.

What not to do: Don’t make changes based on a single data point.

 

Other thoughts.

Try to not use the same group of playtesters for your game all the time. Try to test with as many different groups as you can. Approach other designers, approach people are your local FLGS, ask for feedback from a PnP, use your local game group, etc. Using the same group over and over can lead to bubble thinking and a lack of a fresh perspective on the game. Sometimes a player with new eyes for the game can shed light on a problem that people too close to the game never considered.

Before you playtest with outside playtesters, make your expectations and goals for the playtest clear. If you want them to focus on a specific portion of the game, say that. If you want them to try weird stuff, say that. Helping playtesters understand the specific purpose of the playtest will help those playtesters have direction the type of feedback you’re looking for.

Always be respective of your playtesters’ time and thank them profusely for their help. Ask if it’s OK to include their name in the rulebook if the game is published. You can also ask for contact information if they would like to be notified of any progress made with the game, such as a Kickstarter project launch or if you need more playtesting in the future.

And that’s it! Playtesting from start to finish. We haven’t covered playtesting using a digital platform like Vassal or RollD20.net, nor have we covered what to do when your design isn’t working. Those are topics for another day. 🙂

 

Matt Wolfe is the designer of Wombat Rescue, which is scheduled to be on Kickstarter in May 2015. You can find him on Twitter at @mattwolfe.

Leave a Reply