Usability Testing and What You Need to Know

Usability testing is essential to creating any product or service. It helps designers and business stakeholders better understand what is and isn’t working from a user’s perspective. 

It can be difficult for a team to distance themselves from their work and the needs of the business, making it hard for them to spot errors or things that aren’t working well in the product. 

By having people who represent your users perform usability testing on your product, they will be able to give you an unbiased review of how well your product works with the audience it’s designed for. It also allows designers to fix cumbersome interface issues before a product is fully released to the market. 

What is Usability Testing?

Simply put, a usability test is a tool that a design or development team can utilize to ensure that their product or service performs the way it was intended to. Usability testing can occur as early as the initial design conception, during the production phase, and even years after a product or website has been released to ensure that nothing has broken or become outdated in the technology. 

The reason usability testing is so much more effective than simply allowing the designers to test it is because of direct user input that can be incorporated into the final design. 

User Testing vs. Usability Testing

While user testing and usability testing overlap in some instances, they each provide unique and valuable feedback to the design team. 

User testing, or what is now more preferably called generative research to not imply the user is themself being tested, will come before usability testing in a product’s lifecycle. This form of testing is used to understand the need for a product and its value to its end users as a means of solving a problem or helping them achieve a goal. Generative research can be performed through tools like surveys, interviews, or observation. 

On the other hand, usability testing is putting the product directly into the user’s hands, whether as a prototype or an actual product, so that we can test and measure how they interact with it. We can then use that feedback to iterate on the product, so we can test and measure its performance again with an improved design.  

When Does Usability Testing Happen?

Usability testing can happen at any stage during a product life cycle. In medical product applications, rigorous usability protocols must be satisfied to release a product (see IEC 62366-1) to ensure that they are safe for users according to human factors guidelines. In these cases, the stakes are high for good usability. Usability testing during a product’s design and development is often called formative. Usability testing that takes place directly before (or sometimes after) being released is called summative and is generally much more rigorous with pass/fail criteria that may inhibit a product from being released.

Formative 

Formative usability testing is performed throughout the design process of a product. It allows a design team to get feedback and iterate on designs rapidly while the product is still being designed. 

Formative testing is efficient because a design team does not need a complete product or even a working prototype to begin getting feedback. This type of testing can start with a prototype as long as the sketch is detailed. 

Formative testing allows a design team to test numerous different processes while a design is fine-tuned. For example, for a flight booking app, the process in which a person books a flight could be tested to see if the process flows well and makes sense, where tasks like making a payment are put on the back-burner. 

By getting input back from users early and regularly throughout the process, a design team can iterate on designs quickly and in a low-cost way. 

Summative

Summative usability testing occurs toward the end of the product’s design process. Once a full working prototype or product has been developed, it can then be placed in front of users to interact with what will become the final product. Some companies prefer to do this before a product is released. Others might do this immediately after.

While we recommend that designers perform formative and summative testing for their products, it is not always feasible. If you can only do one type of usability testing, then summative testing is essential. However, if this is only testing you have time for, be prepared that you may be rebuilding or redesigning more significant pieces of the product.

End-stage summative helps designers identify usability issues with the product,  especially problems that make the product too broken to be released. It often involves having users step through every task that the product supports.

What Should be Tested During Usability Testing

The answer to “what does usable mean?” usually differs depending on the product. Most products have different success criteria, use cases, and heuristics that must be tested. For example, we would not typically test a medical device the same way we would test the usability of a retail store, as each has a different set of usability criteria that must be evaluated. Usability tests determine how well a product performs the tasks it was created for or how well it helps users accomplish their goals. Some common things we test include:Task completion efficiency – How successfully are users able to perform tasks in the system? Where is there friction?

• Usability issues – What are the causes behind errors that users experience? (e.g., labeling, signage, navigation, interaction design, etc.) How frequently do these issues occur?

• Key performance metrics – How well does the system perform compared to benchmarks from similar services or previous product iterations?

• Heuristic evaluation – How well does the system comply with usability heuristics in its domain?

• User satisfaction – How satisfied are users with the product’s design? (these kinds of metrics are only viable for more extensive quantitative studies)

Are there opportunities for improvement that we’re missing?

These elements of usability testing are tied to formative and summative testing. There are also more tactical approaches to usability testing, such as comparative testing- how usable is one design versus another? In comparative testing, users will be shown different versions of a product, and usability is measured between both designs. Comparative testing is typical with larger sample sizes in unmoderated settings. 

Who Participates in the Tests?

One of the most significant differences in usability testing is whether the test is moderated or unmoderated. 

In a moderated test, a person representing the design team remains present for the entirety of the usability test, sometimes accompanied by a notetaker or representatives from our client. This allows users to give their feedback instantaneously as they are experiencing it and allows the moderator to ask questions as the test progresses. Typically moderated tests include a small selection of representative users and are recorded.

Moderators can actively watch the participant use the product and note where they tend to linger in the process, what parts seem to frustrate or give them difficulty, and what they seem to enjoy the most. Moderated testing provides the most value to a design team but usually means a smaller participant group within a specific time frame and/or budget. 

In unmoderated testing, on the other hand, the participant will typically receive a digital prototype of the product that they can use. They will narrate on-camera or potentially write a review of their experience afterward. Sometimes these studies are run with larger sample size. This can be efficient and cost-effective, provided that the test has been thoughtfully crafted and the participants are precisely within the user group. That said, the main trade-off here is that there’s not a chance to ask the user any questions during the test or dig deeper into questions they miss or answer shallowly.

The Importance of Continued Usability Testing

Usability testing continues to be beneficial even after a product has been released. This allows designers to see what people continue to enjoy about the app or website and what they are frustrated with and want to be improved. It allows us to capture valuable data from live apps and use that to see where users experience friction compared to how many continue to use it. Most importantly, continued usability testing enables us to iteratively explore, measure, improve and iterate on our designs based on user feedback. Continued usability testing allows us to continually improve the user experience, which leads to more satisfied customers that will be more likely to use our products in the future. 

Want to learn how Grand Studio can help with your next project and build clarity out of complexity?

We’re here to help!