When it comes to usability testing, the biggest concern is always: how many users do I need?
Many companies are reluctant to do a usability test. They overestimate how much it will cost and the amount of users they need, missing a huge opportunity to learn about their customers’ needs and behaviours.
This article outlines why five users are needed for most usability tests, and flags when you need to test more.
Testing five users is cost-effective and practical
Testing five users in one day is cost-effective and practical for a business.
Colleagues, family and friends are not good candidates for usability testing as they’ll likely be familiar with your product, company and role. This can lead to bias. There are user recruitment companies who can source participants for usability tests who aren’t tied to your company.
In a European tech capital like Dublin, usability testing can cost approximately €100 per user with an additional incentive of €50 per user using a recruitment company. In the US, the average agency fee is $107, whereas in the UK, the total agency fee can cost around £100 to £120 per person.
Combined with recording software, usability testing can cost approximately 900 euros, dollars or pounds for an entire day.
Testing helps you uncover important user issues and insights, you really couldn’t make a more worthwhile investment.
Plus, you can do a comprehensive round of user testing in one day with five users. Once you go beyond five users, it’s an extra day and more resources.
To make the most of the business day, allocate an hour time slot to each user with at least thirty minutes break in between. You can use that thirty minutes to gather your thoughts and take notes on the previous session.
Five users secures sufficient user insight
Why is five the optimum number of participants for testing? Usually after three or four users, the same usability issues tend to crop up, there are clear patterns.
In ‘Don’t Make Me Think’, Steve Krug believes that three users are enough:
“The first three users are very likely to encounter nearly all of the most significant problems, and it’s much more important to do more rounds of testing than to wring everything you can out of each round.”
While he’s right in saying that significant problems will repeat, the reality is that users drop out of usability testing all the time. If you’re left with two users, it will be difficult to uncover the findings you need. You need more insurance than three users.
UX industry leader Jakob Nielsen has advocated using five users for years, saying: “Testing with five people lets you find almost as many usability problems as you’d find using many more test participants.
Having smaller user groups can make report writing more effective too. You’re adding more analysis and video compilation time once you go beyond five users. Aim to avoid making unnecessary work for yourself.
Why seeing is believing
Usability testing is just as much about stakeholder management, buy-in and sign off as it is about identifying issues.
You can build a strong case with video footage of user issues.
Colman Walsh (CEO, UX Design Institute) advises presenting clients with less of a report and more video footage. “Seeing is believing. Capturing the usability testing on video really hammers home the message that there were problems and what the problems were.”
He also suggests “following up the video footage with a fairly light report without too much commentary – the report outlines the problems with suggestions for how you might address them.”
Don’t miss the opportunity to make a video highlight reel of your users’ experience. There are many UX recording tools that can help you record in-person and remote usability tests.
Exceptions to the rule of five
There are some exceptions to the rule of five that are worth keeping in mind.
If you’re testing a website or app with lots of functionality, you should test five users for each function.
Imagine you’re designing an airline website and you want to test the purchase and check-in process functions. Rather than trying to cover both features in one test, it’s better to separate them into two. Ten users in total is recommended here, allocating five to the purchase process and five to the check-in process.
Some websites or apps may also have completely different user types with different behaviours. For example, in a property website there could be buyers, renters and sellers or landlords. In this case, you should test five buyers, five renters, five sellers and five landlords to get a good understanding of their different behaviours.
The rule of five might also have to be broken when stakeholders require statistical proof from usability testing. While this isn’t an ideal practice in a qualitative research method, sometimes it’s unavoidable.
Say a senior manager doesn’t believe that the company’s registration and login is ineffective and they want quantitative proof. You may have to test 20 users to show that 12 out of 20 people had difficulty creating a password or 17 out of 20 had problems logging in.
Ideally, try to persuade stakeholders to avoid going down this path beforehand!
Aim to stick with five users whenever possible. It’s ultimately more effective for your UX practice and the company’s goals in general.
