- this is a key takeaway
🐝 this is more key takeaway
The 5-user rule: Cost-effective and optimal usability testing
According to the reputable Nielsen Norman Group, ‘testing with 5 people lets you find almost as many usability problems as you’d find using many more test participants.’
The logic behind their ‘5-user’ suggestion is that as you test more and more people, you uncover fewer new insights at a higher cost. After testing up to 5 people, the same usability issues would continue to be mentioned by additional participants with very little significant change.
💡 Pro Tip
This is a super advanced pro tip
So it’s economical and optimal to test just enough participants who can give you sufficient insights at a low cost. Thus, the 5-user rule.
A quote by a super smart super star.
This rule has been analyzed further by Laura Faulkner in her study. As a result, we can see how the increase in the number of participants influences the percentages of the found usability problems:
When to use more or less than 5 participants
There are instances, of course, where the 5-user rule is not applicable: In some situations, the type of product tested requires a larger number of participants to obtain significant test results. For example, e-commerce websites can be much more complex than software products, so testing as few as five participants may only uncover 35% of their usability problems.
This is a comment or highlight it should grab your attention.
You may also need more or less than 5 participants for your usability study if you:
Have different target audience segments
When a product has multiple user groups, that behave differently, you’ll need to test participants that represent each group.
Kup si ma
Ideálne teraz fakt sa to oplatíSkús zadarmo
For instance, a marketplace app would need participants for both the buyer and seller profiles. In this case, you don’t need to have five participants for each user group since there would be overlapping observations, and you could use as few as 3 or 4 participants for each group.
The overlap of the findings will differ based on the similarity of the user subgroups you need to test. The more similar the groups are, the higher the overlap will be.
Need quantitative data
In conducting quantitative usability studies, your results must be statistically significant to obtain accurate insights. For this, you will need at least 20 to 40 test participants.
Since the focus is on measurable metrics and not qualitative findings, you will need many participants to get enough data that can accurately predict the behaviors of your overall target users.
Use an Agile UX process
In an Agile UX approach, you run multiple usability tests as the product develops since it is an iterative process. In this development style, you conduct tests, use the insights obtained to make changes, and then test the new version – and on and on it goes.
So rather than using five or more participants for each test, you can run multiple tests with as little as 3 participants each since there would be overlapping insights and discoveries. Using fewer participants also helps to save costs on your study.
Pros of using 5±2 participants in your usability study
There have been different optimal numbers of participants suggested by researchers over time. Having a minimum of 3 participants ensures that you capture diversity in your user group while having as many as 7 participants ensures that you uncover almost all the usability problems in the product.
So the ideal number of participants lies in a baseline of 3 to 7 participants for the following reasons:
1. It maximizes the law of diminishing marginal returns
In the usability testing context, the law of diminishing marginal returns states that as you test participants, your study will reach a point where increasing the number of participants will lead to fewer insights obtained. This graph put forward by the Nielsen Norman Group explains it better.
2. Testing 3 to 7 users is cost-efficient
When developing a product, one main reason usability testing faces a lot of pushback from stakeholders is the cost of conducting usability tests. Depending on the type of usability test, the number of participants recruited, as well as the product type among other factors, study costs can range from as little as $25 U.S dollars to $10K U.S dollars and even much more.
From recruiting agency fees to software costs and procuring incentives for your participants like gift cards, there are a lot of expenses and hidden costs incurred in running a single usability test. For example, if you are running a moderated usability test, you need to include the cost of the moderator’s time as well. And since costs rise as the number of participants increases, testing a few participants helps you minimize costs and maximize your budget.
Pro tip: Run a pilot test with 1-2 participants, before you recruit your target audience to test the clarity and comprehensiveness of your tasks, questions, and study overall. This will allow you to eliminate issues before the study goes live – minimizing the risk of running the study and collecting invalid or unusable data leading to wasted time and money.
3. Testing 3 to 7 users is time-efficient
Usability tests often take a lot of time, from setup to test sessions to analyzing the results. Depending on the number of participants and the type of study conducted – moderated or unmoderated, it can take a day or many days for completion.
It is estimated that conducting usability tests could take as long as 11 to 48 hours for only 5 participants. So rather than spending a lot of time testing many participants for your study, test only a few participants that would give you more or less the same results at a lesser time.