x

Register Now to Beegin Your Journey!

Register Now For Free to Beegin Your Journey!

Register Now to Beegin Your Journey!Register for free

Usability Testing Studies: How Many Participants?

Tadeas Adamjak Tadeas Adamjak
Marek Strba Marek Strba
•  22.01.2024

Usability Testing Studies: How Many Participants?

​​Selecting the optimal number of testers for your usability tests can help you save costs while getting maximum insights in a shorter time. So how do you decide on the number of usability testing participants? All the answers can be found in this article...

normálny text

Key Takeaways

  1. this is a key takeaway

🐝 this is more key takeaway

The 5-user rule: Cost-effective and optimal usability testing

According to the reputable Nielsen Norman Group, ‘testing with 5 people lets you find almost as many usability problems as you’d find using many more test participants.’

The logic behind their ‘5-user’ suggestion is that as you test more and more people, you uncover fewer new insights at a higher cost. After testing up to 5 people, the same usability issues would continue to be mentioned by additional participants with very little significant change. 

💡 Pro Tip

This is a super advanced pro tip

So it’s economical and optimal to test just enough participants who can give you sufficient insights at a low cost. Thus, the 5-user rule.

A quote by a super smart super star.

Braňo Mojsej

20+ rokov ex. Alkoholik, Profesionálny abstinent, motivačný hovorca a autor

This rule has been analyzed further by Laura Faulkner in her study. As a result, we can see how the increase in the number of participants influences the percentages of the found usability problems:

When to use more or less than 5 participants

There are instances, of course, where the 5-user rule is not applicable: In some situations, the type of product tested requires a larger number of participants to obtain significant test results. For example, e-commerce websites can be much more complex than software products, so testing as few as five participants may only uncover 35% of their usability problems

This is a comment or highlight it should grab your attention.

You may also need more or less than 5 participants for your usability study if you:

Have different target audience segments 

When a product has multiple user groups, that behave differently, you’ll need to test participants that represent each group.

Kup si ma

Ideálne teraz fakt sa to oplatí

Skús zadarmo bee

For instance, a marketplace app would need participants for both the buyer and seller profiles. In this case, you don’t need to have five participants for each user group since there would be overlapping observations, and you could use as few as 3 or 4 participants for each group.

The overlap of the findings will differ based on the similarity of the user subgroups you need to test. The more similar the groups are, the higher the overlap will be.

Need quantitative data 

In conducting quantitative usability studies, your results must be statistically significant to obtain accurate insights. For this, you will need at least 20 to 40 test participants

toto je koment element používame ak chceme dať nejakú poznámku – napríklad, že editor hovorí, že existuje rozdiel medzi elentom A elementom B ak sa A nerovná B.

Since the focus is on measurable metrics and not qualitative findings, you will need many participants to get enough data that can accurately predict the behaviors of your overall target users. 

https://www.youtube.com/watch?v=XD2hLMedjM4&ab_channel=UXtweak

Use an Agile UX process

In an Agile UX approach, you run multiple usability tests as the product develops since it is an iterative process. In this development style, you conduct tests, use the insights obtained to make changes, and then test the new version – and on and on it goes.

So rather than using five or more participants for each test, you can run multiple tests with as little as 3 participants each since there would be overlapping insights and discoveries. Using fewer participants also helps to save costs on your study.

Pros of using 5±2 participants in your usability study 

There have been different optimal numbers of participants suggested by researchers over time. Having a minimum of 3 participants ensures that you capture diversity in your user group while having as many as 7 participants ensures that you uncover almost all the usability problems in the product. 

So the ideal number of participants lies in a baseline of 3 to 7 participants for the following reasons:

1. It maximizes the law of diminishing marginal returns

In the usability testing context, the law of diminishing marginal returns states that as you test participants, your study will reach a point where increasing the number of participants will lead to fewer insights obtained. This graph put forward by the Nielsen Norman Group explains it better. 

2. Testing 3 to 7 users is cost-efficient

When developing a product, one main reason usability testing faces a lot of pushback from stakeholders is the cost of conducting usability tests. Depending on the type of usability test, the number of participants recruited, as well as the product type among other factors, study costs can range from as little as $25 U.S dollars to $10K U.S dollars and even much more. 

From recruiting agency fees to software costs and procuring incentives for your participants like gift cards, there are a lot of expenses and hidden costs incurred in running a single usability test. For example, if you are running a moderated usability test, you need to include the cost of the moderator’s time as well. And since costs rise as the number of participants increases, testing a few participants helps you minimize costs and maximize your budget.

Pro tip: Run a pilot test with 1-2 participants, before you recruit your target audience to test the clarity and comprehensiveness of your tasks, questions, and study overall. This will allow you to eliminate issues before the study goes live – minimizing the risk of running the study and collecting invalid or unusable data leading to wasted time and money.

3. Testing 3 to 7 users is time-efficient

Usability tests often take a lot of time, from setup to test sessions to analyzing the results. Depending on the number of participants and the type of study conducted – moderated or unmoderated, it can take a day or many days for completion.

It is estimated that conducting usability tests could take as long as 11 to 48 hours for only 5 participants. So rather than spending a lot of time testing many participants for your study, test only a few participants that would give you more or less the same results at a lesser time. 

FAQ

How many respondents does your study need?

Depending on the study question, the test’s design, and the level of confidence that researchers want to place in the results, a different number of participants may be required for usability studies. A general rule of thumb is that 3 to 7 participants are a commonly recommended sample size for usability testing.

However, it is also important to understand that a larger number of participants may be required depending on the research method and the type of data you’re looking to gather. While in usability tests where you’re gathering qualitative data about the user’s behavior 3-7 participants may be enough, it’s different for the quantitative research studies. In card sorting, for example, to get accurate statistics you may need around 30 to 40 respondents and sometimes even more.

 

Share on socials |

Read More