In a recent edition of The Startup Podcast, product management consultant and author, Marty Cagan outlined the power of design prototypes to streamline product development, and this quote could not be more relatable to the topic of prototype user testing:
“Using our engineers to code is kind of our last resort when it comes to discovery, because that’s the slowest, most expensive way to create a prototype… A designer can do 10; 20; 50 iterations in a week.”
Cagan went on to say that design prototypes are a great way to answer questions like: do we have enough value? Is our product feasible? Is it viable?
Prototypes are an essential tool; however, assessing prototypes only based on your opinion and the opinions of those around you lacks the credibility of testing with a wider group and limits the helpful feedback you can gather. Fortunately, prototype user testing, with a significant pool of participants, is within easy reach.
Table of contents
- What exactly is prototype user testing?
- Understanding the benefits of prototype user testing
- Different Methods for Prototype Tests
- Kinds of prototype user tests
- Kinds of tools available for online tests
- Prototype user testing for each project stage
- How to conduct a prototype user test
- What comes after prototype user testing?
- Conclusion: Prototype Testing is imperative for product success
What exactly is prototype user testing?
Prototype user testing involves evaluating a preliminary version of a product with a group of target users to assess its design, functionality, and user experience. The goal is to identify any issues and gather feedback early in the product development process.
Potential users of a product are asked to state their preferences or perform a set of tasks on a prototype of your product, to see how well they can orient themselves within it. Based on the results of these test, logical issues in the structure or design can be uncovered and reworked.
Prototypes can be made of any digital product and are a lot less time consuming to design and redesign than the actual products themselves, making the testing and consequent retesting less time-consuming likewise. That is why it is a valuable and crucial step in the development of your product.
Aspects of prototype user testing covered in this post
If you work in or study digital product development, this post outlines ways to validate ideas quickly and with some conviction. Product Designers and Product Managers, but also Researchers, Writers, and Information Architects can all benefit from prototype user testing of various kinds.
Although almost everyone shares concepts with others, this article will help you actively test and collect both qualitative and quantitative data. Being familiar with Figma is an advantage, however, the tasks in Figma that will be outlined in this post are simple, even for novice users.
User testing is the process of observing how people behave when interacting with the test product. Usability testing is the part of a user test which asks: ‘is this product capable of enabling the target audience’s goal?’ Confusingly, and understandably, it’s common to talk about a prototype usability test, when we actually mean the complete prototype user test.
Understanding the benefits of prototype user testing
If you’ve found this post, I’ll take it that you don’t need too much convincing about the value and importance of prototype user tests. However, there are some perspectives you may not have considered, and which are good to know as you proceed with testing.
Improve product value, discover unexpected ideas
Almost any user test has the potential to bring up responses which challenge the tester’s preconceptions and open pathways to alternatives. These may lead to slight modifications or more significant (and more valuable) pivots in approach.
Prototype User Testing is lower cost than coding
As the Marty Cagan quote earlier indicated, ideas are ‘cheap’, whereas using developers to test ideas is almost always expensive and inefficient.
The cost of coding is threefold: the money you pay the engineer; the time your project needs to wait while the code is made; and the lost opportunity cost, where the engineer could have been adding more value elsewhere. The cost of prototype user testing is a far smaller amount of design time plus the time and money cost of running the test, which is low when using online tools.
During the concept design stage, a prototype can be as simple as a hand-drawn set of screens, and a user test can be created for a design prototype within minutes. Quickly designing and user-testing prototypes is low cost, high-frequency iteration, which helps uncover and illuminate the best ideas to proceed with.
Save a lot by not failing in the market
While we’re talking money, one of the best ways to burn a startup runway is to launch a product which fails to achieve significant product-market-fit. All the hours and funds are put into research, tools, ideation, development, engineering, and marketing. External partners will not see a return if users do not take up the offer of your product.
This is why using prototype user testing early in the product development process will uncover better ways to help users, making a vital contribution to your product’s chance of success.
Get external, somewhat unbiased data from users
Most people working in product development share their ideas with the people they work with. While quick and somewhat valuable, this form of ‘testing’ is easily distorted by group-think, small sample risk, and a reliance on opinions and other undocumented, qualitative results.
Engaging a reasonable scale pool of external testers should give you more certainty, as well as a fairly solid foundation for internal discussions.
Prototype user testing is suitable for all kinds of digital products
Testing prototypes on users is not limited to just one part of a design, one stage of the design process, or one kind of product. It’s suitable for any aspect of a flow or design, at any stage of a product’s life cycle, and can help apps, websites, landing pages, emails, operating systems, or any other digital product.
It’s quick and easy with modern tools
There is a very low barrier for entering the realm of prototype user testing. A prototype can be drawn on paper, and a test set-up with the help of a Prototype Testing tool in just a few minutes.
Find hidden opportunities
Observing what oblivious strangers do when they have a representation of your idea in their hands is equal parts terror, fascination, and education. If the user does not comprehend the prototype in part or in whole, it means that you have found an opportunity for improvement. This could be as small as moving a button or as significant as some kind of pivot; either way leading to higher value and return.
Different Methods for Prototype Tests
You can approach prototype user testing in a way that suits the scale, nature, and stage of the product you want to test with the following considerations:
Low-tech prototype iteration
On the lower end of technology, you can simply draw on paper the screens to be tested and then test them one-on-one with your test participants. Hand drawing is very quick, so you can iterate a few times before settling on which prototypes to test. The drawing quality does not matter as long as it is legible.
Testing in person is, however, slow and implies a smaller sample group. If you want to test many rough ideas very quickly, then in-person tests with colleagues, friends, and family can work. As soon as you have a more defined direction, I urge you to step up your test to an online platform where you can reach more participants. To create the screens, you can use photos of hand drawings or use any computer program you’re comfortable with.
Again, style and drawing quality is not important, but legibility is.
When to use High-tech prototype user testing
Figma and UXtweak (or their equivalent competitors) are the basis of a high-tech user test.
While your Figma drawings can be as simple as rectangles and text, as your product develops, design style becomes more important and plays an important role in user behavior. If you’re a Figma native, starting with first-step prototypes generated in Figma is fine, but certainly do not underestimate the speed and flow-state afforded by hand drawing. We recommend hand drawing as a precursor to any layout in a computer graphics program.
Remote or in-person test participants
If you have a suitable budget of money, time, and colleagues to conduct a test, then more often than not, in-person prototype user testing will yield the most accurate, impactful results. Source participants from your existing users with an email or personal invitation. Many users are motivated to be involved by the prospect of contributing to the development of a product they use. Offering a small incentive like food or a voucher will reflect well on your brand.
On the other hand, for the majority of tests, and as the focus for this post, it is more straightforward to use an online testing participant pool. The investment of money, time, and staff is far lower, and you will gain access to a wider group of participants more easily. The main drawbacks are that you will not be able to observe first-hand how participants act, and you will not be able to ask questions spontaneously in response to their comments and behavior.
A half-way method between in-person and remote is ‘moderated’, meaning your participants are online, but you (and your team) are also online and can interact with them as they take the test. This saves on the cost of a venue and can be spread out over more time if you have less staff to help.
Kinds of prototype user tests
Fundamentally, there are two kinds of test:
- Comparative tests are set up to compare different versions of the same thing.
- Exploratory tests give test participants a realistic scenario and tasks to perform.
If you have intuitively cast a proposed test into one of these categories, consider how the test could benefit from being undertaken in the other categories as well.
Kinds of tools available for online tests
There is an abundance of online tools to use for testing user behavior on a prototype.
- Prototype testing
- User session recordings
- Heatmap generators
- 1:1 and focus group interview platforms
- Direct feedback buttons
- Crash reports
Pro tip: Remember that although prototype user testing is most effective in the earlier phases of product development, it can continue even after a product is deployed to general availability.
Prototype user testing for each project stage
Within the product creation process many product teams aspire to follow something like the double diamond strategy. Prototype user testing can be used with high frequency throughout this framework as follows:
1. Discover
As the first stage, the goal is to observe problems, not test solutions. Use your own or your competitor’s existing products as prototypes for exploratory user testing and get a broad overview of potential issues with the product
2. Define
In the second stage the goal is to narrowly define the problem. Further testing on existing products can be used to refine and pinpoint the issues which users are having.
3. Develop
The third stage is brainstorming a development of a solution to the previously defined problem. This can mean designing a new product prototype and testing the new design of prototype. Prototype user tests at this stage will show the effectiveness of the problem solution and of the new design.
4. Deliver
Visual design should be used to create high-fidelity prototypes. The tests can continue to involve page and navigation UX, but can be more elaborate and cover comprehension. Alternatives in UX and style can be assessed through A/B tests.
5. Post-deployment
This is what happens after the second diamond. Once the product is deployed and being used by actual users, tests can still be run on usability, comprehension, and minor variants in an A/B test.
How to conduct a prototype user test
1. Creating the test plan
To be frank, the planning stage can be very light-touch in prototype user testing. However, the aspects of a plan should still be accounted for, even if they’re not formalized into a written document.
Define | What is the scope of the audience, enquiry, time, and money for the test? |
Requirements | Outline the tools and people required to do the test. |
Rigour | Are there any challenges to research rigor and objectivity, and how will they be overcome? |
Questions | Set and validate the questions to be answered and goals for the outcomes. |
Outcomes | Outline what artifacts will be created and their use. |
2. Creating the Prototype
Once you have an idea of the purpose and procedure for the test, it’s time to make the prototype.
- Hand drawn screens
A great method for creating hand drawn screens is to use sketchboards, which is a way to give your drawings a small degree of structure. In this context, a sketch board is simply a piece of office paper with 2 to 6 equally sized portrait-orientation rectangles printed on it.
Each rectangle represents a screen, without being too specific about the device. Create this quickly, print out a few sheets, and sketch away. Photograph the outcomes with your phone to use as images in tests.
- Making screens in other programs
Many people are comfortable with tools like Word, PowerPoint, Google Slides, Miro, and Adobe Creative Cloud.
These are all perfectly suitable for concept exploration. Use landscape or portrait orientation depending on the target devices for your product. Create your screens and export them as images for testing.
- Making screens in Figma
Figma is the preeminent tool for product design, just ask Adobe, who paid $20B for it!
Thankfully, Figma can be used for free and is simple to learn. You’ll find a lot of Figma tutorials on YouTube and throughout the internet, so we’ll just cover the need-to-know basics here.
3. Creating a Prototype User Testing Study in UXtweak
Creating a prototype user test in UXtweak could not be easier. Here’s what you need to do:
- Go to uxtweak.com and sign up for a free account, or sign in. The free plan allows you to create the test, and you’ll only pay for responses.
- Create a new Prototype Testing Study.
- Paste in your Figma prototype link. Any flows you created in Figma will just work and you can skip the next step. Or, drop in your prototype screen images from any source, including photos of your hand drawn screens.
- Use the UXtweak editor to arrange the screens. You can open each frame to click and drag hotspot areas. In the hotspot popup which appears, you can choose which frame to link to.
- Complete setting up the Study by defining which task to start with.
You’re done with the set up, it’s that simple. To launch your test:
- Copy the link to the study and share it with your test participants. They will be prompted to install a free Chrome Extension to track the results of your test;
- or choose users from UXtweak’s user panel. This is a great way to quickly reach a large user pool to test your prototype.
To better understand how the process works, check out this quick video tutorial:
4. Analyzing the Results
The following enquiries can be applied when reviewing the results of a prototype user test.
- Did the test succeed in progressing with our objectives?
- What patterns can be seen, good and bad, and what qualitative and quantitative evidence is there for these patterns?
- What did the test confirm with some certainty, and what qualitative and quantitative evidence is that based on?
- How far back in the product development process do we need to go to address or extend what we have found? Is it just a small iteration, or does it require full ideation?
- What are the proposed next steps?
5. Create and share a report
UXtweak provides excellent report outputs which you can collate with your analysis and share with your team. Make sure your report is easy to consume for people with less time, but perhaps include detail for those who want to dive in deeper. Be prepared to revise your report based on the feedback and discussions which it sparks.
For the majority of prototype user tests, there is no need to create a thick report. A test report can be as simple as a few bullet points together with UXtweak data visualizations. The most powerful insights gained by a prototype user test may be clear in a small amount of words.
What comes after prototype user testing?
You’ve successfully planned, designed, executed and analyzed your prototype user testing. Congratulations! Now it is time for a redesign and more prototype testing. And then the next step.
Once you’ve analyzed the results of your tests you might have found some issues in the users’ experience. It is a no-brainer that the found issues will have to be fixed within the prototype framework and tested again. This process will have to be reiterated as many times as possible, until you are satisfied with the ease your users are navigating your prototype with and with the prototypes themselves.
When you have finalized the prototypes, it is time for what comes after. Let your developers bring the prototypes to life! Developers will use the prototypes, which at this stage should be of high technical precision and develop an online product based on them. This stage will typically be more lengthy, depending on the complexity of your product.
With conclusion of the development of your product you shouldn’t, however, forget about testing. Usability testing is important in many stages of product development, be it a prototype stage, a newly launched product stage, or when redesigning your product.
Conclusion: Prototype Testing is imperative for product success
If there’s one thing we’ve got from the article above is that Prototype Testing is not just useful but essential to perform if you want your product to succeed. Luckily, UXtweak is always there to help!
Register for your free account and test your prototypes with UXtweak.
