Quick & Easy Usability Tests for Designers, Part 2
This is the second in a series of articles I’m writing to cover a number of quick, easy usability tests that designers can do in very little time and at low or no cost. The first article covered testing button label clarity and effectiveness. This article addresses a test for something more ephemeral—branding and ethos.
Test #2 – Branded Layout
Test category: Branding / Ethos
What we’re testing: A web site home page. However, this type of test may be used on any type of layout where the goal is to communicate a brand’s ethos.
What we want to know: Which version of a design best communicates the desired brand qualities?
Why this matters: At a high level, a brand’s ethos can be supported or degraded by design choices, so it’s important to learn as early as possible how the brand’s target audience reacts to a design direction. Closer to home for many designers, of course, is that this test can help a client see design through their audience’s eyes, potentially saving many fruitless hours of discussion.
When to do this test: Typically this type of test is performed in the early stages of a design or redesign project, in order to validate design concepts and direction before major production work begins (especially in the case of a web project).
What the test tells you: This test will tell you the qualitative effect each design has on the audience. That is, what emotional response does each design evoke? What qualities do audience members associate with each design?
What the test DOESN’T tell you: This test won’t tell you whether the layout achieves any other goals – such as whether it’s easy to use or communicates what the company does. This test is also poor at distinguishing between the effects of quite similar designs.
What you need: For this you need a list of descriptive words, and two to five versions of a layout.
– Layouts: The design approach in each version should be distinctly different from all others in the test, otherwise you won’t get truly reliable results. Many companies have a different designer create each test concept for this exact reason.
– Word list: the word list should contain up to four “target” descriptive words identified by the client as the desired brand or ethos descriptors. Mixed in with these target words add about 4X that number of additional descriptive words.
So, for example, if you have two target words, “sophisticated” and “high-quality,” add 8 more descriptive words into the list. Make sure you select words that are reasonable (no crazy outlier words, please!) and quite different from each other, or you’ll end up with less clear test results. Be sure to include some negative words, too!
A sample list containing “sophisticated” and “high-quality” might look like this when you show it to test participants:
Interesting Dark Sophisticated Intriguing Complicated Colorful Dull High-quality Expensive Solid
How to set up: Arrange the layouts so you can show them to test participants one at a time – one per printed page or per screen—to minimize any cross-influence. Make sure you’re able to vary the order in which you show the layouts to remove that potential skewing factor as well. Keep track of which participant sees which display order. For instance, you might want to keep a list similar to this:
Participant Name Display Order John Smith A-B-C Jane Doe B-C-A Jerry Smith C-B-A Mary Doe A-B-C etc…
Who to get as test participants: As for any usability test, include participants who are as similar as possible to the target audience. This can be a small hassle, but is very worthwhile because you’ll get more authentic, useful feedback.
How many test participants to include: For this type of test, it depends on how many design variations you wish to test. Because you want to mix up the presentation order to avoid skewing the results, the more designs in the test the more participants you should have.
For two designs, 5-7 participants should suffice, assuming they are very similar to the target audience. For each additional design, add 3-4 more participants. Ideally, you want to show each different presentation sequence to at least 3 participants.
How to prepare the test participants: As with all usability tests, you want your participants to feel comfortable giving their genuine feedback. Here are a few quick things to tell them before you get started:
– Thanks for participating!
– You’re helping us improve a design – we really appreciate your help.
– We’re testing the design, NOT YOU. There is no “right” answer to this test.
– We’re looking for your honest, gut-reaction feedback.
– Nobody will be offended or hurt by what you say – everyone involved wants this design to be the best it can be.
How to prepare yourself: Give each design version a reference identifier (such as a letter or number. Do not show this identifier to your test participants, this is only for your reference. Have your design versions ready to show one at a time, and prepare a place for the test participant to sit. Have a notebook and a pen ready to take notes, or you may want to have an assistant take notes for you.
Print out enough copies of the words list so you have one for each design version, for each participant. For instance, if you have 2 design versions and 5 participants, print out 2x5=10 copies of the word list. Label each page with the participant’s name and the design version’s identifier, so you know who said what, and about which version.
Be sure to have several pencils or pens for the participants to use.
Decide how many words you’ll ask the test participants to circle. Usually you’ll ask for the same number as there are client target words. That is, if the client gave you 2 target words, ask the participant to circle 2 words from the list.
How to run the test: This is probably the easiest part! Here are the steps:
– Bring your test participants in one at a time.
– Vary the order in which you show the design versions. For example, show the first participant Design A, then Design B, then C. For the 2nd participant, start on Design C, then A, then B. In other words, mix up the order each time.
– Before you show the first design to the participant, say, “I’m going to show you a couple designs, one at a time. After you’ve looked at each one, I’d like you to circle the 2 words from a list that, to you, BEST describe the design and your reaction to it.” Re-emphasize that you’re looking for their honest first impressions.
– Show the participant the design, and allow them to look at it for about 5 seconds, then turn it face-down or otherwise hide it again.
– Immediately give the participant the word list.
– Once the participant is done circling their words, take the list page from them. If you haven’t already done so, write the participant’s name on the sheet, as well the identifier for the design version they just saw.
– Repeat for each design version.
– Once the participant has responded to all of the design versions, ask them some open-ended followup questions, such as the following, and note their responses:
– “Which design did you like the best, and why?” (do not show them the designs again – the point here is to get some idea what stuck in their minds, and why it did).
– “Do you have any other thoughts on the designs you just saw?”
What to do with the results: Consolidate the test results into a comparison grid like this one:
Ideally, one of your designs will evoke the desired response from a clear majority of participants. If so, you’ve found your winning design direction! If not, the design may have been too similar to each other or you may be farther from a solution than you first thought—which simply means another quick test with updated designs is in order.
Any particular design challenge you’d like me to address in this series? Let me know by posting a comment.
From the NoD Sponsor: