As we approach the start of the New Year and reflect on the work completed in 2011, one study comes to mind when thinking about time and, specifically, user age.

Here at Mediabarn Research Services, we have conducted research studies with users from 2 to 70 years old.  There are very obvious differences among user groups, but one study in particular brought differences based on age to light.

Our client's target market is comprised mainly of adults aged 25-65, skewing somewhat more male than female.  While we expected to find more significant differences in the way gender influences usage of the website, what we uncovered was that age played a bigger role in user expectations and ease of use.

Users at the lower end of the age range had noticeably different expectations regarding nomenclature and content organization than older users.  It seemed that younger users had a more accurate sense of the way the site should be organized and where they would look for information.  Older users seemed to expect less of the site.  Our conclusion was that web users under the age of 40 are familiar with such a variety of sites, and technology use is such a standard practice for them, that they have higher expectations than older users.

What does this mean for researchers and web developers?  Good question!  In the case of websites or applications geared toward the general population, focusing on the user experience of those at the younger end of the spectrum will require developers to meet their expectations, which will help to advance and improve the way we all use technology.

I was talking recently with a colleague who runs the market research department of a large media company.  In addition to evangelizing research, managing her staff, and synthesizing results, she was attempting to conduct a usability study on her own as the moderator.  While I didn’t doubt her ability from a technical standpoint to conduct the interviews, I came away from the conversation with some ideas about how best to conduct the interviews.  In particular, I wondered about  her ability to effectively serve her customers.  In order to moderate 10 hour-long interviews, she had to schedule them around the remainder of her job, which meant that the length of time from the first interview to the last stretched for more than two weeks.  The primary reason was that she would be exhausted if she attempted to conduct more than three interviews in a day, and with meetings and other obligations, there were several days during which she couldn’t conduct any sessions at all.

My question is:  Wouldn’t it be better service to hire a professional moderator to complete the sessions in less than two days, and have a written report of findings and recommendations in the client’s hands in less time than it would take to conduct the sessions internally?

Price always enters this discussion, and hiring a professional moderator certainly adds a cost to the project.  My argument is that there is an opportunity cost to waiting for up to a month before uncovering study results, and there is the very real potential that a development team simply will not wait and will move forward without the benefit of testing.  Then, once results are uncovered, there is the possibility that the offering will require reengineering, again adding cost to the project.  The opportunity and reengineering costs are likely to be far greater than the cost of hiring the professional moderator.

There are lots of reasons to hire a professional moderator – speed, efficiency, unbiased views, professional training – and it's something that should be considered, especially if the customer can be better served with timely results.

We are often asked about sample size when we talk about usability testing. Skeptics find it difficult to believe that we can achieve so much by talking with so few. Here’s our stance on usability sample sizes.

Lots of work has been done to determine the appropriate number of respondents required to gather usability test results. Nielsen Norman Group has talked about this for years and stands by their suggestion that 85% of critical usability issues can be uncovered by testing as few as 5 respondents. We have found this to be true in study after study. There are a few qualifiers that should be kept in mind. The key to gathering accurate and robust findings is to ensure that you’re talking with the right users. People who fit your target profile are the only ones who can provide you with an honest evaluation of your product. Your coworkers or family members, or even an expert review, are no substitute for testing with actual users.

As researchers, in addition to having an eye toward usability, we often recommend testing with a slightly larger sample than 5. We find that many of our clients have a user base that is segmented somehow, usually by one or more demographic factors. In those cases, we typically recommend conducting tests with 8 to 12 users, and sometimes more, in order to get a good representative sample. Even then, it’s rare that we uncover new usability issues after the 5th or 6th interview.

Another note on sample size – so much of usability testing is observational and this allows users to simply do what they do without explaining their reasoning. Other types of research delve deeper into the psyche, and would require more respondents to gain a clearer picture of their thinking and to be able to project that to the overall population. The beauty of usability testing is that we can talk with a few representatives to get that clear picture.

Finally, we live by the motto “Test early and test often.” Ideally, usability testing is part of an iterative design process, whereby the product is designed, tested, redesigned, tested again, tweeked, tested again then launched. So, it doesn’t make sense to talk with large samples of users in each of these stages because it’s not cost-effective and doesn’t get you a big bang for your buck. Instead, talk with fewer users at several stages throughout the design process. And, as we’ve said before, doing some usability testing is always better than doing none at all!

Not sure how many people can say that they love their job, but I truly love what I do – helping to uncover the ways people interact with products and suggesting how to make things easier to use.

As a researcher who focuses primarily on usability, I am constantly amazed at what I learn from respondents.  At the start of each project, I spend time reviewing the stimulus (e.g., website, app, device, etc.) to become familiar with it and to develop the tasks and discussion guide I will use in the research interviews.  A natural part of that process is guessing what the stumbling blocks are going to be.  Here’s the part that I find fascinating, and it happens in nearly every single study:  the users constantly amaze me!  I love conducting the first test session in a study – seeing how an actual user goes through the process of navigating, reactions to design and layout, interpretations of labels and buttons and links.  Invariably, study respondents, who by default fit the profile of the target population, surprise me with their actions and comments.  That’s why l love what I do – because it’s fun for me to learn new things and I can pass those learnings along to my clients so that they can make improvements that make their users happier and more productive.

Take it from someone who conducts usability research for a living – there is no substitute for talking with actual users.  I will always advocate that doing any research is better than doing none at all, but nothing beats sitting down with users in the target audience and watching them use your product.  Even when you think of yourself as an “expert,” I guarantee that they will teach you something you didn’t know.

Heather Nam, our Director of Usability Research, has published an article in UPA’s User Experience Magazine. The article, called “Usability Script and Moderator’s Guide: 2 Tools or 1? Communicating with Study Respondents,” focuses on how a usability script can be used as a key communication tool that guides both moderator and respondent through the conversation that takes place during a usability test. Included is a discussion of situations in which using scripted language isn’t feasible, along with the importance of maintaining a balance between being rigid and being flexible. UPA members can access the full text of the article here.

United States Patent and Trademark Office taps Mediabarn's User Experience Group for its usability and user interface design expertise.  Read more in the press release here.

Mediabarn’s Director of Usability Research, Heather Nam, just published an article on UXmatters, an online magazine providing insights and inspiration for the user experience community. The article, called “Designing User Experiences for Children,” addresses potential pitfalls web designers can encounter when creating a user experience for children, suggests following design conventions that would be applicable for any user whether adult or child, and details specific findings from four recent web site usability studies conducted in the Mediabarn UX Lab with 85 children ages 3 to 9 years old. See Heather's article on the UXmatters site.