Mediabarn offers a full range of services to help you achieve your user testing and research goals.

Learn more about our services »

Mediabarn recruits fresh, engaged respondents for local, national, and international studies.

Learn more about our recruiting »

Minutes from DC, Mediabarn's facility is the ideal location for your next focus group or market research study.

Learn more about our facility »
Monday
Dec012014

Market Research: Code of Ethics

The spectrum of research topics we see in this industry is broad, covering all of the intricate niches of consumer products that fall in between the bookends. Clients work hard to determine what type of consumer best matches their brand profile, typically based on statistical data collected over time and evaluated prior to conducting research. So, when it comes time to test the market, it only makes sense that the strongest feedback will come from those participants with specifications that match the clients consumer base. Research firms are then given a set of criterion from which to draw against through the recruiting phase.

ethical-guide-to-market-research_698x400

We’ve seen it all, from the occasional request for participants without any specific limitations, to requests so specific that it seems nearly impossible to find even one person who might match what the clients need. Regardless of the level of difficulty, ethical values should always be followed or the integrity of the recruit could be severely compromised, resulting in thousands of dollars lost for all parties involved in the process.

In the world of market research, there are many ethical standards in place for the various phases each project surpasses. When it comes to participant recruiting, there are several main points that should be considered and closely followed, regardless of the level of difficulty a project may present.

1. Respecting Respondents: It’s important to remember that the individuals being considered for research participation have already given their valuable time and attention through the screening process. This could take anywhere from two to twenty minutes, and is time for which they are not being compensated. They are often times providing personal information, and confiding in the research firm representing the project. Research interviewers are the face of the firm, but also of the client for whom the work is being conducted. Respondents should be treated with respect and in a professional manner at all phases of every research project. Without them, there wouldn’t be market research!

2. Confidentiality: It isn’t one sided. Each client and respondent alike have a code of confidentiality that must be maintained indefinitely. Great care should always be taken to ensure that sensitive information which could potentially identify participants to third-party sources is kept secure and confidential. The same ethical boundaries should be extended to clients, as well, in order to protect the confidentiality of their studies and prevent bias feedback from respondents. In the event a client requests sensitive information on a participant (such as last name or contact information), permission should be sought from the respondents prior to the release, and if necessary, permissive documentation to keep on file.

3. Persuasive or suggestive recruiting methods: At no time during the screening interview should a recruiter ever attempt to influence or persuade a respondents answer or opinion by emphasizing words or responses, ad-libbing, changing the frame, order or response of questions. It’s extremely important to the integrity of each project that data is sourced and collected impartially so that the concluded research results are a true representation of reality.

4. Honesty: Applicable to recruiting, as well as during the analysis and presentation of findings, participant responses and feedback should be recorded and represented accurately and honestly. Adjusting responses during the screening phase in order to meet a quota or enable qualification is completely intolerable, and poorly positions the integrity of the project.

Additional information about market research ethics can be found on the MRA website at www.marketingresearch.org/code

Friday
Nov072014

What is Wrong with Sandwiches?!!

The above title is what my 10 year old son yelled out of frustration as he watched his hamburger smothered in ketchup fall out of his hands, quickly landing on EVERYTHING. His blaming the burger, but not the hands that held it, made me chuckle. Obviously if the burger was designed differently this would not have happened, right? Or would he have dropped any kind of sandwich at that exact moment because his hands were not holding it correctly? A usability or user issue?

I describe this incident as a very loose parallel to UX issues that we often see during usability testing. How do we know if a usability issue is truly the fault of the interface? The debate has gone on for years, where many would argue that there is no such thing as “user error” –  that it’s always an issue with the product; while some may claim that no interface/design will have a 100% success rate for users.

  • Your participant didn’t see the search bar. Was it because it wasn’t where they expected it to be or was it because it truly was not in a clear or obvious place?
  • A participant doesn’t go to the “correct” link during a task. Did they not understand where to go because the task was not phrased in a way they would have expected, or is the interface really preventing completion?
  • The user didn’t notice the ad you wanted them to see. Was it because it was not prominent enough or did the messaging not resonate or make sense to them?

There are many other examples; those are just a few of the more common ones.

Regardless of what the task and results are – it is important to note during testing that people tend to blame themselves even when it’s not their fault, so be mindful and patient.

Generally, things will resolve to whether or not the interface works within standard expectations. It’s our role as experts to know what those standards are and how to make sure they constantly evolve while technology expands, without reinventing what already works.

With any UI design, if you deviate too far outside of what users generally expect, you run the risk of them missing features or not knowing how it would work. However, if we design for every person’s expectations could there be one perfect interface? Or, more importantly…one perfect spill-proof sandwich?

Friday
Oct242014

7 Tips for Successful Screeners

When it comes to market research, the group of participants taking part are a crucial component to the success of the project. Comprising a faction of high quality, qualified respondents is a process that comes with its own set of challenges, but creating a great group starts with a well built screening questionnaire. Not only does a well written screener aide in the success of the project, but it can also save a lot of time and money on the client end!

So, what makes an effective screening questionnaire? There are several points that should be closely reviewed before stamping a draft with the seal of final approval for fielding.

1. Before you begin wiring your questionnaire, be sure you have a clear outline of what you’re aiming to achieve from this research study. Ensure you have a solid understanding of the intent and purpose, what types of participants would provide the most useful data, and how you will utilize the collected data after the study concludes. All of these factors weigh in on the pertinency of the questions you choose to include for screening.

2. Limit the questions to those that really have the most impact on the scope of the study. By putting in too many questions, you run the risk of turning off a potentially great respondent because the time commitment up front is too long and monotonous, and may also relay a message of a disorganized study. You also run the risk of redundant questions that may inadvertently disqualify participants who are the right fit for the research. Contradictory questions create a lot of confusion for the interviewers and respondents, alike.

3. Be mindful of skip patterns and recruiters notes. While most recruiting firms will pick up on errors, it’s not always clear what direction your skip pattern should take an interviewer as opposed to the direction it is taking them. Also, leaving off notes that give instructions for disqualification can create confusion if there aren’t clear cut guides to quotas. If these errors aren’t caught early, the entire recruiting process may be compromised.

4. When you can, limit excessive quotas. There are certainly instances where an abundance of parameters are absolutely necessary in order to collect the best possible data. But most times, this isn’t the case. Excessive specifications create more obstacles for the recruiting firm to find the multiple needles in the haystack, and also put your project at rick of losing valuable respondents who may be screened out over an insignificant detail. Some quotas are always expected, but be careful about adding quotas to the quotas (and so forth).

5. Keep the wording of the questions limited and easy to follow. Remember, most recruiting firms interview potential candidates over the phone by reading from the script you provide. If the questions are too wordy or complicated, a respondent may lose focus, or misunderstand the question, which may inadvertently lead to a false or inaccurate response. This could compromise the quality of the recruit, but can be avoided by keeping questions simple, short and direct.

6. Review the order of your questions, and ensure it has the correct flow. When possible, include all qualifying questions at the front end of the screener. This will save time on the recruiting end, and also reduce the number of candidates who spend excessive time answering questions only to be turned away at the end. Demographic and other sensitive questions are typically best suited for the back end of the screener, and folks who don’t have to answer these right out of the gate are more inclined to answer. Make sure the questions are aligned in an order that makes logical sense and doesn’t disrupt the flow.

7. Finally, before you send your screening questionnaire out into the field, ask someone to read it out loud to you. Role play, as if you are the candidate and the other person is the interviewer. This will give you first opportunity to hear how the screener reads, and pick up on any potential issues. Also ask that person to look over any skip patterns you may have included to be sure they make sense and don’t elicit confusion. Give your document one last look over to make sure nothing obvious needs correcting before sending it along.

Once the recruiting firm has the screener in hand, ask them to review it and provide any feedback they may have. If there are issues, they will be sure to let you know! It is also important to keep in mind that when the recruit has been fielding and not producing after extensive efforts, it is usually an indication that there is a significant issue with the screener: either the questionnaire or the specifications. If you make the process near impossible, it will be near impossible to fill with quality respondents. Remember, the success of the project starts with a sound screener!

Monday
Jul072014

Work Hard, Play Hard

 

From understanding UX, to learning how to manage working in a fast-paced environment, I’ve gained tons of knowledge since beginning my journey at the ‘Barn (you can read how intimidated I was when I first started here). One thing I’m glad I didn’t have to deal with is getting caught up in the typical “daily grind” that a majority fall victim to here in the DC area.

Many people get lost in their 9-5 business infused life, and in turn are left with stress and frustration. In a recent study performed by APA, about 50% of DC-area residents report having been irritable or angry at least once in the past month due to stress, and 61% of workers reported that heavy workloads have a significant impact on work stress levels. Fortunately, Mediabarn’s mission statement doesn’t include the words “stress” or “frustration”. That’s not to say we don’t get stressed from time to time, but we try and do things to help offset it while here at work.

We all enjoy spending time at the ‘Barn, and it’s a given because we are passionate about what we do. And when you combine that with a little bit of fun, we avoid being part of such a large negative stress statistic. “There is good evidence that if you allow employees to engage in something they want to do, (which) is playful, there are better outcomes in terms of productivity and motivation” says Dr. Stuart Brown, founder of the National Institute for Play. Plus, according to Tim Brown, CEO of IDEO (a global design company), people are willing to take more creative risks in an office where they have the security and comfort to play and not be judged. We’ve found that whether it’s taking a mid-day break to check out the new ice cream shop around the block, or spending the day kayaking on the Potomac River, Mediabarn’s spirited environment keeps our Monday mornings sunny year-round.

If you’re looking to brighten things up around your office, here are some of the ‘Barn’s favorite team building activities that you may want to consider to keep employees happy, healthy and productive. Take it from this ‘Barn-ette, it makes a difference!

 

  • Golf Outing
  • Bowling
  • Museums
  • Baseball games
  • Amusement parks
  • Food cookoffs
  • Happy hours
  • Concerts
  • Monuments
Monday
Jun232014

Recording Mobile Usability Testing

As more of our clients develop sites and apps for mobile devices, we have seen an increased need to conduct usability testing on tablets and smartphones. Finding the right recording solution has been challenging, especially when compared to recording desktop usability sessions.

For desktop usability, there are many different applications that allow us to capture screen interactions along with a picture-in-picture view of a participant’s face. When trying to apply this to mobile device testing, it’s particularly difficult to capture an accurate and clear video recording of the session. Although clients typically want to see the participant’s face, often it’s more important to see what the user’s hands and fingers are tapping and swiping on the mobile device.

An added complication is that the mobile devices are… mobile; in other words, participants move them during the session, making video capture an even bigger challenge. Here at Mediabarn, we made the decision at the onset to develop a mobile testing solution that wouldn’t require any attachments (e.g., cameras, sleds, or wires) to the device. We felt that wrapping a device with mounts and wires would create an unusual user experience.

In order to mimic as lifelike an environment in the usability lab as possible, we offer a variety of flexible solutions and equipment, including wireless HD connections, wiring and power in our ceilings, and a large selection of cameras, mounts and arms. Our recording system allows us to try various approaches and enables us to cater to diverse groups of participants, dependent on the objectives of each unique study. For example, we discovered that testing mobile devices with kids requires a different setup than one we might use with adults. Specifically, kids have a tendency to pull devices into their laps, while adults seem content to use devices on the tabletop. During longer studies, however, we have found that adults tend to pull the device closer to their chest and lap as their arms become fatigued. The bottom line is that our technical approach considers the participants, the type of device, the type of site or app being tested, and the length of study.

We essentially have two main setups:

(1) Camera. We have a number of different camera setups. Generally, we use an “over the shoulder” view or our telescoping arm document camera (our most recent equipment addition). This setup allows for a clear view of the screen as well as what the respondent is doing on the screen. We find that most of our clients are more interested in seeing how participants manipulate what they see on the screen, so this is what we record. Additionally, there are two cameras (one on each side of the participant) showing body language or facial expressions. When we’re testing kids, or for longer studies, we use ceiling cameras, placed to the right or left of the respondent so that their head will not block our view if they lean over the device.

(2) Apple mirroring and Airplay. When it’s more important for our clients to have a crystal clear view of what is on the device screen than to see participants’ hand or finger movements, we use this type of screen capture recording. As with the other technique, the dual cameras capture facial expressions and body movements.

While we would love to have a single solution for recording mobile device usability tests, we have found that maintaining a flexible approach is the best way to fulfill the needs of our clients and their individual projects.

Our Clients
our clients Learn More »
.