Focus groups are a wonderful methodology for website evaluation -- in the right hands.
It’s not unusual during a Blue Bear presentation to be challenged by at least one skeptic. The basis for their uncertain reaction to Blue Bear is our use of the focus group setting. Often, these are usability “engineers” who have either been disappointed themselves by previous focus group experiences or have been told by others that the focus group is a poor methodology for perfecting web sites.
We absolutely disagree. Focus groups are an ideal setting for evaluating and refining websites. But, the groups must be executed correctly. Blue Bear exploits all the advantages of focus group settings while neutralizing the potential negatives. We do this by combining patent pending technology, skillful facilitation and a detailed analytical approach.
More on that shortly. But first, a bit of relevant history will be helpful. That’s because focus groups and human factors -- the precursor to usability -- developed as separate disciplines.
Focus groups were invented during the Second World War, as a way to gain listener reaction to radio programs, war bond campaigns and military training films. These were originally called “focused interviews”. The key challenge was to engage respondents in such a way that they felt free to be open and honest about the topic in question, without straying too far from the strategic objective for the session.
Keeping respondents focused without leading them to a pre-determined result required skillful moderation. But properly facilitated, clients could probe deeper psychology -- how consumer needs are formed, communicated and satisfied. It was an important tool in addition to quantitative research, where results could be tested and confirmed statistically, but only on those closed-ended questions the researcher was smart enough to field in the first place. It wasn’t long before “focus groups” became an important fixture in consumer industries -- particularly at a time of rapid economic growth driven by a red-hot, post-war consumer economy.
Usability sprang from human factors and ergonomics -- the study of how humans interact with “things.” As a discipline, this also emerged during the Second World War, as a way to improve efficiency and effectiveness of systems and machines.
The fundamental concept of “man as an island” needing carefully constructed “bridges” -- customized interfaces with the physical world -- translated easily to computer software. Usability became a necessary core discipline at computer and software companies – as an outgrowth of human factors study. Its fundamental focus was not on gaining psychological insight, but rather on gaining empirical measurements of the ability of a human to perform tasks with external stimulus -- time studies, completion rates, satisfaction levels and other performance measures. A key feature of this work was close scrutiny of each user action and reaction to a stimulus. This typically resulted in a cycle of improvements, each measured empirically (if not statistically) for incremental progress.
The adoption of the Internet as a mainstream media created the first true convergence opportunity for these two fields which, until now, were considered an insoluble “oil and water” combination. The qualitative research movement focused on the inner drivers of peoples’ behavior. The usability research movement focused on the ability of the user to comprehend and use the stimulus presented, in this case the website.
The very outlook and value systems of the people conducting these two types of research are night-and-day different. At the risk of over-generalization, focus group researchers tend to be highly conceptual and empathic. By contrast, usability researchers tend to be linear and logical. It’s not surprising that these two groups might frown across the conference table at each other on occasion. It’s a phenomenon we have seen many times.
The fact is both approaches have merit. The qualitative researchers are right that probing the underlying motivations and needs of consumers/users can open up new vistas that can contribute in a significant way to business results.
The usability engineers are right that the fundamental experience is between man and machine, and the measurement of that experience provides irrefutable direction and clarity – key to defining next steps which also lead to stronger results.
Now, imagine the power of combining these two approaches together without diminution of either approach. The result is “startling new clarity” as one client put it. And, that’s become our mantra. The principles of what we do are simple:
• Provide such an immersive client experience that the client sees their own website (and their competitors’ websites) through the eyes of their target user.
• Provide the client direct exposure to the user as they work with the website, but also listen afterwards to group discussion about their online experience and how it matches needs and expectations.
• Gather all the data from each session as a basis for an exhaustive analysis after the groups – including screen videos, session audio tapes, and database analyses of all typed user verbatims.
• Turn the experience into a comprehensive and actionable final report which provides fact-based support for each recommendation.
Blue Bear combines the empirical discipline of usability research with the creative insight of focus groups in the following ways:
1. We avoid focus group bias by having respondents: 1) work online silently, typing verbatim comments that only they and the client can see, and, 2) do online work first, before engaging in any group discussion.
2. We exploit the advantages of focus groups by creating questions and exercises which probe user needs and expectations in unique ways. Users are not machines, and their inner drives often come from emotional needs which, to some degree, remain unmet. We explore and reveal otherwise unrevealed user thinking with expert moderation skills based upon decades of marketing experience. This is particularly potent when consumers discuss their reactions while the online experience is fresh in their minds.
3. We address the big picture and analyze the details. Success in website evaluation comes from employing both telescopic and microscopic approaches. Telescopically, overall conclusions provide a big-picture support for strategic recommendations. Microscopically, detailed, fact-based findings support specific recommendations which compel clients to take immediate action to improve results.
We detail specific fact-based recommendations from data collected for rigorous post-group analysis, including: 1) all clickstream data, screen videos and user typed verbatims, 2) audio tapes of discussion sessions which are transcribed after sessions, and, 3) customized analysis approaches which specifically address each client’s research objectives.
What’s key is that there is no single approach to Blue Bear analysis. That’s because each client situation is different. We create situations – through careful planning, execution, analysis and reporting that addresses the issues directly and definitively. We create answers, not more questions because we take a custom-tailored approach.
We are strong believers in the dual-action or “whole-brained” approach to website evaluation. Creating websites that win requires not only that users can easily use the site, but also that they want to use the site because it delivers better on their needs than competitive websites.
| about | contact | site map | press room |