Can focus groups be useful for design research? That is a question I hear a lot (be it in my own head or from actual other people).
A discussion along these lines has played out recently on the AnthroDesign mailing list. I've captured the gist of the discussion here. Bridget kicked off with:
...a client wants to conduct focus groups to get people's reactions to different web sites and web functionality. There will be 8-10 people in the room and the web site projected with the moderator driving. Has anyone had experience conducting a group like this? Are there any tips into making this as successful a session as it can be? I've typically conducted usability sessions or concept testing one-on-one.What kind of tasks and questions have built into the discussion guide?
The very smart Steve Portigal replied with (emphasis added by me):
One thing that we find helps us when given these constraints is an individual workbook. We make 'em with really big text and activities like a Likert scale etc. And at every point of evaluation, have people do the workbook FIRST independently and then discuss it. Then you have an artifact afterwards you can collect. We try to do exercises to help move people along to a meaningful place of evaluation from just "hey you are sitting in a room and here's something new and do you like it?" to something closer to a realistic personal evaluation. We might try to get people to do something like "build" what they want instead of evaluating the thing we put in front of them. Or we might do an exercise - be it theatrical or other - to help move the group into a bit more relevant context (i.e., break into two groups and one group is the IT department and the other group is managers who then have to present to each other why this is or isn't a good idea).
I will note that every time we come up with a really interesting and potentially most effective use of the format, it seems like we get shut down by our clients who have engaged us to use this format because they want something safe and familiar, methodologically, and if you spend five minutes building up a role play activity in order to get more context from their evaluation, that means there's five minutes less to cram full of questions about something else that's hard to answer in a focus group room.
Although I've not actually used this myself, I really like this technique. Preetham has a similar suggestion:
Want to echo Steve's comment about workbooks where participants give individual feedback. They are extremely helpful to negate the effects of one or two outspoken individuals.
We have had very good success sending a homework workbook that makes people immerse themselves in the context of use. This also allows them to come into the session with a point of view; makes their feedback so much more valuable and actionable. The cost to make one and send it before hand is negligible when compared to output...
There's a bit of research coming up where the USiT team might be able to employ this technique. Hopefully we can blog about it afterwards.
Care to share your own thoughts or experiences on this technique?
Update: some more useful stuff from AnthroDesign...from Christina:
...One thing I did that worked very well and was very simple was to have blind votes about each design, with each respondent quickly jotting down one MAIN reason for their vote (Repondents' comments included, "clean design," "no contact information," "Not enough info" etc.). I then collected each vote and discussed with the group why they voted how they did (the votes for respondents were not revealed to other respondents).
This helped to not only get something solid for the client in terms of "yes/no" feedback, but also helped to mitigate for the hated "alpha respondent" influencing the meek during groups. It also served as a cross-check to compare the private vote against the public discussion, and allowed everyone to have an equal voice in this area...
Nancy suggests:
This is a situation where I promote the use of design games, to get the players to interact with artifacts and with one another. People are unlikely to tell you anything unexpected in a presentation format. Plus all the critiques of focus groups as a method are likely to be demonstrated, as you've obviously anticipated.
In a similar situation (e.g. getting feedback from existing customers to several candidates for a new visual symbol set), we gathered people for "focus groups" and provided each draft symbol set to a different small team, asking them to mark up a webpage with the symbols corresponding to the meaning distinctions intended. Then the small teams presented to one another and commented on what they preferred about their own solution versus another team's solution (layout, text+symbol vs text alone, size, color, shape, etc.). We got some striking responses, including where a small team was not satisfied with the symbols we provided and they created distinctions on the spot to communicate the meanings we'd requested. Their solution matched one of the other candidate sets of symbols (though of course less polished) which provided additional support for that one. And the client stakeholders were in the room to watch and listen, as observers, during the event.
