My most recent project has taken an interesting turn into keyword search. We're using SOLR for our search engine and the philosophical question on the table is how to fine-tune it or do we fine-tune it?
That is, we are in a tug-of-war over whether the most prominent results should be the products management wants to feature and also, if the results are inherently flawed because some other things are showing up. Specifically, some stakeholders are insisting that we find a way to re-interpret what the user is searching for or override the results the search engine brings back.
Well that sounds rather Orwellian, and maybe not a good thing. At least not something I care to endorse. But let me explain the dilemma a little more.
First, this is an eCommerce site, not social networking or information sharing. It's all about connecting shoppers to products, and consequently, reinforcing marketing goals for the business.
Second, we are grappelling with two problems: iconic brands/products and context-sensitive search. The question at hand is how to get balance between iconic brands and products and the apparent randomness of context-sensitive search.
To illustrate the problem, let's think about grocery shopping. And let's imagine your search word is "Kellogg's". What are you searching for? Breakfast cereal? Probably. And what images are in your mind that signal "Kellogg's" and breakfast cereal? Maybe a box of cornflakes? Not that you want to buy cornflakes or even like them but it's the product that built the company and it's an image that strongly signals your search has landed you in the right place.
That's an iconic brand and an iconic product. And the problem is that people have such strong expectations about iconic brands and products, they may believe the search is broken if these brands or products don't show up according to their expectations.
Think about Kellogg's Cornflakes again. Where did you imagine this product would appear in your imaginary Kellogg's search? Near the top? In the middle? On Peapod, the dominant Chicago-area online grocery, a search on "Kellogg's" sorted by "Best Match", puts Kellogg's Cornflakes in 100th place, trailing a long mix of other cereals, breakfast bars, frozen waffles, and fruit roll-ups. Surprised?
Maybe that is an accurate reflection of the market or maybe the search algorithm is flawed; after all, we don't know Peapod's criteria for "Best Match", or their preferences for sorting results, but it's not hard to imagine a brand manager somewhere being rather upset and insisting something must be wrong if Cornflakes is not among the top 10 or 12 things you see.
We're not in the grocery business but our client makes and distributes several iconic products and brands and carries others in its inventory. These iconic products and brands are not always obvious in our search results, either. A lot of parts, supplies, and accessories show up first when you search solely for the brand name. And yes, this seems a little odd, or at least, hard to explain. To switch analogies, it's like showing you the windshield wipers and floor mats before we show you the car.
Our search engineer and I are convinced it's just a matter of examining the relevancy logic and re-weighting some variables or adding some variables so that the ancillary products lose relevancy. But how finely do you tune the search when you don't have any real data to work with?
To complicate matters, this is a B2B site that is not in production yet and we have no budget for user research; we don't really know what kind of search habits and expectations our customers will have. We know that searching on product numbers (full or part) works fine and that is something our users have been doing for a long time on the legacy site. But soon, they'll have the power of text search over the full product specifications and descriptions and we have no clue how they'll respond to that.
Meanwhile, our stakeholders and product owners are experimenting with keyword search based on their own preferences and guesses, which means a lot of searching for the iconics, and they are seeing too many parts and accessories in the results.
Some of these managers are certain the best bang for their money is to stop spending it on search engine support and start spending it on workarounds; like trapping for brand and product names and running hard-coded queries or highly limited searches instead of letting the search engine do its thing.
The more I struggle with these discussions, the more it hits me that we are getting sucked down a dead-end. If these brands and products are so central to the business and the site's identity, why should we be spending time and energy on the idea that our customers would be typing these brand names into a search box?
If these brands are so central to the business, why should any customer have to search for them? They should be front and center on the home page and every page. There should be "Famous Brands" and "Featured Products" links and lists and icons everywhere you look. All of these hot-button items should be one click away ... not something that customers will have to manually spell out and then hit "Go".
If we haven't put those features and shortcuts in place, (but we mostly have, as well as effective facet filtering *), then all of our time and energy should be on fixing that deficiency. Not fiddling with the search engine.
Let the search engine do its thing and return lots of branded parts and compatible parts for brands. Parts are hard to find; parts require searching. Parts are where you make your margin. ("Give away the razor; make a bundle on the blades.") Let's not take a chance on breaking that before the real users have had a chance to make their habits known.
* You can generally filter a result set in the 1,000's down to less than 50 items, usually under 25, in about two or three screens.
Showing posts with label UX. Show all posts
Showing posts with label UX. Show all posts
Sunday, March 13, 2011
Tuesday, June 29, 2010
Design Research Conference 2010: Speed Talks, Don't Forget This!, and the Panel that Wasn't
Speed Talks
Five-minute speed talks were one feature of the conference. The concept was good -- an open invitation to submit a presentation with a panel of judges to select the best. The results were spotty; all of the presentations had good production values but about half of the topics were trivial (to me). Here are my notes on three of them:
"Adventure Research" - ask a not-entirely together homeless person to give you a tour of San Francisco. "What's All This Fluffy Stuff?" - there was a story wanting to be told but she didn't tell it. "Steampunks" - this is so 1980's; these are some nice ideas for furnishings but why do we have to be subjected to one person's fascination with another couple's fantasy life?
Speed talks that did pay off for me were Lawrence Swaider's presentation of a project to rebrand birth control -- how to package the message for teenage women today; Lorissa MacAllister on using cross-functional teams in healthcare design; and Arturo Pelayo's conjoined twin presentations: Design is Diplomacy.
Arturo was very rude in bundling two presentations into one, chewing up so much time that a later speed talk got bumped from the schedule. But I really liked his first segment where he challenged designers to answer for the social and environmental impacts of their work. The second part was a feel-good piece about empowering children to engage with their community.
The last useful speed talk was on Brains, Behavior and Design. It discussed patterns of irrationality that have implications for design -- for example, losses have more impact than gains; the present counts for more than the future. The project team designed a paper-based toolkit to help designers reframe a solution to better influence people to the desired decision. See BrainsBehaviorAndDesign.com for more information.
Don't Forget This!
We learned that Patricia Moore pioneered the use of disguise to directly experience the target population's reality (see video clip). And we were reminded about the book, "Black Like Me", another pioneering work in this vein.
We got the phrase "New York Verb" and some negotiation advice from Dominic Misino - find out as much as you can about the person; call them by the name they want you to use; find out what's important to them by asking them what they want; know your own bottom line and walk-away point; make them specifically promise, as in, "I promise I will do X, Y, Z".
Ikea Effect - if you do it yourself, you love it more. Heather Reavy called this out as a strategy for getting business buy-in for a design solution. Only rough things out enough that people can imagine possibilities. Big bulletin boards of sticky notes help to surface insights. Frame ideas in words. Balance a description of the feature / solution with evocations of values and aspiration. Leaving space to imagine possibilities gains advocates.
You can see videos of all (almost all?) of the conference presentations on Vimeo, click here for the list.
Earlier I posted some links about design research in China. This is the definitive article, to date: Consumerism in the Wild, Wild East.
The Panel that Wasn't
The ID conferences that I've attended in the past few years have usually concluded with panel discussions. This seems like a good idea and no-brainer to pull-off but usually there's something a little flat and artificial about them. There are generally some decent summary statements or reiterations of key positions but genuine conversations with sparks, ah-has! and gotchas! never really develop. A skillful moderator can bring some tricks to deflate talking heads and lessen the participants' sense of being the equivalent of performing seals - but what do you do when the selected moderator is a talking head himself?
My favorite tweet about this year's panel went something like this, "And now the panel finally gets to talk and ohhh, we're out of time!"
Five-minute speed talks were one feature of the conference. The concept was good -- an open invitation to submit a presentation with a panel of judges to select the best. The results were spotty; all of the presentations had good production values but about half of the topics were trivial (to me). Here are my notes on three of them:
"Adventure Research" - ask a not-entirely together homeless person to give you a tour of San Francisco. "What's All This Fluffy Stuff?" - there was a story wanting to be told but she didn't tell it. "Steampunks" - this is so 1980's; these are some nice ideas for furnishings but why do we have to be subjected to one person's fascination with another couple's fantasy life?
Speed talks that did pay off for me were Lawrence Swaider's presentation of a project to rebrand birth control -- how to package the message for teenage women today; Lorissa MacAllister on using cross-functional teams in healthcare design; and Arturo Pelayo's conjoined twin presentations: Design is Diplomacy.
Arturo was very rude in bundling two presentations into one, chewing up so much time that a later speed talk got bumped from the schedule. But I really liked his first segment where he challenged designers to answer for the social and environmental impacts of their work. The second part was a feel-good piece about empowering children to engage with their community.
The last useful speed talk was on Brains, Behavior and Design. It discussed patterns of irrationality that have implications for design -- for example, losses have more impact than gains; the present counts for more than the future. The project team designed a paper-based toolkit to help designers reframe a solution to better influence people to the desired decision. See BrainsBehaviorAndDesign.com for more information.
Don't Forget This!
We learned that Patricia Moore pioneered the use of disguise to directly experience the target population's reality (see video clip). And we were reminded about the book, "Black Like Me", another pioneering work in this vein.
We got the phrase "New York Verb" and some negotiation advice from Dominic Misino - find out as much as you can about the person; call them by the name they want you to use; find out what's important to them by asking them what they want; know your own bottom line and walk-away point; make them specifically promise, as in, "I promise I will do X, Y, Z".
Ikea Effect - if you do it yourself, you love it more. Heather Reavy called this out as a strategy for getting business buy-in for a design solution. Only rough things out enough that people can imagine possibilities. Big bulletin boards of sticky notes help to surface insights. Frame ideas in words. Balance a description of the feature / solution with evocations of values and aspiration. Leaving space to imagine possibilities gains advocates.
You can see videos of all (almost all?) of the conference presentations on Vimeo, click here for the list.
Earlier I posted some links about design research in China. This is the definitive article, to date: Consumerism in the Wild, Wild East.
The Panel that Wasn't
The ID conferences that I've attended in the past few years have usually concluded with panel discussions. This seems like a good idea and no-brainer to pull-off but usually there's something a little flat and artificial about them. There are generally some decent summary statements or reiterations of key positions but genuine conversations with sparks, ah-has! and gotchas! never really develop. A skillful moderator can bring some tricks to deflate talking heads and lessen the participants' sense of being the equivalent of performing seals - but what do you do when the selected moderator is a talking head himself?
My favorite tweet about this year's panel went something like this, "And now the panel finally gets to talk and ohhh, we're out of time!"
Wednesday, June 2, 2010
Design Research Conference 2010: Day 2, part 2
The afternoon of the second day seemed to carry the most value for the attendees, based on the coffee break buzz. Three speakers focused on design research activities outside of the US - primarily projects in China, India and the African continent. Two of the three speakers talked about getting past cultural barriers and stereotypes. The third speaker gave a concrete framework for evaluating whether a design concept had a viable chance of making a positive impact on third-world conditions.
Cathy Huang: She caught our attention by asking the audience to mimic a monkey in the US, Japan, and China. Each culture has a uniquely different gesture to signal a monkey -- two hands scratching armpits, one hand scratching head, one hand shading eyes looking into distance. So if we can't communicate "monkey" effectively, what else might we get wrong if we try to design and market products and services for consumers in China?
Cathy tackled some typical symbols in Chinese culture: red = happy; shiny = rich or precious; big = royalty or power. A superficial understanding of how these symbols apply in Chinese society could suggest that big red cars will be popular. They're not. And less is not more. In China, a product has to physically express its functionality. A simple exterior suggests a useless item.
Instead, other more subtle values, derived from Chinese philosophy (Doctrine of the Golden Mean), drive Chinese preferences -- social conformity, avoiding attention or suspicion, concealing real wealth, absorbing influences, and utilitarianism.
These conservative values present a variety of problems for design research in China. For example, research results do not equal the real choice. In interviews, users praise one product but choose another for their take-away gift. Likewise popularity does not equal sales. Ikea is a popular destination but more for the experience of being in the store and for getting furniture specs than for shopping. After visiting Ikea, shoppers will have furniture made to order by higher-end craftspeople because the culture values furniture built to last.
Copying (absorbing influences) is a big driver. The Chinese economy and business structures reward results-driven behavior, making it more preferable for businesses to copy existing products rather than innovate. But simply copying or selling a successful model from Country A in China is not guaranteed unless the designer understands where Chinese values and opinion will differ from the consumers in Country A. But Chinese consumers may prefer a product or its copy for completely different reasons from other markets. Designers need to go beyond the intended usefulness of a product and understand the Chinese view of its value. For example, reduced water use was a key feature of a washing machine marketed in Japan and was the reason for its success there. But the exact same machine did not sell as well in China until other aspects about its washing capacity and versatility were highlighted. Functionality was more valuable than water conservation.
Anjali Kelkar: In her opinion, designers researching in foreign countries do not take enough time to empathize. "Why are we in such a tearing hurry? Is our need for rigor too rigid?" In a complex country like India a cultural quirk can become an "insight" when really it is just that, a quirk that does not mean much about the real problem or users under research.
To arrive at an effective level of empathy, her firm devotes a lot of time to pre-research in order to build relationships and get over assumptions. Before arriving onsite, Anjali holds "assumption breaker" sessions with her teams to get everything out in the open. It's a giant brain dump of sticky notes (everything you think or have heard or expect to experience about X) that the teams can re-visit and confirm, dismiss or elaborate on after their initial visits and during research.
She conducts pre-research visits to become familiar with people and their surroundings; including the domestic help. Then she asks her subjects to prepare self-documentation through photos or video. Meanwhile, the research team goes through remote and on-site immersion workshops to experience the culture and setting; revisiting their assumptions and refining their questions. They use the self-doc photos to conduct the research interviews. Interviews are just as likely to be on-the-fly, rather than scheduled. The researchers adapt to the rhythms and interruptions of daily life rather than force a pre-planned schedule. They include any domestic help in their research, if possible, since these are often the real users of home appliances instead of the person who made the purchase decision.
Flexibility is necessary. This means being able to redirect the entire hypothesis and purpose of the research depending on what comes to light during the pre-research visits. For example, one project began with a hypothesis that the economies of small villages could be improved by increasing the availability of repair shops and mechanics. Immediately, the pre-research visit to the area brought the response that what people there really needed was an improved delivery system for drinking water.
Some similar links: Googling Cathy Huang + China Bridge or googling Anjali Kelkar, produce lots of links to interviews and articles with, about, or by these women. But nothing comes up right now that is specifically along the lines of these two talks. So I've added two links here that amplify their themes; a paper from Kaizor on cultural differences in China and one on how ZIBA used cultural immersion in their Chinese design work for Lenovo.
Kevin Starr: I really liked this talk for the specific criteria Kevin shared with us. These criteria come from his work with the Rainer Arnhold Fellows. If you want to design a thing or a service to improve life for a portion of the one billion people living in poverty today, how do you know your idea has a chance of being effective?
One: Know your real mission. Can you state it as a verb + a target + an outcome in nine words or less?
Two: Measure the right thing. Get real numbers, think ahead and get a baseline, determine reasonable re-measurement intervals and sample the right size. Play the "best indicator" game. What one thing can you measure that would give you the best indication you are making a difference? For example, when subsistence farmers have excess produce to sell, they use the proceeds to eat more protein, educate their children, improve their housing, and have more festivals. These are possible measurements. Now consider microfinance. Repayment rates are measurable and certainly show if the investors succeeded but does that measurement say anything about the borrowers standard of living? In one case study, it was found that 25% were better off, 50% the same, and 25% worse off.
Three: Connect the behavior dots. If there was a change can you show that it was due to your project?
Look at the steps in the process that have to be right in order to get the desired result:
1. Identify a need the thing will satisfy
2. Verify there is a demand to satisfy that need
3. Follow a design process (see next paragraph)
4. Realize the thing (viable prototype)
5. Obtain manufacturing capacity
6. Identify / create a distribution channel
7. Establish a market (place and means to procure the thing)
8. Verify the usage behavior is aligned with the thing (will the use feel natural and easy/enjoyable compared to alternatives?)
Having a formal design process to follow increases the likelihood you will have taken the appropriate steps to cover the other points above and will have a plan for contingencies. A formal design process ensures that you have done / will do the necessary research into such things as the physical setting, the users and their customs, previous history of other efforts, ideas about markets and distribution, emerging technology, your competition, and have come up with adequate specifications.
He was scathing on some ventures that failed - the playground merry-go-round as water pump is too hard for kids to push and no fun. The use reverts back to women or animal power. The Life Straw. Does sucking water straight out of a river or pond through a filtered straw really solve everyday water needs, plus you have to replace them eventually. One laptop for every child. Duh...who needs this when you haven't got a school, or health care or food (and where's the money to keep it running?) Even the successful manual treadle pump was slow to start until they got the marketing right.
Next up: Speed talks, odds and ends, and the panel that wasn't.
Cathy Huang: She caught our attention by asking the audience to mimic a monkey in the US, Japan, and China. Each culture has a uniquely different gesture to signal a monkey -- two hands scratching armpits, one hand scratching head, one hand shading eyes looking into distance. So if we can't communicate "monkey" effectively, what else might we get wrong if we try to design and market products and services for consumers in China?
Cathy tackled some typical symbols in Chinese culture: red = happy; shiny = rich or precious; big = royalty or power. A superficial understanding of how these symbols apply in Chinese society could suggest that big red cars will be popular. They're not. And less is not more. In China, a product has to physically express its functionality. A simple exterior suggests a useless item.
Instead, other more subtle values, derived from Chinese philosophy (Doctrine of the Golden Mean), drive Chinese preferences -- social conformity, avoiding attention or suspicion, concealing real wealth, absorbing influences, and utilitarianism.
These conservative values present a variety of problems for design research in China. For example, research results do not equal the real choice. In interviews, users praise one product but choose another for their take-away gift. Likewise popularity does not equal sales. Ikea is a popular destination but more for the experience of being in the store and for getting furniture specs than for shopping. After visiting Ikea, shoppers will have furniture made to order by higher-end craftspeople because the culture values furniture built to last.
Copying (absorbing influences) is a big driver. The Chinese economy and business structures reward results-driven behavior, making it more preferable for businesses to copy existing products rather than innovate. But simply copying or selling a successful model from Country A in China is not guaranteed unless the designer understands where Chinese values and opinion will differ from the consumers in Country A. But Chinese consumers may prefer a product or its copy for completely different reasons from other markets. Designers need to go beyond the intended usefulness of a product and understand the Chinese view of its value. For example, reduced water use was a key feature of a washing machine marketed in Japan and was the reason for its success there. But the exact same machine did not sell as well in China until other aspects about its washing capacity and versatility were highlighted. Functionality was more valuable than water conservation.
Anjali Kelkar: In her opinion, designers researching in foreign countries do not take enough time to empathize. "Why are we in such a tearing hurry? Is our need for rigor too rigid?" In a complex country like India a cultural quirk can become an "insight" when really it is just that, a quirk that does not mean much about the real problem or users under research.
To arrive at an effective level of empathy, her firm devotes a lot of time to pre-research in order to build relationships and get over assumptions. Before arriving onsite, Anjali holds "assumption breaker" sessions with her teams to get everything out in the open. It's a giant brain dump of sticky notes (everything you think or have heard or expect to experience about X) that the teams can re-visit and confirm, dismiss or elaborate on after their initial visits and during research.
She conducts pre-research visits to become familiar with people and their surroundings; including the domestic help. Then she asks her subjects to prepare self-documentation through photos or video. Meanwhile, the research team goes through remote and on-site immersion workshops to experience the culture and setting; revisiting their assumptions and refining their questions. They use the self-doc photos to conduct the research interviews. Interviews are just as likely to be on-the-fly, rather than scheduled. The researchers adapt to the rhythms and interruptions of daily life rather than force a pre-planned schedule. They include any domestic help in their research, if possible, since these are often the real users of home appliances instead of the person who made the purchase decision.
Flexibility is necessary. This means being able to redirect the entire hypothesis and purpose of the research depending on what comes to light during the pre-research visits. For example, one project began with a hypothesis that the economies of small villages could be improved by increasing the availability of repair shops and mechanics. Immediately, the pre-research visit to the area brought the response that what people there really needed was an improved delivery system for drinking water.
Some similar links: Googling Cathy Huang + China Bridge or googling Anjali Kelkar, produce lots of links to interviews and articles with, about, or by these women. But nothing comes up right now that is specifically along the lines of these two talks. So I've added two links here that amplify their themes; a paper from Kaizor on cultural differences in China and one on how ZIBA used cultural immersion in their Chinese design work for Lenovo.
Kevin Starr: I really liked this talk for the specific criteria Kevin shared with us. These criteria come from his work with the Rainer Arnhold Fellows. If you want to design a thing or a service to improve life for a portion of the one billion people living in poverty today, how do you know your idea has a chance of being effective?
One: Know your real mission. Can you state it as a verb + a target + an outcome in nine words or less?
Two: Measure the right thing. Get real numbers, think ahead and get a baseline, determine reasonable re-measurement intervals and sample the right size. Play the "best indicator" game. What one thing can you measure that would give you the best indication you are making a difference? For example, when subsistence farmers have excess produce to sell, they use the proceeds to eat more protein, educate their children, improve their housing, and have more festivals. These are possible measurements. Now consider microfinance. Repayment rates are measurable and certainly show if the investors succeeded but does that measurement say anything about the borrowers standard of living? In one case study, it was found that 25% were better off, 50% the same, and 25% worse off.
Three: Connect the behavior dots. If there was a change can you show that it was due to your project?
Look at the steps in the process that have to be right in order to get the desired result:
1. Identify a need the thing will satisfy
2. Verify there is a demand to satisfy that need
3. Follow a design process (see next paragraph)
4. Realize the thing (viable prototype)
5. Obtain manufacturing capacity
6. Identify / create a distribution channel
7. Establish a market (place and means to procure the thing)
8. Verify the usage behavior is aligned with the thing (will the use feel natural and easy/enjoyable compared to alternatives?)
Having a formal design process to follow increases the likelihood you will have taken the appropriate steps to cover the other points above and will have a plan for contingencies. A formal design process ensures that you have done / will do the necessary research into such things as the physical setting, the users and their customs, previous history of other efforts, ideas about markets and distribution, emerging technology, your competition, and have come up with adequate specifications.
He was scathing on some ventures that failed - the playground merry-go-round as water pump is too hard for kids to push and no fun. The use reverts back to women or animal power. The Life Straw. Does sucking water straight out of a river or pond through a filtered straw really solve everyday water needs, plus you have to replace them eventually. One laptop for every child. Duh...who needs this when you haven't got a school, or health care or food (and where's the money to keep it running?) Even the successful manual treadle pump was slow to start until they got the marketing right.
Next up: Speed talks, odds and ends, and the panel that wasn't.
Friday, May 21, 2010
Design Research Conferce 2010: Day 2
The second day of the conference had a much better sense of theme and structure than the first. The morning focus was on the use of technology in design research and the afternoon talks covered design research activities in Asia and developing countries. This post will cover the technology topics of the day.
Kim Erwin: Discussed how online tools were allowing design research to change. Traditional methods either put the user and researcher in the same time and place - interview/observation - or they separated the researcher from the user by time and place - surveys/logs collected from the user and analyzed via automation or in the researcher's office. Online tools now provide platforms where the user can do self-reporting at different times but the researcher comes to the same place where the user reports -- blogs, forums, narrative building sites. Online tools and platforms support expanded access, mobility, intimacy and collaboration. Kim presented three tools that her student team has been working with: Revelation, QualVu and Civicom.
Revelation allows the researcher to set up projects containing a variety of assignments that the user performs online at different times such as questionnaires, diaries, discussion forums, etc. The researcher can review user inputs and tag them for querying and comparison. QualVu is a tool that lets users create video diaries in their home or workspace. The researcher can access these videos and create accompanying text commentary that is stored with the video by QualVu. The researcher can also pull video clips together to create online reports and presentations. Civicom is a voice and text platform that lets researchers and users phone or text it in. Communication is pull or push, researchers can call users on a regular basis for planned information-gathering and users can text feedback whenever motivated.
Martha Cotten: When stakeholders are in corporate silos such as production, finance, public policy, risk management, etc. how do you get collaboration that is not accidental? Two tools that are good for bringing the team together: GuapoVideo and "Digital Binder". GuapoVideo is another video management tool that lets researchers upload video and then clip, tag, share and chat about it with other members of the research team. GuapoVideo keeps a running log of team inputs that everyone can see.
The "Digital Binder" is a new tool under development at gravitytank. Martha showed us a prototype. The goal is to produce an online version of a research project binder so that all artifacts (text, drawings, video, photos, voice, etc.) can be easily searched, sorted and shared by any team member or stakeholder.
Usman Haque: In an example of technology happening first and then seeking a need it can fill, Usman introduced us to his creation, Pachube . Pachube is a platform for aggregating sensor data. Anything that has a sensor or electronic monitoring device such as electricity meters, heating and cooling systems, etc. can be hooked into Pachube for public or private reporting or to manage buildings or systems to respond automatically to change. As people figure out what they can track and compare, they figure out ways to make use of the technology.
There is a lot of potential for using this information to influence voluntary desirable social behavior around energy consumption. In a example experiment, sensors were wired to plants and lamps in a network of homes. Excess use of your lamp could cause the network to kill someone else's plant. Since members of the network know which locations are the excess consumers, all members regulated their behavior to avoid killing plants. When the network included a lamp and plant at a trade show however, the anonymous show attendees kept the lamp switched on to the selfish, plant-killing setting.
Rob Tannen: This was an interesting discussion of things a designer has to consider about physical behavior when creating products. Because of the broad range of problems, this was not a how-to or what-to do kind of talk but just a series of example to provoke thinking and awareness. Rob suggested we think about a two-dimensional mapping as a starting point: scale & skill. Scale refers to the physical dimension (distance, weight) of the product and skill pertains the training required for use. Example, a cutting tool for gardening vs. a cutting tool for a surgeon.
There is a lack of vocabulary for qualitative ergonomic information so Rob proposed four categories where users can have difficulty or have unintended results: posture (standing or seated), reach (finger/hand, hand/arm, leg), clearance (amount of space to leave free above and around a thing; e.g, keys too close on a keyboard), and strength (minimum and maximum amount of force).
Charting physical space in buildings is important too. Where are people located, what are the flowlines along which they move? What are their lines of sight and the visual clues they look for? (see for inspiration, or entertainment, Synchronous Objects)
For more insights, Rob has a 48 minute presentation on ergonomics as well as additional articles at the Bressler Group and his blog on Fast Company.
Heather Reavey: This was a more general discussion of how designers can connect with their clients to get to breakthrough ideas. I will come back to those thoughts later, in this post, I want to stick with the technology theme. One thing that Heather called out was the problem with too-realistic prototypes. If it's too real, people immediately begin to focus on what's wrong, not what's possible or good. As cautionary tales, she reminded us of zombies, alien invaders, and the failure of a certain digitally animated film. She suggested we investigate Masahiro Mori's Uncanny Valley hypothesis.
Kim Erwin: Discussed how online tools were allowing design research to change. Traditional methods either put the user and researcher in the same time and place - interview/observation - or they separated the researcher from the user by time and place - surveys/logs collected from the user and analyzed via automation or in the researcher's office. Online tools now provide platforms where the user can do self-reporting at different times but the researcher comes to the same place where the user reports -- blogs, forums, narrative building sites. Online tools and platforms support expanded access, mobility, intimacy and collaboration. Kim presented three tools that her student team has been working with: Revelation, QualVu and Civicom.
Revelation allows the researcher to set up projects containing a variety of assignments that the user performs online at different times such as questionnaires, diaries, discussion forums, etc. The researcher can review user inputs and tag them for querying and comparison. QualVu is a tool that lets users create video diaries in their home or workspace. The researcher can access these videos and create accompanying text commentary that is stored with the video by QualVu. The researcher can also pull video clips together to create online reports and presentations. Civicom is a voice and text platform that lets researchers and users phone or text it in. Communication is pull or push, researchers can call users on a regular basis for planned information-gathering and users can text feedback whenever motivated.
Martha Cotten: When stakeholders are in corporate silos such as production, finance, public policy, risk management, etc. how do you get collaboration that is not accidental? Two tools that are good for bringing the team together: GuapoVideo and "Digital Binder". GuapoVideo is another video management tool that lets researchers upload video and then clip, tag, share and chat about it with other members of the research team. GuapoVideo keeps a running log of team inputs that everyone can see.
The "Digital Binder" is a new tool under development at gravitytank. Martha showed us a prototype. The goal is to produce an online version of a research project binder so that all artifacts (text, drawings, video, photos, voice, etc.) can be easily searched, sorted and shared by any team member or stakeholder.
Usman Haque: In an example of technology happening first and then seeking a need it can fill, Usman introduced us to his creation, Pachube . Pachube is a platform for aggregating sensor data. Anything that has a sensor or electronic monitoring device such as electricity meters, heating and cooling systems, etc. can be hooked into Pachube for public or private reporting or to manage buildings or systems to respond automatically to change. As people figure out what they can track and compare, they figure out ways to make use of the technology.
There is a lot of potential for using this information to influence voluntary desirable social behavior around energy consumption. In a example experiment, sensors were wired to plants and lamps in a network of homes. Excess use of your lamp could cause the network to kill someone else's plant. Since members of the network know which locations are the excess consumers, all members regulated their behavior to avoid killing plants. When the network included a lamp and plant at a trade show however, the anonymous show attendees kept the lamp switched on to the selfish, plant-killing setting.
Rob Tannen: This was an interesting discussion of things a designer has to consider about physical behavior when creating products. Because of the broad range of problems, this was not a how-to or what-to do kind of talk but just a series of example to provoke thinking and awareness. Rob suggested we think about a two-dimensional mapping as a starting point: scale & skill. Scale refers to the physical dimension (distance, weight) of the product and skill pertains the training required for use. Example, a cutting tool for gardening vs. a cutting tool for a surgeon.
There is a lack of vocabulary for qualitative ergonomic information so Rob proposed four categories where users can have difficulty or have unintended results: posture (standing or seated), reach (finger/hand, hand/arm, leg), clearance (amount of space to leave free above and around a thing; e.g, keys too close on a keyboard), and strength (minimum and maximum amount of force).
Charting physical space in buildings is important too. Where are people located, what are the flowlines along which they move? What are their lines of sight and the visual clues they look for? (see for inspiration, or entertainment, Synchronous Objects)
For more insights, Rob has a 48 minute presentation on ergonomics as well as additional articles at the Bressler Group and his blog on Fast Company.
Heather Reavey: This was a more general discussion of how designers can connect with their clients to get to breakthrough ideas. I will come back to those thoughts later, in this post, I want to stick with the technology theme. One thing that Heather called out was the problem with too-realistic prototypes. If it's too real, people immediately begin to focus on what's wrong, not what's possible or good. As cautionary tales, she reminded us of zombies, alien invaders, and the failure of a certain digitally animated film. She suggested we investigate Masahiro Mori's Uncanny Valley hypothesis.
Tuesday, March 31, 2009
Is there room for UCD in Agile? part 3
So now we are at Question 3:
Do the UI stories count as "done", even when no software has been written?
"Count" is the operative word here. Do we mean "count" as in "keep track" or "tick off on a list" or do we mean "count" as in adding up some numbers and coming to a conclusion about actuals vs. estimates?
The purpose of saying a story is Done, is to have a means of measuring the team's velocity -- how much working software gets built on average per iteration/ sprint. Thus, in theory, you only really care about Done if the story is a coding story.
But you still need some way of determining if the flow of stories out of the backlog will impede the team's velocity. Now we're talking lean. Are stories getting delivered into the "ready-to-build" backlog fast enough so that the development team can always pull something off the backlog given their average velocity? Or are the developers going to sit around with nothing to do because stories won't be ready? Or better yet, can we move on to the next phase early because we're going to blow through / complete all the stories originally planned for this sprint/phase?
It is important to see the status of those higher-level analysis and UI research tasks that precede the coding stories. If UI work was substantial enough to make it visible as a "story" then you have to be able to say if it is not started, in-progress, or done, and have target expected completion dates so the team can get some kind of assurance about how and when the backlog may grow or shrink.
You can even assign and track analysis estimates and velocity in the same way that you track development velocity. But don't mix the two things together; the velocities are measuring completely different things.
UI research and the subsequent problem definition and solution finding (design thinking) will either add new coding stories, provide details for existing stories, or remove stories. Where the research provides new details for existing stories, that information will increase or decrease the stories's complexity. Likewise, adding and removing stories changes the estimated amount of effort in the backlog.
Being able to see when UI findings are likely to conclude gives the team information about future events that could affect the backlog and helps everyone roadmap the project's progress.
The team needs to be able to see it like this: Our velocity is V. The total point value of our backlog is 4xV so the project should complete in 4 sprints BUT, we still have two UI research tasks to complete and our experience so far on this project indicates that those tasks could add as much as 1xV each in changes to the backlog. So we have a total potential story point increase of 2xV. Therefore, as of today, our best estimate of finishing is in 5 sprints but we may need as many as 6.
(If you are curious about assigning story points for tracking velocity, here's one nice way to do it.)
Do the UI stories count as "done", even when no software has been written?
"Count" is the operative word here. Do we mean "count" as in "keep track" or "tick off on a list" or do we mean "count" as in adding up some numbers and coming to a conclusion about actuals vs. estimates?
The purpose of saying a story is Done, is to have a means of measuring the team's velocity -- how much working software gets built on average per iteration/ sprint. Thus, in theory, you only really care about Done if the story is a coding story.
But you still need some way of determining if the flow of stories out of the backlog will impede the team's velocity. Now we're talking lean. Are stories getting delivered into the "ready-to-build" backlog fast enough so that the development team can always pull something off the backlog given their average velocity? Or are the developers going to sit around with nothing to do because stories won't be ready? Or better yet, can we move on to the next phase early because we're going to blow through / complete all the stories originally planned for this sprint/phase?
It is important to see the status of those higher-level analysis and UI research tasks that precede the coding stories. If UI work was substantial enough to make it visible as a "story" then you have to be able to say if it is not started, in-progress, or done, and have target expected completion dates so the team can get some kind of assurance about how and when the backlog may grow or shrink.
You can even assign and track analysis estimates and velocity in the same way that you track development velocity. But don't mix the two things together; the velocities are measuring completely different things.
UI research and the subsequent problem definition and solution finding (design thinking) will either add new coding stories, provide details for existing stories, or remove stories. Where the research provides new details for existing stories, that information will increase or decrease the stories's complexity. Likewise, adding and removing stories changes the estimated amount of effort in the backlog.
Being able to see when UI findings are likely to conclude gives the team information about future events that could affect the backlog and helps everyone roadmap the project's progress.
The team needs to be able to see it like this: Our velocity is V. The total point value of our backlog is 4xV so the project should complete in 4 sprints BUT, we still have two UI research tasks to complete and our experience so far on this project indicates that those tasks could add as much as 1xV each in changes to the backlog. So we have a total potential story point increase of 2xV. Therefore, as of today, our best estimate of finishing is in 5 sprints but we may need as many as 6.
(If you are curious about assigning story points for tracking velocity, here's one nice way to do it.)
Friday, March 27, 2009
Is there room for UCD in Agile? part 2
In my last post, I started to cover some ground on how Agile and UCD can fit together. These postings come from a discussion on agile usability in which Robert Biddle of Carleton University, asked me three very good questions.
Q1: are the UI designers and programmers within the same team but do different kinds of tasks, or do they do both?
Q2: what are the UI stories (or backlog items) like, especially for user research?
Q3: do the UI stories count as "done", even when no software has been written?
My answer to Question 1 essentially elaborates on my previous post: UI designers and UI programmers are just another flavor of business analyst or programmer and they would collaborate with the other analysts and programmers on all the tasks that need to be done including user research and usability testing.
For example, a team could have a person who has skills in UI design and UI programming. That person could pair up with another analyst to do user research in one iteration, then pair up with a developer to code the UI part of a story(s) in another iteration. A team may also have a person who is a UI designer and experienced in user research but not experienced in UI programming. That type of person would stay completely in the analyst / customer proxy role doing things like user research, product design, acceptance testing, facilitating showcases with the stakeholders, and also any needed product documentation.
The other programmers on the team would also pair up with UCD analysts as needed to have conversations, investigate solutions to a problem interface, and conduct or witness research into the user experience.
Q2: what are the UI stories (or backlog items) like, especially for user research?
We have to be careful with the term "story". "Story" is a thing that developers implement. "Tasks" are things that have to be done to get stories through the pipeline. Tasks are activities people do to figure out what the story should be, realize the story in code, or verify the story is done. You have a UI story if you have something that can be stated like this:
As a mother, I want to be able to search for a doctor who can treat my child's condition and practices close to my home. Acceptance: The search functionality must be easy to find on the home page, you can search by all or part of a medical term (e.g., cancer, scoliosis, juvenile diabetes) and/or a location (city, state, zip), the search results will include the doctor's names, specialities, location(s) and photo.
You have a UI task if you have something that can be stated like this:
Interview/observe 6-10 mothers of children under the age of ten to understand how they would search for and find a doctor for their child on the internet.
Or this:
Interview/observe 6-10 mothers of children under the age of ten to gather information on their expectations for a children's hospital website.
Or this:
Analyze interview results and identify new candidate stories for inclusion in the current or future development phases.
You definitely make the coding stories visible in the master story list and track their progress, but you may not build a backlog of all of the tasks. High-level tasks might be treated "as if" they were stories so it is easy for the entire team to be aware of overall progress and activities for the iteration, but your tracking system would have some type of physical distinction between the analysis/UI tasks and the coding stories.
For example, your information radiator/story wall would likely use color or parallel tracks to distinguish coding stories from other activities. The development track might show stories 2, 3, and 4 in play while your analysis/UI track might show that story 1 is out for customer and UI acceptance testing, detailed requirements discovery is starting for story 5, and user research (UI task) relevant for proposed backlog stories 6, 8 and 9 is in-progress.
A lot of tasks are not made visible because it's just too much noise. But people still report status and blockages in daily stand-ups -- I'm going to be late finalizing the requirements for story 5 because I can't get a meeting with Mr. Y; we need to get the XML spec for the interface to system G before we can complete story 3; our user interviews are proceeding on schedule and we have no problems to report.
I'll cover question 3 in another post since that also has a long-ish answer.
Q1: are the UI designers and programmers within the same team but do different kinds of tasks, or do they do both?
Q2: what are the UI stories (or backlog items) like, especially for user research?
Q3: do the UI stories count as "done", even when no software has been written?
My answer to Question 1 essentially elaborates on my previous post: UI designers and UI programmers are just another flavor of business analyst or programmer and they would collaborate with the other analysts and programmers on all the tasks that need to be done including user research and usability testing.
For example, a team could have a person who has skills in UI design and UI programming. That person could pair up with another analyst to do user research in one iteration, then pair up with a developer to code the UI part of a story(s) in another iteration. A team may also have a person who is a UI designer and experienced in user research but not experienced in UI programming. That type of person would stay completely in the analyst / customer proxy role doing things like user research, product design, acceptance testing, facilitating showcases with the stakeholders, and also any needed product documentation.
The other programmers on the team would also pair up with UCD analysts as needed to have conversations, investigate solutions to a problem interface, and conduct or witness research into the user experience.
Q2: what are the UI stories (or backlog items) like, especially for user research?
We have to be careful with the term "story". "Story" is a thing that developers implement. "Tasks" are things that have to be done to get stories through the pipeline. Tasks are activities people do to figure out what the story should be, realize the story in code, or verify the story is done. You have a UI story if you have something that can be stated like this:
As a mother, I want to be able to search for a doctor who can treat my child's condition and practices close to my home. Acceptance: The search functionality must be easy to find on the home page, you can search by all or part of a medical term (e.g., cancer, scoliosis, juvenile diabetes) and/or a location (city, state, zip), the search results will include the doctor's names, specialities, location(s) and photo.
You have a UI task if you have something that can be stated like this:
Interview/observe 6-10 mothers of children under the age of ten to understand how they would search for and find a doctor for their child on the internet.
Or this:
Interview/observe 6-10 mothers of children under the age of ten to gather information on their expectations for a children's hospital website.
Or this:
Analyze interview results and identify new candidate stories for inclusion in the current or future development phases.
You definitely make the coding stories visible in the master story list and track their progress, but you may not build a backlog of all of the tasks. High-level tasks might be treated "as if" they were stories so it is easy for the entire team to be aware of overall progress and activities for the iteration, but your tracking system would have some type of physical distinction between the analysis/UI tasks and the coding stories.
For example, your information radiator/story wall would likely use color or parallel tracks to distinguish coding stories from other activities. The development track might show stories 2, 3, and 4 in play while your analysis/UI track might show that story 1 is out for customer and UI acceptance testing, detailed requirements discovery is starting for story 5, and user research (UI task) relevant for proposed backlog stories 6, 8 and 9 is in-progress.
A lot of tasks are not made visible because it's just too much noise. But people still report status and blockages in daily stand-ups -- I'm going to be late finalizing the requirements for story 5 because I can't get a meeting with Mr. Y; we need to get the XML spec for the interface to system G before we can complete story 3; our user interviews are proceeding on schedule and we have no problems to report.
I'll cover question 3 in another post since that also has a long-ish answer.
Tuesday, March 24, 2009
Is there room for UCD in Agile?
I'm a member of an agile usability discussion group. There has been a lot of discussion lately about how and whether designers and UCD people can work within an Agile project. There is a lot of concern that methods and aproaches may not mesh -- that Agile starts developing and releasing stuff long before the designer can complete their diagnosis of the problem and create an effective solution.
for most of my time at ThoughtWorks, we did not have any UCD staff but that has changed over the last year or so. Still, every project is different and we shape it according to what works for the situation.
Sometimes, there is no UCD involvement and business analysts and developers just have to make things up based on what they see as making sense and where the client is coming from (often these are applications for the clients internal use ). Sometimes there are projects where the client hired a designer independently of bringing TW in for the implementation and their design is pretty much a static requirement in place at the start or delivered part way through (not very good). And then there are projects where we can have our own UCD person on-board or the outside design firm is committed to working as part of the team through the life of the project. That's when it gets good.
Ideally, there is one or more UCD-skilled person(s) involved on the team through the life of the project. When you have this, the UCD person is utilizing their skills as they play some combination of other roles: product owner/customer, business analyst/SME, developer. This is very much like the way a business analyst would also fit into team.
A business analyst has to be able to understand the business operationally, grasp the strategic objectives of the stakeholders; interview stakeholders and customers/users to discover and verify requirements; be an advocate on the team for the customer/stakeholders; be an advocate for the strategic objectives (sometimes even stakeholders lose track of the greater goals); know enough about technology to have an intelligent conversation with developers so they can advocate back to the stakeholders when choices get hard; and maybe even sometimes do something technical themselves, plus be able to specify and conduct effective acceptance tests so everyone can be assured the story is Done when it's done.
That is not so very different from what the UCD person wants to achieve. Given that, the UCD member of the team is just another flavor of analyst /developer on the team. They concentrate on their specialty but they also collaborate on other just-in-time project tasks as appropriate. For example, maybe doing javascript coding or conducting showcases with the stakeholders, or writing stories and acceptance tests. Programmers also pair up with the UCD analysts as needed to have conversations, investigate technical issues, or research the business problem.
for most of my time at ThoughtWorks, we did not have any UCD staff but that has changed over the last year or so. Still, every project is different and we shape it according to what works for the situation.
Sometimes, there is no UCD involvement and business analysts and developers just have to make things up based on what they see as making sense and where the client is coming from (often these are applications for the clients internal use ). Sometimes there are projects where the client hired a designer independently of bringing TW in for the implementation and their design is pretty much a static requirement in place at the start or delivered part way through (not very good). And then there are projects where we can have our own UCD person on-board or the outside design firm is committed to working as part of the team through the life of the project. That's when it gets good.
Ideally, there is one or more UCD-skilled person(s) involved on the team through the life of the project. When you have this, the UCD person is utilizing their skills as they play some combination of other roles: product owner/customer, business analyst/SME, developer. This is very much like the way a business analyst would also fit into team.
A business analyst has to be able to understand the business operationally, grasp the strategic objectives of the stakeholders; interview stakeholders and customers/users to discover and verify requirements; be an advocate on the team for the customer/stakeholders; be an advocate for the strategic objectives (sometimes even stakeholders lose track of the greater goals); know enough about technology to have an intelligent conversation with developers so they can advocate back to the stakeholders when choices get hard; and maybe even sometimes do something technical themselves, plus be able to specify and conduct effective acceptance tests so everyone can be assured the story is Done when it's done.
That is not so very different from what the UCD person wants to achieve. Given that, the UCD member of the team is just another flavor of analyst /developer on the team. They concentrate on their specialty but they also collaborate on other just-in-time project tasks as appropriate. For example, maybe doing javascript coding or conducting showcases with the stakeholders, or writing stories and acceptance tests. Programmers also pair up with the UCD analysts as needed to have conversations, investigate technical issues, or research the business problem.
In my next post(s), I'll discuss three very specific questions about how this can work.
Subscribe to:
Posts (Atom)