| August 30, 2006
For pleasure reading, I turn to science fiction, and I'm currently reading Chris Moriarty's Spin Control
. In this novel, each chapter opens with a quote, which may be real or fictitious. The quote preceding the chapter titled Information Costs
is not only real, it is highly relevant to our business.
These three assumptions are fundamental: information is partial, information is contaminated, and information costs. – J. Traub (1988).
While I have not been able to track down the source of the quote, I did find the book Complexity and Information
, authored by Joseph F. Traub, and Arthur G Wershcultz. The ideas in the book seem as pertinent to market research as to any other discipline that seeks to make predictions. If these assertions above are true, why do so many people expect research to yield certainties at minimal cost?
In the book's introduction, Traub and Werschulz explain that information is partial because it cannot uniquely identify a particular physical state, and that it is contaminated because it relies on measurements which are subject to error. To illustrate these points, they cite the example of a weather forecast. A forecast combines measurements from a variety of sources: earth-based stations, planes, balloons, ships and satellites. They explain:
"Since the number of measurements is finite, the information is partial. Inevitably, these measurements are contaminated with errors. Hence we can only know the current weather to within some error. Since weather appears to be chaotic, this error is amplified, and limits our ability to forecast the weather to only a matter of days."
(Even this guarded statement seem optimistic to those of us on the East Coast who got soaked last weekend after receiving a forecast of "partly cloudy.")
After providing a couple more examples, Traub and Werschulz conclude their discussion of incomplete and inaccurate information, stating, "At best, we can only guarantee an approximate solution." Then they move on to discuss another dimension of information: the expense associated with gathering it.
The comparison to the practice of market research is obvious. While we do not seek to predict something as chaotic as the weather, we do make predictions based on partial and contaminated information, and that information does have a cost. Implicit in Traub and Wersculz's Introduction is the trade-off between the quantity and quality of information and its cost. They describe a problem-solving algorithm as optimal if "it uses the information to produce an acceptable approximation to the answer at minimal cost."
The same could be said of any research project or analysis and it is worth bearing in mind the next time you write or review a research proposal. Will the information created by the research provide you with an acceptable "approximation" for the smallest possible cost? All too often, I believe, we focus on just one side of this equation, and assume the project with the lowest cost (and quickest turnaround) will be the best choice. Rarely do we focus on the other side of the equation and ask "Will this research give us an acceptable approximation given how we intend to use the information?"
The downside of not considering both sides of this equation is not just the risk of gathering inaccurate or misleading information. It extends to the increased likelihood of making poorly-informed decisions based on that information, and, ultimately, to poor business performance—shortfalls in revenue, weaker share prices and job losses.
We would do well to remember that there are potentially significant ramifications if we fail to approximate correctly. The lost opportunity cost of bad research may not be as destructive and immediate as the failure to predict the path of a hurricane, but it is still considerable, and potentially more insidious for its lack of immediacy.
To sum up:
Research does not deal in certainties; it deals in approximations. And the quality and reliability of those approximations are intrinsically linked to the cost of obtaining the information involved.
At a time when people seem to be looking for research to provide THE answer, quickly, and at a low cost, everyone from senior management to research buyers to research suppliers would do well to remember these facts.