An interesting paper from David Gal and Itamar Simonson that investigates our ability to predict consumer choices in an age of “big data”.

First, preferences are far from static:

“To be sure, in some cases consumers do have strong, precise, stable preferences for particular products or attributes, and they may habitually buy the same options. For instance, some people prefer to buy a 2% organic milk. Likewise, a few consumers may have self-imposed rule as to the highest price they are willing to pay for a water bottle, which prevents them from buying water at airports. When preferences for products or attributes are strong, stable, and precise, consumer choices are relatively easy to predict, such as by simply asking consumers about their preferences.

“However, most of the choices made by consumers that are not habitual or routine are not the result of precise, stable preferences for those products, but are constructed (or discovered) at the time a decision is being made on the basis of interactions among many individual and situational factors.”

After digging into conjoint analysis, recommendation engines, and other predictive tools, Gal and Simonson conclude:

“In contrast, the conclusions from our review reinforce the view that marketing remains as much an art as science, whether or not the analyses produce seemingly precise numbers. Marketers, as much as ever, must rely on their creativity, insight and judgment, as well as trial and error, and often some serendipity, to identify and develop truly new products (and messages) that match dormant (or “inherent”) consumer preferences.”