As marketers, we all want to “build a better mousetrap” — that is, to market a product or service that’s perfectly positioned to entice and encourage customers to buy it.  So market research, and predicting consumer behavior is a critical part of our jobs.

On February 16, 2012, Forbes carried an article titled, “How Target Figured Out A Teen Girl Was Pregnant before Her Father Did.”  It stirred up a lot of controversy about the data that retailers collect, and how it’s used.

Scott Anthony’s book The Little Black Book Of Innovation: How It Works, How to Do It hasn’t generated the same firestorm – although it spells out in detail how to accurately predict consumer behavior before it happens.  Predicting consumer behavior with data-driven marketing and market research are just part of what the book is about, of course, but it’s well worth the $16 asking price for those sections alone.

Anthony, managing director of consulting and investment firm Innosight, writes that sudden flashes of inspiration are good, but a business can’t really rely on them.  “Steve Jobs was right when he said, ‘It is not the customer’s job to know what they want.’”

So if it’s our job as date-driven marketing professionals to know what the customer wants before they know themselves, where do we start?  Peter Drucker explained it this way over 50 years ago, “The customer rarely buys what the company thinks it sells him.”

Market research can uncover part of what consumers want, but even people with the best intentions often provide false or inaccurate information during the research process.  Peter Drucker said that people don’t want quarter-inch drills – they want quarter-inch holes.  So, he wrote, “Businesses need to look at the problems customers are trying to solve, what work-around they are using to solve the problem, and the context in which the problem is being solved.”

Only then, Anthony says, is it time to start designing a product or service.  The book suggests three key steps in predicting consumer behavior – and, of course, using those predictions to create the solutions that consumers will want to buy.

Here are the three key processes Anthony defines.

Get to Context

When A.G. Lafley became CEO of Procter & Gamble in 2000 after the company’s stock had plunged almost 50%, he came to the conclusion that P&G needed to fundamentally reorient itself. The company was world renowned for driving decisions based on deep customer understanding, but Lafley thought that the company had drifted away from that understanding.

His mantra was simple:  The customer is your boss.  In every meeting, he said something like this:  “Fellow P&G-ers, I’d like you to meet your new boss. You may think that I, as your CEO, am boss. That’s not right. You might think that the board of directors to which I report is boss. That’s not right. You might think our shareholders are the bosses. That’s not right. You might think your line manager is boss. That’s not right. We have one and only one boss that matters. The consumer. The consumer is boss.”

Lafley urged his co-workers to hear what the consumer was saying and, much more importantly, tease out what the consumer wanted but couldn’t articulate.  For instance, while Lafley worked on Tide laundry detergent, he found that consumers told market researchers that they loved Tide’s cardboard box packaging. Yet, when Lafley was interacting with a consumer, he noticed that she almost always used a screwdriver or scissors to open the Tide box because she didn’t want to risk breaking her nails. She said she loved the packaging because she didn’t know of any alternatives, but in reality, she had to find a creative way to open the box because of its design limitations.

Watch for Workarounds

Consumers make up for problems with the products they’re offered with creative workarounds.  For instance, market research shows that women hate shopping for blue jeans almost as much as they hate shopping for bathing suits.

In 2009, as part of an ambitious innovation program, VF Corporation, which makes Wranglers and Lee Jeans, began to spend more time with customers in order to understand their frustrations with the buying process.  During a trip to a local store, executives watched as a prospective female customer shopped for a new pair of jeans. She wandered around the endless racks of clothes in the store, picking up pair of jeans after pair of jeans.

Two things stood out:

• The number of pairs of jeans brought into the dressing room.
• The fact that the woman picked up multiple-sizes of every pair she was trying on.

Executives wondered if she had recently gained or lost weight, and was unsure of her size.  In fact it turned out that her experience taught her that the sizes that appeared on the labels of jeans only loosely related to what would actually fit. Her workaround involved bringing in volumes of pairs of jeans in order to find one good fit.

Changing the labeling on their jeans, and creating a campaign that helped women find jeans that were flattering and fit correctly for different body times, resulted in $100 million in incremental revenue for the company – during the worst recession in over 70 years.

Focus on Non-Buyers

The natural tendency is to study existing customers.  Anthony says it’s important to keep doing that.  But also look for people who face some kind of constraint that makes it hard for them to solve a pressing problem.  If you can eliminate the barrier, you tap into a whole new market.

Anthony cites Apple, Southwest, and IKEA as examples of companies that found huge markets just sitting there because consumers had unmet needs. “It takes some mental discipline to look to markets that don’t exist. But that discipline can pay off in the form of growth opportunities that are hidden in plain sight,” Anthony writes, adding, “To get started, write down five things that a coworker or friend can only do by relying on an expert or going to a central location. Think about ideas that would let these people do it themselves.”