It's a textbook tautology: Product success hinges on meeting customer needs. Only when you identify what customers genuinely value can you develop a feature set that survives and thrives past a soft launch. If only it was as simple as asking them. Customers often don't know what they want until they experience it.

Think about the hardcore BlackBerry owners of yore. As they scrolled physical “thumbwheels” and tapped away on rows of ebony-colored QWERTY Tic Tacs, did they dream of a better world? Did any of them pine for a glass touchscreen to replace all that clicking and spinning? Of course not. Yet when they were presented with one, they fled their BlackBerries with surprising speed for early-model iPhones.

It’s the main reason predicting latent needs is difficult. Below I outline how owned, behavioral data can make it easier. As you’ll see, there is figurative “gold” to be found in first-party data. I offer two examples. One is from roughly the era of the birth of the iPhone. The other is as fresh as today’s headlines.

Experiential Market Understanding

To predict latent needs, consider mining experience data the way a scrappy young Netflix did. In their initial years, Netflix conducted experiments to identify hidden viewing interests. The most famous of these is their International Netflix Prize.

This was an open competition for the best algorithm to predict film preferences based on previous viewership and user ratings. Everyone was invited to give it a try, armed with nothing more than a massive, downloadable first-party dataset.

The purse to the winner was a million dollars -- quite a sum in 2009, the depths of the Great Recession. It would go to the team producing an algorithm that increased the accuracy of movie recommendations by the greatest rate. (Most couldn’t reach the qualification level of 10%.)

The steps were as follows:

1. Netflix did the actual data “mining,” a metaphor that for me always evokes the Pacific Northwest gold rush of the late 1800s. The Netflix data team extracted millions of anonymized viewer records, along with the ratings they gave each film, and made the data public for the challenge.
2. Independent data scientists from around the world applied various models, the way gold prospectors might sift through the silt of Klondike streams to find nuggets that could make them rich.
3. Half the data would be withheld from initial data studies, so when a promising algorithm was tested on the untouched control data, improved predictive power could be measured with statistical certainty,

Netflix, by the way, never actually used the winning algorithm as written, though there was indeed the promised payout.

Applying a new algorithm wasn’t the company’s goal. Instead, they learned a great deal about what films were most meaningful to niche segments of users — insights far exceeding the million-dollar price tag! That led to improvements to their platform that increased profits and burnished the brand. It also helped in building their streaming business, where instant, new film recommendations could make the difference between a long-term subscriber and one who cancels in favor of other entertainment options.

Needless to say, this type of contest would not be needed today. Artificial intelligence (AI) can optimize segments for outcomes, such as highly valuing new offerings, much faster than teams of human data scientists ever could.

This analysis of massive data sets using AI has led to a new field of customer insights generation we call Progressive Customer Profiling.

How to Create Content Customers Love: The Team and The Tools

Download this ebook today and we’ll cover the team roles and brand guideline basics writers need to create copy that keeps customers reading and ultimately guides them to conversion.

Register Today

Progressive Customer Profiling

The key to winning the Netflix Prize was to find a number of profiles (sometimes referred to as personas or segments) that were large enough and powerful enough — when activated against — to greatly improve the bottom line. Although the entire Netflix data set only needed to be improved by 10%, the winning algorithm found “veins of gold” in the form of segments that were likely to respond to targeted recommendations at much greater rates than just 1-out-of-10. Back then, the data set was only one source of owned data.

Much has changed.

Today, because of the advent of customer data platforms (CDPs), many owned data sets can be stitched together, expanding the number of attributes upon which AI can apply its predictive models. It’s a level of customer understanding that was unheard of even 10 years ago.

What’s more, this customer knowledge can expand in milliseconds. Thanks to real-time data collection, customer profiling is immediate, and capable of identifying latent needs that are strong but fleeting. As David Ogilvy wrote many years ago in his book on advertising and direct response, “You aren't advertising to a standing army; you are advertising to a moving parade.”

Arguably, some latent needs are neither fleeting nor niche. But most of those truly profound needs are identified by direct research, not behavioral data. Automakers late in the last century undoubtedly knew the cupholder was a good idea by studying the growth of fast-food drive-throughs. News of a lawsuit in 1994 by an elderly woman burned by spilled hot coffee only clinched the deal, making cup holders the ubiquitous vehicle feature it is today.

Many other needs are ephemeral -- not as persistent as the persistent need for a place to plant your soda or coffee. This is where real-time AI insights can shine.

By analyzing customer and prospect data with unprecedented power and speed, the capability of targeted selling and cross-selling opportunities on a site or mobile app has never been more attainable. What’s more, Big Data analysis done by R&D and data science teams, facilitated by tools like Adobe’s Customer Journey Analytics (CJA), can help brands pinpoint promising customer profiles for new product tests.

Progressive customer profiling is not new. In the heyday of print catalogs, segments of customers would receive extremely targeted mini-catalogs. The customers welcomed them, voting with their dollars. 

Today we’re marketing to their children and grandchildren online and in their phones, with bespoke recommendations of far greater potency. Moreover, because we’re meeting legitimate customer needs, these marketing advances are truly a win-win. These advances are growing the bottom line through both immediate sales and long-term customer delight.