Filling the Gaps for a Composable CDP

Back in February, I mentioned that CDP Institute had published a Composable CDP Self-Assessment Tool that asks people what gaps they must fill to convert their current systems into the functional equivalent of a CDP. I recently checked how many responses we’ve received, and was disappointed that there were just fourteen. Obviously this isn’t enough to draw statistically meaningful conclusions, especially when you bear in mind that the audience of CDP Institute members can’t be considered representative of the industry as a whole. But I summarized the answers anyway to see what rough patterns might emerge. They were much more intriguing than I expected.

To set the context: the survey asks the status of 102 customer data management functions in the respondent’s current systems. These are grouped into eleven categories: data capture, data sources, ingestion, data preparation, data storage, identity linking, customer profiles, data sharing, process integration, segment creation, and segment (i.e., audience) output. Answer options are: not needed; needed and available; needed and not available; and don’t know. The most important of these is “needed and not available,” since that’s a gap to be filled. All questions were required.

When I took my first overview of the data, the first, and critically important, observation was that there did seem to be significant clusters in the answers. That was important because it suggested that people were giving accurate answers – at least, to the best of their knowledge – rather than randomly filling in a response. Had the answers been random, I would have expected to see roughly the same distribution of responses to each question, and that was definitely not the case. I’ll guess that requiring answers to all 102 questions filtered out the people who didn’t take the survey seriously.

The next observation was that gaps were more common in some areas than others. The gap percentages (i.e., percentage of "needed and not available" replies) ranged from 46% to 29%. The pattern is clear: the areas with fewest gaps (data sources, data store, data capture, and data sharing) are needed for pretty much any data warehouse, while the most gaps were tied to customer data management (data preparation, identity linking, ingestion, segment output, customer profiles).  This adds to the credibility of the data, even allowing for some confirmation bias.  More important, it's a reminder that you can’t assume a data warehouse built for other purposes will have all the features needed to support CDP applications.

A look individual functions shows something hidden by the category averages: even categories closely related to data warehouses will still have functional gaps for supporting a CDP. For example, while data warehouses are generally good at storing data, they often lack third party data and privacy management. Similarly, although capturing and sharing data are core capabilities for a data warehouse, they often lack real-time connectivity. This reinforces the need to look at requirements in detail when assessing the suitability of your existing warehouse as the base for a CDP.

 

It's also worth looking at the functions without the category filters. This shows more clearly that common data warehouse functions (such as loading structured data) are rarely gaps while CDP-specific functions (such as connections to advertising media, anonymous-to-known profile conversations, and end-user data access) are often gaps.  There are too few responses to make these rankings anything approaching precise. But, even if they were, what any individual company needs would still depend on its own situation.

Given the small sample size of this data set, it’s best to view these results more as anecdotes than an industry survey. But I still believe they offer some useful guidance to CDP vendors and users:

• For vendors: it’s worth noting which categories show the most gaps. The “composable CDP” discussion was initially driven by companies offering reverse ETL features, which correspond roughly to the data sharing, segment creation, and segment output categories. Ironically, these fall in the middle of category gap rankings. The biggest gaps, and probably the greatest opportunity for CDP components, relate to specialized data preparation, identity linking, ingestion and segment output. I'd guess that the reason the "composable" discussion started elsewhere is tools for specialized data prep, identity linking, etc. are already widely available, so there was less room for new entrants.  By contrast, reverse ETL is a relatively new category where it's easier for a new firm to establish itself.  CDP is actually just one application of reverse ETL; in "crossing the chasm" terms, it's a beach head where new vendors can establish themselves and then grow beyond. 
  

From a different perspective, it's also not surprising that more composable CDP vendors are offering products to fill gaps in other CDP categories: it's always easier to sell a new product to an existing customer than to sell an existing product to a new customers.  Offering a more complete set of components begins to make the composable products look more like an integrated suite, losing some of the flexibility of swapping components but also reducing the integration burden that composable vendors otherwise ignore.  It still maintains the core difference between a standard CDP that builds a separate database and a "composable" one that draws from the warehouse.

• For users: the list of common gaps is a helpful reference for ensuring that problematic functions are all considered in a requirements analysis. This is especially important at companies with strong existing data warehouse teams, whose members may not be familiar with CDP requirements.  To the extent that users' own systems match the category gap averages, it may make the most sense to consider custom enhancements to existing capabilities for categories with relatively few gaps, while buying new packaged components for categories with many gaps. Of course, if a company has many gaps across all categories, it probably makes more sense to buy a traditional CDP than to buy, integrate, and maintain a large number of separate components. 

 

 

CDP Success Depends on More than Marketers

The CDP Institute runs periodic roundtables where our members discuss industry issues.  We had one earlier this week on the topic of helping companies make use of an existing CDP.  We chose that topic because we frequently hear that many users don’t know what to do with a CDP after it’s built.

The initial discussion followed the more or less expected path towards solutions such as developing change champions and publicizing success stories to encourage CDP use.  Participants stressed the need for organizational change to allow applications that the CDP makes possible.  This change includes cooperation between teams that had previously not worked together, new processes, and new metrics for cross-channel marketing programs.  We heard that traditional direct marketing organizations are especially likely to struggle precisely because they have such mature single-channel techniques in place.

On the bright side, the group said that real-time capabilities were the most likely to create new opportunities that would make users excited enough to adopt new approaches.  Good to know.

The discussion then turned in an unexpected direction: the shift in CDP leadership from marketers to IT and data teams.  One participant said in the past two years, he had found that 90% of new projects were initiated by IT.  While that’s certainly a trend I’ve observed, the group added several new insights.

• One reason for the shift is that CDP deployment requires more technical skill than most marketing teams can muster.  I have generally argued the reason for greater IT and data team involvement is the broader use of customer data beyond marketing departments.  This makes the CDP an enterprise-level project.  But it’s probably also true that CDPs used more widely throughout the organization are more technically complicated than CDPs used only in marketing.  Broader applications imply more data sources, more types of processing, and connections to more applications.  This all requires greater involvement by the company’s IT and data teams.

• IT and data teams need new skills to succeed with a CDP.  This goes beyond the need to understand specific processes such as identity resolution, which many IT and data teams had not previously needed to support.  It includes the need for those teams to better understand how marketers actually do their jobs.  Without that fundamental understanding, IT and data teams will make poor choices in designing systems to support marketers.  Traditional project-by-project requirements gathering isn't enough.
  

• More specifically, IT and data teams need to recognize that marketers have a different attitude towards data.  One participant observed (and I agree) that IT teams want to make sure everything is correct before they release any data, while marketers are willing to accept some errors in return for quicker results.  It’s an astute observation about culture that both sides should recognize and discuss so they can agree on the right balance for any particular application. 
  

• A bigger role for IT and data implies a smaller role for marketing technologists.  They’re still involved in selecting martech, of course, but are less likely to lead a CDP project than they once were.  This probably applies to other marketing technology projects as well, especially if it applies to departments outside of marketing.  This isn't a death knell for martech staff.  But it does imply a shift in responsibilities, away from designing the martech stack to system administration, support, and analytics.  

Those final two points – about the different attitudes towards data quality and the changing role for martech staff – both point to a need for more CDP education aimed at IT and data teams, rather than just marketers.  That’s something we’ll consider here at CDP Institute and I suspect will be important for industry vendors, consultants, and practitioners as well.