Explaining Martech to a Five Year Old

One piece of paper stands between me and ending the year with a clean desk: a list scribbled on hotel stationery that compares different customer data systems to different types of motor vehicles. Richard Scarry  meets CDP Institute, you might say.

So, just in case your New Year’s plan includes explaining marketing technology to a five year old, here we go:

• Data Warehouse = School Bus. The seats in a bus are lined up in nice neat rows, designed to carry one type of cargo very efficiently. Similarly, a data warehouse is highly structured environment that is very efficient at dealing with specified data types. (See how this works?)

• Data Lake = Moving Van. A moving van is a big box that can hold pretty much anything, although it might be jumbled together in no particular order. A data lake can store any type of data but doesn’t do much to organize it.

• Integration Platform = Delivery Motorcycle. A motorcycle carries small items quickly from one place to another, but doesn’t have any place to store them. An integration platform moves bits of data between systems but doesn’t have its own storage, either.

• Data Management Platform = Fire Truck. A fire truck is a highly specialized vehicle designed to move very quickly and pour out huge volumes of water. A DMP is a highly specialized system that quickly moves huge volumes of data.

• CRM System = Taxi. A taxi carries one person and their baggage directly to a specific destination. A CRM system delivers one customer and their history to a single sales person or call center agent.

• Cloud Platform = Car Carrier. A car carrier is a frame that can hold many unconnected vehicles. A Cloud Platform supports many unrelated systems in the same rack.

Believe it or not, my original list didn’t include an entry for Customer Data Platforms. So let’s give that a moment’s thought and go with…Ice Cream Truck!  They make a lot of noise and everybody loves them.  No, wait, better still...Food Truck!  A food truck collects ingredients from various sources, converts them into delicious meals, and distributes the results to many happy customers.  A CDP combines customer data into profiles that it shares with different systems.  

You’re welcome.

Happy New Year!

Adobe, Microsoft and SAP Announce Open Data Initiative: It's CDP Turf But No Immediate Threat

One of the more jarring aspects in Adobe’s briefing last week about its Marketo acquisition* were several statements that suggested Marketo and Adobe’s other products were going to access shared customer data. This would be the Experience Cloud Profile announced  in March and based on an open source data model developed jointly with Microsoft and stored on Microsoft Azure.**  When I tried to reconcile Adobe’s statements with reality, the best I could come up with was they were saying that Adobe systems and Marketo would push their data into the Experience Cloud Profiles and then synchronize whatever bits they found useful with each application’s data store. That’s not the same as replacing the separate data stores with direct access to the shared Azure files but it is sharing of a sort. Whether even that level of integration is available today is unclear but if we required every software vendor to only describe capabilities that are actually in place, the silence would be deafening.

The reason the shared Microsoft project was on Adobe managers’ minds became clear today when Adobe, Microsoft and SAP announced an “Open Data Initiative” that seemed pretty much the same news as before – open source data models (for customers and other objects) feeding a system hosted on Azure. The only thing really seemed new was SAP’s involvement. And, as became clear during analyst questions after the announcement at Microsoft’s Ignite conference, this is all in very early stages of planning.

I’ll admit to some pleasure that these firms have finally admitted the need for unified customer data, a topic close to my heart. Their approach – creating a persistent, standardized repository – is very much the one I’ve been advocating under the Customer Data Platform label. I’ll also admit to some initial fear that a solution from these vendors will reduce the need for stand-alone CDP systems. After all, stand-alone CDP vendors exist because enterprise software companies including Microsoft, Adobe and SAP have left a major need unfilled.

But in reviewing the published materials and listening to the vendors, it’s clear that their project is in very early stages. What they said on the analyst call is that engineering teams have just started to work on reconciling their separate data models – which is heart of the matter. They didn’t put a time frame on the task but I suspect we’re talking more than a year to get anything even remotely complete. Nor, although the vendors indicated this is a high strategic priority, would I be surprised if they eventually fail to produce something workable.  That could mean they produce something, but it’s so complicated and exception-riddled that it doesn’t meet the fundamental goal of creating truly standardized data.

Why I think this could happen is that enterprise-level customer data is very complicated.  Each of these vendors has multiple systems with data models that are highly tuned to specific purposes and are still typically customized or supplemented with custom objects during implementation. It’s easy to decide there’s an entity called “customer” but hard to agree on one definition that will apply across all channels and back-office processes. In practice, different systems have different definitions that suit their particular needs.

Reconciling these is the main challenge in any data integration project.  Within a single company, the solution involves detailed, technical discussions among the managers of different systems. Trying to find a general solution that applies across hundreds of enterprises may well be impossible. In practice, you’re likely to end up with data models that support different definitions in different circumstances with some mechanism to specify which definition is being used in each situation. That may be so confusing that it defeats the purpose of having shared data, which is for different people to easily make use of it.

Note that CDPs are deployed at the company level, so they don’t need to solve the multi-company problem.*** This is one reason I suspect the Adobe/Microsoft/SAP project doesn’t pose much of a threat to the current CDP vendors, at least so long as buyers actually look at the details rather than just assuming the big companies have solved the problem because they’ve announced they're working on it.

The other interesting aspect of the joint announcement was its IT- rather than marketing-centric focus. All three of the supporting quotes in the press release came from CIOs, which tells you who the vendors see as partners. Nothing wrong with that: one of trends I see in the CDP market is a separation between CDPs that focus primarily on data management (and enterprise-wide use cases and IT departments as primary users) and those that incorporate marketing applications (and marketing use cases and marketers as users). As you may recall, we recently changed the CDP Institute definition of CDP from “marketer-controlled” to “packaged software” to reflect the use of customer data across the enterprise. But most growth in the CDP industry is coming from the marketing-oriented systems. The Open Data Initiative may eventually make life harder for the enterprise-oriented CDPs, although I’m sure they would argue it will help by bringing attention to a problem that it doesn’t really solve, opening the way to sales of their products.  It’s even less likely to impact sales of the marketing-oriented CDPs, which are bought by marketing departments who want tightly integrated marketing applications.

Another indication of the mindset underlying the Open Data Initiative is this more detailed discussion of their approach, from Adobe’s VP of Platform Engineering. Here the discussion is mostly about making the data available for analysis. The exact quote “to give data scientists the speed and flexibility they need to deliver personalized experiences” will annoy marketers everywhere, who know that data scientists are not responsible for experience design, let alone delivery. Although the same post does mention supporting real-time customer experiences, it’s pretty clear from context that the core data repository is a data lake to be used for analysis, not a database to be accessed directly during real-time interactions. Again, nothing wrong with that and not all CDPs are designed for real-time interactions, either. But many are and the capability is essential for many marketing use cases.

In sum: today’s announcement is important as a sign that enterprise software vendors are (finally) recognizing that their clients need unified customer data. But it’s early days for the initiative, which may not deliver on its promises and may not promise what marketers actually want or need. It will no doubt add more confusion to an already confused customer data management landscape. But smart marketers and IT departments will emerge from the confusion with a sound understanding of their requirements and systems that meet them. So it's clearly a step in the right direction.




__________________________________
*I didn't bother to comment the Marketo acquisition in detail because, let’s face it, the world didn’t need one more analysis. But now that I’ve had a few days to reflect, I really think it was a bad idea. Not because Marketo is a bad product or it doesn’t fill a big gap in the Adobe product line (B2B marketing automation).  It's because filling that gap won’t do Adobe much good. Their creative and Web analysis products already gave them a presence in every marketing department worth considering, so Marketo won’t open many new doors. And without a CRM product to sell against Salesforce, Adobe still won’t be able to position itself as a Salesforce replacement. So all they bought for $4.75 billion was the privilege of selling a marginally profitable product to their existing customers. Still worse, that product is in a highly competitive space where growth has slowed and the old marketing automation approach (complex, segment-based multi-step campaign flows) may soon be obsolete. If Adobe thinks they’ll use Marketo to penetrate small and mid-size accounts, they are ignoring how price-sensitive, quality-insensitive, support-intensive, and change-resistant those buyers are. And if they think they’ll sell a lot of add-on products to Marketo customers, I’d love to know what those would be.

** I wish Microsoft would just buy Adobe already. They’re like a couple that’s been together for years and had kids but refuses to get married.

*** Being packaged software, CDPs let users implement solutions via configuration rather than custom development. This is why they’re more efficient than custom-built data warehouses or data lakes for company-level projects.

What's Next for Customer Data Platforms? New Report Offers Some Clues.

The Customer Data Platform Institute released its semi-annual Industry Update today. (Download it here).  It’s the third edition of this report, which means we now can look at trends over time. The two dozen vendors in the original report have grown about 25% when measured by employee counts in LinkedIn, which is certainly healthy although not the sort of hyper growth expected from an early stage industry. On the other hand, the report has added two dozen more vendors, which means the measured industry size has doubled. Total employee counts have doubled too. Since many of the new vendors were outside the U.S., LinkedIn probably misses a good portion of their employees, meaning actual growth was higher still.

The tricky thing about this report is that the added vendors aren’t necessarily new companies. Only half were founded in 2014 or later, which might mean they’ve just launched their products after several years of development. The rest are older. Some of these have always been CDPs but just recently came to our attention. This is especially true of companies from outside the U.S. But most of the older firms started as something else and reinvented themselves as CDPs, either through product enhancements or simply by adopting the CDP label.

Ultimately it’s up to the report author (that would be me) to decide which firms qualify for inclusion.   I’ve done my best to list only products that actually meet the CDP definition.*   But I do  give the benefit of the doubt to companies that adopted the label. After all, there’s some value in letting the market itself decide what’s included in the category.

What’s most striking about the newly-listed firms is they are much more weighted towards customer engagement systems than the original set of vendors. Of the original two dozen vendors, eleven focused primarily on building the CDP database, while another six combined database building with analytics such as attribution or segmentation. Only the remaining seven offered customer engagement functions such as personalization, message selection, or campaign management. That’s 29%.**

By contrast, 18 of the 28 added vendors offer customer engagement – that’s 64%. It’s a huge switch. The added firms aren’t noticeably younger than the original vendors, so this doesn’t mean there’s a new generation of engagement-oriented CDPs crowding out older, data-oriented systems. But it does mean that more engagement-oriented firms are identifying themselves as CDPs and adding CDP features as needed to support their positioning. So I think we can legitimately view this as validation that CDPs offer something that marketers recognize they need.

What we don’t know is whether engagement-oriented CDPs will ultimately come to dominate the industry. Certainly they occupy a growing share. But the data- and analysis-oriented firms still account for more than half of the listed vendors (52%) and even higher proportions of employees (57%), new funding (61%) and total funding (74%).  So it’s far from clear that the majority of marketers will pick a CDP that includes engagement functions.

So far, my general observation has been that engagement-oriented CDPs appeal more to mid-size firms while data and analysis oriented CDPs appeal most to large enterprises. I think the reason is that large enterprises already have good engagement systems or prefer to buy such systems separately. Smaller firms are more likely to want to replace their engagement systems at the same time they add a CDP and want to tie the CDP directly to profit-generating engagement functions. Smaller firms are also more sensitive to integration costs, although those should be fairly small when CDPs are concerned.

There’s nothing in the report to support or refute this view, since it doesn’t tell us anything about the numbers or sizes of CDP clients. But assuming it’s correct, we can expect engagement-oriented vendors to increase their share as more mid-size companies buy CDPs. We can also expect engagement-oriented systems to be more common outside the U.S., where companies are generally smaller. For what it’s worth, the report does confirm that’s already the case.

If the market does move towards engagement-oriented systems, will the current data and analytics CDPs add those features? That’s another unknown. There’s already been some movement: four of the original eleven data-only CDPs have added analytics features over the past year.  But it’s a much bigger jump to add customer engagement features, and sophisticated clients won’t accept a stripped-down engagement system. We might see some acquisitions if the large data and analytics vendors want to add those features quickly. But those firms must also be careful about competing with the engagement vendors they currently connect with. Nor are they necessarily eager to lose their differentiation from the big marketing clouds.  Nor is there much attraction to entering the most crowded segment of the market with a me-too product.

So most data and analytics vendors may well limit their themselves to their current scope and invest instead in improving their data and analytics functions. That will limit them to the upper end of the market but it's where they sell now and offers plenty of room for growth.  Certainly there’s a great deal of room for improved machine learning, attribution, scalability, speed, and automated data management. If I had to bet, I’d expect most data and analytics vendors to focus on those areas.

But I don’t have to bet and neither do you. So we’ll just wait to see what comes next. It will surely be interesting.


_________________________________________________________________________
*CDP is defined as a marketer-controlled system that builds a persistent, unified customer database that is accessible by other systems.

**To further clarify, customer engagement systems select messages for individuals or segments.  Analytics systems may create segments but don't decide which messages go to which segment.  And execution systems, such as email engines, Web content management, or mobile app platforms, deliver the selected messages. 

Wondering How Customer Data Platforms Relate to Other Marketing Systems? Here's a Picture

I was asked the other day about the distinction between Customer Data Platforms and Journey Orchestration Engines. My immediate answer was “Some CDPs are JOEs and some JOEs are CDPs. A CDP is a JOE if has journey orchestration. A JOE is a CDP if its data is accessible to other systems. Think Venn diagram with two intersecting circles.”  It's not clear the answer helped, but it did get me thinking about clarifying with a Venn diagram.  The diagram I originally had in mind was this one, showing that CDPs unify customer data and make it available, while JOEs unify customer data and select messages. Systems that do all three are both a CDP and a JOE.

On reflection, that’s not the right way to draw a Venn diagram. Each circle should represent one set of traits. So the picture should really look like this:

That's fine, but it seems odd that “unify customer data” has no system associated with it. Is there a type of system that just unifies customer data without making it accessible or selecting messages? Come to think of it, there is.  Systems that just do customer matching used to be called Customer Data Integration but I don’t hear that much any more.  Sometimes people talk about Identity Resolution but mostly it seems that Customer Data Integration has been absorbed by the larger category of Master Data Management (MDM) systems, which integrate all kinds of data. So let’s add MDM as the label for that.    

But why stop there?  Let's see how other systems would fit into the diagram. First to come to mind was marketing automation platforms (MAPs), which also select messages (like a JOE) but don’t build a unified customer database or offer open data access. The diagram with MAPs included looks like this:

The next is a Data Lake. It provides open data access like a CDP, but doesn’t build a unified view of the data.  Adding that to the diagram gives us:

Hmm, what about CRM? In many ways its out there with MAP: another system that selects messages but doesn’t build a unified database. So we need to introduce a new split, of marketer-controlled vs. sales controlled. I'll give control a different color for clarity.  Apologies to CRM people that your circle is so tiny; I'm not suggesting anything about the importance of your systems.

Still thinking about control, Data Management Platforms (DMPs) look a lot like Marketing Automation Platforms: they’re marketer-controlled systems that select messages (sort of) but don’t unify data from all sources or provide open access. So unless we want to further subdivide the marketer controlled space, they share the same location as MAPs.

Since control has its own color, Data Lake and MDM jump out as not having an owner. In fact, they’re both typically owned by corporate IT, so we can easily add that circle.

This raises one more question: is there an IT-controlled equivalent of a CDP?  That would be a system that unifies customer data and provides open access but is owned by IT not marketing.  You betcha.  It might be an Enterprise Data Warehouse (EDW) if that has all the access features of a CDP (high speed, flexibility, etc.). But most EDWs don’t meet that standard. So let’s just call it an Enterprise-controlled CDP, or ECDP, if you’re wild and crazy enough to accept a four letter acronym. You’ll remember there’s some debate about whether marketing or IT should really own the CDP.  This doesn't provide an answer but it does give a clearer picture of the question.

I've summarized this information in a table below.  Still confused?  We just posted a answers to Frequently Asked CDP Questions on the CDP Institute blog.  Maybe that will help.