Thursday, September 06, 2018

Customer Data Platforms vs Master Data Management: How They Differ

My wanderings through the Customer Data Platform landscape have increasingly led towards the adjacent realm of Master Data Management (MDM). Many people are starting to ask whether they’re really the same thing or could at least be used for some of the same purposes.

Master Data Management can be loosely defined as maintaining and distributing consistent information about core business entities such as people, products, and locations. (Start here
if you’d like to explore more formal definitions.) Since customers are one of the most important core entities, it clearly overlaps with CDP.

Specifically, MDM and CDP both require identity resolution (linking all identifiers that apply to a particular individual), which enables CDPs to bring together customer data into a comprehensive unified profile. In fact some CDPs rely on MDM products to perform this function.

MDM and (some) CDP systems also create a “golden record” containing the business’s best guess at customer attributes such as name and address. That’s the “master” part of MDM.  It often requires choosing between conflicting information captured by different systems or by the same system at different times. CDP and MDM both share that golden record with other systems to ensure consistency.

So how do CDP and MDM differ? The obvious answer is that CDP manages a lot more than just master data: it captures all the details of transactions and behaviors that are not themselves customer attributes. But many MDM products are components of larger data integration suites from IBM, SAP, Oracle, SAS, Informatica, Talend and others. These also manage more than the identifying attributes of core data objects. You could argue that this is a bit of a bait-and-switch: the CDP-like features in these suites are not part of their MDM products. But it does mean that the vendors may be able to meet CDP data requirements, even if you need more than their MDM module to do it.

Another likely differentiator is that MDM systems run on SQL databases and work with structured data. This is the best way to manage standardized entity attributes.  By contrast, CDPs work with structured, semi-structured and unstructured data which requires a NoSQL file system like Hadoop. But, again, the larger integration suites often support semi-structured and unstructured data and NoSQL databases.  So the boundary remains blurry.

On the practical level, MDMs are primarily tools that IT departments buy to improve data quality and consistency.  Business user interfaces are typically limited to specialized data governance and workflow functions. CDPs are designed to be managed by business users although deploying them does take some technical work. Marketing departments are the main CDP buyers and users while MDM is clearly owned by IT.

One CDP vendor recently told me the main distinction they saw was that MDM takes a very rigid approach to identity data, creating a master ID that all connected systems are required to use as the primary customer ID. He contrasted this with the CDP approach that lets each application work with its own IDs and only unifies the data within the CDP itself.  He also argued that some CDPs (including his, of course) let users apply different matching rules for different purposes, applying more stringent matches in some cases and looser matches in others. I’m not sure that all MDM systems are really this rigid.  But it’s something to explore if you’re assessing how an MDM might work in your environment.

Going back to practical differences, most CDPs have standard connectors for common marketing data sources, analysis tools, and execution systems. Those connectors are tuned to ingest complete data streams, not the handful of entity attributes needed for master data management.  There are certainly exceptions to this among CDPs: indeed, CDPs that focus on analytics and personalization are frequently used in combination with other CDPs that specialize in data collection. MDM vendors are less marketing-centric than CDPs so you’ll find fewer marketing-specific connectors and data models. Similarly, most MDMs are not designed to store, expose, reformat, or deliver complete data sets. But, again, MDMs are often part of larger integration suites that do offer these capabilities.

So, where does this leave a weary explorer of the CDP jungle? On one hand, MDM in itself is very different from CDP: it provides identity resolution and shares standard (“golden”) customer attributes, but doesn’t ingest, store or expose full details for all data types.  On the other hand, many MDM products are part of larger suites that do have these capabilities.

The real differentiator is focus: CDPs are built exclusively for customer data, are packaged software built for business users (mostly marketers), and have standard connectors for customer-related systems. MDM is a general-purpose tool designed as a component in systems built and run by IT departments.

Those differences won’t necessarily show up on paper, where both types of systems will check the boxes on most capabilities lists. But they’ll be clear enough as you work through the details of use cases and deployment plans.

As any good explorer will tell you, there’s no substitute for seeing the ground on foot.

Monday, September 03, 2018

Third Party Data Is Not Dead Yet

Third party data is not dead yet.

It was supposed to be. The culprit was to be the EU’s General Data Protection Regulation, which would cut off the flow of personal data to third party brokers and, even more devastatingly, prevent marketers from buying third party data for fear it wasn’t legitimately sourced. 

The expectations are real.  A recent Sizmek study found that 77% of marketers predicted data regulations such as GDPR would make targeting audiences with third party data increasingly difficult.  In a Demandbase study, 60% of respondents said that GDPR was forcing a change in their global privacy approach.  And 44% of marketers told Trusted Media Brands  that they expected GDPR would lead to more use of first party data vs. cookies.

Marketers say they're acting on these concerns by cutting back on use of third party data. Duke Fuqua’s most recent CMO Survey found that use of online (first party) customer data has grown at 63% of companies in the past two years while just 31% expanded use of third party data.  Seventy percent expected to further grow first party data in the next two years compared with just 31% for third party data.  A Dentsu Ageis survey had similar results: 57% of CMOs were expanding use of existing (first party) data compared with 37% expanding use of purchased data.

The irony is that reports of GDPR impact seem to have been greatly exaggerated. A Reuters Institute study found 22% fewer third party cookies on European news sites after GDPR deployment, a significant drop which nevertheless means that 78% remain.  Meanwhile, Quantcast reported that clients using its consent manager achieved a consent rate above 90%.  In other words, third party data is still flowing freely even in Europe even if the volume is down a little. The flow is even freer in the U.S., where developments like the new California privacy regulation will almost surely be watered down before taking effect, if not blocked entirely by Federal pre-emption.

Of course, what regulation can’t achieve, self-interest could still make happen. There’s at least some debate (stoked by interested parties) over whether targeting ads with third party data is really more effective than contextual targeting, which is the latest jargon for putting ads on Web pages related to the product. Online ad agency Roast and ad platform Teads did an exhaustive study that concluded contextual targeting and demographic targeting with third party data worked about equally well. The previously-mentioned Sizmek study found that 87% of marketers plan to increase their contextual targeting in the next year and 85% say brand safety is a high or critical priority. (Ads appearing on brand-unsafe Web pages is a problem when ads are targeted at individuals, a primary use for third party data.)  The Trusted Media Brands study also listed brand safety as a major concern about digital media buying (ranked third and cited by 58%) although, tellingly, ROI and viewability were higher (first and second at 62% and 59%, respectively).

But third party data isn’t going away.

It’s become increasingly central for business marketers as Account Based Marketing puts a premium on understanding potential buyers whether or not they're already in the company’s own database.  Third party data also includes intent information based on behaviors beyond the company’s own Web site. Indeed companies including Lattice Engines, Radius, 6Sense and Demandbase have all shifted much of their positioning away from predictive modeling or ad targeting based on internal data and towards the value of the data they bring.

Then again, business marketing always relied heavily on third party data. What arguably more surprising is that consumer marketers also seems to be using it more.  Remember that the CMO surveys cited earlier showed expectations for slower growth, not actual declines.  There's more evidence in the steady stream of vendor announcements touting third party data applications.

Many of these announcements are from established vendors selling established applications, such as ad targeting and marketing performance measurements. For targeting, see recent announcements from TruSignal, Thunder, and AdTheorent; for attribution, see news from Viant and  IRI.

But what's most interesting are the newer applications. These go beyond lists of target customers or comparing anonymized online and offline data. They provide something that only third party data can do at scale: connect online and offline identities. This is something that companies like LiveRamp and Neustar have done for years.  But we're now seeing many interesting new players:

Bridg helps retailers to identify previously anonymous in-store customers, based on probabilistic matching against their proprietary consumer database.  It then executes tailored online marketing campaigns.

SheerID verifies the identities of online visitors, enabling marketers to safely limit offers to members of specific groups such as teachers, students, or military veterans. They do this by building connections to reference databases holding identity details..

PebblePost links previously anonymous Web visitors to postal addresses, using yet another proprietary database to make the connections. They use this to target direct mail based on Web behaviors.

You’ll have noticed that the common denominator here is a unique consumer database.  These do something not available from other third party sources or not available with the same coverage.  Products like these will keep marketers coming back for third party data whether or not privacy regulations make Web-based data gathering more difficult.  So don't cry for third party data: the truth is it never has left you.








Tuesday, August 21, 2018

BadTech Is the Next New Thing

Forget about Martech, Adtech, or even Madtech. The next big thing is BadTech.

I’m referring to is the backlash against big tech firms – Google, Amazon, Apple, and above all Facebook – that have relentlessly expanded their influence on everyday life. Until recently, these firms were mostly seen as positive, or at least benignly neutral, forces that made consumers’ lives easier.  But something snapped after the Cambridge Analytica scandal last March.  Scattered concerns became a flood of hostility.  Enthusiasm curdled into skepticism and fear.  The world recognized a new avatar of evil: BadTech.

As a long-standing skeptic (see this from 2016), I’m generally pleased with this development. The past month alone offers plenty of news to alarm consumers:
There's more bad news for marketers and other business people:
Not surprisingly, consumers, businesses, and governments have reacted with new skepticism, concern, and even some action:
But all is not perfect.
  • BadTech firms still plunge ahead with dangerous projects. For example, despite the clear and increasing dangers from poorly controlled AI, it’s being distributed more broadly by Ebay, Salesforce, Google, and Oracle
  • Other institutions merrily pursue their own questionable ideas. Here we have General Motors and Shell opening new risks by connecting cars to gas pumps.  Here – this is not a joke – a university is putting school-controlled Amazon Echo listening devices in every dorm room
  • The press continues to get it wrong. This New York Times Magazine piece presents California’s privacy law as a triumph for its citizen-activist sponsor, when he in fact traded a nearly-impossible-to change referendum for a law that will surely be gutted before it takes effect in 2020.
  • Proponents will overreach. This opinion piece argues the term “privacy policy” should be banned because consumers think the label means a company keeps their data private. This is a side issue at best; at worst, it tries to protect people from being lazy. Balancing privacy against other legitimate concerns will be hard enough without silly distractions.
So welcome to our latest brave new world, where BadTech is one more villain to fear   It's progress that people recognize the issues but we can't let emotion overwhelm considered solutions.  Let’s use the moment to address the real problems without creating new ones or throwing away what’s genuinely good.  We can't afford to fail.

Saturday, August 18, 2018

CDP Myths vs Realities

A few weeks ago, I critiqued several articles that attacked “myths” about Customer Data Platforms. But, on reflection, those authors had it right: it’s important to address misunderstandings that have grown as the category gains exposure. So here's my own list of CDP myths and realities. 

Myth: CDPs are all the same.
Reality: CDPs vary widely. In fact, most observers recognize this variation and quite a few consider it a failing. So perhaps the real myth is that CDPs should be the same. It’s true that the variation causes confusion and means buyers must work hard to ensure they purchase a system that fits their needs. But buyers need to match systems to their needs in every category, including those where features are mostly similar.

Myth: CDPs have no shared features.
Reality: This is the opposite of the previous myth but grows from the same underlying complaint about CDP variation. It’s also false: CDPs all do share core characteristics. They’re packaged software; they ingest and retain detailed data from all sources; they combine this data into a complete view of each customer; they update this view over time; and they expose the view to other systems. This list excludes many products from the CDP category that share some but not all of these features. But it doesn’t exclude products that share all these features and add some other ones. These additional features, such as segmentation, data analysis, predictive models, and message selection, account for most of the variation among CDP systems. Complaining that these mean CDPs are not a coherent category is like complaining that automobiles are not a category because they have different engine types, body styles, driving performance, and seating capacities. Those differences make them suitable for different purposes but they still share the same core features that distinguish a car from a truck, tractor, or airplane.

Myth: CDP is a new technology.
Reality: CDPs use modern technologies, such as NoSQL databases and API connectors. But so do other systems. What’s different about CDP is that it combines those technologies in prebuilt systems, rather than requiring technical experts to assemble them from scratch. Having packaged software to build a unified, sharable customer database is precisely the change that led to naming CDP as a distinct category in 2013.

Myth: CDPs don’t need IT support.
Reality: They sure do, but not as much. At a minimum, CDPs need corporate IT to provide access to corporate systems to acquire data and to read the CDP database. In practice, corporate IT is also often involved in managing the CDP itself. (This recent Relevancy Group study put corporate IT participation at 49%.)   But the packaged nature of CDPs means they take less technical effort to maintain than custom systems and many CDPs provide interfaces that empower business users to do more for themselves. Some CDP vendors have set their goal as complete business user self-service but I haven’t seen anyone deliver on this and suspect they never will.

Myth: CDPs are for marketing only.
Reality: It’s clear that departments outside of marketing can benefit from unified customer data and there’s nothing inherent in CDP technology that limits them to marketing applications. But it’s also true that most CDPs so far have been purchased by marketers and have been connected primarily to marketing systems. The optional features mentioned previously – segmentation, analytics, message selection, etc. – are often marketing-specific. But CDPs with those features must still be able to share their data outside of marketing or they wouldn’t be CDPs.

Myth: CDPs manage only first party, identified data.
Reality: First party, identified data is the primary type of information stored in a CDP and it’s something that other systems (notably Data Management Platforms) often handle poorly or not at all. But nothing prevents a CDP from storing third party and/or anonymous data, and some CDPs certainly do.  Indeed, CDPs commonly store anonymous first party data, such as Web site visitor profiles, which will later be converted into identified data when a customer reveals herself. The kernel of truth inside this myth is that few companies would use a CDP to store anonymous, third party data by itself.

Myth: Identity resolution is a core CDP capability.
Reality: Many CDP systems provide built-in identity resolution (i.e., ability to link different identifiers that relate to the same person).  But many others do not.  This is by far the most counter-intuitive CDP reality, since it seems obvious that a system which builds a unified customer profiles should be able to connect data from different sources.  But quite a few CDP buyers don’t need this feature, either because they get data from a single source system (e.g., ecommerce or publishing), because their company has existing systems to assemble identities (common in financial services), or because they rely on external matching systems (frequent in retail and business marketing). What nearly all CDPs do have is the ability to retain links over time, so unified profiles can be stitched together as new identifiers are connected to each customer’s master ID. One way to think about this is: the function of identity resolution is essential for building a unified customer database, but the feature may be part of a CDP or something else.

Myth: CDPs are not needed if there’s an Enterprise Data Warehouse.
Reality: It’s a reasonable simplification to describe a CDP as packaged software that builds a customer-centric Data Warehouse. But a Data Warehouse is almost always limited to highly structured data stored in a relational database.  CDPs typically include large amounts of semi-structured and unstructured data in a NoSQL data store. Relational technology means changing a Data Warehouse is usually a complex, time-consuming project requiring advanced technical skill. Pushing data into a CDP is much easier, although some additional work may later be required to make it usable. Even companies with an existing Data Warehouse often find a CDP offers new capabilities, flexibility, and lower operating costs that make it a worthwhile investment.

Myth: CDPs replace application data stores.
Reality: Mea culpa: I’ve often explained CDPs by showing separate silo databases replaced by a single shared CDP.  But that’s an oversimplification to get across the concept. There are a handful of situations where a delivery system will read CDP data directly, such as injecting CDP-selected messages into a Web page or exposing customer profile details to a call center agent. But in most cases the CDP will synchronize its data with the delivery system’s existing database. This is inevitable: the delivery systems are tightly integrated products with databases optimized for their purpose. The value of the CDP comes from feeding better data into the delivery system database, not from replacing it altogether.


Myth: CDP value depends on connecting all systems.
Reality: CDPs can deliver great value if they connect just some systems, or sometimes even if they only expose data from a single system that was otherwise inaccessible.  This matters because connecting all of a company's systems can be a huge project or even impossible if some systems are not built to integrate with others.  This shouldn't be used as an argument against CDP deployment so long as a less comprehensive implementation will still provide real value.

Myth: The purpose of CDP is to coordinate customer experience across all channels.
Reality: That's one goal and perhaps the ultimate.  But there are many other, simpler applications a CDP makes possible, such as better analytics and more accurate data shared with delivery systems.   In practice, most CDP users will start with these simpler applications and add the more demanding ones over time.

Myth: The CDP is a silver bullet that solves all customer data problems.
Reality: There are plenty of problems beyond the CDP's control, such as the quality of input data and limits on execution systems.  Moreover, the CDP is just a technology and many obstacles are organizational and procedural, such as cooperation between departments, staff skills, regulatory constraints, and reward systems.  What a CDP will do is expose some obstacles that were formerly hidden by the technical difficulty of attempting the tasks they obstruct.  Identifying the problems isn't a solution but it's a first step towards finding one.

Of course, everyone knows there are no silver bullets but there's always that tiny spark of hope that one will appear.  I hesitate to quench that spark because it's one of the reasons people try new things, CDPs included.  But I think the idea of CDPs is now well enough established for marketers to absorb a more nuanced view of how they work without losing sight of their fundamental value.  Gradual deflation of expectations is preferable to a sudden collapse.  Let's hope a more realistic understanding of CDPs will ultimately lead to better results for everyone involved.

Thursday, August 02, 2018

Arm Ltd. Buys Treasure Data CDP

Customer Data Platform vendor Treasure Data today confirmed earlier reports that it is being purchased by Arm Limited, which licenses semi-conductor technologies and is itself a subsidiary of the giant tech holding company SoftBank. The price was not announced but was said to be around $600 million.

The deal was the second big purchase of a Customer Data Platform vendor in a month, following the Salesforce’s Datorama acquisition. Arm seems a less likely CDP buyer than Salesforce but made clear their goal is to use Treasure Data o manage Internet of Things data. That’s an excellent fit for Treasure Data’s technology, which is very good at handling large volumes of semi-structured data. Treasure Data will operate as a separate business under its existing management and will continue to sell its product to marketers as a conventional Customer Data Platform.

While Arm is an unexpected CDP buyer, the deal does illustrate some larger trends in the CDP market. One is the broadening of CDP beyond pure marketing use cases: as critics have noted, unified customer data has applications throughout an organization so it doesn’t make sense to limit CDP to marketing users. In fact, the time has probably come to remove “marketer-managed” from the formal definition of CDP.  But that’s a topic for another blog post.

A complementary trend is use of CDP technology for non-customer data. Internet of Things is obviously of growing importance and, although you might argue thats IoT data is really just another type of customer data, there’s a reasonable case that the sheer volume and complexity of IoT data rightly justifies considering it a category of its own. More broadly, there are other kinds of data, such as product and location information, which also should be considered in their own terms.

What’s really going on here is that one category of CDPs – the systems that focus primarily on data management, as opposed to marketing applications – is merging with general enterprise data management systems. These are companies like Qubole and Trifacta that often use AI to simplify the process of assembling enterprise data.  These systems do for all sorts of information what a CDP does for customer information. This is a new source of competition for CDPs, especially as corporate IT departments get more involved. There are also a handful of CDP systems, including ActionIQ, Aginity, Amperity, and Reltio, that have the potential to expand beyond customer information. It’s possible that those vendors will eventually exit the CDP category altogether, leaving the field to CDPs that provide marketing-specific functions for analysis and customer engagement. (If that happens, then “marketer-managed” should stay in the definition.)

In any case, the Treasure Data acquisition is another milestone in the evolution of the CDP industry, illustrating that at least some of the systems have unique technology that is worth buying at a premium. I can imagine some of the other data-oriented vendors being purchased for similar reasons. I can also imagine acquisition of companies like Segment and Tealium that have particularly strong collections of connectors to source and target systems. That’s another type of asset that’s hard to replicate.

So we'll see how the industry evolves.  Don't be surprised if it follows several paths simultaneously: some buyers may take an enterprise-wide approach while others limit CDP use to marketing. What I don't yet see is any type of consolidation around a handful of winners who gobble up most of the market share.  That might still happen but, for now, the industry will remain vibrant and varied, as different vendors try different configurations to see which most marketers find appealing.

Wednesday, August 01, 2018

Salesforce Buys Datorama Customer Data Platform: It's Complicated

News that Salesforce had purchased Datorama crossed the wire just as I was starting on two weeks of travel, so I haven’t been able to comment until now. This was purchase was noteworthy as the first big CDP acquisition by a marketing cloud vendor. That the buyer was Salesforce was even more intriguing, given that they had purchased Mulesoft in March for $6.5 billion and that Marketing Cloud CEO Bob Stutz (who announced the Datorama deal) had called CDPs “a passing fad” and said Salesforce already had “all the pieces of a CDP” in an interview in June.

The Salesforce announcement didn’t refer to Datorama as a CDP and Datorama itself doesn’t use the term either. They do meet the requirements – packaged software building a unified, persistent customer database that’s open to other systems – but are definitely an outlier. In particular, Datorama ingests all types of marketing-related data, notably including ad campaign- and segment-level performance information as well as customer-level detail. Their stated positioning as “one centralized platform for all your marketing data and decision making” sure sounds like a CDP, but their focus has been on marketing performance, analytics, and data visualization. Before the acquisition, they told me some of their clients ingest customer-level detail but most do not. So it would appear that while Salesforce’s acquisition reflects recognition of the need for a persistent, unified marketing database (something they didn’t get with MuleSoft), they didn’t buy Datorama as a way to build a Single Customer View.

Datorama’s closest competitors are marketing analysis tools like Origami Logic and Beckon. I’ve never considered either of those CDPs because they clearly do not work with customer-level detail. Datorama competes to a lesser extent with generic business intelligence systems like Looker, Domo, Tableau, and Qlik. These traditionally have limited data integration capabilities although both Qlik and Tableau have recently purchased database building products (Podium Data and Empirical Systems, respectively), suggesting a mini-trend of its own. It’s worth noting that one of Datorama’s particular strengths is use of AI to simplify integration of new data sources. The firm’s more recent announcements have touted use of AI to find opportunities for new marketing programs.

Datorama is much larger than most other CDP vendors: it ranked third (behind Tealium and IgnitionOne) in the CDP Institute’s most recent industry report, based on number of employees found in LinkedIn. The company doesn’t release revenue figures but, assuming the 360 employees currently shown on LinkedIn generate $150,000 each, it would have a run rate of $54 million. (This is a crude guess: actual figure could easily be anywhere from $30 million to $80 million.) Sticking with the $54 million figure, the $800 million purchase price is 15x revenue, which is about what such companies cost. (Mulesoft went for 22x revenue.)  The company reports 3,000 clients, which again is a lot for a CDP but gives an average of under $20,000 per client. That’s very low for an enterprise CDP.  It reflects the fact that most of Datorama’s clients use it to analyze aggregated marketing data, not to manage customer-level details.

Seeing Datorama as more of an marketing analysis system than CDP makes it a little easier to understand why Salesforce continues to work with other CDP vendors. The Datorama announcement was followed a week later by news that Salesforce Ventures had led a $23.8 million investment in the SessionM CDP, which had announced an expanded Salesforce integration just one month earlier  SessionM builds its own database but its main strength is real-time personalization and loyalty. Salesforce in June also introduced Marketing Cloud Interaction Studio, a licensed version of the Thunderhead journey orchestration engine. Thunderhead also builds its own database and I consider it a CDP although they avoid the term, reflecting their primary focus on journey mapping and orchestration. The Salesforce announcement states explicitly that the Interaction Studio will shuffle customers between campaigns defined in the Marketing Cloud’s own Journey Builder, clarifying what any astute observer already knew: that Journey Builder is really about campaign flows, not true journey management.

So, how do all these pieces fit with each other and the rest of Salesforce Marketing Cloud? It’s possible that Salesforce will let Datorama, SessionM, and Interaction Studio independently build their own isolated databases but the disadvantages of that are obvious. It’s more likely that Salesforce will continue to argue that ExactTarget should be the central customer database, something that’s been their position so far even though every ExactTarget user I’ve ever spoken with has said it doesn’t work. The best possible outcome might be for Salesforce to use Datorama as its true CDP when a client wants one, and have it feed data into SessionM, Interaction Studio, ExactTarget, and other Marketing Cloud components as needed.  We'll see if that happens: it could evolve that way even if Salesforce doesn't intend it at the start.

Looking at this from another perspective: the combination of Datorama, SessionM, and Interaction Studio (Thunderhead) almost exactly fills every box in my standard diagram of CDP functions, which distinguishes the core data processing capabilities (ingest, process, expose) from optional analytics and engagement features.  Other Marketing Cloud components provide the Delivery capabilities that sit outside of the CDP, either directly (email and DMP) or through integrations.  The glaring gap is identity linkage, which Datorama didn't do the last time I checked.  But that's actually missing in many CDPs and often provided by third party systems.  Still, you shouldn't be too surprised to see Salesforce make another acquisition to plug that hole.  If you're wondering where Mulesoft fits, it may play a role in some of the data aggregation, indexing, reformatting, and exposing steps; I'm not clear how much of that is available in Datorama.  But Mulesoft also has functions outside of this structure.

In short, it's quite true that Salesforce has all the components of a CDP, especially when you include  Datorama in the mix.


The idea of stringing these systems together raises a general point that extends beyond Salesforce.  The reality is that almost every marketing system must import data into its own database, rather than connecting to a shared central data store. I’ll admit I’ve often drawn the picture as if there would be a direct connection between the CDP database and other applications.  This should never have been taken literally. There are indeed some situations where the CDP data is read directly, such as real time access to data about a single customer. But even those configurations usually require the CDP data to be indexed or extracted into a secondary format: absent special technology, you don’t do that sort of query directly against the primary “big data” store used by most CDPs.

Outside of those exceptions, a subset of CDP data will usually be loaded into the primary data store of the customer-facing applications (email, DMP, Web personalization, etc.). Realistically, those data stores are optimized for their own application and the applications read them directly.  There’s no practical way the applications can work without them.

This is a nuance that was rightly avoided in the early days of CDP as we struggled to explain the concept. But I think now that CDP is well enough understood that we can safely add some details to the picture to make it more realistic and avoid creating false expectations. I'll try to do that in the futre.

Friday, July 27, 2018

Get Ready for CDP Horror Stories as Customer Data Platforms Enter the Trough of Dillusionment

It’s nearly a year since Gartner placed Customer Data Platforms at the top of its “hype cycle” for digital marketing technologies. The hype cycle shouldn’t be taken too literally but it does capture the growing interest in CDPs and reminds us to expect this attention to attract critics.

Sure enough, we’ve recently started to see headlines like “Customer Data Platforms: A Contrarian’s View”, “Why Your Customer Data Platform Is a Failure”  and “CDPs: Yet Another Acronym That Lets Marketers Down”.  It's tempting to dismiss such headlines as competitive attacks or mere attempts to piggyback on wide interest in CDPs.   But we should still take a look at the underlying arguments.   After all, we might learn something.

Let’s start with the “Contrarian’s View”, written by Lisa Loftis, a customer data industry veteran who current works for SAS. She offers to debunk two common CDP “myths”: that “CDPs solve a problem unique to marketing” and that “'marketing-managed' means you don’t need IT’s help”.

Regarding the first myth, Loftis says that systems to match customer identities have been available for decades and that departments outside of marketing also need unified data. Regarding the second, she states its best for marketing and IT departments to work together given the complex technical challenges of marketing systems in general and customer data matching in particular.

She’s right.

That is, she’s right that these technologies are not new, that unified data is useful outside of marketing, and that deploying CDPs requires some technical skills. So far a I know, though, she's wrong to suggest that CDP vendors and advocates (obviously including me) claim otherwise. False belief in these myths are not the reasons marketers buy CDPs.

To put it bluntly, the problem that CDP solves isn’t the lack of technology to build unified customer databases: it’s that corporate IT departments haven’t used that technology to meet marketers’ needs. That failure has created a business opportunity that CDPs have filled. It’s the same reason that people hire private security guards when the government's police fail to maintain order.

And, just as good security guards cooperate with the police, CDP systems must integrate with corporate systems and CDP vendors must work with corporate IT.  CDP vendors have designed their systems to be easier to use than traditional customer matching and management technologies, but that only reduces the technical effort without eliminating it. The remaining technical work may be done by the CDP vendor itself, by a service provider, or even by the corporate IT group. The term “marketer-driven” in the CDP Institute’s formal CDP definition is intended to express this: marketers in control of the CDP, which isn’t the same as doing the technical work.

“Why Your CDP is a Failure” offers an even more provocative headline. But hopes for juicy disaster tales are quickly dashed: author Alan J. Porter of Simple [A] only means that CDPs “fail” because customer data should be shared by all departments. Again, no CDP vendor, buyer, or analyst would ever argue otherwise. There’s no technical reason a CDP can’t be used outside of marketing and some CDP vendors explicitly position their product as an enterprise system. The reason that CDPs are not used outside of marketing is that companies fail to fund enterprise-wide customer databases, not that CDPs can’t deliver such databases. Your CDP is a failure for this reason only if building such a database was its goal. That’s rarely the case.

“CDPs: Yet Another Acronym That Lets Marketers Down” starts with the airy assertion that “When you strip all the nonsensical nuances away from these companies -- the CRMs, the TMSs [tag management systems], the DMPs, the CDPs -- they’re all one simple thing at their cores: identity companies.”  This will be news to people who use such systems every day to run call centers, manage sales forces, capture Web site, run advertising campaigns, and assemble detailed customer histories.

The article continues qirh assertions that “identity isn’t everything”, “brands don’t have a complete understanding of their customers”, and “behaviors without motivations teach us nothing."  Few would argue with the first two while the third is surely overstated.  But the relevance of CDP to all three is questionable.  It seems that author Andy Hunn’s main message is that marketers need the combination of anonymized third party data and survey panel results offered by his own company, Resonate. This may be, but Resonate clearly serves a different purpose from CDPs.  So there's little reason to measure one in terms of the other.

Let me be clear: CDPs are not perfect. Like many new technologies, they are often expected to deliver more than is possible.   We are surely entering the “disillusion” stage of the hype cycle when tales of failed implementations and studies showing mixed satisfaction levels are common (and prove nothing about the technology's ultimate value).  Critical articles can be helpful in clarifying what CDPs do and don’t offer.  It's easy to lose sight of those boundaries in the early stages of a product category, when the main task is building a clear picture of the problems it solves, not on establishing its limits.

This is why the most productive discussion around CDPs right now revolves around use cases. Marketers (and other departments) need concrete examples of how CDPs are being used.  In particular, they need to be told what applications typically become possible when a CDP is added to a company’s marketing technology stack. These generally do one or more thing: combine data from multiple sources, share that data across channels, and rely on real-time access to the assembled data. It's these applications that justify investment in a CDP.

Complaining that CDPs don’t do other things isn’t very helpful – especially if CDP vendors don’t claim they do.  Nor is it a flaw in CDPs if other solutions can achieve the same thing.  Buyers can and should consider all alternatives to solving a problem: sometimes the CDP will be best and sometimes it won’t. It takes a clear understanding of each possibility to make the right choice.   Blanket claims about the value or failures of CDP may be inevitable but they don't really advance that discussion.