Customer Data Platforms vs Master Data Management: How They Differ

My wanderings through the Customer Data Platform landscape have increasingly led towards the adjacent realm of Master Data Management (MDM). Many people are starting to ask whether they’re really the same thing or could at least be used for some of the same purposes.

Master Data Management can be loosely defined as maintaining and distributing consistent information about core business entities such as people, products, and locations. (Start here
if you’d like to explore more formal definitions.) Since customers are one of the most important core entities, it clearly overlaps with CDP.

Specifically, MDM and CDP both require identity resolution (linking all identifiers that apply to a particular individual), which enables CDPs to bring together customer data into a comprehensive unified profile. In fact some CDPs rely on MDM products to perform this function.

MDM and (some) CDP systems also create a “golden record” containing the business’s best guess at customer attributes such as name and address. That’s the “master” part of MDM.  It often requires choosing between conflicting information captured by different systems or by the same system at different times. CDP and MDM both share that golden record with other systems to ensure consistency.

So how do CDP and MDM differ? The obvious answer is that CDP manages a lot more than just master data: it captures all the details of transactions and behaviors that are not themselves customer attributes. But many MDM products are components of larger data integration suites from IBM, SAP, Oracle, SAS, Informatica, Talend and others. These also manage more than the identifying attributes of core data objects. You could argue that this is a bit of a bait-and-switch: the CDP-like features in these suites are not part of their MDM products. But it does mean that the vendors may be able to meet CDP data requirements, even if you need more than their MDM module to do it.

Another likely differentiator is that MDM systems run on SQL databases and work with structured data. This is the best way to manage standardized entity attributes.  By contrast, CDPs work with structured, semi-structured and unstructured data which requires a NoSQL file system like Hadoop. But, again, the larger integration suites often support semi-structured and unstructured data and NoSQL databases.  So the boundary remains blurry.

On the practical level, MDMs are primarily tools that IT departments buy to improve data quality and consistency.  Business user interfaces are typically limited to specialized data governance and workflow functions. CDPs are designed to be managed by business users although deploying them does take some technical work. Marketing departments are the main CDP buyers and users while MDM is clearly owned by IT.

One CDP vendor recently told me the main distinction they saw was that MDM takes a very rigid approach to identity data, creating a master ID that all connected systems are required to use as the primary customer ID. He contrasted this with the CDP approach that lets each application work with its own IDs and only unifies the data within the CDP itself.  He also argued that some CDPs (including his, of course) let users apply different matching rules for different purposes, applying more stringent matches in some cases and looser matches in others. I’m not sure that all MDM systems are really this rigid.  But it’s something to explore if you’re assessing how an MDM might work in your environment.

Going back to practical differences, most CDPs have standard connectors for common marketing data sources, analysis tools, and execution systems. Those connectors are tuned to ingest complete data streams, not the handful of entity attributes needed for master data management.  There are certainly exceptions to this among CDPs: indeed, CDPs that focus on analytics and personalization are frequently used in combination with other CDPs that specialize in data collection. MDM vendors are less marketing-centric than CDPs so you’ll find fewer marketing-specific connectors and data models. Similarly, most MDMs are not designed to store, expose, reformat, or deliver complete data sets. But, again, MDMs are often part of larger integration suites that do offer these capabilities.

So, where does this leave a weary explorer of the CDP jungle? On one hand, MDM in itself is very different from CDP: it provides identity resolution and shares standard (“golden”) customer attributes, but doesn’t ingest, store or expose full details for all data types.  On the other hand, many MDM products are part of larger suites that do have these capabilities.

The real differentiator is focus: CDPs are built exclusively for customer data, are packaged software built for business users (mostly marketers), and have standard connectors for customer-related systems. MDM is a general-purpose tool designed as a component in systems built and run by IT departments.

Those differences won’t necessarily show up on paper, where both types of systems will check the boxes on most capabilities lists. But they’ll be clear enough as you work through the details of use cases and deployment plans.

As any good explorer will tell you, there’s no substitute for seeing the ground on foot.

Salesforce Buys MuleSoft and Offers It as a Data Unification Solutions

The Customer Data Platform industry is doing very well, thank you, with new reports out recently from both Gartner  and Forrester  and the CDP Institute launching its European branch.  But the great question hovering over the industry has been why the giant marketing cloud vendors haven’t brought out their own products and what will happen when they do. Oracle sometimes acts as if their BlueKai Data Management Platform fills the CDP role, while Adobe has made clear they don’t intend to do more than create a shared ID that can link data stored in its separate marketing applications. Salesforce has generally claimed its Marketing Cloud product (formerly ExactTarget) is a CDP, a claim that anyone with experience using the Marketing Cloud finds laughable.

The flaws in all these approaches have been so obvious that the question among people who understand the issues has been why the companies haven’t addressed them: after all, the problems must be as obvious to their product strategists as everyone else and the attention gained by CDP makes the gaps in their product offerings even more glaring. My general conclusion has been that the effort needed to rework the existing components of their clouds is too great for the vendors to consider. Remember that the big cloud vendors built their suites by purchasing separate products.  The effort to rebuild those products would be massive and would discard technology those companies spent many billions of dollars to acquire. So rationalization of their existing architectures, along with some strategic obfuscation of their weaknesses, seems the lesser evil.

We got a slightly clearer answer to the question on Tuesday when Salesforce announced a $6.5 billion purchase of Mulesoft, a data integration vendor that provides connectors between separate systems. In essence, Salesforce has adopted the Adobe approach of not pulling all customer data into a single repository, but rather connecting data that sits in its system of origin. In Salesforce’s own words, “MuleSoft will power the new Salesforce Integration Cloud, which will enable all enterprises to surface any data—regardless of where it resides—to drive deep and intelligent customer experiences throughout a personalized 1:1 journey.”

This is a distinct contrast with the CDP approach, which is to load data from source systems into a separate, unified, persistent database. The separate database has some disadvantages – in particular, it can involve replicating a lot of data – but it also has major benefits. These include storing history that may be lost in source systems, using that history to build derived data elements such as trends and aggregates, and formatting the data in ways that optimized for quick access by marketing systems and other customer-focused applications.

Although the difference between these two approaches is clear, some practical compromises can narrow the distance between them. Most CDPs can access external data in place, reducing the amount of data to be moved and allowing the system to use current versions of highly volatile information such as weather, stock prices, or product inventories. Conversely, a system like Mulesoft can push data into a persistent database as easily as it can push it to any other destination, so it can build some version of a persistent database. In fact, many CDPs that started out as tag managers have taken this approach.

But pushing data into a persistent database isn’t enough. Mulesoft and similar products work with well-defined inputs and outputs, while CDPs often can accept and store data that hasn’t been mapped to a specific schema. Even more important, I’m unaware of any meaningful ability in Mulesoft to unify customer identities, even using relatively basic approaches such as identity stitching. It’s possible to build workarounds, such as calls to external identity management systems or custom-built matching processes. Again, these are solutions employed by some CDP vendors that also lack advanced identity management. But such solutions can be costly, complex, and incomplete. From a buyer’s perspective, they are compromises at best. No one – except a salesperson – would argue they’re the ideal approach.

In short, Salesforce’s purchase of Mulesoft offers a partial solution to the needs that have driven the growth of CDPs. It’s probably the best that Salesforce could do without making the impractical investment needed to rebuild its existing marketing cloud components.  Get ready for a lot more confusion about the best way to build unified customer data. To avoid getting distracted, focus on what marketers really need and let that, not theory or vendor hype, drive your evaluation of the alternatives.

Should Customer Data Platforms Be "Marketer-Controlled"?

Thomas Wieberneit argues in a thoughtful blog post that companies need one platform for consolidated customer data, but that Customer Data Platform isn’t it because the CDP is “marketer-controlled” by definition, and thus doesn’t support other departments.

This hits a nerve. Many of the CDP vendors have told me their systems are used outside of marketing. Just last week, RedPoint unveiled a “Customer Engagement Hub”* that it defines as extending beyond marketing to all customer touchpoints. When I was recently writing a paper for B2B CDP CaliberMind, they listed sales and customer success teams along with marketing as likely buyers of their product.

As these examples suggest, CDP technology can support all customer-facing departments. It’s true that different departmental applications will probably need the CDP data to be presented slightly differently. But applications within marketing also need different formats, and it’s already standard for CDPs to reformat their data for access by external systems. So there’s no technical reason to limit CDPs to marketing.

Indeed, as I argued in my last blog post, the era of isolated marketing technology systems may be coming to an end as companies return to centralized systems to ensure a seamlessly integrated customer experience. In this world, the CDP is a shared enterprise asset, and, as such, clearly not something that marketers build and control for themselves.

So what’s holding me back from changing the definition? Three things:

• History. The fact is that CDPs evolved from systems that were built specifically for marketing. This matters because it means marketing needs drove their design. CDPs built to support sales or customer success teams would probably look a bit different, even if they used the same underlying technology. To give a concrete example, systems built for customer success don’t need to advanced identity resolution because almost all the data they care about arrives with a customer ID attached. I know that history by itself isn’t a good reason to retain the marketer-centric definition of CDP since CDPs have outgrown their origins. But it doesn’t hurt to have something remind users outside of marketing that they should look closely at whether any particular CDP can fully support their needs.

• User control. The primary reason the CDP definition includes “marketer-controlled” is to distinguish CDPs from enterprise data warehouse (or data lake) projects run by corporate IT. That matters because such projects were historically multi-year undertakings that often failed altogether or required so many compromises among enterprise stakeholders that they didn’t meet all of marketing’s particular needs. Moreover, once built, such systems were slow and costly to update, so they didn’t adapt quickly to marketers’ fast-changing environment. This responsiveness is really the biggest change that CDPs introduced into the world of customer data management. As I noted earlier, department-controlled systems may soon be lost in a new round of centralization.  In theory, these new central systems will be vastly more responsive to user needs than the old central systems. But if I were a marketer, I’d be reluctant to give up my own systems until the new ones had proven their agility in practice.

• Departmental buyers. Whatever the long-term future of centralization, CDPs today are almost always bought by departments.  Even when corporate IT groups are managing the process, they are generally reacting to requests from departmental users rather than executing an enterprise master plan of their own. And when you start looking at which departments are making these purchases, marketing is by far the leader.  It’s true that other departments often find uses for a CDP once it’s deployed. Whether marketers really share control of their CDPs when this happens or simply treat other users as guests, I can’t say. From a seller’s standpoint, I can certainly see companies offering their technology in packages aimed at enterprise buyers, which is exactly what RedPoint just did. But abandoning the large marketing-department segment for the nascent enterprise-buyer segment doesn’t sound like a good idea.

So it seems I’ve backed myself into a corner. CDP technologies work at the enterprise level and, logically, CDPs should be enterprise assets. But the main buyers of CDPs are marketing departments and the CDPs' main attraction is departmental control.  So redefining the CDP as an enterprise system would probably reduce its appeal even though it was technically accurate. Replacing "marketer-controlled" with “user-controlled” or “department-controlled” would broaden the scope but still doesn’t describe an enterprise-wide role. We could drop “control” from the definition and get to the heart of the matter, which is responsiveness – so instead of “marketer-controlled” we could have something like “agile” or, more concretely, “easily changed”. But those seem too vague to be useful.

I’m tempted to offer something truly lame, like “Enterprise CDP” as an alternative to plain or Departmental CDP.  It might come to that, or I might ultimately just drop “marketer-controlled” without replacing it. After all, if enterprise systems really do become flexible enough to meet departmental needs in a way that past enterprise data warehouse projects did not, then the distinction will no longer matter.

Bottom line: I’ll leave “marketer-controlled” in place for now but suspect its days are numbered.


________________________________________________________________

*CEH is a Gartner term, which they define as “an architectural framework that ties multiple systems together to optimally engage the customer. A CEH allows personalized, contextual customer engagement, whether through a human, artificial agent, or sensors, across all interaction channels. It reaches and connects all departments, allowing, for example, the synchronization of marketing, sales and customer service processes.” As the term “hub” implies, this definition doesn’t specify creation of a persistent database, something I believe is required to build an effective customer profile. Come to think of it, I'm not sure that "architectural framework" is a system at all. 

Can Customer Data Platforms Make Decisions? Discuss.

I’ve had at four conversations in the past twenty four hours with vendors who build a unified customer database and use it to guide customer treatments. The immediate topic has been whether they should be considered Customer Data Platforms but the underlying question is whether Customer Data Platforms should include customer management features.

That may seem pretty abstract but bear with me because this isn’t really about definitions. It’s about what systems do and how they’re built.  To clear the ground a bit, the definition of CDP, per the CDP Institute, is “a marketer-managed system that creates a persistent, unified customer database that is accessible to other systems". Other people have other definitions but they are pretty similar. You’ll note there’s nothing in that definition about doing anything with data beyond making it available.  So, no, a CDP doesn’t need to have customer management features.

But there’s nothing in the definition to prohibit those features, either. So a CDP could certainly be part of a larger system, in the same way that a motor is part of a farm tractor. But most farmers would call what they’re buying a tractor, not a motor. For the same reasons, I generally don’t to refer to systems as CDPs if their primary purpose is to deliver an application, even though they may build a unified customer database to support that application.

The boundary gets a little fuzzier when the system makes that unified database available to external systems – which, you’ll recall, is part of the CDP definition. Those systems could be used as CDPs, in exactly the same way that farm tractors have “power take off” devices that use their motor to run other machinery.  But unless you’re buying that tractor primarily as a power source, you’re still going to think of it as a tractor. The motor and power take off will simply be among the features you consider when making a choice.*

So much for definitions. The vastly more important question is SHOULD people buy "pure" CDPs or systems that contain a CDP plus applications. At the risk of overworking our poor little tractor, the answer is the same as the farmer’s: it depends it on how you’ll use it. If a particular system offers the only application you need, you can buy it without worrying about access by other applications. At the other extreme, if you have many external applications to connect, then it almost doesn’t matter whether the CDP has applications of its own. In between – which is where most people live – the integrated application is likely add value but you also want to with connect other systems. So, as a practical matter, we find that many buyers pick CDPs based on both integrated applications and external access.  From the CDP vendor’s viewpoint, this connectivity is helpful because it makes their system more important to their clients.

The tractor analogy also helps show why data-only CDPs have been sold almost exclusively to large enterprises. Those companies have many existing systems that can all benefit from a better database.  In tractor terms, they need the best motor possible for power applications and have other machines for tasks like pulling a plow. A smaller farm needs one tractor that can do many different tasks.

I may have driven the tractor metaphor into a ditch.  Regardless, the important point is that a system optimized for a single task – whether it’s sharing customer data or powering farm equipment – is designed differently from a system that’s designed to do several things. I’m not at all opposed to systems that combine customer data assembly with applications.  In fact, I think Journey Orchestration Engines (JOEs), which often combine customer data with journey orchestration, make a huge amount of sense. But most JOE databases are not designed with external access in mind.  A JOE database designed for open access would be even better -- although maybe we shouldn't call it a CDP.

To put this in my more usual terms of Data, Decision, and Delivery layers: a CDP creates a unified Data layer, while most JOEs create a unified Data and Decision layer. There’s a clear benefit to unifying decisions when our goal is a consistent customer treatment across all delivery systems. What’s less clear is the benefit of having the same system combine the data and decision functions. The combination avoids integration issues.  But it also means the buyer must use both components, even though she might prefer a different tool for one or the other.

Remember that there’s nothing inherent in JOEs that requires them to provide both layers. A JOE could have only the decision function and connect to a separate CDP. The fact that most JOEs create a database is just the matter of necessity: most companies don’t have a database in place, so the JOE must build one in order to do the fun stuff (orchestration).  Many other tools, such as B2B predictive analytics and customer success systems, create their own database for exactly the same reason. In fact, I originally classified those systems as CDPs although I’ve now narrowed my definition since the database is not their focus.

So I hope this clarifies things: CDPs can have decision functions but if decisions are the main purpose of the system, it’s confusing to call it a CDP.  And CDPs are certainly not required to have decision functions, although many do include them to give buyers a quick return on their investment. If that seems like waffling, then so be it: what matters is helping marketers to understand what they’re getting so they get what they really need.


_________________________________________________________________
*I’ll guess few of my readers are very familiar with farm tractors. Maybe the more modern analogy is powering apps with your smartphone. For the record, I did work on a farm when I was a lad, and drove a tractor.

Singing the Customer Data Platform Blues: Who's to Blame for Disjointed Customer Data?

I’m in the midst of collating data from 150 published surveys about marketing technology, a project that is fascinating and stupefying at the same time. A theme related to marketing data seems to be emerging that I didn’t expect and many marketers won’t necessarily be happy to hear.

Most surveys present a familiar tune: many marketers want unified customer data but few have it. This excerpt from an especially fine study by Econsultancy makes the case clearly although plenty of other studies show something similar.

So far so good. The gap is music to my ears, since helping marketers fill it keeps consultants like me in the business. But it inevitably raises the question of why the gap exists.

The conventional answer is it’s a technology problem. Indeed, this Experian survey makes exactly that point: the top barriers are all technology related.

And, comfortingly, marketers can sing their same old song of blaming IT for failing to deliver what they need.  For example, even though 61% of companies in this Forbes Insights survey had a central database of some sort, only 14% had fully unified, accessible data.

But something sounds a little funny. After all, doesn’t marketing now control its own fate? In this Ascend2 report, 61% of the marketing departments said they were primarily responsible for marketing data and nearly all the other marketers said they shared responsibility.

Now we hear that quavering note of uncertainty: maybe it’s marketing’s own fault? That’s something I didn’t expect. And the data seems to support it. For example, a study from Black Ink ROI found that the top barrier to success was better analytics (which implicitly requires better data) and explicitly listed data access as the third-ranked barrier.

But – and here’s the grand finale – the same study found that data integration software ranked sixth on the marketers’ shopping lists. In other words, even though marketers knew they needed better data, they weren’t planning to spend money to make it happen. That’s a sour chord indeed.

But the song isn't over.  If we listen closely, we can barely make out one final chorus: marketers won’t invest in data management technology because they don’t have the skills to use it. Or that’s what this survey from Falcon.io seems to suggest.

In its own way, that’s an upbeat ending. Expertise can be acquired through training or hiring outside experts (or possibly even mending some fences with IT). Better tools, like Customer Data Platforms, help by reducing the expertise needed. So while marketers aren't strutting towards a complete customer view with a triumphal Sousa march, there’s no need for a funeral dirge quite yet.

Reltio Makes Enterprise Data Usable, and Then Uses It

I’ve spent a lot of time recently talking to Customer Data Platform vendors, or companies that looked like they might be. One that sits right on the border is Reltio, which fits the CDP criteria* but goes beyond customer data to all types of enterprise information. That puts it more in the realm of Master Data Management, except that MDM is highly technical while Reltio is designed to be used by marketers and other business people. You might call it “self-service MDM” but that’s an oxymoron right up there with “do-it-yourself brain surgery”.

Or not. Reltio avoids the traditional complexity of MDM in part by using the Cassandra data store, which is highly scalable and can more easily add new data types and attributes than standard relational databases. Reltio works with a simple data model – or graph schema if you prefer – that captures relationships among basic objects including people, organizations, products, and places. It can work with data from multiple sources, relying on partner vendors such as SnapLogic and MuleSoft for data acquisition and Tamr, Alteryx, and Trifacta for data preparation. It has its own matching algorithms to associate related data from different sources. As for the do-it-yourself bit: well, there’s certainly some technical expertise needed to set things up, but Reltio's services team generally does the hard parts for its clients. The point is that Reltio reduces the work involved – while adding a new source to a conventional data warehouse can easily take weeks or months, Reltio says it can add a new source to an existing installation in one day.

The result is a customer profile that contains pretty much any data the company can acquire. This is where the real fun begins, because that profile is now available for analysis and applications. These can also be done in Reltio itself, using built-in machine learning and data presentation tools to provide deep views into customers and accounts, including recommendations for products and messages. A simple app might take one or two months to build; a complicated app might take three or four months. The data is also available to external systems via real-time API calls.

Reltio is a cloud service, meaning the system doesn’t run on the client’s own computers. Pricing depends on the number of users and profiles managed but not the number of sources or data volume. The company was founded in 2011 and released its product several years later. Its clients are primarily large enterprises in retail, media, and life sciences.

______________________________________________________________________
* marketer-controlled; multi-source unified persistent data; accessible to external systems

Customer Data Platforms Revisited: The Future of Marketing Data

It’s nearly two years since I introduced the concept of a Customer Data Platform, defined as a marketer-controlled system that builds a multi-source customer database and exposes it to external execution systems.  You may recall that I listed several sets of products as CDPs: B2B predictive lead scoring and customer success management; campaign management with an integrated customer database; and data management platforms to support online advertising. Systems were included only if their data (or derived data such as model scores) was available to other systems for campaigns and messaging.

All those categories have done well since my original posts on the topic. Established vendors have grown quickly and attracted funding; new vendors have joined the mix, also often with substantial funding. So I suppose I could pat myself on the back for spotting an important trend and let it go at that.

But things aren’t quite so simple. A look at the entire CDP ecosystem uncovers important patterns that are hidden when you look at individual vendors or vendor categories. Here's a summary of what I've seen.

Customer Management Functions

CDPs exist because marketers need to coordinate customer (and prospect) interactions across channels. That coordination involves three basic tasks: gathering and unifying customer data from all sources; using that data to select the best treatment for each interaction; and delivering those treatments through the appropriate channel systems. Each of those three tasks has several subtasks. These layers are illustrated by the following diagram, which includes a unified data layer – the classic CDP.

Vendor Categories

So far so good, but it’s really just theory. Things get interesting when you look for specific systems that perform the subtasks. It turns out that there are several categories of specialist systems within each subtask, each doing similar or complementary things in slightly different ways. Connecting the logical flow to actual systems is important because looking at real products tells you what the market is saying: that is, what buyers are willing to pay for and where change is concentrated.

The following table shows what I found when I did this analysis. The list of vendors in each section isn't necessarily comprehensive, especially in crowded segments like B2B marketing automation. I should also stress that I’ve only included Decision-layer vendors who also build their own database. This makes them potential CDPs and means they have many Data-layer functions. In a sublimely liberating act of inconsistency, I have NOT limited the Delivery layer to vendors who build their own database. In fact, most do not.

Investment

The right-most column on the previous table shows the level and types of investment being made in each vendor class. I haven’t collected precise details but the general patterns are pretty strong. The major observation is that current investment is heavily concentrated on the Decision layer, with interest in predictive modeling and message selection (which could also be labeled as personalization). There’s some investment on the Data layer in data gathering vendors, especially along the lines of acquisitions by big companies (Oracle/Datalogix, D&B/NetProspex, etc.). This is a general sign of maturity. Similarly, most recent investment on the Delivery layer has been acquisitions (IBM/Silverpop, Oracle/Responsys, Teradata/Appoxee, etc.), which is a sharp contrast from the heavy venture capital funding a couple of years back. Again, this shows the relative maturity of the space.

(Caveats: although it doesn’t show up in this analysis, I do still see some interesting investment in marketing automation niches such as app marketing, distributed marketing, and agency systems. I’m also increasingly intrigued at the “tag management” vendors on the Data layer (Tealium, Signal, Ensighten, etc.), which are reinventing themselves as data integration hubs. I didn’t see that one coming.)

Implications

It’s tempting to interpret these results are showing that data assembly is a solved problem, allowing marketers to invest Decision systems on the next layer down. But any marketer can tell you, and every survey I’ve seen confirms, that most companies are nowhere near having fully integrated their customer data.

What I think is really going on is that people are investing in Decision systems that build their own multi-source databases, providing both Data and Decision functions in one package. Remember that my original CDP categories included B2B predictive vendors and campaign management vendors who did exactly that. So it seems the proper way to look at things is more along the lines of the following diagram, which shows there are several different ways to solve the customer data integration challenge: you can buy a stand-alone CDP that has only data-level functions; buy a Decision system that also builds an integrated database; or buy a Delivery system that does data, decisions, and execution. As the diagram indicates, most of the Decision vendors do incorporate the CDP functions, while only a few of the Delivery vendors do.

The diagram labels the Data + Decision combination as a “Marketing Platform”.  I think this is reasonably consistent with how most people use the term, since the key feature of a “platform” is its ability to integrate with external systems for delivery and other purposes. I’ve labeled the Data + Decision + Delivery combination as an “Integrated Suite” and used question marks to show that not all suites provide a complete Data solution. This is because many suites aren’t very good at bringing in external data or letting external systems access the data they’ve assembled.

As I noted in the previous section, most of the industry funding and excitement is centered on the Decision layer, which is where the Marketing Platforms live. The practical advantage of those systems over Data-only solutions is obvious: Decision systems deliver a revenue generating application while Data-only systems do not.

But think about that for a moment.  Each Decision system builds its own multi-source database and each integrates separately with the Delivery systems.  Having multiple Decision systems is a nightmare of redundancy:

It seems pretty clear that the better solution is to have a single Decision system controlling everything, which is arguably what most people (and vendors) have in mind when they describe a Marketing Platform. Indeed, this is exactly the direction that most Decision-layer CDPs are headed, by expanding the scope of their products from an initial point solution, such as B2B lead scoring, to encompass other applications. It’s safe to say that the people who built these systems always planned, or at least hoped, to grow in this direction.

Does the growth of Decision-layer CDPs mean that Data-only CDPs will fail? I’ll admit that only a few such systems have appeared in the past two years. But I’m not quite ready to give up on the concept.

Why?  Well, as Tolstoy never said, all good customer databases look alike, but every decision system is different. This means it’s hard to support all types of decisions within a single product. So it does seem that multiple decision systems will appeal to marketers who have the skills to use them and the scale to justify the added expense. Those marketers would benefit from a Data-layer CDP, which would make it easier to deploy best-of-breed decision tools even when those tools lack data unification functions.

The stumbling block for this approach is still the cost of integrating multiple systems: as the diagram shows, there are still plenty of connections in this model. But there’s at least some hope (although I remain skeptical) that newer technologies will make the integration easier. The other bright spot for the Data-only CDPs is that they should be attractive as partners or acquisitions for Decision and Delivery systems that haven’t built their own CDP functions.


And what about the suites? I’ve said for years that the first law of software market development is “suites win”, precisely because most companies will sacrifice best of breed functionality to avoid the costs of integration. Indeed, the big marketing clouds from Oracle, Salesforce.com, IBM, Adobe, and others all include extensive Delivery layer functions. I think it’s fair to say that while their commitment to being “open platforms” is genuine, they see that as a way of letting clients supplement the core functions the suites provide internally.  This is quite different from the idea of a shared Data and Decision platform that specifically avoids offering Delivery services. Still, there’s a  very good chance that a suite which can easily integrate supplementary functions will give marketers enough freedom to overcome the problems of lock-in, while still delivering the convenience of pre-integrated core functions. So I’m not quite ready to abandon “suites win” as a rule, although I’m a bit less certain than previously.

Looking Ahead

It’s fun to handicap the horse race among vendors and categories, but what really matters is the contest itself.   All these smart people and money are finally giving marketers the unified customer databases they so desperately need.  This removes a fundamental obstacle to the cross-channel integrated marketing that everyone recognizes is increasingly important. So let’s look at the view once we've climbed that mountain.

I’d like to tell you I see a new and perfect world, but what's actually there is more mountains.  Once unified databases become available, marketers will face a new set of challenges including:

- more need for predictive models and external data. I only lump those together because they’re already getting a lot of attention. Having a powerful database just makes them even more important.

- new focus on automated content creation and campaign design. Lack of skilled users and adequate content are already huge barriers to effective multi-channel marketing.  Removing the database barrier will only make them stand out even more. So we can expect smart people to address them through technology. Indeed, there is already plenty of activity in these areas but I think it’s fair to say that so far none of vendors have had a major impact. This is arguably the next exciting frontier for marketing technology.

- more developments in cross-channel customer tracking. Again, the need for this has been obvious and some major investments have already been made. Cookies are becoming increasingly inadequate as cookie-hostile channels like mobile become more important. Marketers will soon reach a tipping point (or maybe they already have) where they realize they must abandon cookies and move on to other approaches such as device identification or external identity databases. A new standard will eventually emerge, although I can’t even guess what it might be.

- tighter integration between advertising and marketing technology. These two realms are now largely separate with a few exceptions such as retargeting. But as personalized ad messages become increasingly possible, marketers will have ever-greater incentive to target and, ultimately, coordinate messages across channels using shared data. This is highly dependent on the improved customer tracking, so it might have to wait a bit.

- better marketing attribution. If there’s a last stop on the road to marketing Nirvana, attribution might be it. Once marketers have assembled all that data and associated everything with the right customer, they’ll finally be able to deploy advanced analytical methods to really understand the long- and short-term incremental impact of their marketing efforts. Then, and this itself would be heavenly, we’ll never again hear anyone quote John Wanamaker about not knowing which half of his advertising is wasted.

Recommendations for Marketers

Nirvana is still far distant.  Marketers face immediate choices in how to spend their time and budgets. The trends I’ve just described do have some immediate practical implications. Here are my suggestions:

- Experiment like crazy. The various Decision-layer vendors currently offer different specialties, such as lead scoring vs. product recommendations vs. churn predictions. Vendors in each area are expanding their scope so there’s a good chance you’ll eventually pick one to do almost everything. To have the best odds of making a good selection, you’ll want to learn about as many vendors as possible in advance. So run tests to build an understanding of the applications, technologies, and corporate culture. The good news is that each approach can probably pay for itself in improved performance, so these tests should be more or less self-financing.

- Keep an eye out for new data. Many of the Decision-layer vendors bring their own data to the party, and evaluating that data is one part of understanding what they offer. But there are also other data sources that are not tied to a Decision system. You’ll want to explore these to understand what value they provide value and whether to make them part of your long-term data foundation.

- Plan for integration. You may not have shared customer data or decisions today, but it’s increasingly likely they’re in your future. So every new marketing system should be evaluated in part on its ability to integrate with other systems. This involves sending data to the central database and reading data from it, as well as integrating with Decision-layer systems for predictive models, rules-based selections, optimization, recommendations, personalization, and more. Even if you’re going to use an integrated suite, you’ll want to assess how easily you can supplement its functions by tying into external products, and what kinds of products are already available for integration.

Summary

The stand-alone Customer Data Platform is one solution to the challenge of providing a multi-source, shared marketing database, but it isn't the only option.  Whichever solution marketers ultimately find most appealing, they will benefit from gaining control of their data and moving on to new opportunities that database makes possible..

I've Discovered a New Class of System: the Customer Data Platform. Causata Is An Example.

It has taken me a while to connect the dots, but I’m now pretty sure I see a new type of software emerging. These systems that gather customer data from multiple sources, combine information related to the same individuals, perform predictive analytics on the resulting database, and use the results to guide marketing treatments across multiple channels. This differs quite radically from standard marketing automation systems, which use databases built elsewhere, rarely include integrated predictive modeling, and are focused primarily on moving customers through multi-step campaigns. In fact, the new systems complement rather than compete with marketing automation, which they treat as just one of several execution platforms. The new systems can also feed sales, customer service, online advertising, point of sale, and any other customer-facing systems.

Given how much vendors and analysts love to create new categories, I’m genuinely perplexed that no one has yet named this one. I’ll step in myself, and hereby christen the concept as “Customer Data Platform”.  Aside from having a relatively available three letter abbreviation (see Acronym Finder for other uses of CDP), the merits of this name include:

- “Customer” shows the scope extends to all customer-related functions, not just marketing;
- “Data” shows the primary focus is on data, not execution; and
- “Platform” shows it does more than data management while supporting other systems

But, you may ask, is this really new? Certainly systems for Customer Data Integration (CDI) have been around for decades: these include specialized products like Harte-Hanks Trillium and SAS DataFlux, CDI features within general data management suites like Informatica and Pentaho, and integration within cloud-based business intelligence products like GoodData and Birst. Many of those products have limited capabilities for working with newer data sources like Web sites and social networks, but the real distinction between them and CDPs is that the older systems are mainly designed to assemble data.  Some also provide analytics, but they don't extend to real-time decisions based on predictive models.

Similarly, there have long been specialized systems for real-time interaction management (such as Infor Interaction Advisor and Oracle Real Time Decisions) and for predictive modeling (SAS, IBM SPSS, KXEN). Some interaction managers do create predictive models, and the really big vendors (IBM, SAS, Oracle) have all three key components (CDI, real-time decisions, and predictive models) somewhere in their stables. But systems that closely couple just those features with the goal of feeding data as well as recommendations to execution systems? Those are something new.

By now, you’re probably wondering if I’ll ever get around to actually naming the vendors I have in mind. I’ve recently written about some of them, including Reachforce/SetLogik and Lattice Engines.  I also include RedPoint in the mix, because it has all the key capabilities (database development, predictive models, and real time decisions) even though it also offers conventional campaign management. Others I haven’t yet written about include Mintigo and Gainsight. Of course, each has a different mix of features and its own market position.  Indeed, several have specifically told me they do not compete with the others. Fair enough, but I still see enough similarity to group them together.

All this is a very long-winded introduction to Causata, yet another member of this new class. By now, you can probably guess Causata’s main functions: assemble customer data from multiple sources, consolidate it by customer, place it in an analytics-friendly format, run predictive models against it, and respond in real time to recommendation requests from other systems including Web sites, email, banner ads, and call centers. And you’d be right.

But that’s not the end of the story. With any product, it’s the details that matter. Causata is particularly strong in the data management department, accepting both batch and real-time data feeds and storing data as different types of events (email sent, Web site visit, call center interaction, etc.), each having predefined attributes. The system also has a particularly sophisticated “identity association” service, which looks for simultaneous events involving different identifiers as a way to link them, and can chain identifiers that were linked at different times. When I spoke with Causata about two months ago, the association rules were pretty much the same for all clients, but they promised users would get more control in the future. Users could already choose which types of associations to use in specific queries.

Causata stores the assembled data in HBase, a Hadoop-based database management system that is particularly well suited to large data volumes, many different data types, and ad hoc queries. In addition to the raw data, the system can store derived values such as aggregations (e.g., number of Web page view in past 24 hours) and model scores. Users can run SQL queries to extract data for analysis and predictive modeling in third-party software including QlikView, Tableau, SAS, and R. Prebuilt QlikView reports show the predictive power of different variables for user-specified events. The lack of native analysis and modeling tools creates some friction for users, but also lets them stick with familiar products. So the pros and cons probably cancel each other out.

The system’s decision tools are straightforward. For each situation, users define a “decision engine” that can select among multiple options, such as campaigns, products, or marketing content. These options can have qualification rules. To make a decision, the system can test the options in sequence and pick the first one for which a customer is qualified, or pick the option with the highest predictive model score. Users can also specify a percentage of customers to receive a random option, to gather data for future decisions. An engine can return multiple decisions for situations that require more than one option, such as a Web page with several offers. Causata has some machine learning algorithms to help with the decision process. It plans to expand these to automatically select the best option in a given situation.

Decision engines are called by external systems through a Web services API that can respond in under 50 milliseconds. This is fast enough to manage Web banner ads – something not all interaction managers can achieve. Model scores and other data are updated in real time during an interaction.

Causata can be deployed on-premise by a client or as a cloud-based service. The vendor says a typical implementation starts with three or four data sources and is deployed in about 30 days – very fast for this type of system. In February, Causata introduced prebuilt applications for cross-sell, acquisition, and return programs in financial services, communications, and digital media. These will further speed deployment.

Pricing is based on the number of data sources and touchpoints, with additional charges based on data storage. Cost begins around $150,000 per year.

What's New at DataFlux? I Thought You'd Never Ask.

What with it being Valentine’s Day and all, you probably didn’t wake up this morning asking yourself, “I wonder what’s new with DataFlux?” That, my friend, is where you and I differ. Except that I actually asked myself that question a couple of weeks ago, and by now have had time to get an answer. Which turns out to be rather interesting.

DataFlux, as anyone still reading this probably knew already, is a developer of data quality software and is owned by SAS. DataFlux’s original core technology was a statistical matching engine that automatically analyzes input files and generates sophisticated keys which are similar for similar records. This has now been supplemented by a variety of capabilities for data profiling, analysis, standardization and verification, using reference data and rules in addition to statistical methods. The original matching engine is now just one component within a much larger set of solutions.

In fact, and this is what I find interesting, much of DataFlux’s focus is now on the larger issue of data governance. This has more to do with monitoring data quality than simple matching. DataFlux tells me the change has been driven by organizations that face increasing pressures to prove they are doing a good job with managing their data, for reasons such as financial reporting and compliance with government regulations.

The new developments also encompass product information and other types of non-name and address data, usually labeled as “master data management”. DataFlux reports that non-customer data is is the fastest growing portion of its business. DataFlux is well suited for non-traditional matching applications because the statistical approach does not rely on topic-specific rules and reference bases. Of course, DataFlux does use rules and reference information when appropriate.

The other recent development at DataFlux has been creation of “accelerators”, which are prepackaged rules, processes and reports for specific tasks. DataFlux started offering these in 2007 and now lists one each for customer data quality, product data quality, and watchlist compliance. More are apparently on the way. Applications like this are a very common development in a maturing industry, as companies that started by providing tools gain enough experience to understand how the applications commonly built with those tools. The next step—which DataFlux hasn’t reached yet—is to become even more specific by developing packages for particular industries. The benefit of these applications is that they save clients work and allow quicker deployments.

Back to governance. DataFlux’s movement in that direction is an interesting strategy because it offers a possible escape from the commoditization of its core data quality functions. Major data quality vendors including Firstlogic and Group 1 Software, plus several of the smaller ones, have been acquired in recent years and matching functions are now embedded within many enterprise software products. Even though there have been some intriguing new technical approaches from vendors like Netrics and Zoomix, this is a hard market to penetrate based on better technology alone. It seems that DataFlux moved into governance more in response to customer requests than due to proactive strategic planning. But even so, they have done well to recognize and seize the opportunity when it presented itself. Not everyone is quite so responsive. The question now is whether other data quality vendors will take a similar approach or this will be a long-term point of differentiation for DataFlux.