Do Customer Data Platforms Need Identity Matching? The Answer May Surprise You.

I spend a lot of time with vendors trying to decide whether they are, or should be, a Customer Data Platform. I also spend a lot of time with marketers trying to decide which CDPs might be right for them. One topic that’s common in both discussions is whether a CDP needs to include identity resolution – that is, the ability to decide which identifiers (name/address, phone number, email, cookie ID, etc.) belong to the same person.

It seems like an odd question. After all, the core purpose of a CDP is to build a unified customer database, which requires connecting those identifiers so data about each customer can be brought together. So surely identity resolution is required.

Turns out, not so much. There are actually several reasons.

- Some marketers don’t need it. Companies that deal only in a single channel often have just one identifier per customer.  For example, Web-only companies might use just a cookie ID.  True, channel-specific identifiers sometimes change (e.g., cookies get deleted).  But there may be no practical way to link old and new identifiers when that happens, or marketers may simply not care.  A more common situation is companies have already built an identity resolution process, often because they’re dealing with customers who identify themselves by logging in or who transact through accounts. Financial institutions, for example, often know exactly who they’re dealing with because all transactions are associated with an account that’s linked to a customer's master record (or perhaps not linked because the customer prefers it that way). Even when identity resolution is complicated,  mature companies often (well, sometimes) have mature processes to apply a customer ID to all data before it reaches the CDP. In any of these cases, the CDP can use the ID it’s given and not need an identity resolution process of its own.
- Some marketers can only use it if it’s perfect. Again, think of a financial institution: it can’t afford to guess who’s trying to take money out of an account, so it requires the customer to identify herself before making a transaction. In many other circumstances, absolute certainty isn’t required but a false association could be embarrassing or annoying enough that the company isn’t willing to risk it. In those cases, all that’s needed is an ability to “stitch” together identifiers based on definite connections. That might mean two devices are linked because they both sent emails using the same email address, or an email and phone number linked because someone entered them both into a registration form. Almost every CDP has this sort of “deterministic” linking capability, which is so straightforward that it barely counts as identity resolution in the broader sense.

- Specialized software already exists. The main type of matching that CDPs do internally – beyond simple stitching – is “fuzzy” matching.  This applies rules to decide when two similar-looking records really refer to the same person. It's most commonly applied to names and postal addresses, which are often captured inconsistently from one source to the next. It might sometimes be applied to other types of data, such as different forms of an email address (e.g. draab@raabassociates.com and draab@raabassociatesinc.com). The technology for this sort of matching gets very complicated very quickly, and it’s something that specialized vendors offer either for purchase or as a service. So CDP vendors can quite reasonably argue they needn’t build this for themselves but should simply integrate an external product.

- Much identity resolution requires external data. This is the heart of the matter.  Most of the really interesting identity resolution today involves linking different devices or linking across channels when there’s no known connection. This sort of “probabilistic” linking is generally done by vendors who capture huge amounts of behavioral data by tracking visitors to popular Web sites or users of popular mobile applications, or by gathering deterministic links from many different sources. They then build giant databases (or "graphs" if you want to sound trendy) with these connections.  Even matching of offline names and addresses usually requires external data, both to standardize the inputs (to make fuzzy matching more accurate) and to incorporate information such as address and name changes that cannot be known by inspecting the data itself.  In all these situations, marketers need to use the external vendors’ data to find connections that don’t exist within the marketers’ own, much more limited information. If the external vendor provides matching functions in addition to the data, the CDP is relieved of the need to do the matching internally.

In short, there’s a surprisingly strong case that identity resolution isn’t a required feature in a CDP.  All the CDP really needs is basic stitching and connections to external services for more advanced approaches.  As cross-device and cross-channel matching become more important, CDPs will be more reliant on external vendors no matter what capabilities they’ve built for themselves. One important qualifier is the CDP implementation team still needs expertise in matching, so they can help clients set it up properly. But while it’s great to find a CDP vendor with its own matching technology, lack of that technology shouldn’t exclude a vendor from being considered a CDP.

Reltio Makes Enterprise Data Usable, and Then Uses It

I’ve spent a lot of time recently talking to Customer Data Platform vendors, or companies that looked like they might be. One that sits right on the border is Reltio, which fits the CDP criteria* but goes beyond customer data to all types of enterprise information. That puts it more in the realm of Master Data Management, except that MDM is highly technical while Reltio is designed to be used by marketers and other business people. You might call it “self-service MDM” but that’s an oxymoron right up there with “do-it-yourself brain surgery”.

Or not. Reltio avoids the traditional complexity of MDM in part by using the Cassandra data store, which is highly scalable and can more easily add new data types and attributes than standard relational databases. Reltio works with a simple data model – or graph schema if you prefer – that captures relationships among basic objects including people, organizations, products, and places. It can work with data from multiple sources, relying on partner vendors such as SnapLogic and MuleSoft for data acquisition and Tamr, Alteryx, and Trifacta for data preparation. It has its own matching algorithms to associate related data from different sources. As for the do-it-yourself bit: well, there’s certainly some technical expertise needed to set things up, but Reltio's services team generally does the hard parts for its clients. The point is that Reltio reduces the work involved – while adding a new source to a conventional data warehouse can easily take weeks or months, Reltio says it can add a new source to an existing installation in one day.

The result is a customer profile that contains pretty much any data the company can acquire. This is where the real fun begins, because that profile is now available for analysis and applications. These can also be done in Reltio itself, using built-in machine learning and data presentation tools to provide deep views into customers and accounts, including recommendations for products and messages. A simple app might take one or two months to build; a complicated app might take three or four months. The data is also available to external systems via real-time API calls.

Reltio is a cloud service, meaning the system doesn’t run on the client’s own computers. Pricing depends on the number of users and profiles managed but not the number of sources or data volume. The company was founded in 2011 and released its product several years later. Its clients are primarily large enterprises in retail, media, and life sciences.

______________________________________________________________________
* marketer-controlled; multi-source unified persistent data; accessible to external systems

Dreamforce 2014: Process Is More Important Than Analytics

photo by Dion Hinchcliffe

Salesforce.com’s Dreamforce conference this year was the usual mix of spectacle, congestion, and heart-felt philanthropy. But the main announcements felt fairly slight: a new “analytics cloud” that is primarily about visualization and a mobile app builder for the Salesforce1 platform.

The analytics cloud* is a step forward only because Salesforce has been so far behind: it bulk loads data into a star schema relational database using inverted index for speed, which is a solid but old-fashioned approach. Of course, it’s cloud-based but so are other, newer approaches that are ultimately more flexible and scalable. Solutions to the really hard problems of entity association (matching identifiers for the same person in different systems) and predictive analytics are not included. Nor does the system handle real-time updates or allow queries by external systems for purposes like message personalization. The visualization itself is indeed fast and pretty, but it’s not obviously superior to Birst (also cloud-based), Tableau, or QlikView. The core technology was acquired when Salesforce.com bought EdgeSpring last June.

The mobile app builder for Salesforce1** is the sort of innovation only a geek would love: after all, most people don’t think much about system building in general, let alone get excited about making it easier to build mobile apps for Salesforce. But it’s certainly the more important of the two announcements, because it illustrates how broad the scope of Salesforce has become. The most impressive demonstrations were operational processes such as remote order-taking and customer support, which are far removed from traditional sales automation. They also illustrated how absolutely central mobile devices have become to most business processes, something we all vaguely realize but are still not necessarily acting upon. Business processes need to be reimagined from a mobile perspective, taking into account the possibilities of doing things instantly while on-site at a store, a shopper’s home, traveling, or whatever. This is no longer a new thought, but few companies have actually done it. By providing a drag-and-drop mobile app builder, Salesforce opens up possibilities for companies to innovate along these lines quickly, easily, and cheaply. That’s important to everyone, not just Salesforce geeks.

In fact, the closest thing I had to a deep thought during the conference was that people put too much emphasis on distributing data for decisions and not enough about distributing processes. Demonstrations for tools like Wave always show users drilling into sales data to uncover weak pickle sales at convenience stores in Milwaukee – something that’s exciting the first time but you don’t do on a regular basis. By contrast, a distributed process like better store shelf allocations provides continuous benefits, even though it doesn’t require a human analyst to have a brilliant insight. A really good organization has smoothly running processes that handle each situation according to rules that require little or no judgment. (Of course, a certain amount of discretion by empowered employees is still necessary –but I’d argue the sorts of decisions that make for, say, a great hotel experience have nothing to do with advanced data analysis.)  People like decision management guru James Taylor  have long known this and distinguished operational decisions from strategic decisions, so I guess this isn’t really a new thought, either. But, like the growing centrality of mobile, it’s something that companies need to address by giving them resources. Winners will; losers won’t. It’s that simple.

And while I’m being blunt: two Hawaiian dances in a keynote is two Hawaiian dances too many.


_____________________________________________________________*a.k.a. “Wave”, apparently to justify many Hawaii-themed promotions and an appearance by the Beach Boys.

**called “Lighting”, which suggests it was named separately from Wave, since it's unsafe to surf during electrical storms. But nomenclature notwithstanding, the two systems do seem to work together.

Customer Data Platform Guide Reviews Tools to Build Marketing Databases

Raab Associates’ new Guide to Customer Data Platforms is now available [update: as of 2014, we no longer sell it].

You may not find that news to be fall-off-your-chair exciting. In fact, you’re more likely to wonder whether the world needs yet another report on anything at all. Fair enough. So before telling you what’s in the CDP Guide, I'll tell you why it exists.

Simply put, marketers need better databases. If you’re a working marketer, you almost surely know this from personal experience. But someone who only read industry news and vendor promotions might think all anyone had to do was to plug in the latest cool application and it would immediately be filled with fresh clean data like water from a tap. Dirty big data is our industry’s dirty little secret.


The problem isn’t new but it is getting worse. As customers interact across more channels, marketers need to not just meet them in every new location but recognize them and carry on a continuous conversation from one touchpoint to the next. Marketers can also become more effective by enriching that conversation with information from external sources such as Web pages, social media, and commercial databases. Both the carrot of better results and the stick of customer expectations are ever-more-urgently driving marketers towards building better databases.

The good news is that plenty of smart vendors have also recognized this need and are trying to help marketers on their journey. I call their systems Customer Data Platforms and define them as “a marketer-controlled system that supports external marketing execution based on persistent, cross-channel customer data.”

If there’s one absolutely critical point in that definition, it’s that CDPs put marketers in charge of building their own database. Taking control is the only way that marketers will ever get the databases they desperately need. It’s why CDPs are so important.

But too few marketers know who the CDP vendors are, what they do, and how they differ. The Guide to Customer Data Platforms is designed to provide this information. If the CDP vendors are tour guides on the path to better data, the CDP Guide is the reviews you read to decide which one you’ll hire. As far as we know, no other study serves this purpose.

Given its goal, the heart of the Guide is the vendor profiles: three to five pages on each vendor, describing capabilities for data management, predictive modeling, marketing campaigns, and message delivery, plus background on the vendor’s technology, clients, company history, and pricing. You’ll want to read those closely when you’re selecting a vendor. But first you’ll have to decide whether a Customer Data Platform is something to consider. Here is some information to help make that judgment.

- CDPs are something new. CDPs are systems that help marketers build and update customer databases, and make those databases available to support marketing programs. That may not sound very new, but most B2B marketing automation products today build very limited databases while most B2C marketing automation products rely entirely on an external data warehouse. The systems that do build databases are designed to be used by IT departments, not marketers. And many CDPs provide predictive modeling or best-treatment recommendations that go well beyond the storage functions of a basic data warehouse.

- You still can’t do this at home. CDPs may be tools for marketers, but that doesn’t mean that marketers build the databases themselves. Rather, CDP vendors provide services that build the database with varying degrees of marketer involvement. The difference is that the marketers work directly with the CDP vendors, instead of relying on IT staff that often has other priorities and an imperfect understanding of marketing needs. This makes it much easier and quicker for marketers to get the database they need.

- CDPs are an outgrowth of existing system types. Most CDP systems were created for a purpose that happened to require the same database-building capabilities as a CDP. These purposes fall into three groups which I discussed in last week’s post, so I won’t repeat them here. They’re worth understanding because vendors in each group have a different set of skills, one of which will probably come closest to your needs.

- Convergence is coming. Even though the CDP vendors started with different applications, their shared abilities for identity matching, database management, analytics, and integration will allow them to support more of the same functions over time. As marketers understand the value of their databases more clearly, CDP vendors will be able to focus on selling their data platform features rather than applications the platform supports. Of course, once the platforms themselves are common, vendors will climb the value chain by offering better predictive analytics and cross-channel treatment optimization.

- Details count. CDP features may eventually converge, but for now the systems differ in many small ways that make a big difference. To take one example, nearly every CDP creates predictive models. But some can only predict response to specific promotions, based on who has responded before. Others can do the much more sophisticated analysis needed to predict which offer will best advance a long-term goal such as becoming a new customer. And even among those that model against long-term goals, some can actually estimate the incremental impact of a specific offer and others can just see most common correlations. We found similarly subtle differences in how data is collected (via the vendor’s own Web tags or by importing from other systems), the range of data sources (just marketing automation and CRM or those plus many others), natural language processing to extract useful information from text sources such as Web pages, how much history is kept and how it’s used, program execution, and end-user control. The CDP Guide clarifies these distinctions, but it’s still up to marketers to evaluate which differences will matter in their own business.

The CDP Guide itself contains quite a bit of other useful information, including a formal definition of CDPs, detailed explanations of what to look for in a CDP, and a history of marketing databases starting with ancient Sumer (don’t worry, I skipped the boring parts). Again, the goal is to provide one package with everything you need to get started along the path of buying a CDP system.  From there, it's up to you.