3 Insights to Help Build Your Unified Customer Database

The Customer Data Platform Institute (which is run by Raab Associates) on Monday published results of a survey we conducted in cooperation with MarTech Advisor. The goal was to assess the current state of customer data unification and, more important, to start exploring management practices that help companies create the rare-but-coveted single customer view.

You can download the full survey report here (registration required) and I’ve already written some analysis on the Institute blog . But it’s a rich set of data so this post will highlight some other helpful insights.

1. All central customer databases are not equal.

We asked several different questions whose answers depended in part on whether the respondent had a unified customer database. The percentage who said they did ranged from 14% to 72%:

I should stress that these answers all came from the same people and we only analyzed responses with answers to all questions.  And, although we didn’t test their mental states, I doubt a significant fraction had multiple personality disorders. One lesson is that the exact question really matters, which makes comparing answers across different surveys quite unreliable. But the more interesting insight is there are real differences in the degree of integration involved with sharing customer data.

You’ll notice the question with the fewest positive answers – “many systems connected through a shared customer database” describes a high level of integration.  It’s not just that data is loaded into a central database, but that systems are actually connected to a shared central database. Since context clearly matters, here is the actual question and other available answers:

 The other questions set a lower bar, referring to a “unified customer database” (33%), “central database (42%) and "central customer database” (57%). Those answers could include systems where data is copied into a central database but then used only for analysis. That is, they don’t imply connections or sharing with operational customer-facing systems. They also could describe situations where one primary system has all the data and thus functions as a central or unified database.

The 72% question covered an even broader set of possibilities because it only described how customer data is combined, not where those combinations take place. That is, the combinations could be happening in operational systems that share data directly: no central database is required or even implied.  Here are the exact options:

The same range of possibilities is reflected in answers about how people would use a single customer view. The most common answers are personalization and customer insights.  Those require little or no integration between operational systems and the central database, since personalization can easily be supported by periodically synchronizing a few data elements. It’s telling that consistent treatments ranks almost dead last – even though consistent experiences are often cited as the reason a central database is urgently required.

This array of options to describe the central customer database suggests a maturity model or deployment sequence.  It would start with limited unification by sharing data directly between systems (the most common approach, based on the stack question shown above), progress to a central database that assembles the data but doesn’t share it with the operational systems, and ultimately achieve the perfect bliss of unity, which, in martech terms, means all operational systems are using the shared database to execute customer interactions.  Purists might be troubled by these shades of gray, but they offer a practical path to salvation. In any case, it’s certainly important to keep these degrees in mind and clarify what anyone means when they talk about shared customer data or that single customer view.

2. You must have faith.

Hmm, a religious theme seems to be emerging.  I hadn’t intended that but maybe it’s appropriate. In any event, I’ve long argued that the real reason technologies like marketing automation and predictive modeling don’t get adopted more quickly are not the practical obstacles or lack of proven value, but lack of belief among managers that they are worthwhile. This doesn’t show up in surveys, which usually show things like budget, organization, and technology as the main obstacles. My logic has been that those are basically excuses: people would find the resources and overcome the organizational barriers if they felt the project were important enough.  So citing budgets and organizational constraints really means they see better uses for their limited resources.

The survey data supports my view nicely. Looking at everyone’s answers to a question about obstacles, the answers are rather muddled: budget is indeed the most commonly cited obstacle (41%), followed closely by the technical barrier of extracting data from source systems (39%). Then there’s a virtual tie among organizational roadblocks (31%), other priorities in IT (29%), other priorities in marketing (29%) and systems can’t use (29%). Not much of a pattern there.

But when you divide the respondents based on whether they think single customer view is important for over-all marketing success, a stark division emerges.  Budget and organization are the top two obstacles for people who don’t think the unified view is needed, while having systems that can extract and use the data are top two obstacles for people who do think it’s necessary for success. In other words, the people committed to unified data are focused on practical obstacles, while those who don’t are using the same objections they apply to everything else.


Not surprisingly, people who classify SCV as extremely important are more likely to actually have a database in place than people who consider it just very important, who in turn have more databases than people who consider it even less important or not important at all.  (In case you're wondering, each group accounts for roughly one-third of the total.)

The same split applies to what people would consider helpful in build in building a single customer view: people who consider the single view important are most interested in best practices, case studies, and planning assumptions – i.e., building a business case.  Those who think it’s unimportant ask for product information, vendor lists, and pricing. I find this particular split a bit puzzling, since you’d think people who don’t much care about a unified database would be least interested in the details of building one. A cynic might say they’re looking for excuses (cost is too high) but maybe they’re actually trying to find an easy solution so they can avoid a major investment.

Jumping ahead just a bit, the idea that SCV doubters are less engaged than believers also shows up in at the management tools they use.  People who rated SCV as extremely important were much more likely to use all the tools we asked about. Interestingly, the biggest gap is in use of value metrics. This could be read to mean that people become believers after they measure the value of a central database, or that people set up measurements after they decide they need to prove their beliefs. My theology is pretty rusty but surely there’s a standard debate about whether faith or action comes first.

Regardless of the exact reasons for the different attitudes, the fundamental insight here is that people who consider a single view important act quite differently from people who don’t. This means that if you’re trying to sell a customer database, either in your own company or as a vendor, you need to understand who falls into which category and address them in appropriate terms. And I guess a little prayer never hurt.

3. Tools matter.

We’ve already seen that believers have more databases and have more tools, so you won’t be surprised that using more tools correlates directly with having or planning a database.

Let's introduce the tools formally.  Here are the exact definitions we used and the percentage of people who said each was present in their organization:

Of course, the really interesting question isn’t which tools are most popular but which actually contribute (or at least correlate) with deploying a database. We looked at tool use for three groups: people with a database, people planning a database, and people with no such plans. 

Over all, results for the different tools were pretty similar: people who used each tool were much more likely to have a database and somewhat more likely to plan to build one. The pattern is a bit jumbled for Centers of Excellence and technology standards, but the numbers are small so the differences may not be significant. But it's still worth noting that Centers of Excellence are really tools to diffuse expertise in using marketing technology and don’t have too much to do with actually creating a customer database.

If you’re looking for a dog that didn’t bark, you might have expected companies using agile to be exceptionally likely to either have a database or be planning one. All quiet on that front: the numbers for agile look like numbers for long term planning and value metrics, adjusting for relative popularity. So agile is helpful but not a magic bullet.

What have we learned here? 

Clearly, we've learned that management tools are important and that long term planning in particular both the most common and the best predictor of success.

We also found that tools aren’t enough: managers need to be convinced that a unified customer view is important before they’ll invest in a database or tools to build it.

And, going back to the beginning, we saw that there are many forms of unified data, varying in how data is shared, where it’s stored, how it’s unified, and how it’s used. While it’s easy enough to assume that tight, real-time integration is needed to provide unified omni-channel customer experiences, many marketers would be satisfied with much less. I’d personally hope to see more but, as every good missionary knows, people move towards enlightenment in many small steps.

Pega Customer Decision Hub Offers High-End Customer Journey Orchestration

My previous posts about Journey Orchestration Engines (JOEs) have all pointed to new products. But some older systems qualify as well. In some ways they are even more interesting because they illustrate a mature version of the concept.

The Customer Decision Hub from Pega (formerly PegaSystems) is certainly mature: the product can trace its roots back well over a decade, to a pioneering company called KiQ Limited, which was purchased in 2004 by Chordiant, which Pega purchased in 2010. Obviously the system has been updated many times since then but its core approach to optimizing real-time decisions across all channels has stayed remarkably constant. Indeed, some features the product had a decade ago are still cutting edge today – my favorite is simulation of proposed decision rules to assess their impact before deployment.

Pega positions Customer Decision Hub as part of its core platform, which supports applications for marketing, sales automation, customer service, and operations. It competes with the usual enterprise suspects: Adobe, Oracle, Salesforce.com, IBM, and SAS. Even more than those vendors, Pega focuses on selling to large companies, describing its market as primarily the Fortune 3000. So if you’re not working at one of those firms, consider the rest of this article a template for what you might look for elsewhere.

The current incarnation of Customer Decision Hub ihas six components: Predictive Analytics Director to build offline predictive models, Adaptive Decision Manager to build self-learning real-time models, Decision Strategy Manager to set rules for making decisions, Event Strategy Manager to monitor for significant events, Next Best Action Advisor to deliver decisions to customer-facing systems, and Visual Business Director for planning, simulation, visualization, and over-all management. From a journey orchestration perspective, the most interesting of these are Decision Strategy Manager and Event Strategy Manager, because they’re the pieces that select customer treatments. The other components provide inputs (Predictive Analytics Director and Adaptive Decision Manager), support execution (Next Best Action Advisor), or give management control (Visual Business Director).

Decision Strategy Manager is where the serious decision management takes place. It brings together audiences, offers, and actions. Audiences can be built using segmentation rules or selected by predictive models. Offers can include multi-step flows with interactions over time and across channels. Actions can be anything, not just marketing messages, and may include doing nothing. They are selected using arbitration rules that specify the relevance of each action to an audience, rank the action based on eligibility and prioritization, and define where the action can be delivered.

The concept of “relevance” is what qualifies Decision Hub as a JOE. It measures the value of each action against the customer’s current needs and context,. This is the functional equivalent of defining journey stages or customer states, even though Pega doesn’t map how customers move from one state to another. The interface to set up the arbitration rules is where Decision Hub’s maturity is most obvious. For example, users can build predictive model scores into decision rules and can set up a/b tests within the arbitration to compare different approaches.

Event Strategy Manager lets users define events based on data patterns, such as three dropped phone calls within a week. These events can trigger specific actions or factor into a decision strategy arbitration. It’s another way of bringing context to bear and thus of ensuring each decision is appropriate to the customer’s current journey stage. Like arbitration rules in Decision Strategy Manager, the event definitions in Event Strategy Manager can be subtle and complex. The system is also powerful in being able to connect to nearly any type of data stream, including social, mobile, and Internet of Things devices as well as traditional structured data.

I won't go into details of other Decision Hub components, but they’re equally advanced. Companies with the scale to afford the system can expect it to pay for itself: in one published study, the three-year cost was $7.7 million but incremental revenue was $362 million. Pega says few deployments cost less than $250,000 and most are over $1 million. As I say, this isn’t a system for everyone. But it does set a benchmark for other options.

HubSpot Announces LinkedIn, Facebook Partnerships and Free Marketing Automation Edition at INBOUND Conference

HubSpot held its annual INBOUND conference in Boston last week. Maybe it's me, but the show seemed to lack some of its usual self-congratulatory excitement: for example, CEO Brian Halligan didn’t present the familiar company scorecard touting growth in customers and revenues. (A quick check of financial reports shows those are just fine: the company is expecting about 45% revenue increase for 2016.) Even the insights that Halligan and co-founder Dharmesh Shah presented in their keynotes seemed familiar: I'm guessing you've already heard that video, social, messaging, free trials, and chatbots will be big.

My own attention was more focused on the product announcements. The big news was a free version of HubSpot’s core marketing platform, joining free versions already available of its CRM and Sales systems. (In Hubspeak, CRM is the underlying database that tracks and manages customer interactions, while Sales is tools for salesperson productivity in email and elsewhere.)  Using free versions to grow marketing automation has consistently failed in the past, probably because people attracted by a free system aren't willing to do the substantial work needed for marketing automation success.  But HubSpot managers are aware of this history and seem confident they have a way to cost-effectively nurture a useful fraction of freemium users towards paid status. We'll see.

The company also announced enhancements to existing products. Many were features that already exist in other mid-tier systems, including branching visual workflows, sessions within Web analytics reports, parent/child relationships among business records, and detailed control over user permissions. As HubSpot explained it, the modest scope of these changes reflects a focus on simplifying the system rather than making it super-powerful. One good example of this attitude was a new on-site chat feature, which seems basic enough but has some serious hidden cleverness in automatically routing chat requests to the right sales person, pulling up the right CRM record for the agent, and adding the chat conversation to the customer history.

One feature that did strike me as innovative was closer to HubSpot’s roots in search marketing: a new “content strategy” tool reflecting the shift from keywords to topics as the basis of search results. HubSpot’s tool helps marketers find the best topics to try to dominate with their content.  This will be very valuable for marketers unfamiliar with the new search optimization methods. Still, what you really want is a system that helps you create that content.  HubSpot does seem to be working on that.

With relatively modest product news, the most interesting announcements at the conference were probably about HubSpot’s alliances.  A new Facebook integration lets users create Facebook lead generation campaigns within HubSpot and posts leads from those campaigns directly to the HubSpot database. A new LinkedIn integration shows profiles from LinkedIn Sales Navigator within HubSpot CRM screens for users who have a Sales Navigator subscription. Both integrations were presented as first steps towards deeper relationships. These relationships reflect the growing prominence of HubSpot among CRM/marketing automation vendors, which gives companies like Microsoft and LinkedIn a reason to pick HubSpot as a partner. This, in turn, lets HubSpot offer features that less well-connected competitors cannot duplicate. That sets up a positive cycle of growth and expansion that is very much in HubSpot’s favor.

As an aside, the partnerships raise the question of whether Microsoft might just purchase HubSpot and use it to replace or supplement the existing Dynamics CRM products. Makes a lot of sense to me.  A Facebook purchase seems unlikely but, as we also learned last week, unlikely things do sometimes happen.

ActionIQ Merges Customer Data Without Reformatting

One of the fascinating things about the Customer Data Platform Institute is how developers from different backgrounds have converged on similar solutions. The leaders of ActionIQ, for example, are big data experts: Tasso Argyros founded Aster Data, which was later purchased by Teradata, and Nitay Joffe was a core contributor to HBase and the data infrastructure at Facebook.  In their previous lives, both saw marketers struggling to assemble and activate useful customer data. Not surprisingly, they took a database-centric approach to solving the problem.

What particularly sets ActionIQ apart is the ability to work with data from any source in its original structure. The system simply takes a copy of source files as they are, lets users define derived variables based on those files, and uses proprietary techniques to query and segment against those variables almost instantly. It’s the scalability that’s really important here: at one client, ActionIQ scans two billion events in a few seconds. Or, more precisely, it’s the scalability plus flexibility: because all queries work by re-reading the raw data, users can redefine their variables at any time and apply them to all existing data. Or, really, it's scalability, flexibility, and speed, because new data is available within the system in minutes.

So, amongst ActionIQ’s many advantages are scalability, flexibility, and speed. These contrast with systems that require users to summarize data in advance and then either discard the original detail or take much longer to resummarize the data if a definition changes.

ActionIQ presents its approach as offering self-service data access for marketers and other non-technical users. That’s true insofar as marketers work with previously defined variables and audience segments. But defining those variables and segments in the first place takes the same data wrangling skills that analysts have always needed when faced with raw source data. ActionIQ reduces work for those analysts by making it easier to save and reuse their definitions. Its execution speed also reduces the cost of revising those definitions or creating alternate definitions for different purposes. Still, this is definitely a tool for big companies with skilled data analysts on staff.

The system does have some specialized features to support marketing data. These include identity resolution tools including fuzzy matching of similar records (such as different versions of a mailing address) and chaining of related identifiers (such as a device ID linked to an email linked to an account ID). It doesn’t offer “probabilistic” linking of devices that are frequently used in the same location although it can integrate with vendors who do. ActionIQ also creates correlation reports and graphs showing the relationship between pairs of user-specified variables, such as a customer attribute and promotion response. But it doesn’t offer multi-variable predictive models or machine learning.

ActionIQ gives users an interface to segment its data directly. It can also provide a virtual database view that is readable by external SQL queries or PMML-based scoring models. Users can also export audience lists to load into other tools such as campaign managers, Web ad audiences, or Web personalization systems. None of this approaches the power of the multi-step, branching campaign flows of high-end marketing automation systems, but ActionIQ says most of its clients are happy with simple list creation. Like most CDPs, ActionIQ leaves actual message delivery to other products.

The company doesn’t publicly discuss the technical approach it takes to achieve its performance, but they did describe it privately and it makes perfect sense. Skeptics should be comforted by the founders’ technical pedigree and demonstrated actual performance. Similarly, ActionIQ asked me not to share screen shots of their user interface or details of their pricing. Suffice to say that both are competitive.

ActionIQ was founded in 2014 and has been in production with its pilot client for over one year. The company formally launched its product last month.

Walker Sands / Chief Martech Study: Martech Maturity Has Skyrocketed

Tech marketing agency Walker Sands and industry guru Scott Brinker of Chief Martech yesterday published a fascinating survey on the State of Marketing Technology 2017, which you can download here.  The 27 page report provides an insightful analysis of the data, which there’s no point to me duplicating in depth. But I will highlight a couple of findings that are most relevant to my own concerns.

Martech maturity has skyrocketed in the past year. This theme shows up throughout the report. The percentage of responders classifying their companies as innovators or early adopters grew from 20% in 2016 to 48% in 2017; marketers whose companies invest the right amount in marketing technology grew from 50% to 71%; all obstacles to adoption were less common (with the telling exception of not needing anything new).

Truth be told, I find it hard to believe that things can have shifted this much in a single year and that nearly half of all companies (and 60% of individual marketers) are innovators or early adopters. A more likely explanation is the new survey attracted more advanced respondents than before.  We might also be seeing a bit of “Lake Wobegon Effect,” named after Garrison Keillor’s mythical town where all the children are above average.  Evidence for the latter might be that 69% felt their marketing technology is up to date and sufficient (up from 58%), making this possibly the most complacent group of innovators ever.


Multi-product architectures are most common. I have no problem accepting this one: 21% of respondents said they use a single-vendor suite, while 69% had some sort of multi-vendor approach (27% integrated best-of-breed, 21% fragmented best-of-breed, 21% limited piecemeal solutions). The remainder had no stack (7%) or proprietary technology (4%).

 

But don’t assume that “single-vendor suite” necessarily means one of the enterprise marketing clouds.  Small companies reported using suites just as often as large ones. They were probably referring to all-in-one products like HubSpot and Infusionsoft.

"Best of breed marketers get the most out of their martech tools." That’s a direct quote from the report, but it may overstate the case: 83% of integrated best-of-breed users felt their company was good or excellent at leveraging the stack, compared with 76% of the single-vendor-suite. That not such a huge difference, especially given the total sample of 335. Moreover, companies with fragmented best-of-breed stacks reported less ability (67%) than the single-vendor suite users. If you combine the two best-of-breed groups then the suite users actually come out ahead. A safer interpretation might be that single-vendor suites are no easier to use than best-of-breed combinations.  That would still be important news to companies that think pay a premium or compromise on features because they think suites make are easier to deploy.
  

Integration isn’t that much of a problem. Just 20% of companies cited better stack integration as a key to fully leveraging their tools, which ranked well behind better strategy (39%), better analytics (36%) and more training (33%) and roughly on par with more employees (23%), better defined KPIs (23%), and more data (20%). This supports the previous point about best-of-breed working fairly well, whether or not the stack was well integrated. I would have expected integration to be a bigger issue, so this is a bracing reality check. One interpretation (as I argued last week) is that integration just isn’t as important to marketers as they often claim.

There’s plenty else of interest in the report, so go ahead and read it and form your own opinions. Thanks to Walker Sands and Chief Martech for pulling it together.