3 Insights to Help Build Your Unified Customer Database

The Customer Data Platform Institute (which is run by Raab Associates) on Monday published results of a survey we conducted in cooperation with MarTech Advisor. The goal was to assess the current state of customer data unification and, more important, to start exploring management practices that help companies create the rare-but-coveted single customer view.

You can download the full survey report here (registration required) and I’ve already written some analysis on the Institute blog . But it’s a rich set of data so this post will highlight some other helpful insights.

1. All central customer databases are not equal.

We asked several different questions whose answers depended in part on whether the respondent had a unified customer database. The percentage who said they did ranged from 14% to 72%:

I should stress that these answers all came from the same people and we only analyzed responses with answers to all questions.  And, although we didn’t test their mental states, I doubt a significant fraction had multiple personality disorders. One lesson is that the exact question really matters, which makes comparing answers across different surveys quite unreliable. But the more interesting insight is there are real differences in the degree of integration involved with sharing customer data.

You’ll notice the question with the fewest positive answers – “many systems connected through a shared customer database” describes a high level of integration.  It’s not just that data is loaded into a central database, but that systems are actually connected to a shared central database. Since context clearly matters, here is the actual question and other available answers:

 The other questions set a lower bar, referring to a “unified customer database” (33%), “central database (42%) and "central customer database” (57%). Those answers could include systems where data is copied into a central database but then used only for analysis. That is, they don’t imply connections or sharing with operational customer-facing systems. They also could describe situations where one primary system has all the data and thus functions as a central or unified database.

The 72% question covered an even broader set of possibilities because it only described how customer data is combined, not where those combinations take place. That is, the combinations could be happening in operational systems that share data directly: no central database is required or even implied.  Here are the exact options:

The same range of possibilities is reflected in answers about how people would use a single customer view. The most common answers are personalization and customer insights.  Those require little or no integration between operational systems and the central database, since personalization can easily be supported by periodically synchronizing a few data elements. It’s telling that consistent treatments ranks almost dead last – even though consistent experiences are often cited as the reason a central database is urgently required.

This array of options to describe the central customer database suggests a maturity model or deployment sequence.  It would start with limited unification by sharing data directly between systems (the most common approach, based on the stack question shown above), progress to a central database that assembles the data but doesn’t share it with the operational systems, and ultimately achieve the perfect bliss of unity, which, in martech terms, means all operational systems are using the shared database to execute customer interactions.  Purists might be troubled by these shades of gray, but they offer a practical path to salvation. In any case, it’s certainly important to keep these degrees in mind and clarify what anyone means when they talk about shared customer data or that single customer view.

2. You must have faith.

Hmm, a religious theme seems to be emerging.  I hadn’t intended that but maybe it’s appropriate. In any event, I’ve long argued that the real reason technologies like marketing automation and predictive modeling don’t get adopted more quickly are not the practical obstacles or lack of proven value, but lack of belief among managers that they are worthwhile. This doesn’t show up in surveys, which usually show things like budget, organization, and technology as the main obstacles. My logic has been that those are basically excuses: people would find the resources and overcome the organizational barriers if they felt the project were important enough.  So citing budgets and organizational constraints really means they see better uses for their limited resources.

The survey data supports my view nicely. Looking at everyone’s answers to a question about obstacles, the answers are rather muddled: budget is indeed the most commonly cited obstacle (41%), followed closely by the technical barrier of extracting data from source systems (39%). Then there’s a virtual tie among organizational roadblocks (31%), other priorities in IT (29%), other priorities in marketing (29%) and systems can’t use (29%). Not much of a pattern there.

But when you divide the respondents based on whether they think single customer view is important for over-all marketing success, a stark division emerges.  Budget and organization are the top two obstacles for people who don’t think the unified view is needed, while having systems that can extract and use the data are top two obstacles for people who do think it’s necessary for success. In other words, the people committed to unified data are focused on practical obstacles, while those who don’t are using the same objections they apply to everything else.


Not surprisingly, people who classify SCV as extremely important are more likely to actually have a database in place than people who consider it just very important, who in turn have more databases than people who consider it even less important or not important at all.  (In case you're wondering, each group accounts for roughly one-third of the total.)

The same split applies to what people would consider helpful in build in building a single customer view: people who consider the single view important are most interested in best practices, case studies, and planning assumptions – i.e., building a business case.  Those who think it’s unimportant ask for product information, vendor lists, and pricing. I find this particular split a bit puzzling, since you’d think people who don’t much care about a unified database would be least interested in the details of building one. A cynic might say they’re looking for excuses (cost is too high) but maybe they’re actually trying to find an easy solution so they can avoid a major investment.

Jumping ahead just a bit, the idea that SCV doubters are less engaged than believers also shows up in at the management tools they use.  People who rated SCV as extremely important were much more likely to use all the tools we asked about. Interestingly, the biggest gap is in use of value metrics. This could be read to mean that people become believers after they measure the value of a central database, or that people set up measurements after they decide they need to prove their beliefs. My theology is pretty rusty but surely there’s a standard debate about whether faith or action comes first.

Regardless of the exact reasons for the different attitudes, the fundamental insight here is that people who consider a single view important act quite differently from people who don’t. This means that if you’re trying to sell a customer database, either in your own company or as a vendor, you need to understand who falls into which category and address them in appropriate terms. And I guess a little prayer never hurt.

3. Tools matter.

We’ve already seen that believers have more databases and have more tools, so you won’t be surprised that using more tools correlates directly with having or planning a database.

Let's introduce the tools formally.  Here are the exact definitions we used and the percentage of people who said each was present in their organization:

Of course, the really interesting question isn’t which tools are most popular but which actually contribute (or at least correlate) with deploying a database. We looked at tool use for three groups: people with a database, people planning a database, and people with no such plans. 

Over all, results for the different tools were pretty similar: people who used each tool were much more likely to have a database and somewhat more likely to plan to build one. The pattern is a bit jumbled for Centers of Excellence and technology standards, but the numbers are small so the differences may not be significant. But it's still worth noting that Centers of Excellence are really tools to diffuse expertise in using marketing technology and don’t have too much to do with actually creating a customer database.

If you’re looking for a dog that didn’t bark, you might have expected companies using agile to be exceptionally likely to either have a database or be planning one. All quiet on that front: the numbers for agile look like numbers for long term planning and value metrics, adjusting for relative popularity. So agile is helpful but not a magic bullet.

What have we learned here? 

Clearly, we've learned that management tools are important and that long term planning in particular both the most common and the best predictor of success.

We also found that tools aren’t enough: managers need to be convinced that a unified customer view is important before they’ll invest in a database or tools to build it.

And, going back to the beginning, we saw that there are many forms of unified data, varying in how data is shared, where it’s stored, how it’s unified, and how it’s used. While it’s easy enough to assume that tight, real-time integration is needed to provide unified omni-channel customer experiences, many marketers would be satisfied with much less. I’d personally hope to see more but, as every good missionary knows, people move towards enlightenment in many small steps.

Pega Customer Decision Hub Offers High-End Customer Journey Orchestration

My previous posts about Journey Orchestration Engines (JOEs) have all pointed to new products. But some older systems qualify as well. In some ways they are even more interesting because they illustrate a mature version of the concept.

The Customer Decision Hub from Pega (formerly PegaSystems) is certainly mature: the product can trace its roots back well over a decade, to a pioneering company called KiQ Limited, which was purchased in 2004 by Chordiant, which Pega purchased in 2010. Obviously the system has been updated many times since then but its core approach to optimizing real-time decisions across all channels has stayed remarkably constant. Indeed, some features the product had a decade ago are still cutting edge today – my favorite is simulation of proposed decision rules to assess their impact before deployment.

Pega positions Customer Decision Hub as part of its core platform, which supports applications for marketing, sales automation, customer service, and operations. It competes with the usual enterprise suspects: Adobe, Oracle, Salesforce.com, IBM, and SAS. Even more than those vendors, Pega focuses on selling to large companies, describing its market as primarily the Fortune 3000. So if you’re not working at one of those firms, consider the rest of this article a template for what you might look for elsewhere.

The current incarnation of Customer Decision Hub ihas six components: Predictive Analytics Director to build offline predictive models, Adaptive Decision Manager to build self-learning real-time models, Decision Strategy Manager to set rules for making decisions, Event Strategy Manager to monitor for significant events, Next Best Action Advisor to deliver decisions to customer-facing systems, and Visual Business Director for planning, simulation, visualization, and over-all management. From a journey orchestration perspective, the most interesting of these are Decision Strategy Manager and Event Strategy Manager, because they’re the pieces that select customer treatments. The other components provide inputs (Predictive Analytics Director and Adaptive Decision Manager), support execution (Next Best Action Advisor), or give management control (Visual Business Director).

Decision Strategy Manager is where the serious decision management takes place. It brings together audiences, offers, and actions. Audiences can be built using segmentation rules or selected by predictive models. Offers can include multi-step flows with interactions over time and across channels. Actions can be anything, not just marketing messages, and may include doing nothing. They are selected using arbitration rules that specify the relevance of each action to an audience, rank the action based on eligibility and prioritization, and define where the action can be delivered.

The concept of “relevance” is what qualifies Decision Hub as a JOE. It measures the value of each action against the customer’s current needs and context,. This is the functional equivalent of defining journey stages or customer states, even though Pega doesn’t map how customers move from one state to another. The interface to set up the arbitration rules is where Decision Hub’s maturity is most obvious. For example, users can build predictive model scores into decision rules and can set up a/b tests within the arbitration to compare different approaches.

Event Strategy Manager lets users define events based on data patterns, such as three dropped phone calls within a week. These events can trigger specific actions or factor into a decision strategy arbitration. It’s another way of bringing context to bear and thus of ensuring each decision is appropriate to the customer’s current journey stage. Like arbitration rules in Decision Strategy Manager, the event definitions in Event Strategy Manager can be subtle and complex. The system is also powerful in being able to connect to nearly any type of data stream, including social, mobile, and Internet of Things devices as well as traditional structured data.

I won't go into details of other Decision Hub components, but they’re equally advanced. Companies with the scale to afford the system can expect it to pay for itself: in one published study, the three-year cost was $7.7 million but incremental revenue was $362 million. Pega says few deployments cost less than $250,000 and most are over $1 million. As I say, this isn’t a system for everyone. But it does set a benchmark for other options.

HubSpot Announces LinkedIn, Facebook Partnerships and Free Marketing Automation Edition at INBOUND Conference

HubSpot held its annual INBOUND conference in Boston last week. Maybe it's me, but the show seemed to lack some of its usual self-congratulatory excitement: for example, CEO Brian Halligan didn’t present the familiar company scorecard touting growth in customers and revenues. (A quick check of financial reports shows those are just fine: the company is expecting about 45% revenue increase for 2016.) Even the insights that Halligan and co-founder Dharmesh Shah presented in their keynotes seemed familiar: I'm guessing you've already heard that video, social, messaging, free trials, and chatbots will be big.

My own attention was more focused on the product announcements. The big news was a free version of HubSpot’s core marketing platform, joining free versions already available of its CRM and Sales systems. (In Hubspeak, CRM is the underlying database that tracks and manages customer interactions, while Sales is tools for salesperson productivity in email and elsewhere.)  Using free versions to grow marketing automation has consistently failed in the past, probably because people attracted by a free system aren't willing to do the substantial work needed for marketing automation success.  But HubSpot managers are aware of this history and seem confident they have a way to cost-effectively nurture a useful fraction of freemium users towards paid status. We'll see.

The company also announced enhancements to existing products. Many were features that already exist in other mid-tier systems, including branching visual workflows, sessions within Web analytics reports, parent/child relationships among business records, and detailed control over user permissions. As HubSpot explained it, the modest scope of these changes reflects a focus on simplifying the system rather than making it super-powerful. One good example of this attitude was a new on-site chat feature, which seems basic enough but has some serious hidden cleverness in automatically routing chat requests to the right sales person, pulling up the right CRM record for the agent, and adding the chat conversation to the customer history.

One feature that did strike me as innovative was closer to HubSpot’s roots in search marketing: a new “content strategy” tool reflecting the shift from keywords to topics as the basis of search results. HubSpot’s tool helps marketers find the best topics to try to dominate with their content.  This will be very valuable for marketers unfamiliar with the new search optimization methods. Still, what you really want is a system that helps you create that content.  HubSpot does seem to be working on that.

With relatively modest product news, the most interesting announcements at the conference were probably about HubSpot’s alliances.  A new Facebook integration lets users create Facebook lead generation campaigns within HubSpot and posts leads from those campaigns directly to the HubSpot database. A new LinkedIn integration shows profiles from LinkedIn Sales Navigator within HubSpot CRM screens for users who have a Sales Navigator subscription. Both integrations were presented as first steps towards deeper relationships. These relationships reflect the growing prominence of HubSpot among CRM/marketing automation vendors, which gives companies like Microsoft and LinkedIn a reason to pick HubSpot as a partner. This, in turn, lets HubSpot offer features that less well-connected competitors cannot duplicate. That sets up a positive cycle of growth and expansion that is very much in HubSpot’s favor.

As an aside, the partnerships raise the question of whether Microsoft might just purchase HubSpot and use it to replace or supplement the existing Dynamics CRM products. Makes a lot of sense to me.  A Facebook purchase seems unlikely but, as we also learned last week, unlikely things do sometimes happen.

ActionIQ Merges Customer Data Without Reformatting

One of the fascinating things about the Customer Data Platform Institute is how developers from different backgrounds have converged on similar solutions. The leaders of ActionIQ, for example, are big data experts: Tasso Argyros founded Aster Data, which was later purchased by Teradata, and Nitay Joffe was a core contributor to HBase and the data infrastructure at Facebook.  In their previous lives, both saw marketers struggling to assemble and activate useful customer data. Not surprisingly, they took a database-centric approach to solving the problem.

What particularly sets ActionIQ apart is the ability to work with data from any source in its original structure. The system simply takes a copy of source files as they are, lets users define derived variables based on those files, and uses proprietary techniques to query and segment against those variables almost instantly. It’s the scalability that’s really important here: at one client, ActionIQ scans two billion events in a few seconds. Or, more precisely, it’s the scalability plus flexibility: because all queries work by re-reading the raw data, users can redefine their variables at any time and apply them to all existing data. Or, really, it's scalability, flexibility, and speed, because new data is available within the system in minutes.

So, amongst ActionIQ’s many advantages are scalability, flexibility, and speed. These contrast with systems that require users to summarize data in advance and then either discard the original detail or take much longer to resummarize the data if a definition changes.

ActionIQ presents its approach as offering self-service data access for marketers and other non-technical users. That’s true insofar as marketers work with previously defined variables and audience segments. But defining those variables and segments in the first place takes the same data wrangling skills that analysts have always needed when faced with raw source data. ActionIQ reduces work for those analysts by making it easier to save and reuse their definitions. Its execution speed also reduces the cost of revising those definitions or creating alternate definitions for different purposes. Still, this is definitely a tool for big companies with skilled data analysts on staff.

The system does have some specialized features to support marketing data. These include identity resolution tools including fuzzy matching of similar records (such as different versions of a mailing address) and chaining of related identifiers (such as a device ID linked to an email linked to an account ID). It doesn’t offer “probabilistic” linking of devices that are frequently used in the same location although it can integrate with vendors who do. ActionIQ also creates correlation reports and graphs showing the relationship between pairs of user-specified variables, such as a customer attribute and promotion response. But it doesn’t offer multi-variable predictive models or machine learning.

ActionIQ gives users an interface to segment its data directly. It can also provide a virtual database view that is readable by external SQL queries or PMML-based scoring models. Users can also export audience lists to load into other tools such as campaign managers, Web ad audiences, or Web personalization systems. None of this approaches the power of the multi-step, branching campaign flows of high-end marketing automation systems, but ActionIQ says most of its clients are happy with simple list creation. Like most CDPs, ActionIQ leaves actual message delivery to other products.

The company doesn’t publicly discuss the technical approach it takes to achieve its performance, but they did describe it privately and it makes perfect sense. Skeptics should be comforted by the founders’ technical pedigree and demonstrated actual performance. Similarly, ActionIQ asked me not to share screen shots of their user interface or details of their pricing. Suffice to say that both are competitive.

ActionIQ was founded in 2014 and has been in production with its pilot client for over one year. The company formally launched its product last month.

Walker Sands / Chief Martech Study: Martech Maturity Has Skyrocketed

Tech marketing agency Walker Sands and industry guru Scott Brinker of Chief Martech yesterday published a fascinating survey on the State of Marketing Technology 2017, which you can download here.  The 27 page report provides an insightful analysis of the data, which there’s no point to me duplicating in depth. But I will highlight a couple of findings that are most relevant to my own concerns.

Martech maturity has skyrocketed in the past year. This theme shows up throughout the report. The percentage of responders classifying their companies as innovators or early adopters grew from 20% in 2016 to 48% in 2017; marketers whose companies invest the right amount in marketing technology grew from 50% to 71%; all obstacles to adoption were less common (with the telling exception of not needing anything new).

Truth be told, I find it hard to believe that things can have shifted this much in a single year and that nearly half of all companies (and 60% of individual marketers) are innovators or early adopters. A more likely explanation is the new survey attracted more advanced respondents than before.  We might also be seeing a bit of “Lake Wobegon Effect,” named after Garrison Keillor’s mythical town where all the children are above average.  Evidence for the latter might be that 69% felt their marketing technology is up to date and sufficient (up from 58%), making this possibly the most complacent group of innovators ever.


Multi-product architectures are most common. I have no problem accepting this one: 21% of respondents said they use a single-vendor suite, while 69% had some sort of multi-vendor approach (27% integrated best-of-breed, 21% fragmented best-of-breed, 21% limited piecemeal solutions). The remainder had no stack (7%) or proprietary technology (4%).

 

But don’t assume that “single-vendor suite” necessarily means one of the enterprise marketing clouds.  Small companies reported using suites just as often as large ones. They were probably referring to all-in-one products like HubSpot and Infusionsoft.

"Best of breed marketers get the most out of their martech tools." That’s a direct quote from the report, but it may overstate the case: 83% of integrated best-of-breed users felt their company was good or excellent at leveraging the stack, compared with 76% of the single-vendor-suite. That not such a huge difference, especially given the total sample of 335. Moreover, companies with fragmented best-of-breed stacks reported less ability (67%) than the single-vendor suite users. If you combine the two best-of-breed groups then the suite users actually come out ahead. A safer interpretation might be that single-vendor suites are no easier to use than best-of-breed combinations.  That would still be important news to companies that think pay a premium or compromise on features because they think suites make are easier to deploy.
  

Integration isn’t that much of a problem. Just 20% of companies cited better stack integration as a key to fully leveraging their tools, which ranked well behind better strategy (39%), better analytics (36%) and more training (33%) and roughly on par with more employees (23%), better defined KPIs (23%), and more data (20%). This supports the previous point about best-of-breed working fairly well, whether or not the stack was well integrated. I would have expected integration to be a bigger issue, so this is a bracing reality check. One interpretation (as I argued last week) is that integration just isn’t as important to marketers as they often claim.

There’s plenty else of interest in the report, so go ahead and read it and form your own opinions. Thanks to Walker Sands and Chief Martech for pulling it together.

Singing the Customer Data Platform Blues: Who's to Blame for Disjointed Customer Data?

I’m in the midst of collating data from 150 published surveys about marketing technology, a project that is fascinating and stupefying at the same time. A theme related to marketing data seems to be emerging that I didn’t expect and many marketers won’t necessarily be happy to hear.

Most surveys present a familiar tune: many marketers want unified customer data but few have it. This excerpt from an especially fine study by Econsultancy makes the case clearly although plenty of other studies show something similar.

So far so good. The gap is music to my ears, since helping marketers fill it keeps consultants like me in the business. But it inevitably raises the question of why the gap exists.

The conventional answer is it’s a technology problem. Indeed, this Experian survey makes exactly that point: the top barriers are all technology related.

And, comfortingly, marketers can sing their same old song of blaming IT for failing to deliver what they need.  For example, even though 61% of companies in this Forbes Insights survey had a central database of some sort, only 14% had fully unified, accessible data.

But something sounds a little funny. After all, doesn’t marketing now control its own fate? In this Ascend2 report, 61% of the marketing departments said they were primarily responsible for marketing data and nearly all the other marketers said they shared responsibility.

Now we hear that quavering note of uncertainty: maybe it’s marketing’s own fault? That’s something I didn’t expect. And the data seems to support it. For example, a study from Black Ink ROI found that the top barrier to success was better analytics (which implicitly requires better data) and explicitly listed data access as the third-ranked barrier.

But – and here’s the grand finale – the same study found that data integration software ranked sixth on the marketers’ shopping lists. In other words, even though marketers knew they needed better data, they weren’t planning to spend money to make it happen. That’s a sour chord indeed.

But the song isn't over.  If we listen closely, we can barely make out one final chorus: marketers won’t invest in data management technology because they don’t have the skills to use it. Or that’s what this survey from Falcon.io seems to suggest.

In its own way, that’s an upbeat ending. Expertise can be acquired through training or hiring outside experts (or possibly even mending some fences with IT). Better tools, like Customer Data Platforms, help by reducing the expertise needed. So while marketers aren't strutting towards a complete customer view with a triumphal Sousa march, there’s no need for a funeral dirge quite yet.

Survey on Customer Data Management - Please Help!

I'm working with MarTech Advisor on a survey to understand the state of customer data management.  If you have five minutes or so, could you please fill it out?  Link is here.  And if possible, pass on to people in other companies who could also help.  You'll get a copy of the final report and my gratitude.  Just for reading this far, here's a kitten:

Hull.io Offers A Customer Data Platform for B2B Marketers

The need for a Customer Data Platform – a marketer-controlled, unified, persistent, accessible customer database – applies equally to business and consumer marketing. Indeed, many of the firms I originally identified as CDPs were lead scoring and customer success management vendors who serve primarily B2B clients. But as the category has evolved, I’ve narrowed my filter to only consider CDPs as companies that focus primarily on building the unified data.  This excludes the predictive modeling vendors and customer success managers, as well as the big marketing clouds that list a CDP as one of many components. Once you apply that filter, nearly all the remaining firms sell largely to B2C enterprise clients.

Hull.io is an exception. Its clients are mostly small, B2B companies – exactly the firms that were first to adopt software-as-a-service (SaaS) technologies including marketing automation and CRM. This is no accident: SaaS solves one problem by making it easy to acquire new systems, but that creates another problem because those systems are often isolated from each other. Hull addresses that problem by unifying their data, or, more precisely, by synchronizing it.

How it works is this: Hull has connectors for major customer-facing SaaS systems, such as Salesforce, Optimizely, HubSpot, Mailchimp, Facebook custom audiences, Slack, and Zendesk. Users connect with those systems and specify data elements or lists to synchronize. When data changes in one of customer-facing products, the change is sent to Hull which in turn sends it to other products that are tracking that data.

But, unlike data exchanges such as Zapier or Segment, Hull also keeps its own copy of the data. That’s the “persistent” bit of the CDP definition. It gives Hull a place to store data from enhancement vendors including Datanyze and Clearbit, from external processes called through Javascript, and from user-defined custom variables and summary properties, such as days since last visit. Those can be used along with other data to create triggers and define segments within Hull.  The segments can then be sent to other systems and updated as they change.

In other words, even though the external systems are not directly reading the data stored within Hull, they can still all work with consistent versions of the data.* Think of it as the martech equivalent of Einstein’s’ “spooky action at a distance”  if that clarifies things for you.

To extend its reach even further, Hull.io can also integrate with Zapier and Segment allowing it to exchange data with the hundreds of systems those products support.

Three important things have to happen inside of Hull.io to provide a unified customer view. First, it has to map data from different sources to a common data model – so that things like customer name or product ID are recognized as referring to the same entities even if they come from different places. Hull.io simplifies this as much as possible by limiting its internal data model to two entities, customers and events.  Input data, no matter how complicated, is converted to these entities by splitting each record into components that are tagged with their original meaning and relationships. The splitting and tagging are automatic, which is very important for making the system easy to deploy and maintain.  Users still need to manually tell the system which elements from different systems should map to the same element in the shared data.

The second important thing is translating stored data into the structure needed by the receiving system. This is the reverse of the data loading process, since complex records must be assembled from the simplified internal model. What’s tricky is that the output format is almost always different from the input format, so the pieces have to be reassembled in a different format.  While we’re making questionably helpful analogies, think of this as the Jive Lady translating for the sick passenger in the movie Airplane.

The third key thing is that data relating to the same customer needs to be linked. Hull will do “deterministic” matching to stitch together identities where overlapping information is available – such as, connecting an account ID to a device when someone uses that device to log into their account. Like many other CDPs, Hull doesn’t attempt “probabilistic” matching, which looks for patterns in behavior or data to associate identifiers that are likely to belong to the same person. It does use IP address to associate visitors with businesses, even if the individual is anonymous.

All told, this adds up to a respectable set of CDP features. But Hull co-founder Romain Dardour says few clients actually come to the company looking for a unified, persistent customer database. Rather, they are trying to create specific processes, such as using Slack to send notifications of support tickets from Zendesk. Hull has built a collection of these processes, which it calls recipes. Customers can use an existing recipe or design their own. Dardour said that once clients deploy a few recipes they usually recognize the broader possibilities of the system and migrate towards thinking of it as a true CDP, even if they still don’t use the term.

This is consistent with what I’ve seen elsewhere.  Big enterprises can afford to purchase a unified customer database by itself, but smaller firms often want their CDP to include a specific money-making application. That’s why my original B2B CDPs usually included applications like lead scoring and customer success, while the B2C enterprise CDPs often did not.

The other big divide between Hull and enterprise CDPs is cost. Most enterprise CDPs start somewhere between $100,000 and $250,000 per year and can easily reach seven figures. Hull starts as low as $500 per month, with a current average of about $1,000 and the largest clients topping out around $10,000. Price is based primarily on the number of system connections, with some adjustments for number of contact records, guaranteed response time, data retention period, and special features. Hull has over 1,000 clients, mostly in the U.S. but with world-wide presence. It was founded in 2013.


_________________________________________________________________________________
*You could argue that because the external systems are not reading Hull.io’s data directly, it doesn’t truly qualify as a CDP. I’d say it’s not worth the quibble – although if really massive amounts of data were involved, it might be significant. Remember that Hull.io is dealing with smaller businesses, where replicating all the relevant data is not a huge burden.

Datorama Applies Machine Intelligence to Speed Marketing Analytics

As I mentioned a couple of posts back, I’ve been surveying the borders of Customer Data Platform-land recently, trying to figure out which vendors fit within the category and which do not. Naturally, there are cases where the answer isn’t clear. Datorama is one of them.

At first glance, you’d think Datorama is definitely not a CDP: it positions itself as a “marketing analytics platform” and makes clear that its primary clients are agencies, publishers, and corporate marketers who want to measure advertising performance. But the company also calls itself a “marketing integration engine” that works with “all of your data”, which certainly goes beyond just advertising. Dig a bit deeper and the confusion just grows: the company works mostly with aggregated performance data, but also works with some individual-level data.  It doesn’t currently do identity resolution to build unified customer profiles, but is moving in that direction. And it integrates with advertising and Web analytics data on one hand and social listening, marketing automation, and CRM on the other. So while Datorama wasn’t built to be a CDP – because unified customer profiles are the core CDP feature – it may be evolving towards one.

This isn't to say that Datorama lacks focus. The system was introduced in 2012 and now has over 2,000 clients, including brands, agencies, and publishers. It grew by solving a very specific problem: the challenges that advertisers and publishers face in combining information about ad placements and results. Its solution was to automate every step of the marketing measurement process as much as it could, using machine intelligence to identify information within new data sources, map those to a standard data model, present the results in dashboards, and uncover opportunities for improvement. In other words, Datorama gives marketers one system for everything from data ingestion to consolidation to delivery to analytics.  This lets them manage a process that would otherwise require many different products and lots of technical support. That approach – putting marketers in control by giving them a system pre-tailored to their needs – is very much the CDP strategy.

Paradoxically, the main result of Datorama’s specialization is flexibility. The system’s developers set of goal of handling any data source, which led to a system that can ingest nearly any database type, API feed or file format, including JSON and XML; automatically identify the contents of each field; and map the fields to the standard data model. Datorama keeps track of what it learns about common source systems, like Facebook, Adobe Analytics, or AppNexus, making it better at mapping those sources for future implementations. It can also clean, transform, classify, and reformat the inputs to make them more usable, applying advanced features like rules, formulas, and sentiment analysis. At the other end of the process, machine learning builds predictive models to do things like estimate lifetime value and forecast campaign results. The results can be displayed in Datorama’s own interface, read by business intelligence products like Tableau, or exported to other systems like marketing automation.

Datorama’s extensive use of machine learning lets it speed up the marketing analytics process while reducing the cost. But this is still not a push-button solution. The vendor says a typical proof of concept usually takes about one month, and it takes another one to two months more to convert the proof of concept into a production deployment. That’s faster than your father’s data warehouse but not like adding an app to your iPhone. Pricing is also non-trivial: a small company will pay in the five figures for a year’s service and a large company's bill could reach into seven figures. Fees are based on data volume and number of users. Datorama can also provide services to help users get set up or to run the system for them if they prefer.

News from Krux, Demandbase, Radius: Customer Data Takes Center Stage

If Dreamforce seems a little less crowded than you expected this week, perhaps it's because I didn’t attend. But I’m still tracking the news from Salesforce and other vendors from my cave in Philadelphia. Three announcements caught my eye, all highlighting the increasing attention being paid to customer data.

Salesforce itself had the biggest news yesterday, with its agreement to purchase Krux, a data management platform that has expanded well beyond the core DMP function of assembling audiences from cookie pools. Krux now has an “intelligent marketing hub” that can also load a company’s own data from CRM, Websites, mobile apps, and offline sources, and unify customer data to build complete cross-channel profiles. Krux also allows third party data owners to sell their data through the Krux platform and offers self-service data science for exploration and predictive models. The purchase makes great strategic sense for Salesforce, providing it with a DMP to match existing components in the Oracle and Adobe marketing clouds. But beyond the standard DMP function of generating advertising audiences, Krux gives Salesforce a solid customer data foundation to support all kinds of marketing management.  In particular, it goes beyond the functions in Salesforce ExactTarget, which was previously the designated core marketing database for Salesforce Marketing Cloud. To be clear, there’s no campaign management or journey orchestration within Krux; those functions would be performed by other systems that simply draw on Krux data. Which is exactly as it should be, if marketers are to maintain maximum flexibility in their tools.

Demandbase had its own announcement yesterday: something it calls “DemandGraph,” which is basically a combination of Demandbase’s existing business database with data gathering and analytical functions the Spiderbook system that Demandbase bought in May 2016. DemandGraph isn’t exactly a product but rather a resource that Demandbase will use to power other products. It lets Demandbase more easily build detailed profiles of people and companies, including history, interests, and relationships. It can then use the information to predict future purchases and guide marketing and sales messages. There’s also a liberal sprinkling of artificial intelligence throughout DemandGraph, used mostly in Spiderbook’s processing of unstructured Web data but also in some of the predictive functions. If I’m sounding vague here it’s because, frankly, so was Demandbase. But it’s still clear that DemandGraph represents a major improvement in the power and scope of data available to business marketers.

Predictive marketing vendor Radius made its announcement last week of the Radius Customer Exchange.  This uses the Radius Business Graph database (notice a naming trend here?) to help clients identify shared customers without exposing their entire files to each other. Like Spiderbook, Radius gathers much of its data by scanning the public Web; however, Radius Business Graph also incorporates data provided Radius clients. The client data provides continuous, additional inputs that Radius says makes its data and matching much more accurate than conventional business data sources. Similarly, while there’s nothing new about using third parties to find shared customers, the Radius Customer Exchange enables sharing in near real time, gives precise revocable control over what is shared, and incorporates other information such as marketing touches and predictive models. These are subtle but significant improvements that make data-driven marketing more effective than ever. The announcement also supports a slight shift in Radius’ position from “predictive modeling” (a category that has lost some of its luster in the past year) to “business data provider”, a category that seems especially enticing after Microsoft paid $26.2 billion for LinkedIn.

Do these announcements reflect a change in industry focus from marketing applications to marketing data? I’m probably too data-centric to be an objective judge, but a case could be made. If so, I’d argue it’s a natural development as marketers look beyond the endless supply of sparkly new Martech applications to the underlying foundations needed to support them. In the long run, a solid foundation makes it easier to dance creatively along the surface: so I’d rate a new data-driven attitude as a Good Thing.

Reltio Makes Enterprise Data Usable, and Then Uses It

I’ve spent a lot of time recently talking to Customer Data Platform vendors, or companies that looked like they might be. One that sits right on the border is Reltio, which fits the CDP criteria* but goes beyond customer data to all types of enterprise information. That puts it more in the realm of Master Data Management, except that MDM is highly technical while Reltio is designed to be used by marketers and other business people. You might call it “self-service MDM” but that’s an oxymoron right up there with “do-it-yourself brain surgery”.

Or not. Reltio avoids the traditional complexity of MDM in part by using the Cassandra data store, which is highly scalable and can more easily add new data types and attributes than standard relational databases. Reltio works with a simple data model – or graph schema if you prefer – that captures relationships among basic objects including people, organizations, products, and places. It can work with data from multiple sources, relying on partner vendors such as SnapLogic and MuleSoft for data acquisition and Tamr, Alteryx, and Trifacta for data preparation. It has its own matching algorithms to associate related data from different sources. As for the do-it-yourself bit: well, there’s certainly some technical expertise needed to set things up, but Reltio's services team generally does the hard parts for its clients. The point is that Reltio reduces the work involved – while adding a new source to a conventional data warehouse can easily take weeks or months, Reltio says it can add a new source to an existing installation in one day.

The result is a customer profile that contains pretty much any data the company can acquire. This is where the real fun begins, because that profile is now available for analysis and applications. These can also be done in Reltio itself, using built-in machine learning and data presentation tools to provide deep views into customers and accounts, including recommendations for products and messages. A simple app might take one or two months to build; a complicated app might take three or four months. The data is also available to external systems via real-time API calls.

Reltio is a cloud service, meaning the system doesn’t run on the client’s own computers. Pricing depends on the number of users and profiles managed but not the number of sources or data volume. The company was founded in 2011 and released its product several years later. Its clients are primarily large enterprises in retail, media, and life sciences.

______________________________________________________________________
* marketer-controlled; multi-source unified persistent data; accessible to external systems

History of Marketing Technology and What's Special about Journey Orchestration

I delivered my presentation on the history of marketing technology last week at the Optimove CONNECT conference in Tel Aviv. Sadly, the audience didn’t seem to share my fascination with arcana (did you know that the Chinese invented paper in 100 CE? that Return on Investment analysis originated at DuPont in 1912?) So, chastened a bit, I’ll share with you a much-condensed version of my timeline, leaving out juicy details like brothel advertising at Pompeii.

The timeline* traces three categories: marketing channels; tools used by marketers to manage those channels; and data available to marketers.  The yellow areas represent the volume of technology available during each period. Again skipping over my beloved details, there are two main points:

• although the number of marketing channels increased dramatically during the industrial age (adding mass print, direct mail, radio, television, and telemarketing), there was almost no growth in marketing technology or data until computers were applied to list management in the 1970’s. The real explosions in martech and data happen after the Internet appears in the 1990’s.

• the core martech technology, campaign management, begins in the 1980’s: that is, it predates the Internet. In fact, campaign management was originally designed to manage direct mail lists (and – arcana alert! – itself mimicked practices developed for mechanical list technologies such as punch cards and metal address plates). Although marketers have long talked about being customer- rather than campaign-centric, it’s not until the current crop of Journey Orchestration Engines (JOEs) that we see a thorough replacement of campaign-based methods.

It’s not surprising the transition took so long. As I described in my earlier post on the adoption of electric power by factories (more arcana!), the shift to new technology happens in stages as individual components of a process are changed, which then opens a path to changing other components, until finally all the old components are gone and new components are deployed in a configuration optimized for the new capabilities. In the transition from campaign management to journey orchestration, marketers had to develop tools to track individuals over time, to personalize messages to those individuals, identify and optimize individual journeys, act on complete data in real time, and to incorporate masses of unstructured data. Each of those transitions involved a technology change: from lists to databases, from static messages to dynamic content, from segment-level descriptive analytics to individual-level predictions, from batch updates to real time processes, and from relational databases to “big data” stores.

It’s really difficult to retrofit old systems with new technologies, which is one reason vendors like Oracle and IBM keep buying new companies to supplement current products. It’s also why the newest systems tend to be the most advanced.** Thus, the Journey Orchestration Engines I’ve written about previously (Thunderhead ONE , Pointillist, Usermind, Hive9 ) all use NoSQL data stores, build detailed individual-level customer histories, and track individuals as they move from state to state within a journey flow.

During my Tel Aviv visit last week, I also checked in with Pontis (just purchased by Amdocs), who showed me their own new tool which does an exceptionally fine job at ingesting all kinds of data, building a unified customer history, and coordinating treatments across all channels, all in real time. In true JOE fashion, the system selects the best treatment in each situation rather than pushing customers down predefined campaign sequences. Pontis also promised their February release would use machine learning to pick optimal messages and channels during each treatment. Separately, Optimove itself announced its own “Optibot” automation scheme, which also finds the best treatments for individuals as they move from state to state. So you can add Optimove to your cup of JOEs (sorry) as well.

I’m reluctant to proclaim JOEs as the final stage in customer management evolution only because it’s too soon to know if more change is on the way. As Pontis and Optimove both illustrate, the next step may be using automation to select customer treatments and ultimately to generate the framework that organizes those treatments. When that happens, we will have erased the last vestiges of the list- and campaign-based approaches that date back to the mail order pioneers of the 19th century and to the ancient Sumerians (first customer list, c. 3,000 BCE) before that.




_________________________________________________________________________________
*Dates represent commercialization, not the first appearance of the underlying technology. For example, we all know that Gutenberg’s press with moveable type was introduced around 1450, but newspapers with advertising didn’t show up until after 1600.

** This isn’t quite as tautological as it sounds. In some industries, deep-pocketed old vendors with big research budgets are the technical leaders. 

How Quickly Is the MarTech Industry Growing?

Everyone in marketing knows there’s a lot of new marketing technology, but how quickly is martech really growing? Many people cite changes in Scott Brinker’s iconic marketing technology landscape, which has roughly doubled in size every year since Brinker first published it in 2011. Brinker himself is always careful to stress that his listings are not comprehensive, and anyone familiar with the industry will quickly realize much of the growth in his vendor count reflects greater thoroughness and broader scope rather than appearance of new vendors. But no matter how many caveats are made, the ubiquity of Brinker’s chart leaves a strong impression of tremendously quick expansion.

Fortunately, other data is available. Venture Scanner recently published the the number of companies founded by year for 1,295 martech firms in its database. This shows growth of around 12% per year from 2000 through 2012. (Figures for 2013 and later are almost surely understated because many firms started during those years have not yet been included in the data.)

A similar analysis from CabinetM, which has a database of 3,708 companies, showed a slightly higher rate of 14.5% per year for the same period.* Both sets of data show a noticeable acceleration after 2006: to about 16.5% for Venture Scanner and just under 16% for CabinetM.
 

These figures are still far from perfect. Many firms are obviously missing from the Venture Scanner data. CabinetM has apparently missed many as well: Brinker reported that comparison between CabinetM’s list and his own found that each had about 1,900 vendors the other did not. All lists will miss companies that are no longer in business, so there were probably more start-ups in each year than shown.

But even allowing for such issues, it’s probably reasonable to say that the number of vendors in the industry has been growing at something from 15% to 20% per year. That’s a healthy rate but nothing close to an annual doubling.

Note also that we’re talking here about the number of companies, not revenue.  I suspect revenue is growing more quickly than the number of vendors but can't give a meaningful estimate of how much.

Are particular segments within the industry growing faster than others? CabinetM provided me with a breakdown of starts by year by category.** To my surprise, growth has been spread fairly evenly across the different types of systems. Adtech grew a bit faster than the other categories in 2006 to 2010 and content marketing has grown faster than the average since 2006. But the share of marketing automation and operations have been surprisingly consistent throughout the period covered. So while the number of marketing automation vendors has indeed grown quickly, other categories seem to growing at about the same pace.

So what, if anything, does this tell us about the future?  It's certainly possible some of the drop-off in new vendors since 2013 reflects an actual slowdown in addition to the lag time before new vendors appear in databases. Funding data from Venture Scanner suggests that 2015 may have been a peak year for investments, although 2016 data is obviously incomplete.

 

Another set of funding data, from PitchBook, suggests 2014 was a peak but shows much less year-on-year variation than Venture Scanner. The inconsistency between the two sets of data makes it hard to accept either source as definitive.

So, what does this all mean?  First of all, that people should calm down a bit: the number of martech vendors hasn't been doubling every year.  Second, that industry growth may indeed be slowing, although it's too soon to say for sure.  Third, whatever the exact figures, there are plenty of martech vendors out there and they're not going away any time soon.  So marketers need to focus on a systematic approach to martech acquisition, balancing new opportunities against training and integration costs.
__________________________________________________________________________________
* Here's the actual CabinetM data.  I'm mostly showing this to clarify that my "growth rate" is comparing the number of new companies vs. total industry size, and not the number of new companies this year vs new companies last year.

**CabinetM actually tracks 30 categories.  I combined them into the seven groups used here.

Will Marketing Technologists Kill Martech?

I’ll be giving a speech next week on the evolution of marketing technology, which doesn't follow the path you might think. The new channels that appear on a typical “history of marketing timeline”, such as radio in the 1920’s and TV in the 1950’s, didn’t really trigger any particular changes in the technology used by marketers: planning was still done on paper spreadsheets and copy was typed manually up to the 1970’s. Similarly, marketers up that time worked with the same data – audience counts and customer lists – they had since Ben Franklin and before.

It was only in the 1960’s, when mailing lists were computerized, that new technologies begin to make more data available and marketers get new tools to work with it. Those evolved slowly – personalized printing and modern campaign managers appeared in the 1980’s. The big changes started in the 1990’s when email and Web marketing provided a flood of data about customer behaviors and vendors responded with a flood of new systems to work with it. But it wasn't until the late 2000’s that the number of vendors truly exploded.

I can’t prove this, but I think what triggered martech hypergrowth was Software-as-a-Service (SaaS). This made it easy for marketers to purchase systems without involving the corporate IT department, allowing users to buy tools that solved specific problems whether or not the tools fit into the corporate grand scheme of things. Major SaaS vendors, most notably Salesforce.com, made their systems into platforms that provided a foundation for other systems. This freed developers to create specialized features without building a complete infrastructure. Building apps on platforms also sharply reduced integration costs, which had placed a severe limit on how many systems any marketing department could afford. Easier development, easier deployment, and easier acquisition created perfect environment for martech proliferation.

But every action has a reaction. The growth of martech led to the hiring of marketing technologists, as marketing departments realized they needed someone to manage their burgeoning technology investments. That might seem like a good thing for the martech industry, but it introduced a layer of supervision that restrained the free-wheeling purchases that marketers had been  making on their own. After all, the job of a martech manager is to rationalize and coordinate martech investments, which ultimately means saying “no.”

The quest for rationalization leads to long-term planning, vision development, architecture design, corporate standards, and project prioritization: all the excellent practices that made corporate IT departments so unresponsive to marketers in the first place. The scrappy rebels in martech departments hear the call of order-obsessed dark side and find it increasingly hard to resist.

And it only gets worse (from the martech vendor point of view). As marketing technologists discover just how many systems are already in place, they inevitably ask how they can make things simpler. The equally inevitable answer is to buy fewer systems by finding systems that do more things. This leads to integrated suites – marketing clouds, anyone? – that may not have the best features for any particular function but offer a broad range of capabilities. When the purchase is made by individual marketers focused on their own needs, the best features will win and small, innovative martech vendors can flourish. But when purchases are managed by the central martech department, integration and breadth will weigh more heavily in the decision.  This gives bigger, most established firms the advantage.

In short, martech today is at a crossroads. Martech managers can follow the natural logic of their positions, which leads to greater centralization, large multi-function systems, and increasingly frustrated marketers. Or they can retain their agility and support new, innovative martech vendors, recognizing that near-term efficiency will suffer. Put so starkly, it’s obvious that agility is the better choice, and there is plenty of discussion in the industry of how to maintain it. But the dark side is powerful, relentless, and seductively rational. Martech managers – and the marketers they ultimately serve – must tread carefully to stay on the right path.

ABM Vendor Guide: Differentiators for Result Analysis

...and we wrap up our review of sub-functions from the Raab Guide to ABM Vendors with a look at Result Analysis.

ABM Process	System Function	Sub-Function	Number of Vendors
Identify Target Accounts	Assemble Data	External Data	28
	Select Targets	Target Scoring	15
Plan Interactions	Assemble Messages	Customized Messages	6
	Select Messages	State-Based Flows	10
Execute Interactions	Deliver Messages	Execution	19
Analyze Results	Reporting	Result Analysis	16

As the Guide points out, this category focuses on measurements unique to account-based programs:

Nearly every system will have some form of result reporting. ABM specialists provide account-based result metrics such as percentage of target accounts reached, amount of time target accounts are spending with company messages, and distribution of messages by department within target accounts.

Not surprisingly, most of vendors who do ABM Result Analysis also do some sort of Execution (12 out of 16, to be exact). Another two (Everstring and ZenIQ) didn't fall into the Execution group but came close.  Of the final two, one supports measurement with advanced lead-to-account mapping (LeanData) and one attribution specialist (Bizible). It's important to recognize that many of the Execution vendors will report only results of their own messages.  This is certainly helpful but you'll want to see reports that combine data from all messages to get a meaningful picture of your ABM program results. 

Differenatiators for this group include:

• lead-to-account mapping to unify data
• corporate hierarchy mapping (headquarters/branch, parent/subsidiary, etc.) to unify data
• marketing campaign to opportunity mapping to support attribution
• combine data from marketing automation, Web analytics, and CRM
• track offline channels such as conferences, direct mail, outbound hone calls
• capture detailed interaction history for each Web visit (mouse clicks, scrolling, active time spent, etc.)
• capture mobile app behaviors with SDK as well as Web site behaviors with Javascript tag
• use device ID to link display ads, Web site visits, and form fills to revenue, even when visitors don’t click on ad or Web page
• report within Salesforce CRM on combined information about leads, contacts, accounts, opportunities, campaigns, and owners
• apply multiple attribution methods including first touch, last touch, fractional, etc.
• show account-level descriptive metrics including coverage, contact frequency, visitors, contacts by job title
• show account-level result metrics including reach, engagement, influence, velocity
• show reach, engagement, influence, velocity by campaign, content, persona, segment, etc.
• identify gaps in coverage, reach, or engagement by account and recommend corrective actions

One final reminder: the just-published Guide to ABM Vendors helps marketers understand what tools they need to complete their ABM stack.  It provides detailed profiles of 40 ABM vendors, with contents including:

• introduction to Account Based Marketing
• description of ABM functions
• key subfunctions that differentiate ABM vendors
• vendor summary chart that shows who does what
• explanations of information provided in the report
• vendor profiles including a summary description, list of key features, and detailed information covering  37 categories including data sources, data storage, data outputs, target selection, planning, execution, analytics, operations, pricing, and vendor background.

For more information or to order, click here.