Simple Questions to Screen Customer Data Platform Vendors

I’ve been working for months to find a way to help marketers understand the differences between Customer Data Platform vendors. After several trial balloons and with considerable help from industry friends, I recently published a set of criteria that I think will do the job. You can see the full explanation on the CDP Institute blog. But, since this blog has its own readership I figured I’d post the basics here as well.

The primary goal is give marketers a relatively easy way to decide which CDPs are likely to meet their needs. To do this I’ve come up with a a small list of features that relate directly to working with particular data sources and supporting particular applications. The theory is that marketers know what sources and applications they need to support, even if they're not experts in the fine points of CDP technology.

In other words, read these items as meaning: if you want your CDP to support [this data type or application] then it should have [this feature].

Obviously this list covers just a tiny fraction of all possible CDP features. It’s up to marketers to dig into the details of each system to determine how well it supports their specific needs.  We have detailed lists of CDP features in the Evaluation section of the CDP Institute Library.

The final list also includes a few features that are present in all CDPs (or, more precisely, in all systems that I consider a CDP – we can’t control what vendors say about themselves). These are presented since there’s still some confusion about how CDPs differ from other types of systems.

Now that the list is set, the next step is to research which features are actually present in which vendors and publish the results. That will take a while but when it’s done I’ll certainly announce it here.

Here’s the list:

Shared CDP Features: Every CDP does all of these. Non-CDPs may or may not.

• Retain original detail. The system stores data with all the detail provided when it was loaded. This means all details associated with purchase transactions, promotion history, Web browsing logs, changes to personal data, etc. Inputs might be physically reformatted when they’re loaded into the CDP but can be reconstructed if needed.
• Persistent data. The system retains the input data as long as the customer chooses. (This is implied by the previous item but is listed separately to simplify comparison with non-CDP systems.)
• Individual detail. The system can access all detailed data associated with each person. (This is also implied by the first item but is a critical difference from systems that only store and access segment tags on customer records.)
• Vendor-neutral access. All stored data can be exposed to any external system, not only components of the vendor’s own suite. Exposing particular items might require some set-up and access is not necessarily a real time query.
• Manage Personally Identifiable Information (PII). The system manages Personally Identifiable Information such as name, address, email, and phone number. PII is subject to privacy and security regulations that vary based on data type, location, permissions, and other factors.

Differentiating CDP Features: A CDP doesn’t have to do any of these although many do some and some do many. These are divided into three subclasses: data management, analytics, and customer engagement.

Data Management. These are features that gather, assemble, and expose the CDP data.

     Base Features. These apply to all types of data.

• API/query access. External systems can access CDP data via an API or standard query language such as SQL. It’s just barely acceptable for a CDP to not offer this function and instead provide access through data extracts. But API or query access is much preferred and usually available. API or query access often requires some intermediate configuration, reformatting, or indexing to expose items within the CDP’s primarily data store. Those are important details that buyers must explore separately.
• Persistent ID. The system assigns each person an internal identifier and maintains it over time despite changes or multiple versions of other identifiers, such as email address or phone number. This allows the CDP to maintain individual history over time, even when source systems might discard old identifiers. CDPs that use a persistent ID applied outside of the system do not meet this requirement.
• Deterministic match (a.k.a. “identity stitching”). The system can store multiple identifiers known to belong to the same person and link them to a shared ID (usually the persistent ID). This enables the system to connect identifiers indirectly: for example, if an email linked to an account is opened on a particular device, subsequent activity on that device can also be linked to the account.
• Probabilistic match (a.k.a. “cross device match”). The system can apply statistical methods and rules to identify multiple devices used by the same person, such as computers, tablets, smart phones, and home appliances. While many CDPs rely on third party services for this sort of matching, this item refers only to matching done by the CDP itself.

     Unstructured and Semi-Structured Data. This refers to loading data from unstructured or semi-structured sources such as Web logs, social media comments, voice, video, or mages. These are typically managed with “big data” technologies such as Hadoop. Nearly all CDPs use some version of this technology but it’s only essential if clients have unstructured or semi-structured sources and/or very high data volumes. Some CDPs handle very high data volumes in structured databases such as Amazon Redshift.

• JSON load. The system can accept and store data through JSON feeds without the user specifying in advance the specific attributes that will be included. Additional configuration may later be required to access this data. There are some alternatives to JSON that offer similar capabilities.
• Schema-free data store. The system uses a data store that does not require advance specification of the elements to be stored. Examples include Hadoop, Cassanda, MongoDB, and Neo4J.

     Web Site. This refers to interactions with the company’s own Web site, whether on a desktop computer or mobile device.

• Javascript tag. The system provides a Javascript tag that can be loaded into the client’s Web site and used to capture data about customer behaviors. Some CDP vendors provide full tag management systems but this is not a requirement for this item. This item does require that data captured by the Javascript tag can be associated with a customer record in the CDP database. This is usually done with a Web tracking cookie but sometimes through other methods.
• Cookie management. The system can deploy and maintain Web browser cookies associated with the client’s own Web site. The cookies can be linked to customer records in the CDP database.

     Mobile Apps. This refers to interactions with mobile apps created by the company.

• SDK load. The system offers a Software Development Kit (SDK) that can load data from a mobile app into the CDP database. It must be able to associate the data with individual customers in the CDP database. This is usually done through an app ID. Other SDK features such as message delivery are not a requirement for this item.

     Display Ads. This refers to interactions through display advertising networks, including social media networks.

• Audience API. The system has an API that can send customer lists from the CDP to systems that will use them as advertising audiences. The receiving systems might be Data Management Platforms, Demand Side Platforms, advertising exchanges, social media publishers, or others. Ability to receive information back from the advertising systems is not a requirement for this item.
• Cookie synch. The CDP can match its own cookie IDs with third party cookie IDs to allow the marketer to enrich profiles with external data or reach users through advertising networks.

     Offline. This refers to interactions managed through offline sources such as direct mail and retail stores, where the customer’s primary identifier is name and postal address.

• Postal Address. The system can clean, standardize, verify, and otherwise work with postal addresses. This processing is reduces inconsistencies and makes matching more effective. Systems meet this requirement so long as the address processing is built into system process flows, even if they rely on third party software. Systems that send records to external systems in a batch process do not meet this requirement.
• Name/Address Match. The system can find matches between different postal name/address records despite variations in spelling, missing data elements, and similar differences. As with postal processing, systems can meet this requirement with third party matching software so long as the software is embedded in their processing flows.

     Business to Business. This refers to companies that sell to other businesses rather than to consumers.

• Account-level data. The system can maintain separate customer records for accounts (i.e., businesses) and for individuals within those accounts. This means account information is stored and updated separately from individual information. It also means that selections, campaigns, reports, analyses, and other system activities can combine data from both levels.
• Lead to Account Match. The system can determine which individuals should be associated with which account records, using information such as company name, address, email domain, and telephone number. This excludes processing done by sending batch files to external vendors.

Analytics. These are applications that use the CDP data but don’t extend to selecting messages, which is the province of customer engagement.

• Segmentation. The system lets non-technical users define customer segments and automatically send segment member information to external systems on a user-defined schedule. Ideally, all data would be available to use in the segment definitions and to include in the extract files. In practice, some configuration may be needed to expose particular elements. Systems meet this requirement regardless of whether segments are defined manually or discovered by automated processes such as cluster analysis.
• Incremental attribution. The system has algorithms to estimate the incremental impact of different marketing activities on specified outcomes such as a purchase or conversion. Attribution is a specialized analytical process that relies on the unified customer data assembled by the CDP. Algorithms vary greatly. To qualify for this item, the algorithm must estimate the contribution of different marketing contacts on the final result. That is, fixed approaches such as “first touch” or “U-shaped distribution” are not included.
• Automated predictive. The system can generate, deploy, and refresh predictive models without involvement of a technical user such as a data scientist or statistician. This usually employs some form of machine learning. There are many different types of automated predictive; systems meet this requirement if they have any of them.

 Engagement. This refers to applications that select messages for individual customers. It does not include content delivery, which is typically handled outside of the CDP.

• Content selection. The system can select appropriate marketing or editorial content for individual customers in the current situation, based on the data it stores about them, other information, and user instructions. The instructions may employ fixed rules, predictive models, or a combination. Selections may be made as part of a batch process.
• Multi-step campaigns. The system can select a series of marketing messages for individual customers over time, based on data and user instructions. The message sequence is defined in advance but may change or be terminated depending on customer behaviors as the sequence is executed.
• Real-time interactions. The system can select appropriate marketing or editorial content for individual customers during a real-time interaction. This requires accepting input about the customer from a customer-facing system, finding that customer’s data within the CDP, selecting appropriate content, and sending the results back to the customer-facing system for delivery. The results might include the actual message or instructions that enable the customer-facing system to generate the message.

Will Marketing Technologists Kill Martech?

I’ll be giving a speech next week on the evolution of marketing technology, which doesn't follow the path you might think. The new channels that appear on a typical “history of marketing timeline”, such as radio in the 1920’s and TV in the 1950’s, didn’t really trigger any particular changes in the technology used by marketers: planning was still done on paper spreadsheets and copy was typed manually up to the 1970’s. Similarly, marketers up that time worked with the same data – audience counts and customer lists – they had since Ben Franklin and before.

It was only in the 1960’s, when mailing lists were computerized, that new technologies begin to make more data available and marketers get new tools to work with it. Those evolved slowly – personalized printing and modern campaign managers appeared in the 1980’s. The big changes started in the 1990’s when email and Web marketing provided a flood of data about customer behaviors and vendors responded with a flood of new systems to work with it. But it wasn't until the late 2000’s that the number of vendors truly exploded.

I can’t prove this, but I think what triggered martech hypergrowth was Software-as-a-Service (SaaS). This made it easy for marketers to purchase systems without involving the corporate IT department, allowing users to buy tools that solved specific problems whether or not the tools fit into the corporate grand scheme of things. Major SaaS vendors, most notably Salesforce.com, made their systems into platforms that provided a foundation for other systems. This freed developers to create specialized features without building a complete infrastructure. Building apps on platforms also sharply reduced integration costs, which had placed a severe limit on how many systems any marketing department could afford. Easier development, easier deployment, and easier acquisition created perfect environment for martech proliferation.

But every action has a reaction. The growth of martech led to the hiring of marketing technologists, as marketing departments realized they needed someone to manage their burgeoning technology investments. That might seem like a good thing for the martech industry, but it introduced a layer of supervision that restrained the free-wheeling purchases that marketers had been  making on their own. After all, the job of a martech manager is to rationalize and coordinate martech investments, which ultimately means saying “no.”

The quest for rationalization leads to long-term planning, vision development, architecture design, corporate standards, and project prioritization: all the excellent practices that made corporate IT departments so unresponsive to marketers in the first place. The scrappy rebels in martech departments hear the call of order-obsessed dark side and find it increasingly hard to resist.

And it only gets worse (from the martech vendor point of view). As marketing technologists discover just how many systems are already in place, they inevitably ask how they can make things simpler. The equally inevitable answer is to buy fewer systems by finding systems that do more things. This leads to integrated suites – marketing clouds, anyone? – that may not have the best features for any particular function but offer a broad range of capabilities. When the purchase is made by individual marketers focused on their own needs, the best features will win and small, innovative martech vendors can flourish. But when purchases are managed by the central martech department, integration and breadth will weigh more heavily in the decision.  This gives bigger, most established firms the advantage.

In short, martech today is at a crossroads. Martech managers can follow the natural logic of their positions, which leads to greater centralization, large multi-function systems, and increasingly frustrated marketers. Or they can retain their agility and support new, innovative martech vendors, recognizing that near-term efficiency will suffer. Put so starkly, it’s obvious that agility is the better choice, and there is plenty of discussion in the industry of how to maintain it. But the dark side is powerful, relentless, and seductively rational. Martech managers – and the marketers they ultimately serve – must tread carefully to stay on the right path.

Landscape of MarTech Vendor Directories

I'm making a presentation on marketing technology selection at B2BLeadsCon in New York next week, and had thought to start with the usual Oh-My-God-There-Are-So-Many-Vendors slide to get everybody's attention.  This would ordinarily be Scott Brinker's popular Chief MarTech Landscape but I've recently seen so many variations on the theme that I put together a composite slide instead.  This includes Scott's slide plus versions from Luma Partners, Gartner, MarTech Advisor, Terminus/FlipMyFunnel, and Growthverse.

I considered labeling this a "landscape of landscapes" but quickly realized that (a) it's not all that witty and (b) six vendors isn't enough.  But on further reflection, I recognized that these landscapes are really a type of directory that helps marketers find available products.  This led me to consider other types of online directories, of which there are many.  So I did end up producing a landscape that still isn't as crowded as Scott's but does show the number of information sources available.

As you see, this contains four sets of products: the original six landscapes, divided between the static images and the two interactive options (both very cool).  In addition, there are two directories with analyst ratings, from Gleanster and TopAlternatives.  But the biggest category is the community review sites, of which the best known among marketers are probably G2 Crowd, TrustRadius, and Software Advice.  Because the purpose here is to list tools that help marketers find systems to purchase, I didn't extend the landscape to business directories like Crunchbase, VentureBeat's VB Profiles and Owler.

I did look at every vendor shown in the graphic and can affirm that each includes at least some marketing systems.  There are some interesting differences in approach but, like any good landscape creator, I'll simply give you a set of logos and let you research from there. Again following the tradition of landscape publishers, I make no claims about the completeness of my list or the quality of any of the companies listed.  But I will make your life a bit easier by listing all the links below.  Enjoy!

AlternativeTo
AppAppeal
BestVendor
Chief MarTech
Cloudswave
Credii
DiscoverCloud
GetApp
G2 Crowd
Gartner
Gleanster
Growthverse
Luma Partners 
MarTech Advisor
Osalt
IT Central Station
Serchen
Social Compare
Software Advice
Software Insider (formerly FindTheBest)
Terminus
TopAlternatives
TrustRadius

Design Your Best Marketing Technology Stack and Plan the Transition: Sneak Peek at FlipMyFunnel Conference

Picture posted by Terminus

I can’t decide which is more exciting about next month’s FlipMyFunnel conference in Atlanta (register here and use the code DR50 for a 50% discount): the opportunity to interact with a great collection of speakers and attendees or seeing what the conference organizers at Terminus do with the notion of MarTech Stack Jenga. Based on one cryptic Twitter picture, they’re up to something big.

My own contribution will be a presentation on designing your marketing stack. This is something I’ve done for years as a consultant but it’s now an especially hot topic. Here are some of the key points I’ll be making:

- the stack is based on your business and marketing strategies. I’ve described the importance of strategy before but have now refined my explanation to show how marketing programs, business requirements, and functional requirements connect over-all marketing strategy with martech. The picture below also highlights the importance of planning for future business, marketing, and martech developments.

And I’ve provided a sample template for organizing your requirements by system.

- a winning stack is efficient as well as functional. I'll present a checklist for evaluating your stack design along those dimensions.

- how you draw the stack makes a difference. I’ll argue that a diagram which shows relationships between systems is more helpful than one that simply lists the different components. In the example below, the flow highlights the isolation of sales and service from the rest of the stack – a critical weakness that isn’t apparent when you look at the systems only.

- transition planning must be systematic as well. Companies struggle with transition planning even more than they struggle with stack design.  The goal is to sequence the stack changes so that each new system adds the greatest value with the least disruption. This requires understanding which system changes support each improvement.  This lets you figure out which improvements would be supported by changing any one system, what would then be possible after changing a second system, what is possible after changing a third system, and so on.  The worksheet lets you explore different sequences so you can pick the best one.

This will be easier to understand in person than in writing. Don't take my word for it: join us in Atlanta and see for yourself.

Adobe Marketing Cloud Marches Towards Martech and Adtech Integration

At pretty much the same moment I was publishing my post on the merger of martech and adtech into madtech, Adobe was announcing its latest marketing products, including a press release on uniting “Data-driven Marketing and Ad Tech” . Naturally, this caught my attention.

As you might expect, Adobe’s reality is considerably more complicated than the simplicity of the “madtech” vision. Like the other enterprise software vendors who offer broad martech and adtech solutions, Adobe has built its marketing cloud by buying specialist systems. And, again like its competitors, it has only integrated them to a limited degree.

In Adobe’s case, the various products remain as distinct “solutions” served by a common set of “core services”. The current set of eight solutions includes Analytics (Web, video and mobile analytics, née Omniture Site Catalyst), Social (social publishing, based on Context Optional), Target (Web optimization and personalization, derived from Omniture Test & Target/Offermatica), Experience Manager (Web content management , originally Day Software), Media Optimizer (based on Efficient Frontier and Demdex), Campaign (formerly Neolane); Primetime (addressable TV) and Audience Manager (data management platform, formerly Demdex). Of course, the products have all been modified to some degree since their acquisitions.  But each still has its own data store, business logic and execution components.

Rather than replacing these components with common systems, Adobe has enabled a certain amount of sharing through its core services. In the case of customer data, the “profiles and audiences” core service maintains a common ID that is mapped to identities in the different solutions. This means that even though most customer data stays in the solutions’ own databases, the core service can use that data to build audience segments. There's also an option to load some attributes into the core services profiles themselves.   Audiences, which are lists of IDs, can either be defined in solutions and sent to the core service or built within the core service itself.  Either way, they can then be shared with other solutions. Data from external systems can also be imported to the core service in batch processes and used in segmentation.

Adobe says that data stored in the solutions can be accessed in real time.  I'm skeptical about performance of such queries, but the ability to store key attributes within the core service profiles should give marketers direct access when necessary.  There’s certainly a case to be made that digital volumes are so huge and change so quickly that it would be impractical to copy data from the solutions to a central database. Where external data is concerned, marketers will increasingly have no choice but to rely on distributed data access.

But here’s the catch: Adobe's approach only works if all your systems are actually tied into the central system. Adobe recognizes this and is working on it, but so far has only integrated five of its solutions with the profiles and audiences core service. These are Analytics, Target, Campaign, Audience Manager, and Media Optimizer. The rest will be added over time.

The second big limit to Adobe’s current approach is sharing with external systems. Only Adobe solutions can access other solutions’ data through core services. This makes it difficult to substitute an external product if you already have one in place for a particular function or don’t like Adobe’s solution.

Adobe does connect with non-Adobe systems through Audience Manager, its data management platform, which can exchange data with a company’s own CRM or operational databases, business partners, and external data pools and ad networks. Audience Manager can hold vast amounts of detailed data, but does not store personally identifiable information such as names or email addresses. Audience Manager can also copy Web behavior information directly from Analytics, the one instance (so far as I know) where detailed data is shared between Adobe solutions.


So far, I’ve only been discussing data integration. The various Adobe components also have their own tools for segmentation, decision logic, content creation, and other functions. These are also slowly converging across products: for example, there is an “assets” core service that provides a central asset library whose components can be uploaded to at least some of the individual solutions. The segmentation interface is also being standardized product-by-product. There’s no point in trying to list exactly what is and isn't standard today, since this will only change over time.

The lesson here is that suites are not simple. Marketers considering Adobe or any other Marketing Cloud need to examine the details of the architectures, integration, and consistency among the components they plan on using. The differences can be subtle and the vendors often don’t explain them very clearly. But it pays to dig in: the answers have a big impact on whether the system you choose will deliver the results you expect.

Customer Data Platforms Revisited: The Future of Marketing Data

It’s nearly two years since I introduced the concept of a Customer Data Platform, defined as a marketer-controlled system that builds a multi-source customer database and exposes it to external execution systems.  You may recall that I listed several sets of products as CDPs: B2B predictive lead scoring and customer success management; campaign management with an integrated customer database; and data management platforms to support online advertising. Systems were included only if their data (or derived data such as model scores) was available to other systems for campaigns and messaging.

All those categories have done well since my original posts on the topic. Established vendors have grown quickly and attracted funding; new vendors have joined the mix, also often with substantial funding. So I suppose I could pat myself on the back for spotting an important trend and let it go at that.

But things aren’t quite so simple. A look at the entire CDP ecosystem uncovers important patterns that are hidden when you look at individual vendors or vendor categories. Here's a summary of what I've seen.

Customer Management Functions

CDPs exist because marketers need to coordinate customer (and prospect) interactions across channels. That coordination involves three basic tasks: gathering and unifying customer data from all sources; using that data to select the best treatment for each interaction; and delivering those treatments through the appropriate channel systems. Each of those three tasks has several subtasks. These layers are illustrated by the following diagram, which includes a unified data layer – the classic CDP.

Vendor Categories

So far so good, but it’s really just theory. Things get interesting when you look for specific systems that perform the subtasks. It turns out that there are several categories of specialist systems within each subtask, each doing similar or complementary things in slightly different ways. Connecting the logical flow to actual systems is important because looking at real products tells you what the market is saying: that is, what buyers are willing to pay for and where change is concentrated.

The following table shows what I found when I did this analysis. The list of vendors in each section isn't necessarily comprehensive, especially in crowded segments like B2B marketing automation. I should also stress that I’ve only included Decision-layer vendors who also build their own database. This makes them potential CDPs and means they have many Data-layer functions. In a sublimely liberating act of inconsistency, I have NOT limited the Delivery layer to vendors who build their own database. In fact, most do not.

Investment

The right-most column on the previous table shows the level and types of investment being made in each vendor class. I haven’t collected precise details but the general patterns are pretty strong. The major observation is that current investment is heavily concentrated on the Decision layer, with interest in predictive modeling and message selection (which could also be labeled as personalization). There’s some investment on the Data layer in data gathering vendors, especially along the lines of acquisitions by big companies (Oracle/Datalogix, D&B/NetProspex, etc.). This is a general sign of maturity. Similarly, most recent investment on the Delivery layer has been acquisitions (IBM/Silverpop, Oracle/Responsys, Teradata/Appoxee, etc.), which is a sharp contrast from the heavy venture capital funding a couple of years back. Again, this shows the relative maturity of the space.

(Caveats: although it doesn’t show up in this analysis, I do still see some interesting investment in marketing automation niches such as app marketing, distributed marketing, and agency systems. I’m also increasingly intrigued at the “tag management” vendors on the Data layer (Tealium, Signal, Ensighten, etc.), which are reinventing themselves as data integration hubs. I didn’t see that one coming.)

Implications

It’s tempting to interpret these results are showing that data assembly is a solved problem, allowing marketers to invest Decision systems on the next layer down. But any marketer can tell you, and every survey I’ve seen confirms, that most companies are nowhere near having fully integrated their customer data.

What I think is really going on is that people are investing in Decision systems that build their own multi-source databases, providing both Data and Decision functions in one package. Remember that my original CDP categories included B2B predictive vendors and campaign management vendors who did exactly that. So it seems the proper way to look at things is more along the lines of the following diagram, which shows there are several different ways to solve the customer data integration challenge: you can buy a stand-alone CDP that has only data-level functions; buy a Decision system that also builds an integrated database; or buy a Delivery system that does data, decisions, and execution. As the diagram indicates, most of the Decision vendors do incorporate the CDP functions, while only a few of the Delivery vendors do.

The diagram labels the Data + Decision combination as a “Marketing Platform”.  I think this is reasonably consistent with how most people use the term, since the key feature of a “platform” is its ability to integrate with external systems for delivery and other purposes. I’ve labeled the Data + Decision + Delivery combination as an “Integrated Suite” and used question marks to show that not all suites provide a complete Data solution. This is because many suites aren’t very good at bringing in external data or letting external systems access the data they’ve assembled.

As I noted in the previous section, most of the industry funding and excitement is centered on the Decision layer, which is where the Marketing Platforms live. The practical advantage of those systems over Data-only solutions is obvious: Decision systems deliver a revenue generating application while Data-only systems do not.

But think about that for a moment.  Each Decision system builds its own multi-source database and each integrates separately with the Delivery systems.  Having multiple Decision systems is a nightmare of redundancy:

It seems pretty clear that the better solution is to have a single Decision system controlling everything, which is arguably what most people (and vendors) have in mind when they describe a Marketing Platform. Indeed, this is exactly the direction that most Decision-layer CDPs are headed, by expanding the scope of their products from an initial point solution, such as B2B lead scoring, to encompass other applications. It’s safe to say that the people who built these systems always planned, or at least hoped, to grow in this direction.

Does the growth of Decision-layer CDPs mean that Data-only CDPs will fail? I’ll admit that only a few such systems have appeared in the past two years. But I’m not quite ready to give up on the concept.

Why?  Well, as Tolstoy never said, all good customer databases look alike, but every decision system is different. This means it’s hard to support all types of decisions within a single product. So it does seem that multiple decision systems will appeal to marketers who have the skills to use them and the scale to justify the added expense. Those marketers would benefit from a Data-layer CDP, which would make it easier to deploy best-of-breed decision tools even when those tools lack data unification functions.

The stumbling block for this approach is still the cost of integrating multiple systems: as the diagram shows, there are still plenty of connections in this model. But there’s at least some hope (although I remain skeptical) that newer technologies will make the integration easier. The other bright spot for the Data-only CDPs is that they should be attractive as partners or acquisitions for Decision and Delivery systems that haven’t built their own CDP functions.


And what about the suites? I’ve said for years that the first law of software market development is “suites win”, precisely because most companies will sacrifice best of breed functionality to avoid the costs of integration. Indeed, the big marketing clouds from Oracle, Salesforce.com, IBM, Adobe, and others all include extensive Delivery layer functions. I think it’s fair to say that while their commitment to being “open platforms” is genuine, they see that as a way of letting clients supplement the core functions the suites provide internally.  This is quite different from the idea of a shared Data and Decision platform that specifically avoids offering Delivery services. Still, there’s a  very good chance that a suite which can easily integrate supplementary functions will give marketers enough freedom to overcome the problems of lock-in, while still delivering the convenience of pre-integrated core functions. So I’m not quite ready to abandon “suites win” as a rule, although I’m a bit less certain than previously.

Looking Ahead

It’s fun to handicap the horse race among vendors and categories, but what really matters is the contest itself.   All these smart people and money are finally giving marketers the unified customer databases they so desperately need.  This removes a fundamental obstacle to the cross-channel integrated marketing that everyone recognizes is increasingly important. So let’s look at the view once we've climbed that mountain.

I’d like to tell you I see a new and perfect world, but what's actually there is more mountains.  Once unified databases become available, marketers will face a new set of challenges including:

- more need for predictive models and external data. I only lump those together because they’re already getting a lot of attention. Having a powerful database just makes them even more important.

- new focus on automated content creation and campaign design. Lack of skilled users and adequate content are already huge barriers to effective multi-channel marketing.  Removing the database barrier will only make them stand out even more. So we can expect smart people to address them through technology. Indeed, there is already plenty of activity in these areas but I think it’s fair to say that so far none of vendors have had a major impact. This is arguably the next exciting frontier for marketing technology.

- more developments in cross-channel customer tracking. Again, the need for this has been obvious and some major investments have already been made. Cookies are becoming increasingly inadequate as cookie-hostile channels like mobile become more important. Marketers will soon reach a tipping point (or maybe they already have) where they realize they must abandon cookies and move on to other approaches such as device identification or external identity databases. A new standard will eventually emerge, although I can’t even guess what it might be.

- tighter integration between advertising and marketing technology. These two realms are now largely separate with a few exceptions such as retargeting. But as personalized ad messages become increasingly possible, marketers will have ever-greater incentive to target and, ultimately, coordinate messages across channels using shared data. This is highly dependent on the improved customer tracking, so it might have to wait a bit.

- better marketing attribution. If there’s a last stop on the road to marketing Nirvana, attribution might be it. Once marketers have assembled all that data and associated everything with the right customer, they’ll finally be able to deploy advanced analytical methods to really understand the long- and short-term incremental impact of their marketing efforts. Then, and this itself would be heavenly, we’ll never again hear anyone quote John Wanamaker about not knowing which half of his advertising is wasted.

Recommendations for Marketers

Nirvana is still far distant.  Marketers face immediate choices in how to spend their time and budgets. The trends I’ve just described do have some immediate practical implications. Here are my suggestions:

- Experiment like crazy. The various Decision-layer vendors currently offer different specialties, such as lead scoring vs. product recommendations vs. churn predictions. Vendors in each area are expanding their scope so there’s a good chance you’ll eventually pick one to do almost everything. To have the best odds of making a good selection, you’ll want to learn about as many vendors as possible in advance. So run tests to build an understanding of the applications, technologies, and corporate culture. The good news is that each approach can probably pay for itself in improved performance, so these tests should be more or less self-financing.

- Keep an eye out for new data. Many of the Decision-layer vendors bring their own data to the party, and evaluating that data is one part of understanding what they offer. But there are also other data sources that are not tied to a Decision system. You’ll want to explore these to understand what value they provide value and whether to make them part of your long-term data foundation.

- Plan for integration. You may not have shared customer data or decisions today, but it’s increasingly likely they’re in your future. So every new marketing system should be evaluated in part on its ability to integrate with other systems. This involves sending data to the central database and reading data from it, as well as integrating with Decision-layer systems for predictive models, rules-based selections, optimization, recommendations, personalization, and more. Even if you’re going to use an integrated suite, you’ll want to assess how easily you can supplement its functions by tying into external products, and what kinds of products are already available for integration.

Summary

The stand-alone Customer Data Platform is one solution to the challenge of providing a multi-source, shared marketing database, but it isn't the only option.  Whichever solution marketers ultimately find most appealing, they will benefit from gaining control of their data and moving on to new opportunities that database makes possible..

Seven Marketing Automation Myths to Ignore - Illustrated Edition

I’m sad.

I’ll be giving a speech in Milwaukee next week on marketing automation myths, and early in the preparation process had the idea of illustrating it with mythical creatures from the films of Ray Harryhausen, the stop-action animation genius whose best known images are probably the skeleton warriors in Jason and the Argonauts (1963).

This led to many pleasant hours scrolling through galleries of Harryhausen images. I even found an illustration that vaguely matched the theme of each myth.

But there’s a problem. Every bit of presentation-giving advice, training, and experience I’ve ever had tells me that these illustrations will distract attention from my points rather than reinforcing them. The responsible adult inside of me knows I have to get rid of them while the fun-loving child says, Yeah, but they're just so cool. 

This blog post is my compromise: I’ll publish them here, which will make dropping them from the actual presentation much less painful. **sigh**

So, the illustrated version of my talk goes like this:

Marketing Automation Myth Busting: we start with Mighty Joe Young, Harryhausen’s 1949 tribute to King Kong. I could tell you he’s about to smash some myths, but who are we kidding? It’s just a great image. 

Trouble in Paradise: marketing automation is growing quickly but users are dissatisfied. Maybe that’s not as bad as being attacked by a giant crab, but it’s still problematic. Image from Mysterious Island (1961).

Myth: All systems are the same.  This is an easy mistake because systems all look and sound alike during the buying process. But in fact they differ greatly. The myth leads buyers to think it doesn’t matter which system they purchase, and therefore that they can buy without first defining their requirements. In fact, our research shows that unsatisfied marketers often have purchased a system that didn’t meet their needs. Conversely, the most satisfied users did select based on specific features. The image here is Cyclops from The Seventh Voyage of Sinbad (1958). He has vision problems; it’s hard to see the differences between marketing automation systems. Get it?

Myth: Integration is easy. This echoes the first: all marketing automation products integrate with CRM, so people assume they don’t have to look into the details. But products differ hugely in which systems they connect with, what data they import and export, and how much control uses have over the details. Integration is the single most commonly cited obstacle to success and is linked to the most dissatisfied users. So people really need to ensure that the system they’re buying meets their integration needs. Kali from The Golden Voyage of Sinbad (1974) coordinates fighting with six arms, so she is the goddess of successful integration.

Myth: Failure is the user's fault, not the system's. This myth follows from the first two: if all systems are the same, then failure must be fault of the user. But, as we’ve seen, systems aren’t the same and many failures result from a system that doesn’t meet the user’s needs. Other research shows that users generally overcome obstacles they can control, like organization, training, and staffing levels. Of course, system selection is itself done by users, so they do have some responsibility for any problems. Talos, an animated statue from Jason and the Argonauts, ultimately fails to protect the tomb he is built to guard, so he represents a system that doesn’t work.

Myth: New users should crawl, walk, run.  Many experts – myself included – have suggested that new marketing automation users can safely start without planning by just duplicating their existing programs like email blasts, and then add more sophisticated uses over time.  But our research found that marketers who used more features from the start were happier. My interpretation is that successful marketers took the time to plan and train before deployment, while marketers who didn’t prepare in advance never found the time to learn what they needed. It’s possible to overstate this position – even successful users will add some new features over time. But the point about preparation is important. Kraken, from Clash of the Titans (1981), is a sea monster with no legs, so he never had a chance to move beyond crawling.

Myth: Bigger companies do better.  You might expect that bigger companies would do a better job with marketing automation because they have larger and more sophisticated staffs. They do in fact select more wisely, paying more attention to features and integration than marketers from smaller companies, and less to cost and apparent ease of learning. But they also face more non-technical obstacles such as training, staffing, and organizational barriers. So their over-all satisfaction level is no higher than smaller firms. I chose the giant octopus from It Came from Beneath the Sea (1955) because it’s big – no deeper meaning is intended.

Myth: Marketing automation creates prospects and saves money.  Marketers who expect their system to generate more prospects with less effort are usually disappointed. Marketing automation is basically about nurturing existing leads, not finding new ones, and most companies add staff and budget. Medusa, from Clash of the Titans, is the boss you don’t want to give bad news about system results: her dirty look will turn you to stone.

Myth: Marketing automation has stopped evolving.  Commoditization and consolidation may make marketing automation look like a mature industry.   But there's still plenty of change: new vendors entering the space, existing vendors being bought and repositioning themselves, and expanding scope to include consumer marketing, display ads, external data, better databases, identity resolution across channels, mobile apps and formats, advanced attribution, social promotions and new types of content.  The Beast from 20,000 Fathoms (1953) is a dinosaur who hasn’t evolved one bit.

So what? That’s the end of the myths, but we need to leave on a positive note.  So I end the presentation with some sound, if predictable, advice to prepare carefully, define and select against actual requirements, test integration in advance, deploy quickly, and expect the unexpected. The puzzled look on Troglodyte’s face, from Sinbad and the Eye of the Tiger (1977), represents the confusion marketers feel when wondering what to do next..

Marketers who want help selecting a system could try blowing on a ram's horn like Calibos from Clash of the Titans. Or they can just send me an email at draab@raabassociates.com.

*              *             *

Speaking of art that's amusing if irrelevant, here's a link to a Twilight Zone-themed introduction to a football recruiting show produced by my son Brian.  The apple doesn't fall far from the tree.

New Frontiers in Data Driven Marketing

I recently gave a talk on New Frontiers in Data Driven Marketing, which managed to incorporate Barbie, Fred Astaire and Ginger Rogers, General Winfield Scott, and The Three Stooges. Let’s just say you had to be there. But even without celebrities, I think the list is worth a quick look as you start planning for next year’s marketing programs.

New Challenges

• Integrate ad tech and martech. We’ve seen this coming for some time but it’s now much more obvious as marketing automation vendors like Oracle and Adobe, display ad targeters like Bizo (now part of LinkedIn) and Demandbase, and even tag managers like Signal (formerly BrightTag) and Tealium come at the challenge from different directions. The core issue is that marketing campaigns in advertising, traditional outbound media, and new social and inbound media all target increasingly-identifiable audiences rather than anonymous cookies, site visitors, viewers, or prospect lists. This makes it more possible to work across all media to improve targeting, to coordinate messages for each individual, and to measure the incremental impact of each promotion. This, in turn, requires integrated systems to gather the necessary data in a single location, track interactions with individuals, send appropriate messages, and monitor results. Look for more integration along those lines from big platform players and for cooperation among specialized solutions as they seek to participate in the consolidated approach.

• Extract meaning from big data. Everybody loves big data but few people talk about the downside: sloshing huge buckets of information into a giant data lake means that everybody has to do their own refining before they can do anything useful. Of course, analysts have always spent a lot of time on data prep and veterans will scoff at the implication that most data warehouses are pristine. But the ease of adding new feeds to big data stores, especially of unstructured data, means that users now face a “do it yourself data quality” challenge that's much greater than before. To make things even harder, direct access to data has expanded to many business users who don’t have the data management skills or sensitivity of expert analysts. This is a problem I haven’t seen discussed very much, but you can be certain it is coming to a desktop near you.

• Translate offers across media and campaigns. All that cross-channel coordination means marketers have more ways to present the right message to each individual, which turn means each message much be available in the format of each touchpoint. “Responsive design” addresses one piece of the problem, making it easy for the same Web content to render effectively on different devices. But there are plenty of other touchpoints that responsive design doesn’t reach, including display ads, call centers, and social media. So far, most of the energy related to this issue has been spent in making it easier for a single system to send messages to multiple channels, not in automatically adjusting messages to account for different amounts of content or user mindset in a given context. This is another area that has received little attention so far, especially in terms of refinements like testing and optimization.

New Technologies

• Predictive everywhere.  Most marketers are now familiar with basic predictive modeling applications like lead scoring and content recommendations. But big data and multiplying channels offer them opportunities to do so much more – and, given the alternative of poor customer treatments, they really have no choice.  Happily, the technology to build predictive models has kept up with marketer needs, so it’s increasingly possible for automated systems to build and deploy dozens or hundreds of models with almost no marketer input. This means programs can be designed to incorporate predictive models in all kinds of treatment decisions, from content recommendations to sales call prioritization to banner ad selection. In fact, the technology in this area is probably ahead of marketers, who need to learn how to identify modeling opportunities, to structure programs to use models effectively, and to monitor model results.

• Natural language processing for unstructured data management. Natural language processing (or NLP, as the cool kids say) and unstructured data are different things and both relatively established. I’m listing them here because unstructured data must become at least semi-structured to be useful, through processes such as tagging and indexing. Doing this efficiently at big data volumes requires automated solutions, which is where NLP comes into play. There are plenty of other NLP applications, such as sentiment analysis, speech processing, data gathering, and even some slick “copy generation” methods (for example, Persado and Captora, which I described briefly last June ). But I think making sense of unstructured data is NLP’s killer app.

New Opportunities

• Mobile/local marketing. Okay, maybe not so new. But still at the frontiers, since marketers are struggling to take advantage of what’s unique about mobile systems rather than just treating them as tiny desktops. Mobile apps are one part of this, since they’re separate from regular Web sites and emails. Location- and context-aware programs are another aspect: the potential is obvious even though it’s not yet clear how to best exploit it. There are some pretty serious privacy concerns to address here, although it’s never clear whether those will be real obstacles or evaporate as customers overcome their initial surprise at how much marketers can tell about them and get back to playing Clash of Clans.

• Advanced attribution. I’m talking here about attribution based on a nearly complete view of all customer interactions with a brand: Web and email messages, of course, but also search, display, broadcast and print advertisements, in-store and near-store* interactions, purchase and service histories, social messages and networks, device telemetry, and only the NSA knows what else. Once you have all that data and have managed to link identities across different sources, you can apply some truly whiz-bang analytics to estimate the incremental impact of different messages on short- and long-term customer behaviors. This goes beyond the simplifying assumptions of first-touch, last-touch and fractional attribution approaches. If it works properly, it promises to revolutionize how marketing budgets are managed and to give a substantial business edge to companies that master it first.

• Journey mapping. Another old concept, but one that’s gaining a lot of new attention. I’ll give a shout-out to my friends at SuiteCX who have built some slick mapping tools that I never quite get around to reviewing. If I had to speculate why journey mapping is suddenly so popular, I’d guess it’s because it’s become so obvious that the traditional purchase funnel has exploded into maze of hopscotch courts, with customers leaping from one spot to the next like crickets on a frying pan. Journey mapping is one way to make sense of it all, or at least apply a bit of order to the natural chaos. It relates closely to multi-channel programs, attribution and mobile/local marketing as well, if you think about it. No wonder it’s climbing to be king of the buzz hill.

_______________________________________________________
* I just made that up.

Matching Marketing Technology to Business Strategy: A Starting Framework

As you may know, I’m working with Scott Brinker of Chief Martech blog fame and Third Door Media on the MarTech marketing technology conference set for August 19-20 in Boston.*

When Scott asked for content suggestions during the early stages of the conference planning, my reaction was that one thing everyone needs is a framework for relating marketing technology investments to larger business strategy. Scott was flatteringly enthusiastic and – I totally should have seen this coming – came back two weeks later with the suggestion that I present such a framework.

Bluff called.

Every consulting engagement I do starts with the relationship between business strategy and marketing technology.**   But I've always defined it on a case-by-case basis.  Until Scott asked, I never had a reason to create the framework for a generalized approach.

So, where to begin? The first thing to remember about strategies is that they’re about making choices: a strategy is a method for reaching a goal; knowing the method gives you a way to decide whether a particular choice is consistent with it.  More concretely, “make money” is a goal; “make money by building high quality products for which customers will pay a premium” is a strategy. Even a strategy as simple as that one tells you a great deal about where to invest, what kinds of systems to build, what kinds of marketing to perform, whom to hire, and much else. I do this analysis for my clients so we can be sure to find marketing systems that are consistent with their business approach and resources.

But here’s the thing: just about every discussion of marketing technology starts off with the (true) fact that today's customers demand a highly personalized experience informed by detailed knowledge of their history. The implication is that every business needs to adapt a strategy of customer intimacy. But if everyone has the same business strategy, then shouldn’t everyone use the same technology strategy as well?

[crickets]

I think the answer is no, for two reasons. First, everyone doesn’t have the same business strategy. Second, even the same business strategy could result in different technology approaches.

• Business strategies still differ. It’s true that customers today expect every company to track their interactions and respond intelligently across all touchpoints.†   But precisely because that expectation is universal, meeting it isn’t enough by itself to be a successful strategy. Remember, strategy is about choices and there’s no choice about whether to meet those expectations. It's HOW you meet those expectations that involves choices that are strategy-driven.

• Tech strategies can vary for the same business strategy.  Let’s say your business strategy is to be a low cost provider. You still have to deliver a reasonable degree of customer intimacy, just as you still have to deliver a reasonable degree of product quality. In this case, the goal of your tech strategy is always the same: “to deliver customer intimacy at low cost”. But the method could be “through extensive automation”, “through low labor cost via outsourcing”, or “through limiting services provided”. Each would imply a different technical approach.

Okay, the notion of a strategy-to-technology framework seems to make sense. [cheers]  So what would that framework actually look like?

It would start with strategic options. The classic big three are high quality, low cost, and high service (i.e., close customer relationships). Each implies different requirements for marketing, product design, production, customer support, and administration, which in turn drive technology, core competencies, and organization.††

Those requirements are the goals of the technology strategies. Methods to meet them are technology options, such as integrated suites, best of breed systems, and platforms-and-apps.  These are modified by other parameters such as in-house vs. outsource, scope of channels, sophistication level, resources, and scale. Note that some of these are choices while others are constraints.

The really critical challenge in setting your technology strategy is understanding the implications of each option so you can pick the technology strategy that best meets the business strategy requirements. Once the technology strategy is chosen, the final step is mapping the choices to the different layers of the marketing technology architecture: Data, Decisions, and Deployment (I just made that up but we all adore alliteration). The resulting framework would indeed be specific enough to give useful guidance. [more cheers]

I’m guessing this sounds like gibberish to at least some of you. Fair enough. Here’s an example of the framework for an online retailer trying to compete with Amazon by offering customers even lower prices. As you read this chart from left to right, each column provides guidance to the column that follows it.

The recommended strategy in this example is to buy an integrated suite and outsource its operation, accepting limited flexibility in return for lowest possible cost. You might not feel that suites actually give the lowest cost, but that's okay.  In fact, I recommended a suite specifically to make the point that the platform-and-app architecture seen as the future of marketing tech by many people (myself definitely included) isn’t always the right solution. Nor, alas, can a framework force you to make the best choice. What it can do is provide clarity about the options being chosen, so people can argue their case if they disagree, and then pull in the same direction once a decision is made.

So that’s what I’m thinking at the moment. It could well change by the time I reach Boston in August. Join us then and find out.

______________________________________________________________________________

* The conference will provide a much-needed vendor-agnostic forum for understanding marketing technologies and how organizations can use best them. If you’re reading this blog, you should attend. Click here for more info and early bird pricing.

** You do know I make my living by helping companies define their marketing technology requirements and select tools, right? Some people seem not to realize this.

† Well, maybe not every company.  Some still don’t identify individual customers. But even former bastions of anonymity like consumer packaged goods manufacturers now increasingly connect to customers through targeted advertising, social media, and promotions. So we can modify the claim to state that customers expect this tracking when they know the data could be available. It doesn’t change the rest of this analysis.

†† This is all basic “strategy map” stuff based on work by Robert S. Kaplan and David P. Norton. Wikipedia tells me that strategy maps are obsolete but I still find them useful.

Acquisitions Reshape the Marketing Automation Industry: Growth at the Bottom, Room in the Middle, Fog at the Top

Raab Associates officially released the new edition of our B2B Marketing Automation Vendor Selection Tool (VEST) yesterday. This is our flagship report on the industry, with nearly 200 data points on 23 vendors and separate ratings for micro-business, small to mid-size companies, and enterprise marketing departments. There are quite a few vendor comparisons out there, but none come close to the level of detail in the VEST – and details are what you really need to select a system. I personally suggest that anyone interested in the industry buy a copy for themselves and another for someone they love. See www.raabguide.com/vest for details.

I genuinely enjoy catching up with the vendors while preparing the VEST, but must admit that my favorite part of the process is analyzing the data once it’s assembled. Sadly, the wave of acquisitions that swept the industry in the past year has made this harder: many major vendors are now part of a public company, which severely restricts the information they can share. We’ve probably passed a tipping point where so much information is hidden that I can’t draw a clear picture of industry growth rates or competitive positions.

The table below shows the data available and highlights the holes. I’ve grouped the vendors into three buckets based on the market sectors they serve: micro-business (under $5 million revenue), small to mid-size business ($5 to $500 million), and large enterprises (over $500 million).

You’ll immediately see that the “not reported” information is concentrated among companies serving mid-size and enterprise clients, which is where all the acquisitions to date have taken place. Neolane is an exception but only because they provided the VEST information just before Adobe acquired them in June. I doubt we’ll see new numbers from them in the future. Marketo was mostly missing until they provided key figures in their earnings call this afternoon. Thanks, guys.

I've summarize my thoughts on this data with three oh-so-catchy phrases: growth at the bottom, opportunity in the middle, and fog at the top.

Growth at the Bottom: the green shading in the client growth column highlights companies reporting a year-on-year increase of 60% or more. What jumps out is the concentration at the top of the chart, in the micro-business sector. Four of the five micro-business vendors grew more than 60% and the fifth (Venntive) grew at a far-from-shabby 54%. There’s too much missing data in the other sectors to say for certain that the micro-business vendors are growing the fastest, but it sure looks that way. My interpretation is that the micro-business sector is the least mature and still presents the greatest untapped opportunity – even if buyers are still limited to the small proportion of business owners who are “tech geeks”.

Room in the Middle: Marketo's client count increased just 36% from mid-2012 to mid-2013 (although they’re projecting 54% revenue growth for 2013 vs. 2012).  We can no longer see the growth rates for mid-market heavy weights Pardot and Eloqua, but I’d be surprised if they beat Marketo.  They're certainly not close to the 67% to 90% rates reported by LeadFormix, Act-On, and eTrigue. I suspect Pardot, Eloqua and Marketo will increasingly focus on selling to enterprises, and in Marketo’s case on expanding footprint within existing clients. If so, this might open the way to faster growth by the next tier of mid-market vendors, who are mostly still private.  (LeadFormix is the exception, but seems to be pretty much left alone by its corporate parent). The clear winner in this scenario is Act-On, which has ample venture funding and has indeed been growing very rapidly. They are already the first vendor since Pardot to break the 150-employee barrier (blue shading). Silverpop and HubSpot might also benefit but neither is fully focused on standard B2B marketing automation. Other vendors would need outside funding to squeeze through what will probably be a briefly open window.

Fog at the Top: My visibility into enterprise B2B marketing automation was always clouded because of cross-over by B2C vendors including IBM, SAS, Teradata, and Neolane. It is now completely obscured except for sporadic glimpses of details that vendors choose to reveal. But even if everyone shared all their data with me, the enterprise picture would remain foggy because enterprises are increasingly integrating marketing automation with advertising , sales, service, and Web management. This makes it increasingly meaningless to treat marketing automation as a distinct category. Of course, that integration is exactly why the enterprise vendors purchased all those marketing automation systems in the first place.

If integration really happens at the top then we'll end up with a bizarre symmetry, since the enterprise market will be mirroring the integrated sales / CRM / Web / ecommerce products already bought by micro-businesses.  This would leave stand-alone marketing automation as a niche product for mid-tier companies. It would be a very large niche, but squeezed between broader suites from above and below and, eventually, challenged from within by integrated suites built for mid-market companies. The obvious response from marketing automation vendors is to build those broad suites themselves or to create platforms that are the foundation of such suites. That’s exactly what the larger mid-tier companies are doing, but it’s an expensive proposition. Any small mid-market companies who want to play must grab whatever fleeting opportunity the market offers today for growth, before they are locked out for good.

Customer Data Platforms: My New Whitepaper Explains the Excitement

You may have noticed that I've been uncharacteristically aggressive in promoting the Customer Data Platform concept.  Sorry, but I just can't help it: a new system category is even more rare than a new B2C marketing automation system (which, as yesterday's post pointed out, is much rarer than a new beetle).  More important, I think the category itself is a very important development that could really help marketers solve some big problems.  So it's worth several shoves to get the ball rolling.

Along those lines, I am embarrassingly excited to report my formal whitepaper explaining the CDP in depth has just been published.  It was sponsored by ReachForce but they had no influence on the actual content. (In fact, they probably would have preferred something a bit more on-message for their own marketing, so let me thank them for their indulgence.)  You can download it here.  Comments are welcome!

Provenir Adds Social Listening to Customer Decisions: Another Customer Data Platform

I’m still collecting examples to illustrate my new category of Customer Data Platform (CDP) systems. The latest is Provenir, a company founded in 1992 that has long sold a system to make credit risk and fraud decisions in real time. Over the past year, the company has added “social listening” capabilities and begun offering itself to marketing agencies as a customer interaction manager. It has met with good success and is now offering its “social listening platform” more broadly. *


It’s a slight stretch to call Provenir a CDP, because it doesn’t manage a permanent customer database.  Rather, like most interaction managers, it calls data from external sources during each decision.  But Provenir does have some customer matching capabilities and stores at least some information internally. Moreover, it completely meets the other three CDP criteria: predictive modeling, real-time decisions/recommendations executed through external systems, and a non-technical user interface. It’s also sold as the “glue” connecting data sources, modeling, and execution systems, which is exactly the role played by a CDP.  So, what the heck…welcome to the club!

Provenir is organized around process flows, which cover a particular task such as reacting to a Web site visit. Users define each process by building a flow chart, or, as the cool kids call them today, a graph.** These, um, graphs***, can contain branches, loops, and other advanced structures.  The nodes can also contain other graphs that define a subprocess in more detail. Nodes can perform a wide range of operations including data gathering, calculations, updates, decisions, and messages to external systems. Although setting these up is inevitably rigorous, Provenir makes it as painless as possible by providing help such as letting users draw lines to map fields from one system to another; building rules through score cards, tables and decision trees; and warning if a flow is incomplete.

Provenir relies on external systems to assemble, integrate, and store customer data.  Users can build matching processes with system graphs, although the vendor recommends connecting to other products to load reference data or do advanced "fuzzy" matching.  Provenir can monitor source systems for selected events and issue queries to assemble data as needed. The social listening features can monitor Twitter for keywords and Tweets by specified individuals.  These can trigger process flows that can retweet a message, send a direct Twitter message to the poster, or respond through another channel. The system can also monitor and post messages on Facebook. Other channels will be added over time.

Predictive modeling in Provenir is also done in external systems. The system can import PMML code or call models in SAS, R, or even Excel. Data mapping functions can automatically extract the list of required variables from PMML, do basic transformations and calculations when loading model inputs, and manage parameters, constants, and local variables.

Decisioning is Provenir’s greatest strength. The process flow…I mean graph…is inherently very flexible, and the ability to define rules as tables, trees, score cards, and other formats adds even more power. Users can set up champion/challenger tests as splits within a process flow; results are stored in a database for analysis and reporting. Users can also build simulated data sets, containing specified distributions of particular variables, and use these to forecast results of their flow designs. Such simulation is one mark of a mature decision system.

Provenir has some built-in messaging capabilities, but most decisions are executed externally.  The system has been connected with email, Web content management, call centers, campaign management, text messaging, and other execution platforms.

Pricing for Provenir’s social listening product is based on the size of the customer database. Starting price can be as a low as several thousand dollars per month. The system is usually sold on a Software-as-a-Service (SaaS) basis, but on-premise licenses are also available.


_______________________________________________________________
* For extra credit, compare and contrast Provenir’s primary Web site  with the site for their listening division.

** Defined in Wikipedia as “mathematical structures used to model pairwise relations between objects”.

*** Would it be even cooler to call them grafs or, better still, grafz?