Surprising Results in Customer Data Platform Survey

The Customer Data Platform Institute (CDPI) recently surveyed its members about their customer-facing systems and CDP deployment plans.  (Click here to download the full report.)  While CDP Institute members are obviously not typical marketers (being smarter, richer, and better looking), the answers still provide some intriguing insights into the marketplace.

Let’s start with the general state of customer facing systems. One-third reported they had many disconnected systems, just over one-third reported (37%) they had many systems connected to a central platform of some sort (9% unified customer database, 9% unified database and orchestration system, or 19% marketing automation or CRM platform), 6% said one system does almost everything, and the remaining 23% said they had some other configuration or didn’t know.

I’ve compared these results below with several other surveys that asked similar questions.

The one thing that immediately jumps out in this comparison is the CDPI Member survey showed a much lower percentage of replies for “one primary system”. Otherwise, the answers across all surveys are very roughly similar, showing about 30% to 50% of companies having many connected systems and many disconnected systems. This suggests more integration than I’d expect, but it depends on how much integration those “many connected” systems really represent.

The CDPI Member survey also asked about plans regarding Customer Data Platforms. Nineteen percent had a CDP already deployed, 18% had deployment in progress, and 26% planned to start deployment within the next 12 months. The remaining 38% either planned to start after 12 months (4%), had no plans to deploy (19%) or didn’t know (14%). I haven’t seen any other survey that asks this question but have no doubt that CDP deployment and plans are much higher for CDPI members than the rest of the industry average.

Where things get really interesting is when we explore how the same people answered these two questions. At first blush, you’d assume the 19% with a deployed CDP would be the same 18% who said they had many systems connected to a unified customer platform, either by itself or with an orchestration engine attached. Not so much. Here’s the actual cross tab of the results.

What you see (in the yellow cells) is just 42% of the people who said they had a deployed CDP also said they had many systems connected to a unified database. If we allow that a deployed CDP could be present in companies where one customer-facing system does almost everything or the customer facing systems are connected to marketing automation or CRM, then 74% of the deployed CDPs are covered.

I take the remaining 26% as a healthy reminder that just having a CDP doesn’t guarantee all your systems will connect to that CDP, either by feeding data into it or reading data from it. In fact, we know that many CDPs support analytics without being connected to delivery systems, so this really shouldn’t come as a surprise.

On a more encouraging note, as the green cells highlight, a good majority of in-process and planned CDP deployments are at companies with many disconnected systems or many systems connected to a marketing automation or CRM.  Those are the companies most in need of data unification. So it does appear that the CDP message is reaching its target audience and CDPs are being used as intended.

The survey also asked about company revenue, business type (B2B vs B2C), and region. Comparing those with current systems and CDP deployment also gave some interesting and unexpected results. But there's no point in repeating them here since you can download the full report and see for yourself.  Enjoy!

Here's a Game to Illustrate Strategic Planning

My wife is working on a Ph.D. in education and recently took a course on strategic planning for academic institutions. Her final project included creating a game to help illustrate the course lessons. What she came up with struck me as applicable to planning in all industries, so I thought I’d share it here.

The fundamental challenge she faced in designing the game was to communicate key concepts about strategic planning. The main message was that strategic planning is about choosing among different strategies to find the one that best matches available resources. That’s pretty abstract, so she made it concrete by presenting game players with a collection of alternative strategies, each on a card of its own. She then created a second set of cards that listed actions available to the players. Each action card showed which strategies the action supported and what resources it required. There were four resources: money, faculty, students, and administrative staff.  To keep things simple, she assumed that total resources were fixed, that each strategy contributed equally to the ultimate goal, and that each action contributed equally to whichever strategies it supported. 

In other words, the components of the game were:

- One goal. In the case of my wife’s game, the goal was to achieve a “top ten” ranking for a particular department within a university. (It was a good goal because it was easily understood and measured.)

- Four strategies. In my wife’s game, the options were to build up the department, cooperate with other departments at the university, cooperate with other universities, or promote the department to the media and general public.

- A dozen actions. Each action supported at least one strategy (scored with 1 or 0) and consumed some quantity of the four resources (scored from 0 to 3 for each resource). Actions were things like “run a conference”, “set up a cross-disciplinary course” and “finance faculty research”.

- Four resources, each assigned an available quantity (i.e., budget).

As you can tell from the description, the action cards are the central feature of the game.  Here's a concrete example, where each row represents one action card:

The fundamental game mechanism was to pick a set of actions.  These were scored by counting how how many supported each strategy and how many resources they consumed.  The resource totals couldn't exceed the available quantities for each resource.  The table below shows scoring for a set of three actions.

 In this particular example, all three actions support "cooperate with other departments", while two support "build department" and one each supports "cooperate with other universities" and "promote to public".  Resource needs were money=8, faculty=6, student=5 and administration= 1.  Someone with these cards could choose "cooperate with other departments" as the best strategy -- if the resources permitted.  But if they were limited to 7 points for each resource, they might switch the "fund scholarship" card for the "extracurricular enrichment" card, which uses less money even though it consumes more of the other resources.  That works because, with a budget of 7 for each resource, the player can afford to increase spending in the other categories.

As this example suggests, the goal of the game is to get players to think about the relations among strategies, actions and resources, and in particular how to choose actions that fit with strategies and resources.

Although the basic scoring approach is built into the game, there are many ways my wife could have played it:

- Predefine available resources and let different players draw different action cards.  They would then decide which strategy best fit the available cards and resources. 

- Give different strategy cards to different players and put all action cards face up on the table.  Players then each choose one action card in turn, trying to assemble the best set of actions for their assigned strategy.

- Randomly select the resource levels at the start of the game and let all players use all action cards.  The winner is whoever first finds combination of actions that yields the most points for any strategy without exceeding the resources available.

- Split the class into two teams, gave each team two strategy cards and a set of action cards, and let the winner be whichever team finds the combination of actions that comes closest to using all available resources. (That’s the one she chose.)

Other rules are possible, along with refinements such as making some strategies more valuable than others at reaching the goal and making some actions more effective than others at supporting a given strategy. But my wife had ten minutes to explain, play, score and discuss the game, so the simplifications made sense for her situation.

What I like about this game is that it clearly identifies the elements of the strategic planning process and shows how they’re related. Specifically, it highlights that:

- different strategies can reach the same goal. Identifying available strategies and choosing among them is an important part of strategic planning that's often not clearly recognized.

- different actions can support different strategies. This has two implications: strategies are initially chosen in part based on what actions are available and, later, actions are evaluated based on how well they support the chosen strategy.

- different actions can compete for the same resources. In the short run, the combination of actions must be chosen to maximize the value achieved from the resources available. In the long run, resources are not fixed, so organizations can decide which resources they need to support the actions they need for strategic success.

- different strategies are best suited to different combinations of resources. This is the ultimate message of the game. Actions are just intermediaries to help understand how specific strategies and specific resources are related.

I hope you find this interesting and perhaps even useful. It’s more thought experiment than actual game.  But if you’re inspired to create your own physical version, do send me pictures.

2017 Retrospective: Things I Didn't Predict

It’s the time of year when people make predictions. It’s not my favorite exercise: the best prediction is always that things will continue as they are, but what’s really interesting is change – and significant change is inherently unpredictable. (See Nassim Nicholas Taleb's The Black Swan  and Philip Tetlocks' Superforecasting on those topics.)

So I think instead I’ll take a look at surprising changes that already happened. I’ve covered many of these in the daily newsletter of the Customer Data Platform Institute (click here to subscribe for free). In no particular order, things I didn’t quite expect this year include:

- Pushback against the walled garden vendors (Facebook, Google, Amazon, Apple, etc.) Those firms continue to dominate life online, and advertising and ecommerce in particular. (Did you know that Amazon accounted for more than half of all Black Friday sales last week?) But the usual whining about their power from competitors and ad buyers has recently been joined by increasing concerns among the public, media, and government. What’s most surprising is it took so long for the government to recognize the power those companies have accrued and the very real threat they pose to governmental authority. (See Martin Gurri’s The Revolt of the Public for by far the best explanation I’ve seen of how the Internet affects politics.)  On the other hand, the concentrated power of the Web giants means they could easily converted into agents of control if the government took over.  Don’t think this hasn’t occurred to certain (perhaps most) people in Washington.  Perhaps that’s why they’re not interested in breaking them up.  Consistent with this thought: the FCC plan to end Net Neutrality will give much more power to cable companies, which as highly regulated utilities have a long history of working closely with government authorities. It’s pitifully easy to imagine the cable companies as enthusiastic censors of unapproved messages.

- Growth in alternative personal data sources. Daily press announcements include a constant stream of news from companies that have found some new way to accumulate data about where people are going, who they meet, what they’re buying, what they plan to buy, what content they’re consuming, and pretty much everything else. Location data is especially common, derived from mobile apps that most people surely don’t realize are tracking them. But I’ve seen other creative approaches such as scanning purchase receipts (in return for a small financial reward, of course) and even using satellite photos to track store foot traffic. In-store technology such as beacons and wifi track behaviors even more precisely, and I’ve seen some fascinating (and frightening) claims about visual technologies that capture peoples’ emotions as well as identities. Combine those technologies with ubiquitous high resolution cameras, both mounted on walls and built into mobile devices, and the potential to know exactly who does and thinks what is all too real. Cross-device matching and cross-channel identity matching (a.k.a. “onboarding”) are part of this too.

- Growth in voice interfaces. Voice interfaces don't have the grand social implications of the preceding items but it’s still worth noting that voice-activated devices (Amazon Alexa and friends) and interfaces (Siri, Cortana, etc.) have grown more quickly than I anticipated. The change does add new challenges for marketers who were already having a hard time figuring out where to put ads on a mobile phone screen.  With voice, they have no screen at all.  Having your phone read ads to you, or perhaps worse sing a catchy jingle, will be pretty annoying. To take a more positive view: voice interfaces will force innovation in how marketers sell and put a premium on agent-based services that make more decisions for consumers. Of course, that's only positive if the agents actually work in consumers’ interest. If the agents also serve other masters – such as companies that pay them to send business their way – consumers can easily be harmed. But at least they’ll have more time for things other than shopping.

- Retailers focus on convenience. Speaking of shopping: retailers with physical stores continue to panic about the growth of Amazon and other ecommerce vendors. What I find surprising isn’t the panic, but that their main reaction has been to introduce innovations like “BOPIS” (buy online, pick up in store) that focus on shopper convenience. Nothing will ever be more convenient than ordering remotely and having stuff delivered, so this is a game they’re guaranteed to lose. It’s clear to me that the future of in-store retail depends on creating entertaining, enjoyable experiences, and specifically on human interactions that online merchants can’t duplicate. Those interactions could be with store personnel, friends, and other shoppers. I’ll violate my rule against predictions and say here that categories of in-store retail that aren’t inherently entertaining will vanish (think: grocery shopping except maybe for fresh produce).  But I'll hedge my bet by not saying when.

- Marketing clouds increase their share. I don’t recall seeing any actual data on this, but my distinct impression from talking with buyers is that the major marketing clouds (Adobe, Salesforce, Oracle) are being bought by more companies. This shouldn’t have surprised me: I’ve spoken for years about “Raab’s Law”, which says that integrated suites always beat best-of-breed components in the long run. It seemed briefly that marketing technology would be different, largely because cloud-based systems make integration so much easier than before.  The vast profusion of specialized martech systems seemed to support this view. But it’s clear we’re now seeing “martech fatigue” set in, as marketers tire of purchasing an endless array of new systems they then barely use. One bolt of lightning to illuminate this was a recent Gartner survey that found martech spend is now falling. This bodes ill for independent martech vendors and suggests that the long-awaited consolidation may finally be at hand. The question really is how long the momentum of wilder times will carry many of today’s martech point solutions before they fall.

- Interest in self-service technology. I’ve seen several recent announcements related to the idea that marketers and non-technical users in other departments will develop their own systems using varying types of advanced technology. Chatbots, predictive models, and entire business process integrations have all been offered as things business users could create for themselves, often with a little help from artificially intelligent friends. Bosh, I say. Marketers are already overwhelmed by the complexity of their tools and in particular by the challenges of connecting separate systems. The cloud and AI might make this easier but they don’t make it easy. The growth in marketing clouds shows marketers voting with their budgets to avoid integration. To be clear, what surprises me is that people think self-service will work or even that it’s desirable. They should know better.

- No Customer Data Platform acquisitions. Okay, maybe I’m the only person who thinks about this. But it’s still pretty odd that none of the big marketing clouds has yet purchased a CDP vendor. (The only deal I recall was Campaign Monitor buying Tagga and those are not major players.) I can think of many ways to explain this: cloud vendors don’t see the problem, they think they’re already solved it, they don’t want to admit they haven’t solved it, they don’t want to reengineer existing products to use a CDP, they prefer to buy companies with large market share, they think they can build their own CDPs, etc. But, ultimately, the big marketing clouds haven’t purchased a CDP because their clients haven’t pushed them for one. The marketing clouds will act quickly once they start losing deals because clients want CDP functions the vendors can’t provide. Maybe the limited degree of integration within the existing marketing cloud architectures is enough, or maybe buyers don't realize it's inadequate until after they’ve made the purchase.  I fully expect acquisitions to happen – oops, another prediction – but not very soon. And if the clouds continue to increase their share without adding a CDP, maybe it won’t happen at all.

- No uptake on the idea of “personal network effects”. There’s no question that I’m the only person thinking about this one. But I continue to believe the concept (described here) is central to understanding how things really work in today’s online economy. I'm surprised other people haven't picked up on it, especially as they pay more attention to the power of the walled garden vendors. For example, anti-trust regulators are struggling with the fact that firms like Google, Facebook and Amazon are not monopolies in the conventional sense, are missing the point that such firms can monopolize the attention (and, thus, purchases) of individual consumers. If anybody wants to co-author a Harvard Business Review article on this, let me know. (I’m serious.)

So much for surprises. Lest you get the impression that I’m always wrong, I'll list some things that haven’t surprised me one bit.

- No pressure on privacy. While many people keep expecting consumers to really start caring about personal privacy, I’ve never seen any reason to think that will happen. Experience has shown that even the smallest reward is enough for people to expose pretty much everything there is to know about themselves. Sometimes it doesn’t even require paying money; just a bit more convenience or recognition is enough. If anything, people are getting more used to everything being public and thus less concerned about keeping anything private. They suspect, probably correctly, that it’s a losing battle.

- Lack of unified customer data. Marketers have talked for years about the need for a complete customer view and the integrated, omni-channel customer experience that’s needed to support it. It’s possible there’s actually been some progress: while surveys used to show 10-15% of marketers said they had a unified view, I’m now often seeing figures in the 25-40% range. I don’t believe the real numbers are anywhere that high but maybe they indicate a little improvement. Even so, given the important assigned to the topic, you might expect the problem would mostly be solved by now. I’m not surprised it hasn’t been: building a unified view is tough, more because of organizational obstacles than a lack of technology. So things will continue to move slowly.

- AI bubble remains unburst. Surely it’s time for people to stop getting excited every time they hear the term “artificial intelligence”? Apparently not; pretty much every new product I see announced, including vacuum cleaners and doorknobs, has an AI component. People will eventually expect AI to be built into everything, just as they expect electricity, plastics, and other previous miracle technologies. But it takes a long time for people to recognize what’s possible, so I’m not surprised they still find even basic AI features to be amazing.

Do Customer Data Platforms Need Identity Matching? The Answer May Surprise You.

I spend a lot of time with vendors trying to decide whether they are, or should be, a Customer Data Platform. I also spend a lot of time with marketers trying to decide which CDPs might be right for them. One topic that’s common in both discussions is whether a CDP needs to include identity resolution – that is, the ability to decide which identifiers (name/address, phone number, email, cookie ID, etc.) belong to the same person.

It seems like an odd question. After all, the core purpose of a CDP is to build a unified customer database, which requires connecting those identifiers so data about each customer can be brought together. So surely identity resolution is required.

Turns out, not so much. There are actually several reasons.

- Some marketers don’t need it. Companies that deal only in a single channel often have just one identifier per customer.  For example, Web-only companies might use just a cookie ID.  True, channel-specific identifiers sometimes change (e.g., cookies get deleted).  But there may be no practical way to link old and new identifiers when that happens, or marketers may simply not care.  A more common situation is companies have already built an identity resolution process, often because they’re dealing with customers who identify themselves by logging in or who transact through accounts. Financial institutions, for example, often know exactly who they’re dealing with because all transactions are associated with an account that’s linked to a customer's master record (or perhaps not linked because the customer prefers it that way). Even when identity resolution is complicated,  mature companies often (well, sometimes) have mature processes to apply a customer ID to all data before it reaches the CDP. In any of these cases, the CDP can use the ID it’s given and not need an identity resolution process of its own.
- Some marketers can only use it if it’s perfect. Again, think of a financial institution: it can’t afford to guess who’s trying to take money out of an account, so it requires the customer to identify herself before making a transaction. In many other circumstances, absolute certainty isn’t required but a false association could be embarrassing or annoying enough that the company isn’t willing to risk it. In those cases, all that’s needed is an ability to “stitch” together identifiers based on definite connections. That might mean two devices are linked because they both sent emails using the same email address, or an email and phone number linked because someone entered them both into a registration form. Almost every CDP has this sort of “deterministic” linking capability, which is so straightforward that it barely counts as identity resolution in the broader sense.

- Specialized software already exists. The main type of matching that CDPs do internally – beyond simple stitching – is “fuzzy” matching.  This applies rules to decide when two similar-looking records really refer to the same person. It's most commonly applied to names and postal addresses, which are often captured inconsistently from one source to the next. It might sometimes be applied to other types of data, such as different forms of an email address (e.g. draab@raabassociates.com and draab@raabassociatesinc.com). The technology for this sort of matching gets very complicated very quickly, and it’s something that specialized vendors offer either for purchase or as a service. So CDP vendors can quite reasonably argue they needn’t build this for themselves but should simply integrate an external product.

- Much identity resolution requires external data. This is the heart of the matter.  Most of the really interesting identity resolution today involves linking different devices or linking across channels when there’s no known connection. This sort of “probabilistic” linking is generally done by vendors who capture huge amounts of behavioral data by tracking visitors to popular Web sites or users of popular mobile applications, or by gathering deterministic links from many different sources. They then build giant databases (or "graphs" if you want to sound trendy) with these connections.  Even matching of offline names and addresses usually requires external data, both to standardize the inputs (to make fuzzy matching more accurate) and to incorporate information such as address and name changes that cannot be known by inspecting the data itself.  In all these situations, marketers need to use the external vendors’ data to find connections that don’t exist within the marketers’ own, much more limited information. If the external vendor provides matching functions in addition to the data, the CDP is relieved of the need to do the matching internally.

In short, there’s a surprisingly strong case that identity resolution isn’t a required feature in a CDP.  All the CDP really needs is basic stitching and connections to external services for more advanced approaches.  As cross-device and cross-channel matching become more important, CDPs will be more reliant on external vendors no matter what capabilities they’ve built for themselves. One important qualifier is the CDP implementation team still needs expertise in matching, so they can help clients set it up properly. But while it’s great to find a CDP vendor with its own matching technology, lack of that technology shouldn’t exclude a vendor from being considered a CDP.

No, Users Shouldn't Write Their Own Software

Salesforce this week announced “myEinstein” self-service artificial intelligence features to let non-technical users build predictive models and chatbots. My immediate reaction was that's a bad idea: top-of-the-head objections include duplicated effort, wasted time, and the potential for really bad results. I'm sure I could find other concerns if I thought about it, but today’s world brings a constant stream of new things to worry about, so I didn’t bother. But then today’s news described an “Everyone Can Code” initiative from Apple, which raised essentially the same issue in even clearer terms: should people create their own software?

I thought this idea had died a well-deserved death decades ago. There was a brief period when people thought that “computer literacy” would join reading, writing, and arithmetic as basic skills required for modern life. But soon they realized that you can run a computer using software someone else wrote!* That made the idea of everyone writing their own programs seem obviously foolish – specifically because of duplicated effort, wasted time, and the potential for really bad results. It took IT departments much longer to come around the notion of buying packaged software instead of writing their own but even that battle has now mostly been won. Today, smart IT groups only create systems to do things that are unique to their business and provide significant competitive advantage.

But the idea of non-technical workers creating their own systems isn't just about packaged vs. self-written software. It generally arises from a perception that corporate systems don’t meet workers’ needs: either because the corporate systems are inadequate or because corporate IT is hard to work with and has other priorities. Faced with such obstacles to getting their jobs done, the more motivated and technically adept users will create their own systems, often working with tools like spreadsheets that aren’t really appropriate but have the unbeatable advantage of being available.

Such user-built systems frequently grow to support work groups or even departments, especially at smaller companies. They’re much disliked by corporate IT, sometimes for turf protection but mostly because they pose very real dangers to security, compliance, reliability, and business continuity. Personal development on a platform like myEinstein poses many of the same risks, although the data within Salesforce is probably more secure than data held on someone’s personal computer or mobile phone.

Oddly enough, marketing departments have been a little less prone to this sort of guerilla IT development than some other groups. The main reason is probably that modern marketing revolves around customer data and customer-facing systems, which are still managed by a corporate resource (not necessarily IT: could be Web development, marketing ops, or an outside vendor). In addition, the easy availability of Software as a Service packages has meant that even rogue marketers are using software built by professionals. (Although once you get beyond customer data to things like planning and budgeting, it’s spreadsheets all the way.)

This is what makes the notion of systems like myEinstein so dangerous (and I don’t mean to pick on Salesforce in particular; I’m sure other vendors have similar ideas in development). Because those systems are directly tied into corporate databases, they remove the firewall that (mostly) separated customer data and processes from end-user developers. This opens up all sorts of opportunities for well-intentioned workers to cause damage.

But let’s assume there are enough guardrails in place to avoid the obvious security and customer treatment risks. Personal systems have a more fundamental problem: they’re personal. That means they can only manage processes that are within the developer’s personal control. But customer experiences span multiple users, departments, and systems. This means they must be built cooperatively and deployed across the enterprise. The IT department doesn't have to be in charge but some corporate governance is needed. It also means there’s significant complexity to manage, which requires some sort of trained professionals need to oversee the process. The challenges and risks of building complex systems are simply too great to let individual users create them on their own.

None of this should be interpreted to suggest that AI has no place in marketing technology. AI can definitely help marketers manage greater complexity, for example by creating more detailed segmentations and running more optimization tests than humans can manage by themselves. AI can also help technology professionals by taking over tasks that require much skill but limited creativity: for example, see Qubole, which creates an “autonomous data platform" that is “context-aware, self-managing, and self-learning”. I still have little doubt that AI will eventually manage end-to-end customer experiences with little direct human input (although still under human supervision and, one hopes, with an occasional injection of human insight). Indeed, recent discussions of AI systems that create other AI systems suggest autonomous marketing systems might be closer than it seems.

Of course, self-improving AI is the stuff of nightmares for people like Nick Bostrom, who suspect it poses an existential threat to humanity. He may well be right but it’s still probably inevitable that marketers will unleash autonomous marketing systems as soon as they’re able. At that point, we can expect the AI to quickly lock out any personally developed myEinstein-type systems because they won’t properly coordinate with the AI’s grand scheme. So perhaps that problem will solve itself.

Looking still further ahead, if the computers really take over most of our work, people might take up programming purely as an amusement. The AIs would presumably tolerate this but carefully isolate the human-written programs from systems that do real work, neatly reversing the “AI in a box” isolation that Bostrom and others suggest as a way to keep the AIs from harming us. It doesn’t get much more ironic than that: everyone writing programs that computers ignore completely. Maybe that’s the future Apple’s “Everyone Can Code” is really leading up to.

____________________________________________________________
*Little did we know.  It turned out that far from requiring a new skill, computers reduced the need for reading, writing, and math.

TrenDemon and Adinton Offer Attribution Options

I wrote a couple weeks ago about the importance of attribution as a guide for artificial intelligence-driven marketing. One implication was I should pay more attention to attribution systems. Here’s a quick look at two products that tackle different parts of the attribution problem: content measurement and advertising measurement.

TrenDemon

Let’s start with TrenDemon. Its specialty is measuring the impact of marketing content on long B2B sales cycles. It does this by placing a tag on client Web sites to identify visitors and track the content they consume, and then connecting client CRM systems to find which visitor companies ultimately made a purchase (or reached some other user-specified goal). Visitors are identified by company using their IP address and as individuals by tracking cookies.

TrenDemon does a bit more than correlate content consumption and final outcomes. It also identifies when each piece of content is consumed, distinguishing between the start, middle, and end of the buying journey. It also looks at other content metrics such as how many people read an item, how much time they spend with it, and how many read something else after they’re done. These and other inputs are combined to generate an attribution score for each item. The system uses the score to identify the most effective items for each journey stage and to recommend which items should be presented in the future.

Pricing for TrenDemon starts at $800 per month. The system was launched in early 2015 and is currently used by just over 100 companies.

Adinton

Next we have Adinton, a Barcelona-based firm that specializes in attribution for paid search and social ads. Adinton has more than 55 clients throughout Europe, mostly selling travel and insurance online. Such purchases often involve multiple Web site visits but still have a shorter buying cycle than complex B2B transactions.

Adinton has pixels to capture Web ad impressions as well as Web site visits. Like TrenDemon, it tracks site visitors over time and distinguishes between starting, middle, and finishing clicks. It also distinguishes between attributed and assisted conversions. When possible, it builds a unified picture of each visitor across devices and channels.

The system uses this data to calculate the cost of different types of click types, which it combines to create a “true” cost per action for each ad purchase. It compares this with the clients’ target cost per actions to determine where they are over- or under-investing.

Adinton has API connections to gather data from Google AdWords, Facebook Ads, Bing Ads, AdRoll, RocketFuel, and other advertising channels. An autobidding system can currently adjust bids in AdWords and will add Facebook and Bing adjustments in the near future. The system also does keyword research and click fraud identification. Pricing is based on number of clicks and starts as low as $299 per month for attribution analysis, with additional fees for autobidding and click fraud modules. Adinton was founded in 2013.  It launched its first product in 2014 although attribution came later.

Further Thoughts

These two products are chosen almost at random, so I wouldn’t assign any global significance to their features. But it’s still intriguing that both add a first/middle/last buying stage to the analysis. It’s also interesting that they occupy a middle ground between totally arbitrary attribution methodologies, such as first touch/last touch/fractional credit, and advanced algorithmic methods that attempt to calculate the true incremental impact of each touch. (Note that neither TrenDemon nor Adinton’s summary metric is presented as estimating incremental value.)

 Of course, without true incremental value, neither system can claim to develop an optimal spending allocation. One interpretation might be that few marketers are ready for a full-blown algorithmic approach but many are open to something more than the clearly-arbitrary methods. So perhaps systems like TrenDemon and Adinton offer a transitional stage for marketers (and marketing AI systems) that will eventually move to a more advanced approach.

 An alternative view would be the algorithmic methods will never be reliable enough to be widely accepted.  This would see these intermediate systems as about as far as most marketers ever will or should go towards measuring marketing program impact. Time will tell.

Flytxt Offers Broad and Deep Customer Management

Some of the most impressive marketing systems I’ve seen have been developed for mobile phone marketing, especially for companies that sell prepaid phones.  I don’t know why: probably some combination of intense competition, easy switching when customers have no subscription, location as a clear indicator of varying needs, immediately measurable financial impact, and lack of legacy constraints in a new industry. Many of these systems have developed outside the United States, since  prepaid phones have a smaller market share here than elsewhere.

Flytxt is a good example. Founded in India in 2008, its original clients were South Asian and African companies whose primary product was text messaging. The company has since expanded in all directions: it has clients in 50+ countries including South America and Europe plus a beachhead in the U.S.; its phone clients sell many more products than text; it has a smattering of clients in financial services and manufacturing; and it has corporate offices in Dubai and headquarters in the Netherlands.

The product itself is equally sprawling. Its architecture spans what I usually call the data, decision, and delivery layers, although Flytxt uses different language. The foundation (data) layer includes data ingestion from batch and real-time sources with support for structured, semi-structured and unstructured data, data preparation including deterministic identity stitching, and a Hadoop-based data store. The intelligence (decision) layer provides rules, recommendations, visualization, packaged and custom analytics, and reporting. The application (delivery) layer supports inbound and outbound campaigns, a mobile app, and an ad server for clients who want to sell ads on their own Web sites.

To be a little more precise, Flytxt’s application layer uses API connectors to send messages to actual delivery systems such as Web sites and email engines.  Most enterprises prefer this approach because they have sophisticated delivery systems in place and use them for other purposes beyond marketing messaging.

And while we’re being precise: Flytxt isn’t a Customer Data Platform because it doesn’t give external systems direct access its unified customer data store.  But it does provide APIs to extract reports and selected data elements and can build custom connectors as needed. So it could probably pass as a CDP for most purposes.

Given the breadth of Flytxt’s features, you might expect the individual features to be relatively shallow. Not so. The system has advanced capabilities throughout. Examples include anonymizing personally identifiable information before sharing customer data; multiple language versions attached to the one offer; rewards linked to offers; contact frequency limits by channel across all campaigns; rule- and machine learning-based recommendations; six standard predictive models plus tools to create custom models; automated control groups in outbound campaigns; real-time event-based program triggers; and a mobile app with customer support, account management, chat, personalization, and transaction capabilities. The roadmap is also impressive, including automated segment discovery and autonomous agents to find next best actions.

What particularly caught my eye was Flytxt’s ability to integrate context with offer selection.  Real-time programs are connected to touchpoints such as Web site.  When a customer appears, Flytxtidentifies the customer, looks up her history and segment data, and infers intent from the current behavior and context (such as location), and returns the appropriate offer for the current situation. The offer and message can be further personalized based on customer data.

This ability to tailor behaviors to the current context is critical for reacting to customer needs and taking advantage of the opportunities those needs create. It’s not unique to Flytxt but it's also not standard among customer interaction systems. Many systems could probably achieve similar outcomes by applying standard offer arbitration techniques, which generally define the available offers in a particular situation and pick the highest value offer for the current customer. But explicitly relating the choice to context strikes me as an improvement because it clarifies what marketers should consider in setting up their rules.

On the other hand, Flytxt doesn't place its programs or offers into the larger context of the customer lifecycle.  This means its up to marketers to manually ensure that messages reflect consistent treatment based on the customer's lifecycle stage.  Then again, few other products do this either...although I believe that will change fairly soon as the need for the lifecycle framework becomes more apparent.

Flytxt currently has more than 100 enterprise clients. Pricing is based on number of customers, revenue-gain sharing, or both. Starting price is around $300,000 per year and can reach several million dollars.

When to Use a Proof of Concept in Marketing Software Selection -- And When Not

“I used to hate POCs (Proof of Concepts) but now I love them,” a Customer Data Platform vendor told me recently. “We do POCs all the time,” another said when I raised the possibility on behalf of a client.

Two comments could be a coincidence.  (Three make a Trend.)  But, as the first vendor indicated, POCs have traditionally been something vendors really disliked. So even the possibility that they’ve become more tolerable is worth exploring.

We should start by defining the term.  Proof of Concept is a demonstration that something is possible. In technology in general, the POC is usually an experimental system that performs a critical function that had not previously been achieved.  A similar definition applies to software development. In the context of marketing systems, though, a POC is usually not so much an experiment as a partial implementation of an existing product.  What's being proven is the system's ability to execute key functions on the buyer's own data and/or systems. The distinction is subtle but important because it puts the focus on meeting the client's needs.  

Of course, software buyers have always watched system demonstrations.  Savvy buyers have insisted that demonstrations execute scenarios based on their own business processes.  A carefully crafted set of scenarios can give a clear picture of how well a system does what the client wants.  Scenarios are especially instructive if the user can operate the system herself instead of just watching a salesperson.  What scenarios don’t illustrate is loading a buyer’s data into the system or the preparation needed to make that data usable. That’s where the POC comes in.

The cost of loading client data was the reason most vendors disliked POCs. Back in the day, it required detailed analysis of the source data and hand-tuning of the transformation processes to put the data into the vendor’s database.  Today this is much easier because source systems are usually more accessible and marketing systems – at least if they’re Customer Data Platforms – have features that make transformation and mapping much more efficient.

The ultimate example of easier data loads is the one-click connection between many marketing automation and CRM “platforms” and applications that are pre-integrated with those platforms. The simplicity is possible because the platforms and the apps are cloud-based, Software as a Service products.  This means there are no custom implementations or client-run systems to connect. Effortless connections let many vendors to offer free trials, since little or no vendor labor is involved in loading a client’s data. 

In fact, free trials are problematic precisely because so little work goes into setting them up. Some buyers are diligent about testing their free trial system and get real value from the experience. But many set up a free trial and then don't use it, or use it briefly without putting in the effort to learn how the system works.  This means that all but the simplest products don’t get a meaningful test and users often underestimate the value of a system because they haven’t learned what it can do.

POCs are not quite the same as free trials because they require more effort from the vendor to set up.  In return, most vendors will require a corresponding effort from the buyer to test the POC system.  On balance that’s a good thing since it ensures that both parties will learn from the project.

Should a POC be part of every vendor selection process? Not at all.  POCs answer some important questions, including how easily the vendor can load source data and what it’s like to use the system with your own data.  A POC makes sense when those are critical uncertainties.  But it’s also possible to answer some of those questions without a POC, based on reviews of system documentation, demonstrations, and scenarios. If a POC can’t add significant new information, it’s not worth the time and trouble.

Also remember that the POC loads only a subset of the buyer’s data. This means it won't show how the system handles other important tasks including  matching customer identities across systems, resolving conflicts between data from different sources, and aggregating data from multiple systems. Nor will working with sample data resolve questions about scalability, speed, and change management. The POC probably won’t include fine-tuning of data structures such as summary views and derived variables, even though these can greatly impact performance. Nor will it test advanced features related to data access by external systems.

Answering those sorts of questions requires a more extensive implementation.  This can be done with a pilot project or during initial phases of a production installation. Buyers with serious concerns about such requirements should insist on this sort of testing or negotiate contracts with performance guarantees to ensure they’re not stuck with an inadequate solution.

POCs have their downsides as well. They require time and effort from buyers, extend the purchasing process, and may limit how many systems are considered in depth.  They also favor systems that are easy to deploy and learn, even though such systems might lack the sophistication or depth of features that will ultimately be more important for success.

In short, POCs are not right for everyone. But it’s good to know they’re more available than before. Keep them in mind as an option when you have questions that a POC is equipped to answer.

Wizaly Offers a New Option for Algorithmic Attribution

Wizaly is a relatively new entrant in the field of algorithmic revenue attribution – a function that will be essential for guiding artificial-intelligence-driven marketing of the future. Let’s take a look at what they do.

First a bit of background: Wizaly is a spin-off of Paris-based performance marketing agency ESV Digital (formerly eSearchVision). The agency’s performance-based perspective meant it needed to optimize spend across the entire customer journey, not simply use first- or last-click attribution approaches which ignore intermediate steps on the path to purchase. Wizaly grew out of this need.

Wizaly’s basic approach to attribution is to assemble a history of all messages seen by each customer, classify customers based on the channels they saw, compare results of customers whose experience differs by just one channel, and attribute any difference in results to that channel   For example, one group of customers might have seen messages in paid search, organic search, and social; another might have seen messages in those channels plus display retargeting. Any difference in performance would be attributed to display retargeting.

This is a simplified description; Wizaly is also aware of other attributes such as the profiles of different customers, traffic sources, Web site engagement, location, browser type, etc. It apparently factors some or all of these into its analysis to ensure it is comparing performance of otherwise-similar customers. It definitely lets users analyze results based on these variables so they can form their own judgements.

Wizaly gets its data primarily from pixels it places on ads and Web pages. These drop cookies to track customers over time and can track ads that are seen, even if they’re not clicked, as well as detailed Web site behaviors. The system can incorporate television through an integration with Realytics, which correlates Web traffic with when TV ads are shown. It can import ad costs and ingest offline purchases to use in measuring results. The system can stitch together customer identities using known identifiers. It can also do some probabilistic matching based on behaviors and connection data and will supplement this with data from third-party cross device matching specialists.

Reports include detailed traffic analysis, based on the various attributes the system collects; estimates of the importance and effectiveness of each channel; and recommended media allocations to maximize the value from ad spending.  The system doesn't analyze the impact of message or channel sequence, compare the effectiveness of different messages, or estimate the impact of messages on long-term customer outcomes. As previously mentioned, it has a partial blindspot for mobile – a major concern, given how important mobile has become – and other gaps for offline channels and results. These are problems for most algorithmic attribution products, not just Wizaly.

One definite advantage of Wizaly is price: at $5,000 to $15,000 per month, it is generally cheaper than better-known competitors. Pricing is based on traffic monitored and data stored. The company was spun off from ESV Digital in 2016 and currently has close to 50 clients worldwide.

Attribution Will Be Critical for AI-Based Marketing Success

I gave my presentation on Self-Driving Marketing Campaigns at the MarTech conference last week. Most of the content followed the arguments I made here a couple of weeks ago, about the challenges of coordinating multiple specialist AI systems. But prepping for the conference led me to refine my thoughts, so there are a couple of points I think are worth revisiting.

The first is the distinction between replacing human specialists with AI specialists, and replacing human managers with AI managers. Visually, the first progression looks like this as AI gradually takes over specialized tasks in the marketing department:

The insight here is that while each machine presumably does its job much better than the human it replaces,* the output of the team as a whole can’t fundamentally change because of the bottleneck created by the human manager overseeing the process. That is, work is still organized into campaigns that deal with customer segments because the human manager needs to think in those terms. It’s true that the segments will keep getting smaller, the content within each segment more personalized, and more tests will yield faster learning. But the human manager can only make a relatively small number of decisions about what the robots should do, and that puts severe limits on how complicated the marketing process can become.

The really big change happens when that human manager herself is replaced by a robot:

Now, the manager can also deal with more-or-less infinite complexity. This means we no longer need campaigns and segments and can truly orchestrate treatments for each customer as an individual. In theory, the robot manager could order her robot assistants to create custom messages and offers in each situation, based on the current context and past behaviors of the individual human involved. In essence, each customer has a personal robot following her around, figuring out what’s best for her alone, and then calling on the other robots to make it happen. Whether that's a paradise or nightmare is beyond the scope of this discussion.

In my post a few weeks ago, I was very skeptical that manager robots would be able to coordinate the specialist systems any time soon.  That now strikes me as less of a barrier.  Among other reasons, I’ve seen vendors including Jivox and RevJet introduce systems that integrate large portions of the content creation and delivery workflows, potentially or actually coordinating the efforts of multiple AI agents within the process. I also had an interesting chat with the folks at Albert.ai, who have addressed some of the knottier problems about coordinating the entire campaign process. These vendors are still working with campaigns, not individual-level journey orchestration. But they are definitely showing progress.

As I've become less concerned about the challenges of robot communication, I've grown more concerned about robots making the right decisions.  In other words, the manager robot needs a way to choose what the specialist robots will work on so they are doing the most productive tasks. The choices must be based on estimating the value of different options.  Creating such estimates is the job of revenue attribution.  So it turns out that accurate attribution is a critical requirement for AI-based orchestration.

That’s an important insight.  All marketers acknowledge that attribution is important but most have focused their attention on other tasks in recent years.  Even vendors that do attribution often limit themselves to assigning user-selected fractions of value to different channels or touches, replacing the obviously-incorrect first- and last-touch models with less-obviously-but-still-incorrect models such as “U-shaped”, “W-shaped”,  and “time decay”.  All these approaches are based on assumptions, not actual data.  This means they don’t adjust the weights assigned to different marketing messages based on experience. That means the AI can’t use them to improve its choices over time.

There are a handful of attribution vendors who do use data-driven approaches, usually referred to as “algorithmic”. These include VisualIQ (just bought by Nielsen), MarketShare Partners (owned by Neustar since 2015) Convertro (bought in 2014 by AOL, now Verizon), Adometry (bought in 2014 by Google and now part of Google Analytics), Conversion Logic, C3 Metrics, and (a relatively new entrant) Wizaly. Each has its own techniques but the general approach is to compare results for buyers who take similar paths, and attribute differences in results to the differences between their paths. For example: one group of customers might have interacted in three channels and another interacted in the same three channels plus a fourth. Any difference in results would be attributed to the fourth channel.

Truth be told, I don’t love this approach.  The different paths could themselves the result of differences between customers, which means exposure to a particular path isn’t necessarily the reason for different results. (For example, if good buyers naturally visit your Web site while poor prospects do not, then the Web site isn’t really “causing” people to buy more.  This means driving more people to the Web site won’t improve results because the new visitors are poor prospects.) 

Moreover, this type of attribution applies primarily to near-term events such as purchases or some other easily measured conversion.  Guiding lifetime journey orchestration requires something more subtle.  This will almost surely be based on a simulation model or state-based framework describing influences on buyer behavior over time. 

But whatever the weaknesses of current algorithmic attribution methods, they are at least based on actual behaviors and can be improved over time.  And even if they're not dead-on accurate, they should be directionally  correct. That’s good enough to give the AI manager something to work with as it tells the specialist AIs what to do next.  Indeed, an AI manager that's orchestrating contacts for each individual will have many opportunities to conduct rigorous attribution experiments, potentially improving attribution accuracy by a huge factor.

And that's exactly the point.  AI managers will rely on attribution to measure the success of their efforts and thus to drive future decisions.  This changes attribution from an esoteric specialty to a core enabling technology for AI-driven marketing.  Given the current state of attribution, there's an urgent need for marketers to pay more attention and for vendors to improve their techniques. So if you haven’t given attribution much thought recently, it’s a good time to start.

__________________________________________________________________________
* or augments, if you want to be optimistic.

Customer Data Platforms Spread Their Wings

I escaped from my cave this week to present at two conferences: the first-ever “Customer Data Platform Summit” hosted by AgilOne in Los Angeles, preceding Shop.org, and the Technology for Marketing conference in London, where BlueVenn sponsored me. I listened as much as could along the way to find what’s new with the vendors and their clients. There were some interesting developments.

• Broader awareness of CDP. The AgilOne event was invitation-only while the London presentation was open to any conference attendee, although BlueVenn did personally invite companies it wanted to attend. Both sets of listeners were already aware of CDPs, which isn’t something I’d expect to have seen a year or two ago. Both also had a reasonable notion of what a CDP does. But they still seemed to need help distinguishing CDPs from other types of systems, so we still have plenty more work to do in educating the market.

• Use of CDPs beyond marketing. People in both cities described CDPs being bought and used throughout client organizations, sometimes after marketing was the original purchaser and sometimes as a corporate project from the start. That was always a potential but it’s delightful to hear about it actually happening. The widely a CDP is used in a company, the more value the buyer gets – and the more benefit to the company’s customers. So hooray for that.

• CDPs in vertical markets. The AgilOne audience were all retailers, not surprisingly given AgilOne’s focus and the relation of the event to Shop.org. But I heard in London about CDPs in financial services, publishing, telecommunications, and several other industries where CDP hasn’t previously been used much. More evidence of the broader awareness and the widespread need for the solution that CDP provides.

• CDP for attribution. While in London I also stopped by the office of Fospha, another CDP vendor which has just become a Sponsor of the CDP Institute. They are unusual in having a focus on multi-touch attribution, something we’ve seen in a couple other CDPs but definitely less common than campaign management or personalization. That caught my attention because I just finished an analysis of artificial intelligence in journey orchestration, in which one major conclusion was that multi-touch attribution will be a key enabling technology. That needs a blog post of its own to explain, but the basic reason is AI needs attribution (specifically, estimating the incremental value of each marketing action) as a goal to optimize against when it's comparing investments in different marketing tasks  (content, media, segmentation, product, etc.)

If there's a common thread here, it's that CDPs are spreading beyond their initial buyers and applications.  I’ll be presenting next week at yet another CDP-focused event, this one sponsored by BlueConic in advance of the Boston Martech Conference. Who knows what new things we'll see there?

Vizury Combines Web Page Personalization with a Customer Data Platform

One of the fascinating things about tracking Customer Data Platforms is the great variety among the vendors.

It’s true that variety causes confusion for buyers. The CDP Institute is working to ease that pain, most recently with a blog discussion you’re welcome to join here.  But for me personally, it’s been endlessly intriguing to trace the paths that vendors have followed to become CDPs and learn where they plan to go next.

Take Vizury, a Bangalore-based company that started eight years ago as an retargeting ad bidding platform. That grew into a successful business with more than 200 employees, 400 clients in 40 countries, and $30 million in funding. As it developed, the company expanded its product and, in 2015, released its current flagship, Vizury Engage, an omnichannel personalization system sold primarily to banks and insurance companies. Engage now has more than a dozen enterprise clients in Asia, expects to double that roster in the next six months, and is testing the waters in the U.S.

As often happens, Vizury’s configuration reflects its origins. In their case, the most obvious impact is on the scope of the system, which includes sophisticated Web page personalization – something very rare in the CDP world at large. In a typical implementation, Vizury builds the client’s Web site home page.  That gives it complete control of how each visitor is handled. The system doesn't take over the rest of the client's Web site, although it can inject personalized messages on those pages through embedded tags.

In both situations, Vizury is identifying known visitors by reading a hashed (i.e., disguised) customer ID it has placed on the visitor’s browser cookie. When a visitor enters the site, a Vizury tag sends the hased ID to the Vizury server, which looks up the customer, retrieves a personalized message, and sends it back to the browser.  The messages are built by templates which can include variables such as first name and calculated values such as a credit limit.  Customer-specific versions may be pregenerated to speed response; these are updated as new data is received about each customer. It takes ten to fifteen seconds for new information to make its way through the system and be reflected in output seen by the visitor.

Message templates are embedded in what Vizury calls an engagement, which is associated with a segment definition and can include versions of the same message for different channels. One intriguing strength of Vizury is machine-learning-based propensity models that determine each customer’s preferred channel. This lets Vizury send outbound messages through the customer’s preferred channel when there’s a choice. Outbound options include email, SMS, Facebook ads, and programmatic display ads. These can be sent on a fixed schedule or be triggered when the customer enters or leaves a segment. Bids for Facebook and display ads can be managed by Vizury’s own bidding engine, another vestige of its origins. Inbound options include on-site and browser push messages.

If a Web visitor is eligible for multiple messages, Vizury currently just picks one at random. The vendor is working an automated optimization system that will pick the best message for each customer instead. There’s no way to embed a sequence of different messages within a given engagement, although segment definitions could push customers from one engagement to the next. Users do have the ability to specify how often a customer will be sent the same message, block messages the customer has already responded to, and limit how many total messages a customer receives during a time period.

What makes Vizury a CDP is that it builds and exposes a unified, persistent customer database. This collects data through Vizury's own page tags, API, and mobile SDK; external tag managers; and batch file loads.  Data is unified with deterministic methods including stitching of multiple identifiers provided by customers and of multiple applications on the same device. The system can do probabilistic cross-device matching but that's not reliable enough for most financial service applications.  Vizury doesn’t do fuzzy matching based on customer names and addresses, which is not a common technique in Asia.

The system includes standard machine learning algorithms that predict product purchase, app uninstalls, and message fatigue in addition to channel preference and ad bidding. Results can be applied to tasks other than personalization, such as lead scoring.  Algorithms are adapted for each industry and trained on the client’s own data. Users can't currently apply machine learning to other tasks.

Vizury uses a typical big data stack including Hadoop, Hive, Pig, Hbase, Flume, and Kafka. Clients can access the data directly through Hadoop or Hbase.  Standard reports show results by experience, segment, and channel, and users can create custom reports as well.


Pricing for Vizury is based on the number of impressions served, another echo of its original business. Enterprise clients pay upwards of $20,000 per month, although U.S. pricing could be different.

B2B Marketers Are Buying Customer Data Platforms. Here's Why.

I’m currently drafting a paper on use of Customer Data Platforms by B2B SaaS marketers.  The topic is more intriguing than it sounds because it raises the dual questions of  why CDPs haven’t previously been used much by B2B SaaS companies and what's changed.  To build some suspense, let’s first review who else has been buying CDPs.

We can skip over the first 3.8 billion years of life on earth, when the answer is no one. When true CDPs first emerged from the primordial ooze, their buyers were concentrated among B2C retailers. That’s not surprising, since retailers have always been among the data-driven marketers. They’re the R in BRAT (Banks, Retailers, Airlines, Telcos), the mnemonic I’ve long used to describe the core data-driven industries*.

What's more surprising is that the B's, A's, and T's weren't also early CDP users.  I think the reason is that banks, airlines, and telcos all capture their customers’ names as part of their normal operations. This means they’ve always had customer data available and thus been able to build extensive customer databases without a CDP.

By contrast, offline retailers must work hard to get customer names and tie them to transactions, using indirect tools such as credit cards and loyalty programs. This means their customer data management has been less mature and more fragmented. (Online retailers do capture customer names and transactions operationally.  And, while I don’t have firm data, my impression is that online-only retailers have been slower to buy CDPs than their multi-channel cousins. If so, they're the exception that proves the rule.)

Over the past year or two, as CDPs have moved beyond the early adopter stage, more BATs have in fact started to buy CDPs.  As a further sign of industry maturity, we’re now starting to see CDPs that specialize in those industries. Emergence of such vertical systems is normal: it happens when demand grows in new segments because the basic concepts of a category are widely understand.  Specialization gives new entrants as a way to sell successfully against established leaders.  Sure enough, we're also seeing new CDPs with other types of specialties, such as products from regional markets (France, India, and Australia have each produced several) and for small and mid-size organizations (not happening much so far, but there are hints).

And, of course, the CDP industry has always been characterized by an unusually broad range of product configurations, from systems that only build the central database to systems that provide a database, analytics, and message selection; that's another type of specialization.  I recently proposed a way to classify CDPs by function on the CDP Institute blog.** 

B2B is another vertical. B2B marketers have definitely been slow to pick up on CDPs, which may seem surprising given their frenzied adoption of other martech. I’d again explain this in part by the state of the existing customer data: the more advanced B2B marketers (who are the most likely CDP buyers) nearly all have a marketing automation system in place. The marketers' initial assumption would be that marketing automation can assemble a unified customer database, making them uninterested in exploring a separate CDP.  Eventually they'd discover that nearly all B2B marketing automation systems are very limited in their data management capabilities.  That’s happening now in many cases – and, sure enough, we’re now seeing more interest among B2B marketers in CDPs.

But there's another reason B2B marketers have been uncharacteristically slow adopters when it comes to CDPs.  B2B marketers have traditionally focused on acquiring new leads, leaving the rest of the customer life cycle to sales, account, and customer success teams.  So B2B marketers didn't need the rich customer profiles that a CDP creates.  Meanwhile, the sales, account and customer success teams generally worked with individual and account records stored in a CRM system, so they weren't especially interested in CDPs either.  (That said, it’s worth noting that customer success systems like Gainsight and Totango were on my original list of CDP vendors.)

The situation in B2B has now changed.  Marketers are taking more responsibility for the entire customer life cycle and work more closely with sales, account management, and customer success teams. This pushes them to look for a complete customer view that includes data from marketing automation, CRM, and additional systems like Web sites, social media, and content marketing. That quest leads directly to CDP.

Can you guess who's leading that search?  Well, which B2B marketers have been the most active martech adopters? That’s right: B2B tech marketers in general and B2B SaaS product marketers in particular. They’re the B2B marketers who have the greatest need (because they have the most martech) and the greatest inclination to try new solutions (which is why they ended up with the most martech). So it’s no surprise they’re the earliest B2B adopters of CDP too.

And do those B2B SaaS marketers have special needs in a CDP?  You bet.  Do we know those needs are?  Yes, but you’ll have to read my paper to find out.

_______________________________________________________
*It might more properly be FRAT, since Banking really stands for all Financial services including insurance, brokers, investment funds, and so on.  Similarly, Airlines represents all of travel and hospitality, while Telco includes telephone, cable, and power utilities and other subscription networks.  We should arguably add healthcare and education as late arrivals to the list.  That would give us BREATH.  Or, better still, replace Banks with Financial Services and you get dear old FATHER.

**It may be worth noting that part of the variety is due to the differing origins of CDP systems, which often started as products for other purposes such as tag management, big data analytics, and campaign management.   That they've all ended up serving roughly the same needs is a result of convergent evolution (species independently developing similar features to serve a similar need or ecological niche) rather than common origin (related species become different over time as they adapt to different situations).  You could look at new market segments as new ecological niches, which are sometimes filled by specialized variants of generic products and are other times filled by tangentially related products adapting to a new opportunity.

My point here is there are two separate dynamics at play: the first is market readiness and the second is vendor development.  Market readiness is driven by reasons internal to the niche, such as the types of customer data available in an industry.  Vendor development is driven by vendor capabilities and resources.  One implication of this is that vendors from different origins could end up dominating different niches; that is, there's no reason to assume a single vendor or standard configuration will dominate the market as a whole.  Although perhaps market segments served by different configurations are really separate markets.

AgilOne Adds New Flexibility to An Already-Powerful Customer Data Platform

It’s more than four years since my original review of AgilOne, a pioneering Customer Data Platform. As you might imagine, the system has evolved quite a bit since then. In fact, the core data management portions have been entirely rebuilt, replacing the original fixed data model with a fully configurable model that lets the system easily adapt to each customer.

The new version uses a bouquet of colorfully-named big data technologies (Kafka, Parquet, Impala, Spark, Elastic Search, etc.) to support streaming inputs, machine learning, real time queries, ad hoc analytics, SQL access, and other things that don’t come naturally to Hadoop. It also runs on distributed processors that allow fast scaling to meet peak demands. That’s especially important to AgilOne since most of its clients are retailers whose business can spike sharply on days like Black Friday.

In other ways, though, AgilOne is still similar to the system I reviewed in 2013. It still provides sophisticated data quality, postal processing, and name/address matching, which are often missing in CDPs designed primarily for online data. It still has more than 300 predefined attributes for specialized analytics and processing, although the system can function without them. It still includes predictive models and provides a powerful query builder to create audience segments. Campaigns are still designed to deliver one message, such as an email, although users could define campaigns with related audiences to deliver a sequence of messages. There’s still a “Customer360” screen to display detailed information about individual customers, including full interaction history.

But there’s plenty new as well. There are more connectors to data sources, a new interface to let users add custom fields and calculations for themselves, and workflow diagrams to manage data processing flows. Personalization has been enhanced and the system exposes message-related data elements including product recommendations and the last products browsed, purchased, and abandoned. AgilOne now supports Web, mobile, and social channels and offers more options for email delivery. A/b tests have been added while analytics and reporting have been enhanced.

What should be clear is that AgilOne has an exceptionally broad (and deep) set of features. This puts it at one end of the spectrum of Customer Data Platforms. At the other end are CDPs that build a unified, sharable customer database and do nothing else. In between are CDPs that offer some subset of what AgilOne offers: advanced identity management, offline data support, predictive analytics, segmentation, multi-channel campaigns, real time interactions, advanced analytics, and high scalability. This variety is good for buyers, since it means there’s a better chance they can find a system that matches their needs. But it’s also confusing, especially for buyers who are just learning about CDPs and don’t realize how much they can differ. That confusion is something we’re worrying about a lot at the CDP Institute right now. If you have ideas for how to deal with it, let me know.

Self-Driving Marketing Campaigns: Possible But Not Easy

A recent Forrester study found that most marketers expect artificial intelligence to take over the more routine parts of their jobs, allowing them to focus on creative and strategic work.

That’s been my attitude as well. More precisely, I see AI enabling marketers to provide the highly tailored experiences that customers now demand. Without AI, it would be impossible to make the number of decisions necessary to do this. In short, complexity is the problem, AI is the solution, and we all get Friday afternoons off. Happy ending.

But maybe it's not so simple.

Here’s the thing: we all know that AI works because it can learn from data. That lets it make the best choice in each situation, taking into account many more factors than humans can build into conventional decision rules. We also all know that machines can automatically adjust their choices as they learn from new data, allowing them to continuously adapt to new situations.

Anyone who's dug a bit deeper knows two more things:

• self-adjustment only works in circumstances similar to the initial training conditions. AI systems don’t know what to do when they’re faced with something totally unexpected. Smart developers build their systems to recognize such situations, alert human supervisors, and fail gracefully by taking an action that is likely to be safe. (This isn’t as easy as it sounds: a self-driving car shouldn’t stop in the middle of an intersection when it gets confused.)

• AI systems of today and the near future are specialists. Each is trained to do a specific task like play chess, look for cancer in an X-ray, or bid on display ads. This means that something like a marketing campaign, which involves many specialized tasks, will require cooperation of many AIs. That’s not new: most marketing work today is done by human specialists, who also need to cooperate. But while cooperation comes naturally to (most) humans, it needs to be purposely added as a skill to an AI.*

By itself, this more nuanced picture isn’t especially problematic. Yes, marketers will need multiple AIs and those AIs will need to cooperate. Maintaining that cooperation will be work but presumably can itself eventually be managed by yet another specialized AI.

But let’s put that picture in a larger context.

The dominant feature of today’s business environment is accelerating change. AI itself is part of that change but there are other forces at play: notably, the “personal network effect” that drives companies like Facebook, Google, and Amazon to hoard increasing amounts of data about individual consumers. These forces will impose radical change on marketers’ relations with customers. And radical change is exactly what the marketers’ AI systems will be unable to handle.

So now we have a problem. It’s easy – and fun – to envision a complex collection of AI-driven components collaborating to create fully automated, perfectly personalized customer experiences. But that system will be prone to frequent failures as one or another component finds itself facing conditions it wasn’t trained to handle. If the systems are well designed (and we’re lucky), the components will shut themselves down when that happens. If we’re not so lucky, they’ll keep running and return increasingly inappropriate results. Yikes.

Where do we go from here? One conclusion would be that there’s a practical limit to how much of the marketing process can really be taken over by AI. Some people might find that comforting, at least for job security. Others would be sad.

A more positive conclusion is it’s still possible to build a completely AI-driven marketing process but it’s going to be harder than we thought. We’ll need to add a few more chores to the project plan:

• build a coordination framework. We need to teach the different components to talk to each other, preferably in a language that humans can understand. They'll have to share information about what they’re doing and about the results they’re getting, so each component can learn from the experience of the others and can see the impact its choices have elsewhere.  It seems likely there will be an AI dedicated specifically to understanding and predicting those impacts throughout the system. Training that AI will be especially challenging. In keeping with the new tradition of naming AIs after famous people, let's call this one John Wanamaker. 

• learn to monitor effectively. Someone has to keep an eye on the AIs to make sure they’re making good choices and otherwise generally functioning correctly. Each component needs to be monitored in its own terms and the coordination framework needs to be monitored as a whole. Yes, an AI could do that but it would be dangerous to remove humans from the loop entirely. This is one reason it’s important the coordination language be human-friendly.  Fortunately, result monitoring is a concern for all AI systems, so marketers should be able to piggyback on solutions built elsewhere. At the risk of seeming overly paranoid, I'd suggest the monitoring component be kept as separate as possible from the rest of the system.

• build swappable components.  Different components will become obsolete or need retraining at different times, depending on when changes happen in the particular bits of marketing that they control. So we need to make it easy to take any given component offline or to substitute a new one. If we’ve built our coordination framework properly, this should be reasonably doable. Similarly, a proper framework will make it easy to inject new components when necessary: say, to manage a new output channel or take advantage of a new data source.  (This is starting to sound more like a backbone than a framework.  I guess it's both.)  There will be considerable art in deciding how what work to assign to a single component and what to split among different components. 

• gather lots of data.  More data is almost always better, but there's a specific reason to do this for AI: when things change you might need data you didn’t need before, and you’ll be able to retrain your system more quickly if you’ve been capturing that data all along. Remember that AI is based on training sets, so building new training sets is a core activity.  The faster you can build new training sets the faster your systems will be back to functioning effectively. This makes it worth investing in data that has no immediate use. Of course, it may also turn out that deeper analysis finds new uses for data even when there hasn’t been a fundamental change. So storing lots of data would be useful for AI even in a stable world.

• be flexible, be agile, expect the unexpected, look out black swans, etc.  This is the principle underlying all the previous items, but it's worth stating explicitly because there are surely other methods I haven't listed. If there’s a true black swan event – unpredictable, rare, and transformative – you might end up scrapping your system entirely. That, in itself, is a contingency to plan for. But you can also expect lots of smaller changes and want your system to be robust while giving up as little performance as possible during periods of stability.

Are there steps you should take right now to get ready for the AI-driven future? You betcha. I’ll be talking about them at the MarTech Conference in Boston in October.  I hope you’ll be there!


____________________________________________________________________________________
*Of course, separate AIs privately cooperating with each other is also the stuff of nightmares. But the story that Facebook shut down a chatbot experiment when the chatbots developed their own language is apparently overblown.**

** On the other hand, the Facebook incident was the second time in the past year that AIs were reported to have created a private language.  And that’s just what I found on the first page of Google search. Who knows what the Google search AI is hiding????