Customer Data Platform Industry Profile: A Look Inside the Numbers

My snarky twin at the Customer Data Platform Institute just published a new report on the CDP industry. Since few industry vendors release financial or business details, the report relies on public sources including Owler for revenue estimates, Crunchbase for funding history, and LinkedIn for employee counts. Most vendors did provide client counts, and several privately shared other information where the public data was clearly wrong. You can download the report here. I'll wait while you do that.  (Sound of fingers tapping.)


Okay, you've downloaded it, right?  Good.

As you see, the report only presents figures for the industry as a whole. We feel those are reasonably accurate but that data for individual vendors are too unreliable to show separately. That may sound illogical but bear in mind that figures for the larger vendors are more reliable, so many errors that are significant for individual small vendors don’t materially change the total. Also remember that some vendors provided information in confidence and we made estimates of our own for some others.

I do feel I can safely publish statistics for three groups within the industry.  This gives some additional insight without exposing any proprietary or misleading vendor data.  The groups are based on each vendor's original business.  They are:

• Tag managers. This may seem an unlikely starting point, but it actually makes sense.  Tag management was originally about collecting data once (when a Web page loaded) and then sharing it with other systems that would otherwise have their own tags. This gave the Web site owner more control over what went where and reduced the number of tags on each page.  The data sharing was similar to what happens in integration platforms/data hubs  like Jitterbit and Zapier. So tag managers were always about data distribution.  To become true CDPs, the tag vendors had to ingest data from additional sources and send the data to a persistent database. Ingesting new sources can be challenging but vendors could grow incrementally by choosing which sources to accept.  Feeding a persistent data is basically just adding a new destination for data sharing.  So the transition to CDP offered a reasonable path to escape being a commodity tag manager.

• Campaign managers. I’m using this term loosely to include companies that offered any sort of marketing message selection. It includes systems that do email, Web site messages, mobile app messages, and omnichannel campaigns. These vendors all started out as CDPs in the sense that they always built unified customer databases. Among other things, this meant that most included reasonably robust cross-channel identity resolution. These vendors didn’t necessarily start by sharing their database with other systems.  But they do it now or I wouldn't consider them a CDP.

• Data assembly systems. This is a bit of a catch-all category but almost every system in this group was designed primarily to create a customer database that would be accessible to other systems.  Intended uses included analytics, marketing execution, or both. (I say "almost" because the group includes two systems that built databases primarily to support their own attribution services.) There’s more variety within this group than the other two.  But many vendors provide advanced identity resolution and all are strong at providing external access.

Here are key statistics for each group.

original purpose:	vendors	funding	2016 revenue	customer count	revenue / customer	employee count	revenue / employee
Tag Management	6	$356 million	$118  million	13,500	$9,000	840	$141,000
Campaign Management	8	$106 million	$108 million	1,000	$108,000	520	$207,000
Data Assembly	13	$246 million	$100 million	3,000	$34,000	920	$108,000

Here are some observations:

• Revenue is split about evenly among the three groups. That’s a bit surprising because tag management is an older and more established category than the others, so you might have expected it to have more revenue.  Vendors in the other categories do tend to be newer, smaller, and growing more quickly.

• Tag management vendors have many more customers and earn less revenue per customer. This largely reflects the original tag management products, which are sold to many non-enterprise customers. But the tag management vendors also have hundreds of enterprise clients.  Many of those clients are building the large-scale customer databases we expect to call a CDP.  The tag management group also includes a couple of vendors who specialize in building CDPs for smaller companies. These are not as expensive as the enterprise installations.  The campaign management vendors average about $100,000 per customer, which is what you'd expect for an enterprise CDP.  Revenue per customer is just $34,000 for the data assembly vendors, but that's largely due to one vendor with 2,000 clients.  Without that vendor, revenue per customer figures for the data assembly group would be $88,000. Backing out non-CDP clients is why the report puts the number of CDP customers for the entire industry at 2,500.

• Revenue per employee is generally in line with what we expect to see at growing Software as a Service companies. The standout here is the campaign management group, which has an impressively high figure of $207,000 per employee.  This suggests the campaign managers have a high value-added business.  The relatively low amount of external funding is more evidence that campaign managers throw off considerable cash from their own operations. The much lower revenue per employee for the data assembly companies, $108,000, is more typical of new SaaS ventures.  Indeed, several of these are just starting to earn revenue from their first clients.  These data assembly companies have attracted considerably more funding than the campaign managers, giving them a cushion to invest in growth. (If you’re wondering about that company with 2,000 clients, its revenue per employee is similar to others in its group.)

There are other nuances to consider in assessing these figures. For example, several vendors do business through agencies, which makes it harder to count clients and to compare revenue per client.  But the over-all picture that emerges is a healthy industry that is already attracting substantial revenue and funding.

The report projects a 50% annual growth rate, which yields an estimated $1 billion revenue for 2019.  The projection is based on public and private reported growth rates, which actually averaged much higher than 50% on a revenue-weighted basis.  The report used 50% to be conservative.  While past performance doesn't guarantee future growth, I think CDP revenues will if anything accelerate because most marketers still don't realize what a CDP can do for them.  As more of them get the message, CDP adoption should skyrocket. So the future is bright indeed.

NGData Gives Enterprise Marketers a Customer Data Platform of Their Own

If you read my recent post on Customer Data Platforms Revisited very, very closely, you might have noticed it listed a category of data vendors who “store unified profiles and expose to other systems”, which is pretty much the core definition of a Customer Data Platform. You would also have noticed that category had only two members, Aginity and NGData. I reviewed Agnity back in November 2013 and when I spoke with them more recently, found they were still doing pretty much the same thing and growing nicely. But, until today, I’ve never discussed NGData.

I wish I could come up with a really cool reason for that, but it’s only because NGData just recently came to my attention. The company itself has been serving clients in late 2012. As my classification suggests, they are in the business of assembling client data from multiple systems, using it to build detailed customer profiles, and making the results available to execution systems like campaign management, call centers, Web sites, and mobile apps.

The technology involved is Hadoop as the primary data store and HBase to expose the profiles to external systems. Data is unified mostly with customer IDs supplied by the source systems, although NGData can also build cross reference tables to associate related IDs and do some probabilistic matching to link related devices. The profiles include both raw data and calculated metrics such as trends, exceptions, signals, affinities, predictions of fraud risk, churn likelihood, and next product to buy.  The predictions can be based on conventional predictive methods like regression or on integrated machine learning. Often the company is able to use regression models that the client has built already for other data sources. The base version of the company’s system, called Lily Enterprise, has about 700 such metrics, and clients can add their own. (There’s also an open source version of Lily, which has only the data management components of Lily Enterprise.)  External systems access profiles via SQL queries against HBase, API calls, or file transfers.

This may all sound pretty straightforward and should be familiar to readers of this blog. But out there is the real world, most companies are still struggling with the challenges of assembling customer data and making it accessible.  Those firms will find this is pretty novel stuff. NGData stresses its ability to easily add new data sources and to retain huge amounts of detail, something it inherits from Hadoop. Complex metrics like exceptions and trends are also relatively unusual.  They make it much easier for execution systems to act intelligently, since the hard work of surfacing opportunities and selecting responses is largely done in advance.  Predefining those metrics as part of the base system speeds initial deployment.   One proof point: NGData says clients can typically start running programs based on its system in four to five months, and sometimes as quickly as three months. Conventional data warehouse projects usually take more than one year and many are never delivered.*

NGData stresses that marketers get better results when they can look at all the data associated with each individual, rather than treating people as members of broad segments. Again, no reader of this blog will disagree.  But I did still like their example of using behaviors to identify people who are likely to call customer support with a particular problem, and then preempting those calls with personalized messages about how to solve the problem. Everybody wins: the customer appreciates the proactive service, and company fields fewer phone calls. The underlying point, which NGData stresses often, is that they're going beyond insight to producing an actionable result.

NGData also highlights the value of responding to customer behaviors in real time or near real time.  It can do this because it immediately updates its metrics in Hbase whenever it receives new information. Still is more old news, but conventional solutions often update their scores and recommendations nightly or even less often. To be fair, NGData can only update in real time if the source systems are providing immediate updates – which often isn't the case.  The vendor does have its own Web tags to capture real time information about Web visits, which lets it use in-session behavior to drive product and page recommendations.

The one thing you may not expect about Lily is that it’s old-style on-premise software, not software-as-a-service. That’s mostly because current clients are big companies in financial services, telecommunications, and media, industries that have been reluctant to let precious company data outside their walls (although plenty of hackers still manage to get it, he added snarkily). It’s also not clear how much SaaS would benefit NGData, since each client’s data store would remain separate in any case and the software needs to be installed on relatively few desktops. For what it's worth, Aginity is also on-premise and also serves primarily enterprise clients. 

I suspect it's no coincidence that the two "pure" Customer Data Platform vendors both focus on enterprises.  Smaller firms need to justify a CDP by combining it with a specific application, but big companies can afford a separate CDP project that they'll later tie to separate execution systems.  Perhaps this will change as CDPs get cheaper, the model is better understood, and integration with execution systems becomes easier.

NGData has about 25 current paying clients in Europe and the U.S. Pricing is based on volume of data and/or number of customers. Total cost for software and services starts around $150,000 to $200,000 and can go much higher. As I said, this is enterprise software.

________________________________________________________________________________
*This is one of those things that “everybody knows” but few cab document. Here’s one paper that says data warehouses “usually” take 12 to 36 months, cost $1 to $1.5 million, and have a 70% failure rate. It’s not clear where the author got his data but it all sounds about right.

BlueConic Selects Targeted Messages Using a Cross-Channel Marketing Database

This blog has mentioned BlueConic in passing a couple of times but never quite gotten around to reviewing it in detail. Until now.

The delay may seem surprising, since BlueConic qualifies as a Customer Data Platform, a type of system I’ve been arguing will play an increasingly central role in marketers’ futures. For those of you who haven’t been paying close attention, a CDP is defined as a

• marketer-controlled system that
• supports external marketing execution based on
• persistent, cross-channel customer data.

This definition distinguishes CDPs from traditional marketing automation products, which do their own execution, and from real-time interaction managers, which lack persistent data stores. CDPs are important because few marketers have been able to build adequate cross-channel databases and because connecting those databases with execution systems has been difficult. The databases and connections are needed because today's customers expect personalized, coordinated treatments across all channels.  Call it the “Amazon fallacy”: customers believe that since Amazon.com can give them highly personalized treatments, so can everyone else.

Anyway, back to BlueConic. The system has two main capabilities, which are to maintain customer profiles and to deliver targeted messages. The profiles can be based on data imported from other systems via batch processes or APIs or captured by BlueConic itself.  It does this with “listeners” that can read data from forms or monitor behaviors via Javascript tags on Web pages, emails, and other media. “Listeners” can also create interest rankings and scores based on user behavior. All of these become available as attributes on the customer profile, which in turn can create customer segments and drive targeted messages.

The messages are delivered by what BlueConic calls “dialogues”, each of which sends a single message to a single location (email, text message, section on a Web page, etc.) to a specified customer segment.  Messages tailored to customer interests would require creating a separate segment for each message.  Similarly, presenting a sequence of messages would require a separate dialogue for each step in the sequence.  If a customer is eligible for several dialogues at once, the system currently relies on an optimizer to pick best-responding option and will soon let users create rules to further guide the results. There is no built-in predictive modeling but the optimizer can continuously test alternative messages within a dialogue and automatically deploy the winner. Users can also apply a frequency cap to dialogues to limit the number of times any customer sees the same message.

BlueConic’s integration features are more extensive than its decision management. The system can capture data entered into forms even if the form ise not submitted. Users can insert message contains into an existing Web page without writing HTML code. Profiles capture  customer identifiers provided by other systems and are automatically merged when two profiles are linked to the same external ID.  Data is exchanged with other sources and execution systems via REST APIs. There are standard integrations with Twitter, Facebook, and Salesforce.com, as well as a system development kit for integration with mobile apps. The underlying data store is Apache Cassandra running on Amazon Web Services, which is highly flexible and scalable at moderate cost.

Integration and data management are what make BlueConic most interesting from a CDP perspective, since those are the core CDP functions. A “pure" CDP would provide only those services while leaving decision management and message delivery to other systems. I expect “pure” CDPs to appear, but most marketers prefer a broader solution, like BlueConic, to assembling the components for themselves. Pure CDPs will become more attractive as integration becomes easier through more standard APIs and connectors, a promise that cloud-based systems often make but are just starting to deliver.

BlueConic’s pricing is already data-centric: fees are based on numbers of profile and channels, not interactions or messages. Prices start around $1,000 per month although most clients pay more. Current implementations are mid-size and enterprise firms in B2C industries including retail, publishing, financial services, utilities, telecommunications, sports and travel. The system has about 70 current customers and is sold both directly and to partners such as ad agencies, other software vendors, and marketing service providers.

Latest Marketing Technology Acquisitions Bypass Marketing Automation Vendors

On May 6, AOL and Google announced their respective acquisitions of attribution vendors Convertro  and Adometry. Since then, Acxiom purchased online/offline identity matching vendor LiveRamp and SAP bought behavioral targeting vendor SeeWhy, tag management vendor Tealium announced connectors to several email platforms, and the Phillies lost nine of 14 baseball games.

That last item is just included to see if you’re paying attention. But the others share at least two important connections: one for what they are, and another for what they’re not.

The positive connection is that all of these are about integrating customer data. Convertro and Adometry both import individual-level contacts across online and offline channels and use these with other data to estimate to incremental impact of each marketing message. (See my 2013 post on Adometry for details.)  Although the attribution vendors' primary focus is helping to measure and improve the results of digital advertising, the data they assemble incorporates all types of interactions and can ultimately support other types of analysis and execution. LiveRamp, a partner of both Convertro and Adometry, connects individual identities across different media such as email and Web sites. SeeWhy supports retargeting across ad networks, email messages, and during Web site visits, again relying on its ability to assemble a unified profile. Tealium, whose tags often capture data for those other solutions, is leveraging its position at the headwaters of the data stream by building its own persistent profiles and treatment rules.

These announcements all illustrate the increasingly central role that data is playing in marketing technology. But we’re not talking about just any type of customer data here: in each case, we’re talking about data related to display advertising, and about connecting that to data from a company’s direct customer interactions. The continued integration of those two worlds – which have been almost totally separate – is a tremendously important trend that will be driving marketing technology over the next several years.

It’s also worth noting that Convertro, Adometry, and SeeWhy all perform sophisticated statistical analysis while doing data assembly as an auxiliary function.  The obvious implication is that advanced analytics will become increasingly important, which is certainly correct. Less obviously, the data assembly could be performed in other systems without Convertro, Adometry or SeeWhy losing their primary value. Indeed, given the redundancy of having each system build its own central customer database and of each vendor investing in those essential-but-peripheral capabilities, both users and vendors would benefit from having separate systems that specialize in building the customer database and share the result with these other products. Yes, we’re talking here about Customer Data Platforms.

This brings us to what all these systems are not. The particular not-ness that I have in mind is not being a marketing automation system. After years of big marketing automation acquisitions, these transactions show industry giants investing in a different type of marketing technology. In this context, it’s relevant that SAP recently decided to resell Adobe’s marketing cloud rather invest in its own solution. I’m not ready to say that the great marketing automation boom is over or that marketing automation is approaching commodity status. But I do think that industry attention is shifting more to central analytics and decisioning in one direction and to advanced execution systems in the other. As stand-alone products in each of those areas become more sophisticated, marketing automation systems, which straddle both, may take a less central role in marketers' technology planning.

Aginity Puts a Customer Data Platform on an Analytical Appliance

When your only tool is a hammer, everything looks like a nail.  I’ve been illustrating the point recently by asking whether every system I see is really a Customer Data Platform (CDP). The question comes up because nearly every customer management system builds its own customer database, which is one core function of a CDP.  What distinguishes CDPs is that they make their database accessible to other execution systems and add some type of customer management intelligence. This intelligence ranges from behavior flags, segment codes, or predictive model scores to treatment recommendations to full-blown campaign management. Sometimes the enriched data is all that’s exposed to the execution systems, although usually the underlying customer profiles are available as well. Often the CDPs support just one stage of the customer life cycle, such as acquisition or retention: this in itself doesn’t disqualify a system, since I expect that they’ll expand in the future. The other key feature is that CDPs are designed to be run by marketers, not IT staff, even though IT will usually play a role in connecting to company-managed data sources.

I bring all this up partly to clarify that I'm actually being more selective than you might think in deciding what to call a CDP and partly because I’m writing today about Aginity, which refers to itself as a “customer insight appliance” but I think can rightly be classified as a CDP.  This in turn matters because CDPs solve a critical problem – marketers’ need for better customer databases – so identifying the widest possible range of CDP vendors increases the chances of each marketer finding a solution that fits her requirements.

On to Aginity itself. Functionally, the system is organized into layers for data loading, database management and analytics, and data consumption, which is exactly the model you’d expect from a CDP.  Where it differs from most CDPs is the underlying technology.  Aginity runs on a Netezza or similar "massively parallel processing" (MPP) data appliance that would typically run on-premise at the client, rather than being accessed remotely in a “Software as a Service” (SaaS) model.

Of course, most marketers couldn’t care less about this difference. They might care more if Aginity was a tool for IT departments, but in fact marketers can control most Aginity functions beyond the initial connections with source systems, and those connections require IT help even for SaaS systems.

Digging a bit deeper into the technical details (and feel free to skip the rest of this paragraph; it will not be on the final exam), Aginity uses a combination of relational and Hadoop data stores, which lets it add new data sources without formal data modeling.  It uses a simple wizard that lets non-technical users add new data elements and expose them on a metadata layer. The system automatically generates scripts to load new data and distribute it appropriately on the data appliance.   The system doesn't do the type of "fuzzy matching" needed to associate customer identities across different platforms when no direct link is available; it relies on the client or external partners to make those connections.

Once loaded, the data can be queried directly via SQL, typically using Aginity’s free Query Workbench, which is widely used for MPP databases throughout the industry.  Or the data can be published using other Aginity tools that create data marts for external analysis and execution systems. The publishing tools can be run by a marketing analyst, although Aginity says most clients let IT staff use them so IT can enforce quality standards, governance rules, backup management, and similar best practices.

The net result is that Aginity can have a new customer database available to marketing in 90 days or less (often much less), compared with the six to twelve months this typically requires. It’s this speed and flexibility that make me consider Aginity a tool for marketers – and thus a CDP – rather than a tool for IT departments.

Aginity also provides some analytical and customer management features of its own. These include ability to add derived attributes such as lifetime value calculations and segment codes to customer records. These attributes can call on any data gathered by the system, a critical advantage of a CDP. Customer lists can be fed to external systems for direct execution, such as sending an email, or can be loaded into data marts that external systems access with their own segmentation and campaign management tools. Aginity currently provides a range of analysis features including dashboards, profile reports, and segment migration over time. It relies on external systems for advanced analytics such as predictive modeling and plans tighter integration with such systems to allow more precise control over customer treatments.

Aginity was founded in 2006 as a service firm to assemble data for analytics and marketing execution. Its current product, first released in January 2012, is based on tools it developed as a service agency. The company’s clients are concentrated among large retailers but include some ecommerce, manufacturing, and other industries that handle large amounts of customer data.