Thursday, May 09, 2024

Filling the Gaps for a Composable CDP

Back in February, I mentioned that CDP Institute had published a Composable CDP Self-Assessment Tool that asks people what gaps they must fill to convert their current systems into the functional equivalent of a CDP. I recently checked how many responses we’ve received, and was disappointed that there were just fourteen. Obviously this isn’t enough to draw statistically meaningful conclusions, especially when you bear in mind that the audience of CDP Institute members can’t be considered representative of the industry as a whole. But I summarized the answers anyway to see what rough patterns might emerge. They were much more intriguing than I expected.

To set the context: the survey asks the status of 102 customer data management functions in the respondent’s current systems. These are grouped into eleven categories: data capture, data sources, ingestion, data preparation, data storage, identity linking, customer profiles, data sharing, process integration, segment creation, and segment (i.e., audience) output. Answer options are: not needed; needed and available; needed and not available; and don’t know. The most important of these is “needed and not available,” since that’s a gap to be filled. All questions were required.

When I took my first overview of the data, the first, and critically important, observation was that there did seem to be significant clusters in the answers. That was important because it suggested that people were giving accurate answers – at least, to the best of their knowledge – rather than randomly filling in a response. Had the answers been random, I would have expected to see roughly the same distribution of responses to each question, and that was definitely not the case. I’ll guess that requiring answers to all 102 questions filtered out the people who didn’t take the survey seriously.

The next observation was that gaps were more common in some areas than others. The gap percentages (i.e., percentage of "needed and not available" replies) ranged from 46% to 29%. The pattern is clear: the areas with fewest gaps (data sources, data store, data capture, and data sharing) are needed for pretty much any data warehouse, while the most gaps were tied to customer data management (data preparation, identity linking, ingestion, segment output, customer profiles).  This adds to the credibility of the data, even allowing for some confirmation bias.  More important, it's a reminder that you can’t assume a data warehouse built for other purposes will have all the features needed to support CDP applications.

A look individual functions shows something hidden by the category averages: even categories closely related to data warehouses will still have functional gaps for supporting a CDP. For example, while data warehouses are generally good at storing data, they often lack third party data and privacy management. Similarly, although capturing and sharing data are core capabilities for a data warehouse, they often lack real-time connectivity. This reinforces the need to look at requirements in detail when assessing the suitability of your existing warehouse as the base for a CDP.


It's also worth looking at the functions without the category filters. This shows more clearly that common data warehouse functions (such as loading structured data) are rarely gaps while CDP-specific functions (such as connections to advertising media, anonymous-to-known profile conversations, and end-user data access) are often gaps.  There are too few responses to make these rankings anything approaching precise. But, even if they were, what any individual company needs would still depend on its own situation.

Given the small sample size of this data set, it’s best to view these results more as anecdotes than an industry survey. But I still believe they offer some useful guidance to CDP vendors and users:

  • For vendors: it’s worth noting which categories show the most gaps. The “composable CDP” discussion was initially driven by companies offering reverse ETL features, which correspond roughly to the data sharing, segment creation, and segment output categories. Ironically, these fall in the middle of category gap rankings. The biggest gaps, and probably the greatest opportunity for CDP components, relate to specialized data preparation, identity linking, ingestion and segment output. I'd guess that the reason the "composable" discussion started elsewhere is tools for specialized data prep, identity linking, etc. are already widely available, so there was less room for new entrants.  By contrast, reverse ETL is a relatively new category where it's easier for a new firm to establish itself.  CDP is actually just one application of reverse ETL; in "crossing the chasm" terms, it's a beach head where new vendors can establish themselves and then grow beyond. 
From a different perspective, it's also not surprising that more composable CDP vendors are offering products to fill gaps in other CDP categories: it's always easier to sell a new product to an existing customer than to sell an existing product to a new customers.  Offering a more complete set of components begins to make the composable products look more like an integrated suite, losing some of the flexibility of swapping components but also reducing the integration burden that composable vendors otherwise ignore.  It still maintains the core difference between a standard CDP that builds a separate database and a "composable" one that draws from the warehouse.
  • For users: the list of common gaps is a helpful reference for ensuring that problematic functions are all considered in a requirements analysis. This is especially important at companies with strong existing data warehouse teams, whose members may not be familiar with CDP requirements.  To the extent that users' own systems match the category gap averages, it may make the most sense to consider custom enhancements to existing capabilities for categories with relatively few gaps, while buying new packaged components for categories with many gaps. Of course, if a company has many gaps across all categories, it probably makes more sense to buy a traditional CDP than to buy, integrate, and maintain a large number of separate components. 



Wednesday, May 01, 2024

CDP Success Depends on More than Marketers

The CDP Institute runs periodic roundtables where our members discuss industry issues.  We had one earlier this week on the topic of helping companies make use of an existing CDP.  We chose that topic because we frequently hear that many users don’t know what to do with a CDP after it’s built.

The initial discussion followed the more or less expected path towards solutions such as developing change champions and publicizing success stories to encourage CDP use.  Participants stressed the need for organizational change to allow applications that the CDP makes possible.  This change includes cooperation between teams that had previously not worked together, new processes, and new metrics for cross-channel marketing programs.  We heard that traditional direct marketing organizations are especially likely to struggle precisely because they have such mature single-channel techniques in place.

On the bright side, the group said that real-time capabilities were the most likely to create new opportunities that would make users excited enough to adopt new approaches.  Good to know.

The discussion then turned in an unexpected direction: the shift in CDP leadership from marketers to IT and data teams.  One participant said in the past two years, he had found that 90% of new projects were initiated by IT.  While that’s certainly a trend I’ve observed, the group added several new insights.

  • One reason for the shift is that CDP deployment requires more technical skill than most marketing teams can muster.  I have generally argued the reason for greater IT and data team involvement is the broader use of customer data beyond marketing departments.  This makes the CDP an enterprise-level project.  But it’s probably also true that CDPs used more widely throughout the organization are more technically complicated than CDPs used only in marketing.  Broader applications imply more data sources, more types of processing, and connections to more applications.  This all requires greater involvement by the company’s IT and data teams.

  • IT and data teams need new skills to succeed with a CDP.  This goes beyond the need to understand specific processes such as identity resolution, which many IT and data teams had not previously needed to support.  It includes the need for those teams to better understand how marketers actually do their jobs.  Without that fundamental understanding, IT and data teams will make poor choices in designing systems to support marketers.  Traditional project-by-project requirements gathering isn't enough.

  • More specifically, IT and data teams need to recognize that marketers have a different attitude towards data.  One participant observed (and I agree) that IT teams want to make sure everything is correct before they release any data, while marketers are willing to accept some errors in return for quicker results.  It’s an astute observation about culture that both sides should recognize and discuss so they can agree on the right balance for any particular application. 

  • A bigger role for IT and data implies a smaller role for marketing technologists.  They’re still involved in selecting martech, of course, but are less likely to lead a CDP project than they once were.  This probably applies to other marketing technology projects as well, especially if it applies to departments outside of marketing.  This isn't a death knell for martech staff.  But it does imply a shift in responsibilities, away from designing the martech stack to system administration, support, and analytics.  

Those final two points – about the different attitudes towards data quality and the changing role for martech staff – both point to a need for more CDP education aimed at IT and data teams, rather than just marketers.  That’s something we’ll consider here at CDP Institute and I suspect will be important for industry vendors, consultants, and practitioners as well.  

Friday, April 26, 2024

CDP Overview: How We Got Here, Where We're Going, and What Could Get in the Way

Has anyone asked you recently about the past, present, and future of Customer Data Platforms?  No?  That's odd; people ask me those questions

all the time.  Here are my current answers. 

It’s eleven years since the term “customer data platform” appeared in 2013.  What stages has it gone through over that time?

The first stage was just to recognize that CDP was a separate category of software.  What was new about CDP was that it was packaged software that was building a customer database.  Before then, customer databases were custom projects, like a data warehouse, and the only packaged marketing software was applications like campaign management systems or predictive modeling tools.  The earliest CDPs actually bundled the database building capability with an application.  But, fairly soon, vendors realized that there was more value in the database building features, which were rare, than in the applications themselves, which were fairly common.

Once we had identified the CDP category, the next stage was convincing people that it was important.  The concept itself was easy to grasp (“all customer data in one place”).  But there was initial skepticism about whether it was a legitimately separate category or just another name for existing technologies such as CRM, data warehouses or DMPs.  So we spent most of our time explaining the difference between CDP and other systems that also built customer databases.  In fact, the formal CDP Institute definition – "packaged software that builds a persistent, unified customer database accessible to other systems" – is carefully crafted so that each term identifies a differentiator between CDP and some other type of system.  “Packaged software”, for example, distinguishes CDP from data warehouses and data lakes, which are custom built.  I won’t go through the other elements point-by-point.

Once the concept was reasonably well defined, we faced skepticism about need for separate CDP database, compared to just reading existing source data in real time to build profiles.  That’s what ”persistent” refers to in the CDP definition.  It took the big martech vendors including Salesforce and Adobe several years to accept that.  The reason is  that building profiles on demand by assembling data in real time just takes too long.

Even after category was established, there was great confusion about what really qualified as a CDP.  That’s why the CDP Institute launched its RealCDP certification program, which expanded on our definition by defining five key requirements: assemble all data types, keep all details, retain the data indefinitely (subject to regulatory constraints), build unified profiles, and share the data with other systems.  We later added a sixth point relating to two real time capabilities: access to individual customer profiles, for things like call centers and website personalization, and real time event triggers, for things like responding to dropped shopping carts.  Of course, we are just one voice among many, and are easily drowned out by vendor promotions. So, unfortunately, some of that confusion persists to this day.

Part of the reason for the continuing confusion is a debate over whether CDP just builds profiles or also should include data activation capabilities such as analytics and personalization.  Our view is they’re not essential, but over time, we’ve seen the industry split between CDPs that only build profiles and CDPs that include activation functions.  The majority of CDP vendors provide activation, so that’s clearly what most buyers want.

More recently, we’ve seen interest in using customer data beyond marketing, which makes IT and data teams more interested in CDP projects.  That’s a big change because when CDP was just a marketing tool, IT teams were very happy to let marketers buy the CDP becaise it kept the marketers happy without consuming IT resources.  Now, CDP is too important to be left to marketers.  This also makes the CDPs that just build profiles more appealing in some situations – and we have in fact seen slightly faster growth in that group most recently.

IT involvement in turn brings more interest in companies building their own CDP, and leveraging their existing data lakes and warehouses as the foundation.  This has been called ‘composable CDP’ although it’s not really the right use of the term ‘composable’.  Most people now agree that 'warehouse-based' is a better label.  Semantics aside, the problem is that IT teams often underestimate the requirements for a proper CDP and, thus, the work involved in creating one.  So it’s important to ensure they do a thorough job of scoping the project in advance, so they make the best choice.

The other thing we’ve seen, more or less continuously, is expansion of CDP into new industries.  Originally they were used primarily in retail and media.  Then, they grew more common in financial services, hospitality, and telecom, which are all industries that traditionally had pretty good customer data systems.  Most recently, we’re seeing CDPs in education, healthcare, and government applications.

We’re also seeing CDP used more for advertising applications, as companies lean more heavily on first-party data to replace the loss of third-party cookies and replace other targeting methods that become harder as privacy rules become more stringent.  

In fact, CDP also supports other privacy-related applications, such as closer control over ensuring that contacts are authorized by consumer consent, and providing data clean rooms for privacy-safe data sharing.  Funneling all customer list creation through a CDP is one way to avoid breaking privacy rules.

Where do you see the industry headed next?  

The biggest issue right now is the movement towards ‘composability’.  We can’t control whether IT and data departments try to build their own CDP-equivalent.  But we can educate them about the actual requirements and we can give them tools that make it easier to meet those requirements.  Many CDP vendors are now breaking apart their systems to provide modules that companies can use as components in building an internal system.  Of course, that helps those vendors to survive a transition to a composable CDP world, but it also helps to maintain the reputation of the industry as a whole.  What we don’t want to see is composable CDP projects fail because that makes people question the value of the CDP concept itself.

Closely related to the composable trend is a trend for ‘no copy’ access to external data by a CDP.  This means the CDP can read data from other systems without copying it into the CDP data store.  That used to be more or less impossible at scale, because finding, reading, and integrating masses of external data took too long.  With today’s technologies, that process can be much faster, so it becomes a more practical alternative.  Some common use cases are reading things that change quickly and are only relevant at the moment you need them, like inventory levels or local weather conditions.  Of course, this ability also blurs the distinction between a traditional CDP, which imported all its data, and a warehouse-based CDP, which works with data stored externally.  That’s okay but it does add still more confusion to the discussion.  

I’ll also make a side note that the original CDP skeptics argued for reading source system data on demand, so it may seem this proves they were right.  But so far I don’t think you can have a CDP that only works by reading external data on demand, because some processes like identity resolution and data aggregation still take too long to do purely on the fly.  So I see the future CDP as having a core of data that it does copy and store in its own database, for things like the identity graph that tells how to combine data from different sources into each customer’s profile.  Whether that database is a CDP or data warehouse doesn't matter: either way, the data will have been copied from the original source system and preprocessed for use by the CDP.  The core data could well be supplemented with data read on-demand from external systems, which could be the original source or a data warehouse or data lake.  At least in theory, this would give you the best of both worlds: minimal data copying but maximum performance.

Putting aside composability, CDPs will need to adjust to new privacy rules by adding features like encryption, advanced privacy policy management, and data clean rooms.  They’ll adjust to new media and new data types, like audio and video, which need to be not just stored but analyzed in ways that are currently close to impossible.  And, of course, they’ll adjust to growing AI capabilities, which will make it easier to perform some functions like adding new data sources, matching identifiers that belong to the same person, building predictive models, and analyzing business results.  AI will also add to the volume and complexity of data that CDPs have to handle, which itself may require new technology to support.  For example, if AI starts to create a huge variety of content personalized for each individual, that makes result analysis vastly more complicated.  Somehow the CDP will need to deal with that.

On a more prosaic level, I expect to see more CDPs that are tailored to the needs of particular industries.  That’s a typical development for a mature software category.  Specialist systems can be more cost-effective to build and deploy, because they use special data structures and features tailored to a particular industry’s needs.  This will bring down the cost of CDPs, which has been an obstacle to expanding the base of CDP purchasers.  

And, as I’ve already mentioned, I expect to see CDPs used more widely beyond marketing.  One thing we’ve learned in recent years is that customers expect a personalized experience every time they interact with a company, whether it’s before they buy a product or after they start using it.  That requires making customer data available at every interaction point.  Beyond direct customer interactions, teams like product development and operations planning also can benefit from using customer data.

Are there any particular obstacles or threats to the CDP’s continued success?

Certainly CDPs will have to adapt to the changes we’ve already discussed in data types and volumes, in the users for customer data, and in whatever technical developments change the best way for CDPs to be built.  Individual vendors may struggle to keep up.  But I think the need for complete, sharable customer profiles is here to stay.  So to the extent that that’s the core definition of a CDP, the future of the industry is secure.

That said, I do see two major threats to the CDP industry as we know it today.  

The first is technical: the cloud databases from specialists like Snowflake and Databricks and general cloud platforms like Google Cloud, AWS, and Microsoft Azure.  All those vendors are increasingly eyeing the giant pile of customer data held in a CDP and wanting it for themselves.  To get it, they’re adding applications to build profiles, such as data quality and identity resolution, and to do analytics and marketing.  Sometimes they make the applications to be features to their own database systems; sometimes they build direct integrations with specialist applications through a marketplace of some sort.  Either way, this cuts out the CDP, which has traditionally been an intermediary between enterprise data stores and business applications.  Again, this comes back to the notion of the warehouse as the primary customer data store, which makes the CDP database unnecessary.  I think threat is limited today because most IT departments don’t have the resources to build those systems for themselves. But as new tools make it easier to build those systems, the threat becomes more important.  There are really just two ways for CDPs to respond to this threat:

  • one is to become platforms themselves, which is to say, to replace the data warehouse itself.  That’s not as crazy as it sounds, since the CDP is really a set of tools that builds the customer database, so it can work on top of a Snowflake or Google BigQuery or whatever database the client wants.  In this world, the CDP evolves from a tool to build customer profiles to a tool to build data warehouses in general.  Difficult but not impossible, at least for the largest CDP vendors who have the resources to compete.
  • The other option is to become applications on top of the warehouse, offering specialized capabilities like cross-channel journey orchestration.  The would build on CDPs’ existing capabilities to share their profiles with activation systems, and to themselves do activation functions that apply across all channels, such as predictive models and best offer selection.  It’s actually an easier path for most CDP vendors, since it continues their role as intermediaries between data sources and business applications.  But it’s also a fairly narrow role and there will be lots of competition.  Specialization by industry, company size, and other variables will be the key to avoiding that competition by becoming the best choice in a particular niche.

The second threat isn’t technical, but the ability of organizations to actually make use of a CDP once it’s built.  We run into this all the time, when companies tell us their staff doesn’t know what to do with a CDP or doesn’t have the skills to do what they’d like.  It’s a problem that will only get worse as customer data grows more complicated and there are more possibilities for using customer data.  Maybe AI will solve the problem, and that’s something CDP vendors need to invest in to make happen.  But there’s no guarantee that AI will be the answer, and my guess is that even AI will need skilled users to take advantage of the possibilities it creates.  

Of course, if the CDP turns out not to be useful, the staff members will blame the CDP, not their own lack of skill.  But it doesn’t really matter who takes the blame: if CDPs don’t create value, companies won’t be willing to invest in them.  So it’s really important for the CDP industry to help train CDP users, not just in the technical details of how to use a CDP, but in the business programs that a CDP can support. 

Monday, April 15, 2024

Send in the Clouds: Martech Moves to Cloud Platforms

Last weekend brought the intriguing rumor that Salesforce is in late stage talks to acquire data integration vendor Informatica.   This followed the previous week’s rumor that Google-parent Alphabet is consideringan offer for Salesforce-competitor HubSpot, and came the same week as a slew of partnership announcements tied to Snowflake’s Marketing Data Cloud Forum.

You don’t need a crazy wall to see how these events are connected.  Cloud database vendors including Google and Snowflake are expanding into marketing applications.  This poses an obvious threat to customer cloud vendors like Salesforce, in good part because it expands the ability of IT and data teams to build their own solutions rather than buying them.  Viewed in this light, Informatica helps Salesforce to compete by strengthening its ties with IT and data teams.  Of course, this is the same benefit that Salesforce sought with its Tableau, Slack and Mulesoft acquisitions, although you could argue those were all primarily tools for business users while Informatica is used primarily within IT and data teams.

A stronger strategic argument is that an Informatica that is properly integrated with Salesforce Data Cloud would help those teams build more powerful databases within Salesforce.  This would make Data Cloud a more viable alternative to Google Cloud or Snowflake as the company’s primary customer data store.  In Salesforce’s wildest dreams, Data Cloud might even replace data warehouses.

Let’s put aside the question of whether Salesforce could or should become an enterprise data warehouse supplier.  The immediate question is whether it can retain its position as a primary platform for customer data and customer-facing applications, or that role will be taken over by cloud platform vendors like Google Cloud, Amazon Web Services, and Microsoft Azure and cloud database specialists like Snowflake and Databricks. 

I don’t know the answer.  But what does seem clear is that customer systems will run on platforms, whether from Google, Salesforce, Snowflake, or someone else.   

The platform approach isn’t particularly new but it’s not the way things have always been.  Remember that most customer systems – even today – are stand-alone, self-contained products that maintain their own databases.  These are the dreaded silos we keep hearing about.  The trick is to connect these silos, initially at the data level by copying their data into a central warehouse or Customer Data Platform, and ultimately at the decision and delivery levels through shared journey orchestration and messaging. 

Platforms solve the data problem, since all systems run on a common data store.  They don’t necessarily unify the decision and delivery layers, since those functions are handled by applications built on the platform.  Those boundaries are not clear: platform vendors sometimes add decision and delivery functions.  Indeed, the risk that the platform vendor will incorporate your application is one that application developers must accept as the price for accessing the platform vendor’s customers.  Data clean rooms are a good example: once independent applications, they are now offered as core platform services.  Machine learning and artificial intelligence are following the same path.

The transition from silos to platforms is far from complete, but the industry direction is clear.  The entry of cloud and cloud data platforms as alternatives to customer clouds is likely to accelerate the change.  As we’ve already seen, platforms are a mixed blessing for application developers: they gain access to a larger audience but risk of being undercut by a platform product extension.  (The good news is those extensions often involve acquiring a leading application.)  

Application vendors also benefit from the platform providing data management functions that the application vendor would otherwise have to build for themselves.  This simplifies application development, freeing resources to improve the primary application functions.  Unfortunately, it also makes it easier for other companies to build competing applications.  This lower barrier to entry (and to continued survival) is one reason the number of marketing applications has increased so sharply in the past decade. 

The only way for application developers to escape this "app trap" is to themselves become platforms.  Many aspire to this and some, including HubSpot, have had some success moving down that path.  But most are too small to support a robust app marketplace and certification program or to attract a critical mass of application vendors.  

And what does all this mean for marketers and other business users?  Since the platform model isn’t new, we can already draw on experience.  It’s a mixed bag: buyers have a greater choice of applications for any particular task, which is mostly good but does create a burden of having to choose.  Buyers are tightly bound to whichever platform they select, since switching platforms means switching the related applications as well.  This creates a quasi-monopoly situation, with the potential for higher prices and poorer service that comes with it.  Raise your hand if you’ve seen that already.  This will only become worse as platforms become more common, making it harder for independent application vendors to survive and, thus, harder for companies to assemble their own ‘best of breed’ architecture from separate point solutions.  In other words, you may miss those silos when they’re gone.

It doesn’t have to be this way.  The alternative to a platform architecture is a true composable architecture, where different applications are written to common standards that are not controlled by any particular platform.  The ecommerce world has managed this to some extent, with ‘headless’ solutions and standards enforced by the MACH alliance.  Standard data access languages like SQL are another example.  That the customer data world will come up with similar standards seems doubtful, although there’s always hope.   

It’s also possible that integration platforms will let different systems interoperate even without shared standards.  That’s certainly their goal but so far results are limited by the effort to accurately capture all the nuances of one system and present them to another without losing anything in translation. AI might well change things by making this translation easier.  But AI might change a lot of things, so let’s not count on any of them quite yet.

For the immediate future, then, we can expect a handful of platform vendors to control customer data at a growing number of companies.  Those companies will be captives of the platforms, but it will be a fairly enjoyable captivity, with many pleasing applications to choose from and few obstacles to working as they please.  Every so often they’ll pull back a curtain and see the bars on the windows. Some will be so unhappy that they'll escape.  But most will accept the limits of their chosen platform and get on with their work.



Friday, February 16, 2024

Composable vs Packaged CDP: How Can We Help?

“Composable CDP” has been consuming much attention at CDP Institute as we wrestle with how best to help buyers who are considering it as an option.  The Institute published a Composable CDP Self-Assessment tool a few weeks ago, which gives a checklist of the functions required to replicate the profile-building capabilities of a standard CDP.  This was intended as the centerpiece of our Composable CDP Knowledge Hub but has attracted fewer users than expected.   The explanation I find most plausible is that few people can answer the Self-Assessment’s detailed questions about existing systems and requirements. 

In itself, this isn’t terrible: much of the value in the Self-Assessment comes from giving users a comprehensive checklist of items to consider, thereby highlighting the true scope of a Composable project.  Still, it’s clear the Self-Assessment isn’t helping buyers as much as we had expected, so we’re considering what other tools might be more useful.

The discussions surrounding this have helped to clarify my own understanding of where Composable CDP fits into the greater scheme of things.  The key insight is that the choice to use a Composable or packaged CDP is ultimately a tactical decision that addresses just one stage of the CDP project.  It doesn’t affect the previous stage, which is requirements definition, or the following stages, which are deployment and activation.  To overstate the case just a bit, marketers and other CDP users don’t care how their CDP gets built; they just want to use the resulting profiles to do their jobs.  To the extent that CDP users do care about the Composable vs packaged decision, it’s because that choice impacts the cost, timing, and quality of their system.

This insight in turn clarifies the distinction between knowing enough to make the Composable vs packaged decision, and actually deciding which is the best choice.  The knowledge needed to make the decision includes:

  • Defining business requirements, which requires business users who understand their needs. 
  • Understanding existing systems, so you can identify gaps that must be filled to meet business requirements
  • Understanding the organization’s capabilities for filling the gaps with either a Composable or packaged solution.  These capabilities relate to technical resources including staff skills and budgets.  I believe that assembling a Composable solution requires more extensive technical resources than buying a packaged CDP, although some Composable advocates might disagree.
  • Estimating the costs of delivering a Composable solution and a packaged solution so you can compare them.

The Self-Assessment tool addresses only the first two of those items: it asks users what they need and what their current systems can provide.  So, even if the users could answer all the questions, it wouldn’t provide all the information needed to make a sound decision.  Again, this isn’t exactly a flaw, since those are two important types of information.  But it is a limitation.

Only after all the information listed above has been gathered can the company address the second question of whether Composable or packaged is its best choice.  Answering this question depends on the combination of capabilities and costs.  That is, for Composable to be a viable option, the company needs to have adequate technical resources to assemble a Composable solution, and for Composable to be the best option, it has to offer the best combination of cost and value.  

(If we assume that either option delivers the same value, then the only difference is cost.  In theory, every system that meets same requirements will deliver the same value.  But in the real world, there will be differences in the quality of the resulting systems.  These will depend on the specific tools that are chosen, not simply on whether the solution is Composable or packaged.  This means that buyers must evaluate specific alternatives for either approach.)  

Circling back to the original question of how the Institute can help users who are considering a Composable CDP, it’s clear that few people will have all the information they need available when they start the process.   So the best we can do is to provide a framework that identifies the four kinds of information to collect and, perhaps, provides a place to store it over time.   That could be as simple as a spreadsheet, which isn’t very exciting from a technical standpoint.  But if it’s what our members need the most, then that’s what we’ll deliver.

Addendum:  Taking things a step further, here's what the tables described above could look like.  It would be interesting to collect separate answers from business users and IT/data teams, who very likely would disagree in many places.  I could easily convert these tables into an online format and have the system do some light analysis, including a comparison of answers from business users vs IT/data teams.  But we'd still run into the same problem, that it takes time assemble answers.  So this pushes me back to a spreadsheet that people can fill out at their leisure. You can download that here.

1 & 2: Requirements & Existing Systems

Data Sources

Not needed

Needed and Fully Available

Needed and Partly Available

Needed and Not Available

Don’t Know

- Website


- Data warehouse/

- data lake

- Email

- Third party data

- eCommerce

- Web advertising

- Social media advertising

- Point of Sale

Profile Building Functions

- Capture data

- Ingest data

- Prepare data

- Store data

- Link data

- Build profiles

- Share profiles

- Integrate profiles

- Segment profiles

Activation Functions

- Audience selection

- Predictive analytics

- Campaign definition

- Real time interactions

- Cross-channel orchestration


3. Resources

Rating (1=low, 5=high)

- Customer data infrastructure

- IT/Data team: customer data experience

- IT/Data team availability for this project

- Business users: customer data experience

- Business user availability for this project

- Training resources

- Measurement resources

- Budget for this project

- Management Support


4. Cost

Hard costs (enter dollar amounts):



- Licenses/Vendor Fees

- Labor/Development & Integration Costs

- Operations (hardware, software, on-going maintenance & upgrades)

Other Considerations: (1=disadvantage, 2=same, 3=advantage)

- Time to Value

- Delivery Risk (time, cost overruns)

- Performance Risk (scalability, performance)

- Roadmap (control over features)

- Security/privacy (control over data)

- Quality (best components, competitive advantage, vendor, usability/consistent interface)