When to Use a Proof of Concept in Marketing Software Selection -- And When Not

“I used to hate POCs (Proof of Concepts) but now I love them,” a Customer Data Platform vendor told me recently. “We do POCs all the time,” another said when I raised the possibility on behalf of a client.

Two comments could be a coincidence.  (Three make a Trend.)  But, as the first vendor indicated, POCs have traditionally been something vendors really disliked. So even the possibility that they’ve become more tolerable is worth exploring.

We should start by defining the term.  Proof of Concept is a demonstration that something is possible. In technology in general, the POC is usually an experimental system that performs a critical function that had not previously been achieved.  A similar definition applies to software development. In the context of marketing systems, though, a POC is usually not so much an experiment as a partial implementation of an existing product.  What's being proven is the system's ability to execute key functions on the buyer's own data and/or systems. The distinction is subtle but important because it puts the focus on meeting the client's needs.  

Of course, software buyers have always watched system demonstrations.  Savvy buyers have insisted that demonstrations execute scenarios based on their own business processes.  A carefully crafted set of scenarios can give a clear picture of how well a system does what the client wants.  Scenarios are especially instructive if the user can operate the system herself instead of just watching a salesperson.  What scenarios don’t illustrate is loading a buyer’s data into the system or the preparation needed to make that data usable. That’s where the POC comes in.

The cost of loading client data was the reason most vendors disliked POCs. Back in the day, it required detailed analysis of the source data and hand-tuning of the transformation processes to put the data into the vendor’s database.  Today this is much easier because source systems are usually more accessible and marketing systems – at least if they’re Customer Data Platforms – have features that make transformation and mapping much more efficient.

The ultimate example of easier data loads is the one-click connection between many marketing automation and CRM “platforms” and applications that are pre-integrated with those platforms. The simplicity is possible because the platforms and the apps are cloud-based, Software as a Service products.  This means there are no custom implementations or client-run systems to connect. Effortless connections let many vendors to offer free trials, since little or no vendor labor is involved in loading a client’s data. 

In fact, free trials are problematic precisely because so little work goes into setting them up. Some buyers are diligent about testing their free trial system and get real value from the experience. But many set up a free trial and then don't use it, or use it briefly without putting in the effort to learn how the system works.  This means that all but the simplest products don’t get a meaningful test and users often underestimate the value of a system because they haven’t learned what it can do.

POCs are not quite the same as free trials because they require more effort from the vendor to set up.  In return, most vendors will require a corresponding effort from the buyer to test the POC system.  On balance that’s a good thing since it ensures that both parties will learn from the project.

Should a POC be part of every vendor selection process? Not at all.  POCs answer some important questions, including how easily the vendor can load source data and what it’s like to use the system with your own data.  A POC makes sense when those are critical uncertainties.  But it’s also possible to answer some of those questions without a POC, based on reviews of system documentation, demonstrations, and scenarios. If a POC can’t add significant new information, it’s not worth the time and trouble.

Also remember that the POC loads only a subset of the buyer’s data. This means it won't show how the system handles other important tasks including  matching customer identities across systems, resolving conflicts between data from different sources, and aggregating data from multiple systems. Nor will working with sample data resolve questions about scalability, speed, and change management. The POC probably won’t include fine-tuning of data structures such as summary views and derived variables, even though these can greatly impact performance. Nor will it test advanced features related to data access by external systems.

Answering those sorts of questions requires a more extensive implementation.  This can be done with a pilot project or during initial phases of a production installation. Buyers with serious concerns about such requirements should insist on this sort of testing or negotiate contracts with performance guarantees to ensure they’re not stuck with an inadequate solution.

POCs have their downsides as well. They require time and effort from buyers, extend the purchasing process, and may limit how many systems are considered in depth.  They also favor systems that are easy to deploy and learn, even though such systems might lack the sophistication or depth of features that will ultimately be more important for success.

In short, POCs are not right for everyone. But it’s good to know they’re more available than before. Keep them in mind as an option when you have questions that a POC is equipped to answer.