How to Build a CDP RFP Generator

Recent discussions with Customer Data Platform buyers and vendors have repeatedly circled around a small set of questions:

• what are the use cases for CDP? (This really means, when should you use a CDP and when should you use something else?)
• what are the capabilities of a CDP? (This really means, what are the unique features I’ll find only in a CDP? It might also mean, what features do all CDPs share and which are found in some but not others?)
• which CDPs have which capabilities? (This really means, which CDPs match my requirements?)
• can someone create a standardized CDP Request for Proposal? (This comes from vendors who are now receiving many poorly written CDP RFPs.)

These questions are intertwined: use cases determine the capabilities users need; requirements are the heart of an RFP, and finding which vendors have which capabilities is the goal of vendor selection. These connections suggest the questions could all be answered as part of one (complicated) solution. This might involve:

1. defining a set of common CDP use cases
2. identifying the CDP capabilities required to support each use case
3. identifying the capabilities available in specific CDPs
4. having buyers specify which use cases they want to support
5. auto-generating an RFP that lists the requirements for the buyer’s use cases
6. creating a list of vendors whose capabilities match those requirements

What’s interesting is that steps 1-3 describe information that could be assembled once and used by all buyers, while steps 5 and 6 are purely mechanical. So only step 4 (picking use cases) requires direct buyer input.  This means the whole process could be made quite easy.

(Actually, there’s one more bit of buyer input, which is to specify which capabilities they already have available in existing systems. Those capabilities can then be excluded from the RFP requirements. The capabilities list could also be extended to non-CDP capabilities, since most use cases will involve other systems that could have their own gaps.  These nuances don’t change the basic process.)

As a sanity check, I’ve built a small Proof of Concept for this approach using an Excel spreadsheet.  I'm happy to say it works quite nicely.  I'll share a simplified version here to illustrate how it works.  In particular, I'll show just a few capabilities, use cases, and (anonymous) vendors. .

We’ll start with the static data.

The columns are:

• Capability: a system capability.
• Description: a description of the capability. This can both help users understand what it is and be a requirement in the resulting RFP. Or, we could create separate RFP language for each capability. This could go into more detail about the required features.
• CDP Feature: indicates whether the capability would be found in at least some CDPs. The CDP RFP can ignore features that aren't part of the CDP, but it's still important to identify them because they could create a gap that makes the use case impossible.  For example, consider the first row in the sample table, whether the Web system can accept external input.  This isn't a CDP feature but it's needed to deliver the use case for Real time web interactions.
• Use Cases: shows which capabilities are needed for which use case. For items that relate to a specific channel, each channel would be a separate use case.  In the sample table, Single Source Access is specifically related to the Point of Sale channel while Real Time Interactions are specifically related to Web
• Vendor Capabilities: these indicate whether a particular vendor provides a particular capability.

The second table looks at the items that depend on user input. The only direct user inputs are to choose which use cases apply (not shown here) and to indicate which capabilities already exist in current systems.  All other items are derived from those inputs and the static data.

 

The columns are:

• Nbr Use Cases Needing: this shows how many use cases require this capability. It’s the sum of the capability values for the selected use cases.
• Already Have: this is the user’s input, showing which of the required capabilities are already available. In the sample table, the last row (site tag) is an existing capability.  Since it exists, you can leave it out of the RFP.
• Nbr Gaps: the number of use cases that need the capability, excluding capabilities that are already available. These are gaps. Using the number of cases, rather than a simple 1 or 0, provides some sense of how important it is to fill each gap.
• Nbr CDP Gaps: the number of gap use cases that might be enabled by a CDP. The first row iIn the example, Web – accept input (ability of a Web site to accept external input) isn’t a CDP attribute, so this value is set to zero.
• Gaps Filled by Vendor: the number of CDP Gaps filled by each vendor, based on the vendor capabilities. A total at the bottom of each column shows the sum for all capabilities for each vendor.  This gives a rough indicator of which vendors are the best fit for a particular user.

The main outputs of this process would be:

• List of gaps, prioritized by how many use cases each gap is blocking and divided into gaps that a CDP could address and gaps that need to be addressed by other systems.
• List of CDP requirements, which is easily be transformed into an RFP. A complete RFP would have additional questions such as vendor background and pricing.  But these are pretty much the same for all companies so they can be part of a standard template. The only other input needed from the buyer is information about her own company and goals. And even some goal information is implicit in the use cases selected.
• List of CDP vendors to consider, including which vendors fill which gaps and which have the best over-all fit (i.e., fill the most gaps). This depends on having complete and accurate vendor information and will be a sensitive topic with vendors who hate to be excluded from consideration before they can talk to a potential buyer.  So it's something we might not do right away.  But it’s good to know it’s an option.

Beyond the Basics

We could further refine the methodology by assigning weights to different use cases, to capabilities within each use case, to existing capabilities, and to vendor capabilities. This would give a much more nuanced representation of real-world complexity. Most of this could be done within the existing framework by assigning fractions rather than ones and zeros to the tables shown above. I’m not sure how much added value users would get from the additional work, in particular given how uncertain many of the answers would be.

We could also write some rules to make general observations and recommendations based on the inputs, such as what to prioritize. We could even add a few more relevant questions to better assess the resources available to each user and further refine the recommendations. That would also be pretty easy and we could easily expand the outputs over time.

What’s Next?

But first things first. While I think I’ve solved the problem conceptually, the real work is just beginning. We need to refine the capability categories, create proper RFP language for each category, define an adequate body of use cases, map each use case to the required capabilities, create the RFP template, and research individual vendor capabilities. I’ll probably hold off on the last item because of the work involved and the potential impact of any errors.

Of course, we can refine all these items over time.  The biggest initial challenge is transforming my Excel sheet into a functioning Web application. Any decent survey tool could gather the required input but I’m not aware of one that can do the subsequent processing and results presentation. A more suitable class of system would be the interactive content products used to generate quotes and self-assessments. There are lots of these and it will be a big project to sort through them. We’ll also be constrained by cost: anything over $100 a month will be a stretch. If anybody reading this has suggestions, please send me an email.

In the meantime, I’ll continue working with CDP Institute Sponsors and others to refine the categories, use cases, and other components. Again, anyone who wants to help out is welcome to participate.

This is a big project.  But it directly addresses several of the key challenges facing CDP users today. I look forward to moving ahead.

When to Use a Proof of Concept in Marketing Software Selection -- And When Not

“I used to hate POCs (Proof of Concepts) but now I love them,” a Customer Data Platform vendor told me recently. “We do POCs all the time,” another said when I raised the possibility on behalf of a client.

Two comments could be a coincidence.  (Three make a Trend.)  But, as the first vendor indicated, POCs have traditionally been something vendors really disliked. So even the possibility that they’ve become more tolerable is worth exploring.

We should start by defining the term.  Proof of Concept is a demonstration that something is possible. In technology in general, the POC is usually an experimental system that performs a critical function that had not previously been achieved.  A similar definition applies to software development. In the context of marketing systems, though, a POC is usually not so much an experiment as a partial implementation of an existing product.  What's being proven is the system's ability to execute key functions on the buyer's own data and/or systems. The distinction is subtle but important because it puts the focus on meeting the client's needs.  

Of course, software buyers have always watched system demonstrations.  Savvy buyers have insisted that demonstrations execute scenarios based on their own business processes.  A carefully crafted set of scenarios can give a clear picture of how well a system does what the client wants.  Scenarios are especially instructive if the user can operate the system herself instead of just watching a salesperson.  What scenarios don’t illustrate is loading a buyer’s data into the system or the preparation needed to make that data usable. That’s where the POC comes in.

The cost of loading client data was the reason most vendors disliked POCs. Back in the day, it required detailed analysis of the source data and hand-tuning of the transformation processes to put the data into the vendor’s database.  Today this is much easier because source systems are usually more accessible and marketing systems – at least if they’re Customer Data Platforms – have features that make transformation and mapping much more efficient.

The ultimate example of easier data loads is the one-click connection between many marketing automation and CRM “platforms” and applications that are pre-integrated with those platforms. The simplicity is possible because the platforms and the apps are cloud-based, Software as a Service products.  This means there are no custom implementations or client-run systems to connect. Effortless connections let many vendors to offer free trials, since little or no vendor labor is involved in loading a client’s data. 

In fact, free trials are problematic precisely because so little work goes into setting them up. Some buyers are diligent about testing their free trial system and get real value from the experience. But many set up a free trial and then don't use it, or use it briefly without putting in the effort to learn how the system works.  This means that all but the simplest products don’t get a meaningful test and users often underestimate the value of a system because they haven’t learned what it can do.

POCs are not quite the same as free trials because they require more effort from the vendor to set up.  In return, most vendors will require a corresponding effort from the buyer to test the POC system.  On balance that’s a good thing since it ensures that both parties will learn from the project.

Should a POC be part of every vendor selection process? Not at all.  POCs answer some important questions, including how easily the vendor can load source data and what it’s like to use the system with your own data.  A POC makes sense when those are critical uncertainties.  But it’s also possible to answer some of those questions without a POC, based on reviews of system documentation, demonstrations, and scenarios. If a POC can’t add significant new information, it’s not worth the time and trouble.

Also remember that the POC loads only a subset of the buyer’s data. This means it won't show how the system handles other important tasks including  matching customer identities across systems, resolving conflicts between data from different sources, and aggregating data from multiple systems. Nor will working with sample data resolve questions about scalability, speed, and change management. The POC probably won’t include fine-tuning of data structures such as summary views and derived variables, even though these can greatly impact performance. Nor will it test advanced features related to data access by external systems.

Answering those sorts of questions requires a more extensive implementation.  This can be done with a pilot project or during initial phases of a production installation. Buyers with serious concerns about such requirements should insist on this sort of testing or negotiate contracts with performance guarantees to ensure they’re not stuck with an inadequate solution.

POCs have their downsides as well. They require time and effort from buyers, extend the purchasing process, and may limit how many systems are considered in depth.  They also favor systems that are easy to deploy and learn, even though such systems might lack the sophistication or depth of features that will ultimately be more important for success.

In short, POCs are not right for everyone. But it’s good to know they’re more available than before. Keep them in mind as an option when you have questions that a POC is equipped to answer.

Rating the Crowd-Sourced Marketing Software Review Sites

What began as a whimsical “landscape of landscapes” led to the serious realization that crowd-sourced review sites are the most common type of vendor directory.  Fifteen of the 23 sources listed in my original graphic fell into that category. This begged for a deeper look at the review sites to understand how they differ and which, if any, could replace the work of professional reviewers (like me) and software guides (like my VEST report).

The first question was which sites draw a big enough crowd to be useful. I used Alexa traffic rankings, which are far from perfect but good enough for this sort of project. (Compete.com gave similar rankings except that TrustRadius came in lower, although still in the top 10.)  After adding two review sites that I learned about after the original post, I had 17 to consider. In order of their Alexa rankings, they were:

Since crowd wisdom without a crowd can’t be terribly effective, I limited further analysis to the top 10 sites. Of these, AlternativeTo.net, SocialCompare, and Cloudswave were different enough from the standard model that it made sense to exclude them. This left seven sites worth a closer look.

The next question was coverage by the sites of marketing technology. Every site except TrustRadius covered a broad range of business software from accounting to human resources to supply chain as well as CRM and marketing. TrustRadius was more focused on customer-related systems although it still had business intelligence and accounting. The numbers of categories, subcategories, and marketing subcategories all differed widely but didn’t seem terribly significant, apart from SoftwareInsider and DiscoverCloud looking a bit thin. Differences in the numbers of products in the main marketing categories also didn't seem meaningful  – although they do illustrate how many products there are, in case anyone needs reminding.

What did look interesting was the number of ratings and/or reviews for specific products. I sampled leading marketing automation vendors for different sized companies. It turns out that G2Crowd and TrustRadius had consistently huge leads over the others. I didn’t check similar statistics for other software categories, but this is probably the one that counts for most marketers.

Of course, quality matters as well as quantity. In fact, it probably matters more: my primary objection to crowd-sourced software reviews has always been that users’ needs for software are so varied that simple voting based on user satisfaction isn't a useful indication of how a system will for any particular buyer.  This is different from things like restaurants, hotels, and plumbers, where most buyers want roughly the same thing.

Software review sites address this problem by gathering more detail about both the products and the reviewers. Detailed product information includes separate numeric ratings on topics such as ease of use, value for money, and customer support; detailed ratings on specific features; and open-ended questions about what reviewers liked most and least, how they used the system, and what they’re recommend to others. Reviewer information on all sites except Software Advice starts with verifying that the user is a real person through requiring a LinkedIn log-in. This lets the review site check the reviewer’s name, title, company, and industry, although these are not always fully displayed. Some sites verify that the reviewer actually uses the product. Some provide other background about the reviewer’s activities on the review site and how their work has been rated.

I can't show how each vendor handles each of those items without going into excruciating detail. But the following table gives a sense of how much information each site collects. Of course, reviewers don’t necessarily answer all these questions. (Caution: this information is based on a relatively quick scan of each site, so I’ve probably missed some details. If you spot any errors, let me know and I’ll correct them.)  When it comes to depth, TrustRadius and DiscoverCloud stand out, although I was also impressed by the feature details and actual pricing information in G2Crowd.

The number and depth of reviews are clearly the most important attributes of review sites.  But they also differ in other ways.  Selection tools to identify suitable vendors are remarkably varied – in fact, the only filter shared by all sites is users' company size. Industry is a close second (missing only in DiscoverCloud), while even selections based on ratings are found in just four of the seven sites. Only three sites let users select based on the presence of specific features, an option I believe is extremely important.

 Looking beyond selection tools: most sites supplement the reviews with industry reports, buyer guides, comparison grids, and similar information to help users make choices. Several sites let users ask questions to other members.

So, back to my original question: can crowd-sourced review sites replace professional software reviews? I still don’t think so: the coherent evaluation of a practiced reviewer isn’t available in the brief comments provided by users, even if those comments are accompanied by information about specific product features. This may sound like self-serving mumbo-jumbo, but I do think a professional reviewer can articulate the essence of many products more effectively than users who report only on their personal experience. (Yes, I really just wrote "articulate the essence".)

But whether sites can replace professional reviewers is really the wrong question.  What matters is the value the review sites offer on their own. I’d say that is considerable: given enough volume, they indicate the rough market share of different products, the types of users who buy each system, and what worked well or poorly for different user types. User comments give a sense of what each writer found important and how they reached their judgements.  This in turn lets readers assess whether that reviewer’s needs were similar to their own. Buyers still need to understand their own requirements, but that’s something that no type of review can replace.

Seven Marketing Automation Myths to Ignore - Illustrated Edition

I’m sad.

I’ll be giving a speech in Milwaukee next week on marketing automation myths, and early in the preparation process had the idea of illustrating it with mythical creatures from the films of Ray Harryhausen, the stop-action animation genius whose best known images are probably the skeleton warriors in Jason and the Argonauts (1963).

This led to many pleasant hours scrolling through galleries of Harryhausen images. I even found an illustration that vaguely matched the theme of each myth.

But there’s a problem. Every bit of presentation-giving advice, training, and experience I’ve ever had tells me that these illustrations will distract attention from my points rather than reinforcing them. The responsible adult inside of me knows I have to get rid of them while the fun-loving child says, Yeah, but they're just so cool. 

This blog post is my compromise: I’ll publish them here, which will make dropping them from the actual presentation much less painful. **sigh**

So, the illustrated version of my talk goes like this:

Marketing Automation Myth Busting: we start with Mighty Joe Young, Harryhausen’s 1949 tribute to King Kong. I could tell you he’s about to smash some myths, but who are we kidding? It’s just a great image. 

Trouble in Paradise: marketing automation is growing quickly but users are dissatisfied. Maybe that’s not as bad as being attacked by a giant crab, but it’s still problematic. Image from Mysterious Island (1961).

Myth: All systems are the same.  This is an easy mistake because systems all look and sound alike during the buying process. But in fact they differ greatly. The myth leads buyers to think it doesn’t matter which system they purchase, and therefore that they can buy without first defining their requirements. In fact, our research shows that unsatisfied marketers often have purchased a system that didn’t meet their needs. Conversely, the most satisfied users did select based on specific features. The image here is Cyclops from The Seventh Voyage of Sinbad (1958). He has vision problems; it’s hard to see the differences between marketing automation systems. Get it?

Myth: Integration is easy. This echoes the first: all marketing automation products integrate with CRM, so people assume they don’t have to look into the details. But products differ hugely in which systems they connect with, what data they import and export, and how much control uses have over the details. Integration is the single most commonly cited obstacle to success and is linked to the most dissatisfied users. So people really need to ensure that the system they’re buying meets their integration needs. Kali from The Golden Voyage of Sinbad (1974) coordinates fighting with six arms, so she is the goddess of successful integration.

Myth: Failure is the user's fault, not the system's. This myth follows from the first two: if all systems are the same, then failure must be fault of the user. But, as we’ve seen, systems aren’t the same and many failures result from a system that doesn’t meet the user’s needs. Other research shows that users generally overcome obstacles they can control, like organization, training, and staffing levels. Of course, system selection is itself done by users, so they do have some responsibility for any problems. Talos, an animated statue from Jason and the Argonauts, ultimately fails to protect the tomb he is built to guard, so he represents a system that doesn’t work.

Myth: New users should crawl, walk, run.  Many experts – myself included – have suggested that new marketing automation users can safely start without planning by just duplicating their existing programs like email blasts, and then add more sophisticated uses over time.  But our research found that marketers who used more features from the start were happier. My interpretation is that successful marketers took the time to plan and train before deployment, while marketers who didn’t prepare in advance never found the time to learn what they needed. It’s possible to overstate this position – even successful users will add some new features over time. But the point about preparation is important. Kraken, from Clash of the Titans (1981), is a sea monster with no legs, so he never had a chance to move beyond crawling.

Myth: Bigger companies do better.  You might expect that bigger companies would do a better job with marketing automation because they have larger and more sophisticated staffs. They do in fact select more wisely, paying more attention to features and integration than marketers from smaller companies, and less to cost and apparent ease of learning. But they also face more non-technical obstacles such as training, staffing, and organizational barriers. So their over-all satisfaction level is no higher than smaller firms. I chose the giant octopus from It Came from Beneath the Sea (1955) because it’s big – no deeper meaning is intended.

Myth: Marketing automation creates prospects and saves money.  Marketers who expect their system to generate more prospects with less effort are usually disappointed. Marketing automation is basically about nurturing existing leads, not finding new ones, and most companies add staff and budget. Medusa, from Clash of the Titans, is the boss you don’t want to give bad news about system results: her dirty look will turn you to stone.

Myth: Marketing automation has stopped evolving.  Commoditization and consolidation may make marketing automation look like a mature industry.   But there's still plenty of change: new vendors entering the space, existing vendors being bought and repositioning themselves, and expanding scope to include consumer marketing, display ads, external data, better databases, identity resolution across channels, mobile apps and formats, advanced attribution, social promotions and new types of content.  The Beast from 20,000 Fathoms (1953) is a dinosaur who hasn’t evolved one bit.

So what? That’s the end of the myths, but we need to leave on a positive note.  So I end the presentation with some sound, if predictable, advice to prepare carefully, define and select against actual requirements, test integration in advance, deploy quickly, and expect the unexpected. The puzzled look on Troglodyte’s face, from Sinbad and the Eye of the Tiger (1977), represents the confusion marketers feel when wondering what to do next..

Marketers who want help selecting a system could try blowing on a ram's horn like Calibos from Clash of the Titans. Or they can just send me an email at draab@raabassociates.com.

*              *             *

Speaking of art that's amusing if irrelevant, here's a link to a Twilight Zone-themed introduction to a football recruiting show produced by my son Brian.  The apple doesn't fall far from the tree.

Vendor Selection Best Practices, Predictive Marketing Explained, Content Marketing Integration, and Other New Papers on Raab Web site

I've just gotten around to posting a  bunch of new white papers in the Resources section of the Raab Guide Web site.  Ordinarily you would need to register to view these, but I spent the whole afternoon putting this together, so I want as many people as possible to see them.  Here you go:

• Technology Evaluation Best Practices describes the challenges that marketers face in selecting the right systems and lists best practices to follow for success.

• What Matters Most In Selecting Marketing Automation explores survey results that highlight common errors when buying a marketing automation system and offers recommendations for how to avoid them.

• Why Modern B2B Marketers Need Predictive Marketing explains why predictive modeling has become essential to B2B marketers and describes specific applications.

• Defining Your Marketing Technology Strategy presents a framework for coordinating your marketing systems and links to an online tool that analyzes your current situations and recommends how to improve your systems.

• Content Marketing Integration Workbook provides a set of checklists to help marketers understand how to integrate content marketing with their other marketing programs at the strategic, operational, and technical levels.

• Questions to Ask When Selecting Your Customer Data Platform helps marketers to understand what a CDP is, whether they need one, and what specific questions to ask vendors when trying to pick the right system.

• The Customer Data Platform describes an emerging class of products that combine marketing database management, centralizing treatment decisions, and integration with execution systems.

Acquisitions Reshape the Marketing Automation Industry: Growth at the Bottom, Room in the Middle, Fog at the Top

Raab Associates officially released the new edition of our B2B Marketing Automation Vendor Selection Tool (VEST) yesterday. This is our flagship report on the industry, with nearly 200 data points on 23 vendors and separate ratings for micro-business, small to mid-size companies, and enterprise marketing departments. There are quite a few vendor comparisons out there, but none come close to the level of detail in the VEST – and details are what you really need to select a system. I personally suggest that anyone interested in the industry buy a copy for themselves and another for someone they love. See www.raabguide.com/vest for details.

I genuinely enjoy catching up with the vendors while preparing the VEST, but must admit that my favorite part of the process is analyzing the data once it’s assembled. Sadly, the wave of acquisitions that swept the industry in the past year has made this harder: many major vendors are now part of a public company, which severely restricts the information they can share. We’ve probably passed a tipping point where so much information is hidden that I can’t draw a clear picture of industry growth rates or competitive positions.

The table below shows the data available and highlights the holes. I’ve grouped the vendors into three buckets based on the market sectors they serve: micro-business (under $5 million revenue), small to mid-size business ($5 to $500 million), and large enterprises (over $500 million).

You’ll immediately see that the “not reported” information is concentrated among companies serving mid-size and enterprise clients, which is where all the acquisitions to date have taken place. Neolane is an exception but only because they provided the VEST information just before Adobe acquired them in June. I doubt we’ll see new numbers from them in the future. Marketo was mostly missing until they provided key figures in their earnings call this afternoon. Thanks, guys.

I've summarize my thoughts on this data with three oh-so-catchy phrases: growth at the bottom, opportunity in the middle, and fog at the top.

Growth at the Bottom: the green shading in the client growth column highlights companies reporting a year-on-year increase of 60% or more. What jumps out is the concentration at the top of the chart, in the micro-business sector. Four of the five micro-business vendors grew more than 60% and the fifth (Venntive) grew at a far-from-shabby 54%. There’s too much missing data in the other sectors to say for certain that the micro-business vendors are growing the fastest, but it sure looks that way. My interpretation is that the micro-business sector is the least mature and still presents the greatest untapped opportunity – even if buyers are still limited to the small proportion of business owners who are “tech geeks”.

Room in the Middle: Marketo's client count increased just 36% from mid-2012 to mid-2013 (although they’re projecting 54% revenue growth for 2013 vs. 2012).  We can no longer see the growth rates for mid-market heavy weights Pardot and Eloqua, but I’d be surprised if they beat Marketo.  They're certainly not close to the 67% to 90% rates reported by LeadFormix, Act-On, and eTrigue. I suspect Pardot, Eloqua and Marketo will increasingly focus on selling to enterprises, and in Marketo’s case on expanding footprint within existing clients. If so, this might open the way to faster growth by the next tier of mid-market vendors, who are mostly still private.  (LeadFormix is the exception, but seems to be pretty much left alone by its corporate parent). The clear winner in this scenario is Act-On, which has ample venture funding and has indeed been growing very rapidly. They are already the first vendor since Pardot to break the 150-employee barrier (blue shading). Silverpop and HubSpot might also benefit but neither is fully focused on standard B2B marketing automation. Other vendors would need outside funding to squeeze through what will probably be a briefly open window.

Fog at the Top: My visibility into enterprise B2B marketing automation was always clouded because of cross-over by B2C vendors including IBM, SAS, Teradata, and Neolane. It is now completely obscured except for sporadic glimpses of details that vendors choose to reveal. But even if everyone shared all their data with me, the enterprise picture would remain foggy because enterprises are increasingly integrating marketing automation with advertising , sales, service, and Web management. This makes it increasingly meaningless to treat marketing automation as a distinct category. Of course, that integration is exactly why the enterprise vendors purchased all those marketing automation systems in the first place.

If integration really happens at the top then we'll end up with a bizarre symmetry, since the enterprise market will be mirroring the integrated sales / CRM / Web / ecommerce products already bought by micro-businesses.  This would leave stand-alone marketing automation as a niche product for mid-tier companies. It would be a very large niche, but squeezed between broader suites from above and below and, eventually, challenged from within by integrated suites built for mid-market companies. The obvious response from marketing automation vendors is to build those broad suites themselves or to create platforms that are the foundation of such suites. That’s exactly what the larger mid-tier companies are doing, but it’s an expensive proposition. Any small mid-market companies who want to play must grab whatever fleeting opportunity the market offers today for growth, before they are locked out for good.

Social and Mobile Features Head the List of New Marketing Automation Capabilities

I’m getting ready for the next edition of the B2B Marketing Automation Vendor Selection Tool (VEST). This is based on nearly 200 questions to vendors, mostly about product features. The first step in the process is to update the list of questions. This is based on a review of recent vendor announcements plus my own feeling for what’s important. What emerges is an interesting portrait of industry trends in product development.

You won’t be surprised to learn that most of the changes involve social and mobile marketing, today's two hottest areas in marketing in general. We’ll get back to those in a bit. But first, I’d argue the single most important result is just how few changes there really were. B2B marketing automation is far from mature in terms of market penetration, but the mix of product features is pretty well set. Most of vendor announcements I reviewed were about common features that particular vendors had been lacking or were enhancing.  Social and mobile are the exceptions, but both are still very small contributors to most B2B marketing programs. I saw much more activity around features that were new last year, such as dynamic content and integration with Webinar systems and with Microsoft Dynamics CRM.

So exactly what new social and mobile features are now on my list? The previous report already included basic social capabilities including sharing marketing content to social media, tracking responses generated from social media, and monitoring social media activity. The new VEST expands that list to include:

- track social media influence: individual-level tracking mechanism that can identify the number of times a recipient has shared a promotion to social media and the number of responses generated the shared promotions. This information is part of the contact profile of the individual.

- create social media posts: deliver messages through social media, such as Twitter posts and Facebook updates. These messages can be created and then scheduled for future delivery.

- create social forms: create forms that are delivered within a third-party social media system such as Facebook.

- create social promotions: create social promotions such as contests, polls, ratings, etc.

- social sign-on and data capture: recipients can register using third-party social credentials, such as their Facebook ID. This gives access to information stored within the third-party social media system and allows communication through that system.

- build social profile: capture information about a specified individual by searching public information across multiple social media systems. This information includes social media handles and social activity such as posts, comments, and questions answered. The information is added to the individual profile and activity history.

The broad range of these features represents both a maturation of B2B social marketing and uncertainty about what will ultimately prove useful. We can expect more social features in the near future, although I suspect some will later be abandoned when it turns out they’re not especially effective in a B2B context.

On to mobile.  My previous list of mobile features was limited to text messaging. I’ve expanded that to add:

- mobile formats: generate Web and email versions in formats tailored to delivery on mobile devices such as smartphones and tablets.

- mobile CRM: salespeople can access the system on mobile platforms such as smartphones and tablets.

- mobile reporting: users can access reports on mobile platforms such as smartphones and tablets.

- mobile administration: users can set up campaigns and create content on mobile platforms such as smartphones and tablets.

Only the first of these, mobile formats, is about delivering marketing messages. The others are all about marketers and salespeople accessing the system on their own mobile devices. That’s clearly the current focus on mobile marketing automation, although it’s safe to expect more mobile marketing in the future – such as location-based promotions, which are notably absent so far.

I also added three entries in other categories. These were:

- app marketplace: the vendor has a formal app marketplace that lets third party applications connect to its product without custom integration.

- real time recommendations: rules and/or predictive models can recommend the best treatment for a customer as an interaction takes place within system-managed content such as a Web page.

- real time interactions: rules and/or predictive models can recommend the best treatment for a customer as an interaction takes place within an external platform such as a call center or Web site. This requires features to collect information about the interaction from the external platform, to match this information against the system's own database of contacts profiles and history, to make recommendation using the available information, and to deliver the recommendation to the external platform. .

These features all expand the scope of B2B marketing automation, mostly be connecting it with other systems. In one sense that's the opposite of the previous new entries, which were about adding features to marketing automation itself.  But both approaches aim to place marketing automation at the center of a company’s customer management infrastructure. Since other products, including CRM and Web sites, are also reaching for that position, we’ll see how widely these features get adopted. My sense is they’ll be more successful at small companies, where the labor savings of a unified system are most important because technology resources are most constrained.

None of the features I’ve added are currently available in more than a handful of systems.  Some may not yet be present in any. Few marketers this year will choose a system primarily because these particular features are present.  But we'll find over time which are really important.

New Workbook: Estimating the Cost of Marketing Automation

We released two more vendor selection workbooks last week, both sponsored by Eloqua and available for free on the RaabGuide Web site. One is about estimating the cost of a marketing automation system and the other is about evaluating vendor services.

The cost workbook was a particular challenge because the subject is so complex. After much thought, I came up with four cost categories:

• direct system costs: the actual price paid for the marketing automation software itself. This is where most buyers focus their analysis, but it’s a tiny fraction of the value at play.  Background research for the workbook suggested that automation costs are from 1% to 5% of an average marketing budget. This means that the direct system cost is pretty much insignificant compared with the marketing budget it will help to manage. To put it another way: even a small improvement in the other 95% to 99% of marketing costs can easily pay for a marketing automation system.

• operations costs: other costs related to running the marketing automation system, such as staff time and costs of related systems. Most of these are marketing operations costs, and there may be others in sales and IT.  You’re already incurring many of them, so the analysis has to identify how much they’ll increase or decrease as a result of marketing automation. This is really hard since it takes a detailed understanding of your current processes and how they'll change.  But the stakes are high: operations costs are about 25% of a typical marketing budget.

• marketing program costs: the expenses for specific marketing programs, such as advertising, trade shows, email, etc. These are the other 75% of the typical marketing budget. Marketing automation can reduce these costs substantially, both through reduced waste and through shifting funds to more effective programs.

• revenue: many marketers shy away from building revenue gains into their marketing automation calculation, but revenue is ultimately the reason for their investment. From an analytical perspective, it’s important not to double-count revenue gains (assuming the same marketing budget) and cost savings (from a lower marketing budget).  It's also important to recognize that revenue isn’t 100% profit. The workbook describes how to do this.  To keep things in perspective: marketing costs are under 10% of revenue at most firms, meaning that direct marketing automation are under 0.5% of revenue.

Analyzing all four items in depth is a big job. The good news is you don’t usually need to assess them all at once. 

• building a business case for marketing automation needs only a rough estimate for direct system costs, since they’re so small compared with the revenue, marketing program costs, and operations.  

• comparing marketing automation systems lets you focus on the system costs and changes in marketing operations costs, since those may vary considerably from one product to another. The impact on program costs and revenues should be about the same unless you're considering systems with widely different capabilities.  But hopefully you identified your needs earlier in the process, so you'll only be comparing similar systems once you reach the final evaluation..

It's useful to understand these cost categories, but the real work is gathering the details for each component.  This is where the workbook comes in: it lists of specific items to consider, so you have a framework to help ensure your analysis is complete. This is important: to take a real-world example, one of my consulting clients recently received quotes from two marketing automation vendors, one of which included email delivery and one of which did not. Recognizing the difference made it easy to prepare a true apples-to-apples comparison, but we could have easily missed it until later the process if we had not used a formal framework.

Vendor Selection: Writing a Good Requirements Document

My last two posts (not counting this morning’s detour into Marketo-land) described common errors marketers make when selecting marketing automation systems. How did we come to this?

I see two reasons:

• Marketers are like everybody else. Remember all that yammering about how today’s buyers do their own research, don’t talk to sales until late in the process, and get their information from social media rather than experts? Today’s marketers buy that way too. So the carefully structured, professionally managed selection process is a thing of the past.

• Marketers are marketers.  This means they’re facing more change and a less clear future than other types of buyers, and they’re less experienced with purchasing technology. It’s no wonder they can’t define their requirements as well as someone buying a new accounting system.

But all is not lost. Marketers can do a better job of system selection if they try. Specifically, they can do two things: improve the selection process itself and look beyond features to assess the vendors. This post will focus on the selection process and the next will talk about judging vendors.

As I’ve already written more than once, the key to sound selection process is a good set of requirements. These should be packaged into a formal requirements document so you have them all in one place, easily organized and available to share with vendors. But don’t think you’re writing the document for vendors. Instead, imagine you’ll submit it to the Chief of the Prussian General Staff, who just might slice your ear off if you do a less than thorough job.

Here's what he'll be looking for:

• Background: a general description of your business, including the products, company size, and industry characteristics. This gives a vendor an idea of your key issues and what sort of solution would be appropriate. Remember: a solution that’s too sophisticated for your needs can be as ineffective as one that’s too simple.

• Marketing process: describe your current methods for customer acquisition, relationship development, and retention. Include a channel-by-channel breakdown of your major marketing programs, with the volumes, spending and results for each. Your goals are to define the scope of your required solution and to help prioritize different capabilities. 

• Existing systems: describe the current marketing systems, including the technology, how they’re used, and known problems. This provides additional context for judging the scope of change that’s desired and what’s needed to achieve it.

• Project objectives: only now are you ready to state your goals for this project. You’ve waited this long because the objectives only make sense in light of your current situation. The goals you state here should be as specific as possible, so you can later check that proposed solutions  address them.

• Data sources: describe the internal and external systems that will feed your marketing automation platform. A simple marketing automation deployment might integrate only with CRM. But more complex scenarios could include inputs from Web analytics, order processing, point of sale, accounting, and elsewhere. Present this information in a table with record counts and transaction volumes so it can be used to size and price your solution.

• Required functions: this translates your project objectives into specific system requirements. These include data preparation as well as marketing execution. They wouldn’t generally extend to non-functional requirements like vendor background and pricing, although you could include them here if you’re concerned you’ll forget about them otherwise. Even though these are functional requirements, don’t be too specific in how things should work: you want enough flexibility for each vendor to showcase the best way to use their system. This part of the document is where you're most likely to need outside help: it takes an expert to know what functions are implied by each project objective.

• Use case scenarios: here’s the place to get specific. Pick several key processes, such as specific marketing programs, and describe in full detail how you want them set up. This would include segmentation rules, content creation, processing logic, CRM integration, lead scoring, and any other tasks required to run the program. You’ll later ask the vendors to demonstrate how they would perform those tasks.. The key is to define real projects for your business, not vendor-chosen examples that showcase their strengths and bypass their weaknesses.

These same elements should appear in pretty much any requirements document. What will differ is the degree of detail: I’ve written some requirements documents that are three pages long and some that are thirty. The right scale depends on the complexity of your situation. But even a simple requirements document is well worth the trouble, both to clarify your own thinking and to communicate that thinking to potential vendors.

Marketers Do a Bad Job Selecting Marketing Automation Systems

I presented my Seven Deadly Sins of Marketing Automation Software Selection during last week’s Webinar with Neolane. (To replay the Webinar, click here.)  If you’re wondering how many companies actually commit those sins, the sad answer is: a lot. Here are some statistics.

• About half of buyers consider only one system, I’m told by various vendors. Some may have known exactly what they needed in advance, but most are just buying the first system that seems to do what they need. And it’s a safe bet they haven’t analyzed their requirements well enough to understand those needs correctly.

• 66% of buyers base their selection process on meetings within marketing. This isn’t bad in itself, but many don’t talk to anyone else. You do also have to wonder how other 34% make a decision if they’re NOT talking to anyone in marketing. (This and the following figures come from the CMO Council study “Driving Revenue Through Customer Relevance”, which I analyzed in detail last year).

• 42% of buyers rely on online research. Again, not a bad source in itself, but far from sufficient. The real problem is comparing this figure and the previous 66% to…

• 25% of buyers consult with in-house IT. Think about that: 75% of CMOs are making a major system investment WITHOUT consulting their IT group. This would be fine if most marketers were experts at technology acquisition. But they’re not. Software-as-a-Service  makes it possible for marketers to purchase and deploy a marketing automation system without help from IT, but that doesn’t make it a good idea.

• 19% of buyers do a formal needs assessment and Request for Proposal (RFP). Again, this means the other 81% are buying a system without a formal buying process. Maybe some are just skipping the RFP, which isn't always needed. But I know from my own experience that plenty of marketers don’t do a needs assessment either. That's a big problem: you can't make a sound choice without one. Remember: when you don't know where you're going, any road will take you there.

• 25% do a pilot deployment. A pilot isn’t essential if you’ve run a good selection process. But for the vast majority of marketers who haven't run a good process, a pilot is their last line of defense before buying the wrong system. That so few run one means the most are buying blindfolded and hoping for the best. Let’s just say that this is not a good idea. 

The Seven Deadly Sins of Marketing Automation System Selection

I’ll be giving a Webinar this Thursday on evaluating marketing automation software, sponsored by Neolane. Part of the content will be a list of Seven Deadly Sins of Marketing System Selection.  I thought that was worth a blog post of its own. So here goes.

1. Ignoring Users. Selection teams often don’t take the the time to understand how future users of the system do their jobs today. The justification may be that everything will change anyway, or that every marketing department has similar needs, or that the users themselves don’t know what they need. The cost of skipping this step is that you don’t learn about existing business processes and user skills. This means you don’t identify what processes need to be changed and what training your users will need.  The immediate result is you can’t factor those items into your vendor evaluation. Longer term, your deployment will take longer since you’ll have to stop to gather this information before you can proceed.

2. Lack of Purpose. It’s frightening how often I ask someone how they expect to use their new marketing automation system and am told they don’t know. Buyers who don’t set business objectives have no way to judge what the system should do or to measure its success after the fact. Ideally you’ll have specific, quantifiable goals in terms of numbers of qualified leads, costs, and revenue created. But even general goals like supporting Webinars or running nurture campaigns are enough to give useful direction. Remember the old saying: “When you don’t know where you’re going, any road will take you there.”

3. No Requirements. Even marketers who know what they want often don’t translate those desires in specific system requirements. This is probably the most common sin of all. Formal, written requirements provide a framework to prioritize your needs, explore them with vendors, and make a complete, consistent assessment of what you learn. Without written requirements as a reference, your project can easily descend into chaos: something that made for great medieval artwork, but in real life is no fun at all.

4. Talk Only to Leaders.  Buyers often limit their consideration to a handful of vendors who are anointed as industry leaders by analysts or simply gain the most attention in social media. The theory seems to be that the most popular products do the best job of meeting a broad spectrum of needs, and are thus most likely to suit the buyer.  It’s an argument that only makes sense to people who don’t know their actual requirements. Think of it this way: would you only consider three best-selling automobiles (Ford F-150 pickup, Chevy Silverado pickup, and Toyota Camry)? Of course not, because you have specific requirements that those products probably don’t meet. Chances are you also have a few marketing automation needs that less popular systems actually perform best. You won’t know unless you look.

5. Let the Vendor Drive. Marketers who don’t know what they want often rely on the vendors to tell them what’s important. At best, the salesperson takes the time to understand your business and demonstrates how her system can best meet your needs. But that’s not the same as defining the best solution. More likely, the salesperson will hand you a list of what her system does best and hope you evaluate everyone else against it. It’s true that some salespeople will walk away from a deal if it’s a poor fit, but now you’re relying on the kindness of strangers – and you remember how that worked out for Blanche DuBois. (Poorly.)

6. Focus on Functions. We all love our bells and whistles, and salespeople love to show them. But functionality isn’t the only thing you need to consider in a vendor.  In fact, given that most systems can meet your basic needs, functions may not be the most important differentiator. You also need to consider how well the vendor will train and support you, whether their underlying technology can meet your present and future needs (there’s those pesky requirements again!), their familiarity with your industry, and how likely they are to remain in business. It's harder to answer these questions than sit through a demo, but they’re critical to your project’s success.

7. Work Without Experts. This is the Original Sin from which all others flow. It takes expertise to define objectives, gather requirements, screen the vendors, and run a smooth process. Marketers, like B2B buyers everyewhere, are increasingly trying to do it all without help – and most of them don’t have the time or skills to succeed. If you’re among the have-nots, see whether your IT department or procurement team have the skills to help. If not, find an external expert who specializes in marketing automation systems (for example, Raab Associates). Chances are, their fee will be less than the value of the time you’d spend doing the work for yourself. More important, you’ll end up with a better decision sooner, greatly increasing the final return on your marketing automation investment.

B2B Marketing Automation Industry Size and Segments

As I mentioned yesterday, our new B2B Marketing Automation Vendor Selection Tool (VEST) asks vendors to estimate the number of clients in each of four size categories.

This provides an interesting overview of the industry. The segments are defined based on revenue. Installation counts are:

Looking at the raw percentages doesn’t make much sense since businesses in each group are quite different. There’s a strong case to be made that micro-businesses in particular have such different needs that their vendors are not really part of the same industry as the rest of B2B marketing automation. I’ve described those differences in this post and go into them in our Vendor Selection Workbook (different from the VEST, and free on the Raab Guide site.)

But if you do want to consider all these vendors as one industry, the minimum adjustment to make is to account for differences in price. The table below calculates revenues using reasonable assumptions about revenue per client in each segment:

Combined with the previous chart, this shows the micro-business segment represents 61% of clients but just 17% of industry revenues. At the other extreme, large business represents just 6% of clients but 28% of revenue. The small- and mid-size companies are the heart of the industry , with 55% of the revenue from 33% of the clients.

The $257.5 million revenue estimate is reasonable but it excludes revenues from B2B marketing automation vendors not in the VEST report and the B2B revenues of B2C marketing automation firms. So I’d estimate total industry revenue at $325* million for 2011. This represents a 50% growth over my estimate for 2010. That is consistent with the growth rate I reported yesterday.

The figures also shed light on the ever-popular question of penetration rates. The table below shows company counts by revenue range from business list compiler Manta. But not all of these are B2B marketers. Looking at the industry categories, I'd put the estimated market at half the total.


The 26.7% figure for the large company category is clearly too high, but that's easy to explain: big companies have lots of divisions, so many vendors have sold to a little piece of those firms. There’s certainly still plenty of opportunity left. It’s possible the 3% figure for mid-size firms reflects some of this effect as well.

Figures for the first three categories are more intriguing. They're much lower than the usual estimates that 5% to 10% of companies have marketing automation. Either the surveys behind those estimates are incorrect or my market definition is too broad.

It’s probably a bit of each: surveys tend to reach people who have above-average interest in the topic, and my 50% figure is based on categories that could potentially use marketing automation, not the categories that have deployed it so far. A count of the pioneer companies, basically tech and manufacturing industries, would reduce the estimated market to anything from one quarter to one tenth the numbers shown. This would translate to penetration rates of 10% to 30%, which is more in line with current estimates.

But I’d argue that the market is already growing beyond this core group, so the long-term potential is considerably larger. That’s great news – so long as vendors don’t get stuck in the current niche and so long as competitors from the CRM, email, Web software, Web advertising or other industries don’t swoop in and snatch it all away.

______________________________________________________________
*The original version of this post estimated $300 million. On consideration, I raised the estimate to $325 million because
- my revised estimate for 2010 was $225
- the 52% growth rate in the previous post was in number of clients, but growth is faster in the higher-priced segments, so the revenue growth would be higher
- average prices are probably rising a bit in the mid-sized segment and big segments, so revenue would rise faster than client counts
- the client counts were gathered in May and June, so they are not quite mid-year figures

I would have gone higher, but the large-company figures are probably overstated in my estimates because many of the 1,200 installations are small, departmental systems that wouldn't generate anything near $60,000 per year.

Ranking B2B Marketing Automation Vendors: Part 3

Summary: The first two posts in this series described my scoring for product fit. The third and final post describes scoring for vendor strength. And I'll give a little preview of the charts these scores produce...without product names attached.

Beyond assessing a vendor's current product, buyers also want to understand the current and future market position of the vendor itself. I had much less data to work with relating to vendor strength and there are many fewer conceptual issues. From a buyer’s perspective, the big questions about vendors are whether they’ll remain in business, whether they’ll continue to support and update the product, and whether they understand the needs of customers like me.

As with product fit, I used different weights for different types of buyers. As you'll see below, the bulk of the weight was assigned to concentration within each market. This reflects the fact that buyers really do want vendors who have experience with similar companies. Specific rationales are in the table. I converted the entries to the standard 0-2 scale and originally required the weights to add to 100. This changed when I added negative scoring to sharpen distinctions among vendor groups.


These weights produced a reasonable set of vendor group scores – small vendors scored best for small buyers, mixed and special vendors scored best for mid-size buyers, and big vendors scored best for big buyers. QED.


I should stress that all the score development I've described in these posts was done by looking at the vendor groups, not at individual vendors. (Well, maybe I peeked a little.) The acid test is when the individual vendors scores are plotted -- are different kinds of vendors pretty much where expected, without each category being so tightly clustered together that there's no meaningful differentiation?

The charts below show the results, without revealing specific vendor names. Instead, I've color-coded the points (each representing one vendor) using the same categories as before: green for small business vendors, black for mixed vendors, violet for specialists, and blue for big company vendors.






As you can see, the blue and green dots do dominate the upper right quadrants of their respective charts. The other colors are distributed in intriguing positions that will be very interesting indeed once names are attached. This should happen in early to mid January, once I finish packaging the data into a proper report. Stay tuned, and in the meantime have a Happy New Year.

Ranking B2B Marketing Automation Vendors: Part 2

Summary: Yesterday's post described the objectives of my product fit scores for B2B marketing automation vendors and how I set up the original weighting for individual elements. But the original set of scores seemed to favor more complex products, even for small business marketers. Here's how I addressed the problem.

Having decided that my weights needed adjusting, I wanted an independent assessment of which features were most appropriate for each type of buyer. I decided I could base this on the features each set of vendors provided. The only necessary assumption is that vendors offer the features that their target buyers need most. That seems like a reasonable premise -- or at least, more reliable than just applying my own opinions.

For this analysis, I first calculated the average score for each feature in each vendor group. Remember that I was working with a matrix of 150+ features for each vendor, each scored from 0 to 2 (0=not provided, 1=partly provided, 2=fully provided). A higher average means that more vendors provide the feature.

I then sorted the feature list based on average scores for the small business vendors. This put the least common small business features at the top and the most common at the bottom. I divided the list into six roughly-equal sized segments, representing feature groups that ranged from rare to very common. The final two segments both contained features shared by all small business vendors. One segment had features that were also shared by all big business vendors; the other had features that big business vendors didn't share. Finally, I calculated an average score for the big business vendors for each of the six groups.

What I found, not surprisingly, was that some features are more common in big-company systems, some are in all types of systems, and a few are concentrated among small-company systems. In each group, the intermediate vendors (mixed and special) had scores between the small and large vendor scores. This is additional confirmation that the groupings reflect a realistic ranking by buyer needs (or, at least, the vendors’ collective judgment of those needs).


The next step was to see whether my judgment matched the vendors’. Using the same feature groups, I calculated the aggregate weights I had already assigned to the those features for each buyer type. Sure enough, the big business features had the highest weights in the big business set, and the small business weights got relatively larger as you moved towards the small business features. The mid-size weights were somewhere in between, exactly where they should have been. Hooray for me!



Self-congratulation aside, we now have firmer ground for adjusting the weights to distinguish systems for different types of buyers. Remember, the small business scores in particular weren’t very different for the different vendor groups, and actually gave higher scores to big business vendors once you removed the adjustment for price. (As you may have guessed, most features in the “more small” group are price-related – proving, as if proof were necessary, that small businesses are very price sensitive.)

From here, the technical solution here is quite obvious: assign negative weights to big business features in the small business weight set. This recognizes that unnecessary features actually reduce the value of a system by making it harder to use. The caveat is that different users need different features. But that's why we have different weight sets in the first place.

(As an aside, it’s worth exploring why only assigning lower weights to the unnecessary features won’t suffice. Start with the fact that even a low weight increases rather than reduces a product score, so products with more features will always have a higher total. This is a fundamental problem with many feature-based scoring systems. In theory, assigning higher weights to other, more relevant factors might overcome this, but only if those features are more common among the simpler systems. In practice, most of the reassigned points will go to basic features which are present in all systems. This means the advanced systems get points for all the simple features plus the advanced features, while simple systems get points for the simple features only. So the advanced systems still win. That's just what happened with my original scores.)

Fortified with this evidence, I revisited my small business scoring and applied negative weights to items I felt were important only to large businesses. I applied similar but less severe adjustments to the mid-size weight set. The mid-size weights were in some ways a harder set of choices, since some big-company features do add value for mid-size firms. Although I worked without looking at the feature groups, the negative scores were indeed concentrated among the features in the large business groups:


I used the adjusted weights to create new product fit scores. These now show much more reasonable relationships across the vendor groups: that is, each vendor group has the highest scores for its primary buyer type and there’s a big difference between small and big business vendors. Hooray for me, again.


One caveat is that negative scores mean that weights in each set no longer add to 100%. This means that scores from different weight sets (i.e., reading down the chart) are no longer directly comparable. There are technical ways to solve this, but it's not worth the trouble for this particular project.

Tomorrow I'll describe the vendor fit scores. Mercifully, they are much simpler.