Raab VEST Report: Testing, Data Quality and Content Management Still Lag in Marketing Automation Products

My last post looked at data from our just-released B2B Marketing Automation Vendor Selection Tool (VEST) to understand general industry trends and identify the greatest areas of improvement. Today we’ll look at the VEST data to see what’s still hard to find. As before, the charts show three columns: change in feature availability over the past year among core marketing automation vendors; current availability among core vendors; and current availability among enterprise vendors. See the previous post for details on the calculations.

Split Tests: Formal split testing of different content versions or customer treatments is the heart of marketing optimization, but many B2B marketers still don’t have the time or resources to do it. Given the lack of demand, it’s not too surprising that many vendors don’t offer strong testing features.  Still, I feel they have something approaching a moral obligation to provide these features and encourage their use.  Note that splits within lists, the one testing capability that is fairly common, is actually the hardest for marketers to use.  Testing features are much more available among enterprise systems, whose clients are more likely to conduct tests as a matter of course.

Value-Based Selection: This is arguably the next step after dynamic content (see my previous post), since it uses calculated values rather than user-crafted rules to select marketing contents or campaign actions. Like dynamic content, it reduces the complexity of marketing programs while allowing them to be more targeted. It's still much harder to find than dynamic content although it is becoming more available.  Again, enterprise vendors have a substantial lead over the core systems.

Integrate with Direct Mail Printer: This is admittedly a small tactical issue, but it's interesting in its own way.  There’s apparently a resurgence of interest of direct mail generally and post cards in particular as a way to avoid ever-more-cluttered email inboxes and social media channels. This is one of the few features that are more common among micro-business systems than the core group.

Project and Content Management: These features are most important for large marketing departments that need coordinate work of many people. Most core marketing automation systems can track the creation and last change date of an item. But serious administration requires much more detailed control over who makes changes, approvals, and project management. As marketing programs get more complicated at all sizes of companies, these features will become increasingly important.

Data Quality: These are features that give marketers more control over the data that goes into their systems. Like split testing, data quality is widely recognized as important but often ignored. Availability of these features actually went down last year because several new core vendors provided below-average support. Enterprise vendors, with their more sophisticated client base, support these features fully.

Data Management: These features each reflect a certain degree of data management sophistication, although there’s a reasonable case that a separate company table doesn’t matter much in practice. The opportunity table is critical for revenue analysis, and you see here that it’s widely available. Custom tables are needed to extend the marketing database beyond inputs from the CRM system. They used to be fairly rare but are now available in more than half of the core products.   But half full also means half empty, so buyers still need to check carefully to ensure a particular vendor supports their needs.

Reviewing this list of features, only value-based selection is really cutting edge.  The rest have long been standard for consumer marketing automation products and enterprise B2B.  They're missing from core B2B marketing automation systems because most of their clients are smaller, less sophisticated companies who haven't needed them.  This may never change for vendors focused on small marketing departments.  But vendors serving larger companies will add these features as their clients discover they need them.

For more information about the B2B Marketing Automation VEST report, please visit www.raabguide.com/vest.

New Marketing Automation Report: Venture Funding is Key to Success

I released the 2012 edition of our B2B Marketing Automation Vendor Selection Tool (VEST) report today, an event that deserves more hoopla that I’ve given it. The VEST provides by far the most detailed, objective information available on industry vendors. It includes nearly 200 data points on 21 products, thumbnail sketches of each vendor’s strengths and weaknesses, and three industry quadrants showing leaders in different market segments. It’s also interactive: you can change the weights assigned to different items and watch the vendors zoom around the quadrant as a result. For those of us who don’t get out much, that’s downright exciting.

Although the VEST is primarily intended to help people who are buying a marketing automation system, its database also provides a statistical portrait of the industry. After rooting around in the numbers like pig hunting truffles, here’s what I dug up:

Core marketing automation is growing fastest. We classify industry vendors into four groups:
- micro business vendors (Infusionsoft and OfficeAutoPilot)
- HubSpot (a category of its own because it’s not quite standard marketing automation and is big enough to treat separately)
- enterprise vendors (Neolane, Aprimo, Silverpop, and Oracle)
- core B2B marketing automation (everybody else: Eloqua, Marketo, Pardot, Genius, Act-On, et, al.)

The enterprise vendors don’t release meaningful installation counts – some refuse to provide any data and others don’t distinguish B2B from B2C clients. So we’ll exclude them from further analysis. The table below shows growth for the remaining groups:

As you see, core vendors grew almost twice as fast as the micro-business group and considerably faster than HubSpot. There was some speculation last year that the micro-business vendors were growing the fastest. Myth busted.

Revenue grew faster than installation counts. The previous table shows that combined growth across all categories is 46%. But that doesn’t mean much because the selling prices are so different. Adjusting for revenue per client, I estimate that industry revenue grew about 55% last year.  I could show you my calculations, but then I'd have to...well, you know.

New leaders can still emerge, but venture funding is required. A year ago, the three largest core vendors were Eloqua, Marketo, and Genius, and Act-On was an also-ran. Today, the Marketo has more clients than Eloqua (although not more revenue), Pardot has replaced Genius in the third position, and Act-On is coming up fast.

It's no mystery why the market remains fluid: venture capital lets new entrants shoulder their way to the top. Of the five top-ranked core vendors, all but Pardot has substantial venture funding. None of the remaining ten core vendors do – and their average growth is much slower. Self-funded firms can survive but it’s unlikely they will become leaders.

Specialization is increasing. We ask vendors for client counts in four segments: micro-business (under $5 million revenue); small business ($5 to $20 million revenue), mid-size business ($20-$500 million revenue) and big business ($500 million and higher). Most gave us answers, although Act-On was a prominent hold-out.

It’s no surprise that the micro vendors sell almost exclusively to micro clients. HubSpot’s base is much more diverse, although the micro and small sectors still account for 75% of its base. Of the core vendors, Marketo and Genius are the most small business oriented, with the two smallest categories accounting for over half of their customers. Pardot is tightly focused on small and mid-size clients, reflecting their disciplined sales approach. Eloqua has by far the most big-business clients of any core vendor, a proportion that has grown dramatically since the first VEST report one year ago.

For more information on the new VEST report, visit www.raabguide.com/vest.

B2B Marfketing Automation Vendor Selection Tool: What’s Inside and Why

Summary: Our new B2B Marketing Automation Vendor Selection Tool (VEST) has been carefully crafted to help marketers at every step of the selection process. I think it’s worth walking through the main components to explain why they’re there.

Here's a screen-by-screen look at the components of the VEST. For more information or to order, please click here.

Explanations

What It Is: This is basic information for people who are just starting to explore marketing automation. It includes a general introduction suggesting how to use the VEST and then provides explanations of what marketing automation means and why it’s important, an overview of the state of the industry, advice on running a selection project, and details on the vendor scoring.

Why It’s There: Many buyers are new to marketing automation. They need a coherent explanation of what it is, why it matters, how it fits into the larger scheme of marketing technology, and how to go about selecting a tool. I think the industry veterans will also find these materials interesting, but they’re really aimed at bringing the newbies up to speed.

Sector Charts


What It Is: This is the vendor landscape chart that users love and analysts are apparently obligated to produce. It uses our vendor scores to plot the relative positions of products in terms of how well they fit buyer needs. This lets us place “leaders” in the upper right quadrant. There are four versions: one each based on weights for small, mid-size and large businesses, plus a custom chart with the user’s own weights. Sliders make it easy adjust the weights assigned to broad categories within product and vendor fit.


Why It’s There: The chart makes it easy for each user to identify the most likely candidates, quickly reducing the consideration set to something manageable. More important, having alternative sets of weights, allowing custom weights, and making easy to adjust category weights all encourage buyers to recognize that there’s no "one true leader" and therefore to think about what weights are really relevant to their own needs.

Vendor Profiles


What It Is: This gives concise descriptions of the strengths, weaknesses, market position, and most suitable clients for each vendor. These are accompanied by charts displaying key factoids, such as the number of clients, number of employees and year founded; the position of the vendor in each of the three sector charts; and the relative strength of specific categories within the product and vendor fit scores.

Why It’s There: Now that buyers have tentatively identified their best candidates, they can look here to get a better sense of the products. The descriptions are based on Raab Associates’ detailed product research, and thus highlight information not captured in the numeric scores. For the first time in the VEST, this section introduces the category details within the score totals. This provides the next level of detail and lets buyers to see how vendor strengths actually line up with their priorities.

Item Detail


What It Is: This shows the nearly 200 specific items used in scoring the vendors. It provides the detailed definitions used in rating each item for each vendor (typically on a scale of 0 to 2) and shows the weights assigned to each item in the small, mid-size and large scoring schemes. It also gives users another opportunity to view and adjust the category weights.

Why It’s There: This introduces the actual items used in the scoring, encouraging them to look even deeper below the surface. The definitions include explanations of when and why each item matters, helping to further the users’ understanding of important-but-subtle product differences. Showing the variation of weights for the same item in the different scoring schemes implicitly encourages users to consider what weight makes the most sense for them.

Compare Vendors


What It Is: This lets users select any three vendors and compare them side-by-side. Screens start with a summary view that shows the product and vendor fit totals and the sum of both raw and weighted values for the categories. Users can then drill into each category to see the item-level ratings and weighted scores for all three vendors.

Why It’s There: This lets users drill into the vendor details at the finest possible level, seeing exactly what is driving the category scores and exactly how the vendors differ. Showing the sum of the raw values along with the weighted values graphically illustrates the impact of the category weights on the summary scores, encouraging users to ensure that the category weights reflect their own priorities. By this point in the process, users should understand which items they care about most.

Custom Weights


What It Is: This lets users set the item and category weights they’ll use in their custom scoring. They can apply the standard small, mid-size or large weights as a starting point. They can also save their weights as a scenario to use in another session. They can save any number of those scenarios.

Why It’s There: This lets users create their own custom scores, based by now on a deep understanding of their own needs and the information embedded within the VEST. Custom scoring won’t make the selection decision for anyone, but it will facilitate comparisons between vendors and highlight key items to research in detail.