Tuesday, October 22, 2013

Marketing Automation User Satisfaction: Clearly, There's Room for Improvement (and maybe a little vodka)

Last week’s post on marketing automation and its discontents prompted several questions about whether the level of dissatisfaction is any higher with marketing automation than other systems. To some extent, this is asking whether the glass is half empty or half full; and, as the illustration suggests, the answer matters less than the fact that there’s room for improvement. But I do have some data to share on the question of relative dissatisfaction.

The first insights come from G2 Crowd, a research firm that ranks software based on user ratings and social data. I have my doubts about comparing software this way* but users certainly know whether or not they're happy.  The folks at G2 were kind enough to reformat some of their data for me.**

According to the G2 figures, marketing automation users are in fact more enthusiastic about their choices than almost anyone else. CRM in particular has a vastly worse rating, but even email, Web analytics, and Web content management show more detractors and fewer promoters. I’m not sure how to interpret this – is the average marketing automation system really easier and better than those other types of software?  Or is something else going on: maybe satisfaction is lowest in the most mature categories, like human resources, enterprise resource management, and accounting, because experienced users are the most demanding?

A second set of insights comes from Ascend2 and Research Partners, which asked its panel which inbound marketing tactics they considered most effective and most difficult to execute. Here we see a very different story: marketing automation and lead nurturing (listed separately) are clear outliers in a bad way: among the less effective tactics and the hardest to execute. In fact, they are the only two tactics where the difficulty score was significantly higher than the effectiveness score (i.e., above the diagonal line in the chart below).***

The Ascend2 study also found that 18% of respondents used marketing automation extensively, while 43% made limited use of it, and 39% didn’t use at all. This is similar to the BtoB study I cited last week, which found that just 26% of marketing automation users had fully adopted their system.  I believe those effectiveness vs. difficulty ratings hint at the reason for those results: most marketers don’t fully deploy marketing automation because they find it too much work compared with the benefit they’d gain. In other words, the hurdle to marketing automation adoption is not laziness, but a rational evaluation of the return from investments in marketing automation vs. other activities.

That rational judgment could still be wrong.  After all, marketers who haven’t fully deployed marketing automation don’t know how effective it really is. Ascend2 addressed this by asking marketers to rate their performance and comparing answers of the 12% self-rated “very successful” with the 20% who rated themselves “not successful”.

Those answers contain some positive news: of the very successful group, 45% were extensive users of marketing automation, compared with just 9% of the not successful.

But even the very successful marketers gave marketing automation only the fifth-highest effectiveness rating, which doesn’t differ much from the sixth-highest rating in the not successful group.

Similarly, the very successful marketers rated marketing automation as sixth most difficult (actually, tied for fifth) while the not successful marketers ranked it as fourth-hardest. In other words, marketing automation is indeed a bit easier than it seems before you start, but even the most experienced and most successful marketing automation users consider it pretty darn hard and just modestly effective.

So what we have here is a mixed message: marketing automation does correlate with success and its users might even be relatively satisfied, but it's still a lot of work for limited results.  You read that as good news or bad, but, either way, it shows the need for more work before marketing automation can reach its full potential.


* My basic objection is that users have different needs, so a system that satisfies one user may not be good for another.

** G2’s explanation: “The data for this chart comes from the over 7,400 enterprise software surveys users have completed on G2 Crowd as of Friday 10/18/13. For every product review we ask "How likely is it that you would recommend this product to a friend or colleague?" on a 0-10 scale. We segment reviewers that rate a product 9-10 as Promoters, 7-8 as Passives, and 0-6 as Detractors. The product segmentation data is aggregated to determine Net Promoter Score at a category level.”

***It's barely possible that the answers would be different if the Ascend2 study had asked about marketing in general rather than "inbound marketing purposes".  But I doubt it.


Unknown said...

Thanks for the post David,
I am a little bit skeptical about G2 Crowd data. They rely on vendors to reach out to their users for those survey results. For example, G2 Crowd work with a marketing automation vendor and the vendor sends out an email to their users to encourage them to participate in that survey. My assumption is that, the vendors are being very picky about to whom they reach out to. I believe they reach to users who they think will give them a great feedback.
In the meantime it is easier for G2 Crowd to reach out to users' of more mature products like CRM or e-Commerce by their selves(not through vendors). This is mainly because their market penetration is a lot higher than market automation and there are more users out there using these tools. Thus the NPS scores for more mature products are more realistic than marketing automation scores.
Of course this is all speculation but this is how I interpret G2 Crowd's data.

Anonymous said...

Hi David,

These results seem odd to me. I don't feel like "content marketing" and "marketing automation" belong in the same list of things. MA is inward-focused, creating systems and processes. Content marketing is outward-facing, creating awareness, branding, and leads.

My feeling on this is that both are an absolute requirement for all marketing departments, and marketing automation is a pre-requisite before running effective content marketing. But that is not to say that it is *more important*. It is just *what you should do first*.

Marketing automation without content marketing is...well, a little silly. But content marketing without marketing automation is a total cluster****.

My two cents.
VP Marketing, RJMetrics

David Raab said...

Thanks for the comments, Anil and Tristan. Regarding the validity of G2's data, there are always issues with sample selection in these surveys, but you may be right that there is a bigger problem with products that have smaller installed bases, either because of vendor-driven audiences or perhaps just because the early adopters are inherently more enthusiastic.

Tristan, I'm still struggling with how to distinguish "content marketing" from marketing in general, since both aim at "creating awareness, branding, and leads". I personally tend to equate marketing automation with lead nurture campaigns, but perhaps you're right that it's actually more about systems and processes than campaigns. Definitions are the hardest part of working with surveys, since you never really know what definitions the respondents have in mind.

As Donald Rumsfeld didn't quite say, you analyze the surveys you have.

Unknown said...

David, thanks for an insightful post and stirring up good conversation.

Anil, I appreciate your comments and concerns about G2 Crowd. We are doing a significant amount of in-house community building both online via social networks, as well as at big industry events, to encourage the broadest range of input possible. We use this same process for every category. Additionally, some vendors are encouraging their customers to share reviews, but they are reaching out to all their customers. For example HubSpot had a booth at their Inbound conference encouraging all customers to share reviews on both G2 Crowd and the AppExchange.

As our G2 Crowd sites gains mind-share we believe it will become an outlet for both happy and upset customers to share their experiences. We believe in the premises defined so well by James Surowiecki in the "Wisdom of Crowds" where more input from a broader crowd leads to the better insights than any one person can provide.

Jon Miller said...

It's worth pointing out that the Ascend data asked people about INBOUND marketing effectiveness. That's why things like content stand out at as top performing. If you look at the entire demand process, the story may be different. Personally, I don't think it's right to generalize the inbound marketing results.

David Raab said...

Hi Jon. It's possible the answers would be different if the question had not been about inbound, but I suspect people don't generally make such fine distinctions when responding to these sorts of surveys. For example, the group actually rated lead nurturing and scoring as more effective than marketing automation, even though I see nurturing and scoring as even less related to content marketing (i.e., acquisition) than marketing automation.

On the other hand, the survey itself was about inbound marketing, so there's a pretty good chance that the people who responded were content marketing enthusiasts, which would help explain why they rated content marketing so highly.

Of course, the larger point of the series of posts is that there are many surveys giving similar answers.

Tom De Baere said...

Hi David,

Thanks for blogging about this study. I noticed some uncertainty in the comments about whether content marketing and marketing automation belong in the same category.

I believe every company investing in marketing automation must have strong content marketing processes under control before even thinking of investing in marketing automation. To me content is the fuel that makes the marketing automation engine run.

The same thing for social media marketing: that doesn't work without strong content.

That's why I believe content marketing has to be in this list.

About the study: if it is indeed the case that people interested in content marketing or inbound marketing responded the most, in that case someone should make a broader survey to include other priorities of decision makers that do not have content marketing as their top priority.

Hope this add.


Tom De Baere