SAS Survey on Marketing Measurements Gives Odd Results

SAS Institute recently released a survey of senior executives’ attitudes towards marketing performance measurement. The primary result was what you’d expect: depending on the medium, just 7% to 14% of executives rate their marketing measurements “very effective”.

But the details were perplexing. Respondents found direct mail to be the significantly less well measured than other media: 33% said direct mail measurements were either “very” or “somewhat” effective, compared with 50% to 58% for advertising, collateral, public relations and events.

Since most marketers would consider direct mail to be the most rather than least measurable of channels, you have to wonder what caused this result. It may be that the people surveyed don’t use much direct mail—39% said they either don’t measure direct mail effectiveness or don’t know whether they measure it, roughly twice as many as the 17% to 22% who gave the same answers for other media. According to the study, the survey was conducted among “large and midsize companies across a range of industries,” which doesn’t tell us much about who they were but does suggest few were direct marketers.

The other striking result was that respondents ranked “operations/supply chain management” in seventh place (43%) on their list of divisions that play a vital role in achieving strategic goals. Sales (64%), customer service (61%), strategy/planning (59%), marketing (55%) and even product development (54%) and IT (45%) all ranked higher. Only finance (39%) and human resources (20%) were deemed less important.

Personally I would rank operations pretty much as number 1 when it comes to achieving goals, so it really surprised me to see such different results. Perhaps the managers in the survey really believe they are in commodity businesses, or perhaps they just have a narrow definition of “operations” that excludes most customer-facing activities. (Note that sales and customer service, which certainly are customer-facing, ranked numbers 1 and 2.) Still, any way you slice it, operational experiences make up the bulk of customer interactions and largely determine future behavior. It’s just hard to imagine managers not recognizing that.

Another way to look at this is that my reactions are classic examples of people refusing to accept information that contradicts their pre-existing beliefs. I’ll concede this is a possibility: maybe direct mail really is least measurable, and maybe operations are not all that strategic. Or at least maybe most managers really believe those things, whether or not they’re correct. But before I accept such odd results, I’d really want to know more about how the questions were asked, who answered them, and whether other research leads to similar conclusions. I certainly wouldn’t spend company money on the assumption that they’re correct.