ABM Vendor Guide: Differentiators for Result Analysis

...and we wrap up our review of sub-functions from the Raab Guide to ABM Vendors with a look at Result Analysis.

ABM Process	System Function	Sub-Function	Number of Vendors
Identify Target Accounts	Assemble Data	External Data	28
	Select Targets	Target Scoring	15
Plan Interactions	Assemble Messages	Customized Messages	6
	Select Messages	State-Based Flows	10
Execute Interactions	Deliver Messages	Execution	19
Analyze Results	Reporting	Result Analysis	16

As the Guide points out, this category focuses on measurements unique to account-based programs:

Nearly every system will have some form of result reporting. ABM specialists provide account-based result metrics such as percentage of target accounts reached, amount of time target accounts are spending with company messages, and distribution of messages by department within target accounts.

Not surprisingly, most of vendors who do ABM Result Analysis also do some sort of Execution (12 out of 16, to be exact). Another two (Everstring and ZenIQ) didn't fall into the Execution group but came close.  Of the final two, one supports measurement with advanced lead-to-account mapping (LeanData) and one attribution specialist (Bizible). It's important to recognize that many of the Execution vendors will report only results of their own messages.  This is certainly helpful but you'll want to see reports that combine data from all messages to get a meaningful picture of your ABM program results. 

Differenatiators for this group include:

• lead-to-account mapping to unify data
• corporate hierarchy mapping (headquarters/branch, parent/subsidiary, etc.) to unify data
• marketing campaign to opportunity mapping to support attribution
• combine data from marketing automation, Web analytics, and CRM
• track offline channels such as conferences, direct mail, outbound hone calls
• capture detailed interaction history for each Web visit (mouse clicks, scrolling, active time spent, etc.)
• capture mobile app behaviors with SDK as well as Web site behaviors with Javascript tag
• use device ID to link display ads, Web site visits, and form fills to revenue, even when visitors don’t click on ad or Web page
• report within Salesforce CRM on combined information about leads, contacts, accounts, opportunities, campaigns, and owners
• apply multiple attribution methods including first touch, last touch, fractional, etc.
• show account-level descriptive metrics including coverage, contact frequency, visitors, contacts by job title
• show account-level result metrics including reach, engagement, influence, velocity
• show reach, engagement, influence, velocity by campaign, content, persona, segment, etc.
• identify gaps in coverage, reach, or engagement by account and recommend corrective actions

One final reminder: the just-published Guide to ABM Vendors helps marketers understand what tools they need to complete their ABM stack.  It provides detailed profiles of 40 ABM vendors, with contents including:

• introduction to Account Based Marketing
• description of ABM functions
• key subfunctions that differentiate ABM vendors
• vendor summary chart that shows who does what
• explanations of information provided in the report
• vendor profiles including a summary description, list of key features, and detailed information covering  37 categories including data sources, data storage, data outputs, target selection, planning, execution, analytics, operations, pricing, and vendor background.

For more information or to order, click here.

6 Key Marketing Measures That Don't Include Revenue

Ask most marketers how they measure performance, and they’ll tell you they look at results: incremental revenue or return on investment if they’re available, or response rates if they're not. Industry experts take a similar approach, focusing largely on the need for better revenue measures. The situation – and barely concealed frustration – is captured perfectly in the headline from a recent Forrester Consulting study sponsored by Silverpop: “Response Metrics Are Used To Evaluate Success, Leaving Customer Or Business Impact Metrics Largely Ignored”.

I agree that revenue is important, but humbly suggest that there’s more to marketing measurement than ROI. Marketers need several types of information – and shouldn’t let the quest for performance measures prevent them from meeting other requirements.

Here are six non-value measures that marketers should build into their reporting systems.

• Benchmarks. Sure, marketing’s job is to generate revenue, but is generating $1 million good or bad? The only way to know is by placing the number in context, which could be this year’s marketing plan or last year’s actual results. Even if marketers can’t measure revenue, they can  set benchmarks for metrics like number of leads generated, funnel conversion rates, or cost per order. In fact, measuring components that contribute to results gives better insights into marketing performance than reporting on the results themselves. For this reason, marketers who can’t directly measure marketing-generated revenue should think twice before creating complex indirect estimates that are hard to understand and have limited credibility. The money would probably be better spent on reports that provide a clearer picture of what’s actually happening.

• Projections. Past results are interesting but the future is more important. Again, the real need is to understand the factors that determine future results, such as response rates and funnel velocity. Changes in these can give early warning of risks and opportunities.  The trick is to distinguish real trends from random variations, so marketers react quickly without chasing too many false alarms.

• Operations. It’s easy to make mistakes in setting up a marketing program, especially one with multiple stages, lead scoring models, and decision rules. Even careful testing can’t always capture all program steps or contingencies. Marketers need reports on the number of people in each program stage and receiving each message, and they need a model that lets them know whether those numbers are reasonable. Reports should show both program-to-date and weekly or daily results: cumulative data show major errors such as bad program logic, and short-term results capture small problems, such as a missed processing step, that could get lost in a program-to-date aggregate.

• Exceptions. Projections and benchmarks put data in context, but marketers don't have time to comb through every figure.  They need exception reports to highlight the most important variations, both positive and negative.  Marketers also need tools to drill into the exceptions so they can understand what happened and identify new opportunities.

• Testing.  Formal tests are the most certain way to understand the impact of marketing projects, but they often require special reporting tools such as ways to compare results for different customer groups over time.  Incremental revenue is the ultimate measure for test evaluation, but often other metrics such as response rates or velocity are easier to capture and more directly relevant.  Reaping the full benefit of tests also requires systems to distribute results and catalog findings for future reference.

• Strategic Goals. Marketing plans should be based on corporate strategy, but long-term goals fade into the background once marketers start making tactical choices based on day-to-day results. The reporting system should provide direct measures of strategic objectives – things like penetration of new market segments and exploration of new channels – so marketers can see the cumulative impact of deviations from the original plans. Many strategic goals, such as process change, staff training, and systems deployment, are not measured in revenue at all.

The types of measures I’ve just described don’t replace revenue and ROI reporting.  Rather, they meet needs that revenue reports alone cannot. Ideally, all these types of information will be combined in a marketing dashboard that provides a quick overview of critical information and allows drilling into details when necessary. Marketers should realize that the contents of this dashboard will change over time as their focus shifts to different programs and strategic goals. They should also recognize that good reporting will generate new questions as it uncovers risks and opportunities that would otherwise have gone undetected.  The system should make those questions easier to answer, but marketers shouldn't expect their total work to decrease.  What they can expect is that better reporting will increase the value created by their efforts: yet another new metric, Return on Reporting, should go up.

Youcalc: On-Demand Analytics Without Stored Data

Summary: Youcalc is an on-demand analytics vendor with 130 prepackaged applications primarily for sales and marketing reporting. Unlike its competitors, youcalc it reads data directly from other Software-as-a-Service systems rather than loading it into its own database. This saves money and simplifies installation but has some drawbacks too. Still, it's an intriguing alternative to the standard approach.

Youcalc is fundamentally different from other on-demand analytics vendors like Birst, Cloud9 Analytics, Gooddata and Pivotlink: while those vendors all query data stored in their system, youcalc queries the source data directly. That is, youcalc provides analytical applications that read from an existing system, typically a Software-as-a-Service vendor like Salesforce.com or Google AdWords.

Although this sounds like a subtle difference, the implications are huge. It means that youcalc doesn’t need the infrastructure to build and store client databases, thereby reducing its costs dramatically.

It also means that youcalc can to give each new client immediate access to standard applications, since there is no need to adjust for differences in their data. Although this is possible with prebuilt applications at other on-demand analytics vendors, youcalc has made it more central to their business model. In fact, youcalc extends this to related community concepts such as user-contributed enhancements, forums, tagging and rating of popular applications.

I’m intrigued by the youcalc approach but do see some disadvantages. One is that data integration capabilities are limited: the current version of the system can only combine data sources that already share common keys or are linked with an existing cross reference table. I suppose it’s technically possible to allow more sophisticated data matching, but any processing will still be limited by the need to repeat it each time the data is read from its sources and loaded into memory.

A second, more fundamental limitation is that the system can’t access historical data, such as point-in-time snapshots of information which is not retained in operational systems. At best, youcalc could point to an externally-built data warehouse as a source – but now you’re back doing all the database development that youcalc is supposed to avoid. No free lunch here, folks.

Still, there are those cost savings. Youcalc is priced at an astonishingly low $19.95 per user per month, which gives access to 130+ prebuilt applications for products including Salesforce.com, SugarCRM, Google AdWords, Google Analytics, MailChimp and 37Signals’ BaseCamp (project management) and Highrise (contact management). There’s also a free version that is excludes some of the more powerful applications. The full set is available for a 30 day free trial.

Unfortunately, these prices may not last. CEO Rasmus Madsen told me the company plans eventually to charge higher fees for applications linked to higher priced source systems.

None of this would matter if the youcalc applications and underlying technology weren't worth having. But I found them quite impressive.

Applications can contain multiple objects such as charts and lists. They can also contain drop-down selection boxes to filter components and select alternative chart dimensions. A single application can have multiple pages linked by menus. Users can embed images, text notes and external URLs, and have control over style details such as type fonts and background colors. Although the presentation is nowhere near as advanced as products like Tableau or TIBCO Spotfire, it is competitive with other on-demand analytics systems.

Most current youcalc applications display a single chart from a single data source, such as “Time-Day Distribution for Google Analyzer”. But users can change the contents by selecting different dimensions (e.g., date range) and metrics (e.g. visits, new visitors, bounces, etc.). Some applications combine multiple data sources, such as the “AdWords Campaign ROI Overview for Salesforce.com” that compares cost from Google AdWords with revenue from Salesforce.com.

Users can modify these applications or create their own from scratch (although all the existing applications were built by youcalc). Development is done with Java-based desktop software that runs on Windows, Mac or Linux PCs. The interface involves dragging different components onto a whiteboard and then configuring and connecting them. There are two different whiteboards, one to show the actual application and another to display the flows used to construct each object. These flows begin with connection to an external data source and then send the data through functions to apply formulas, convert formats, create summaries, and perform other tasks. Parameters of each function can be edited during the set-up or connected to objects like drop-down menus for end-user interaction. A completed application can be saved as a stand-alone Web page, a mobile phone Web page, embedded within an external page, or deployed as a widget on an iGoogle home page.

None of this requires actual programming, and basic tasks should be easy enough for a skilled spreadsheet jockey. More demanding activities, such as connecting to an in-house data source, take considerable technical understanding. (The system doesn’t query in-house resources directly; rather, it sends a message to a “listener” on the in-house system, which runs the specified query and transmits the results back as an XML data stream.) Connections for standard sources such as Salesforce.com are very simple since they’re prebuilt: users just enter their log-in credentials and the system does the rest.

If youcalc has an Achilles heel, it will turn out to be data volume. The system accesses standard sources (Salesforce.com, AdWords, etc.) through their APIs, which often limit the number of records that can be pulled at once. Youcalc connectors can submit new calls until all the data has been read, but this is still awkward and will probably be slow for large volumes.

In addition, the data must be loaded into system memory during each user session. This also imposes some practical limits—we’re probably talking in the multi-gigabyte range—even though youcalc runs in the Amazon data cloud, which gives it access to very large servers. Madsen says the largest current installation works with data for 150 Salesforce.com users.

Youcalc was launched in its current form at the end of 2008, although the company has been working on its core technologies since 2003. Madsen said that 4,000 accounts were created in the first six months since launch, and there are currently more than 7,000 application sessions per week. Most are from small businesses, which makes sense for any number of reasons including price, functionality and ease of deployment. The company hopes eventually to attract larger firms as well.

LucidEra and Birst Blaze New Trails for On-Demand BI

I spent a few minutes last week on the Web sites of about eight or nine on-demand business intelligence vendors, and within a few days received emails from two of them ostensibly asking about much earlier visits where I must have registered with my email address. Given my current obsession with demand generation systems, I’m pretty sure this was no coincidence: they had deposited a cookie linked to my email address during the earlier visit, and used this address to react when I returned. I suppose I should admire this as good marketing, although the disingenuousness of the messages was a bit disturbing. I suppose they felt that was better than the creepy feeling I might get if they said they knew I had visited. [Postscript: I later spoke with one of the salespeople, who sadly assured me it was a total coincidence. She only wished her firm could react so effectively.]

(I was about to coin the word “disingenuity” to mean something that is ingeniously disingenuous [i.e., cleverly deceptive], but see that the dictionary already lists it as a synonym for disingenuousness. Pity. )

Whatever. The reason I was looking at the on-demand BI sites was I’d spoken recently with two vendors in the field and wanted to get some context. One of the two was LucidEra , which was giving me an update since my post about them in July.

They’re doing quite well, thanks, and most excited about a new services offering they call a “pipeline healthcheck”. This is a standardized analysis of a company’s sales pipeline to find actionable insights. LucidEra says it has been tremendously successful in demonstrating the value of their system and thus closing sales. Apparently, many marketers never learned how to analyze the information buried within their sales automation systems, simply because it wasn’t available back when they were being trained. So doing it for them, and helping them learn to do it for themselves, adds great value.

This reinforced one of my few really profound insights into the software business, which is that the marketing software vendors who succeed have been the ones who provide extensive services to help their clients gain value from their systems. (Well, I think it's profound.) Interestingly, when I told LucidEra I have recently been applying this insight to demand generation vendors, they said they had recently switched to a new demand generation vendor and—this is the interesting part—found the new system was so much simpler to use that very little vendor support was necessary. That’s an interesting tidbit, although it doesn’t necessary confirm my service-is-essential thesis. Perhaps it needs a corollary of some sort when the applications are obvious or the users are already trained. Facts can be so pesky.

The other vendor on my mind was Birst. I actually spoke to them back in early September, but their product announcement was under embargo until September 30 and in any case I’ve been focused since then on the demand generation guide (have I mentioned http://www.raabguide.com/ yet today?) I’m glad to get back to Birst, though, because I was quite intrigued by what they showed me. Basically they claim to have fully automated the entire business intelligence implementation process: loading the data, designing the warehouse, identifying interesting information, and creating dashboards to display the results.

I’ll admit to being skeptical of how well they can do this, but the company’s managers have some excellent credentials and Birst itself is a project of a Success Metrics, which has been providing Web-based opportunity discovery to insurance and pharmaceutical sales forces since 2006. They offered me an online workspace to play with the tool, but I haven’t had time to take them up on it. (I think their Web site makes that same offer to anyone.)

I did spend a few minutes playing with a prebuilt demo on the Web site: it’s a reasonable user interface for ad hoc analysis and building dashboard reports. There was a lag of up to five seconds between each click when I was working with the data, which would quickly get annoying if I were trying to do real work. Part of the lag may be caused by the underlying technology, which generates relational OLAP cubes on the fly in response to user queries. But it also appears the system uses a traditional Web interface, which redraws the screen after each click, rather than AJAX and similar technologies which provide a smoother, faster user experience.

I don’t want to dwell on the Birst user interface, partly because I haven’t tested it thoroughly and partly because you can judge it for yourself, but mostly because their more important claim is the automated implementation. As I said last March, I think the labor involved with building the system is the biggest obstacle to on-demand BI, so Birst’s claim to have solved this is the real news.

It would take some serious testing to assess how good a job Birst’s automated systems can really do. Still, the system can be useful even if it’s not perfect and it will presumably improve over time. So if you’re thinking about on-demand business intelligence, either for a specific purpose or just to better understand what’s possible, Birst is certainly worth a look.

Incidentally, my quick scan of other on-demand business intelligence vendors (Autometrics, BlinkLogic, Good Data, oco, OnDemandIQ, and PivotLink) showed that only oco made a similar claim about having automated the implementation process.

On the other hand, Good Data, PivotLink LucidEra and possibly oco are using in-memory or columnar databases (PivotLink’s is in-memory and columnar: they win). In theory these should give quicker response than Birst’s on-the-fly OLAP cubes, although actual performance depends on the implementation details. (Speaking of experience, Birst’s database technology has been running at Success Metrics for several years, and has scaled to the terabyte range. I don’t know what scales the other vendors have reached.) It also seems to me that in-memory and columnar databases should be particularly compatible with automated implementation because their simpler structures and greater efficiency make them more forgiving than conventional databases if the automated design is less than optimal. But no one in this particular group of vendors seems to have put the two together.

I don’t know when I’ll have time to give all these other vendors the attention they deserve. But based on what I’ve heard from LucidEra and Birst, and seen on the other vendors’ Web sites, I’m more optimistic about the potential of on-demand business intelligence than I was back in March.