Tuesday, November 25, 2008
Marketbright sees its most important differentiator as a sophisticated architecture designed to coordinate marketing activities throughout a large organization. This doesn't strike me as a very effective selling point: buying a product because of its architecture is the software equivalent of reading Playboy for the articles. (Do I get credit for resisting the temptation to link to Playboy.com?) What really matters are the features facilitated by this architecture. According to the company Web site, these include “full document repository and asset management, multi-currency budget planning and management and a range of integrated collaboration features”. Now that's something to get excited about. Hubba hubba, eh?
At least this clarifies which end of the demand generation market will find Marketbright most attractive. Indeed, Pilcher told me that product sells best to people who have worked in large organizations and seen first-hand what it takes to support collaboration within marketing. These people may currently be working in small firms, so Marketbright has ended up with customers of all sizes. Pricing ranges from $20,000 to $200,000 per year based on the modules and number of users, so the system is financially competitive facross the spectrum.
Not having seen the product, I don’t know whether its sophisticated management features come at the price of end-user complexity. This is a common trade-off. One hint of Marketbright’s approach may be that Pilcher recommends his clients build separate campaigns for different customer segments, rather than “boiling the ocean” by creating a single campaign with branches to handle all contingencies. This suggests that Marketbright has at least tried to keep things as simple.
Pilcher and I had a lengthy subsequent email discussion about usability, “violently agreeing” that it’s an important though elusive measure. My final conclusion was similar to the positions I’ve taken before: usability has to be measured separately for different functions, levels of campaign sophistication, and user skill sets. Where I may have changed my mind is a grudging agreement that it’s legitimate to summarize the details into simple measures that could be plotted in a graph. The obvious ones are usability and functionality scores. I still fear this could mislead by obscuring important information: for example, deep functionality in a few areas could generate the same score as limited functionality across many areas. (Pilcher proposed the number of channels as a separate dimension, but then a system with weak functionality in many channels scores better than a system that is strong in just a few. I consider that equally misleading.) But if a two-dimensional summary offers an attractive entry point which in turn leads to deeper exploration, it’s better than scaring people away by showing them the details at the start.
Wednesday, November 19, 2008
But I’ve also been approached by some of the other demand generation specialists. My original set of products was based on a general knowledge of which companies are most established, plus some consultation with vendors to learn who they felt were their main competitors. So far the original list of Eloqua, Vtrenz, Marketo, Manticore Technology and Market2Lead has proven a good set of choices. Yet there are so many more vendors I could add. How to choose?
The general rule is pretty obvious: pick the vendors that people are most interested in. We do, after all, want people to buy this thing. Of course, you want some wiggle room to add intriguing new products that they may not know about. Still, you mostly want to the report to include the vendors they are already asking about.
But although the general rule is obvious, which vendors are most popular is not. Fortunately, we have the Internet to help. It offers quite a few ways to measure interest in a vendor: Web searches, blog mentions, Google hits, and site traffic among them. All are publicly available with almost no effort. After a close analysis of the alternatives, I have decided the Alexa.com traffic statistics are the best indicator of vendor market presence. (You can read about the analysis in fascinating detail on my marketing measurement blog, MPM Toolkit.)
The table below shows the Alexa rankings and share statistics for the current Guide entries, the four marketing automation vendors already mentioned, and a dozen or so contenders.
Already in Guide:
|Unica / Affinium*|
|Other Demand Generation:|
The figures themselves need a little explaining. The Alexa rank is a “combined measure of page views and number of users”, with the most popular site ranked number 1, next-most-popular ranked number 2, etc. (In case you're wondering, the top three are Yahoo!, Google and YouTube.) Alexa share represents “percent of global Internet users who visit this site”. The rank and share figures correlate closely, but share is probably for comparing sites, since the ratio directly reflects relative traffic. That is, a share figure twice as large as another share figure indicates twice as many visitors, while a rank that is one half as large as another rank doesn’t necessarily mean twice as much traffic.
The figures for the existing vendors, in the first block of the table, give pretty much the ranking you’d expect. One wrinkle is that Vtrenz is owned by Silverpop, so Silverpop.com presumably siphons off a great deal of traffic from Vtrenz.com. On the other hand, Silverpop is a major email service provider in its own right, so a large share of the Silverpop.com traffic probably has nothing to do with Vtrenz. In any event, I’ve listed both sites in the table. Vtrenz is clearly a major vendor, so nothing is at stake here except bragging rights.
What’s more interesting is the figures for the Marketing Automation group. Unica is quite popular, while the other vendors are much less visited. This doesn’t particularly surprise me, although seeing Alterian, Aprimo and Neolane rank well below Manticore Technology and Market2Lead is odd. Perhaps these vendors are more obscure than I had realized. Still, they are much larger firms and do much more marketing than Manticore or Market2Lead. Interestingly, the other measure I found somewhat credible, IceRocket’s count of blog mentions, ranks Alterian, Aprimo and Neolane considerably higher than Manticore and Market2Lead. (See the MPM Toolkit post for details.) So the marketing automation vendors are probably a little more important to potential Guide buyers than the Alexa numbers suggest.
But my real concern was the Other Demand Generation group. Here, the Alexa figures do provide some very helpful insights. Basically they suggest that Marketbright, Pardot, Marqui and ActiveConversion, are all pretty much comparable in market presence to Manticore and Market2Lead. I spoke with Marketbright and Pardot this week and connected with ActiveConversion some time ago. Based on those conversations, this seems about right. (Marqui is a special case because they fell on financial hard times and the assets were recently purchased.) Rankings fall off sharply for the other vendors on the list, providing a reasonable cut-off point for the next round of Guide entries.
Of course, nothing is set in stone. Perhaps one of the smaller vendors can convince me that they have something special enough to justify including them. Plus there is still the question of whether I should invest the effort to expand the Guide at all, and what sequence I do the additions. But, whatever the final result, it’s nice to have an objective way to measure vendor market presence.
Thursday, November 13, 2008
After a few bon mots that probably no one else will find clever ("Usability is hard to measure; features are easy to count" "Small hard facts beat big blurry realities") I got to describing the steps in a usability-aware selection process:
- define business needs
- define processes to meet those needs
- define tasks within each process
- identify systems to consider, then, for each system:
- determine which users will do each task
- determine how much work each task will be
- compare, rank and summarize the results
As a point of comparison, it's steps 3, 5 and 6 that differ from the conventional selection process. Step 3 in a conventional process would identify features needed rather than tasks, while steps 5 and 6 would be replaced with research into system features.
What I realized as I was writing this was that the real focus is not on usability, but on defining processes and tasks. Usability measures are something of a by-product. In fact, the most natural way to implement this approach would be to score each system for each task, with a single score that incoporates both functionality and as ease of use. Indeed, as I wrote not long ago, standard definitions of usability include both these elements, so this is not exactly an original thought.
Still, it does mean I have to restructure the terms of the debate (at least, the one inside my head). It's not usability vs. features, but process vs. features. That is, I'm essentially arguing that selection processes should invest their effort in understanding the company business processes that the new system must support, and in particular in which the tasks different users will perform.
The good news here is that you'll eventually need to define, or maybe redefine, those processes, tasks and user roles for a successful implementation. So you're not doing more work, but simply doing the implementation work sooner. This means a process-focused evaluation approach ultimately reduces the total work involved, as well as reducing implementation time and improving the prospects for success. By contrast, time spent researching system features is pretty much a waste once the selection process is complete.
Of course, this does raise the question of whether the feature information assembled in the Raab Guide to Demand Generation Systems is really helpful. You won't be surprised to find I think it is. This is not so much because of the feature checklist (truly my least favorite section) but because the Guide tries to show how the features are organized, which directly impacts system usability. Plus, of course, the absence of a necessary feature makes a system unusable for that particular purpose, and that is the biggest usability hit of all. What the Guide really does is save readers the work of assembling all the feature information for themselves, thereby freeing them to focus on defining their own business processes, tasks and users.
In conclusion, you should all go and buy the Guide immediately.
Thursday, November 06, 2008
(I was about to coin the word “disingenuity” to mean something that is ingeniously disingenuous [i.e., cleverly deceptive], but see that the dictionary already lists it as a synonym for disingenuousness. Pity. )
Whatever. The reason I was looking at the on-demand BI sites was I’d spoken recently with two vendors in the field and wanted to get some context. One of the two was LucidEra , which was giving me an update since my post about them in July.
They’re doing quite well, thanks, and most excited about a new services offering they call a “pipeline healthcheck”. This is a standardized analysis of a company’s sales pipeline to find actionable insights. LucidEra says it has been tremendously successful in demonstrating the value of their system and thus closing sales. Apparently, many marketers never learned how to analyze the information buried within their sales automation systems, simply because it wasn’t available back when they were being trained. So doing it for them, and helping them learn to do it for themselves, adds great value.
This reinforced one of my few really profound insights into the software business, which is that the marketing software vendors who succeed have been the ones who provide extensive services to help their clients gain value from their systems. (Well, I think it's profound.) Interestingly, when I told LucidEra I have recently been applying this insight to demand generation vendors, they said they had recently switched to a new demand generation vendor and—this is the interesting part—found the new system was so much simpler to use that very little vendor support was necessary. That’s an interesting tidbit, although it doesn’t necessary confirm my service-is-essential thesis. Perhaps it needs a corollary of some sort when the applications are obvious or the users are already trained. Facts can be so pesky.
The other vendor on my mind was Birst. I actually spoke to them back in early September, but their product announcement was under embargo until September 30 and in any case I’ve been focused since then on the demand generation guide (have I mentioned http://www.raabguide.com/ yet today?) I’m glad to get back to Birst, though, because I was quite intrigued by what they showed me. Basically they claim to have fully automated the entire business intelligence implementation process: loading the data, designing the warehouse, identifying interesting information, and creating dashboards to display the results.
I’ll admit to being skeptical of how well they can do this, but the company’s managers have some excellent credentials and Birst itself is a project of a Success Metrics, which has been providing Web-based opportunity discovery to insurance and pharmaceutical sales forces since 2006. They offered me an online workspace to play with the tool, but I haven’t had time to take them up on it. (I think their Web site makes that same offer to anyone.)
I did spend a few minutes playing with a prebuilt demo on the Web site: it’s a reasonable user interface for ad hoc analysis and building dashboard reports. There was a lag of up to five seconds between each click when I was working with the data, which would quickly get annoying if I were trying to do real work. Part of the lag may be caused by the underlying technology, which generates relational OLAP cubes on the fly in response to user queries. But it also appears the system uses a traditional Web interface, which redraws the screen after each click, rather than AJAX and similar technologies which provide a smoother, faster user experience.
I don’t want to dwell on the Birst user interface, partly because I haven’t tested it thoroughly and partly because you can judge it for yourself, but mostly because their more important claim is the automated implementation. As I said last March, I think the labor involved with building the system is the biggest obstacle to on-demand BI, so Birst’s claim to have solved this is the real news.
It would take some serious testing to assess how good a job Birst’s automated systems can really do. Still, the system can be useful even if it’s not perfect and it will presumably improve over time. So if you’re thinking about on-demand business intelligence, either for a specific purpose or just to better understand what’s possible, Birst is certainly worth a look.
Incidentally, my quick scan of other on-demand business intelligence vendors (Autometrics, BlinkLogic, Good Data, oco, OnDemandIQ, and PivotLink) showed that only oco made a similar claim about having automated the implementation process.
On the other hand, Good Data, PivotLink LucidEra and possibly oco are using in-memory or columnar databases (PivotLink’s is in-memory and columnar: they win). In theory these should give quicker response than Birst’s on-the-fly OLAP cubes, although actual performance depends on the implementation details. (Speaking of experience, Birst’s database technology has been running at Success Metrics for several years, and has scaled to the terabyte range. I don’t know what scales the other vendors have reached.) It also seems to me that in-memory and columnar databases should be particularly compatible with automated implementation because their simpler structures and greater efficiency make them more forgiving than conventional databases if the automated design is less than optimal. But no one in this particular group of vendors seems to have put the two together.
I don’t know when I’ll have time to give all these other vendors the attention they deserve. But based on what I’ve heard from LucidEra and Birst, and seen on the other vendors’ Web sites, I’m more optimistic about the potential of on-demand business intelligence than I was back in March.