Here is the AXIS itself, with QlikView alone at the top:
And therein lies the problem. Fond as I am of QlikView, I can’t imagine any measure by which it would be a dominant business intelligence tool, let alone rank so far above its competitors. So I was more than a little perplexed to see the Aberdeen ranking, especially because I wasn’t aware that Aberdeen produced something similar to the Forrester “waves” and Gartner “magic quadrants”.
Indeed, it turns out that the AXIS program is brand new. The BI report is the second to be issued but will be followed by several per quarter. Aberdeen’s materials describe its reports as “unlike other comparative products that primarily focus on feature and functionality”, but to my eye it looks pretty similar. Apparently what makes it “the market’s first technology solution software provider assessment tool that is truly customer-centric” is that one of its two dimensions, “value delivered”, is based on Aberdeen’s surveys that identify best-in-class companies.
Exactly how this relates to the vendor rankings is unclear, although I could find out if I paid $895 for the report. Maybe a higher “value delivered” ranking means the product is used by more best-in-class companies, although I rather doubt it given QlikView's still-limited market penetration. More likely, the higher ranked products are viewed by users as delivering greater value, perhaps with some weighting towards the relative performance of the companies doing the ranking. I’ve no doubt that QlikView users are more enthusiastic than any other vendors’, both because it truly is a great product but also because it’s still in the relatively early adapter stage where users tend to be highly motivated and vocal. Perhaps Aberdeen looks at the features desired by best-in-class companies and compares those to the features delivered by the different products, although this too seems unlikely.
The second dimension of “market readiness” is described by Aberdeen as based on “evaluation of responses to a standardized vendor questionnaire, analyst briefings, public records and customer interviews.” This sure sounds like a conventional feature-and-function assessment to me.
As someone who has been evaluating software for many years, I fully appreciate the appeal of these sorts of matrices. Vendors love them because, if they’re ranked near the top, it gives them something to crow about. Buyers love them because they can save work by considering only the top-ranked alternatives (even though vendors piously warn this is inappropriate). The analyst firms love them because they get lots of publicity, both directly because the press loves a horse race and from the winning vendors who promote them.
I have always avoided producing such rankings, even when people ask for them, precisely because they make it too easy for buyers to avoid the essential work of assessing products against their own needs. Still, the commercial advantages of the rankings are so great that I may yet feel impelled to produce them. That being the case, I can’t really criticize Aberdeen for rolling out their own. But I do hope they make a serious effort at educating people on what the "AXIS" means and how it should and shouldn’t be used.
David, hello. We'd be happy to provide more insight into the Aberdeen AXIS methodology, and give you more direct insight into the Value Delivered and Market Readiness scores. To your point, the AXIS focuses less on feature/function, and more on how business users thrive when partnering with top-performing technology vendors.
ReplyDelete