Sunday, February 07, 2016

Marketing attribution systems: a quick look at the options

I’ve seen a lot of attribution vendors recently. If you're a regular reader here, you saw my reviews of Claritix (last week) and BrightFunnel (in December).  Last week caught up with Jeff Winsper of Black Ink, which I'll hopefully review before too long.  Bizible also popped up recently although I don’t recall the occasion; possibly something related to their interesting survey on “pipeline marketing” and attribution methods.

My rational brain knows that there’s probably no reason for this flurry of sightings beyond pure coincidence. But it’s human to see patterns where they don’t exist, so I did find myself wondering if attribution is becoming a hot topic. I can easily come up with a good story to explain it: marketing technology has reached a new maturity stage where the data needed for good attribution is now readily available, the cost of processing that data has fallen far enough to make it practical, and the need has reached a tipping point as the complexity of marketing has grown. So, clearly, 2016 will be The Year of Attribution (as Anna Bager and Joe Laszlo of the Internet Advertising Bureau have already suggested).

Or not. Sometimes random is just random. But now that this is on my mind, I've taken a look at the larger attribution landscape.  Quick searches for "attribution" on G2 Crowd and TrustRadius turned up lists of 29 and 17 vendors, respectively – neither including Brightfunnel or Claritix, incidentally.  A closer look found that 13 appeared on both sites, that each site listed several relevant vendors that the other missed, and that both sites listed multiple vendors that were not really relevant. For what it's worth, eight vendors of the 13 vendors listed on both sites were all bona fide attribution systems -- which I loosely define to mean they assign fractions of revenue to different marketing campaigns.  I wouldn't draw any grand conclusions from the differences in coverage on G2 Crowd and TrustRadius, except to offer the obvious advice to check both (and probably some of the other review sites or vendor landscapes) to assemble a reasonably complete set of options.

I've presented the vendors listed in the two review sites below, grouping them based on which site included them and whether I qualified them as relevant to a quest for an attribution vendor.  I've also added a few notes based on the closer look I took at each system in order to classify it.  The main questions I asked were:
  • Does the system capture individual-level data, not just results by channel or campaign?  You need the individual data to know who saw which messages and who ended up making a purchase.  Those are the raw inputs needed for any attempt at estimating the impact of individual messages on the final result.  
  • Does the system capture offline as well as online messages?  You need both to understand all influences on results.  This question disqualified a few vendors that look only at online interactions.  In practice, most vendors can incorporate whatever data you provide them, so if you have offline data, they can use it.  TV is a special case because marketers don't usually know whether a specific individual saw a particular TV message, so TV is incorporated into attribution models using more general correlations.
  • How does the vendor do the attribution calculations?  Nearly all the vendors use what I've labeled an "algorithmic" approach, meaning they perform some sort of statistical analysis to estimate the attributed values.  The main alternative is a "fractional" method that applies user-assigned weights, typically based on position in the buying sequence and/or the channel that delivered the message.  The algorithmic approach is certainly preferred by most marketers, since it is based in actual data rather than marketers' (often inaccurate) assumptions.  But algorithmic methods need a lot of data, so B2B marketers often use fractional methods as a more practical alternative.  It's no accident that the only B2B specialist listed here, Bizible, is the only company that uses a fractional method, as do B2B specialists BrightFunnel and Claritix.  It's also important to note that the technical details of the algorithmic methods differ greatly from vendor to vendor, and of course each vendor is convinced that their method is by far the best approach.
  • Does the vendor provide marketing mix models?  These resemble attribution except they work at the channel level and are not based on individual data.  Classic marketing mix models instead look at promotion expense by channel by market (usually a geographic region, sometimes a demographic or other segment) and find correlations over time between spending levels and sales.  Although mix models and algorithmic attribution use different techniques and data, several vendors do both and have connected them in some fashion.
  • Does the vendor create optimal media plans? I'm defining these broadly to include any type of recommendation that uses the attribution model to suggest how users should reallocate their marketing spend at the channel or campaign level.  Systems may do this at different levels of detail, with different levels of sophistication in the optimization, and with different degrees of integration to media buying systems. 
Of course, there are plenty of other points that differentiate these systems.  But this list should be a useful starting point if you're considering a new attribution system -- as well as a reminder of the need to define your requirements and drill into the details before you make a final selection.

Attribution Systems

G2 Crowd and TrustRadius
  • Abakus: individual data; online and offline; algorithmic; optimal media plans
  • Bizible: individual data; online and offline; fractional; merges marketing automation plus CRM data; B2B
  • C3 Metrics: individual data; online and TV; algorithmic; optimal media plans 
  • Conversion Logic: individual data; online and TV; algorithmic;optimal media plans
  • Convertro: individual data; online and offline; algorithmic; mix model; optimal media plans; owned by AOL
  • MarketShare DecisionCloud: individual data; online and offline; algorithmic; mix models; optimal media plans; owned by Neustar
  • Rakuten Attribution: individual data; online only; algorithmic; optimal media plans; formerly DC Storm, acquired by Rakuten marketing services agency in 2014
  • Visual IQ: individual data; online and offline; algorithmic; optimal media plans
G2 Crowd only
  • BlackInk: individual data; online and offline; algorithmic; provides customer, marketing & sales analytics 
  • Kvantum Inc.: individual data; online and offline; algorithmic; mix models; optimal media plans
  • Marketing Evolution:  individual data; online and offline; algorithmic; mix model; optimal media plans
  • OptimaHub MediaAttribution  individual data; online and offline; attribution method not clear; data analytics agency with tag management, data collection, and analytics solutions
    TrustRadius only
    • Adometry: individual data; online and offline; algorithmic; mix models; optimal media plans; owned by Google
    • ThinkVine: individual data; online and offline; algorithmic; mix models; optimal media plans; uses agent-based and other models
    • Optimine:  individual data; online and offline; algorithmic; optimal media plans
    Other Systems

    G2 Crowd and TrustRadius

    G2 Crowd only
    • Adinton: Adwords bid optimization and attribution; uses Google Analytics for fractional attribution
    • Blueshift Labs: real-time segmentation and content recommendations; individual data but apparently no attribution
    • IBM Digital Analytics Impression Attribution: individual data; online only; shows influence (not clear has fractional or algorithmic attribution); based on Coremetrics
    • LIVE: for clients of WPP group; does algorithmic attribution and optimization
    • Marchex: tracks inbound phone calls
    • Pathmatics: digital ad intelligence; apparently no attribution
    • Sizmek: online ad management; provides attribution through alliance with Abakus
    • Sparkfly: retail specialist; individual data; focus on connecting digital and POS data; campaign-level attribution but apparently not fractional or algorithmic
    • Sylvan: financial services software; no marketing attribution 
    • TagCommander: tag managemenet system; real-time marketing hub with individual profiles and cross-channel data; custom fractional attribution formulas
    • TradeTracker: affiliate marketing network
    • Zeta Interative ZX: digital marketing agency offering DMP, database, engagement and related attribution; mix of tech and services

    Tuesday, February 02, 2016

    Claritix Assembles Marketing Data for Analysis: Maybe That's Enough

    Most of the work in any marketing analytics project is integrating data from multiple systems. Claritix carries this insight to one logical conclusion by offering a system that does data assembly, basic reporting, and little else.  No fancy attribution methodologies or custom journey maps here (although they’re on the way). I’m not fully convinced this is enough to justify using Claritix but am open to the possibility. Here’s a deeper look.
     

    As I just said, Claritix’s chief function is assembling customer data from multiple sources. The system has prebuilt connectors to import data from popular vendors including Salesforce.com, Marketo, Hubspot, SAP, SugarCRM, and Facebook. It can connect with others through standard APIs. The imported data is loaded into MongoDB, a NoSQL database that offers great flexibility and ease of deployment. Claritix applies sophisticated algorithms to cleans the data and match contacts based on similarity.  It also uses matches created elsewhere such as lead IDs used to synchronize CRM and marketing automation data or cookie IDs imported from Google Analytics. The matching happens at both the contact and account level. Imported data includes contacts, funnel stages, campaigns, channels, revenue, and content.


    Users can access this data through dashboards, charts, and views. There are different dashboards for the main data types (campaigns, funnel stages, channels, etc.). These provide basic information such as impressions, engagements, visits, deals and revenue by campaign, or sources, stages, conversion rates, and average duration by funnel stage. The specific measures depend on the data type. Users can drill into details down to the contact level. Views can show results for user-defined segments.

    Claritix also lets users assemble information into binders, which are contain pages that are snapshots of dashboards, charts, and notes. These can be exported to PDF or slides or viewed directly within Claritix. Binders can update themselves at regular intervals. Collaboration features let users attach virtual “sticky notes” to screen images and share these via Slack or Claritix’s own communication channels.

    So far as I know, that’s pretty much all that the system does. There is no capability, for example, to write the assembled data back to source systems for their own use.  Claritix tells me this has been quite sufficient for their initial clients, who have liked the fact that set-up is virtually all automated or handled by the vendor.  This has let them assemble data across multiple systems in ways that would otherwise have been impossible or hugely expensive. Certainly price is an advantage: Claritix starts at $1,000 per month for up to 10,000 contacts in the database, with the cost per contact decreasing for higher volumes. A system with more advanced reporting, such as Brightfunnel (which I reviewed in December and has been a consulting client) starts at $3,000 per month or higher. Still, you have to decide whether you’ll need the features that Claritix is missing; if so, you’ll end up missing many of the beneifts that good marketing measurement provides.   As Captain Planet used to say, the power is yours.

    Thursday, January 28, 2016

    Real Magnet Offers Complex Campaigns Without the Flow Charts

    I recently saw a useful distinction between AI – artificial intelligence, which is machines replacing people – and IA – intelligent assistance, which is machines helping people. Real Magnet, an email service provider turned marketing automation vendor with over 1,000 clients, doesn’t position itself as either. But its flagship feature is letting marketers create sophisticated, multi-step campaigns by answering handful of questions in a template. The remaining work to implement the marketers’ choices is done by the system. That sounds like Intelligent Assistance to me.

    This piques my interest because I’ve long argued that the chief roadblock to wider use of marketing automation is the difficulty of setting up campaigns, and have offered Artificial Intelligence as the solution. That is, I have been looking for systems that automatically design campaigns (or deliver optimal customer treatments without campaigns), thereby removing the roadblock by doing the work on the marketers' behalf. This has always felt a bit optimistic, but, then, so do self-driving cars. An Intelligence Assistance approach seems like a more plausible near-term alternative – analogous to the “driver assist” features already finding their way into automobiles.

    Of course, many marketing automation systems use templates as part of their campaign set-up. What sets Real Magnet apart is the entire set-up is done through the templates. The system does offer a conventional workflow builder (which is quite nice, in fact) but it's not needed for campaigns that fit the standard templates.  Users do have the ability to convert template campaigns to the workflow format for customization. .
    Campaign Steps

    To make things a bit more concrete: the Real Magnet campaign picker starts out by asking the user to select their industry from a list. The system then presents a choice of industry-appropriate campaign types such as subscription renewals, webinar promotions, birthday and anniversary messages, and welcome kits. Once a campaign is chosen, the system presents three or four steps with a few questions per step: for example, steps for a standard email campaign are select the audience, select the messages and intervals, and schedule the execution. Most of these selections are themselves made by picking from predefined options or templates, with the ability for users to set up new options as needed.  Real Magnet support staff is also available to set up options when clients need help.

    Campaign Workflow
    Campaign templates can include multiple steps and branching flows, such as follow-up messages to people don’t complete a registration process. Campaign steps can include many types of actions, from sending messages to assigning group membership, setting field values, managing point totals, suppressing further communications, or directing the flow to another branch or block within a branch. Over-all, these options make Real Magnet a very powerful system.

    Campaigns can also run processes such as a/b tests, lead scoring, landing pages, segmentation, or suppression lists. In other words, pretty much any task that would ordinarily require complex set-up can be created through a template. Not surprisingly, Real Magnet reports its users – typically small marketing departments with limited resources – find this very appealing. Those are exactly the kinds of users who struggle to deploy advanced features in most marketing automation systems.

    Real Magnet also provides several levels of campaign reporting, from a dashboard with summary statistics to performance by individual messages within a campaign to lists of campaign participants. Reports vary based on the campaign details.  They often include engagement rates and achievement of user-specified goals. Reports can also consolidate results for groups of campaigns.

    The Real Magnet database is largely limited to a single record per customer, although the system does track promotion history and related events such as form files and survey completions. Users can add custom fields to the customer record but not custom tables. The system integrates with major CRM and association management systems, and can access their contents to some degree. It also stores social media handles for Facebook and Twitter, can send messages through those systems, and can create scores based on social media behaviors such as retweets and likes.

    Real Magnet started business in 2000 as an email service provider. More than half of the company's 1000-plus clients are trade and professional associations, with additional concentrations in education and publishing. Pricing of the Real Magnet system is based on number of emails sent. Packages start around $200 to $300 per month.

    Monday, January 25, 2016

    Avention DataVision Gives Sales and Marketing Systems Unified Access to B2B Customer Data Quality and Alerts

    My look last week at True Influence’s InsightBASE, a relatively new-fangled approach to intent data, was karmically balanced by a conversation with Avention, a old-line data aggregator that traces its roots to CD-ROM business lists from Lotus OneSource. The folks at Avention had reached out to discuss their latest product, DataVision, which extends Avention’s reach from sales enablement to marketing systems.  The goal is giving clients a single data source to support both departments.

    DataVision lets clients upload customer lists to be cleaned and enhanced by matching against Avention’s own master file, which is itself compiled from some seventy sources. Sales and marketing systems can then access the results in an online database, providing all departments with a single, consistent view of their consolidated data. The information includes both companies and contacts and supplements standard profile information with event-based "signals" derived from news reports, company Web sites, and social media postings. Clients can set up alerts based on signals and can acquire new names that are similar to their current customers.

    If this sounds familiar, it’s because Reachforce, InsideView, SalesLoft and other data vendors offer similar services. Predictive modeling vendors including Leadspace, Lattice Engines, Mintigo, and Everstring also provide enhancement and signal-based alerts, although usually with less depth of detail. The biggest difference is those vendors usually send the enhanced information back to client systems rather than keeping it in an external database which sales and marketing systems access directly.

    But different isn’t necessarily better. No one will discard their CRM or marketing automation database and use the DataVision file instead. There’s simply too much other information within the sales and marketing systems. So, in practice, DataVision will be used to update a company’s existing databases, pretty much the same as its competitors. The data may be a bit fresher, since any query to DataVision will return the latest information available to Avention. DataVision also provides some nice tools to visualize the distribution of a client’s customers across geography, industry, company size, and other dimensions, and to compare those distributions with the entire Avention universe of known firms. Again, these features are useful even if they are not necessarily unique.

    In short, Avention DataVision is a solid option when you’re looking to clean and enhance your company’s customer and prospect data – something every firm needs to do. Intent data and predictive modeling are not part of the mix yet, but it’s easy to imagine those being added in the future. Whether Avention is your best choice will depend on your specific situation.  The only way to know is to define your exact requirements, test several sources, and evaluate the results. The good news is you have lots of vendors to choose from, so you have a good chance of finding one that fits your needs.

    Wednesday, January 20, 2016

    True Influence InsightBASE Simplifies Use of B2B Intent Data

    Intent data is one of hottest topics in marketing today – see, for example, Oracle’s recent purchase of AddThis. But while the promise of intent data is irresistible – “reach prospects with demonstrated interest in your product!” – the reality has been less appealing. Even setting aside issues of accuracy and coverage, there are problems with both advertising and email, the two primary applications for intent data.  Advertising can reach large numbers of people but just a tiny fraction will click on an ad and a tiny fraction of those will provide contact information. Intent-based email lists are obviously contactable but volumes are often quite low.

    B2B lead generation vendor True Influence  today announced a new product to help fill these gaps. InsightBASE monitors intent signals – in the form of visits to Web pages with relevant content – and notifies clients when there is surge in activity for companies on a target list. The notifications can be loaded as lists into a marketing automation or CRM system, where they can trigger advertising, sales calls, or other actions. Clients also receive contact names, email addresses, and phone numbers at those companies. The contact data is drawn from True Influence’s master list of 30 million business contacts, which are continuously verified to ensure deliverability. Although the names are not tied directly to Web visits, they are selected by job title and level, so they should be appropriate. Visits are tied to companies based on the user’s Web domain – a typical approach although one that can’t misses many sessions from home offices and mobile devices.

    So, what distinguishes InsightBASE from other intent-based products? The main difference is that users get the company and contact lists. This contrasts with many intent-based advertising vendors, who serve ads to qualified audiences but don't tell clients exactly whom they’re reaching.  InsightBASE also differs from predictive marketing vendors who use intent data as inputs to their scoring systems and in some cases also provide lead lists: although predictive models almost surely do a better job of isolating the best prospects than InsightBASE’s simple profiles plus surge tracking, the models add considerable cost and complexity.

    True Influence also says its partner network gives it access to more intent data than anyone else.  That's possible but I haven’t done the research to confirm it. Nor is it necessarily important, since activities on some Web sites are less significant than others. The value of data from True Influence, or anyone else, can only be resolved through tests, which will probably give different results for different purposes.

    The mechanics of InsightBASE are straightforward. Users set up a campaign by either uploading their own list of target companies or making selections from True Influence’s own database of more than three million Web domains. Selections can use standard filters such as industry, location, number of employees, or domain type (such as .edu or .gov). They can also be based on use of specific technologies, allowing marketers to target competitors’ customers. The next step is to specify keywords to use as indicators of intent. True Influence has its own list of about 5,000 keywords and uses them to do its own classification of Web pages.  It can add new keywords as needed.  Finally, InsightBASE runs a report showing how many of the target companies visited pages with the specified keywords over the past thirty days, and whether their activity increased, decreased, or remained the same compared with previous periods. This gives a good indication of the potential volume of future activity.

    Once the campaign begins, users can extract lists of domains that exceeded a specified activity level or had change in activity. They can export the domains, contacts associated with the domains, or both. InsightBASE has standard integrations with Marketo, Oracle Eloqua, and Salesforce.com. Once the lists are loaded into those systems, they can be used for email, advertising, sales calls, or other purposes.

    Pricing for InsightBASE is based on the number of domains monitored, starting at $2,500 per month for 2,500 domains with discounts for higher volumes. There are no separate fees for additional campaigns, contact names, or supporting services. True Influence reports that its initial tests showed companies with activity surges responded to promotion emails at four times the rate of non-targeted companies.

    Tuesday, January 19, 2016

    OneSpot Offers Automated Content Selection Targeted at Long Term Results

    As you know from previous blog posts, I’ve been borderline obsessed recently with systems that automatically create multi-step campaign flows. So when I saw that OneSpot calls its product a “content sequencing engine” you can bet they had my attention. When I read that “OneSpot’s machine learning technology serially delivers multiple pieces of content to your users based on their interests and digital journey stage,” I thought I might have found the Holy Grail itself.

    OneSpot was already on my list of interesting companies because they automatically reformat content to use in different channels. This is a good example of applying artificial intelligence to reduce the workload on marketing departments so they can deliver more targeted messages at lower cost. The company had been doing this and programmatic ad buying since its start in 2012.

    The content sequencing engine is a more recent addition. According to OneSpot Chief Marketing Officer Adam Weinroth, one of the things that make the engine special is that it finds the best content to generate repeat engagement rather than immediate response. Another is that it delivers the content through advertising on external Web sites as well as in email, a company’s own Web site, mobile and social. In other words, OneSpot's “sequencing” is about coordinating messages in different channels, not delivering groups of messages in a fixed order. Since my own quest has been automated creation of ordered messages, OneSpot isn't the Grail I seek.  But it's still quite special: as Weinroth points out, there are many systems to select offers for ecommerce but few for content marketing. Even fewer support advertising along with other channels.

    OneSpot deploys several major pieces of technology to make this happen. A content analytics engine automatically classifies existing content without manual tagging. The classification categories (a.k.a. taxonomy) are themselves created automatically. This automation removes one of the largest bottlenecks in deploying high volumes of content. The reformatting engine then prepares the content to be distributed across multiple channels, again without manual labor. The automated recommendations are based on a profile that includes results from all the channels supported by OneSpot. This cross-channel perspective is what lets OneSpot base recommendations on repeat engagement rather than immediate response. It also lets the system bid on advertising to non-customers as well as messages to current customers. OneSpot also has its own real-time bidding engine. It is integrated with major Web ad exchanges, which Weinroth said allows it to potentially bid on 36 million impressions per minute.

    Other aspects of OneSpot are more conventional. The system uses a Javascript tag to integrate with company Web sites and third party ad sites. Email is personalized through API connections with email service providers. Advertising audiences are selected by defining targeting criteria against standard Web audiences. Customers and prospects are identified through Web cookies, hashed (anonymized) email addresses linked to cookies, and third party services for device targeting.

    OneSpot continues to extend its technology. Weinroth showed me a beta version of a report that compares demand for each topic with the number of existing content pieces for that topic and the impact of content with that topic on reengagement. While OneSpot won’t actually create the additional content, this is still another step towards replacing manual tasks with automation.

    Pricing for OneSpot starts north of $100,000 for an annual contract. The actual fees are based on traffic volumes and channels supported. Weinroth said most clients use the system in at least two channels, typically Web site messages and Web ad retargeting.

    Tuesday, January 05, 2016

    Rating the Crowd-Sourced Marketing Software Review Sites

    What began as a whimsical “landscape of landscapes” led to the serious realization that crowd-sourced review sites are the most common type of vendor directory.  Fifteen of the 23 sources listed in my original graphic fell into that category. This begged for a deeper look at the review sites to understand how they differ and which, if any, could replace the work of professional reviewers (like me) and software guides (like my VEST report).

    The first question was which sites draw a big enough crowd to be useful. I used Alexa traffic rankings, which are far from perfect but good enough for this sort of project. (Compete.com gave similar rankings except that TrustRadius came in lower, although still in the top 10.)  After adding two review sites that I learned about after the original post, I had 17 to consider. In order of their Alexa rankings, they were:



    Since crowd wisdom without a crowd can’t be terribly effective, I limited further analysis to the top 10 sites. Of these, AlternativeTo.net, SocialCompare, and Cloudswave were different enough from the standard model that it made sense to exclude them. This left seven sites worth a closer look.

    The next question was coverage by the sites of marketing technology. Every site except TrustRadius covered a broad range of business software from accounting to human resources to supply chain as well as CRM and marketing. TrustRadius was more focused on customer-related systems although it still had business intelligence and accounting. The numbers of categories, subcategories, and marketing subcategories all differed widely but didn’t seem terribly significant, apart from SoftwareInsider and DiscoverCloud looking a bit thin. Differences in the numbers of products in the main marketing categories also didn't seem meaningful  – although they do illustrate how many products there are, in case anyone needs reminding.



    What did look interesting was the number of ratings and/or reviews for specific products. I sampled leading marketing automation vendors for different sized companies. It turns out that G2Crowd and TrustRadius had consistently huge leads over the others. I didn’t check similar statistics for other software categories, but this is probably the one that counts for most marketers.


    Of course, quality matters as well as quantity. In fact, it probably matters more: my primary objection to crowd-sourced software reviews has always been that users’ needs for software are so varied that simple voting based on user satisfaction isn't a useful indication of how a system will for any particular buyer.  This is different from things like restaurants, hotels, and plumbers, where most buyers want roughly the same thing.

    Software review sites address this problem by gathering more detail about both the products and the reviewers. Detailed product information includes separate numeric ratings on topics such as ease of use, value for money, and customer support; detailed ratings on specific features; and open-ended questions about what reviewers liked most and least, how they used the system, and what they’re recommend to others. Reviewer information on all sites except Software Advice starts with verifying that the user is a real person through requiring a LinkedIn log-in. This lets the review site check the reviewer’s name, title, company, and industry, although these are not always fully displayed. Some sites verify that the reviewer actually uses the product. Some provide other background about the reviewer’s activities on the review site and how their work has been rated.

    I can't show how each vendor handles each of those items without going into excruciating detail. But the following table gives a sense of how much information each site collects. Of course, reviewers don’t necessarily answer all these questions. (Caution: this information is based on a relatively quick scan of each site, so I’ve probably missed some details. If you spot any errors, let me know and I’ll correct them.)  When it comes to depth, TrustRadius and DiscoverCloud stand out, although I was also impressed by the feature details and actual pricing information in G2Crowd.


    The number and depth of reviews are clearly the most important attributes of review sites.  But they also differ in other ways.  Selection tools to identify suitable vendors are remarkably varied – in fact, the only filter shared by all sites is users' company size. Industry is a close second (missing only in DiscoverCloud), while even selections based on ratings are found in just four of the seven sites. Only three sites let users select based on the presence of specific features, an option I believe is extremely important.


     Looking beyond selection tools: most sites supplement the reviews with industry reports, buyer guides, comparison grids, and similar information to help users make choices. Several sites let users ask questions to other members.

    So, back to my original question: can crowd-sourced review sites replace professional software reviews? I still don’t think so: the coherent evaluation of a practiced reviewer isn’t available in the brief comments provided by users, even if those comments are accompanied by information about specific product features. This may sound like self-serving mumbo-jumbo, but I do think a professional reviewer can articulate the essence of many products more effectively than users who report only on their personal experience. (Yes, I really just wrote "articulate the essence".)

    But whether sites can replace professional reviewers is really the wrong question.  What matters is the value the review sites offer on their own. I’d say that is considerable: given enough volume, they indicate the rough market share of different products, the types of users who buy each system, and what worked well or poorly for different user types. User comments give a sense of what each writer found important and how they reached their judgements.  This in turn lets readers assess whether that reviewer’s needs were similar to their own. Buyers still need to understand their own requirements, but that’s something that no type of review can replace.