SiteCore Adds Analytics and Marketing To Web Content Management

Summary: SiteCore has added extensive analytical and marketing features to its Web content management system. The integrated analytics should save considerable effort for marketers. Channel-specific marketing automation is less appealing but should help to keep marketing automation vendors on their toes.

I commented last month that more Web content management system (CMS) vendors are adding marketing automation features. One of my examples was SiteCore, so I can’t point to them again as further proof of that assertion. But I did have a good talk last week with SiteCore VP Marketing Darren Guarnaccia, who clarified why this is happening and made a strong case for the integrated approach.

For those of you who (like me) are unfamiliar with SiteCore: it is an eight year old provider of Microsoft .NET-based Web content management systems, with over 1,600 mid-to-large sized customers running more than 20,000 dynamic Web sites including Sara Lee, Toshiba, Omni Hotels and Dollar Rent-a-Car/Thrifty. In other words, it is a substantial player in a crowded market.

According to Guarnaccia, the company has seen control over the CMS selection process steadily migrate from IT departments to marketers over the past four years. The trend is most pronounced at mid-sized firms, where IT is generally less powerful than at very large companies. During this time, it’s become clear that marketers need features that go beyond editing Web pages, to helping them do a better job of understanding and reacting to customers. SiteCore describes this as closing an “actionability chasm” between analytics and execution.

The chasm is created by the traditional approach of using analytical systems (often sold as externally hosted services) that are separate from the underlying content management system. Capturing detailed information with such systems involves much more than adding one code snippet to a shared page template. At a minimum, each page must be given its own ID and, more realistically, pages must be given multiple tags to facilitate analysis. Companies running several separate analytical systems may need several sets of tags.

The practical result of such an arrangement is that marketers and their Web teams quickly fall behind in their tagging, and end up with incomplete and unreliable analytics. Building analytics into the CMS allows users to avoid some tags altogether and makes it easier to reuse the rest. Integrated analytics also allow the system to track visitors with first-party cookies, which are less likely to be erased than the third-party cookies used by some stand-alone analytical products.

Integration also makes it easier to coordinate activities such as personalization, behavior-based targeting, and tests. The logic for these potentially overlapping functions can be all managed as part of one Web page definition, rather than separately.

For example, SiteCore supports lead scoring by assigning content scores (for technology, marketing, sales, pricing, tech support, etc.) to each Web page or, potentially, to components within a page. The lead score for each visitor’s interest in each category is the sum of the category scores for all the pages that person has visited. The same information can be used to identify the visitor’s business role or assign a persona.

The advantage of page-based scoring is that the scores adjust automatically to new Web contents. Otherwise, the company must rely on one team of workers to add new content and a separate team to incorporate the new content into the scoring rules.

Guarnaccia offered a Web site marketing maturity model that started with traffic statistics and extended to user experience statistics, content profiling, segmentation, conversion tracking, campaign management, sales enablement (using the IP address to identify visitor location and company), testing and optimization, and real time personalization. He said these are all present at no extra charge within the latest version of SiteCore, which was released at the end of June as the SiteCore Online Marketing Suite. An online demonstration confirmed they are indeed available, and at an impressively high level of sophistication.

SiteCore organizes these features around the individual Web pages. Attributes for each page include interest scores already mentioned, plus goals and other events that are logged to the visitor's history profile when the page is viewed. A page view can also trigger actions including test execution, personalization, data updates, parameter setting, sales alerts, and calls to external scripts. The system also captures the usual Web analytics data such as traffic volume, referring and exit pages, and on-site search terms. It can also use the visitor history to play back the sequence of pages viewed during a Web session.

This page-centric view of the world makes sense for a CMS vendor, but it's a pretty big switch from the campaign-centric view of most marketers and most marketing automation systems. In fact, the biggest objection to CMS-based marketing automation may be that it assumes everything is centered on the Web site.

Guarnaccia didn’t see it that way. He suggested that marketers will use separate systems for each channel. I think that SiteCore’s main goal is to replace stand-alone Web analytics and personalization systems, not to provide cross-channel marketing automation. Still, the company does plan move beyond Web marketing by adding outbound email campaigns in a few weeks. It will also support emails triggered by Web page visits.

My own take is that building analytics into the CMS makes sense, but I doubt marketers want new silos in the form of channel-specific marketing systems. If so, SiteCore’s marketing features will be most appealing to companies that interact with customers primarily through the Web. For those firms, the Web site could reasonably be the core customer management system. Systems for other channels then would connect with the Web database in the same way that auxiliary channels are (sometimes) now integrated with a central Customer Relationship Management (CRM) system.

The CMS-based model relates to other industry trends: integration between marketing automation and sales systems, and, more broadly, absorption of marketing automation into operational systems. For companies where the Web site is the primarily operational system, these are exactly the same thing. For companies where the Web and CRM are both important independent systems, marketing automation is an ally they may both wish to annex.

For now, though, SiteCore is working to cooperate with CRM rather than replace it. The system can scan IP address registries to identify a visitor’s geographic location and company, and then use the results to route leads, alert sales people, and aggregate data at the company level. SiteCore has built data synchronization for Salesforce.com and Microsoft Dynamics and will add other systems as clients request them. If further integration is needed, other systems can access the SiteCore databases directly.

This access is simplified because SiteCore is traditional on-premise software, not an externally hosted service. Pricing is based on the number of concurrent users and servers. A single server license starts as low as $15,000, although an average installation runs about $90,000. The vendor provides several days of classes, including about two days for marketing users.

The reasons for CMS vendors to add marketing automation functions are clear: to differentiate themselves and to capture budget now spent on analytical and marketing systems. It makes perfect sense for companies selecting a new CMS to prefer integrated analytics, and in some cases to add integrated marketing automation. It’s less likely that companies will discard an otherwise-satisfactory existing CMS just to get these features. But the normal replacement cycle runs three to five years, according to Guarnaccia, so it won't be long before most marketers find themselves with integrated analytical features and new marketing automation options. Even if marketers don't use all of those features, the possibility will encourage stand-alone marketing automation vendors to improve their own products to keep pace.

Operational Systems Should Be Designed with Testing in Mind

A direct marketing client pointed out to me recently that it can be very difficult to set up tests in operational systems.

There is no small irony in this. Direct marketers have always prided themselves on their ability to test what they do, in pointed contrast to the barely measurable results of conventional media. But his point was well taken. Although it’s fairly easy to set up tests for direct marketing promotions, testing customer treatments delivered through operational systems such as order processing is much more difficult. Those systems are designed with the assumption that all customers are treated the same. Forcing them to treat selected customers differently can require significant contortions.

This naturally led me to wonder what an operational system would look like if it had been designed from the start with testing in mind. A bit more reflection led me to the multivariate testing systems I have been looking at recently—Optimost, Offermatica, Memetrics, SiteSpect, Vertster. These take control of all customer treatments (within a limited domain), and therefore make delivering a test message no harder or easier than delivering a default message. If we treated them as a template for generic customer treatment systems, which functions would we copy?

I see several:

- segmentation capabilities, which can select customers for particular tests (or for customized treatment in general). You might generalize this further to include business rules and statistical models that determine exactly which treatments are applied to which customers: when you think about it, segmentation is just a special type of business rule.

- customer profiles, which make available all the information needed for segmentation/rules and hold tags that identify customers already tagged for a particular test

- content management features, which make existing content available to apply in tests. Some systems provide content creation as well, although I think this is a separate specialty that should usually remain external.

- test design functions, that help users create correctly-structured tests and then link them to the segmentation rules, data profiles and content needed to execute them. These design functions also include parameters such as date ranges, preview features for checking that they are set up correctly, workflow for approvals, and similar administrative features.

- reporting and analysis, so users can easily read test results, understand their implications, and use the knowledge effectively.

I don’t mean to suggest that existing multivariate testing systems should replace operational systems. The testing systems are mostly limited to Web sites and control only a few portions of a few pages within those sites. They sit on top of general purpose Web platforms which handle many other site capabilities. Using the testing systems to control all pages on a site without an underlying platform would be difficult if not impossible, and in any case isn’t what the testing systems are designed for.

Rather, my point is that developers of operational systems should use the testing products as models for how to finely control each customer experience, whether for testing or simply to tailor treatments to individual customer needs. Starting their design from the perspective of creating a powerful testing environment should enable them to understand more clearly what is needed to build a comprehensive solution, rather than trying to bolt on particular capabilities without grasping the underlying connections.

Proving the Value of Site Optimization

Eric’s comment on yesterday’s post, to the effect that “There shouldn’t be much debate here. Both full and fractional designs have their place in the testing cycle” is a useful reminder that it’s easy to get distracted by technical details and miss the larger perspective of the value provided by testing systems. This in turn raises the question posed implicitly by Friday’s post and Demi’s comment, of why so few companies have actually adopted these systems despite the proven benefits.

My personal theory is it has less to do with a reluctance to be measured than a lack of time and skills to conduct the testing itself. You can outsource the skills part: most if not all of the site testing vendors have staff to do this for you. But time is harder to come by. I suspect that most Web teams are struggling to keep up with demands for operational changes, such as accommodating new features, products and promotions. Optimization simply takes a lower priority.

(I’m tempted to add that optimization implies a relatively stable platform, whereas things are constantly changing on most sites. But plenty of areas, such as landing pages and check out processes, are usually stable enough that optimization is possible.)

Time can be expanded by adding more staff, either in-house or outsourced. This comes down to a question of money. Measuring the financial value of optimization comes back to last Wednesday's post on the credibility of marketing metrics.

Most optimization tests seem to focus on simple goals such as conversion rates, which have the advantage of being easy to measure but don’t capture the full value of an improvement. As I’ve argued many times in this blog, that value is properly defined as change in lifetime value. Calculating this is difficult and convincing others to accept the result is harder still. Marketing analysts therefore shy away from the problem unless pushed to engage it by senior management. The senior managers themselves will not be willing to invest the necessary resources unless they believe there is some benefit.

This is a chicken-and-egg problem, since the benefit from lifetime value analysis comes from shifting resources into more productive investments, but the only way to demonstrate this is possible is to do the lifetime value calculations in the first place. The obstacle is not insurmountable, however. One-off projects can illustrate the scope of the opportunity without investing in a permanent, all-encompassing LTV system. The series of “One Big Button” posts culminating last Monday described some approaches to this sort of analysis.

Which brings us back to Web site testing. Short term value measures will at best understate the benefits of an optimization project, and at worst lead to changes that destroy rather than increase long term value. So it makes considerable sense for a site testing trial project to include a pilot LTV estimate. It’s almost certain that the estimated value of the test benefit will be higher when based on LTV than when based on immediate results alone. This higher value can then justify expanded resources for both site testing and LTV.

And you thought last week’s posts were disconnected.

Is Taguchi Good for Multivariate Testing?

I’ve spent a lot of time recently talking to vendors of Web site testing systems. One topic that keeps coming up is whether Taguchi testing—which tests selected combinations of variables and infers the results for untested combinations—is a useful technique for this application. Some vendors use it heavily; some make it available but don’t recommend it; others reject it altogether.

Vendors in the non-Taguchi camp tell me they’ve done tests comparing Taguchi and “full factorial” tests (which test all possible combinations), and gotten different results. Since the main claim of Taguchi is that it finds the optimum combination, this is powerful practical evidence against it. On the theoretical level, the criticism is that Taguchi assumes that there are no interactions among test variables, meaning results for each variable are not affected by the values of other variables, when such interactions are in fact common. Moreover, how would you know whether interactions existed if you didn’t test for them? (Taguchi tests are generally too small to find interactions.)

Taguchi proponents might argue that careful test design can avoid interactions. But the more common justification seems to be that Taguchi makes it possible to test many more alternatives than conventional A/B tests (which change just one item at a time) or full-factorial designs (which need a lot of traffic to get adequate volume for each combination.)

So, the real question is not whether Taguchi ignores interactions (it does), but whether Taguchi leads to better results more quickly. This is possible even if those results not optimal, because Taguchi lets users test a wider variety of options with a given amount of traffic. I’m guessing Taguchi does help, at least for sites without huge visitor volumes.

Incidentally, I tried to do a quick classification of which vendors favor Taguchi. But it’s not so simple, because even vendors who prefer other methods still offer Taguchi as an option. And some alternative methods can be seen more as refinements of Taguchi than total rejections of it. So I think I’ll avoid naming names just now, and let the vendors speak for themselves. (Vendors to check: Offermatica, Optimost, Memetrics, SiteSpect, Vertster.)

SiteSpect Does Web Tests without Tags

I had a long and interesting talk yesterday with Larry Epstein at SiteSpect, a vendor of Web site multivariate testing and targeting software. SiteSpect’s primary claim to fame is they manage such tests without inserting any page tags, unlike pretty much all other vendors in this space. Their trick, as I understand it, is to use a proxy server that inserts test changes and captures results between site visitors and a client’s Web server. Users control changes by defining conditions, such as words or values to replace in specified pages, which the system checks for as traffic streams by.

Even though defining complex changes can take a fair amount of technical expertise, users with appropriate skills can make it happen without modifying the underlying pages. This frees marketers from reliance on the technical team that manages the site. It also frees the process from Javascript (which is inside most page tags), which doesn’t always execute correctly and adds some time to page processing.

This is an intriguing approach, but I haven’t decided what I think of it. Tagging individual pages or even specific regions within each page is clearly work, but it’s by far the most widely used approach. This might mean that most users find it acceptable or it might be the reason relatively few people use such systems. (Or both.) There is also an argument that requiring tags on every page means you get incomplete results when someone occasionally leaves one out by mistake. But I think this applies more to site analytics than testing. With testing, the number of tags is limited and they should be inserted with surgical precision. Therefore, inadvertent error should not be an issue and the technical people should simply do the insertions as part of their job.

I’m kidding, of course. If there’s one thing I’ve learned from years of working with marketing systems, it’s that marketers never want to rely on technical people for anything—and the technical people heartily agree that marketers should do as much as possible for themselves. There are very sound, practical reasons for this that boil down to the time and effort required to accurately transfer requests from marketers to technologists. If the marketers can do the work themselves, these very substantial costs can be avoided.

This holds true even when significant technical skills are still required. Setting up complex marketing campaigns, for example, can be almost as much work in campaign management software as when programmers had to do it. Most companies with such software therefore end up with experts in their marketing departments to do the setup. The difference between programmers and these campaign management super users isn’t really so much their level of technical skill, as it is that the super users are part of the marketing department. This makes them both more familiar with marketers’ needs and more responsive to their requests.

Framing the issue this way puts SiteSpect’s case in a different light. Does SiteSpect really give marketers more control over testing and segmentation than other products? Compared with products where vendor professional services staff sets up the tests, the answer is yes. (Although relying on vendor staff may be more like relying on an internal super user than a corporate IT department.) But most of the testing products do provide marketing users with substantial capabilities once the initial tagging is complete. So I’d say the practical advantage for SiteSpect is relatively small.

But I’ll give the last word to SiteSpect. Larry told me they have picked up large new clients specifically because those companies did find working with tag-based testing systems too cumbersome. So perhaps there are advantages I haven’t seen, or perhaps there are particular situations where SiteSpect’s no-tag approach has special advantages.

Time, and marketing skills, will tell.