Wednesday, June 08, 2011

Coremetrics Offers a Foggy View of Lifecycle Analysis

I stumbled over an Adexchanger interview yesterday with John Squire, the Chief Strategy Officer of IBM Coremetrics. It first caught my eye because the headline read “IBM’s Vision for the Marketer”, which is always a topic of interest. Then I noticed it was touting new reporting feature called Coremetrics Lifecycle, which the company describes as “the industry’s first application geared to enable online marketers to track and understand how customers progress through long-term conversion lifecycles.”

This was intriguing. On one hand, I’ve seen plenty of systems that track customers through the buying process, including Eloqua, Marketo, Leadformix, Clear Saleing, C3 Metrics, and Encore Media Metrics. So the claim to be first is questionable. But, on the other hand, seeing another vendor offer this sort of analysis reinforces the importance of the concept.

But a closer look at Lifecycle itself was disappointing. The product does allow tracking of individual Web site visitors over time, which is the foundation of lifecycle analysis. But, in my opinion, a lifecycle tracking system reports on movement of customers across stages within the lifecycle. That is, it shows conversions from one stage to the next. This implies reports that show the previous stages of customers who enter a new stage (“where they came from”), and show the destinations of customers who leave a stage (“where they went”). These are typically represented as a matrix showing all combinations of previous and current stages, or a flow chart that highlights the most common before-and-after pairs.

Lifecycle does none of this. Rather, it lets users define any number of segmentation schemes and count the number of customers in each segment. It does report how many customers entered each segment during a specified time period, but not where they came from. In fact, there is no requirement for a logical progression from one segment to the next, which to me is what a lifecycle implies.

Lifecycle has some other useful features. It can report on the most common marketing treatments received by people who moved into a segment, giving some insight into treatment effectiveness. It calculates the average number of days and Web sessions that customers spend in a segment, which is a limited velocity measure. It also lets users select segment members and send them messages through Coremetrics’ products for email, display ad retargeting, and Web site personalization, although it's not clear the process can be automated.

But a proper lifecycle analysis tool would go much further. It would calculate the end-to-end completion rates, show the drop-off from one stage to the next, estimate the incremental impact of specific treatments, project future segment counts, and show changes in these measures over time. So while I’m pleased that Coremetrics is promoting the concept of lifecycle analysis, I’m disappointed that its product doesn’t deliver a real lifecycle measurement solution.

Addendum - June 19, 2011

After the original post and IBM's comment on it, I reviewed the Lifecycle product with the Coremetrics team. This uncovered no substantive errors in the original post, although a couple of points could have been stated more clearly.

- the system supports two types of lifecycles, one requiring that customers progress through the stages in sequence and one that does not. Users specify the type when they set up a new lifecycle. In both cases, the stages are defined by selection rules created by the user.

- there is a limit of six stages per lifeycle.

- for sequential lifecycles, the system will warn the user if the selection rules are not inherently sequential. (An inherently sequential rule might be based on the number of purchases made; you can't make three purchases without having previously made two. Other stage definitions, such as downloading a white paper or leaving a comment, might come in any order and, therefore, are not inherently sequential.)

- in a sequential lifecycle, the system will not allow customers to advance outside of sequence even if the definitions would allow it. Nor does it report on customers who would qualify for a later stage but cannot reach it because they didn't qualify for a previous one.

- the system's primary report shows the number of customers within each stage during a specified date range. Think of this as an inventory. A "Migrator" report shows how many customers entered their current stage during the report period: for example, there were 500 customers in stage 3, of whom 200 first entered stage 3 during this period. This gives some sense of movement, but it's not the classic funnel analysis showing the percentage of customers in each stage who eventually move to the next stage.

- users can run the standard reports against "segments", which could be defined as anything including a cohort of customers who entered the system during a specified time period. A Lifecycle inventory report for such a cohort would show how many customers reached each stage and got no further. This is the information needed to build a classic funnel analysis, although users would have to extract the data and manipulate it to produce an actual funnel report. This would be done outside of Coremetrics, because there is no end-user report writer.

- reports show the average number of days and Web sessions it takes customers to reach each stage (i.e., since they first entered the system), not the number of days and sessions spent in each stage, as I wrote originally.

- users do have the option to create a recurring process that automatically selects customers in a particular stage and sends them an email or other message. The system could apply a few rules to this process, such as eliminating people who had been selected previously. But more sophisticated controls would be handled outside of Coremetrics, in the message delivery system.

- the system can profile customers in each stage against many attributes (products purchased, geography, social network membership, etc.) in addition to marketing contents received. But, as I wrote originally, the reporting only shows the percentage of customers in each stage who match a particular attribute: this is far from measuring influence, for reasons I'll explain in a future post.

- we confirmed that the system doesn't do projections of future inventory counts, report on out-of-sequence customer movements, or allow customers to migrate backwards into lower-ranked stages (as might happen if stages were based on recency or ratios).

I'm happy to have clarified these matters but none of this changes my original assessment: Lifecycle is a useful product that falls far short of serious life stage analysis.


bbk937 said...

Thank you for the review of our product. We would like to offer a few clarifications on what you have written:

- IBM Coremetrics Lifecycle does report on the the movement of users across lifecycle stages (or milestones) including the opportunity to view specific registrants and/or cookies moving through each stage.
- Default Lifecycle reports do require a strict progression. A visitor must have achieved a preceding milestone stage before moving to the next. Thus, visitors moving into a new stage must be coming from the preceding one. That said, the product also offers a flexible option to consider milestone achievement independent of order which becomes quite useful for targeting actions, isolated analysis and lifecycle scenarios that may not be so structured.
-The product allows for much more than simply diving into the most common marketing treatments that advance customers to new lifecycle stages. It also offers an understanding of many types of correlated analysis. For example, users can evaluate which content, products, site promotions, etc are most influential on progression as well as other factors such as the impact of mobile devices, social networks, geography, videos or other.

- We appreciate your validation of the concept. This is the first release of the product and aimed to address key client use cases. We agree that it can be much more and have big plans for growing the product and tying it to even more automated action.

IBM Coremetrics Product Management

David Raab said...

Thanks for the clarifications. Your comments raise a number of additional questions. For example, how do you ensure that the segment definitions create a strict progression? Is movement through stages reported for cohorts, and if so, how are these created? Is anything other than simple correlation used to measure "influence"? As we have already discussed in private, the best approach is to set up a briefing so we can go through all this in detail and I can present a more complete report.