Demand Generation Usability Scores - Part 2

Usability Items for Simple Marketing Programs
(note: this is a slightly revised version of the original post, reflecting vendor feedback.)

Yesterday's post described the background of this usability scoring project and gave scores for several items that apply to both simple and complex marketing programs. This post will continue with the scores for items that apply to usability for simple campaigns.

Build a campaign as a list of stages. Users can build a simple campaign by defining a linear sequence of stages. As discussed in last Friday's post , I see this is one of two truly key features for making simple campaigns easy to build. The primary alternative, laying out a campaign on a Visio-style flow chart, is harder for most marketers to grasp.

In case you're wondering, the practical difference between a list and a flow chart is that stages in a flow chart are linked by separate decision icons, while stages in a list are connected directly. Many list interfaces allow decision rules within each stage, so there's not necessarily a functional difference. But embedding the rules gives a much cleaner, simpler view of the campaign flow. Of course, this only works for simple flows, because any complex branching is hidden and would soon become unmanageable.

Build a campaign as a list of stages
Eloqua	Manticore Technology	Market2Lead	Marketbright	Marketo	Neolane	Silverpop Engage B2B
0	0	1	1	1	0	1

Build linear campaigns outside of a larger campaign structure. The linear campaigns described above can be created independently, rather than as part of a larger structure involving multiple linear campaigns. This is the other key to simple campaign usability because it allows the linear campaigns to be added and removed directly. Otherwise, they must be fit into a larger structure which adds complexity that is unnecessary for a simple marketing program. The flow of leads among these independent campaigns may be managed either through explicit routing (i.e., one campaign directly sends leads to another campaign) or implicitly (i.e., each campaign has its own entry conditions, and leads flow to whichever campaigns they are qualified for.)

Build linear campaigns outside of a larger campaign structure
Eloqua	Manticore Technology	Market2Lead	Marketbright	Marketo	Neolane	Silverpop Engage B2B
0	0	1	1	1	0	0

Define campaign schedules only at the campaign level, not for individual steps. Execution schedules are specified for the campaign as a whole, and not separately for individual stages. This reduces setup labor, eliminates an item from the user interface, and makes it easier to understand how the campaign will function. The campaign schedules are often derived from the update schedules of the lists that feed the campaign. Neolane is scored with a half point because it allows stage-level schedules but can be configured to hide the capability.

Define campaign schedules only at the campaign level, not for individual steps.
Eloqua	Manticore Technology	Market2Lead	Marketbright	Marketo	Neolane	Silverpop Engage B2B
0	1	1	1	1	0.5	0

Decision rules are built with functions for specific data types, such as ‘X Web site visits in past Y days'. This contrasts with generic rule building interfaces, which require users to know which field holds a particular type of data and may require complex specifications for calculations. Every system scored here meets this requirement in one way or another, but some other demand generation products do not.

Decision rules are built with functions for specific data types
Eloqua	Manticore Technology	Market2Lead	Marketbright	Marketo	Neolane	Silverpop Engage B2B
1	1	1	1	1	1	1

Campaign stages send leads directly to sales. A stage within the flow for each campaign specifies when the system will transfer qualified leads to sales. This is easier to understand than having a separate function that scans for transfer opportunities independently of campaigns. Silverpop gets a half point because it has rules within each campaign that scan for opportunities.

Campaign stages send leads directly to sales
Eloqua	Manticore Technology	Market2Lead	Marketbright	Marketo	Neolane	Silverpop Engage B2B
1	1	0	1	1	1	0.5

Training time during implementation. This reflects the amount of training the vendors say they provide for end-users during system implementation. It is included here on the assumption that less training time indicates an easy-to-use system. Vendors providing less than one day of training are scored with a 1; those providing more than one day are scored with a 0. Where vendors offer different versions of their system, the scores are based on training for the simplest version with complete functionality. In practice, of course, vendors offer different training packages which are tailored to the client's situation.

Training time during implementation
Eloqua	Manticore Technology	Market2Lead	Marketbright	Marketo	Neolane	Silverpop Engage B2B
0	1	0	1	1	0	1

Total for Simple Items

Total for Simple Items
Eloqua	Manticore Technology	Market2Lead	Marketbright	Marketo	Neolane	Silverpop Engage B2B
2	4	4	6	6	2.5	3.5

Combined Score for Simple Programs

The total usability score for simple marketing programs is the sum of simple items listed in this post plus the shared items listed yesterday. (Figures for Eloqua were changed from the original values based on information provided by the company.) It's no surprise that Marketo has the highest score. The three-way tie for second place among Manticore Technology, Market2Lead and Marketbright is more intriguing. The rating for Market2Lead in particular reflects its just-released new interface, which goes a long way to simplifying what had previously been a very complicated system.

Combined Score for Simple Programs
	Eloqua	Manticore Technology	Market2Lead	Marketbright	Marketo	Neolane	Silverpop Engage B2B
Shared	6	6	6	4	6	6	5.5
Simple	2	4	4	6	6	2.5	3.5
Combined	8	10	10	10	12	8.5	9

Tomorrow's post will look at scores for complex campaigns.