Will CDP Buyers Consider Private Clouds as On-Premise Deployment?

Most Customer Data Platforms are Software as a Service products, meaning they run on servers managed by the vendor. But some clients prefer to keep their data in-house. So before releasing the CDP Vendor Comparison report – now available here – I added a line for on-premises deployment.

This seemed like a perfect fit: a clear yes/no item that some buyers consider essential. But it turned out to raise several issues:

- on-premises vs on-premise. I originally used “on-premise”, which is how the term is typically rendered. One of the commenters noted this is a common error. A bit of research showed it’s been a topic of discussion but on-premise is now more widely used relating to computer systems.  On-premises actually sounds a bit pedantic to me, but I’m using it to avoid annoying people who care. (Interestingly, no one seems too concerned about whether to use the hyphen. I guess even grammar geeks pick their battles.)

- private clouds. Several vendors argued that on-premises is an old-fashioned concept that’s largely been replaced by private clouds as a solution for companies that want to retain direct control over their systems and data. This resonated: I recalled seeing this survey from 451 Research showing that conventional on-premises [they actually used “on-premise”] deployments now account for just one-quarter of enterprise applications and the share is shrinking.

Percentage of Applications by Venue:
24% Conventional (on-premise, non-cloud)
18% on-premise private cloud
15% hosted private cloud
14% public cloud
13% off-premise non-cloud
Source: 451 Research, Strategy Briefing: Success Factors for Managing Hybrid IT, 2017

My initial interpretation of this was the on-premises private clouds meet the same goals as conventional on-premises deployments, in the sense of giving the company’s IT department complete control. But in discussions with CDP vendors, it turned out that they weren’t necessarily differentiating between on-premises private clouds and off-premise private clouds, which might be running on private servers (think: Rackspace) or as “virtual private servers” on public clouds (think: Amazon Web Services). Clearly there are different degrees of control involved in each of these and companies that want an on-premises solution probably have their limits on how far they’ll go in the private cloud direction.

- public clouds. One vendor speculated that most remaining conventional deployments are old systems that can’t be migrated to the cloud. The implication was that buyers who could run a CDP in the cloud would gladly do this instead of insisting on an on-premises configuration. This survey from Denodo suggested otherwise: while it found that 77% of respondents were using a public cloud and 50% were using a virtual private cloud, it also found that 68% are NOT storing “sensitive data” in the public cloud. Presumably the customer data in a CDP qualifies as sensitive. I don't know whether the respondents would consider a “virtual private cloud” as part of the public cloud.  But I think it’s reasonable to assume that a considerable number of buyers reject external servers of any sort as an option for CDP deployment, and that “on-premises” (including on-premises private clouds) is a reasonable term to describe their preferred configuration.

Quantivo Offers High-Volume Customer Analytics at a Modest Price

These are exciting times in the world of analytical systems. The Web has created new demands to handle unprecedented data volumes and semi-structured data. Cloud-based deployment offers near-infinite hardware scalability and flexibility.  Acquisitions by enterprise software giants have opened opportunities for smaller, more nimble alternatives. The result has been an explosion of companies using new techniques for managing and analyzing huge data volumes. Many, including Vertica, Aster Data, Greenplum, and Netezza, have also been assimilated by enterprise vendors. Others, including Paraccel, Kognitio, SAND Technology, and 1010data, have grown steadily while remaining independent.

Quantivo is part of this latest technical flowering. Its core is a data structure that sits somewhere between columnar – now the most common approach for analytical processing – and hierarchical indexes. Specifically, Quantivo identifies unique pairs of values within incoming records and stores each pair only once. It then builds new pairs by identifying unique combinations of each pair with another element (which might itself be previously-identified pair). For example, the first analysis might find all transactions that involve a specific product on a single date. The second pass might find all transactions with that product/date pair that were made by a given customer. The process repeats, working up a hierarchy of combinations. The system also tracks the number of times each pair occurs and the specific records involved, so the full detail of the original data can always be reconstructed. The system doesn’t build pairs for all possible combinations of data elements, although users can define several hierarchies if they want the same element to be part of several pairs. Indexes also allow analysis across pairs that are not directly built into the data.

Quantivo doesn’t hide the details of its approach, but it doesn’t talk much about them, either. This likely reflects a (correct) judgment that users will care more about the system’s benefits than how it works. Those benefits include data compression (typically 10% of the original volume), fast response, high scalability, flexible schemas, handling unstructured data, and efficient processing of queries that cause problems in standard SQL. The choice of hierarchy also inherently organizes data around “concepts”, which can be different from the physical structures of the inputs. Queries are relatively simple because users see only a flat list of data elements; relationships are managed automatically, behind the scenes.

Yes there are trade-offs. The data must be processed during the load – a computation-intensive task that takes longer than creating a simple columnar database. The schema must be designed, which requires some technical expertise and can lead to slower response for queries outside the expected paths. Non-SQL queries require a Quantivo-built user interface, meaning users cannot stick with their familiar SQL-based business intelligence tools.

Some of these issues are mitigated by Quantivo’s other major differentiator: it was designed from the start as a true multi-tenant cloud-based solution.  This means it can easily spread its workload across multiple servers and replicate data as needed to support higher data volumes (billions of rows), reduce load times (typically one to two hours per day for daily updates), and improve response time. Support for multiple hierarchies and queries across hierarchies also minimize the price of making a bad decision during initial schema design, since even unplanned queries will execute reasonably efficiently. Designing a schema is relatively simple: in fact, Quantivo plans to add self-service data provisioning, including schema design, in the near future.

The Quantivo user interface is purposely designed to look like other business intelligence tools: users get a list of measures and dimensions, which they drag into place to create pivot tables. They can create filters, define calculated values, add summary levels, and drill down to details. There are simple graphics such as bar and pie charts, but no advanced visualization.

The unique power of the system lies within the filters, which can select data that’s outside the standard dimensions. For example, a filter could select all transactions for customers who purchased a specific product – the type of market basket analysis that’s hard with traditional SQL queries. The system also calculates “association metrics” which compare the data across two groups, such as products in the baskets including product A vs. products in baskets including product B.

Quantivo pricing is based on data volume and complexity. A typical client pays $40,000 to $50,000 per year for 100 million to one billion rows of data, making the system remarkably affordable for a product of its type.  Most clients are using the system for customer data analytics, and the vendor has created connectors for SAP, Salesforce.com, Marketo, Responsys, ExactTarget, Omniture, Google Analytics, IBM, Microsoft, and NCR systems. Quantivo launched as a cloud-based service in 2008 and has an undisclosed number of paying clients.

Cloud-Based QlikView Still Isn't Available as a Service

Summary: Pay-as-you-go pricing would make QlikView easier to buy, but the company doesn't offer this option. To make a stronger business case for the purchase, include the value of shifting work from IT to business users, and of producing results faster.

Last week’s post about QlikView 9.0 prompted an inquiry from a manager who has been trying for a year to convince his company to consider the product. Having run into this issue many times, I easily felt his pain and we speculated a bit on what might help things along.

One obvious tactic would be to purchase QlikView on a pay-as-you-go basis, presumably cloud-based. But a quick check with QlikView confirmed that they don’t allow this and have no plans to change.

The closest they come is to let their partners offer QlikView-based applications as a service. For example, they pointed me to SportsDataHub, which lets users analyze football statistics for $40 per year. But the key point about this and similar QlikView services is that you can only access data loaded by the partner. You can't define and load your own data sources directly. At best, you might be able to create your own reports based on the loaded data. (See QlikTech Marketing SVP Anthony Deighton's comment on this post for a little more on the subject.)

I don’t understand QliiView’s reluctance to adopt a Software-as-a-Service model. It has proven viable for many other software companies, including other business intelligence vendors. To me, it seems a natural extension of the company’s “seeing is believing” sales approach as well as a good way to sidestep the barriers raised by corporate IT.

In fact, QlikView’s tremendous ease-of-use makes it an excellent fit for the SaaS model, because business users can deploy it for themselves with minimal technical support. In our conversation last week, QlikTech's Deighton said the majority of clients already implement the system without purchasing any external services. If there was ever a piece of software suited to SaaS, this is it.

Be that as it may. The lack of a proper SaaS offering left my correspondent with several avenues to pursue:

- find a QlikView partner who would build an appropriate application and sell it to him on a services basis. This doesn’t seem very plausible because he probably won’t be able to commit enough funding to make the project worthwhile for the partner. I mean, if he had that much money, he could just buy the software outright in the first place.

- use an alternative system that costs less. Yes, QlikView is unique and wonderful, but products from ADVIZOR Solutions, Lyzasoft, Tableau Software and TIBCO Spotfire offer some of the same advantages at a much lower entry price. Again, this is far from ideal, and it might not work at all because I didn’t explore precisely which aspect of QlikView my correspondent found attractive. Still, it’s better than nothing.

(Vaguely related aside: today, people often cite author Jim Collins’ phrase “good is the enemy of great” as a reason to avoid compromise. Previously, I was more likely to see Voltaire’s “the best is the enemy of the good,” which means that compromise is better than nothing. I’m sure this reversal says something important about our society, although I can’t say what. You're welcome.)

- Find a way to sell QlikView internally. Of course, my correspondent had already been trying, so his question was whether I had any new ideas for how. This actually prompted some very deep thinking over the weekend, which will show up in my Information Management magazine column over the next several months. To summarize four pages in 100 words, there are two approaches to consider:

- do a cost of ownership analysis showing the savings from letting business users perform tasks currently done by IT. Traditional cost analysis compares the time it takes IT to do the work with one tool vs. another. This hides rather than highlights the advantages of QlikView and similar products.

- do a “time to result” analysis that measures the time spent waiting for IT to deliver solutions through multiple iterations. This applies to many analytical databases, not just QlikView, because their flexibility reduces the time spent building conventional BI structures like star schemas and data cubes.

Perhaps one of these will work. I hope so, because we could all benefit from finding ways to take advantage of what new technologies like QlikView have to offer.

Webinar April 2: Does On-Demand Business Intelligence Make Sense?

I just realized I hadn't mentioned this upcoming Webinar in the blog. It's on April 2 at 2 p.m. Eastern.

I'll be talking about whether and when to use on-demand business intelligence. Webinar is sponsored by Birst, an on-demand BI vendor. (I guess that eliminates most of the suspense about how I'll answer to the title question.) Birst co-founder Paul Staelin will also present, and of course we'll take questions.

Please register here.