Beautiful BABI: SiSense PrismCubed Offers Business Intelligence for Business Analysts

Summary: SiSense PrismCubed offers a reasonable option for a business-analyst business intelligence system. It’s probably a little harder to use than some competitors, but gives a bit more power and flexibility in return.

SiSense PrismCubed, officially launched this past August, is another member of the growing set of business intelligence systems aimed at empowering business analysts to build their own applications. I’ve also written about QlikView and Lyza, and think there are others.

What distinguishes these tools from other business intelligence systems is they let non-technical users manipulate source data in more sophisticated ways than a spreadsheet or report writer. Specifically, data from several sources can be merged on a common key, filtered, aggregated and processed through complex formulas.

This sort of manipulation has traditionally required SQL programmers, OLAP cube designers, or similar technical experts. Allowing business analysts to do it without having to learn deep technical skills is precisely what lets them build applications with minimal external assistance. (I say "minimal" because technical staff must still handle connections to the source data.)

These systems also provide report creation and distribution. But unlike business-analyst data manipulation, those capabilities are also found in other business intelligence products.

You’ll note that my definition does NOT specify a particular database technology, such as in-memory or columnar, that the data is updated in real time, that the system is targeted at mid-sized businesses, or that results are distributed pervasively through the organization. Those have all been proposed as ways to classify business intelligence systems, and several of the products in my business-analyst business intelligence (BABI--how cute!) category fall into one or another such group. But I think it’s a mistake to focus on those other features because they don’t get at real value provided by these tools, which is the flowering of applications made possible when business analysts can create them independently.

Now that I've defined a new type of application, complete with the all-important acronym, the next step is defining an evaluation framework to help compare the competitors. I’d like to claim I do this through deep research and brilliant insights into user needs, but, in fact, I generally start with the features in the existing systems. This runs the risk of missing some critical requirement that no vendor has yet uncovered, but it saves a ton of work. And I can still argue that I’m piggybacking on the vendors’ own deep research and insights as embodied in their products.

In any event, a starter set of review criteria for BABI systems (sorry, but I find the acronym irresistible) would include:

- combine data from multiple, heterogeneous sources (relational databases, CSV files, Excel tables, etc.)

- allow non-technical users to define processing flows to manipulate the data (merge, filter, aggregate, calculate)

- present the manipulated data in a structure that’s suitable for reporting and visualization

- allow non-technical users to create applications including reports, visualizations, and (optionally) additional functions such as data refresh and export

- allow other users to view (and optionally interact with) the applications

- meet reasonable performance standards for data load, storage, response time and scalability

- use appropriate technology (actually, I don’t care if the thing is powered by hamsters. But understanding the underlying technology helps to predict where problems might arise.)

- affordable pricing (not exactly a criterion, but important nevertheless)

Obviously these criteria could be much more detailed, and no doubt they will grow over time. But for now, they provide a useful way to look at PrismCubed.

1. Combine data from multiple sources: PrismCubed provides a wizard to connect with different data sources, including SQL Server, Oracle, CSV files, Excel and Amazon S3 logs (which earns them extra coolness points). The system can read the database schemas directly, saving users the need to define basic data structures. Users have the option modify structures if they desire. A connection can be live (i.e., the source is requeried each time a report is run) or reloaded on demand from within a completed application. This provides real-time data access, which isn’t always available in business intelligence systems. The system can also reload data automatically on a user-specified schedule.

2. Allow non-technical users to manipulate source data: PrismCubed does a particularly good job here. At a basic level, users can write complex formulas to add derived fields to a table during the import process. More important, a drag-and-drop interface lets them build complex visual processing flows from standard icons including data definition, filtering, inclusion or exclusion, unions, and top or bottom selects. These flows can combine multiple data sources and include branches that generate separate output sets that are all available to use in applications.

3. Present the manipulated data for reporting: the system automatically classifies input data as dimensions (text, dates, etc.) and measures (numbers which can be aggregated). Users can override the system’s assignments and can add new dimension fields during the data load. They can create derived measures at any time. Once the load is complete, the system presents the dimensions and measures in an “ElastiCube” available for reports and other applications.

4. Create reports and other applications: the system provides a remarkably rich development environment. Users build applications by dropping different types of objects (which the vendor calls widgets) onto dashboard pages. Widgets can make selections; display data in pivot tables, charts, calendars, and images; and execute actions including refresh data, jump to different pages, query external data sources, edit data, and export to Excel. A dashboard can have multiple pages.

The primary reporting widget is the pivot table, which itself is built by dragging dimensions into rows and columns, and the measures into cell values. Users can apply filters to widgets, such as selecting the top 10 values for a dimension. These filters can be static (a fixed list) or dynamic (reselected each time the dashboard is updated). PrismCubed also provides special features for time series calculations such as period-to-period growth and differences. That's a nice touch, because those can be quite difficult to define with conventional reporting systems.

Reporting widgets can be connected to the ElastiCube dimensions and measures or directly to SQL data sources. Users can also specify whether selections made in one widget affect the data displayed in other widgets. There are actually three options here, including complete independence, direct links from one widget to another, and global impact on other widgets. This gives more flexibility than systems that automatically apply global selections, but does force users to do more work in specifying which approach they want.

Widgets, filters and other components can be stored in a central repository and reused across applications.

5. Share applications: Users can export dashboard contents to Excel tables or can copy an entire dashboard as a static PDF. Applications, including underlying ElastiCubes, can be copied and run on another user’s PC. In addition, a Web server due for release this month (October) will let dashboard creators publish their dashboards to a central server, where other users will be able to access and modify them. The server will provide fine-grained control over what different users are allowed to change.

6. Scalability and Performance: SiSense has tested the PrismCubed engine on multiple terabytes of data. It cited one client who loaded 30 million telephone call detail records in 30 to 90 minutes. Loaded data usually takes somewhat less disk space than the original source. The system currently requires a complete reload to add new data to an existing ElastiCube, although the vendor plans to add incremental appends by the end of November. Once the data is loaded, reports within applications usually update in seconds.

7. Technology: PrismCubed stores data in a columnar data structure. It also stores a dimension map for each column, but doesn’t preaggregate the data along the dimensions. As with other columnar databases, this avoids the need for specialized data structures to handle particular queries. When data has not been preloaded into the system, PrismCubed can also run the same query across multiple external data sources.

Although PrismCubed stores the entire ElastiCube on disk, it only loads into memory the columns required for a particular query. This lets it can handle larger data sets than purely in-memory systems without massive hardware. There might be some problems if the selected columns for a query exceeded the system’s available memory.

PrismCubed runs on Windows PCs with the .NET framework installed. On 64 bit systems, this means the amount of potential memory is virtually unlimited. Although PrismCubed itself is new, a previous version of the product using the ElastiCube database engine was launched in September 2008 and has more than 6,000 users.

8. Pricing: PrismCubed is priced on an annual subscription basis, which is unusual for this type of product but common among hosted BI vendors. SisSense offers several versions of PrismCubed, ranging from a free Viewer that can only access dashboards created elsewhere, to a $1,500 per year Professional edition that allows full creation of dashboards and ElastiCubes. There are also a free version (limited to 2,000 rows of data), a $300 per year Personal edition (which can create dashboards but not share them), and a $700 per year Analyzer that can build and share dashboards but not ElastiCubes. Server pricing wasn’t quite set when I spoke with SiSense but will probably be around $3,500 per year per server.

These prices are quite reasonable compared with similar vendors, even considering the recurring annual subscription fees, particularly because the end-user Viewer is free. Price details are published on the vendor’s Web site.