Simba Technologies' CEO Corner

Data Warehousing and Ad Hoc Queries

It is interesting to see how various vendors believe that data warehouses and data warehouse appliances will help when it comes to ad hoc query performance.  A lot of databases are tuned to perform well for certain types of queries.  When the user of the database asks something unexpected, performance can grind to a halt.  Stephen Swoyer at TDWI wrote a very good article entitled "In Depth: Closing the Ad Hoc Query Performance Gap for Good" http://www.tdwi.org/News/display.aspx?ID=9034 about this issue.

Performance of ad hoc queries in a data warehouse or data warehouse appliance is a major issue.  As more data is gathered, more people want to try to glean actionable information from that data.  The better able the users are to glean actionable information, the more successful they will be.  There are many companies in this space including:

- illuminate

- Teradata

- Netezza

- DATAllegro

- Dataupia

- Kognitio

- Sybase

- ParAccel

- InfoBright

- Vertica

And of course, the traditional companies like Oracle, IBM, and Microsoft should not be forgotten as well as SAP with its NetWeaver BI (BW) and BIA offerings.  BI is powerful but to get the most value, you will need a database engine that performs.

Good article on Microsoft's acquisition of DATAllegro...

Stephen Swoyer at TDWI wrote a good piece on Microsoft's acquisition of DATAllegro entitled "Analysis: What's Behind Microsoft's DATAllegro Acquisition?" http://www.tdwi.org/News/display.aspx?ID=9056.  Definitely worth a read.  I too expect Microsoft to say more at their BI Conference which runs from October 6-8 and is located in Seattle http://www.microsoft.com/bi/conference/.  I went to the first conference last year and found it gave good insight into Microsoft's BI directions.  I expect there will be much more this year especially now that PPS is a shipping product.

The "last 18 inches"...

I was reading the article entitled "Visualize This: A Fresh Perspective on Business Intelligence Systems" by Angela Shen-Hsieh http://www.dmreview.com/specialreports/2008_88/10001663-1.html.  Angela comments "I like to refer to this visual approach as “the last 18 inches” - that is, the distance between the computer screen and the human brain. Focusing on this last and most vital link of the data chain is essential to getting value out of the massive investments in IT infrastructure that companies have made. After all, if we can’t get information to our brains in a meaningful way, then we’re just drowning in numbers."  This is indeed a very important piece of the BI puzzle - how to get information from data.  The data warehouse appliance has done a good job of scaling and allowing us to process more and more data.  The big question is how to understand and comprehend all this data?

How will Cloud Computing change BI and Data Warehousing?

This is a question I have been pondering for some time.  How will the Cloud Computing change BI and Data Warehousing?  In theory, it should be much easier to get a system up an running.  This is the same reason why the data warehouse appliance is so compelling.  Give me a box, let me plug it in and load my data, and off I go to do the work I really need to do.  The idea of Cloud Computing is even easier.  This is the CPU power/storage that I need, where do I load my data, and off I go to do the work I really need to do.

I work in the technology industry and sometimes we forget that the users of our software/technology don't really care how it works.  Rather, they care that software/technology enables them to do their jobs.  Cloud Computing holds that great promise.

In his article entitled "Send in the Clouds" http://www.bireview.com/bnews/10001601-1.html, JimEricson talks about how "Cloud computing will impact the data warehousing/BI equation".  I think this is very pertinent.  We have to get BI to more users and along with the data warehousing appliance, cloud computing will help to make BI more pervasive. 

SAP NetWeaver BI (SAP BW) and Microsoft Excel 2007 – MDX Connectivity

I was poking around the SAP Developer Network (http://www.sdn.sap.com) website this weekend and found a presentation entitled "SAP NetWeaver BI 7.0 Native Microsoft Excel 2007 Integration"
(https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/7083c7d3-d1ab-2a10-08ae-c8470148c7c2). Although these slides are dated January 2008, they are worth looking at. Slides 3 and 4 talk about the competitive positioning of Excel 2007 and the business case for Excel vs BEx Analyzer. Slides 5 and 6 show some architecture. In slide 6, I would like to point out that irrespective of how you access NetWeaver BI – ODBO, XMLA, or BAPI – you will hit the same MDX Processor. Many people have asked me if there are advantages of one API vs another and my answer has been that if it is just about the MDX, there is no difference because it is the same MDX Processor that underlies all the different APIs. ODBO and XMLA are industry standard APIs whereas BAPI is an SAP specific API. BAPI does have some slight advantages in that it exposes things that are SAP specific but again, these are minor. When it comes to supporting the MDX Query Language, the SAP MDX Processor has been upgraded to better work with Excel 2007. Slide 9 tells you more about what SAP Support Packages are needed to enable the Excel 2007 ODBO/MDX connectivity.

What do you think a Data Warehouse Appliance is?

A recent TDWI survey asked people "What do you think a data warehouse appliance is?"  It is interesting that 53% of respondents replied that a data warehouse appliance is "server hardware and database software built specifically to be a data warehouse platform".  Also interesting is that only 19% answered that they don't know.  These results again point to how data warehouse appliances are becoming much more mainstream and adoption is picking up.  You can read more about the survey in Philip Russom's article "Defining the Data Warehouse Appliance" http://www.tdwi.org/Publications/display.aspx?id=7784.

IBM Cubing Services - MDX and ODBO - Excel Pivot Tables

In case you missed it, on May 6, IBM announced support for OLE DB for OLAP (ODBO) in their Cubing Services product.  You can read the official notice at http://www-01.ibm.com/common/ssi/rep_ca/3/897/ENUS208-113/ENUS208-113.PDF.  DB2 Warehouse edition is now called InfoSphere Warehouse.

In October 2007, IBM shipped MDX Query Language support in this product.  However, you had to access the MDX functionality via a proprietary interface.  At the Information OnDemand conference in October 2007, IBM had a demo of Business Objects Voyager working with Cubing Services.  Voyager is an OLAP reporting tool from Business Objects which connects to MDX data sources.  Voyager works with Microsoft Analysis Services, SAP BW, and some of the Business Objects Performance Management applications.  It was nice to see Voyager working with Cubing Services, albeit using a proprietary API.  Now Cubing Services supports ODBO so it is possible to connect any ODBO client to Cubing Services.  I have not yet played with this new version of Cubing Services.  I hope to soon.

One thing I am told is that Microsoft Excel works with Cubing Services via ODBO and MDX.  However, IBM's MDX Query Language implementation is what I call the MDX 1999 variant.  The MDX Language Specification is part of the OLE DB for OLAP Specification which Microsoft last published in 1999, therefore I call it MDX 1999.  Microsoft has of course added a lot of extensions to the MDX Query Language and the MDX supported by Microsoft Analysis Services 2005 is what I call MDX 2005.

Excel 2007 uses the MDX Query Language to connect to OLAP data sources.  Excel 2007 is an adaptive product - it will adapt to the MDX supported by the underlying data source.  Therefore, if you connect Excel 2007 to Microsoft Analysis Services 2005, it will function differently than if you connect it to Microsoft Analysis Services 2000.  Excel 2007 is able to determine what MDX Query Language features are supported by the underlying data source and expose functionality accordingly.

Since Cubing Services supports MDX and ODBO, you can connect Excel 2007 to Cubing Services.  However, only the MDX 1999 variant is supported and Excel 2007 on Cubing Services will function like Excel 2007 on Microsoft Analysis Services 2000.  So far, only SAP BW 7.0 and Microsoft Analysis Services 2005 support the MDX 2005 variant.  I am not sure of IBM's timeline for supporting MDX 2005.  Also, Cubing Services does not yet support XMLA.  I expect XMLA support to come relatively soon as this is technically easier to develop than ODBO and much easier to develop than MDX 2005 functionality.  If I was a betting man, I would expect IBM to announce XMLA at their next Information OnDemand conference.

Pivot Tables in Google Spreadsheets - Panorama Did It!

I wrote a while back about Pivot Tables in Google Spreadsheets.  Well, I have seen it and it is true.  Panorama has been able to embed a portion of their NovaView application within Google Spreadsheets to have Pivot Table like functionality.  What they do is interesting because they source the data from the Google Spreadsheet and seem to build a cube out of it on the fly.  The data in the Google Spreadsheet needs to be in a particular format so right now things are very simplified.

I spoke to Oudi Antebi at Panorama and he said their plan is to eventually source data from Microsoft Analysis Services, SAP BW, and anything else that supports the MDX Query Language from a server point of view.  Nonetheless, a Pivot Table on Google Spreadsheet data is very interesting and powerful.  You can easily load data from whatever your data source into the Google Spreadsheet.  Panorama's interface to build the cube is quite straightforward.  The tool is quite slick.  The concept is very powerful.  Pivot Tables and BI are something that Microsoft is focusing on for future versions of Excel and the fact that Panorama is able to quickly adapt their NovaView product to help Google get ahead of Microsoft has the potential to be a real game changer.

Nigel Pendse at the OLAP Report wrote a commentary on Panorama and Google which is also worth a quick read... http://www.panorama.com/newsletter/2008/apr/olap-report-commentary.html.

Business Objects Polestar and SAP BIA

During the keynotes at Sapphire, there was a lot of discussion about the new Business Objects Application called Polestar.  Polestar was demoed against SAP's Business Intelligence Accelerator (BIA).  The keynote demo showed Polestar working against over a billion records in BIA and performing extremely well.

I went onto the show floor to get a closer look.  At the BOBJ pod, I met Nigel Stoodley from BOBJ who gave me a very good demo.  Even though the demo on the show floor was not Polestar against BIA, it was nonetheless very impressive.  Polestar allows you to do ad hoc analysis of your data.  The current Polestar product has a Google-like interface to start with where you enter your query.  It then goes against the Universe and finds potential matches for your query.  You select a match and then a very nice graphical user interface comes up showing details of your selection.  You can drill down in the data, you can compare data, and you can get different types of visualizations - bar charts, pie charts, etc.  It is a simple yet powerful GUI that allows a non-analyst user to navigate the data and get a better understanding.  It's a very nice product and worth trying out if you want something like this.

At the demo pod, I asked Nigel more about how Polestar goes against various data.  Nigel explained to me that Polestar is built to go against a Universe.  Therefore, any BOBJ customer that has a Universe (or multiple Universes) can get Polestar and immediately be able to use the product.  I then queried further to understand how Polestar worked with BIA.  Nigel explained that SAP/BOBJ built a Universe on top of the BIA.  Data was loaded into the BIA and the Universe allowed Polestar to access this data.

While Polestar is shipping, the ability to connect to BIA is not readily available.  Let's hope SAP brings this out soon.  In the keynote, my understanding is that they demoed the Polestar / BIA solution live and were connecting to a BIA server that was in Walldorf, Germany.  Apparently, they were connecting live all the way from Orlando to Walldorf.  What was not explained was what the hardware configuration of the BIA system was that was able to process over a billion records so fast.

I really enjoyed Hasso's keynote this year.  I thought he did a really good job.  Hasso is a good speaker and he had the audience engaged, interested and amused.  I really cracked up when he swore on stage.  Hasso seems to be a strong proponent of TREX and BIA.

SAP Sapphire - Patrick Lencioni

The first official day of Sapphire was very busy.  They main keynotes were Patrick Lencioni and Tom Brokaw.  I have to admit that other than the keynotes, I only attended 1 session (lots of networking and not enough time in the day).

The session that I attended was by Doug Merritt and Marge Breya.  Doug owns what is now called POA - performance optimization applications and Marge owns the IDD (Information Discovery and Delivery) piece from BOBJ.  This was a good session to attend after attending the session by Dan Kearnan et al yesterday.  Doug and Marge gave an overview of the general direction of the combined SAP and BOBJ in the BI space.  They have this new application called Polestar.  Very interesting UI and they demo Polestar sitting on top of BIA.  Quite slick.  I want to go find this on the show floor and actually play with it myself.  Will write more when I get the chance.

I also want to mention the keynote by Patrick Lencioni.  I have read his first two books (they were very good) and then didn't read any of his other material.  The session today reminded me that I should read his newer material.  Patrick spoke about "The Five Dysfunctions of a Team".  Patrick is a really good speaker and I quite enjoyed his talk.  If you get a chance to see him speak, do it.  After the keynote, I purchased the book and got it autographed.  I spent a few minutes talking to Patrick.  Very interesting person.  A summary of the five dysfunctions are:

1. Absence of Trust

2. Fear of Conflict

3. Lack of Commitment

4. Avoidance of Accountability

5. Inattention to Results

I especially like #2.  You can't just accept a non-committal yes.  You really need to get people to discuss and debate the issue so that you get their real input.  Now that I have the book, I have some good reading material for the flight back home on Wednesday.