Connect – a Simba Technologies Blog

Amazon's new Big Data service - DynamoDB

Jim Ericson at Information Management writes about DynamoDB - Amazon's new Big Data service here: http://www.information-management.com/news/Amazon_big_data_cloud-10021812-1.html.  Worth a quick read.  DynamoDB builds on SimpleDB, and Ericson writes that "since it is a non-traditional database, DynamoDB will face a limited audience that the company hopes will grow as the service is approached and tested."  One thing about the so-called NoSQL Big Data databases is that they are simple to use and very powerful.  The challenge is to use these within the existing IT landscape.  Both Cloudera and Microsoft are promoting ODBC drivers as the way to bridge between the SQL and NoSQL worlds.  I think this is a good idea but one that needs more than the basic ODBC that these folks are touting.

Understanding the Oracle-Cloudera Big Data offering...

The big news of the day is likely Oracle's announcement of the specifics of their Big Data Appliance. As usual, TheReg has done a great job running down the specs. What I want to focus in on is the connectors that were announced today. The details are still somewhat thin but here's what I understand to be their offerings:

1.  “Oracle Loader for Hadoop, which moved data from Oracle 11g R2 databases to Hadoop data stores.”

Sounds like SQOOP to me.

2.  “Oracle Data Integrator for Hadoop, a twist on the existing Data Integrator tool that can automatically generate MapReduce code to chew on data and bring data sets into view of Oracle databases.”

This must be Cloudera’s ODBC driver that Cloudera announced late last year together with Tableau. I wonder how they are addressing the HiveQL/SQL gap.

3.  “Direct Connection for HDFS, and this essentially makes a section of the HDFS file system holding mapped and reduced data to be viewed as an Oracle database table.”

This sounds like a great idea. If this connector bypasses Thrift, it offers the potential for the highest performance.

4.  “R Connector for Hadoop. With this, Oracle has indeed taken the open source R statistical analysis package and added optimized math libraries to link it to the various data stores in the Big Data Appliance stack. This connector is not based on the Hadoop-friendly R tools from Revolution Analytics.”

 A specialized driver to address R.

 

Specifics aside, what looks clear is that integration/connectivity with "the establishment" is important. So what do you think and/or need? If you're trying to fit Hadoop into your landscape, drop me a line and let me know.

 

 

Microsoft Isotope CTP ultra-slick...

The Christmas and New Year season is finally passed. Or, at least, the turkey is out of the way. Whether one has gotten over the holiday season is another matter.

If anyone paid attention, Microsoft released a CTP of Isotope just before Christmas (see here). The easiest way to explain Isotope is that  it is "Hadoop for Windows".

I had some time to try it out and I was pleasantly surprised by how slick the package is. For one, the CTP is hosted on Azure so the pre-requisite for evaluating it is absolutely minimum. And the Metro look is refreshing.

The most useful bit is the inclusion of an ODBC driver and a VSTO pane that provides integration with Excel. The VSTO pane provided functionality equivalent to Microsoft Query since Microsoft Query doesn't currently run. (Of course, HiveQL isn't SQL so there will be limitations down the line.) But I was able to poke the ODBC driver using other tools. On the whole, this CTP is ultra-slick at showing Hadoop's power and demonstrates how Microsoft can harness its vast repertoire of technologies to provide a compelling offering.

If you haven't signed up yet, do.

 

Using Hive from ODBC

Hadoop and Hive are definitely on everyone's mind these days. Whenever I see the words 'big data', connecting to Hadoop and Hive are not far. There is a latent demand for ODBC connectivity to Hive. To recap, nearly every top vendor has announced and/or released some form of integration with Hadoop & Hive:

• IBM: http://www-01.ibm.com/software/data/bigdata/enterprise.html
• Microsoft: http://www.microsoft.com/bigdata
• Oracle: http://www.oracle.com/us/technologies/big-data/index.html
• Pentaho: http://www.pentaho.com/hadoop/

There is of course Hive's own ODBC driver. But it seems that many people are looking in particular for a Windows driver.

Looking further, Hive's HiveQL is both a subset and extension of SQL-92. Supporting the full SQL-92 would be especially useful when integrating Hive ODBC into legacy apps.

So what are your challenges with integrating Hive into your solution and/or landscape? Drop me a line.

 

ODBC, Business Intelligence, and Big Data

The recent announcements by Microsoft about Big Data are very good news for those of us who work in the OLAP and BI space.  Basically, it looks like Microsoft is going to allow you to do OLAP and BI from Analysis Services on top of Hadoop and Hive.  A very glod blog post by Andrew Burst is worth a read.  "Microsoft will also make available an ODBC driver for Hive, the data warehousing front-end for Hadoop developed by FaceBook. What’s the big deal about an ODBC driver?  The combination of that driver and Hive will allow PowerPivot and SQL Server Analysis Services (in its new “Tabular mode”) to connect to Hadoop and query it freely.  And that, in turn, will allow any Analysis Services front end, including PowerView (until last week known by its “Crescent” code name), to perform enterprise-quality analysis and data visualization on Hadoop data.  Not only is that useful, it’s even a bit radical."  You can read the full post here: Putting the 'BI' in Big Data

Microsoft announces support for ODBC on Linux

Yes, you read the title correctly.  Microsoft is about to release a Linux ODBC driver for SQL Server.  This was announced yesterday at the SQL Server PASS conference.  For those of us who are ODBC fans (like me), this is great news.  With the news that Microsoft will deprecate the OLE DB provider for SQL Server and now with the news that we will get an official Microsoft Linux ODBC driver for SQL Server, it just re-inforces the fact that when Microsoft and Simba developed the ODBC specification 20 years ago, we got it right.  ODBC is the most prevalent database connectivity protocol out there and combined with SQL is a must have for any data source that wants to interoperate today.

There is a good blog post by Brian Swan from Microsoft that you can read here: Microsoft Announces SQL Server ODBC Driver for Linux!

ODBC is the gateway to Big Data

I am at the Microsoft SQL Server PASS conference this week and the big news today was Microsoft's announcement that they are supporting Hadoop.  So, how is Microsoft supporting Big Data?  Through ODBC of course.  Specifically, Microsoft will introduce an ODBC driver for Hive (the Hadoop query engine) on Windows.  This should be released next year.  It is interesting because other Simba customers including Teradata Aster Data use Simba's technology to provide ODBC drivers.  As much as Big Data and the Cloud are interesting, at the end of the day, you still need to connect applications and data sources and you need to connect to the desktop.  For 20 years, ODBC has been a standard of connectivity and in the world of Big Data and Cloud, ODBC continues to be the most widely used data connectivity interface.  You can read more about Microsoft's ODBC for Big Data.

ODBC is the gateway to the Cloud

It's interesting that it is SQL Azure that is pushing Microsoft to invest more in their ODBC driver and deprecate their OLE DB provider.  We are seeing the same thing in the ODBC SDK business.  We continue to partner with companies who have a Cloud-based or hosted solution and while these are great solutions, their customers still want to connect products like Microsoft Excel to these cloud data sources.  There are really two solutions to do this.  The first is to export the data from the cloud to some sort of delimited text file and then import this text file into Excel.  This is a very cumbersome solution.  The second is to build an ODBC driver that will talk to the web services interface of the cloud data source and allow Excel to connect directly and dynamically.  More and more of the cloud vendors are seeing the power of such an ODBC for cloud or ODBC for web services solution.  We just recently had a customer in Europe that build an ODBC driver using the C# programming language in a week using our SimbaEngine ODBC SDK.  When they showed the solution to their management, they were amazed at how easy it was to build and yet how functional the solution was.

Vectorsoft ODBC Driver Performance Optimization

Vectorsoft has been a long-time Simba customer. They have an interesting product called CONZEPT 16 - an integrated database development environment and system. They have just released a new version (5.6) of their ODBC driver based on the SimbaEngine SDK 8 for CONZEPT 16. Not only are they getting the immediate benefits of the support for ODBC 3.52, Unicode and 64-bit,  but they are also taking advantage of improved performance and the advanced performance features in the new generation of SimbaEngine. After implementing pass-down of filters and joins, they were excited to share a blog with us (http://blog.conzept16.de/2011/08/ODBC-Treiber-Performanceoptimierung, note in German). According to Robert’s testing, the new driver has vastly outperformed the old driver which is based on SimbaEngine SDK 7. The performance boost is especially dramatic (orders of magnitude) for queries with filters on non-indexed columns and with joins. This totally makes sense since operations (or functions) are “shipped” closer to the data store rather than the data. The operations are not limited to the above mentioned, they can also be aggregations, “select from” queries and etc. What’s powerful about this feature is it allows the driver specific glue code (DSII) to dynamically select operations to pass down and automatically leave the rest to the SimbaEngine – a truly collaboratively approach to SQL processing.

More about Microsoft's move back to ODBC standard

Kurt Mackie at Redmondmag.com wrote more about Microsoft Shifting To ODBC for SQL Server.  The next version of SQL Server - Denali - is expected later this year and we expect more news at the PASS Summit in October.  I will be attending PASS again this year and will post further updates.  One interesting thing to watch is how Microsoft Analysis Services will connect to relational data sources in the future.  Today, your options are OLE DB and ADO.NET.  My expectation is that the importance of ADO.NET will increase with the deprecation of OLE DB.

More about Microsoft's plans to consolidate around ODBC and deprecate OLE DB

Microsoft's recent announcement that they would focus more on ODBC and deprecate OLE DB has raised a lot of questions.  Amina Saify at Microsoft recently posted some questions and answers.  Some interesting points were:

1. "When it comes to uniform data access to SQL Server from different platforms, ODBC has always been a better choice and that was consistently quoted by all of our customers in various surveys, SDRs and forums. By fully aligning with ODBC, Microsoft will be focusing on one set of industry standard APIs that are widely used by many of our customers."  This is a very powerful statement.  At Simba, as the leader in data connectivity, people are always asking us what open standards based interfaces they should implement for their databases and data sources.  My answer has always been ODBC first followed closely by JDBC.

2. "More customers are adopting cloud these days. To enable client applications running on various different platforms to connect to cloud, Microsoft chose ODBC as the de-facto standard client connectivity API for native client applications connecting to SQL Azure."  I am curious as to what Microsoft's position is around JDBC access to Azure?

3. " In most of the performance comparison tests we have done, ODBC seem to perform better."  Not only does ODBC perform better, it is also supported on pretty much any platform out there - Windows, Linux, Solaris, HP/UX, AIX, and MacOS.  Talk about ubiquity.

You can read Amina's post here: Microsoft is Aligning with ODBC for Native Relational Data Access or and I have reproduced it below as well:

FAQ:

 

Question1: Microsoft created OLE DB after ODBC, isn’t deprecating OLE DB a step going backwards?

Answer: Microsoft has been supporting ODBC through all the releases of SQL Server for relational data access. This includes adding new functionality to the ODBC drivers whenever a corresponding feature is added to SQL Server. OLE DB was introduced primarily to provide uniform data access to non-relational data as well as relational data. But it is a Microsoft proprietary technology that worked only on Microsoft platforms. When it comes to uniform data access to SQL Server from different platforms, ODBC has always been a better choice and that was consistently quoted by all of our customers in various surveys, SDRs and forums. By fully aligning with ODBC, Microsoft will be focusing on one set of industry standard APIs that are widely used by many of our customers.

 

Question2: Why is Microsoft making this change now? What are the advantages?

Answer: More customers are adopting cloud these days. To enable client applications running on various different platforms to connect to cloud, Microsoft chose ODBC as the de-facto standard client connectivity API for native client applications connecting to SQL Azure. If you are writing applications that run against standalone SQL Server and SQL Azure, or if you are planning to port your application to SQL Azure, using ODBC APIs make the transitions pretty seamless. ODBC APIs use industry standard interfaces and are very simple and straight forward to use. Using multiple result sets, memory management and specifying required/optional properties in OLE DB is more complex. Overall, customers get the benefit of easy programming and cross-platform support with this alignment and can now build applications that can be uniformly ported between various platforms.

 

Question3: How long will the current SQL Server OLE DB provider be supported?

Answer: SQL Server OLE DB will be supported for 7 years from the Denali launch, the life of Denali support, to allow you a large window of opportunity for changing your applications before the deprecation. It will also be supported through the complete life of Denali extended support.

 

Question4: What is being deprecated? What happens to other Microsoft products that use and depend on OLE DB?

Answer: This announcement is about the deprecation of Microsoft SQL Server OLE DB provider only. Other Microsoft OLE DB providers, 3^rd party OLE DB providers and the OLE DB standard will be supported as per their respective terms. Other Microsoft SQL Server features and products that are built on top of OLE DB or use OLE DB, like Distributed Query (Linked Server), SQL Server Integration Services, SQL Server Analysis Services etc., will continue to be supported as per their respective terms. Microsoft will take care of replacing the underlying dependency. Providers like ADO.Net which can run on top of OLE DB will not support OLE DB once the latter is deprecated. At that time the clients using the underlying provider need to update their application to use a different provider.

 

Question5: Are there any features in OLE DB that are missing in ODBC? How about performance?

Answer: Most of the features related to relational data access in SQL Server are already available in ODBC. The specific implementation and usage may be different between these providers and if there are features that have better implementation in the current OLE DB, Microsoft will port such functionality to ODBC based on customer demand and feedback. In most of the performance comparison tests we have done, ODBC seem to perform better. We will publish a more detailed performance analysis document in the coming months.

 

---

Amina Saify

Microsoft endorses ODBC over OLE DB

Wow!  About a decade and a half ago, Microsoft said that OLE DB would be the successor to ODBC.  Now, Microsoft announces that "The next release of Microsoft SQL Server, Code Name “Denali,” will be the last release to support OLE DB."  For those of us who have been in the industry for so long, this is great news that Microsoft has finally brought some clarity to this.  Simba has many customers who use our SimbaEngine SDK for data connectivity solutions for ODBC, JDBC, OLE DB, and ADO.NET and so people often ask us for advice as to which standard to support.  We have maintained for years that ODBC and JDBC are the primary relational database connectivity standards and it is great to see this news from Microsoft today to give more clarity to the situation.  Just to be clear, I am quoting the entire Microsoft announcement here:

"Today we are announcing that Microsoft is aligning with ODBC for native relational data access – the de-facto industry standard. This move supports our commitment to interoperability and our partners and customers’ journey to the cloud. SQL Azure is already aligned with ODBC today.

The next release of Microsoft SQL Server, Code Name “Denali,” will be the last release to support OLE DB. OLE DB will be supported for 7 years from launch, the life of SQL Server Code Name “Denali” support, to allow you a large window of opportunity for change before deprecation. We encourage you to adopt ODBC in any future version or new application development.

Conversations with our customers and partners have shown that many of you are already on this path. The marketplace is moving away from OLE DB and towards ODBC, with an eye towards supporting PHP and multi-platform solutions. Making this move to ODBC also drives more clarity for our C/C++ programmers who can now focus their efforts on one API. "

You can read the actual blog entry here: Microsoft Aligning with ODBC.

ODBC and JDBC for the Cloud

I was talking to one of our customers recently that has a solution that lives in the Cloud.  It is a very powerful solution and they have been gaining a lot of traction.  One issue the end users have is they want to be able to analyze their data in certain ways.  Our customer was constantly working to build various reports for their end users.  Then our customer found that their end users would take the data from these custom reports and load it into Excel.  Epiphany.  They realized if they had a simple link to Excel, they would no longer have to build all these custom reports and their end users would be free to do whatever they needed.  That was when our customer approached Simba to see how we could help.  They had a very well documented web services interface to their Cloud solution.  One of our developers took 3 days to quickly build a custom ODBC driver that allowed Excel to pull data directly from the Cloud solution.  Our customer was amazed.  If they had realized they just needed an ODBC driver for their product, it would have literally saved thousands of person hours building the custom reports and endless end user frustration.  As much as solutions are moving to the Cloud, people still like their desktop tools like Excel and all they want is the ability to quickly analyze their data in Excel.  A custom ODBC or JDBC driver enables you to pull any data into Excel or Crystal Reports or any tool of your choice.

‘tis the season for predictions…

On news feeds and podcasts over the last few weeks, there have been lots of predictions of what 2011 has in store for us.  Sprinkled subtly among the hot themes (more mobile, more social, more collaborative), I noticed something interesting in the background—there was a lot of talk about data.  How to handle the ever-increasing growth of data.  How to make data more open and accessible.  How to make sense of increasingly interconnected data.

Whether we’re talking about the latest apps for a smartphone, the latest network that links people together, or the latest technology that helps people work together, at the core there is always the data that is either generated by it or needed by it.  20 years ago, we might have been talking about the latest desktop database technology and how we were going to analyze the data from it and combine that with data from a mainframe system.  It’s still the same problem—we’ve got some large sets of data from different sources and we want to slice & dice that data in a way that makes sense to us.

It’s telling that 20 years ago, the first ODBC driver allowed Excel users to pull in data from diverse sources like dBase and ISAM and produce their own custom reports & charts—and we’re still using the same tools to answer tough business questions today.  The range of data connectivity tools has expanded to include ODBC, JDBC, OLEDB, ADO.NET, OLAP, ODBO, XMLA, MDX and more, but the fundamentals haven’t changed.  My (admittedly rather safe) prediction for 2011 is that the need for connectivity to diverse data sources will continue to grow (and keep growing well into the future).   If there is some unusual or custom data source that you would like to connect your applications to, or pull into your servers from, please drop us a line as it may be simpler than you think.

On news feeds and podcasts over the last few weeks, there have been lots of predictions of what 2011 has in store for us.  Sprinkled subtly among the hot themes (more mobile, more social, more collaborative), I noticed something interesting in the background—there was a lot of talk about data.  How to handle the ever-increasing growth of data.  How to make data more open and accessible.  How to make sense of increasingly interconnected data.

Whether we’re talking about the latest apps for a smartphone, the latest network that links people together, or the latest technology that helps people work together, at the core there is always the data that is either generated by it or needed by it.  20 years ago, we might have been talking about the latest desktop database technology and how we were going to analyze the data from it and combine that with data from a mainframe system.  It’s still the same problem—we’ve got some large sets of data from different sources and we want to slice & dice that data in a way that makes sense to us.

It’s telling that 20 years ago, the first ODBC driver allowed Excel users to pull in data from diverse sources like dBase and ISAM and produce their own custom reports & charts, and we’re still using the same tools to answer tough business questions today.  The range of data connectivity tools has expanded to include ODBC, JDBC, OLEDB, ADO.NET, OLAP, ODBO, XMLA, MDX and more, but the fundamentals haven’t changed.  My (admittedly rather safe) prediction for 2011 is that the need for connectivity to diverse data sources will continue to grow (and keep growing well into the future).   If there is some unusual or custom data source that you would like to connect your applications to, or pull into your servers from, please drop us a line as it may be simpler than you think.

OData gaining traction

It’s been interesting to watch the list of producers of live OData Services grow over the last few months.  The early adopters (like Netflix, some municipal & federal government agencies, and several Microsoft projects) have been joined by the likes of Facebook, eBay and twitpic.  The list is even gaining a sense of humour, as evidenced by the entry for Nerd Dinner:  “Nerd Dinner is a website that helps nerds to meet and talk, not surprisingly it has adopted OData.”  You can check out the complete list here:  http://www.odata.org/producers.

As of this writing, the list is up to 33 services with a good mix between large/high-profile entities with wide-ranging application possibilities and small initiatives often focused on just a slice of the tech community itself.  We need both, because it’s the big names with brand recognition that catch people’s attention, but small-scale tinkering by techies can be an excellent incubator from which new & creative ideas will eventually emerge.

OData, the technology, is “the web-based equivalent of ODBC, OLEDB, ADO.NET and JDBC” (Chris Sells, via MSDN).  Just as the ODBC specification and its descendants have enabled data access between clients and servers across widely disparate hardware and OS platforms, the OData specification promises data access between “consumers” (applications using standard web technologies) and “producers” (data sources residing on various web server platforms).

OData, looked at as a movement, is creating an open-ended environment with a whole new range of business decisions for companies to deal with.  At the simplest level, the questions for a company choosing to publish an OData service are similar to those you would consider for a web application:

• Scope:  Will the data be accessible only via a corporate Intranet, or will it be available on the open Internet?
• Authentication/Security:  Will a user have to login to access the data?  Will some portion of the data be available without logging in?  The Netflix example is a good one to consider here—their movie catalogue is available to the general public, but a user’s movie queue is restricted to authenticated users only.
• Will the data be Read only or Read-write?
• What subset of fields from the database will be exposed?
• What range of data from the database will be exposed?

The big difference, of course, is that you’re not the one creating the application.  You may also create an OData consumer application in addition to the OData producer data source, but the key is that you’re not the only one doing so and there are potentially many applications consuming your data feed.  This moves you up to a higher level of business questioning:

• How might our data potentially be combined with other data sources (either by us or by others) and what value does that potentially add for our customers?
• What type of new business relationships are possible with other producers of OData feeds?
• What is the value to our business of this data (i.e. do we need to charge for access to it)?

If your company makes a product that generates data in some form, adding an ODBC driver opens it up to a wide range of general-purpose reporting tools so your customers can slice & dice the data to their hearts’ content.  The benefits of providing data connectivity to the desktop are proven and clear.  If you’re already doing this, what are your company’s plans for providing data connectivity to the web?  Let us know, we’d love to hear from you.

SAP HANA now available - supports MDX and SQL

SAP just announced that their High performance ANalytical Appliance (HANA) is now available.  HANA supports both the SQL and MDX query languages.  SQL is supported via ODBC.  I am not sure if they also support JDBC, but if not, it would make sense this would come very soon.  MDX is supported via ODBO.  XMLA is not supported yet but most BI applications and especially Excel are more ODBO-centric so this is fine.  Since Excel only works with ODBO providers, I would much rather have ODBO than XMLA.

Some comments about HANA from Jake Klein (VP Product Management at SAP): http://sapteched.news-sap.com/2010/12/01/still-wondering-what-are-in-memory-and-sap-hana/.

ODBC 3.8 Applications and Drivers

Platform support for ODBC 3.8 has been available under Windows 7 (and Windows Server 2008 R2) for a bit more than a year now.  In the world of ODBC (as with many other industry standards), we see a bit of a chicken and egg effect with the roll out of updates to the standard:  Applications need to be enhanced to take advantage of the new capabilities, but the ODBC drivers also need to implement the new features before applications can use them.  Fortunately, one of the hallmarks of ODBC is its strong emphasis on compatibility, so these things don’t need to happen at the same time.  Updated drivers that make the new features available can be used by older applications and applications with enhancements for the new features can still work with older drivers.  The end user doesn’t see all the benefits until both the application and the driver are at the same level but they get the full functionality of the lower level regardless.

From the outset, it’s been expected that the upcoming “Denali” release of the SQL Server Native Client would be first out of the gate with database support for the new features.  This was confirmed last week by Jimmy Wu in his post on the SQLNCli team blog.  For ODBC application developers where SQL Server is your data source, this means you can now start to take advantage of these new features:

• With Streamed Output Parameters, you will be able to retrieve extremely large data objects (such as the new varbinary(max), varchar(max), and nvarchar(max) objects introduced with SQL Server 2008 R2) without the ODBC driver having to allocate enough buffer space to contain the entire object at once.
• ODBC C-Type Extensibility will allow you to get data objects from a database to your application even if the data types don’t exist in the ODBC specification.  Prior to ODBC 3.8, the ODBC driver would have to convert the data into a supported ODBC format (like SQL_C_BINARY) and your application would have to reconstruct it.
• Asynchronous Connection Operations and improved connection pool management mean that your application can be designed for larger numbers of users with less-frequent access requirements (such as with cloud-based services and large-scale corporate applications).

Simba is committed to ODBC 3.8 and will be adding support for it in a future release of SimbaEngine ODBC SDK, bringing these advanced features to the full range of Windows, UNIX and Linux platforms that we support.  We’re interested in hearing from client application developers if you’re starting down the road to take advantage of these features.  We are also interested to hear from developers using database driver SDKs who feel that these features can be passed through elegantly to their data source in ways that will enhance functionality and performance.

Build, buy, or something in-between?

It’s not the sort of question that usually comes up over hors d'œuvres, but given the type of event I was at, I guess it’s not too surprising…  It was a gathering for Product Managers in local tech companies and I found myself chatting with someone whose company made GPS-enabled systems for tracking fleets of vehicles.  Their system of course produced lots of data and reports to go with it but they found that customers really wanted to slice & dice the data themselves, using something like Excel.  They’ve had a simple “export” feature available for years, but their customers really wanted “live” access to the data, so they were looking at ODBC as their solution.

Which gets me to the question—when they found out that the company I worked for made an ODBC SDK, they asked me why someone would use our SDK rather than Microsoft’s own ODBC SDK?  After all, Microsoft’s is essentially free as part of your MSDN subscription and it includes things you’ll need like a driver manager, installer DLL, test tools and even sample applications.  They had already looked at some other commercial solutions for pre-packaged ODBC drivers and determined that they weren’t viable because either the configuration options were too limited or they couldn’t package the driver the way wanted to deliver it to their customers.

I agreed that given their needs, they were definitely candidates to “build-it” rather than “buy-it” but what Simba offers is a third option that falls between the two extremes.  Microsoft’s ODBC SDK is a low-level C language API and a full, commercial implementation built with this API is a major undertaking, with low level data checks, sequence checks and error reporting needed.  It also assumes that you have a database engine that supports SQL and provides at least basic relational database functions.  Our SimbaEngine ODBC SDK is actually complementary to Microsoft’s, it is a higher-level SDK that makes it easy for you to build a driver to the Microsoft ODBC specification.  If your data is stored in flat files or a non-relational format, we also include a SQL Engine to provide the SQL processing required for ODBC.

Since this is quite a common question for us (although I have to admit, it’s the first time I’ve been asked over wine & cheese…), we have a white paper that goes into more detail, so I sent them a link to it and I thought it would be worth sharing the story here as well.  Here’s the link if you’re curious to learn more: http://www.simba.com/docs/Using-SimbaEngine-ODBC-SDK-vs-Microsoft-ODBC-SDK.pdf

Hiding things in plain sight…

It’s a good time to be making the switch from 32-bit to 64-bit Windows because most of the issues from the early days of 64-bit (driver problems for everything ranging from your sound card to your printer) are behind us.  You might be making the switch to take advantage of more powerful applications, like 64-bit Microsoft Excel 2010 that lets you work with much larger pieces of data.  Or, you might be making the switch to just get past the 3.5(ish) gigabyte RAM ceiling in 32-bit Windows and you’re going to stick with your old 32-bit applications for a while longer.

If you’re in the latter camp and you use an ODBC driver to connect a client application (like Excel or Crystal Reports) to a back-end database system, you will get to play a little game of hide and seek…

The ODBC Data Source Administrator is a tool you will be familiar with and it looks exactly the same in 64-bit Windows as it did in 32-bit Windows, but there is a catch.  When you access the program the way you normally would, via the Control Panel, you get a version which only configures 64-bit drivers.  If you are still using 32-bit applications, they won’t see any of these drivers.  Fortunately, Microsoft has included the old 32-bit ODBC Data Source Administrator, but you have to access it directly by navigating through the file system.  We have published a “How-To” that goes into a bit more detail http://www.simba.com/docs/HOW-TO-32-bit-vs-64-bit-ODBC-Data-Source-Administrator.pdf, but the short answer is that you need to create a shortcut to “C:\WINDOWS\SysWOW64\odbcad32.exe” and use this to create data sources for your 32-bit applications.

It’s a good time to be making the switch from 32-bit to 64-bit Windows because most of the issues from the early days of 64-bit (driver problems for everything ranging from your sound card to your printer) are behind us.  You might be making the switch to take advantage of more powerful applications, like 64-bit Microsoft Excel 2010 that lets you work with much larger pieces of data.  Or, you might be making the switch to just get past the 3.5(ish) gigabyte RAM ceiling in 32-bit Windows and you’re going to stick with your old 32-bit applications for a while longer.
If you’re in the latter camp and you use an ODBC driver to connect a client application (like Excel or Crystal Reports) to a back-end database system, you will get to play a little game of hide and seek…
The ODBC Data Source Administrator is a tool you will be familiar with and it looks exactly the same in 64-bit Windows as it did in 32-bit Windows, but there is a catch.  When you access the program the way you normally would, via the Control Panel, you get a version which only configures 64-bit drivers.  If you are still using 32-bit applications, they won’t see any of these drivers.  Fortunately, Microsoft has included the old 32-bit ODBC Data Source Administrator, but you have to access it directly by navigating through the file system.  We have published a “How-To” that goes into a bit more detail <http://www.simba.com/docs/HOW-TO-32-bit-vs-64-bit-ODBC-Data-Source-Administrator.pdf>, but the short answer is that you need to create a shortcut to “C:\WINDOWS\SysWOW64\odbcad32.exe” and use this to create data sources for your 32-bit applications.

Minding your bits...

I note that Window's transition from 32-bit to 64-bit has been an on-going saga for the last few years. The 64-bit variant of Windows XP was a big step forward. Another major milestone was just recently crossed when Microsoft released the first 64-bit variant of Microsoft Office as part of the Office 2010 launch in May.

For those who were around when Windows last made a machine-word-size transition--from 16 to 32-bit--things will be a deja vu of sorts.

I've noticed that this time around, everyone seems to be jumping onboard the biggest, baddest Excel (64-bit 2010) with a lot more enthuasism than I recall the last time around.

One impediment lurking in the background that many people may be caught off-guard by is the availability of the appropriate 64-bit database drivers/providers. Simba's been working hard to catch up here.

So what are people's experience with 64-bit Excel so far? How has your transition to 64-bit been? What sorts of difficulities have you ran into when you made the leap to 64-bit?

The case for a custom ODBC driver

I was talking to a customer recently and they were telling me they have a very high performance database that scales very well, but they have poor reporting.  They were complaining that they spent so much time on building such a great database product and all their customers see is that they cannot get the reports they need.  So what to do?

The answer is simple, build a custom ODBC driver for the database.  The database product already supports SQL so all they need is ODBC.  The customer complained that they didn't have anyone in the company who understands ODBC.  Again, the answer is simple.  I asked one of our engineers to work with the customer and they literally had Crystal Reports talking to their database using the custom ODBC driver in about a week.  The customer was amazed.  They went to one of their users and gave them a demo of the proof of concept ODBC driver.  The user loved it.  Our customer was ecstatic because they now had access to all ODBC compliant applications to show off their database.  Win-win-win.

EMC buys GreenPlum

Interesting news yesterday about EMC's acquisition of data warehouse appliance company GreenPlum.  GreenPlum had of course partnered heavily with Sun Microsystems which was recently acquired by database giant Oracle.  GreenPlum has some interesting customers including eBay, NYSE Euronext, and NASDAQ OMX Group.  GreenPlum plays in the same space as veterans Teradata and Oracle and upcomers like Aster Data, Kognitio, ParAccel, InfoBright, Vertica, etc.  This is a very interesting space which saw Microsoft acquire DataAllegro and coming out with SQL Server 2008 R2 Parallel Edition as well as SAP's pending acquisition of Sybase.  SAP is also working on their own in-memory column-oriented technology and their upcoming HANA appliance.  As the volume of data continues to grow exponentially, the ability to store and analyze large amounts of data continues to be a challenge for every enterprise and so we will continue to see more action in this space.  As the databases scale to be larger and larger, one question that always comes up is how do we access and analyze all of this data.  The SQL query language accessed via ODBC and JDBC drivers continues to be the language of choice in the database world.  The MDX query language accessed via ODBO and XMLA is now the de-facto language of choice for analytics.

I was recently asked who supports MDX and the answer is, everyone.  The major database players - Oracle, IBM, Microsoft, and SAP all support MDX query language.  The major BI players - SAP/Business Objects, Microsoft, Oracle/Hyperion, and IBM/Cognos all support the MDX language.  Microsoft Excel is on virtually every desktop in the enterprise and if you look at Microsoft's recently released Excel 2010, the BI capabilities in Pivot Tables has been extended with the addition of sparklines, slicers, and even write-back.  Excel Pivot Tables talks to analytic databases using the MDX query language.  SQL and MDX are both necessary for analytics today.

The Kingdom of Excel

Whoa!  At the Gartner BI Summit, they suggested we "Wave the white flag on using Excel for business intelligence".  What breakthrough thinking!  This is something we at Simba have seen for years.  Many BI/OLAP vendors have come to us looking for an MDX Query Language solution for their OLAP/data warehousing products. We have found that a key driver for supporting the MDX Query Language was connectivity with Excel and specifically Excel Pivot Tables.  Microsoft continues to invest heavily in the BI capabilities of Excel and having an MDX Provider for your OLAP/multi-dimensional data source allows Excel to connect natively.  Native Excel connectivity is something we have now brought to Oracle OLAP cubes and we will even be at the Oracle Collaborate 10 conference next week where we have two sessions on showing users how to use Excel to connect to Oracle 11g database: http://www.simba.com/news/Simba-Provides-Free-Training-Connecting-Excel-PivotTables-to-Oracle-OLAP-at-COLLABORATE.htm.

Excel connectivity is a very important feature of any data source be it relational, multi-dimensional, ISAM, object-oriented, XML, etc.  Excel can connect to anything that has an ODBC or an ODBO interface.  Once someone can see how their data can be pulled live into Excel and how an Excel user can then get creative in their tool of choice, it becomes a very powerful reason to have an ODBC driver or ODBO provider.

You can read more about what Gartner is saying here: http://searchbusinessanalytics.techtarget.com/news/2240018042/Gartner-BI-Summit-Wave-the-white-flag-on-using-Excel-for-business-intelligence.

The new efficiency/normal according to Steve Ballmer

I was reading Steve Ballmer's latest email entitled "The New Efficiency" (http://www.microsoft.com/mscorp/execmail/2009/09-29NewEfficiency.mspx) in which he basically says the new normal in business is that "people borrow less, save more, and spend with much greater caution."  This is a simple yet powerful premise to do more with less.  At Simba, we have always tried to help our customers do more with less.  Simba started out helping ISVs deliver custom ODBC and JDBC drivers to their customers to increase value.  With our ODBC and JDBC SDKs, our ISV customers were able to quickly and efficiently build open standards based data connectivity for their databases which then gave their customers tremendous advantage in they could now use a wide variety of applications with these databases.  Then, Simba broadened what we did by also adding an OLAP SDK that supports ODBO and XMLA.  Today, Simba is going a step further.  We are now adding value directly for enterprise customers.  Our new MDX Provider for Oracle OLAP lets any user of Oracle OLAP connect to their data directly from Microsoft Excel.  The user gets to leverage all the power of Excel and Oracle OLAP without having to either use an Excel spreadsheet add-in which requires learning new software or a separate mid-tier server which can be quite expensive and take time to install and configure.

ODBC 3.80 is here!

Finally, ODBC 3.80 is here!  According to Pak-Ming Cheung from Microsoft, “We’re excited to announce that version 3.8 of the Microsoft ODBC DM (Driver Manager) will be released in Windows 7 and Windows Server 2008 R2”.  You can read more in his blog post at: http://blogs.msdn.com/data/archive/2009/07/06/odbc-dm-3-80-in-windows-7-and-windows-server-2008-r2.aspx.

 

Basically, there are 4 main new features in ODBC 3.80:

1.       Streamed Output Parameters

2.       ODBC C-Type Extensibility

3.       Asynchronous Connection Operation

4.       Better Management in ODBC Connection Pooling

 

As a leader in the data connectivity space in general and ODBC in particular, Simba is definitely committed to supporting ODBC 3.80.  If you would like more information, please don’t hesitate to contact us to discuss further.