Sunday, March 30, 2008

Hibernate Dialect for DB2

I have been working for about a week on making our Dodeca product work on DB2. I finally got it done today. First let me explain a little about how Dodeca works to set the context for my post.

Dodeca runs on metadata that describes every part of the application. All report definitions, Essbase connections, point of view selectors, toolbars and even custom code are stored in a relational database and are retrieved to the client via web services on 'as-needed' basis. The system is so completely run by metadata that an application can literally be exported to a single zip file. We often create prototypes for customers, ship them an exported zip of the metadata and, within minutes, they have their prototype up and running on their system.

On the middle tier, we use an open source Object-Relational Mapping ("ORM") technology called Hibernate. Hibernate takes care of persisting the Plain Old Java Object ("POJO") representations of the metadata in our server to a relational database. Hibernate uses a 'Dialect' to create the SQL syntax necessary for a particular database flavor. We have used Hibernate with a number of databases but most frequently with MySQL, SQL Server and Oracle. We have been able to get each of these databases running with a minimum of effort. My guess is that not as many people have used Hibernate with DB2 or that they haven't used it like we are using it.

In particular, we are storing the key metadata field in a Character Large OBject ("CLOB") field. Support for CLOB data among the relational database vendors is quite varied; DB2 does something quite different. After some research, I found that the CLOB fields produced by our Hibernate schema generation were very small. In fact, they were defaulting to a size of 255 bytes. That is pretty small for any piece of metadata, particularly an Excel file. I also found that DB2 CLOBs used a number and a scaling factor in their DDL language like this:

ENCODED_ARTIFACT CLOB (1024 M )

I looked at the Dialect base class in Hibernate and found an interesting method to override:

public String getTypeName(int code, int length, int precision, int scale) throws HibernateException

Based on my experience with Hibernate over the past couple of years, I guessed that the arguments passed to this function were obtained from the Hibernate mapping file. I tested that assumption with the following mapping:

<property name="EncodedArtifact" column="ENCODED_ARTIFACT" type="text" length="16" scale="2">

I was correct and decided to use the scale option to convert to int passed in to the method to a string in the format that DB2 could understand. In fact, I decided to let Hibernate create the initial string for a CLOB type and then I would modify it to fit my needs. I wrote the following class and used my new Dialect, com.appliedolap.dodeca.hibernate.dialect.DB2Dialect, in my hibernate.properties file.

Here is the Java code for the dialect:


package com.appliedolap.dodeca.hibernate.dialect;

import org.hibernate.HibernateException;
import java.sql.Types;

/**
* An SQL dialect for DB2 with fixes for Clob column creation for Dodeca.
*
@author Tim Tow
*/

public class DB2Dialect extends org.hibernate.dialect.DB2Dialect {

public DB2Dialect() {
super();
}

/**
* Get the name of the database type associated with the given
* {
@link java.sql.Types} typecode with the given storage specification
* parameters. Note: this code calls the parent class getTypeName method
* unless the type is a CLOB. If the type is a CLOB, the length field from
* the Hibernate mapping is used as the length of the CLOB. Valid values are
* from 1 to 99999.
*
* The scale field is used to specify the scale of the CLOB. Valid values are 0 - bytes, 1 - kilobytes
* 2 - megabytes, and 3 - gigabytes. The default is megabytes.
*
*
@param code The {@link java.sql.Types} typecode
*
@param length The datatype length
*
@param precision The datatype precision
*
@param scale The datatype scale
*
@return the database type name
*
@throws org.hibernate.HibernateException If no mapping was specified for that type.
*/

public String getTypeName(int code, int length, int precision, int scale) throws HibernateException {
// validate the length
if (length < 1 length > 99999) {
length =
1024;
}

// seed the typename
String result =
super.getTypeName( code, length, precision, scale );

// fix up the clob type for DB2 to create it properly
if (code == Types.CLOB) {
StringBuffer buffer =
new StringBuffer(16);

// add the first part of the string
buffer.append(result.substring(
0, result.indexOf(")")));

// add the scale for bytes/kbytes/mbytes/gbytes
switch (scale) {
case 0:
buffer.append(
" ");

break;
case 1:
buffer.append(
" K ");

break;
case 2:
buffer.append(
" M ");

break;
case 3:
buffer.append(
" G ");

break;
default:
buffer.append(
" M ");

break;
}

buffer.append(
")");

result = buffer.toString();
}

return result;
}
}

Tuesday, March 25, 2008

Signup time for ODTUG

After a bunch of work, the ODTUG agenda is finally posted and if you want to keep up on the latest coming in the Essbase world and have access to the people who know, this is the place to be. Here is a link to the conference agenda:

http://www.odtugkaleidoscope.com/hyperion.html

My friend Edward Roske has already blogged about the trial and tribulations of putting together the track. His post, "Best Essbase Conference Ever?", is a good read. Here is the link:

http://looksmarter.blogspot.com/2008/03/best-essbase-conference-ever.html

Finally, there is a Hyperion-specific registration page that has just been published by the ODTUG team at:

http://www.prairiesystemsgroup.com/odtug_hyperion_registration.html

Hopefully I will get some technical content posted here again in the next few weeks. I am off to Denver the week before Collaborate (again in Denver) for OBIEE training; I hope to post on some of what I learn there.

Until then, back to writing a custom Hibernate Dialect for DB2 (the as-shipped Hibernate DB2 Dialect doesn't create CLOB fields properly nor does it properly write a where clause if the column is an integer.. Arghhh!)

Monday, March 3, 2008

ODTUG Update

I have an update on the ODTUG Conference that I wanted to share with everyone. Edward Roske and I have been working hard on putting together a killer program that will attract people who are interested in learning, in-depth, what there is to know about Essbase. Edward has proposed a session schedule that just blows me away. It will be very unlike the Solutions Conferences over the past few years that really focused on customer success stories along with a moderated dose of 'what has recently shipped'. Rather, this conference will be all technical, period. We hope to get the sessions nailed down this week so we can make the agenda public.

Another thing I like about this conference is that, instead of the traditional golf outings that *used* to be a fixture at Solutions (years ago), this conference has a community service event on Saturday. Although the French Quarter and the Garden Districts of New Orleans have recovered relatively well, much of New Orleans is still reeling from the effects of Katrina. I haven't had the opportunity to get back to New Orleans since flying a couple of AngelFlight missions just after the storm and I look forward to helping again.

If you are an Essbase developer, mark your calendars for June 15-19 for New Orleans. Keep an eye on the website at http://www.odtugkaleidoscope.com/ for upcoming Hyperion announcements.

Monday, February 4, 2008

Oracle Developer Tools User Group Call for Papers (Act fast!)

Sorry I haven't had time to post anything for the last couple of weeks. I plan to start a series on how to install the Hyperion 9.3.1 Suite when I can get enough time to do the proper job of it. I have been working 'half-days' (12 hours, get it?), however, for the past couple of months and haven't had much time for anything. I did, however, force myself to go fly my airplane on Saturday. You know it hasn't been flying enough when, for the second time in a row, the battery was low and the plane needed to get a jump start. It had been a couple of weeks since it had been flown and I thought I had better make sure the battery was charged up before I fly over to Atlanta on Wednesday (tentatively) to go to the Hyperion User Group meeting. I bet you didn't know you could jump-start an airplane, did you?

I did, however, get an interersting email over the weekend from some of the officers in the the Oracle Developer Tools User Group ("ODTUG"). They have decided to pursue a Hyperion technology track at the annual Kaleidoscope 2008 conference scheduled for New Orleans June 15 to June 19. Hyperion-oriented people may want to attend the conference to see the latest on both Essbase technologies and OBIEE. This conference appears as though it will have content that has been been missing, for the most part, from the Hyperion 'Solutions' conferences over the past few years where there seemed to be slanted a bit more towards marketing than technical. Information for the Kaleidoscope conference can be found at

http://www.odtugkaleidoscope.com/


Although submissions for the ODTUG conference have been closed for a while, they just this past weekend reopened submissions for Hyperion related content. Submissions are scheduled to end February 16th. You can enter your submissions at

http://www.prairiesystemsgroup.com/odtug_hyperion_abstracts.html


I hope that many Hyperion experts are willing to share their knowledge in New Orleans.

Thursday, January 17, 2008

Why you can't use SQL to query Essbase

Over the years, I have seen many people ask why you can't use SQL to query Essbase and I have given it a bunch of thought. I saw a similar post today on OTN and decided to finally put it into writing.

Keep in mind that, despite the fact that I have been doing Essbase full time for the last 12 years or so, I did a bunch of SQL stuff before that. In fact, I spent a bunch of time back when I was an 'accountant' working as a technical lead on an Oracle Financials implementation. So here goes with the explanation.

Essbase is not accessible via SQL as the structure of a cube is quite different than that of a table.

If you think about what SQL does, it retrieves a dynamically determined set of rows each of which are specified by a defined set of columns. Conversely, when retrieving data from a cube (in the simplest case), you are retrieving a dynamically determined set of rows and a dynamically determined set of columns filtered by a 'slice' which are determined by "page fields" or headers (which is very comparable, conceptually to a SQL where clause). In the more complex case, you are retrieving sets of sets of dynamically determined rows defined by sets of sets of dynamically determined columns.

Another way of visualizing it would be to think about how you would structure SQL to select a range of *cells* in an series of spreadsheets, say range "C64:M400". That isn't too hard when you think about a single worksheet, but how about when there are 78 worksheets in your workbook that you need the same cells from? What if there were separate subdirectories on your server with the same workbook for all 563 sales centers in your company and you needed the same cells from all of those? Well, at this point we are essentially talking about querying 4 dimensions.. What if your company tracked 12 dimension? What would the SQL look like to get those results?

As you can see, the structure of multidimensional data doesn't lend itself very well to SQL. That being said, there is a 'SQL-like' language that was built to do multidimensional queries, "MDX". Here is a sample MDX query from the 9.3.1 APS java sample code:

SELECT
Union(CrossJoin({[Sales], [Profit]}, {[Actual], [Budget]}),
Union(CrossJoin([Total Expenses].Children, {[Actual]}),
{([Opening Inventory], [Variance]), ([Additions], [Variance %])}))
ON COLUMNS,
CrossJoin([200].Children, {[East], [West]} ) DIMENSION PROPERTIES [Product].Ounces
ON ROWS
FROM Sample.Basic
WHERE {[Jan]}";

MDX query functionality is exposed in the Hyperion/Oracle products primarily via the C and Java APIs and via web services using XML for Analysis ("XMLA"). I did not find any MDX related functionality in the VB API declarations file; that is probably due to the large number of structures and API calls that would have to be defined/supported combined with the fact that VB has a sunset that is fast approaching.

Tuesday, January 8, 2008

New Hyperion Blog Link

I have just put a new Hyperion blog link in my 'Cool Links'. It is the 'Look Smarter' blog written by my friend Edward Roske at interRel Consulting in Dallas. Edward has been working with Essbase since 'the world was flat', so to speak, and is also the author of the only book that has ever been published about Essbase. He also has the coolest office I have ever been in at 'The Rangers Ballpark in Arlington'; his office overlooks the outfield and, on the day I was there, the Rangers were playing the Detroit Tigers.

Edward's blog has some great technical information, particularly on the future of Hyperion products, and is entertaining as well. The blog is published at http://looksmarter.blogspot.com/

Monday, January 7, 2008

Advanced Security Manager

Oracle recently took down the HDN website and not all of the content, including third party downloads, is currently available on the OTN site. Of course, someone posted on OTN their need to get ahold of the OlapUnderground Advanced Security Manager. In response, I got in contact with the OlapUnderground team and they have sent me the Advanced Security Manager (along with the source code).. We are considering taking over development and management on the ASM but, in the meantime, I have made the Advanced Security Manager files available on our website:

http://www.appliedolap.com/olapunderground/downloads/temp/secmon1.zip

I will leave this download, as received from the OlapUnderground team, on our website until it is determined if it will be available on OTN or until another home is found for it. If we decide to take on management of the ASM development, we will create a separate page for it on our website.

I guess the question is this. How many people actually still use the Advanced Security Manager? Should we spend any cycles working it at all? Note: I have promised not to release the source code unless it is cleaned up a bit. Thoughts anyone?