Outrageous..

New Zealand is a beautiful country and we had a wonderful time on the weekend visiting the Coromandel Peninsula.  I am now at the NZOUG conference, but I am outraged and need to let you know one thing in case you ever come here to visit.  Leave your internet devices at home and don't plan to communicate unless you are prepared to empty your pocket. 

I am staying at the conference hotel, The Pullman, where internet connectivity is $28.50/day/device.   I was told I could upgrade to have access to the 'The Club' level for $40/person/day and that I would have unlimited internet.  Given the fact that Darlene and I have 3 devices between us, and for the free snacks and drinks in the Club level, I decided I would upgrade us.  

When I connected, it seemed strange that I was being prompted to login and agree to the $28.50/day even though I had the 'Club' status, so I went to the front desk.   I was getting charged for both the club and the daily rate.  What?  Apparently, the clerk at check-in told me about the free internet in the 'Club' level but failed to mention that the internet was free only when you were on their computers in the 'Club'.  So, with my devices *and* the 'Club', I was being charged $165/day for internet access.  That is actually more than I paid for the room!  On top of that, I had paid an extra $300 this month just to have phone/3G service on my iPhone in New Zealand.   The hotel manager did agree to cancel my 'cCub' level access and remove those charges which was a nice gesture, the the idea that we are still paying nearly $90/day for internet access that drops when you move from room to room has left a bad taste in my mouth.

On a side note, on our first day in New Zealand, we stayed next door at The Quadrant hotel.   I liked the rooms better and they had free internet.  Next time, I will stay there the entire time. 

OK..  I feel better now..   Besides, Edward Roske is here with his wife; he told me they have 4 devices!

The Limits of SSPROCROWLIMIT are History

At Open World, in light of the announcement of Exalytics, I suggested during a panel discussion that the limit for SSROWPROCLIMIT be raised.  In case you are not familiar with the setting, let me tell you about it and its implications.   SSPROCROWLIMIT is an Essbase config file setting that controls the maximum number of spreadsheet rows Essbase processes on a Spreadsheet Add-in request. It is used only when suppress missing rows is turned on.  The rows subject to the limit are counted before suppression.

The problem with this setting is the small number of rows allowed, as the default value is 250,000 and the maximum value is 500,000.  That probably wasn't a bad limit when this setting was introduced in the very early days of Essbase.  At that time, I was running a fast Pentium machine with 1 Mb of RAM and 2 1Gb SCSI drives.  At that time, 500,000 rows was huge.   Now, with Exalytics and 1 TB of RAM which is, if I did the math correctly, more that a million times more memory than my old Pentium, the same 500,000 row limit seems quite confining.  At the same time, Essbase outlines keep growing.  I have a customer who has 380,000 members in one dimension in a BSO database!  If they tried to zoom in, suppress missing, on this and any other dimension, Essbase would need an SSPROCROWLIMIT of 760,000 rows if the second dimension had only 2 members.   Further, the row limit enforced by this setting considers the number of rows before suppression. This means that it doesn't matter whether the query actually returns more than 500,000 rows or just a single row after suppression, the limit is used internally during the suppress missing processing.

I sent a quick email to the Essbase program manager after Open World and, like magic, he let me know that the latest Essbase version, 11.1.2.1.102, has removed the upper limit.  Sure enough, I checked the readme.html for the new patch and here it is:

New Maximum for SSPROCROWLIMIT

(11730559)

The Essbase.cfg setting SSPROCROWLIMIT no longer has a maximum value of 500,000. The maximum number of spreadsheet rows Essbase processes on a request can now be set to 16,384 or higher. It is not recommended to use a limit higher than 500,000.

I haven't tried it yet, but I am sure I will in the near future.   I know a number of customers who will be excited about this change.

Guest Blog Entry for ODTUG

I have written a guest blog entry about my upcoming trip to speak at Oracle conferences in New Zealand and Australia.  In the post, I discuss a couple of my plans for the trip and talk a bit about the sessions I am going to present.  The blog entry now posted at http://odtug.wordpress.com/2011/10/20/tim-tows-heading-to-australia/.

Dodeca 6 is Here!

We have been working very hard over the past year and are pleased to announce we are ready to release Dodeca, version 6.  In past major releases of Dodeca, we have focused on a theme and this release is no different.  The theme for Dodeca 6 is high performance.

The performance enhancements encompass changes to both the Dodeca server and the Dodeca client.  The server changes include a major refactoring of both the Dodeca and the Dodeca-Essbase server components.  The changes to both servers include:

• Implementation of a new, object-oriented framework that makes it easier for us to add new functionality in the future.  It also makes it easier for customers and partners to extend the server functionalities of Dodeca.
• Significantly improved performance reading and writing the xml used to communicate with the client.  In fact, in the case of large documents, xml read/write performance has been improved by more than 2000%.
• A new logging infrastructure that allows customers to better understand the internal operations of the servers.  Each server has both a timed log and an untimed log.  The timed log allows customers to see internal timings in the server which can be very useful for performance tuning.

Additionally, to both improve performance and to add flexibility for dynamically generated Essbase templates, we have also added relational caching of the Essbase outline.  Though this capability is optional, we think it is a nice new functionality.

The client also has some impressive performance enhancements.  These enhancements include:

• Significantly improved performance reading and writing the xml used to communicate with the Dodeca-Essbase service.
• The new Accelerator that can improve the overall performance of our .NET client by up to 400%.
• We have also added some new functionality that we think users will love.  For example, Dodeca now features:
• Multi-level Essbase undo/redo.
• An Essbase Unknown Members explorer
• Customizable Excel AutoCalculate to display sums, averages, etc of a selected range in the status bar.
• New options for inserting/deleting rows, columns and worksheets.
• Over 10 new workbook script methods and functions.  The new functions and methods enable Dodeca workbook scripts to call a custom web service, to execute custom code on the Dodeca-Essbase server, to get the dimension name represented at a given member cell and to obtain a GUID from the operating system, among other features.

The performance we are seeing in Dodeca is quite astonishing.  We have a number of customers who have the desire to retrieve large amounts of data from the Essbase server.  Oracle, with its new ultra-fast Exalytics hardware, validates this customer requirement by producing hardware to support large and fast Essbase operations.  Yesterday, I did a zoom-in during a webcast demo that returned nearly 190,000 cells of data; it was about 9,500 rows by 20 columns.  The operation completed in about 2 seconds.  The means Dodeca was able to make a request to Essbase, Essbase retrieved the data, and the data was returned into the spreadsheet, at a rate of approximately 95,000 cells per second.

I also was doing some stress testing a few months ago and did a large zoom-in operation.  Here is a screenshot of the spreadsheet after the zoom.

This retrieve, where the grid size was 803,530 rows by 9 columns, processed through our server in just over 42 seconds.  How do I know that?   I could see it in the Dodeca logs:

The items circled in the log indicate the grid size before the zoom-in, the grid size after the zoom-in and the milliseconds to complete the entire transaction.  Since this screenshot was taken, we have done further work on the logs to separate items that have sizes/times associated with them and things that don’t have associated times.  Further, the default formatting for the timed logs use pipe-delimiters between the fields so the log can be imported directly into a relational database, or even into Excel, for further analysis.

I have covered a bunch of things in this blog entry but there are a bunch of new things in Dodeca 6.  When we began work on the servers, I put in many, many 80+ hour weeks researching and working on conceptual designs for the server.  I was telling someone recently about my experiences during that time.  Basically, I didn’t shower for days at a time and my hair gets greasy after 1 day, so I wore a toque (or stocking hat).  I wore one of my many hooded sweatshirts, kept my headsets on and was generally anti-social.  My family barely got a word in and most of my friends at Starbucks didn’t even recognize me.  The person to whom I was telling this was another programmer and familiar with the concept.  He said to me, “So, you went into Unabomber mode!”  I guess you could call it that.  For everyone who reads my blog on a regular basis, perhaps you can see why my posts have become a bit scarce over the last year.

In summary, this post talks about a few of the key aspects of Dodeca 6.  I will delve into some of these topics deeper as time goes by, but for now, I will leave this post with a bit of a puzzler for you.  Below is a screenshot of one of the new samples created using the out-of-the-box functionality of Dodeca 6.   This sample view features write-back to Essbase, but it breaks the rules of Essbase a bit.   Tell me, what are at least 3 of the normal Essbase rules we broke and how did we do it?  I will send a free Dodeca T-Shirt to the first 5 correct answers posted in the comments below.

Note that I have to approve the comments before they show up in the comments, so I will know who the first 5 people are.  For those of you who have already seen this demonstrated, you are ineligible to win but, then again, you probably already have a bunch of Dodeca T-Shirts.  Winners need to send me email once you have been identified as a winner and include your name/address/size so I can mail the shirt.  Also, if you want to win a shirt, use your real name when you post the comment as I don’t want to try to manage sorting out who the real ‘anonymous’ is.

So, how did we make this work?

Using the Java API to Logout Users From a Server

There was a question on the Network54 board today regarding the Java API and logging off all users from a given application.   I quickly wrote up a quick (but untested) bit of Java API code as an example, but decided to post it here as Network54 mangles the formatting of code examples.   Without further ado, here is the code in a more readable format:

void disconnectUsersOfApplication(IEssOlapServer server, String applicationName) throws EssException {
    // get the connections to the server
    IEssIterator connections = server.getConnections();

    // loop the connections
    for (int i = 0; i < connections.getCount(); i++) {
        // cast to a connection info object
        IEssOlapServer.IEssOlapConnectionInfo connection = 
         (IEssOlapServer.IEssOlapConnectionInfo)connections
         .getAt(i);

        // if the connection is to the target app
        if (connection.getConnectedApplicationName()
         .equalsIgnoreCase(applicationName)) {
            // log them off
            connection.logoffUser();
        }
    }
}

Essbase Strategic to Oracle Fusion

We have been hearing that Essbase is a core technology in the Oracle Fusion strategy, but what is Fusion and how does Essbase fit?

In general, Fusion is the next generation set of applications from Oracle. Oracle has acquired a number of companies over the past several years and each company, like Hyperion, brought compelling functionality to the Oracle family.  Each product they acquired, however, presented significantly different user experiences.  Under the covers, the acquired products were also significantly different as they were built on a variety of technical frameworks.

In Fusion applications, Oracle is bringing together the world class applications they have acquired into a single suite.  Fusion is not simply a marketing slogan nor is it simply a creative packaging of the acquired products.  Instead, Fusion is a completely new product line that takes the best ideas from the acquired products and incorporates them into a single, rationalized product.  These products have been rewritten from scratch using a single UI platform, the Oracle Application Development Framework ("ADF") which is an industry standard Java technology that runs on Oracle Middleware.  Oracle ADF delivers a standard set of components which are available to customers and partners and when customers want, or need, to extend the Fusion applications, the extension will be written in the same ADF Framework that Oracle uses to build the products.

So, how does Essbase fit in?  I had previously heard that Essbase will be incorporated into the Fusion Financials General Ledger as the standard allocation engine.  I was thrilled to hear this week that the Fusion Financials General Ledger will also automatically spin off Essbase analytic cubes.  This is great news for all of us that love Essbase as it places Essbase into *every* Oracle customer.  With all of the love Oracle has given Essbase this year, this has been the best Oracle Open World ever!

Oracle Annouces the Exalytics Business Intelligence Machine

Larry Ellisom opened Open World with a keynote on the theme of Extreme performance and focused on how the Oracle Exadata and Exalogic servers have hardware and software engineered together for performance.

The Exa* line feature both parallel hardware and software.  That means the servers, network, storage, vm's, OS, database and middleware all run parallel with no single point of failure.  The Exa* line uses a Hybrid Columnar Compression algorithm to compress the data in memory.  The computers also use Infiniband network connectivity within the machine which provides a 10x performance increase over TCP/IP.  Combining the compression with the network performance gains, Oracle is claiming 100x performance increases.

Likewise, the Exa* line has massive amounts of RAM inside the machine.  How much?  You guessed it.  10x more.  Combining the 10x compressed memory with the 10x more RAM means 100x more data in memory.

In summary, the Exa* line is fast.  Oracle cited results from customers including BNP Paribas where the data warehouse ran 17x faster with no other changes.

So, what does this have to do with Essbase?  Larry Ellison also announced the new Exalytics Intelligence Machine.  The marketing bulletpoints on this announcement included extreme in-memory analytics, speed of thought analysis, and instantaneous business intelligence.   As many of you remember, Speed of Thought Analysis was one of the Arbor or Hyperion marketing slogans; it is great to see it back!

The details of the machine, though, are really cool.  The hardware features include:

  1 Tb of RAM (which could contain 5 to 10 Tb of compressed data)
  40 cores (4 x 10 Core Intel Xeon CPUs)
  Disk read rates of up to 200 Gb/sec
  40 Gbps Infiniband internal network
  1-10 GBps ethernet external network

Exalytics isn't just hardware.  It comes pre-installed with software as well:

  BI Foundation Suite: OBIEE
  In-memory parallel Analytics
  In-memory parallel Essbase
  In-Memory Parallel TimesTen

So, what the heck is In-memory parallel Essbase?   The short answer is that it is Essbase tuned to work with in-memory data.  So, what does that mean?  They have implemented an adaptive in-memory cache that decides what get stored in memory.  The algorithm adapts to change to the workload and can auto-tune or be tuned by administrators.

What I have learned talking with my sources is that the Essbase server engine has a number of new optimizations that enables it to better utilize large amounts of memory.  Apparently, these optimizations will also be in the upcoming 11.1.2.2 version of Essbase regardless of whether Essbase is running on the Exalytics hardware.  In other words, the improvements will benefit all Essbase customers but the improvements can be more fully leveraged on the Exalytics hardware.

Thomas Kurian, who is the Executive Vice President of Product Development at Oracle, introduced the details of Exalytics in his keynote.  At the ACE Directors meeting last week, we were honored to have Thomas speak to our group where we were briefed on Exalytics.  I have heard Thomas speak a number of times since the Hyperion acquisition and, in my opinion, I think Thomas is truly one of the smartest people in the world.  He is responsibile for hundreds of products yet, last week, I heard Thomas answer a question about support on a specific virtual machine.  Not only did he have the answer, but he referenced a specific device driver file. Wow!

Oracle had an Exalytics machine on display outside the keynote.  Here is a picture of the Exalytics machine (in the foreground; that is an Exadata database machine in the background). 

So, yes, I actually got to touch the new machine.  Oracle didn't announce availability or pricing on the machine.  Of course, I am sure your friendly neighborhood Oracle sales rep would be happy to answer those questions for you.

On the screen in the background, Oracle was displaying the new OBIEE running against Essbase.  It appears they have implemented much of the same type of functionality that was previously available in the Hyperion Visual Explorer add-in for Excel.  This type of visualization is very helpful for spotting patterns in large datasets and I expect it will be very popular.

I look forward to working on an Exalytics based system soon.  I know a number of customers who look forward to having one of these systems.