For reference I’ve used the three basic setups for an Application (These aren’t the only three).

• Single Threaded
• Multithreaded (Using Locks for Synchronization)
• Multithreaded (Using data structures which support Lockless Concurrency)

These can be loosely related to TM1 like so:

• TM1 8.4 and below - Single Threaded
• TM1 9.0 to 9.5 - Multithreaded (Using Locks)
• TM1 9.5 and above - Multithreaded (Lockless)

When the first TM1 version came out which supported Multithreading we heard the crowd roar “It’s slower and it uses more memory!” and even now with what I assume to be a Lockless model (Parallel Interaction) we hear the same. So let me explain what is happening here and in which way the new multithreaded environment is faster.

Theoretical Users vs. Processing Time

Take a moment to digest the chart above. The first thing you’ll notice is that without any kind of multithreading (aka Single Threaded) the processing time increases in a linear fashion with the user load.

Lesson 1: Stripping aside all caching and performance enhancing techniques, Two users performing the same action in a single threaded environment will take twice as long as one user would. (Pretty straight forward huh?)

Now take a closer look at the multithreaded lines. You’ll see that they handle multiple users better (in that the trend is no longer so steep), but you can also see that they incur a base processing time penalty in servicing even a single user.

Lesson 2: Multithreading typically incurs its own cost to processing, but unlike the cost of Single Threaded Actions this cost does not (often) increase with additional user load.

With any mechanism, the algorithms are not as straight forward as I have made them appear here. Specific methodologies or Bad Coding can incur their own additional performance degradation.

I would love to hear from IBM on this one as to how close to the mark I’ve hit.

Class dismissed – any questions?

Software Installation

CubeSpy Setup & Configuration

Software Features

Software Appearance

TM1 Operations Console

TM1 10.1 Operations Console - Home Screen

CubeSpy

CubeSpy Home Screen

CubeSpy 1.5.9 - User Load Charts x2

CubeSpy 1.5.9 - Event Viewer

CubeSpy Monitor

CubeSpy is a TM1 Monitoring application which supports many convenient administrative features such as message broadcasting and a 24/7 User Log. Additional information and a Trial version can be downloaded here.

1. Open a command prompt and run the following (updating paths to match that of your own environment).

Command Prompt - Navigate to your TM1 JRE Directory

Command:

keytool -import -file "C:\Program Files\ibm\cognos\tm1\bin\ssl\applixca.pem" -keystore "C:\Program Files\ibm\cognos\tm1\bin\jre\6.0\lib\security\cacerts" -storepass "changeit"

Command Prompt - Enter "yes" and hit Enter

2. Start “IBM Cognos TM1 Applications”. To do this simply open “IBM Cognos Configuration”, right click “TM1 Application Server” and click “Start”.

3. (Optional – If you are going to create a specific group of users in TM1 who can access the Operations Console) Open a TM1 Client such as Architect or Perspectives and create the group.

4. Update the configuration file for the Operations Console called “OpsConfig.xml” with your adminhost, server name and security group (Possibly created in step 3).

Note: the path to this file will be similar to:

C:\Program Files\ibm\cognos\tm1\webapps\tm1operationsconsole\WEB-INF\configuration\

TM1 10.1 Operations Console Config File

Note that at this point the official guides tell the user to turn off Javascript Debugging and Caching in Internet Explorer. I’m not going to do this as:

a. The Javascript Debug Prompt only pop’s up when a error in the web site has occurred. (Is IBM sweeping bugs under the rug here?)
b. I don’t want to slow down web browsing by disabling caching for every site I visit just for one application.

5. Browse to the TM1 Operations Console: http://YourServerName:YourPort/tm1operationsconsole

TM1 10.1 Operations Console

6. Once you have logged in:

• Click the “Add Operation Group” Button and enter a Name.
• Right click on the Group and click “Add Adminhost”
• Right click on the Adminhost and Click “Add TM1 Server”

If you navigated the labyrinth successfully, you can now click on the newly added TM1 Server and view a list of connected users along with their activities.

Please note that Turbo Integrator and the Advanced Rules editor both haven’t changed when accessed through Perspectives.

Note: This will be a two post Review. In this post we will simply show you what data we used and how it all looks – Step by Step.

Exercise Sales Data: Sales Sample Data.xls

You can download it above if you want to step through this post with me.

Sales Data File

First lets launch Insight and create a new Workspace. Its at this point (if you’ve also just finished installing TM1 10.1) that you realize IBM really like their splash screens.

IBM Cognos Insight

Once you drag the sales challenge data onto Insight you’ll notice it is giving us a break down of each customer vs. each other. While not unexpected this doesn’t give us any actual Insight … Yet but simply pivoting the results around you can view Pivots such as “All Product Sales by Sales Channel” or “All Countries by Sales Channel”.

Cognos Insight - After Importing Sales Data

Cognos Insight - After a quick play around

Once you are familiar with the Workspace its time to cleanup the Dimensions and Fix up a couple of data misinterpretations (Such as Customer ID and Name as two separate Dimensions) and build a Cool Sales Dashboard.

So there you have it. A Dashboard that could have been made in 10min if I hadn’t been trying so hard to get under the hood and see how it all works. What I don’t show here is the dimension editor, rules editor and all the bells and whistles which let you do mappings, aliases and hierarchies etc on the fly. Stay tuned for the reviews conclusion.

Prerequisites:

• Microsoft Visual C++ 2005 SP1 Redistributable Package
• Microsoft .NET 3.5 SP1 Framework
• Microsoft Chart Controls for Microsoft .NET Framework 3.5.
• Microsoft Excel

Getting Started:

To kick of you’ll need either an installation CD or Archive. When downloading directly from IBM’s website you’ll be using a “.tar.gz” archive. This can be extracted with winZip/7Zip or WinRAR.

TM1 10.1 Installation Directory

Within the Installation directory you should find the file called “issetup.exe”. Open “issetup.exe” It you are using a version of windows which has UAC (User Access Control) enabled (such as Windows 7) run the setup as an Administrator.

Installation Page 1

Installation Page 2 - Terms and Conditions

Installation Page 3 - Software Location

Installation Page 4 - Software Features & Components

When selecting software features and components note that Cognos Insight and Performance Modeler are both not installed by default. You will need to tick these individually for them to be installed. Match your window to the image above to ensure they will be installed.

Installation Page 5 - Shortcut Folder Title

Installation Page 6 - Installation Summary

Installation Page 7 - Installing Components

Installation Page 8 - Success!

TM1 10.1 Installed Successfully. Check that the installed applications exist within the Start Menu like so:

Below is a high level overview of the components which make up IBM Cognos TM1 version 10.

TM1 10.1 - Application Overview

The brand new components here are:

1. IBM Cognos Insight – Enables instant ad-hoc detailed analysis of data files.
2. IBM Cognos Performance Modeler – Used in building new analysis models.
3. IBM Cognos TM1 Operations Console – A new web based frontend for TM1 Top (and competition for CubeSpy)

Problem:

Although some additional components of TM1 have built-in email functionality, TM1 does not have email functionality built into its ETL Engine. Text emails will not suffice, you want to employ HTML encoding in your email to provide a more ascetically appealing email.

Solution Process:

1. Within a TI Process output text to a file to be used as the body of the email to be sent.
2. Execute a email script command to send off the email to recipients.

Executing the TI script:

Email script is parameter driven and will prompt you for parameters to be pushed into the TI process:

• pRecipient : yourName@yourCompany.com.au ; nextName@yourCompany.com.au ( note use a  semi column between each email address when emailing multiple users.
• pSubject:  add a comment.
• pBody: add a comment .
• pAttachFile: file location of where your file attachment resides.

Email script global variables will need changing before execution of script:

• sFileName: file location of generated txt file used as the body of the email to be sent.
• sSender: notifications@yourCompany.com.au ( who the email is from)
• sSMTPSvr: SMTP or Exchange Server used for emailing. This is often SMTP.yourCompany.com.au ( SMTP server )

Solution:

Screenshot of Email: (TM1EmailHtml.zip)

HTML Email alert

Notes:

• Due to 255 character limit restriction within TM1 (prior to version 9.4), instead of emailing individuals you should set up a email group instead of separating each email address separately i.e. use emailGroup@yourCompany.com.au instead of yourName@yourCompany.com.au ; nextName@yourCompany.com.au.

Often when writing dimension update processes from a datasource which is updated automatically we use a standard procedure.

1. Delete all the Elements
2. Add Top Level Consolidations
3. Cycle through (Metadata) Adding Elements and Element Components
4. Cycle through (Data) Adding Aliases

But in some cases we are only given a partial data feed.  For example we may be reporting off 3 years sales but the datasource for the Product dimension is broken into a file per year. If we were to use the DimensionDeleteAllElements() function within the first process (which imports the first years products), the additional products from subsequent years will not exist until the next process is executed – The data associated with the subsequent year’s products will be lost when those products aren’t added into the Dimension immediately as part of the same Process/ Chore Execution.

Note: You can still use DimensionDeleteAllElements in the first of three processes without losing data if the processes are always executed together in a Chore – but this is reeks of potential for user error.

In the same example, if we were to skip the DimensionDeleteAllElements() and an element moved from one consolidation to another (from a change to the current years file). As there is no cleanup part to the overall process, the product will reside under both consolidations producing garbage data.

The Solution:

The solution is to break down the structures/hierarchies within the dimension without deleting any elements instead of calling the DimensionDeleteAllElements() function. To do this the TI Script needs to loop through each element in the dimension and remove each child. While explained simply enough in Pseudo code, Loops in Turbo Integrator are avoided by most people. So here is a standard script you can use – plug and play. All you need to do is define the variable “DimName”

e.g.
DimName = 'Product';

#Unwind the dimension we are updating
iElm = 1;
ElmCount = DIMSIZ(DimName);
WHILE(iElm < ElmCount);
    sElm = DIMNM(DimName,iElm);
    ChildCount = ELCOMPN(DimName,sElm);
    WHILE(ChildCount > 0);
        sChildElm = ELCOMP(DimName,sElm,ChildCount);
        DIMENSIONELEMENTCOMPONENTDELETE(DimName, sElm, sChildElm );
        ChildCount = ChildCount - 1;
    END;
    iElm = iElm + 1;
END;

The Bedrock Equivalent:

Bedrock provides a simple solution here in a single line.

EXECUTEPROCESS('Bedrock.Dim.Hierarchy.Unwind.All',
                                'pDimension','Product','pDebug',0);

We have an allocation cube with the following dimensions:

• Financial code (10 n elements)
• Year (2008, 2009, 2010, 2011, 2012)
• Month (12 months plus YTD consolidations)
• Scenario (actuals, budget rounds)
• Allocation phase (7 phases, populated by Turbo Integrator)
• Entity (combined with country in 1 dimension)
• Cost center (several hundreds)
• Product (several thousands)
• Relation_entity (the counterpart dimension of Entity, for eliminations in a Reporting cube)
• Relation_cost center (the counterpart dimension of Cost center, for eliminations in a Reporting cube)
• Account (several thousands)

As you can tell, this is a heavy cube. It contains actuals and budget data, for very detailed analysis dimensions. Several millions of data points are loaded using TI in a “base” phase. After that other processes will populate the other allocation phases. No rules are used for allocations. Lookup cubes containing percentages aid us in carrying out the allocations.

The reporting cube is similar, and goes as follows:

• Product type (a Text attribute on the Product dimension)
• Financial code
• Year
• Month
• Scenario
• Allocation phase
• Entity
• Cost center
• Product
• Account

The reporting cube is rules-based. The rules differ for allocation phases. For certain phases, we want to carry over the respective data from the allocations cube. For other phases, however, we want to eliminate certain activity within the (multinational) group: this will be both at the Cost center level and at the Entity level. Therefore, we added the counterpart dimensions for Cost center and Entity (see above): they contain the same elements, but with a prefix to ease the eliminations.

This is the logic to be followed (Cost center = CC, Entity = E):

• For an n-level CC and E, we show the respective values from the other cube;
• For an n-level CC and a consolidated E, we show the total activity in that CC and E, EXCLUDING the activity in that CC between the n-level E’s of that Group E;
• For an n-level E and a consolidated CC, we show the total activity in that CC and E, EXCLUDING the activity in that E between the n-level CC’s of that Group CC;
• For consolidated E’s and CC’s, we have a double elimination. We show the total activity in that E and CC, EXCLUDING all activity in the n-level E’s and CC’s among themselves.

I hope you are still with me; the most difficult part of the article is finished by now. Given the above logic of eliminations, we need to use TM1 calculation rules. As far as I know, there is no solution with consolidation structures and TI processes to achieve the same result. The chosen selection for Cost center and Entity determines what eliminations are done. If the structure of the Cost centers or Group entities changes, the eliminations will change as well.

Judicious use of the 2 counterpart dimensions in the allocations cube (and a prefix for element names) allows us to use generic and relatively simple rules. We carry over a number from the allocations cube, and subtract either 0, 1 or 2 consolidated values: 0 subtractions for case A above, 1 subtraction for cases B and C, 2 subtractions for case D.

But then feeding these rules… This poses a big challenge for Cost centers and Entities: every single number in an allocation phase would feed a big number of consolidated Cost center AND consolidated Entity elements. The cartesian product is there and the result is that the cube explodes as the data volume of the allocation cube grows. An 8 GB RAM server machine quickly became insufficient. The model went out of memory when top consolidated values were retrieved in the Reporting cube. This is obvious.

What exactly did the trick to feed the cube rules? Since data loads in the allocation cube are monthly without manual data entry, it was not a big problem to copy over the base data from the allocations cube to the corresponding n-level cells in the Reporting cube. By doing this, the existence of the data at n-level serves the purpose of feeding the rules at C-level. We even skip the need for a rule at n-level Cost center and Entity because the copy process takes care of this. We needed to “feed” the consolidated values so that they show up in Execute Viewer (Advisor in the IBM Cognos Express suite) and to be able to use zero suppression. The model went from an estimated 10 GB RAM memory and server load times of more than an hour, to about 800 MB RAM memory and 1 minute of load time.

Kudos to my colleague Luc Cusiel for coming up with this nice solution.

Author:

Wim Gielis (currently working for Aexis Belgium)
http://www.wimgielis.be, and choose “TM1″
wim.gielis@gmail.com