Agile (BI) in the context of large enterprises & corporate governance

It has been quite a while since my last blog post. The major reason for this is that my company IT-Logix decided to start its own blog. So I wrote a few articles, all of them in German. Therefore if you are interested in this post in German, you’ll find it here.

At IT-Logix we’re tackling the Agile topic for more than three years now. The main question is how you can apply the agile principles in a BI/DWH environment. You cannot “create” agility directly. It is much more a side effect of a combination of methods, concepts and technologies which are all agility oriented. The starting point is often – this is true for me too – the Agile Manifesto. The values and principles written down in the Agile Manifesto describe the “mindset” which is the most important component on your journey to more agility. Because no method, no concept and no tool solve any problem if the involved people don’t get the “sense” behind it.

If someone hears “agile” they typically think about Scrum. Scrum is a project method which has been around in the software development space for quite a time and now emerges more and more in other  domains – including BI and DWH projects. The circumstance that many people associate Agile and Scrum directly show that the inventors of Scrum were established a pretty good marketing around it. Still, you should consider three things here:

  1. What is called Scrum in certain organisations often is far away from the basic idea of Scrum, namely the agile mindest. Michael James described this recently on Twitter as follows:
    Anti-Agile
    This often leads to frustration – an amusing blog post puts it in a nutshell: http://antiagilemanifesto.com/
  2. Alongside the strong marketing of Scrum we find a proprietary terminology. An iteration is called a sprint, a daily coordination meeting “daily scrum”, a ToDo list becomes a backlog etc. This brings the danger of understanding and hence communication problems between stakeholders inside and outside the project organisation, especially senior management.
  3. Scrum is only one out of multiple agile practices – and for itself alone not always the most suitable method regarding its application in the BI/DWH environment.

Looking at my customer I experience again and again aspects which contradict an iterative project approach. Even just the project setup requires a project request with estimated efforts, a project plan and milestones. From an architectural perspective one needs to consider an already existing overall architecture, typically data models and systems which are already in place. During implementation often multiple teams of the organisation are involved, there are a lot of dependencies and therefore (unexpected) waiting times. How can you incorporate these surrounding conditions into a specific project method and still embrace the values of the Agile Manifesto?

In autumn 2014 I got to know the Disciplined Agile Delivery (DAD) framework of Scott Ambler. In my eyes Scott and his co-authors very well manage to bridge the gap between Scrum as a pure development method and the needs of an enterprise environment. Thereby DAD doesn’t reinvent things but provides a structure to put different methods in a bigger, overarching context.

DAD provides different project lifecycles. The basic “Agile” lifecycle contains many elements out of Scrum but resigns to use a proprietary language. An important aspect in addition is the inception phase where the necessary coordination with the “big picture” can happen.

Iterations a primarily based on a time box approach. That means you fix the time frame and vary the scope of an iteration in a way that it fits into this time slot. Depending on your environement this can become pretty tricky. Especially if other teams from which you depend do not follow this (time boxed) approach. That’s the reason why DAD provide further lifecycles, e.g. the lean lifecycle:

The basic structure remains the same, but the time boxed iterations are missing. Thereby the flexibility increases, but equally the danger of getting bogged down. At this place we need a certain discipline – the name “Disciplined” Agile Delivery has its roots in this reality.

To better get to know DAD we invited Scott Ambler for a two days workshop at IT-Logix by end of January 2015. As with Scrum, you can take a first certification level after such a workshop. Therefore I can proudly call me a Disciplined Agilist Yellow Belt ;-)

To sum it up: Once more one will need a good portion of “common sense” and not method devoutness. What works in practice counts. The DAD framework gives me and our customers a meaningful guard railing to realise the added values of an agile mindset. At the same time DAD is open enough to care for the individual cutomer situations. My own contribution is to fill the described, generic processes with BI specific aspects and apply it in practice.

Event recommendation 1: As a member of the Zurich Scrum Breakfast Club I meet with other agile practitioners and interested people. The event is organized in the OpenSpace format, more infos about it here. Interested in joining us? Let me know! I can bring along a guest for free.

Event recommendation 2: Join me during the sapInsider conference BI2015 at Nice by mid of June: I’ll explain how I applied the DAD framework to a SAP BusinessObjects migration project. More information about my session here.

To end this blog post a few impressions of our DAD workshop with Scott Ambler:

Foto 5 Foto 4 Foto 2 Foto 1

 

How to speed up connection between Translation Manager, Universe Designer and CMS

At a recent customer project my teammate and I were facing an issue with importing Webi documents and universes into Translation Manager (all on BO4.1 SP4) getting the following error message:

org.apache.axis2.AxisFault: Unable to find servers in CMS, servers could be down or disabled by the administrator (FWN 01014)

There is a KB article describing this error and a solution: http://service.sap.com/sap/support/notes/1879675

One hand we followed the instructions in the KB article and specified the hostname for each server. Although our server is multihomed (that means has more than one network interfaces) we didn’t thought about this before hand because the second network interface goes into a backup LAN and is never used for communication with e.g. the client tools. In addition everything worked fine so far – until we got the issue with Translation Manager. Still, just setting the hostname was not enough. On the server side there is the Windows firewall enabled, therefore we had to assign static request ports to several services. We already did so before our issue with the Translation Manager for the CMS, Input & Output FRS, all Webi Processing Server. We used TCPView to analyze on which ports Translation Manager was opening connections. As long as there were requests on ports which we didn’t specify in the CMC (some random port number e.g. 56487) we narrowed down all the services which Translation Manager establishes a connection. We had to specify a port for all of the following servers:

  • The Adaptive Processing Server (APS) hosting the DSL-Bridge Service
  • The APS hosting the Client Auditing Proxy Service
  • The APS hosting the Search Index Service
  • The APS hosting the Translation Service
  • The WACS (Web Application Container Server)

Besides the issue with Translation Manager we had another issue with creating new universe connections in the Universe Design Tool. When creating a new connection we had to wait up to 30 minutes ( ! ) to get to the wizard page where you can select from the list of available database drivers. Still, after 30 minutes everything worked fine and could create the connection successfully. Based on our experience with Translation Manager we run again TCPView and found that we need to assign a port number to both connection servers (32 and 64 bit) in CMC. Having done this, creating a new connection now works without any waiting time.

After all: If you have firewalls between your BO server and client tools, just assign a port to all servers available and open the port on the firewall (the only exception might be for the Event Server as I’m really not aware of any communication between this server and a component outside the BO server)

Optimizing my dashboard: Creating a visual draft

Remember my last post about the Webi dashboard? As mentioned at the end of that post aimed to show a technical trick to put an interactive kind of a button onto a Webi report. Now this post is the first in a series of posts how to optimize the layout of the initial dashboard. Let’s start with creating a draft of our optimized dashboard layout. The advantage of such a draft is that it is not yet implemented with the actual BI tool but using either pen and pencil or a wireframing tool. I did the later and chose the cloud edition of Balsamiq. To get a quick start you can use a 30 days trial account.

Let me explain briefly how the tool works. After that I’ll explain some of my thoughts behind the chosen layout.
Within the editor you can drag and drop sketched objects like a window container, rectangles, text, buttons etc.

Balsamiq_Editor

My dashboard in the Balsamiq editor

Looking at the available chart objects, I’m not really satisfied:

Balsamiq_Charts

Default chart elements in Balsamiq

Therefore I added my own images representing typical IBCS chart types (IBCS stands for International Business Communication Standards. I wrote a short blog post about IBCS here.). The images are based on the graphomate add-on for SAP DesignStudio and BusinessObjects Dashboards.

IBCS_charts

Typical IBCS chart types (created with graphomate)

After all, my inital dashboard draft looks like this:

Page01.jpeg

My dashboard draft (chart view 1)

At the top you can see some reserved space for an appropriate title. Providing this is a major requirement stated by IBCS as one of the top ten proposals:

I adapted this to a BI specific title concept where I distinguish between general title elements (like the organization or global query filters) and object specific titles, e.g. for the table or a chart. For the table I used the default element of Balsamiq, maybe I will update this later on with an IBCS optimized one. For now it is just a placeholder.

The charts in this first view will show current year values and previous year values (where as the current year will be indicated within the global filters area) for revenue and margin. To make the analysis of available data more straight forward, I decided to add two variance charts, one for absolute values and one with percentage values. Again, this is one of the top 10 elements in IBCS:

You might have discovered the symbolic button to switch the chart view. The second view looks like this:

Page02.jpeg

My dashboard draft (chart view 2)

The header and table areas stay the same. In the lower area with charts I now show historical values for revenue with actual and plan values. Instead of putting everything into one big chart I decided to use small multiples for the top 5 product lines (based on total revenue over time) as well as one chart for all other product lines. Depending on how it will look like in Webi we might decide to show more product lines or add another topic to the dashboard (as we still have space left on the bottom right corner).

In this blog I showed how to create a simple dashboard draft using a wireframeing tool like Balsamiq. In addition I pointed out how to apply two of the top ten IBCS proposals in this conceptual phase.

Update May 5, 2015: Internally at IT-Logix we continued the implementation of the above shown mockup; we used both Design Studio and Web Intelligence. Unfortunately the time for writing a blog post was missing so far. Still, if you are interested in the Web Intelligence part, have a look at the following sapInsider BI2015 session held by my teammate Kristof Gramm: http://bit.ly/1EGKFtA He will show how you can do pretty amazing things with Webi (by the way, using this link you’ll be granted a 300€ discount on the conference registration ;-)

A (Webi) dashboard built by a business (power) user

This blog post is inspired by a recent customer request to challenge their decision to use Design Studio for some “dashboard requirements”. Showing how you can create a dashboard in Webi doesn’t mean I told the customer not to use Design Studio. Much more it is to show that finally a dashboard as well as every other type of BI front end solution is made up of requirements and not primarily by the tool you build the solution. Please refer to my Generic Tool Selection Process for more details as well as my post regarding BI specific requirements engineering.

Having said this, let’s have a look at how we can use latest Webi 4.1 features to quickly build an interactive dashboard without the need of (much) scripting. First of all here is what the final result looks like:

01_DashboardOverview1

You can select values from the left side bar (Product Lines), you can select States by directly clicking into the table and you can switch from the bar chart to a line chart. Here you see it in action:

The first step to achieve this, is to create the basic table and the two charts. Until the dynamic switch is implemented, I placed them side by side. Next add a simple input control in the left side bar:

02_SimpleInputControl 03_SimpleInputControlDepend

Next thing is to define the table as an additional input control – right click the table and choose “Linking” and “Add Element Link”,  choose the two chart objects as dependencies:

04_TableAsInputControl 05_TableAsInputControlDepend

Next we need to create the “switch” to toggle the two charts. As I would like to position this switch at the top right corner of the chart, I again use a table input control. To generate the two necessary table values (namely “Bar Chart” and “Line Chart”) I prepared a simple Excel spreadsheet:

08_ExcelContent

In 4.1 you can now finally upload this sheet directly into the BO repository:

07_UploadExcel

If you need to update the Excel sheet later on, this is now feasible as well:

09_UploadExcelReplace

Finally, in Webi add the Excel sheet as a second query:

10_ExcelQuery    10_ExcelQueryDetails

In the report we need now two tables: A visible one to represent the chart switch and a (hidden – see the “Hide always” option) dummy table to act as a dependency for the first:

13_HiddenDummyTable  12_HideDummyTable

The most tricky part is to create a variable to retrieve the selected value:

15_VarSelectedChartType

Here the formula for copy / paste:

=If( Pos(ReportFilterSummary(“Dashboard”);”Chart Type Equal “) > 0)
Then Substr(ReportFilterSummary(“Dashboard”);Pos(ReportFilterSummary(“Dashboard”);”Chart Type Equal “) + Length(“Chart Type Equal “);999)
Else “Bar Chart”

(The idea for this formula I grabed from David Lai’s Blog here)

Finally you need to configure the hide formula for both charts:

16_DynamicallyHideChart

That’s it.

Conclusion

Positive: I’m not too technical anymore (I do more paperwork than I wish sometimes…). Therefore I don’t consider me a “developer” and I like solutions for the so called “business (power) user” more and more. Therefore I like Webi. It took me about 60 minutes to figure out how to create this kind of interactive dashboard. I didn’t need to install anything – I could do everything web based. Except for one single formula (which I didn’t need to write myself)  I could click together the above sample. And I dare to say it looks like some kind of a dashboard :-) In addition I have all the basic features of Webi like a broad range of data source support, plenty of export possibilities, Office integration and so on. Even integrating an Excel spreadsheet as a data source is now finally a no-brainer.

Negative: Clearly, Webi is not a “design tool”. For example I wasn’t able to show icons for my chart switch instead of the text lables. Putting a background image to the table doesn’t work well if the table is used as input control. When I discussed this prototype with the customer they also mentioned that there are still too many options end users might get confused with (e.g. that there is a “filter” section showing whether the Bar Chart or the Line Chart value is chosen). In Webi you can’t change that. Toolbars, tabs etc. are just there where they are. Live with it or choose a different tool.

Bottom line: Have a look at my Generic Tool Selection Process and the mentioned hands-on test. The above example is exactly what I mean with this: Create a functional prototype in one or two tools and then do a fact based decision depending on your requirements and end user expectations.

Important remark: This post focused on the technical aspect of the dashboard. The visual representation doesn’t yet fit to best practices mentioned in my earlier articels (e.g. about SUCCESS) In a next blog post I will outline how to optimize the existing dashboard in this regard.

Join my teammate Kristof Gramm during sapInsider’s BI2015 conference at Nice (June 16-18): He will go into much more details about how you can use Web Intelligence as a dashboard tool for business users. Use this link to see more infos and save 300€ on your conference registration!

Getting to Know the German Monster – The Importance of Proximity in a Globalized World

Just recently I published a blog about the following topic on sap.com

Thanks for reading!

IBCS – an emerging standard for business #dataviz

In my previous post I introduced the SUCCESS model of Rolf Hichert. In this post I’d like to introduce you to the subject of IBCS – the International Business Communication Standards. IBCS’ aim is to “foster the level of understanding in reports and presentations” (Source: IBCS). Currently, IBCS consists of two main parts:

Notation of meaning
The part “Notation of meaning” describes basically the semantics of a standardized business communication language. It covers all aspects of meaning in the context of business communication and suggests an appropriate notation.

Design of components
The part “Design of components” covers rather the syntactical aspects of a standardized business communication language. It describes the basic report elements and specific rules to use them for the design of objects such as tables and charts. Several objects and additional elements make up complete pages. Although this part should not consider aspects of meaning, but only define the “grammar” of a unified communication language, some overlap to the “Notation of meaning” (semantics) is inevitable. (Source: IBCS)

IBCS and SUCCESS are closely related, therefore I will start with some explenations about how the two topics are positioned against each other. First of all SUCCESS and IBCS can be looked at from a time perspective. Clearly, SUCCESS was developed first:

Folie1

From this point of view IBCS  is a refinement of certain aspects of SUCCESS (namely the Unify part in SUCCESS). This refinement can be seen in the following perspective too:

Folie2

SUCCESS provides only generic guidelines without concrete implementation details, e.g. the following one:

UnifyStandardDimensions

Souce & Copyright: HICHERT + PARTNER

This guideline just tells you: If you visualize this year’s revenue with blue in chart number 1, you should also use blue in chart number 2. But the guideline doesn’t tell you to always use blue. If you wish to take brown for current revenue, it’s OK. Just use brown whenever you visualize current revenue. This and many of the other guidelines (some of them are mentioned in my previous blog post) in SUCCESS I therefore attribute to what I call “common sense” or data visualization basic recommendations. Now IBCS comes into play. Whereas SUCCESS is rather generic IBCS defines the details and writes them down. As indicated above IBCS does this in mainly two parts:

– the notation of meaning clearly defines which elements have which meaning:

UnifyStandardDimensionsIBCS

Source: IBCS

This way business communication should be simplified through standardization the same way as you are used to it with geographical maps for example. Blue always means water, north is always at the top of the map etc. This “semantic layer” is something which differentiates IBCS from other data visualization and information design concepts. Most if not all of them focus on generic recommendations only. In contrast, IBCS wants to harmonize the visualizations in a business context and make them therefore easier to understand.

– the second aspect of IBCS is the design of components

Let’s have a look at the SUCCESS guidline first, e.g. for chart design:

Source & Copyright: HICHERT + PARTNER

Source & Copyright: HICHERT + PARTNER

IBCS gives you much more information, e.g. regarding legends:

LegendsIBCS

Source: IBCS

You can look at IBCS and SUCCESS from a third perspective too:

Folie3

In this perspective IBCS is the solid base with all the detailed rules to consider for efficient data visualizations in business communications. Yet the IBCS rules are hard to digest (as every other industry standard…) SUCCESS can be seen as an implementation methodology of them (as IBCS and SUCCESS are mostly congurent to each other). SUCCESS is an acronym of seven verbs – if you act on them, you’ll see that implementing IBCS is pretty straight forward.

IBCS is work in progress and it is open-source (based on the Creative Commons BY-SA license). Its further development is orchestrated by the IBCS Association which again is run by HICHERT + PARTNER. Simply create a login on ibcs-a.org and start to add your own ideas to further refine and extend the standard.

If you want to learn more about IBCS, SUCCESS and how it can be implemented, join me during the BOAK conference this autumn. It takes place on Tuesday, September 16th in Zurich / Switzerland. In the data visualization track (sessions E1 to E4) you’ll find several sessions dedicated to IBCS. Jürgen Faisst, CEO of HICHERT + PARTNER will elaborate on the goals of IBCS. My friend Lars Schubert will demonstrate how you can apply IBCS using the IBCS certified software “graphomate” as part of SAP Design Studio or SAP BusinessObjects Dashboards. I myself will present together with a customer showing how we implemented an IBCS oriented design in Web Intelligence. The day after I will teach these best practices during a one day workshop. A sneak peek on the Webi reports you can find on my Hichert Certified Consultant page. Looking forward to seeing you during the BOAK conference!

What do you think about IBCS and the semantic layer? Do you think it is worth the effort to harmonize data visualizations in a business context? Just add a comment now!

Simple guidelines for more efficient report & dashboard design

It is already for more than three years that I’m engaged with the topic of business information design. It was during a regular Crystal Reports project. We were about to finalize the IT concept phase and the project was on track. Crystal Reports was already selected as the tool to go forward with. When suddenly something happened: The sponsor of the project, the company’s CFO, asked us if we can implement what he called a “Hichert chart”. I found that Hichert is a German business information design specialist and at first sight he had some weird ideas regarding chart design. So weird we couldn’t really implment them with Crystal Reports and – long story short – we couldn’t finish the project as the CFO’s requirement regarding Hichert charts was too strong. Not even other tools within the BusinessObjects portfolio were able to fullfil it. So the customer finally chose an Excel based solution…

Of course I was annoyed but at as well curious. How powerful this Hichert stuff must be in order to turn a project upside down? This was the start of my journey into business information design. According to Wikipedia this field is about the following:

“Information design is the practice of presenting information in a way that fosters efficient and effective understanding of it. The term has come to be used specifically for graphic design for displaying information effectively, rather than just attractively or for artistic expression.” (Source)

In this and future blog posts I’d like to illustrate how you can achieve this more efficient understanding of data and information in the context of Business Intelligence.

There is already a lot of helpful content available, let me name just a few sources I use in my daily work:

For this article I’d like to introduce the SUCCESS model of Hichert. SUCCESS is an acronym for the following verbs:

Say – Unify – Condense – Check – Enable – Simplify – Structure

Every verb corresponds to a collection or category of design guidelines. Most of these guidelines are commonly accepted best-practices – and you find them as recommendations not only with Hichert but with many of the above people. The only difference with SUCCESS is, that the recommendations are put into about 120 concrete, numbered and illustrated  guidelines of “Dos & Don’ts” regarding information design. This very structured approach makes it very easy to get started with the topic. You can find everything summarized on the SUCCESS poster:

SUCCESS Poster

SUCCESS Poster (Source & Copyright: HICHERT+PARTNER)

You can download your own copy here.

Let me briefly explain what each category is about:

Say: This is about having a message to tell or in general about meaningful content in your reports and dashboards. A good example is the following guideline:

Replace Traffic Lights

Replace Traffic Lights (Source & Copyright: HICHERT+PARTNER)

Another important, yet pretty easy to implment guideline tackles the matter of having a clear title concept which enables the reader to quicker grasp what he or she is looking at:

Title Concept (Source & Copyright: HICHERT+PARTNER)

Title Concept (Source & Copyright: HICHERT+PARTNER)

Unify: This category of guidelines is about the statement “what looks the same should mean the same” and the other way round. The following guideline shows suggestions how you might unify the look of tables and charts – for now the important part is not yet what the chart or table looks like but that charts and tables always follow the same design pattern.

Unify Tables & Charts (Source & Copyright: HICHERT+PARTNER)

Unify Tables & Charts (Source & Copyright: HICHERT+PARTNER)

Condense: Condense is about increasing the information density on a given report or dashboard page / screen. A few simple but powerful guidelines:

Use empty space (Source & Copyright: HICHERT+PARTNER)

Use empty space (Source & Copyright: HICHERT+PARTNER)

Use Small Multiples (Source & Copyright: HICHERT+PARTNER)

Use Small Multiples (Source & Copyright: HICHERT+PARTNER)

Check: This category is about ensuring quality. One topic for example is to choose an appropriate chart type and the usage of propre scales:

Choose appropriate chart type (Source & Copyright: HICHERT+PARTNER)

Choose appropriate chart type (Source & Copyright: HICHERT+PARTNER)

Don't cut axes (Source & Copyright: HICHERT+PARTNER)

Don’t cut axes (Source & Copyright: HICHERT+PARTNER)

Enable: The guidelines in the Enable category are not about information design itself but how to best introduce the guidelines of the other categories into an organization:

Present alternatives (Source & Copyright: HICHERT+PARTNER)

Present alternatives (Source & Copyright: HICHERT+PARTNER)

Create a Rollout Plan (Source & Copyright: HICHERT+PARTNER)

Create a Rollout Plan (Source & Copyright: HICHERT+PARTNER)

Simplify: This category brings together a lot of guidelines which are all about avoiding noise and other distracting elements in the report and dashboard design:

Avoid meaningless elements (Source & Copyright: HICHERT+PARTNER)

Avoid meaningless elements (Source & Copyright: HICHERT+PARTNER)

Avoid 3D charts (Source & Copyright: HICHERT+PARTNER)

Avoid 3D charts (Source & Copyright: HICHERT+PARTNER)

Omit Long Numbers (Source & Copyright: HICHERT+PARTNER)

Omit Long Numbers (Source & Copyright: HICHERT+PARTNER)

Structure: Similar to the Say category, Structure is more about the content itself than its visual representation. The guidelines here describe how to group data:

Exhaustive Structures (Source & Copyright: HICHERT+PARTNER)

Exhaustive Structures (Source & Copyright: HICHERT+PARTNER)

Mutually Exclusive Structures (Source & Copyright: HICHERT+PARTNER)

Mutually Exclusive Structures (Source & Copyright: HICHERT+PARTNER)

 

Show Structures (Source & Copyright: HICHERT+PARTNER)

Show Structures (Source & Copyright: HICHERT+PARTNER)

Most of the above shown guidelines can be implemented straight away with most BI tools including SAP’s Web Intelligence and Crystal Reports. During my upcoming blog posts I’d like to look more closesly at several further aspects both from technical as well as conceptual perspective. This will also answer the question why we had troubles implementing “Hichert charts” in the customer project mentioned at the beginning.
In the meanwhile I’m happy to get your feedback what you think about SUCCESS and to learn about your experience regarding information design in the context of Business Intelligence.

Follow

Get every new post delivered to your Inbox.

Join 657 other followers