Agile Business Intelligence Maturity Model

As outlined in my previous blog agility in business intelligence projects can’t be produced directly. Instead you should invest into professionalism, standardization and automation. In this post I’m showing an overview of concrete building blocks to support you on this way.

In my Agile Business Intelligence Maturity Model (ABIMM) I’ve collected many building blocks and arranged them in a practical sequence. An overview you can find in the following illustration:

Agile Business Intelligence Maturity Model

Agile Business Intelligence Maturity Model

We can extract a few key messages from this model:

  1. You can’t increase agility directly – you can only reduce the amount of needed upfront design. By doing this agility is increased automatically.
  2. A reduction of upfront design leads inevitably to higher risks – risks you need to deal with actively, e.g. by using a version control system or solutions for test automation (cf. my blog post here). As long as such basic infrastructure elements aren’t available you should be very cautious with introducing iterative, incremental procdures like e.g. Scrum. (A very illustrative presentation about Agility requires Safety you can find here)
  3. All beginnings are difficult: The building block “Agile Basics & Mindeset” represents an enormous hurdle in many cases. As long as an organization doesn’t experience a top down transformation towards agile values and principles (cf. e.g. the Agile Manifesto), it doesn’t make much sense to start with it bottom-up.
  4. The gulf can be overcome by buying the necessary tools for test automation, version control and training for employees. This can typically happen within the boundaries of the already existing infrastructure. But to overcome the chasm, todays often heterogenous, multi layered BI tool landscapes aren’t suited very well. That’s one reason why I’ve become a big fan of data warehouse automation and tools like WhereScape. Products like WhereScape RED institutionalize the usage of design patterns in an integrated development environment. Only for this reason e.g. refactoring on the level of the data model and hence iterative data modeling becomes feasible with realistic effort. At the same time tools like WhereScape provide you with an ultra high degree of automation for the deployment of new and changed artefacts.

A more detailed explanation of the Agile BI Maturity Model can be found in my recent article in the German TDWI journal “BI-Spektrum“.

Here you find the English translation of my article!

Many thanks to my company IT-Logix and all the great staff working with me. You are the indispensable foundation for all my BI related work. Don’t forget, you can hire me as a consultant 😉

Following you’ll find the literature list on which the different building blocks and the model itself is based on:

[AmL12] Ambler Scott W., Lines Mark: Disciplined Agile Delivery: A Practitioner’s Guide to Agile Software Delivery in the Enterprise, IBM Press, 2012

[AmS06] Ambler Scott W., Sadalage Pramod J.: Refactoring Databases: Evolutionary Database Design, Addison-Wesley Professional, 2006

[Bel] Belshee Arlo: Agile Engineering Fluency http://arlobelshee.github.io/AgileEngineeringFluency/Stages_of_practice_map.html

[BiM] Memorandum für Agile Business Intelligence: http://www.tdwi.eu/wissen/agile-bi/memorandum/

[Col12] Collier Ken: Agile Analytics, Addison-Wesley, 2012

[CoS11] Corr Lawrence, Stagnitto Jim: Agile Data Warehouse Design: Collaborative Dimensional Modeling, from Whiteboard to Star Schema, DecisionOne Press, 2011

[Hug12] Hughes Ralph: Agile Data Warehousing Project Management: Business Intelligence Systems Using Scrum, Morgan Kaufmann, 2012

[HuR09] Humble Jez, Russell Rolf: The Agile Maturity Model – Applied to Building and Releasing Software, http://info.thoughtworks.com/rs/thoughtworks2/images/agile_maturity_model.pdf, 2009

[Kra14] Krawatzeck Robert, Zimmer Michael, Trahasch Stephan, Gansor Tom: Agile BI ist in der Praxis angekommen, in: BI-SPEKTRUM 04/2014

[Sch13] Schweigert Tomas, Vohwinkel Detlef, Korsaa Morten, Nevalainen Risto, Biro Miklos: Agile maturity model: analysing agile maturity characteristics from the SPICE perspective, in Journal of Software: Evolution and Process, 2013 (http://www.sqs.com/de/_download/agile_maturity_wiley_2013_final.pdf)

Parts of this blog have first been published in German here.

Advertisements

7 Responses to Agile Business Intelligence Maturity Model

  1. Do you see Data Vault fitting in your framework as an iterative modeling methodology?

    What about the use of cloud-based data warehouse platforms & BI tools – do you see those as enabling technologies?

    • rbranger says:

      Hi Kent, thanks for your comment. Indeed, during my presentations around ABIMM I position Data Vault in the frame of “Identify & Apply Design Patterns”. It is by the way a design patterns which shows well how standardization drives automation.
      Cloud-based data warehouse platforms (e.g. like Snowflake 😉 are really helpful in the area of agile infrastructure. Think about cloning developer sandboxes (one for each developer), separate testing systems (for test automation for example) or running an A/B-setup for your production system – as far as I worked with cloud-based infrastructure this is much easier to achieve (although it can become expensive in the end). Still wondering how you can best do continous integration and push button deployments. In cloud environments scripting possibilities are not always as flexible as with certain on-premise tools. Can you share experience on this from your side?
      Cheers
      Raphael

      • Thanks. Glad to have someone objective confirm what I have been thinking regarding Data Vault and Snowflake in relation to agile (and automation). The fast clone feature in Snowflake is definitely helping customers be more agile from a DevOps perspective,

        The best example I have at the moment of push button deployment is in fact with WhereScape. They showed me an initial demo of deploying a data warehouse and loads to Snowflake last week. So that, as you know, is a push button from the WhereScape UI.

        Other than that I have not gotten any details of specific examples from the field but do believe we have a few customers working on it. I expect they are using Python scripting. (Snowflake itself is really software in the cloud, and we are an agile shop doing CI internally and doing regular upgrade deployments to AWS – but I have no idea what our engineers are using to do it.)

        I do have one colleague who has developed a product (using Jenkins in part) that does CI with an OBIEE repositories. I will have to ask him if they have deployed that with cloud-based targets yet.

  2. Great to see WhereScape moving ahead with Snowflake! Looking forward to test it on my own soon.

  3. Pingback: AgileBI: How Corporate culture influences the development approach | My life as a BI consultant

  4. Pingback: AgileBI workshop London | My life as a BI consultant

  5. Pingback: My life as a (BI) consultant: Update Spring 2017 | My life as a BI consultant

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: