It’s been a while since I’ve posted here about the role of Agile Scrum in BI development teams and how you can be successful implementing Agile Scrum in your BI solution development teams. For a review, check out my post from almost exactly 1 year ago here.
That’s back when I was the BI Technology Director for Razorfish where we took a Big Data Analytics platform to market for online advertising analytics. Since then, Razorfish & Atlas were sold to Facebook & Omnicom and I moved back to the software vendor side of the business, with Penatho.
In both cases, we built BI and Analytics solutions using Agile. Pentaho is a bit simpler to implement because we are building Java applications with teams of developers who are already familiar with working in Agile teams and environments.
At Razorfish, we were really employing teams of BI developers: ETL developers, data modelers, cube developers, report designers and data scientists. Not only are many of those roles not comfortable naturally in an Agile environment, many of these roles have worked for years in their career previously working solo, not even in teams at all.
BI engineers are indeed developers. You do not have to write code in Java, .NET or C++ to be a “developer”. An engineer writing ETL and models is writing code, testing, working from requirements and troubleshooting. That means that Agile can work and it also means that “BI Developer” is a role that is very important, deserves the same level of respect and can be very difficult and tedious at times. Give some love back to your BI Developer!!
Sprint Reviews CAN work in BI / Analytics solutions and are VITAL to your success. In the Java development world, this may seem like a no-brainer. But I run into a lot of resistance when building Analytics solutions using Scrum. Many folks believe that building a data model, ETL, analytics, reports and demoing all of that in a 3 / 4 week iteration is not possible. It is possible, I have worked in this environment for years and will take you out of that “go off and build-it” approach to business intelligence that has killed many BI projects over the years. Sprint Reviews are your friend!
Like Loading...
I’ve been following these posts and am hungry for more tips and details on exactly how one can harness the more tangible benefits of Test Driven Development (TDD) in a Data Warehousing / BI project.
My vision is to build a foundation of TDD tests that will become the regression testing for the BI solution we are providing to an Investment Reporting group, but there isn’t alot of precedent to follow, so any hints you have would be greatly appreciated!
Enter your comment here...
Notify me of new comments via email.
Blog at WordPress.com.
Paras Doshi's Blog on Analytics, Data Science & Business Intelligence.
Databases et al...
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
