Agile methodologies have seen wide adoption among development organizations in recent years. Business Intelligence (BI) teams have wisely joined this movement toward Agile. An Agile project, made up of many short “sprints”, is iterative by design. In building an analytical dashboard, for example, the challenge is to perfect how to tell a story with your data. So, having an iterative process with mechanisms for collecting feedback (Agile) is extremely helpful.

However, Agile alone does not guarantee project success. For the next BI project that you plan to run with Agile methodology, focus your team on four key areas to make your project run smarter and smoother.

1. Preparation is key. It’s all about being prepared for Sprint 1. Run a short “pre-project” to provide the framework for project success.
2. Database mindfulness. The database is central to any BI project. Early sprints should have a focus on database changes or data integration work to keep the database from being a roadblock to the UI/UX focused tasks.
3. UI/UX for user adoption. Start building the UI/UX component and seek feedback from the project owner & stakeholders both early and often. Take advantage of Agile and iterate on the UI/UX design to drive higher user adoption.
4. Application optimization.  Aim for conciseness. Eliminate extraneous information. Look for opportunities to combine visualizations and dashboards. Combine datasets if possible and eliminate unnecessary dimensions & metrics.

Read on for more details on how to incorporate these four keys areas into your Agile project.

Preparation is Key

“By failing to prepare, you are preparing to fail.” — Benjamin Franklin

To ensure the team will hit the ground running with sprint 1, the first step is to be prepared. The pre-project team will be smaller and typically includes the product owner, scrum master, database architect, and BI architect. Consider the following activities for the pre-project:

• User story brainstorming.  Hold a series of meetings to discuss the objectives of the project, who the users are, what are the sources of data, and key performance indicators. Review any existing system or reports that the new BI project will replace. Leverage these meetings to write an initial set of epics user stories for your backlog.
• Wireframing.  Define the main dashboards that are in-scope for your project (these will define your epics). At a high level, brainstorm a list of key dimensions and key performance indicators (KPIs) that are needed for each dashboard.
• Data integration planning.  The goal is to understand the scope of database development work that is needed to achieve the BI user stories and wireframes.
• Sprint planning.  Define an objective/goal for each sprint. The sprint 1 goal is key for the project kickoff. Expect that other sprint goals will change as the project progresses.
• Ceremony planning.  Set expectations with the team on the Agile ceremonies and time commitments expected during the project (daily standups, sprint planning, sprint retrospectives, sprint demos, etc). Schedule all meetings in advance.
• Agile Training.  Run training for team members new to Agile or who haven’t been on an Agile team recently. If possible, secure an Agile coach to support the first sprint or two.
• Infrastructure.  Ensure development infrastructure is ready to go before the start of Sprint 1. Secure access to a scrum tool and assign licenses to all team members.

While the above tasks are recommended to ensure a successful sprint 1, plan to maintain this momentum by doing user story creation, wireframing, planning in advance of every sprint.

Database Mindfulness

BI projects invariably require some database work in order to achieve the goals of your dashboard or visualizations. Many projects require a new database or have significant data integration requirements.

In the case of light database work, your team needs someone who can play the role of data architect, defining database subtasks for any UI/UX user stories that cannot be achieved by the existing database. Keep user stories as specific as possible so that work can proceed on components that are not dependent on the database change. Depending on the nature of the database change, these stories may span multiple sprints to allow time for both the data integration work and the UI/UX work. Consider using mock data or placeholders in order to keep your UI/UX work moving forward until the database changes are available.

If there is significant database work to complete, dedicate the first several sprints to build your foundational reporting data mart or warehouse (or integrating your new data sets into an existing warehouse). One of the biggest mistakes that we’ve seen is to try and do the BI development in parallel with significant database development. BI developers will largely spin their wheels without a database in place and loaded with representative data. It may make sense to delay adding the BI team to the project until they have a database to work with (though a BI architect will be useful during the database design and build to ensure compatibility with the BI tools and anticipated requirements). If you cannot defer the BI team members to later sprints, then focus these team members on to fleshing out the user stories with wireframing and prototyping during these early, database focused sprints.

User Interface/User Experience (UI/UX) for User Adoption

Iterative development is a key strength of Agile, so begin work on the user interface immediately and start to gather feedback. Don’t let data integration work stand it the way of progress; create prototypes using mock data. End of sprint demos should be built into your schedule to give opportunities to share progress on user stories with the project owner and stakeholders. Consider additional demos with the product owner mid-sprint if new UI or visualization concepts are ready for review. Be sure to set expectations early with the product owner that their engagement is required at the sprint demos and to encourage the attendance of other stakeholders who can provide feedback. Update user story acceptance criteria or add new items to the backlog to properly track the suggestions received at the demos. Expect to get UI/UX change requests with every sprint demo, so have an agreed-upon plan for when to freeze the UI/UX to meet deadlines for testing and production migration.

By creating your UI concepts early and seeking feedback often, the overall application has the chance to take form. This yields later sprints that are used to refine the overall UI/UX experience that drives higher user adoption.

Bonus Tip: If possible, include a UI/UX designer from the start of project to help craft professional looking dashboards. Most developers are not skilled in UI/UX design.

Application Optimization

Fast performance is a critical and achievable goal for any BI application. For blazing fast BI applications, first, strive for concise visualizations. Second, aim for as few datasets as possible, where those datasets are no larger than necessary. In the early sprints, nail down the BI application design and how to tell your data story. This will guide what visualizations, data elements, metrics, and filters are needed. Look for opportunities to combine dashboards/pages and eliminate any extraneous information from each visualization. Conciseness is key. Once the UI/UX design is crystalized, shift focus towards optimizing your datasets. Ensure you have the leanest possible datasets (remove unnecessary rows, data elements, and metrics). On-the-fly joins between datasets can be costly to performance, so it is important to consolidate datasets whenever possible. As each dashboard is completed, continue to revisit the datasets and look for opportunities to combine them or make them leaner.

High performance is critical for a successful BI application. Take advantage of Agile by focusing on optimizing your dashboards and visualizations in the early sprints. Use the later sprints to consolidate your datasets and make them as lean as possible.

Summary

The Agile development methodology can be a great fit for BI projects. To maximize your opportunity for project success, consider four key areas. One, run a “pre-project” to prepare the project team for sprint 1. Two, be mindful of the database tasks that are required and organize the project to minimize situations where the database hinders overall progress on your user stories. Three, emphasize the importance of building your user interface components as early as possible and seeking feedback as often as possible. This will help you refine your UI/UX and drive higher user adoption. Four, plan to optimize the user interface and the datasets, driving towards high performance.

D2Strategy provides our clients with the strategies, solutions, and cultural transformations to gain actionable intelligence from their data assets. Our differentiator is our innate ability to truly understand your complex business needs and translate them into the best fit and most cost-effective technology solution. Our data scientists and engineering team uses the latest technologies to deliver innovative analytics, BI, and data warehousing solutions.

Looking to get a competitive advantage?  Email Michael Haggerty at mhaggerty@d2strategy.com.