4 Must-Know Principles for BI Project Success

THE 4 PRINCIPLES

Over the years, we’ve found that much of the conventional wisdom about BI, data management, and reporting technology is flawed, and sometimes even downright backwards. We’ve distilled these hard-won nuggets of wisdom down into four best practices:

1. It’s not about the tech (the BI mistake almost everyone is making)

2. Your front end is only as good as your back end

3. Beware technical debt

4. You have weeks, not months

 

Some of these principles may seem a little mysterious right now but don’t worry. We’ll cover each of them in detail.

 

PRINCIPLE 1: IT’S NOT ABOUT THE TECH

People who work in BI are passionate about new technology. After all, it’s the tech that makes BI possible in the first place. But here’s the kicker: In our experience, the biggest driving factor of great BI results is NOT the technology being used, but the underlying human structure. That’s right! People power still trumps tech.

If you’ve been in the BI game long, chances are you or someone you know has had the experience of paying through the nose for expensive/shiny visualization tools only to discover that the reports are still plagued by data chaos (how much more humiliating and painful can it get?!) We all want to believe the utopian promises of total reporting automation, but reality is reality. Your needs are too specific and too unique for there to be a one-size-fits-all tech solution. It’s that simple. Reporting tech can do wonderful things for you, but you have to have your underlying human structure setup first or you’ll be throwing away your money/reputation.

 

Avoiding The Tech Trap

So how do you do it right? How do you avoid the tech trap? Good human structures are built on four pillars: vision, standards, self-reliance, and results.

Vision: It’s essential that you create a BI vision, then get the rest of your organization to “buy-in”. This is the sales component of your job. You may be averse to thinking of yourself as “selling” something, but it’s critical that you communicate and describe what the future looks like and why your BI solution makes it awesome. It needs to be tangible. As you sell your vision, remember that most of your listeners are (almost exclusively) interested in what’s in it for them and their teams. Speak in terms of benefits and positive outcomes, not technical specs or crazy jargon. For example, “These are the decisions you’ll be able to make using data once it’s easy to analyze…” Or “You’ll save [x] hours per week once the process is automated…”

Standards: Part of your vision should include common best practices for how your data warehouse will be built, used, and maintained. These ground rules are called “standards”, and there are nearly countless plusses for having them. Short-term, standards make sure your results are consistent and repeatable. Long-term, standards reduce your coding workload and keep your data warehouse from becoming dependent on the knowledge of individual developers. Standards also play a critical role in reducing technical debt, but we’ll save that topic for a future email…

Self-reliance: Gone are the days that BI managers could slough off responsibilities like database administration, server management, and data extraction (e.g., API calls, NoSQL databases, etc.) to the IT department. Relying on another department to manage your database or server creates an unnecessary point of failure. And if your system performs poorly, users don’t blame the system admins, they blame the BI team (you). That’s no bueno. Good BI managers get the people they need on their team then take on ownership of every point in the BI process, from data extraction to reporting. This gives you more power to enforce standards, minimizes points of failure, and makes sure you’re not held responsible for other departments’ flubs!

Results: Keeping your BI vision alive and maintaining momentum requires delivering real, tangible, measurable results to the rest of your organization. In (another) future email, we’ll go into more detail on strategies for managing and prioritizing your BI reporting objectives so you can deliver results that matter at a pace that keeps everyone happy.

Don’t get us wrong, picking the right technology stack for your company is important, but it simply won’t matter whose dashboard’s you’re using if you don’t get your vision, standards, and processes lined up first. For example, a guitarist can have the best rig money can buy, but his ability to actually sound good still depends on his personal skill, not the guitar. As important as tech is, it’s only part of the BI equation.

 

PRINCIPLE 2: YOUR FRONT END IS ONLY AS GOOD AS YOUR BACK END

A major problem in BI today is that visualizations are getting all the attention. There is no shortage of companies offering flashy dashboards and one-size-fits-all solutions. We counter with our own Xerva mantra: Your front end is only as good as your back end.

To illustrate our point, imagine a restaurant with a great atmosphere, great service, but terrible food. Obviously, this is a recipe for disaster. Atmosphere and service are important, but if you don’t get what’s happening back in the kitchen right first, it’s all for naught.

We like this restaurant analogy because we’ve found it’s a really handy way to visualize the relationship between your data warehouse (the kitchen in the back where the food is prepped) and your reporting

technology (the dining area up front where the food is presented). Also, we were hungry when we wrote this email. Anyways…

The point is, you can have the best visualization tech money can buy, but it’s all useless if your data isn’t in order. The good news here is that most of the crazy reporting problems in the front end can be fixed by building a good kitchen (i.e., a good data warehouse). Fix the data first, and the rest will be much more likely to fall into place.

 

BUILDING A GOOD “KITCHEN”

So how do you build your back end? A good data warehouse always…

Provides “One version of the truth”: The idea here is that everyone in your organization is able to go to the same officially “blessed” or sanctioned source (your data warehouse) to get the information they need. This way only data that has been cleaned, conformed, and had accurate business logic applied to it is used to make business decisions. For example, by defining all the logic around customers into just one customer dimension table, you help your business by creating consistency. Plus, when the logic changes, you only need to make one update in one place to make sure the reports that use that customer dimension also use the new logic.

Meets the needs of the whole business: Often BI managers get so caught up in delivering a few mission critical metrics to top executives, that they forget to meet the needs of the rest of the company. You can do better! With a good project management system in place, you should be able to meet the data reporting needs of your entire company. If you fall short of this goal, departments will start throwing their own reporting solutions together, and your “one version of the truth” will quickly be engulfed in data chaos.

Is adaptable to emerging business needs: The needs of your business will change over time, so build your data warehouse and reporting systems to be flexible. For example, embed as little logic in reports as possible. Instead, keep your logic defined centrally in the data warehouse. It also helps to apply business logic as late as possible while building your data warehouse. This allows you to avoid dependencies cropping up after your business logic is applied.

Building a good data warehouse that has the three features above can be a daunting task, but using the principles in our previous email (vision, standards, self-reliance, and results) to establish a good underlying human structure, along with some helpful strategies we’ll be sharing in our next email, you’ll be well on your way to getting the job done!

 

PRINCIPLE 3: BEWARE TECHNICAL DEBT

We can’t over emphasize the importance of getting the data right first. That being said, sometimes BI managers fail to anticipate the huge amounts of technical debt (the ongoing cost to maintain a technology solution beyond its implementation) that will be incurred by their data warehouses.

Sometimes, we want to believe that if we can just nail the setup, the data warehouse won’t require any more attention. We’ll be free, along with our team, to move on to other projects and everything will be rainbows and kittens forever.

The hard reality (at least for the foreseeable future) is that no matter how well a data warehouse is built, it will require time and effort to maintain. Code breaks, bad data slips into the system, or unforeseen possibilities necessitate refactoring.

If the BI manager is unprepared for these inevitable issues, developers can get locked in or “married” to a project in a never-ending cycle of fixes and patches that suck resources and kill company buy-in.

BI managers who are unprepared for these inevitable issues are soon buried alive along with their teams by technical debt. It is not a happy place to be! Developers get locked in or “married” to a project in a never-ending cycle of fixes and patches that suck resources and kill company buy-in. Eventually it becomes impossible to keep up with all the maintenance your data warehouse needs and suddenly your data is a mess again! Fortunately, it doesn’t have to be this way.

 

HOW TO AVOID TECHNICAL DEBT

While some technical debt is inevitable, in can be kept at a manageable level with a little foresight. Here are a few handy tips:

Plan for it. Simply knowing that your data warehouse will require some upkeep is huge. Instead of getting blindsided like BI novices, let your organization know what to expect and budget some of your team’s time for routine maintenance.

Develop in phases (optimize later). Imagine trying to build a house (roof, walls, plumbing, etc.) all at once, instead of in phases. You’d end up with a total mess that was 100x more expensive and time consuming to build. The same principle goes for building data warehouses. Plan out your data warehouse construction process in phases so that you can start with nailing the foundational stuff (accuracy and consistency), then move on to fine-tuning and optimizing later.

Use standards. As you hopefully remember from the first email in this course, standards are ground rules for how your data warehouse will be built, used, and maintained. Standards are the number one way to reduce technical debt which is why having them is so critical!

Establish an approved tech stack. A “tech stack” is the combination of software/applications you use to store, analyze, and visualize your data. Having a uniform tech stack for your entire organization slashes technical debt by reducing the number of applications you and your team need to work with. Keep your tech stack pure by requiring any tech stack additions or changes to be approved by you first.

 

PRINCIPLE 4: YOU HAVE WEEKS, NOT MONTHS

Starting out, your first job is to sell your BI vision to the rest of your organization. After all, gathering data from all your different departments and synthesizing it into one accessible source requires coordination, cooperation, and buy-in at almost every level.

If you do it right, your organization will be excited and optimistic about the picture of the future you’ve painted, but if you fail to deliver in a timely manner, you’ll find yourself losing momentum and consensus. Once that happens, you’ll find yourself fighting an increasingly uphill battle.

Believe it or not, this is the number one reason BI projects fail to take off: failing to deliver results fast enough.

Our rule of thumb is that you have weeks, not months to deliver results before people start to lose faith in your vision and try out other solutions. When other departments “go rogue” it makes you look bad, cripples your ability to deliver “one version of the truth”, and exacerbates the problems already going on in “the kitchen”, not to mention the tangled mess it will create down the road (often you’re the one who has to maintain their rogue decision).

 

BUYING TIME

You can buy yourself time by getting key indicators out to your company ASAP. Start by creating a prioritization matrix. A prioritization matrix lists and organizes business processes according to their importance and difficulty.

Use the prioritization matrix to build a short-term and long-term roadmap for your project. Publish the roadmap where everyone in the business can see it. Revisit the roadmap on (at least) a quarterly basis. The roadmap should include a 1-year plan, but also span 2+ years into the future. Go for the low hanging fruit first (things that your org wants most and that are easy to deliver on). This way the tasks that add the most value for the smallest amount of effort get done first.

When you have a plan for your whole company and everyone gets it, executives will be more likely to help prioritize needs. You may even find other departments voluntarily give you budget so you can tackle their data problems for them!

Getting the data accurate, organized, and modeled correctly the first time around is way easier than putting it off and playing catch up later. Manage expectations by being honest about the time and resources you’ll need to get the job done.

 

WRAPPING UP

That’s it! You’ve learned all 4 key BI principles in this blog post:

1. It’s not about the tech (the BI mistake almost everyone is making)

2. Your front end is only as good as your back end

3. Beware technical debt

4. You have weeks, not months

 

We hope some of the insights we’ve shared will be helpful to you. Good luck with your next BI project!