How to Select the Best BI Tool for You

Post by 
Reading time: 5 min

This is a series of articles about how to think about a BI tool evaluation and selection, and how to create a weighted scorecard to guide your decision.

Series Outline

Functional Requirements

Functional Requirements is a category of capabilities that tools have or do not have, or rather, it’s a categorization of what you’d like your users to be able to do. This is likely the category of requirements that your Business Users are most interested in.

Top 11 Functional Requirements for Your BI Tool Evaluation

  1. Advanced Analytics
  2. Augmented Analytics
  3. Dashboarding and Data Visualization
  4. Data Management
  5. Data Querying
  6. Embedded Analytics
  7. Geospatial Visualizations and Analysis
  8. Internet of Things (IoT) Analytics
  9. Mobile BI
  10. Platform
  11. Reporting
Technical Requirements

Technical Requirements is a category of capabilities about how the tools are deployed, accesses, and secured. This is likely the category of requirements that your IT and Engineering Teams are most interested in.

Top 6 Technical Requirements for Your BI Tool Evaluation

  1. Availability & Scalability
  2. Data Source Connectivity
  3. Deployment
  4. Extensibility
  5. Operating Systems
  6. Security
Vendor Requirements

Vendor Qualification Requirements cover the stability, viability and ease of working with a tool vendor. This is likely the category of requirements that your Finance Team is most interested in.

Top 5 Vendor Requirements for Your BI Tool Evaluation

  1. License Types
  2. Professional Services and Maintenance
  3. Training
  4. User Support
  5. Vendor Information

Why you should trust us

We've spent years gathering and refining our knowledge, traveling the world to learn about new products, trends and new versions of tools we already know. It’s a lot to keep up with and although there’s a lot we don’t know, we know enough to build a company that solely does product research and reviews, but we'd rather actually build stuff.

"Those who can't do [consult]" so since we actually can help with tool selection and implementation, we’ll keep our pencils sharp and our sleeves rolled up so we can keep helping companies to get the analytics they deserve.

Our process

  1. We scour to find the best categories and criteria. Some might be openly available while others might be behind "email walls". We add in other things that we feel like are missing, and most importantly, we listen to the community to see what we still needs to be added.
  2. We fill out the information on tools and vendors based on our experience as developers and technologists, not product marketers. (They have their place, but we like brass tacks.)
  3. With this in hand, we reach out to the vendors to fill in any remaining blanks.
  4. We scrutinize their responses for "marketing fluff" like some vendors saying they have live data connections or real time data when they actually don't, it doesn't function well at scale, or it technically works but it's a poor user experience.
  5. We build a Weighted Scorecard. Each evaluation is documented to include at least Evaluation Date, Evaluation By, Version number, Product name (since some companies have a suite of products), Priority / Importance.
  6. We finalize the scorecard by multiply out the results as Rating x Priority to get an absolute score.
  7. We review the results with your team and determine next steps with demos, testing, or a pilot project with your actual data.

Our Scorecard Methodology


Each feature is rated on a scale of 0-3:

0 - Does not apply, or cannot do it

1 - Does it, but not very well

2 - Does it well and/or is an expensive add-on product

3 - Best example in the market

Priority (Importance)

Each feature is scored on a scale of 0-3, with a weight applied according to how critical each is for your team and use case.

Our default is to use every other digit in a Fibonacci sequence since the difference between “Nice to have” and “Want to have” is significant, and the difference between “Want to have” and “Mission critical” is even more significant. (0), 1, 1, 2, 3, 5, 8

0 - Not applicable = x0 points

1 - Nice to have = x1 point

2 - Want to have = x3 points

3 - Mission critical = x8 points


We finalize the scorecard by multiply out the results as Priority x Rating to get an absolute score. These scores can be compared against other products but should not be evaluated as e.g. 475/650 points which would imply a score of 73% “good”. We feel like this is faulty and may lead to an unfair advantage or disadvantage a since a lot of the potential features may not be relevant to you. It’s much better to evaluate absolute scores against the others in your comparison set and see what stands out.

“All models are wrong, but some are useful.”

At the end of the day, you will be making a decision that is largely subjective. Saying that a feature is “nice to have” vs “want to have” or that a tool does something “but not very well” are all subjective inputs. However, if after the exercise you see that one vendor/tool scores significantly lower than the others, that may be a good reason to drop it from your short list. But if two tools score similarly, we advise to not just select the highest scoring tool. Rather, go back and look at why and where each of those tools scored as they did and make a decision that’s right for your use case.

A note about money

From time to time we may refer to how a particular tool in the market works, but we don't get paid by any of these vendors to get included in any of this. In fact, it costs us a lot of time and money to compile this research.

So… Why are we giving this away for free?

Because if we can hop on a call for a few minutes and help you, or if you can review these materials and go on to successfully implement analytics at your company, then you don't need Mashey. If you feel like you could use some help, give us a shout.

Enjoy the series, and please, reach out to us if you feel like something is missing or misrepresented. We consider this a living set of information that we will come back to and update frequently.

Need help with your BI tool selection? Book a call with us and we’ll see if we can help. If we can point you in the right direction with a short phone call, great! If you’d like to hire us to do an evaluation and selection for you, contact us and we’ll make a selection together.

Join Our
Data Analytics

Expect to hear from us about once a month.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
THere's More

Post You Might Also Like

All Posts
Change Management

Change Management: Managing Change

This is the second post in our series on increasing analytics adoption through change management. This post discusses how to manage change during project implementation.

Top 5 Platform Functions of BI Tools

Here are the top platform functions of BI tools to consider as part of your evaluation.

Top 10 Reporting Features of BI Tools

Here are the top reporting features of BI tools to consider as part of your evaluation.
Change Management

How Can a Data Analytics Project Succeed on Paper but Fail in Practice?

Change management alone will not cause a project to succeed, but the absence of change management almost certainly increases the likelihood of failure.
Change Management

How Analytics Will Change Your Organization: Part 1

Analytics is everywhere. In this article, we look at the many ways in which analytics is changing the face of modern industry.