In the BI-ginning
The early stages of BI in any company starts with reporting. Reporting is the ability of retrieving rows of data and columns with some totals and subtotals where the report reader will use to make decisions about the topic of interest. A sales report split by regions will allow a sales manager to identify if they are advancing in the right direction. Matching the sales report with other data such as the sales objectives would be the next step in order to have a clearer and more focused view of the state of the business topic. An indicator is calculated that is a representation of how well a certain activity is performing or not. In recent years, and as the maturity and new generations become more proficient in handling data and more complex software tools, the concept of “Self Service BI” came into light. In essence, it is the ability for Subject Matter Experts (SME), managers, business owners, non-technical consumers to perform many of the tasks that until then only Technical Experts could perform. The ability to communicate with the data via languages such as SQL all the way to producing documents (reports and dashboards) was left to the Technical Experts to perform, requiring extensive communication, translation of business requirements into technical ones, and of course adding time and resources to the ability to analyze data. This required the ability to produce data models that could provide any type of metric on demand. The concept of “cubes”, not tabular reports, came into current use where the Technical Expert was now responsible to create Data Warehouses and produce data cubes that business users would then consume using pivot tables and exploratory tools. Cubes and subsequent technologies such as in-memory databases and columnar/vertical databases, allowed the business user to be more self-sufficient but it still requires the Technical Expert to be part of the process which forces the discussion of defining requirements. The need for the technical experts to be even more involved than before is that even with exploratory tools such as cubes, the need to define “what if” scenarios and predictive models requires even more technical knowledge than simple data analysis. In addition to a strong data management understanding, it requires advanced statistics, software programming and broader understanding of the assumptions affecting that specific domain.
Figure 1 – Key areas of functionality in Business Intelligence.
1 – Data Connectivity
The first step in the road to visualize you data is to actually get to your data. Many moons ago when we decided that ClicData should have its own built in data connectivity capabilities we were told by many stating that this was not an issue. As business moved to the cloud and no longer held their data in local databases, the issue re-surfaced once more and again we were told that many of them have APIs (Application Programming Interfaces). Still, today, you will be hard pressed to find one API that looks exactly the same as the next one. Much like file and database standards, the proliferation of formats continues to this data and even the most serious technical team is challenged with keeping up to date with the different ways of connecting to data. So the native ability to easily access the data for all your company’s data sources must be #1 on your list. Without it you will be resorting to manual exports prone to limitations, errors and adding to your manual day to day activities which eventually will make your BI extinct due to the amount of effort and most importantly the fact that if the responsible is on vacation, so is your BI platform.
2 – Data Management
Assuming that all your data is in one place thanks to all the connectors that enable you to “suction” the data out of all your core systems, the next step is to make sure that relevant data talks to each other. There are always small “gotchas” when dealing with different systems. The way dates are stored, the identifiers for items such as products, employees, clients, bad emails, missing data, mismatched characters and so many other issues. As such a dashboard and reporting application must have native capabilities to directly (not having to send the data to another system for processing) to make these changes rapidly without too much hassle.
3 – Visualizations
Not everyone agrees with Stephen Few and Perceptual Edge in stating that pie charts and using multiple colors and effects on charts should never be used. For one reason or another, some do like their 3D pie chart with 30 slices and if that provides them with the visual information that they need then so be it. This to say that a dashboard and reporting platform should have as many types of visualizations, even innovative ones while also attempting to direct the users to select the best suited ones. Using a pie chart is not the best visualization if you have more than 5 or so categories, and using a line chart to measure unrelated categories is also not applicable. So the software should not restrain the users on its formatting capabilities but rather on how to use them better.
4 – Publication
Have you ever read the Dilbert cartoon about unspoken objectives? Well, there is no point in creating a company reporting system and keeping the information on your computer desktop is there? So a good software should have the ability to share access – securely – to any user on your team based on a distribution list or team security list. But not as static PDFs or PowerPoints, but rather as dynamic web pages that can be interacted with by the users where they can filter, drill down, navigate and even export to static document formats for when they are “on the road”.
5 – Automation
Here’s the thing. You can accomplish many if not all of the above with a lot of programming, copying and pasting, Excel macros and shared drives to name a few of the things you will need. But you will need to do it day after day, week after week and month after month. Is this really the best use of your time and effort? Is it sustainable? And if you are not able to accomplish these tasks, will your employees, customers, partners lose confidence in your reporting? Probably. So automation is key. Without automation, any business reporting initiative is pretty much dead from the get go.
6 – No Hardware/No Software
This goes without saying but the days of having to buy a server and installing it in your company’s computer room are over. Fight the good fight if you want to but third party hosting for your company’s software and data is happening except in some niche and computing intensive domains. There is no way that most companies can compete in security, cost, facilities and stability with the likes of Amazon and Microsoft Azure, among others. More importantly – you shouldn’t unless that is your business. If your business is not hosting then save yourself money, space, time, frustration and resources, increase your security, safety, reliability and backup and go cloud. Also, make sure that your BI platform and dashboard solution requires no software installation. Are you using a MacBook? Do you want to keep buying licenses for everyone interested in creating dashboards or looking at data? If so, make sure your dashboard editor is not tied to Windows or Mac. Ensure that it is fully web and that it runs smoothly at least on Chrome, Safari, Edge and Firefox. This is total freedom without the hassle of operating systems, heavy downloads and installations.
7 – Mobile
Related to hardware and software, one thing that your dashboard solution must do well is the ability to access dashboards on your mobile device especially if you are always on the go. A mobile sales force equipped with a phone capable of showing them their performance intermediately after a sale is not only encouraging for the users but it is also habit forming. The more they use dashboards the more data focused they will be and the more they will use their BI tools to make decisions. Most BI initiatives suffer from lack of interest and use and most times it is just because they are not accessible easily and directly. Most tools you need to access via the company portal, VPN, passwords and logins through a laptop or desktop. By bringing the dashboards to their mobile, you remove those barriers.
8 – Integrated User Security
Similar to the mobile capabilities, where if your users do not have access easily and directly to a BI platform they will soon lose interest and use it less over time, the ability to integrate user security into their existing security is a great feature to have. This means that the users do not have to enter their usernames and passwords in several systems each time, once to login to Windows or Mac, again for email, again for their CRM or Finance system, and again for BI. Too many passwords! This increases security risk and it adds more work for your IT staff since they will be resetting passwords every week. By introducing Single Sign-On (SSO), or the ability to login once and subsequently login to other applications using the same identity, it makes it more secure, more centralized (only 1 set of credentials and permissions to manage), and easier for the users.
9 – Alerts
Are you expecting your users and executives to rigorously and frequently access your beautiful dashboards every day? Think again. They have lots of other stuff to do. But if something is happening and they should be aware they will be coming back to you asking why you didn’t tell them about it. Why was that delivery late to our key customer? Why are 60% of our invoices over 90 days late? Why is branch XYZ not performing? And why didn’t I know about it? Or kore importantly why didn’t the BI tool tell me about it. Well, that is what alerts do. Basically you enter thresholds and once a calculated value reaches those thresholds an alert with a nice dashboard explaining whats happening is sent via email or mobile alert.
10 – Embedded Analytics
It is true that reporting is something that all comes last. Many think that as long as the data exists somewhere that we can report on it. If that is true or not, what is very common is that business users use transactional type of applications such as Accounting, ERP, CRM and other similar software much more frequently than the analytical ones such as your BI. One reason is because those systems have reporting builtin for their own data which can provide them with their basic needs and secondly, well, that is their main tool for doing their work. If you directly embed the dashboards in those applications, dashboards that can extend the information and provide insights for your business users then you don’t even need to have another system for them to login and access their reports. More importantly you can make those dashboards actionable. For example by clicking on a pie of the pie chart, you can then display a screen in the main application with the data that makes up that pie. This provides value and makes it very easy for your users to take advantage of the information you are providing.
BI – The Next Generation
So until now we covered the basics. With the above capabilities and functionalities you get a stable dashboard and reporting system for your company or clients. But we need to be more demanding because we have more and more data, more complex business scenarios where the user is capable and in need of wanting to do more without technical assistance. So here are what I believe are the next features that you need to look for and they all involve Advanced Analytics, Machine Learning and Predictive Analytics – all things within the Artificial Intelligence context as well as some other items which may be very useful in integrating data into your reporting solution much easier and faster. These are just teasers and I recommend you to read a few more blogs on this subject such as “What is AI’s Role in Your Business’ Future?” and “The Future of Business Intelligence”
1 – Intelligent Data Monitoring and Cleansing
What if instead of you executing queries, the BI platform would be able to tell you that during a certain data refresh certain anomalies were detected. Values out of normal, peaks or valleys, unusual patterns, outliers would be corrected or alerts would go out immediately before they are displayed to the users. If your data extends to millions of rows or billions then having an automated validation system in place is the way to do it but the issue with it is that it only validates for the case scenarios that you thought off. How about the other cases you have not yet faced or did not think about? This is where AI can help by analyzing and segmenting the data a variety of ways to raise those exceptions to you for further analysis.
2 – Data Storyboard and Narrative
Instead of receiving charts and tables of data how about a movie narrating a story highlighting key metrics that have taken place and things to look out for? You receive an email you open it and a voice starts telling the story “Yesterday, region X experienced an unusual decrease in service calls of 27%…” and simultaneously a video showing you a chart over the last 7 days and compare to last year. Natural Language Generation (NLG) along with intelligent video editing capabilities will make this a reality in the next 3-5 years.
3 – Data Lineage
Knowing where the data came from is just as important as knowing what to do with it. Understanding that you are looking at a chart of data that came from several data sources, that has been “massaged” by certain data analysts or certain algorithms have been applied to make it more consistent is critical to identifying issues with the data itself. This however is highly complex as many BI systems standardize the data into their format eliminating the original source and origins and hence data analysts and users don’t understand where they are getting the data from. Better built systems will have the intelligence to keep tabs on data as it goes along its stages.
4 – Predictive Analytics and What If?
It goes without saying that Predictive Analytics, or the ability to statistically start to forecast certain events or data, is the most interesting aspect of AI as it relates to business. If your BI has the capability to directly and at any time proposed predictive models, select the best suited ones and process further data and make predictions using that data then you can do start taking what if scenarios into account. For example in a churn scenario, apply a retention campaign and measure its ROI as shown in the dashboard below.
I hope the above functionalities will provide you with insight on selecting or improving your existing BI platform. If you think we missed something just let us know and we will happily add to the list. In the meantime, feel free to check our comprehensive Business Intelligence solution, that includes all the features mentioned above. Of course.