ClicData provides the ability to extract, transform and load your data into the platform so that you can prepare the calculations for your dashboards.
Not a lot of technical knowledge is required since we provide visual capabilities of data manipulation.
However, there are a couple of advanced features and concepts that need to be mastered by the Editors or Admins.
ClicData’s Cache Feature
In today’s blog, I am going to explain a new feature we have released recently, the cache functionality.
The cache feature is useful if you have a complex data transformation (ETL) process in the Data section on ClicData. The cache can be applied only to views, merges and fusions, and not on top of data sources. View the example below:
Scenario of What Your Data Looks Like
In this scenario, the user imported a couple of data sources from different systems: from a database (Source 1 and Source n), from a CRM system (Table 1 and Table n), and a flat file containing budget information. He builds data transformations on Views (cleanse of data and builds other calculated columns), and then includes the Views in a Fusion. The tables imported from the CRM system are merged in order to combine information from different tables into one data set. On top of the Fusion and Merges, there are additional Views built which are preparing the data for different dashboards. Additional calculated columns are added at this step.
What Happens on Dashboards With No Cache
Once you load a dashboard on editor mode or via the Live Link, each widget goes back to the data and requests the updated data rows. ClicData goes to each view, merge and fusion and processes the data transformations. In case you have complex calculations not only on the data side but also on the dashboard, the results might take longer to display.
What Happens on Dashboards When the Data is Cached
To improve the performance for our end-users, our hard-working engineering team implemented the cache functionality. Basically, if you cache a data set, the cache is updated when source is updated and stored as a stand-alone table.
Let’s go back to our example. ClicData refreshes the data sources based on a schedule. This means that each merge, fusion and view is evaluated and updated automatically. When the cache is enabled, after the data sets are updated, the data is “stored” on our database. When the widget “calls” the data, the calculations are not processed each time, but we retrieve the result directly from the static tables.
How to Cache Your Tables
To enable the cache, you need to access the data set properties. In this screen, you can find the cache switch.
You should not cache all the data sets on your account. Going back to the above example, the cache should only be enabled for View 3, View 4 and View B. There is no reason to cache a merge since the data is not “called” directly from the dashboard.
Challenges
- All Views which are not cached take 2 – 10 MB of storage space on your ClicData account. If they are cached, the storage occupied increases.
- If you use every minute data refresh, the cache functionality will not suite your needs since the tasks will not have time to “cache” the tables and ClicData would need to access the live data.
In case of further questions, do not hesitate to open a support ticket at support@clicdata.com, and we will be happy to help you!