Why should you combine Snowflake and ClicData?
Since we added the Snowflake connector, many of our customers have asked us about Snowflake, wanting to know more about it. Some were confused, too, because both Snowflake and ClicData offer a cloud-based, scalable data warehouse feature, and people didn’t understand why we’d want to offer a connector that syncs data from Snowflake to ClicData. In fact, with our Data Stream feature recently released we can now push data from ClicData to Snowflake.
Why do we connect to a different data warehouse platform, and what are the use cases and benefits for our customers?
What is Snowflake?
Snowflake is an enterprise-ready, cloud-based app that creates a data warehouse in a cloud platform such as Amazon or Azure. It can be scaled on-demand to allow work with more or less data, depending on requirements, and it can also scale on-demand on the processing of the data.
For example, if you need near-real-time data loads for complex transformations but have relatively few complex queries in your reporting, you can script a massive Snowflake warehouse for the data load and scale it back down after it’s completed—all in real-time.
It is a highly technical platform built on multiple cloud providers and requires strong development skills to use and maintain. But it is highly flexible and ideal for large volume data, cyclic loads of data, and high frequency/high volume transactional tables—at the billion-row level—such as those used in IoT, telecommunications, and website traffic.
Snowflake is built specifically for the cloud, and it’s designed to address many of the problems found in older, hardware-based data warehouses, such as limited scalability, data transformation issues, and delays or failures due to high query volumes.
Best Use Cases for Snowflake
Snowflake is a layer over existing databases available in Amazon and Azure—and soon, if not already, in Google, too. This layer provides easier, but still highly technical, access to these underlying SQL and NoSQL data stores.
Snowflake would benefit from some technical development in its interfaces, but due to its highly scalable and built-for-volume architecture, the app can handle just about anything you can throw at it. More about Snowflake here.
On the other hand, Snowflake is not a full Business Intelligence platform. It does not provide data visualization, publications, or automation. It is definitely not for the end-user but rather for the data management team.
It is a great platform for high volume and/or high-frequency transactional data such as websites, telecommunications, and financial data. It also provides a great alternative to other warehouses such as SAP HANA that can be very difficult to implement.
ClicData with Snowflake
At ClicData, we wanted to provide a connection to Snowflake because we saw the immediate value for customers who have built their data warehouse in Snowflake but need to provide analytics and visualization to external users without impacting their Snowflake architecture and without the high costs associated with purchasing additional modules and tools.
With high volumes of data stored in Snowflake, ClicData can be used to use Snowflake’s querying power to bring extracts of data ready for consumption within ClicData—periodically, on-demand, or in real-time.
Furthermore, one might want to add complementary data—data that is not required to be present in Snowflake or that it is too dynamic or too costly to store in Snowflake. With this additional data, more insightful reports and dashboards can be built.
Also, using a streamlined copy of the data from Snowflake does not incur the same costs to use the data since ClicData doesn’t charge more when processing data.
We also solve the “More Dashboards Viewers, the Higher the Cost” problem by having ClicData plugged in directly to Snowflake.
Snowflake charges per processing and storage, which can be scaled up or down as needed. But if you have a consistent number of dashboard viewers or data processing tasks, scaling up or down is irrelevant, and you will always be “up.” If your users invariably use dashboards and reports, then your usage will be consistently high, and you will end up paying more.
This creates an issue in utilization objectives since part of the objective is to make your company data-driven. But if by being more data-driven, the cost is higher, the tendency is to lower costs and, as a result, to lower the use of data analytics.
ClicData’s operating principle is different, though. It is why we do not charge more for data processing, for the number of dashboards and reports you need, or for how often they are used. We try to encourage more communication around the data by not increasing the price linearly with use.
Snowflake with ClicData
Just as there are many advantages to bringing data across from Snowflake into ClicData, there are also advantages to having ClicData supply data to Snowflake.
Snowflake does not offer connectors to external data sources, databases, or systems. That type of work is typically done via Snowflake’s API integration or through highly customized development work. But now, Snowflake customers can utilize ClicData to connect to the hundreds of systems it connects to and feed that data via our API, data stream, or SQL direct connection (on dedicated tiers only) to Snowflake.
In the end…
What matters the most is what solution is best for your needs and how easily you can use data to your advantage. If you do require large-scale data power and scalability and have built your data warehouse in Snowflake, you can use ClicData as a data ingestion tool or as a data visualization tool. Either or both options will drive results faster and potentially even lower costs in your overall data and Business Intelligence strategy.
