With the advent of Internet-scale applications generating massive amounts of data, there is a need to streamline the underlying database architecture. Historically, databases were the systems for storing data. In contrast, the real value of any data is discovered only after processing it.
More...
In-database analytics is the bridge between data storage and data processing. It merges the two functions. Conventionally, these are separate systems. However, with in-database analytics, they are integrated into one cohesive system. This results in faster and better analytics performance.
Evolution of In-Database Analytics and Advantages
Over the years, databases have evolved as a result of multifaceted challenges faced by applications. There have been a lot of improvements around the innards of database system mechanisms. A few groundbreaking innovations include optimizations around data access, data models, storage engines, replication, and clustering mechanisms. Some of them have also garnered many opinionated debates, like the hotly contested SQL vs. NoSQL argument in several tech forums.
The in-database analytics adds yet another dimension. However, instead of improving the internal mechanisms, it adds a new capability. A database system augmented with in-database analytics transforms into a big data processing engine.
Gartner coined the term “X Analytics” as part of their top 10 Data & Analytics trends for 2020. It is an umbrella term that refers to the growing application of analytics for different types of data, such as text, video, and audio. Therefore the impetus for modern database architecture is focused on processing & analytics rather than storage alone. Consequently, in-database analytics is slated to become a de-facto feature in all database/data warehousing products in the future.
Advantages for Data Analysts
In-database analytics serves as a tool for data analysis jobs performed directly on the stored dataset. Some of the apparent advantages for data analysts include:
- 1Faster Processing: Having an in-database analytics layer on top of a database makes the raw data accessible to the analytics layer almost instantaneously. This approach circumvents the overheads of moving large datasets from database storage to separate analytics apps.
- 2Less Complexity: In-database analytics leverages the database’s query language to embed logic that drives the analytical outcomes. Thereby, it reduces the complexities of writing separate code, creating and managing apps for performing analytics.
- 3AI Enablement: Artificial Intelligence is the ultimate holy grail of all analytics. With the support of in-database analytics, data analysts can access an immediate playground to perform AI-related workflows directly on the database.
Advantages for Businesses
Beyond the technical advantages, businesses have a lot to gain from in-database analytics. The enterprises of the 21st century rely on business intelligence (BI) to achieve data-driven decisions. Analytics is at the heart of BI applications, and the in-database way is set to revolutionize BI in several ways:
- 1Realtime Reports and Visualization: The technical advantages of faster data access and parallel processing shortens the lead time to generate BI reports, providing a realtime experience.
- 2Analytics Backed by Current Data: By virtue of speed and the design, in-database analytics is always performed on the most recent data, avoiding inconsistencies in data access.
- 3Enhanced Predictive Analytics: In-database analytics provides the least intrusive way of streamlining data warehousing capabilities for achieving some level of predictive analytics to identify future business opportunities and risks.
In-database Analytics Use cases
In-database analytics is a fairly technical subject. Therefore, its use cases revolve around intricate tasks around data warehousing, such as building more efficient ETL pipelines, self-service analytics apps, and other routine data analytics tasks.
However, from an industry standpoint, there are some broader scenarios where in-database analytics shines compared to traditional analytics.
E-Commerce: In-database analytics expedites the predictions around supply and demand to perform realtime pricing and discount offer generation for e-commerce products. This situation also applies to any form of online trading or bidding application that operates at scale.
Fintech: Fraud detection is one of the most common workflows in an online financial transaction. The use of in-database analytics can provide an immediate assessment of such risks. There are similar workflows in organizations that have multiple decision points and require prompt resolution. In-database analytics is the preferred choice to speed up such processes.
Online Security: Many online security threats like DDoS attacks can also be predicted more efficiently by leveraging in-database analytics. Speed of prediction is the winner here, and in-database analytics boosts the efficiency of the threat prediction algorithms by many folds.
Apart from that, any application that deals with behavioral data or crowdsourced data also needs in-database analytics for delivering quicker insights.
5 In-Database Analytics Platforms to Jumpstart Your Next BI Project
If you are in the process of exploring BI platforms with in-database analytics capabilities, then here are some of the platforms to check out.
Bipp.io
Bipp.io is a BI platform for data analysts. It has a rich set of features that aid the data analysis workflow, from connecting databases, creating datasets, writing SQL queries, and building custom models and dashboards. In addition, in-database analytics is baked right into the Bipp.io platform and integrates seamlessly with enterprise data warehouses.
bippLang is a powerful yet concise data modeling language that unifies all the analytics-related questions into a set of SQL compliant queries. Furthermore, it makes it easy for both business users and technical BI analysts to work with the model, thereby alleviating the need for know-how on SQL queries and technical nitty-gritty.
Looker
Looker is a BI platform for big data. The Looker platform offers a modular way of building data analytics workflows using Looker blocks. Some of the commonly used blocks are the Source block that pulls data from common data sources, Data blocks that enrich the data with pre-modeled external data, and the Analytics block that employs design patterns to transform data.
Looker’s embedded analytics feature and the LookML modeling language provide an effective toolchain for leveraging in-database analytics. However, there is a steep learning curve for picking up on LookML.
Sisense
Sisense is a data analytics platform that offers proprietary, high-performance analytical databases, known as ElastiCubes. Customers of Sisense can build data models on top of Elasticubes to offer custom analytics experiences.
Like the other platforms, Sisense also offers dashboards, reporting, custom analytics with embeddable analytics widgets, live data sources, supporting visual data modeling framework, and in-database analytics capabilities through ElastiCube.
Domo
Domo is a platform that leverages your BI and data investments to create intelligent apps that drive action from your data. Using Domo’s flexible app-building framework, companies can build new data experiences in record time.
Domo also offers embedded analytics to streamline reporting and provide self-service analytics to business teams. Domo also has advanced ETL tools to define custom data workflows using drag and drop visual tools. However, it does not offer full in-database analytics and data modeling language capabilities.
Tableau
Tableau is one of the early pioneers in visual analytics. It offers a great visualization platform to integrate with data sources, interact with data in numerous ways, and collaborate with stakeholders to deliver data insights and stories in a visually intuitive manner.
Tableau offers embedded analytics and data modeling capabilities to streamline analytics workflows and aid in efficient data governance. Tableau has multiple deployment options from desktop to the server, with cloud, on-prem options, and the in-database capability largely depends on the deployment configuration.
Where There is Data, There Should be Analytics
The key for in-database analytics is keeping data and the analytics processing logic as close as possible. And as we just saw, it makes a lot of sense. However, every technology has its pros and cons, and in-database analytics is no different.
The very idea of incorporating data processing within the database brings computation into play, in addition to storage. Computing resources have their constraints, based on the volume of data processed or the speed of realtime report generation expected. These and a few other factors like the cost and complexity of implementation are important trade-offs that you must consider when deciding on an in-database analytics-capable database platform.
Nevertheless, data-centric information architectures will be the mainstay of an organization’s data governance strategy in the coming years. One of the examples of this transformation is the data mesh architecture. Coupled with AI and the need for realtime data processing, in-database will be a default component in future data platforms.
