Cloud Data Platform

Data Infrastructure

Data Platform

Data Strategy

Creating a native cloud platform for data analysis

Industries:

Finance & Insurance - Retail & FMCG - Transportation - Energy & Utility - Life Science - Industrial

Solutions:

Data Infrastructure - Data Platform - Data Strategy

Technologies:

Aws - Azure - Google Cloud Platform - Confluent - Blindata - Snowflake - Databricks

Overview

The ability to elastically scale storage and computation resources together with the pay-as-you-go pricing model makes the Cloud the ideal environment to design modern data platforms capable of overcoming the traditional limitations of on-premises data warehouses and data lakes.

For this reason, integration, storage and data analysis workloads are among the first use cases activated in the cloud transformation paths of many companies.
However, in order to take full advantage of the Cloud to increase development agility while reducing management costs, it is necessary to rethink the architecture of data platforms.

Challenges

The main problems of data platforms on prem are:

Inability to scale resources elastically. At times of high load the platform is often in trouble while at times of low load it pays for idle resources.
Inability to scale storage and computation independently. Having to increase one must also increase the other. The scaling unit is the server within the cluster.
High operational costs to configure and manage complex, distributed architectures that are often composed of multiple technologies developed by different vendors.

75% of all databases

will be deployed or migrated to a cloud by 2022

Gartner

50% of new system deployments in cloud

ill be based on a cloud data management ecosystem by 2023

Gartner

Solution

The cloud data platform leverages the peculiarities of the cloud to acquire, transform, store and make accessible a potentially unlimited amount of data while reducing operational costs and increasing development agility.

Once acquired in batch or real-time mode, data is stored and consolidated within the data lake by means of on-demand integration processes.

On top of the data lake consisting of a scalable and cost-effective storage system (i.e., object storage), query engines are grafted to access the data. Depending on the type of access pattern, it is possible to have different types of query engines.

It is also possible to have a data virtualization tool to provide analytic consumers with a single semantic layer of access to the data thereby masking the underlying fragmentation into multiple query engines.

Integrate

Acquisition and consolidation of data from both batch and real-time systems

Store

Storage of raw data and all subsequent consolidation layers in low-cost storage systems with potentially unlimited storage possibilities

Query

Access to stored data with different query engines depending on the type of query and underlying workload

The complete route

1. Setup

Definition of the technology infrastructure and data architecture. Connection to data sources.

2. Land

First complete data product with data acquisition and consolidation pipeline. Exposure to analytic consumers through a centralized semantic layer.

3. Expand

Addition of new data products and coverage of new data domains.

Benefits

Potentially infinite storage and computing capacity

Elastic resource scaling

Possibility to scale storage and computation separately

Consumption-based pricing model

Reduced integration costs between data lake and data warehouse

Reducing operational costs

Simplified and controlled data access thanks to the unified semantic layer

Greater development agility

Success Stories

Arcese – Velocity

Arcese S.p.A. invests in technologies that enable real time as...

Selex – Retail Analytics in Cloud

Selex builds new retail analytics platform in Cloud with Snowflake

Events

Video Talk

Cloud Data Fabric 2021

Date and Time: 20/10/2021

Main Topics In this webinar, we talked about how to create a Data Fabric in the cloud and modern. We talked about the characteristics of this type of architecture, its...

Online Webinar

Digital Integration Hub 2021

Date and Time: 07/07/2021

Main topics In this webinar, DIH will be presented as an architectural pattern that can decouple legacy systems from consumers and, at the same time, make data available at various...

Hybrid Event

IKN Forum Retail 2021 – Chapter 1

Date and Time: 24/06/2021

Process Mining for Customer Journey Analysis: Reconstructing and analyzing the real behavior of the customer in a complex ecosystem. Quantyca participated in Forum Retail 2021, the IKN Italy event designed...

Hybrid Event

IKN Forum Retail 2021 – Chapter 2

Date and Time: 28/10/2021

Quantyca participated in IKN – Forum Retail 2021, the IKN Italy event designed for its #onforumretail community. Francesco Gianferrari Pini, Co-Founder of Quantyca, and Matteo Gabanini, Business Intelligence IT Manager...

Video Talk

Snowflake, una Data Platform coi fiocchi 2020

Date and Time: 13/03/2020

Quantyca, a technology consulting company specialized in Data Management and a partner of Snowflake, has discussed the main challenges in today’s market and the competitive advantages of using Snowflake. This...