We would like to keep you up to date with the latest news from Digitalisation World by sending you push notifications.
Digital Newsletter
Each week our editor Phil Alsop rounds up the most popular articles, videos and expert opinions. We compile this into a Digital Newsletter and send it straight to your inbox every week.
Digital Magazines
We'll let you know each time a new edition of Digitalisation World is released so that you're always kept up-to-date with the latest and greatest news and press releases.
Video Magazines
The Digitalisation World Video magazine contains the latest Zoom interviews with experts in the industry.
A lakehouse architecture simplifies data and AI for organizations. In the past, data teams had to maintain proprietary data warehouses for BI workloads and data lakes for data science and machine learning workloads, because no single data platform could meet the performance needs of BI and the flexibility needs of data science. Expensive and complicated to maintain, this coexistence of legacy architectures has created data silos that slow innovation and stifle data team productivity. A lakehouse addresses this by running all workloads through a single architecture.
Shell chose Databricks to be one of the foundational components of its Shell.ai platform. “Shell has been undergoing a digital transformation as part of our ambition to deliver more and cleaner energy solutions. As part of this, we have been investing heavily in our data lake architecture. Our ambition has been to enable our data teams to rapidly query our massive datasets in the simplest possible way. The ability to execute rapid queries on petabyte scale datasets using standard BI tools is a game changer for us. Our co-innovation approach with Databricks has allowed us to influence the product roadmap and we are excited to see this come to market.” Dan Jeavons, GM Data Science
“It is no longer a matter of if organizations will move their data to the cloud, but when. A lakehouse architecture built on a data lake is the ideal data architecture for data-driven organizations and this launch gives our customers a far superior option when it comes to their data strategy,” said Ali Ghodsi, CEO and co-founder of Databricks. “We’ve worked with thousands of customers to understand where they want to take their data strategy, and the answer is overwhelmingly in favor of data lakes. The fact is that they have massive amounts of data in their data lakes and with SQL Analytics, they now can actually query that data by connecting directly to their BI tools like Tableau.”
SQL Analytics is built on Delta Lake, an open format data engine that adds reliability, quality, and security, to a customer’s existing data lake. Customers are able to avoid storing multiple copies of data, as well as locking data up in proprietary formats. To deliver BI-performance on a data lake, SQL Analytics makes use of two unique innovations. First, it provides easy-to-use auto-scaling endpoints that keep query latency consistently low under high user load. Second, it uses Delta Engine, Databricks’ unique polymorphic query execution engine, to complete queries quickly against both large and small data sets. With native connectors for all major BI tools, including Tableau and Microsoft Power BI, customers can easily integrate SQL Analytics into their existing BI workflows to conduct analytics on much fresher, more complete data than ever before. SQL Analytics also provides a SQL-native query and visualization interface to allow analysts, data scientists, and developers without access to traditional BI tools to build dashboards and reports that can be easily shared within their organization.
“Now more than ever, organizations need a data strategy that enables speed and agility to be adaptable,” said Francois Ajenstat, Chief Product Officer at Tableau. “As organizations are rapidly moving their data to the cloud, we’re seeing growing interest in doing analytics on the data lake. The introduction of SQL Analytics delivers an entirely new experience for customers to tap into insights from massive volumes of data with the performance, reliability and scale they need. We're proud to partner with Databricks to bring that opportunity to life.”
The lakehouse architecture is widely supported by Databricks partners including:
“Databricks SQL Analytics is a critical step in the most important trend in the modern data stack: the unification of traditional SQL analytics with machine-learning and data science,” said George Fraser, CEO at Fivetran. “Companies make huge investments in centralizing and curating data, and they should be able to make those investments once and then implement multiple analytical paradigms in a unified environment. The Lakehouse architecture supports that."
