This is a session recaps a session by Paul Turley, Director & BI&A Competency Lead at 3Cloud, delivered 3Cloud’s Envision Summit.
Get ready to be blown away by Microsoft’s sensational lake house architecture, powered up with the incredible capabilities of Power BI! This innovation is set to change the entire industry. Imagine a single, unified platform that lets you create a comprehensive end-to-end data warehouse and analytics solution, right there in Power BI itself! Brace yourself as BI & DW developers unlock the full potential of cutting-edge technologies like SPARK, delta lake, parquet, Python notebooks, pipelines, Kusto, SQL, and so much more, all integrated within the Power BI platform. It’s a dazzling symphony of possibilities, where you can orchestrate mind-boggling enterprise solutions using just one extraordinary product. Discover the limitless wonders of this groundbreaking new platform.
Conventional Power BI Solutions
In a conventional Power BI solution, there is a distinct separation of work efforts. The process begins with data engineering, a crucial step in Enterprise BI, especially in the Microsoft world, where dimensional models are heavily relied upon. This involves shaping, cleansing, and preparing data for analytical reporting. Typically, a data engineering team is brought in to build pipelines, stage the data, and possibly work through a medallion architecture before storing it in a relational database, data mart, or data lake as fact and dimension tables. Once this data engineering work is completed, the solution is handed over to the BI development team. However, this transition often takes longer than anticipated, resulting in waiting times and inefficiencies before BI developers can start building tabular models, data sets, and visualizing and reporting on the data in Power BI.
Microsoft Fabric
Microsoft Fabric introduces a unified platform that consolidates various tasks and roles into one system, simplifying the overall process. Although specialized skill sets are still required, with experts focusing on tasks like data engineering pipelines, notebooks, and orchestrating data movement, the platform facilitates these roles within a unified framework. Specialized roles, such as data science engineers, can continue to work on their specific tasks, such as handling streaming data sets. The introduction of a lake house architecture allows for hosting data storage, Delta pipelines, and notebooks using the Spark engine. This unified platform enables the presentation of data in dimensional form, making it accessible for consumption and reporting in Power BI.
How Does Fabric Work?
Microsoft Fabric works by integrating mature components that have been developed over several years, bringing together technologies from Synapse Data Warehousing, Synapse Analytics, Data Factory, Synapse Data Science, Real-time Analytics, and Power BI into a unified platform. This is not about reinventing the wheel or recycling old components but rather leveraging existing, mature technologies. The platform offers a seamless experience with a web interface and a straightforward tool set. Users can employ various languages such as T-SQL, Python, R, Scala, and native Spark languages, as well as KQL for streaming data sets. Analysis Services use DAX and Power Query within a serverless compute engine, all built on top of a unified security layer named One Security. Data is stored in a unified format, specifically Parquet Delta Storage, referred to as “one lake.” The platform introduces new concepts while building upon familiar components that users may have already encountered or be using.
Fabric Key Concepts
In the context of Fabric, some key concepts to grasp include:
- Workspaces: Similar to Power BI workspaces, these remain familiar, but within them, there are new objects to understand.
- Lake House: Built on Azure Data Lake Gen 2 storage, the Lake House concept is crucial. It’s essentially a unified platform, akin to OneDrive over a data lake.
- One Lake: Think of One Lake as a single drive over the data lake, where data can be stored in various formats (e.g., CSV, Excel), but the native format is Parquet. Delta, which sits on top of Parquet, is referred to as Delta storage or Delta tables.
- Shortcuts: These are objects related to pipelines, notebooks, and a new concept for Power BI called “Next Generation Power BI,” which utilizes a storage mode named “Direct Blink.”
These concepts form the foundation of the Fabric platform, and understanding them is crucial for effective utilization.
Fabric: Data Analytics for the Era of AI
The focus of the Fabric platform is on empowering AI and Next Generation modern analytics, centered around the Lake House concept. This approach is “Lake Centric” and open, offering APIs and methods for working with data at various levels within the platform. The key takeaway is the emphasis on having a single copy of data, eliminating the need for multiple copies for reporting purposes. The platform maintains raw, bronze, silver, and gold layers through a medallion architecture, but the central idea is to avoid unnecessary duplication of data. Instead, Fabric promotes the concept of “One Lake” or “OneDrive for data,” providing a single SAS Lake for the entire organization, provisioned automatically without the need for manual setup of storage accounts, resource groups, or specific security configurations. This streamlined approach consolidates all aspects of data management in one place within the platform.
Microsoft Fabric Demo
To see a demonstration of Fabric in action, watch Microsoft Fabric: The Magic of a Unified Analytic Reporting Platform on our YouTube channel.
Get Started
Connect with 3Cloud! Let’s discuss how Fabric can impact your business today.
