Last week at Build, Microsoft announced that Azure Synapse Analytics is moving to Public Preview. For most of us, this represents the first real chance to get hands on with the truly new elements of Azure Synapse since the exciting announcements at Ignite (which I wrote about 6 months ago). In fact, I’ve been excited about the potential of Azure to truly transform the rules of the game with the advent of true, cloud-scale analytics for some time now.
Over a year ago, Azure SQL DW, the origin of Synapse Analytics, had established itself as the clear top performer in providing traditional data warehouse technology with the provisioning, scale, and performance of the cloud. With the release of the updated feature set into Public Preview, we are seeing the next phase in industry leadership as we move towards a unified experience, integrated services, and a platform that truly delivers insights for all!
Not long ago, we used this simplified (trust me, I know!) slide to provide a high level overview of a Modern Data Platform. If you noticed it, Azure Synapse Analytics in this diagram is really just the initial re-brand of Azure SQL DW, as the real transformation of Synapse’s capabilities remained in the realm of announcements and private previews until last week…
Now granted it’s only now in Public Preview (and subject to continued improvements!), Synapse is redrawing the landscape dramatically:
In fact, for some applications, the column on the right can be omitted as Synapse provides integrated Spark notebooks and integration or Azure Machine Learning for ML/AI as well as native Power BI modeling and reporting capabilities! The new Azure Synapse Studio interface alone has dramatically improved the experience of numerous roles within any organization that runs on data.
The Modern Data Platform: not just logical anymore!
For the last few years, the Modern Data Platform has been a logical concept that, in application, required the stitching together of numerous disparate technologies with various types and states of user interface, security integration, languages, and experiences in order to make everything deliver on its promise. It worked – one of the benefits of using Azure is Microsoft’s ongoing ability to provide service integration and security – but it was somewhat more challenging than it needed to be for more basic scenarios in order to bring a full spectrum of capabilities to the most complex scenarios. In technology at least, c’est la vie, non?
But Azure Synapse Analytics is making a clear statement… equal parts unified experience (Synapse Studio), service integration (ADF, Power BI, AMLS, etc), and net new services (serverless/on demand SQL, Spark pools, etc), we can expect a far more streamlined technical landscape as we implement modern data platform solutions for organizations moving forward. Organizations exploring the expansion of their analytical capabilities with Spark have a minimal-friction access point with Synapse, and a more unified focal point to build their teams and skillsets around as they look to leverage the capabilities of Azure.
A look towards the future
Much of the Public Preview has been closing some loops on capabilities and interfaces that I personally have been excited about for the past 6 months, but one feature in particular that was not previously on my radar is worth calling out. Initially, it is very limited just now, but the pace of change is both rapid and inevitable, and this is one to keep an eye on!
Hybrid Transactional Analytical Processing, or HTAP, implemented in Synapse as Azure Synapse Link, is not a brand-new concept. SQL Server has been blurring the lines between Transactional and Analytical data with column-store indexes and in-memory transactional tables for years, and the HTAP acronym is not owned by Microsoft. However, what we see in the Synapse Preview is a cloud-native HTAP implementation that combines Azure Cosmos DB with Synapse Analytics – it’s built for cloud-scale analytics and may well represent a significant paradigm shift that reduces costs and improves accessibility for powerful AI and BI across a wide range of scenarios. This one is going to be fun to explain to my less technical friends, as my own excitement will make it harder to slow down and help connect the dots on why this is just so cool…
In case this is still feeling a bit fuzzy on what it means, here is a nice picture from Microsoft that highlights the new Synapse Link allowing Synapse to reach across to Cosmos DB and directly interact with the available analytical store in that system. No data movement/transformation to get in the way… just a much broader canvas and faster tools to use when working in the analytics studio!
(source: https://docs.microsoft.com/en-us/azure/cosmos-db/synapse-link)
Although this presently only works with Cosmos DB, the concept will certainly extend to additional data repositories in the future. This has the potential to dramatically reduce the ETL/ELT workload for certain data sets while dramatically increasing the real-time or near-time analytical capabilities surfaced by Synapse – and I for one am excited for that future!