Are you interested in learning how to combine your data with the ability to query data on your terms? Azure Synapse Analytics is a boundless analytics service that combines data integration, enterprise data warehousing, and big data analytics into a single platform. This tool enables users to bring their worlds together by providing a combined experience for ingesting, exploring, preparing, managing, and serving data.
In a recent webinar by Lead Data Architect, Brian Custer, we examined the Azure Synapse Analytics unified analytics workspace and discussed the major use cases and scenarios for using the platform. Brian also spends time on examining the Azure interface.
Additional topics covered:
How to set up a workspace in Azure
How to work with the major features of the studio
Analyzing NYC taxicab data
If you’re looking for a unified analytics workspace to engineer and analyze big data, Azure Synapse is the answer, and this webinar will give you a great introduction to Synapse. You can watch the entire webinar by clicking the link below.
Hopefully, you’ve already heard about Azure Synapse. One of the newest Azure offerings, Synapse is a limitless analytics service that brings together, data warehousing, enterprise data warehousing, and big data analytics. I’d like to tell you about the “launchpad” for all things Synapse – Synapse workspaces.
With Synapse workspaces you can do many things including development, ETL, ELT, DevOps, Azure ML, and Power BI. When you create a new workspace, you automatically get a serverless pool.
When you go into your workspace, you’ll see Activity Hubs in Synapse Studio. These hubs organize the tasks needed for building analytics solutions.
Synapse Studio is divided into Activity hubs; there are currently 6 hubs:
Overview and data – where we can explore all our structured/unstructured data
Develop – the development hub where you can use workbooks, SQL, etc.
Orchestrate, monitor, and manage – these look like Azure Data Factory, they have the same look & feel and do the same jobs.
Pools are comparable to databases in Synapse Analytics. There are 3 kinds of pools; serverless, dedicated, and Spark.
Each workspace has a serverless pool by default.
In my serverless pool cheat sheet (To see this, please watch my video included), you’ll see we have Spark, Cosmos DB, and Azure Data Lake store as the 3 data sources you can use.
The input types are Parquet, CSV, and JSON. Parquet is the better way to do things as it’s compressed, so it only has to read a large piece of the data in a compressed format into memory. In other words, it doesn’t have to go back to the well as often.
Also, think about portioning your data in some logical way when you begin working with this. This way, if your data doesn’t belong to a certain partition, it won’t go looking for it somewhere it’s not.
Also, think about different landing zones for different data (refer to the flow chart on my cheat sheet).
In my video, I demo how to use a Synapse Analytics workspace so be sure to check that out. I’ll walk you through how a workspace, as well as how serverless, works.
My advice is to go in the Azure portal and give Azure Synapse Analytics workspace a try. Play around with it and see what you can do.
If you have questions about Azure Synapse Analytics, either how to use it or how to implement it in your organization, reach out to us. Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or [email protected].
I’m here with some exiting news from Microsoft! Last week at a digital conference, Satya Nadella announced the general availability of Azure Synapse Analytics and the preview of Azure Purview, a unified data governance service. Azure Synapse Analytics has been gaining traction while in preview and adding Azure Purview gives businesses the ability to get the most of out their data and analytics.
Let’s talk about Azure Purview. This is a comprehensive data governance service that helps organizations discover all data across the organization. Demos at the digital conference showcased different ways you can use Purview for governance. Some key things are the ability to go multi-cloud, not only in Azure, but others as well. You can also connect with your on-prem environment and your Azure data assets.
For quite some time, those of us in the data disciplines have worked to inventory all the different aspects of data, like column, database and table names, etc., and put all those pieces into a common repository, often referred to as a data dictionary. Microsoft has been working for years to create a product that would be comprehensive enough to help most people with their governance and compliance needs. We’ve now got this with Azure Purview.
Some key highlights pointed out are:
A business glossary – no need to manually build a data dictionary.
Automated data classification – allows you to know things like data type (Social Security number for instance). You also have custom options and can schedule future scanning and classification on a routine basis. This way you’re getting continual updates, as opposed to a data dictionary where you get snapshot in time unless you manually update.
Cloud-based search facility – gives you the ability to find things quickly and easily across a broad series of data assets.
Data lineage and reporting – supports the end to end data lifecycle.
Power BI facilities
I feel Azure Purview is a very strong offering. Without it I would have either create my own versions of these pieces or using something like Embarcadero, which I used years ago. Another thing to note is that the experience is very similar to the canvas workspace experience in Azure Synapse Analytics, so if you’ve been working with that, it will feel very familiar.
The next part of Microsoft’s announcement is that Azure Synapse Analytics is now generally available. Azure Synapse Analytics is a limitless analytics service which brings together traditional data warehouse and big data analytics in one offering. It brings these together for a unified experience to ingest, prepare, manage, and serve data for immediate machine learning and BI applications. I, and many of our customers, have been using this great product a lot, so this going GA is surely exciting news.
Some noteworthy things with Azure Synapse Analytics are:
A new native cloud distributed SQL engine
Deep integration with Spark
Flexible query options such as serverless and dedicated
Integration with Power BI and machine learning
TPC-H benchmark at petabyte scale
Native Row Level Security (this is not possible with Amazon Redshift or Google BigQuery)
Native ML integration for the citizen data scientist
Code management – by that their talking about Azure DevOps as another piece that plays well with it.
Power BI integration to Teams which I found to be kind of cool
Again, great announcements with both the general availability of Azure Synapse Analytics and the public preview of Azure Purview. These two products combined empowers teams to remove data silos and leverage all data for analytics and data governance.
Need further help with these or any Azure product or service? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or [email protected].
Why are these announcements so important? Because to thrive, businesses need to harness the power of their data.
One of the best ways to harness the power of your data is to remove data silos. While not a new concept, achieving this has been a constant challenge in the history of data and analytics, as many ecosystems continue to be complex and heterogeneous. Organizations must break down all silos to the get the most out of data and analytics, in a consolidated, secure, and compliant manner. Azure Synapse Analytics and Azure Purview remove silos by creating a unified data analytics and governance service.
Now that we’ve given you an overview of the announcements, let’s break down the details of each Azure service.
What is Azure Synapse Analytics? Azure Synapse Analytics is a limitless analytics service that brings together traditional data warehousing and big data analytics – into one offering! Azure Synapse brings these two worlds together with a centralized experience to ingest, prepare, manage, and serve data for immediate BI and machine learning applications.
Azure Synapse Analytics Highlights
New cloud native distributed SQL engine
Deep integration with Spark
Flexible service query options – Serverless + Dedicated
Power BI + ML integration
Azure Synapse Link – Enables real-time data analytics with link to your operational database
TPC-H benchmark at PB scale
Native Row Level Security – This is not possible with Amazon Redshift, Google BigQuery
Citizen Data Scientist capabilities with direct ML integration
Code Management – Automation, code sync to dev/master branch, and end to end deployment lifecycle
Power BI integration from inside the Synapse service
Ability to add Power BI reports into Teams for end users
What is Azure Purview? It is critical to enable these capabilities through a comprehensive data governance solution. An organization that does not know where its data is, does not know what its future will be. Azure Purview is a unified data governance service that helps organizations achieve a complete understanding of their data. Azure Purview empowers users to discover all data across the business, track lineage of data and create a business glossary wherever it is stored: on-premise, across clouds, in SaaS applications, or in Power BI.
Azure Purview Highlights
Key important and rich features: Compliance, Catalog and Data Map
Sourcing from on-prem, SaaS, and multi-cloud services
Data Map: search/browse data by tech/non-tech domain
No manual building of data dictionary
Data scanning by clicking on a button
Automated data classification with custom options
Schedule for future scanning and classification
Easy cloud based data search
Data lineage and reporting
Free scanning: On-premise SQL Servers, Power BI Service, data sensitivity labeling for O365 E5 customers
The combination of Azure Synapse Analytics and Azure Purview enables organizations to develop the capabilities needed to empower their teams to leverage all data for analytics and data governance, silo-free.
Your sales team is the backbone of your business. Today I’d like to talk about using analytics to build a more effective sales team. This is a big topic for many organizations and I’ll give you some insight into how to use analytics to make your sales teams go from good to great.
With all the focus on cloud and Azure, have you ever wondered if you can run your traditional BI workloads within Azure? You may ask, should we be putting our traditional BI workloads that we’ve been using like the SQL Stack, SSIS, SSAS and relational databases in Azure; can we even do that?
There are many organizations like yours that are trying to figure out how to take that first step into the cloud this year. Maybe it will help to hear what we are seeing our customers make the highest priority as they are moving into the cloud.