Azure

Azure Purview

As data volume and growth continue to rise, businesses are turning to data governance and data catalog programs to help them manage their data properties and create searchable self-service experiences. Azure Purview is a data catalog tool that provides users with access to a searchable catalog of business and technical assets.

In this recent webinar, presented by Senior Data Consultant, Steven Wise, and Lead Data Architect, Darren Zwonitzer, we explored the implementation process and how to resolve challenges in a secure environment. We also examined a few best practices, as well as lessons learned from various Purview engagements.

Additional topics covered:

  • How to install, configure, and secure a typical installation
  • How to customize and extend the business glossary
  • How to define and use custom classifications
  • How to use the API to add custom elements

If you are searching for a means to provide business and technical data users with self-service capabilities or starting a data governance program, then this webinar is right for you. You can watch the complete webinar below.

Be sure to join us each week for our free webinars that happen every Tuesday at 11 AM ET. Our webinars cover a wide range of Azure topics and are presented by industry experts. Check out what webinars are coming up on our events calendar.

Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or  [email protected].

 

3CloudAzure Purview
Read More

Modern Data Warehouse: What It Is, Why You Need It, How to Use It

Analytics has evolved tremendously since the 90’s when we began constructing data warehouses. The technique is new, the tools are very different, and the demands are even different! Companies, for example, achieve a strategic edge by machine learning, also supplied by IoT and smartphones. The huge volume of data and high-speed causes problems with scalability.

In this recent webinar presented by Cloud Data Architect, Rowland Gosling you will see an outline of new methods and tools used to construct a data warehouse. The presentation also discusses the value of a data warehouse and ways to enhance the data capabilities within your business.

 

Additional topics covered:

  • How the modern data warehouse updates exist within analytic strategies
  • Mapping to today’s business goals for analytics
  • Typical end-to-end workflows that are used to solve problems
  • Using all the data available to create a more accurate picture of the business
  • How do we scale?
  • How are we going to stay competitive?
  • Can we consolidate to one approach to analytics?

 

If you are a business leader, technical staff, or a company interested in a consolidated, modern approach to analytics, then this webinar is right for you. You can watch the complete webinar below.

 

Be sure to join us each week for our free webinars that happen every Tuesday at 11 AM ET. Our webinars cover a wide range of Azure topics and are presented by industry experts. Check out what webinars are coming up on our events calendar.

Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or  [email protected].

3CloudModern Data Warehouse: What It Is, Why You Need It, How to Use It
Read More

Modern Data Warehouse: What It Is, Why You Need It, How to Use It

Analytics has evolved tremendously since the 90’s when we began constructing data warehouses. The technique is new, the tools are very different, and the demands are even different! Companies, for example, achieve a strategic edge by machine learning, also supplied by IoT and smartphones. The huge volume of data and high-speed causes problems with scalability.

In this recent webinar presented by Cloud Data Architect, Rowland Gosling you will see an outline of new methods and tools used to construct a data warehouse. The presentation also discusses the value of a data warehouse and ways to enhance the data capabilities within your business.

 

Additional topics covered:

  • How the modern data warehouse updates exist within analytic strategies
  • Mapping to today’s business goals for analytics
  • Typical end-to-end workflows that are used to solve problems
  • Using all the data available to create a more accurate picture of the business
  • How do we scale?
  • How are we going to stay competitive?
  • Can we consolidate to one approach to analytics?

 

If you are a business leader, technical staff, or a company interested in a consolidated, modern approach to analytics, then this webinar is right for you. You can watch the complete webinar below.

 

Be sure to join us each week for our free webinars that happen every Tuesday at 11 AM ET. Our webinars cover a wide range of Azure topics and are presented by industry experts. Check out what webinars are coming up on our events calendar.

Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or  [email protected].

3CloudModern Data Warehouse: What It Is, Why You Need It, How to Use It
Read More

Key Strategies to Enable DevOps Transformation

Are you looking to learn more about DevOps and DevOps Transformations? In this recent webinar, 3Cloud Sr. Director of DevOps, Noel da Costa focuses on key strategies for enabling DevOps transformation that can help you jumpstart your DevOps initiatives. Noel has over 25 years experience in App Dev and 10+ years specialized in DevOps.

This webinar covers:

  • What DevOps is
  • Why you should implement DevOps
  • Key strategies to enable DevOps

If you are new to DevOps, DevOps is the union of people, products, and processes that help our end-users to maximize the distribution of quality and value between production and operations. DevOps is a cultural change; it involves changing how people work and the environment in which they collaborate. Noel will discuss the importance of people, process, and products within DevOps.

The presentation then dives into the business benefits enabled by DevOps, including:

  • Shorter cycle times and faster delivery of value to market
  • Improved quality and availability
  • Optimized resources and waste reduction/elimination
  • Innovation & better customer experience achieved through experimentation.

Noel also covers challenges that DevOps can help your business overcome such as long delivery lead times of products and services to market, lack of collaboration between business and IT decision makers, difficulty staying compliant with regulatory and security requirements, and many more.

If you’re looking to learn about DevOps and all it can do to transform your business, this is the webinar for you. This deep dive presentation covers all you need to know about DevOps and adopting a model of continuous improvement. You can watch the complete webinar below.

Be sure to join us each week for our free webinars that happen every Tuesday at 11 AM ET. Our webinars cover a wide range of Azure topics and are presented by industry experts. Check out what webinars are coming up on our events calendar.

Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or  [email protected].

3CloudKey Strategies to Enable DevOps Transformation
Read More

An Intro to Azure Purview

Have you heard about one of the newest additions to Azure services, Azure Purview? This new addition to Azure services is currently in Preview and is a unified data governance tool that lets you easily create a map of your data landscape. I’m here to tell you more about Purview and share one of my favorite features.

In my video included in this post, I’ll walk you through a demo of how to utilize Azure Purview.

  • I’ll start with a view of a fictional data map. On that map, I have data centers and can group my data centers together.
  • I can also add many different data sources such as SQL Servers, Teradata, Hive Metastore, SAP, Azure Data Lake Storage Gen2, Azure SQL Databases, Power BI, as well as others, and the list is growing every day.
  • It allows for classification of sensitive data and there are built in systems classifications, as well as create custom classifications. So, if you store bank routing numbers, for example, Purview already knows what that is and how it should be formatted. You can then apply this classification to that field in the data map.
  • An example for a custom classification might be in a Police Department, where you may want to have a classification for the computer-aided dispatch and case numbers. And almost every organization has a custom employee ID. You can create those classifications and find them in multiple systems by using the automated data discovery process.
  • Purview creates a catalog that is easily searchable by your business users. They simple enter in a term and the interface will show matches and suggestions for different assets and places they can find them.
  • In my opinion, the coolest feature is the data lineage. Here’s how it works:
    • In my demo I have an example of a Power BI Campaign Analytics dashboard. It we start at the dashboard and work our way back, we’ll see there are two visualizations, campaign revenue and digital campaigns.
    • We can see that those come from a dataset that is embedded in the Power BI workspace. It also shows that both datasets come from some final data that had a prep and transform associated to it, along with showing the five different data sources.
    • The benefit here is being able to show our users where the data comes from, what happens to it, where it ends up and how it gets used can help them to understand the complexity of the data.

I am a strong proponent of data governance, but few organizations do it as it seems like a giant undertaking. I feel it’s one of the most foundational activities an organization can take to increase their data literacy and IQ. That’s why Purview is so exciting! It allows businesses to easily start cataloging their data and will show immediate value to the stakeholders in the organization.

Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or  [email protected].

 

Leslie AndrewsAn Intro to Azure Purview
Read More

Introduction to Azure Integration Services

Do you want to learn how to leverage and integrate disconnected business systems? In a recent webinar, 3Cloud’s Application Modernization Practice Director, Nick Althoff, teaches you what Azure Integration Services is and how you can leverage it to integrate disconnected business systems.

Azure Integration Services is Microsoft’s cloud offering that consists of a set of serverless Platform as a Service (PaaS) components that can be used together to do this integration. For those that are unfamiliar, when you are running something serverless you are leveraging compute (RAM and processor) that Microsoft is responsible for maintaining, no spinning up VMs or installing an operating system. Plus, you’ll benefit from event-driven scale to scale your workloads effortlessly, as well as save money with sub-second billing.

The bulk of this presentation digs into not only an overview of Azure Integration Services but into each of the components that make up Integration Service: API Management, Logic Apps, Event Grid, Service Bus, Function Apps and Event Hub. You’ll come away with a sense of how these components fit in the Azure Integration Services platform.

So, if you’re looking to integrate your disconnected business systems using Azure Integration Services, this webinar is for you. You can watch the complete recorded webinar below.

Be sure to join us each week for our free webinars that happen every Tuesday at 11 AM ET. Our webinars cover a wide range of Azure topics and are presented by industry experts. Check out what webinars are coming up on our events calendar.

Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or  [email protected].

3CloudIntroduction to Azure Integration Services
Read More

Daylight Savings Time Changes in Power Query

In this post, I’ll show how to accommodate for Daylight Savings Time changes in a Power BI file. The video I’ve included will show the formulas that I use and how they adapted from Daylight Savings Time to Standard Time. There are many ways to do this and it is best to find a way that accommodates what you need to do, but I’ll walk through how I do it for my needs within Power Query.

  • The first thing I do in Power Query is to identify the start and end date of Daylight Savings Time. In the US, the start date is the second Sunday in March and end date was the first Sunday in November and I’ve added the time of 2:00am.
  • You could look these up and make a list of them for the next 8 or 10 years but to do it formulaically, I use a DateTime.LocalNow and Date.Year to pull the year out of whatever year we are in.
  • In the Applied Steps area in my query settings, I create a small temporary calendar table using that the year that will start in March and end in November. I create a list of those dates under source and then I convert that list into the actual dates.
  • No I have a calendar table with column1 being all dates listed from the start of March thru November. Using the Added Custom fields in Applied Steps, I add a column for month number and one for number of the day of the week (0-6).
  • Since I’m only interested in Sundays in March and November, I filter this table to show me only Sundays (day of week 0) in those months. More specifically, I need to know the second Sunday in March and the first Sunday in November. To do that, I want to group rows by their month number and then add an index column.
  • If I click on one of the columns in my table, I can see a Sunday index which is counting in order (1st, 2nd, 3rd Sunday). If I remove the other columns and just keep the one with the index and expand it, I see I have all the information pieces I need (month number, Sunday index columns).
  • Next, I’ll add a little helper column (GetDays) that will combine the month number and Sunday index columns together. I filter that helper column for the 2nd Sunday in March and the 1st Sunday in November.
  • So, I have all I need but I just need to add a time, so I’ll put a time component in and combine it with the date, so I have a DateTime column.
  • It’s important to note that when we do comparisons in Power Query between dates, date times or date time zone, you want to be consistent with your format. If I’m going to compare date time zones, I need to put my date times into a date time zone. I also need to ensure that the time zone is the same, so I’m going to use a consistent switch. I’ll add a time zone to my start and end dates.
  • I’ve added two queries (StartTime and EndTime) in which I’ll reference that DateTime list. In my formulas, I’ll write in the DateTime in March as the min and the end DateTime in November as the max. I’ll also need to add the time zone to each and make sure I’m consistent. Check out my video for more detail of the formulas.
  • The logic here is an If/Then statement. If the switch time zone which I’ve set to -4 (which is Eastern Time during Daylight Savings), is greater than the start time and less than the end time, then we are in Daylight Savings and the time is correct.
  • If it’s not in between those two dates, then we’re not in Daylight Savings and we need to subtract an hour from it because now we’re going to fall back. In other words, we’re going to be 5 hours behind instead of four. If I refresh it again, it should match the time that’s on my computer. In my SwitchTime Zone column, I see the time that it was in the summer and in my DateTime with DST Correction column I see the correct time now which matches my computer, so I know that this is working correctly.
  • But how do I know it’s working to the exact second? I’ll just duplicate my three queries and create a testing query. In this testing query I will arbitrarily set the start and end time to be a specific day and time to check back on my table to see if it matches. This testing helps ensure that it’s working perfectly.

My video below goes into more detail and may make this written demo clearer. I hope my peek into how I can accommodate for Daylight Savings within Power Query is helpful. This is certainly not the only way to do this, but it works well for me, so I thought it was worth sharing.

Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or  [email protected]

Steve WiseDaylight Savings Time Changes in Power Query
Read More

Azure Data Factory 101

Are you just starting out with Azure Data Factory? In this post, I’ll give you an introduction to Azure Data Factory, covering what it is, how it works and how to set it up. Within the video included this post is a short demo of how to create and access the platform.

What is Data Factory?

  • Here is a clear definition that I found from the Cloud Academy. Azure Data Factory is a cloud-based data integration service that allows you to create data-driven workflows in the cloud for orchestrating and automating data movement and data transformation.

How Data Factory works?

  • The key components of Data Factory are pipelines, activities, and datasets.
  • Pipelines are made up of activities. There are 3 types of activities:
    • Movement – the copy activity
    • Transformation – including Azure Functions, HD Insight, Stored Procedures, Spark and Databricks
    • Control – ForEachLoops, If Condition Branching, Wait Times, and Validations
  • Datasets represent the inputs and the outputs of the activities.
  • Linked Services – these are the connection strings and authentication for all types of sources for the data sets.
  • Data Flows – are the results of the datasets where you can apply logic and transform the data.
    • Mapping Data Flows are graphical with drag and drop functionality.
    • Wrangling Data Flows are more like using Power Query or M.
  • Integration Runtime – allows you to do data integration across different network environments. There are three types of runtimes: Azure, Self-hosted, and Azure SSIS. Depending on where the data is that you need to copy will determine which of these is appropriate for the use case.

In the video below, I provide a brief walk through of how to access and create in Azure Data Factory. Please check it out, as I think it is a good resource for those just starting out.

Our Azure Every Day series is another great resource. 3Cloud consultants posts weekly blogs and videos around all things Azure. You’ll find all current and past blogs on our website or by clicking here.

Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or  [email protected].

Leslie AndrewsAzure Data Factory 101
Read More

Tabular Modeling & Calculation Techniques Beyond the Basics in Power BI

Do you want to learn how to address real-world issues with Power BI and DAX? In a recent webinar, Principal Consultant, Paul Turley, teaches you what you need to know. Paul covers many-to-many relationships, using disconnected tables and takes a look at using composite models.

This presentation takes a deep dive into Tabular modeling and calculation and includes demos. Paul’s agenda covers:

  • Dimensional modeling basics (just a quick run through as this is a more advanced presentation)
  • Filter propagation in Tabular models
  • Effects of single and bi-directional relationships as you are creating relationships in your data model
  • A look at scenarios where data model relationships can or cannot be used to achieve requirements or may or may not solve business problems
  • Dynamic measures built on a dimensional data model
  • A demo on DirectQuery in a Gen 1 composite model with Import mode
  • A first look at new Gen 2 DirectQuery using Power BI datasets which is the next generation of composite model capabilities which is now in public preview

All these topics and demos are based on a business requirement or business problem context, so the presenter will set up a business problem and work through how it can be solved with dimensional modeling.

So, if you’re looking to learn about using Power BI and DAX to address real-world business issues, then this webinar is for you. You can watch the complete webinar below.

Be sure to join us each week for our free webinars that happen every Tuesday at 11 AM ET. Our webinars cover a wide range of Azure topics and are presented by industry experts. Check out what webinars are coming up on our events calendar.

Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or  [email protected].

3CloudTabular Modeling & Calculation Techniques Beyond the Basics in Power BI
Read More

A Few Tips When Creating Power BI Mobile Reports

Power BI is the best BI and reporting tool out there, and so is Power BI Mobile. Microsoft has made mobile access a first-class experience in the Power BI platform. All the heavy lifting like authentication, data security and network connectivity is all automatically delivered out of the box.

Just download the app and sign in once, then sit back and enjoy access to all your reports and dashboards. You’ll also get integration with your mobile device’s virtual assistant, so you can ask Siri for the sales report and she’ll pull it up for you to view.

Awesome stuff, right? But as a developer, you may not know how to deliver an optimized view of your analysis for mobile consumption. Here, I’ll walk you through creating a mobile version of your report. I’ll point out the major features, as well as some valuable tips and tricks.

  • In the Power BI Desktop view, you can look at the design for the mobile view by clicking on View and then Mobile Layout. This gives me a blank canvas and on the right side is a visualizations pallet, which at this point only lists the visualizations from my desktop.
  • I can pull over one of my desktop visuals and add my report title and company logo, but honestly, it’s not that great. I want to change the way it looks or even create a totally different visual just for my mobile layout, but unfortunately, I’m stuck with the visuals that I have on my desktop.
  • To do this, here’s my first big tip for a work around; you’ll need to create visuals, put them on your desktop version, and then hide them. Let’s walk through this:
    • The title bar ‘Sales Analysis’ on my mobile view is huge so I want to make it smaller. I go back to my desktop and make a copy of my title bar and make one with a smaller font size.
    • Next, I go over to my mobile layout, get rid of the large one and add the smaller one.
    • Here’s the problem, when I go back to the desktop, I want to get rid of that smaller font title text box and when I delete it, it will disappear from the mobile view as well.
    • If I try to hide it in the desktop view, it will also hide it in my mobile view.
    • So, my trick is to hide that text box with the smaller font behind a larger text box in my desktop view. All I need to do is lineup that mobile only text box on top of the other one, go to Format and click Send to the Back. You can also do this in the Selection panel on the right side and drag the text box to the bottom of the list, and it will put it behind everything else higher on the list.
  • Next, I want to create a visual that is more meaningful in my mobile view. In my demo, I create a copy of my bar chart visual and I filter it to show only the most recent 3 months of my data.
  • I’ll shrink that down a bit and hide it behind the larger bar chart on my desktop. That new 3-month bar chart will appear in my mobile visualizations pallet and I can pull that over onto the canvas.
  • When I save this and publish it, I’ve created one version that users who view it from the desktop will get the desktop view. Users that view it from their phone will see the desktop view if they are in horizontal mode but will get the mobile view if they are in vertical mode.

When I’m on my phone and open the Power BI app, I see two different icons. The mobile version I created and published has a different icon with a phone on it. This way end users can quickly identify the version that is intended for the mobile use. I can rotate my phone horizontally and see the desktop version, but the goal of creating a mobile view of reports is to just have distinct data (in my case, the most recent 3 months) available for a quick update.

I can also add a quick Siri shortcut by simply telling Siri that when I say, ‘open mobile sales report’, my phone will go right to that mobile report. These tips should be super helpful when creating Power BI mobile reports.

Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or  [email protected].

Chris SilvernailA Few Tips When Creating Power BI Mobile Reports
Read More