Azure SQL Data Warehouse

My List of Top Contributors in the Data/BI Community

As I read the many posts from those in the community who I follow, I am reminded that the community brain trust is much greater than any individual. As a writer and blogger, I’m occasionally compelled to express an original thought or opinion that I think is uniquely my own. However, we work in a world where everything comes from somewhere and there are many contributors who I trust and rely upon for advice and cutting-edge information. This “corner” of my blog is to highlight these community contributions that I find informative.

Paul TurleyMy List of Top Contributors in the Data/BI Community
Read More

Azure Storage Options

Azure Storage Options (1)

We all know life can get hectic. Here at Pragmatic Works, we’re no different. But one of our goals is to learn something new about Azure every day, as things are constantly changing and being updated. Many people are still learning all the amazing things they can do within the Azure cloud and we want to help. Our posts in our Azure Every Day series are a great way to learn more about Azure each week.

There are many options for data storage, how do you know which is right for your data? Today I’d like to discuss storage in relation to the architecture of the modern data warehouse and to shed some light on your options.

 

 

Tom WardAzure Storage Options
Read More

Azure Data Factory v2 Parameter Passing: Linked Services

icon_azure@2xSometimes I get so involved in my repeatable processes and project management that I forget to look up.  Such is the case of the December 2018 ability to parameterize linked services.  I could not rollback and rework all the ADF components this impacted which had already gone to production, but oh hooray!  Moving forward, we can now have one linked service per type, one dataset per linked service and one pipeline per ingestion pattern.  How sweet is that?  Au revoir to the days of one SSIS package per table destination.  

This is blog post 3 of 3 on using parameters in Azure Data Factory (ADF).  Blog post #1 was about parameterizing dates and incremental loads.  Blog post #2 was about table names and using a single pipeline to stage all tables in a source.  Today I am talking about parameterizing linked services.

Disclaimer: Not all linked service types can be parameterized at this time.  This feature only applies to eight data stores:

  • Azure SQL Database
  • Azure SQL Data Warehouse
  • SQL Server
  • Oracle
  • Cosmos DB
  • Amazon Redshift
  • MySQL
  • Azure Database for MySQL

Concept Explained:
The concept is pretty straightforward and if our  goal is to have one linked service per type, a parameterized Azure SQL Database linked service might look like this:

Tom WardAzure Data Factory v2 Parameter Passing: Linked Services
Read More

Backup Retention for Azure PaaS Database Services

Backup Retention for Azure PaaS Database ServicesRecently, Pragmatic Works has been on engagements helping our customers to migrate their on-premises databases to the Azure cloud. We’ve been asked to migrate such things as: SQL databases to a SQL database in the cloud; migrations off their APS or PDWs utilizing Azure SQL Data Warehouse; and MySQL and PostgreSQL databases to the cloud.

Tom WardBackup Retention for Azure PaaS Database Services
Read More