Are you looking to reuse file connections you have set up in Azure Data Factory prior to using Data Flows? Maybe you have a case where you’re building some pipelines and you’re pulling data from files, etc. and you want to use reuse those for a Data Flow. When I tried this I discovered that you cannot reuse these existing text file data sets, but you must recreate them. In today’s post, I want to share a quick tip on how to recreate these file connections to use in Data Flows.
Step 1 (Screenshot below): Create a new Data Flow in Azure Data Factory using your work canvas. Under the settings pick a data set and point it towards the file that you have previously set up. On the left side, you should see your previously made data sets. I have a couple previously created that are still there, but not available as selections in my Source Data set list within the Data Flow, so they will need to be recreated.
Step 2: When I create a a new data set and point it towards my Azure Data Lake linked service, I’ll now see the menu shown in the screenshot below. There is an option for delimited text, which I was using in my case. I choose delimited text and set up the data set. If you look at the JSON beneath it, you can see they have made some changes in the formatting.
When I move back to the Data Flow (see below), you will see the drop-down for source data and the newly created data set is now an option available to be used in my Data Flow. Note the files you previously made will not be available, but only the ones you recreate as new are available for selection.
While you may not be able to reuse previously existing connections, I hope this quick tip will make your process of recreating these files easier.
Need further help? Our expert team and solution offerings can help your business with any Azure product or service, including Managed Services offerings. Contact us at 888-8AZURE or [email protected].