Upload Files to the Lakehouse with Microsoft Fabric

Video Transcription:

Hi my name is Steve Hughes. I work for 3Cloud in their Data and Analytics team. Today I’d like to talk about uploading files into your lakehouse. So as you open up your brand new lakehouse and you’re thinking, “How do I get data here?” First of all, you can see this, the big set of bullets in the middle, how to get data in your lakehouse using either data flow, a pipeline, a notebook, or even a shortcut.

Of those, the first 3 represent an option to get data in your lakehouse by using an ETL process of some kind. A shortcut is something that we’ll be demoting later, but not not in this demo. So look for a new demo for that one.

Now, you can also go up to Git Data and you’re going to see right away that you have similar options and you have the note open notebook over here that you can use, but what if you just have some files you want to get up into the lakehouse, you can start working them with them right away. Typically, we would upload files in that scenario, so the first thing we’re going to do is take a look at explore and you see we have tables now. Tables are those files that have been properly formatted for use as delta tables, and that’s not what we’re going to try to do today. First thing we’re going to do is just try to get some raw files uploaded into our file section in our lakehouse to give us some source data to start working with.

First of all we’d like to add a sub folder in this case. So when you do the dropdown you can see right here, we can add a shortcut here or we can do grab a new subfolder or upload which well do next. But let’s go with a new subfolder. As a part of that process we want to give that folder a name. So in our scenario I’m going to paste a name in there. Let’s call it Samples. Just so I have a good place to locate all the files we’re using in this demo all right. So now I have a Samples folder subfolder. Let’s go into that folder you can currently see there’s no content. We can have more subfolders to upload files, so let’s just start with a simple file upload and we are going out to here and going to select upload files and we are going to open up our dialog here to find files. As you can see. I have my data for a restaurant demo loaded up here and I am going to grab this first JSON file. You see this is receipt data.

We’re going to open that up and then we’re going to upload it. So once you click open, you upload it. You have the option to overwrite if that file already exists in our case, that doesn’t matter, so we’re just going to go ahead and hit upload. You can see it’s a small file, so it loads up here quickly. What’s really cool about this is that we can verify the data is what we expect. We can preview the JSON data in properly formatted and you can see here that this is a number of restaurant receipts stored as JSON objects in a file. So now we can see that data. It’s like, alright, cool, now, what about a situation where I have a folder of data that I want to work with? Well, that’s equally easy to upload. So if we go over here, choose upload and then select upload. Folder. We’re go to go to the same dialogue that we had before. I’m going to grab the Restaurant Data Files Folder and we are going to upload that Data.

As you see here, we did get the warning and ask only if we trust a site in this case we do, so we’re going to continue with they upload there and then we’re once again going to click upload one more time and that is going to bring all that data into the restaurant data files. We can expand that folder and we can open one of the CSV that this is still held here and see a preview. So now you’ve seen CSV data and JSON data loaded in. We’re not just limited to the more common data types. We can actually pull some older data types across to older being a relative term. Of course, we can also bring in pretty much any data we want to in here. But in our case we’re going to keep it limited to those type data file types. So we’re going to actually pull in an XML file as well. So you can see that i’re just going to upload a file here which has our restaurant information in it.

We’re going go out to our folders again and go up a level. And select this file here which is our XML say. Open. Then upload that. And we can browse our Restaurant XML. And there you go you can see that we can hold multiple types of data in here, and we can use this data for later to work with our work within our lakehouse and move it through the process. Reporting so realistically.

This is a quick way to get data up into the lakehouse, especially if you’re working with samples and just trying to get data uploaded quickly. This is an option that you have. No data is loaded into tables. In order to fall in the table section, it needs to be in a Delta table format. We have not actually done that. We’re still looking at it in its raw state. If we want to move that over, we’ll use one of the ETL processes to change the type of the data and land it on Delta. But for now you can see it’s fairly straightforward upload files in here to start working with your lakehouse on the new Microsoft Data and Analytics platform. Hopefully you’re learn something new here today. Thank you for your time. If you want to reach out to 3Cloud for more support on working with the new platform or anything else related to Azure and Power BI, please reach out for more information. Once again, thank you for joining us today.