Mount Azure BLOB Storage as File System on Docker container

Written by: Chirag (Srce Cde)


Blob storage is an object store to store vast amounts of unstructured data, blob storage is an option. It uses a flat namespace to store the data. The blob storage can have multiple containers and each container contains the blob objects/data.

Suppose, there is a requirement to mount to the blob storage container to access the data from the docker container. Now, since the blob storage is not an actual file system so there is a need for a middleware that can help to mount the object/blob storage as the file system. And in this case, the middleware that we are going to use is Blobfuse (v1). However, the blob storage also supports NFS 3.0 protocol but for the purpose of this article, we will not use it.

BlobFuse is a virtual file system driver for Azure Blob storage. BlobFuse allows you to access your existing block blob data in your storage account through the Linux file system. BlobFuse uses the virtual directory scheme with the forward-slash ‘/’ as a delimiter.

For the purpose of this article, we build the flask application and bundle it as a docker container. The purpose of the flask app is to load the images (icons) stored in the Blob Storage. Hence, we will mount the Blob Storage on the Ubuntu filesystem and subsequently, bind the mount with the container as a part of volumes.



Hands-On


Create blob storage & upload files

Go to Azure Portal → search for Storage Accounts → Create the Blob storage account as below.



Post creating the Azure Blob Storage, go to that resource → click Containers under Data storage → click + Container and fill in the required details and click on Create



As mentioned above, upload the images within the container. I have uploaded a couple of images.



Launch Virtual Machine

Search for Virtual Machines on Azure Portal and create Azure Virtual Machine



Click on Networking to open port 8000. Click on Add inbound port rule and open port 8000. Its because the flask application will be running on port 8000.



As a next step, connect to VM via Visual Studio Code since we are going to write some code and docker config files. But you can also SSH into the instance for the same.

After a successful remote connection, execute the below commands to setup docker.

  • sudo apt-get update
  • sudo apt-get install docker-compose

Setup Blobfuse

To install Blobfuse, add the Microsoft package repository by executing the below commands

  • wget https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb
  • sudo dpkg -i packages-microsoft-prod.deb
  • sudo apt-get update
  • rm -r packages-microsoft-prod.deb
Install blobfuse
  • sudo apt-get install blobfuse

Authorize blobfuse to access storage by creating the config file and adding accountName, accountKey & containerName in the config file

  • touch /home/ubuntu/fuse_connection.cfg

accountName containerblobstorage
accountKey storageaccesskey
containerName icons

Get the Access key from Access keys under Security + networking and copy the Key



For more information about the setup check here

Create a temp mount path & mount folder

Execute the below command to create both

  • sudo mkdir -p /mnt/resource/blobfusetmp
  • sudo mkdir -p /mnt/containermnt

Post mount path/folder creation, execute the below command to mount the blob storage on the Ubuntu system


sudo blobfuse /mnt/containermnt --tmp-path=/mnt/resource/blobfusetmp  --config-file=/home/ubuntu/fuse_connection.cfg

/mnt/containermnt is where the blob storage is mounted

If the command is successful, you can execute ls /mnt/containermnt to test if the mount is successful.

Note: If the VM is rebooted, then the storage will unmount and it needs to be mounted manually


Deploy the application

Now, it is time to write some code. Go ahead and create a directory in your VM and copy the code from here

The directory structure looks as follows



app.py contain the flask code which will render display.py from the templates directory. display.py will load all the images from static/images folder. The requirements.txt contains the information about the python packages to be installed.

We will mount the /mnt/containermnt on static/images folder via docker-compose.yml file. Dockerfile contains all the definitions/commands to create an image

Within docker-compose.yml the flaskapp_sevice is defined. Port 8000 is exposed to the host, for accessing the application via the public IP address of the Virtual Machine. /mnt/containermnt is mapped to the static/images directory in the container


version: '3'
services:
  flaskapp_service:
    container_name: flaskapp
    build: 
      context: ./
      dockerfile: Dockerfile
    ports:
      - "8000:8000"
    volumes:
      - /mnt/containermnt:/project/flaskapp/static/images

Finally, execute the below command to build the image and run the container.

sudo docker-compose up --build -d

Once, the container is up and running copy the public IPv4 address of the virtual machine and open a new tab in the browser & navigate to http://yourpublicip:8000

This is how the output looks like in my case



Here, we have successfully mounted the blob storage on the docker container and the flask application is able to load the data from there.

I hope this is helpful and can cater to the number of use cases.


References:



YouTube       Twitter       GitHub       LinkedIn