Abstract
I always get this question – how can I download Azure
blob storage files in Azure Linux VM? When I say use Azure CLI (Command Line
Interface) then next question asked is – Do you have step by step guide?
Well, this blog post is the answer to both questions.
The high level approach is outlined below –
- Provision Linux Azure VM in a subnet. [Of course this step is out of scope of this article. For detailed steps refer - this guide .]
- Install Azure CLI in Linux VM
- level Upload sample files to azure storage and then download them in a folder in Linux VM.
This article assumes you
understand
Azure storage
and related concepts.
Wow, this is first blog post from me on Linux and Azure.
Prepare you Azure Linux VM
I have provisioned an
Azure Linux VM with OS as CentOs 7.2. Added this VM in a Subnet, with NSG
having only port 22 inbound open. Also I have attached a public IP to this VM
so that I can make SSH to this VM from anywhere over port no 22. Step by step
guide link is already shared above.
If you are having
different OS than CentOS then commands in below steps will change however high
level approach remains same.
Install CLI in Azure CentOS VM
First make SSH to
your Linux VM and run command “sudo su” [without double quotes]. So in
subsequent steps we will not face awesome “access denied” or “permission denied”
errors. Or we don’t have to add “sudo” word in every command we run.
There are two
versions of Azure CLI –
1.0
–
This stuff is written in node.js and supports both Classic [old way of doing
things on Azure] and ARM [new fancy way of doing things on Azure].
2.0
–
To make this version impressive Microsoft calls it “Next generation CLI” and is
written in python. Only supports ARM mode.
I will be using 2.0
version. Hence I need Python as well installed on the Linux Azure VM. So let’s
first install python latest version on Azure CentOS VM.
Let’s make sure that
yum is up to date by running below command –
sudo yum -y update
-y flag tells system
that “relax, we are aware that we are making changes, hence do not prompt for
confirmation and save our valuable time”. This command execution will take good
amount of time.
Next install
yum-utils using below command –
sudo yum -y install
yum-utils
Now we need install
IUS (Inline with upstream stable). Don’t get scared by name. This is community
project which will ensure that whatever version we install for Python 3, we will
get the most stable version. Run below command to install IUS –
sudo yum -y install
https://centos7.iuscommunity.org/ius-release.rpm
After IUS now we can
install recent version of Python. As of writing, the recent version is 3.6 but
I will install 3.5 to be on safer side. In python 3.5 version I see 3.5.3 is
the latest so let’s install it.
sudo yum -y install
python35u-3.5.3 python35u-pip
To verify, simple
run below command and it’s output should be 3.5.3.
python3.5 -V
Now install the
required prerequisites on CentOS using below command –
sudo yum
check-update; sudo yum install -y gcc libffi-devel python-devel openssl-devel
Finally, back to installation of CLI 2.0 -
curl -L
https://aka.ms/InstallAzureCli | bash
This may prompt you
to download CLI in which directory. Press enter to keep the default path of
installation which would be “/root/lib/azure-cli”. Similarly keep pressing
enter if more prompts are displayed.
Restart command shell
to take changes effect –
Exec -l $SHELL
Just type “az”[without
quotes] and it should you Azure cli commands information in CentOs. This means
installation of Azure CLI 2.0 on Linux is successful.
Run below command to
list storage related commands.
az storage -h
Upload sample files to Azure Storage
This step is straight
forward. Use Azure portal and create one standard [not premium] ARM based storage
account. Create container and upload 4 sample files in the container. It would
look like below –
Add Azure account to CLI
Run command as shown below. It will prompt you with a code
and link to enter the code. After this you will be asked for login using
existing azure related credentials. Successful login will show you the
subscriptions associated to your account as below –
Set storage account and download the blob
Now set credentials for
storage account.
export AZURE_STORAGE_ACCOUNT=YourStorageAccount
export AZURE_STORAGE
ACCESS_KEY=YourStorageAccountKey
Create a directory
named as test1 using the command. This is the directory in which we will
download blob contents. -
mkdir test1/
After this run below
command to download the blob file in test1 folder
az storage blob
download -c sample -n File1.txt -f /test1/File1.txt
az storage blob
download -c sample /test1
Change directory to
test by command cd test1 and run ls -l. This should list File1.txt as shown
below.
Limitation
Using Azure CLI you
can’t download all the blob from a container. You have to download each and
every blob individually; bulk download of azure blobs is not supported.
Resolution to Bulk download
To download all
blobs from a container instead of Azure CLI, we will need to use Azure XPlat
CLI. Or we can also use Powershell as it is open source now [although I have not tried yet]. It’s common to refer many approaches to achieve one task when you are in open
source.
J
Azure XPlat CLI is a
project that provides cross platform command line interface to manage Azure. Refer
documentation here - https://github.com/Azure/azure-xplat-cli.
But this is another blog on another day.
Conclusion
So I hope now you understand how easy it is to download
azure blob storage contents in Linux Virtual Machine.
Please provide your valuable comments. Good news is its
free!!
Keep Downloading!!
