Uploading large file to Azure blob storage and httpRuntime maxRequestLength

Recently I was told to write a code for uploading large size files to Azure Blob storage from an aspx page. I asked for specifications of size of files. I was told that file to be uploaded will vary from 1MB to anything.

When I had actually implemented a code for file upload, things were working smoothly for the file up to 4MB. For all other files having size more than 4MB, I was receiving many errors.

Those errors included, time out errors, maxRequestLength exceeded, operation could not be completed, and my favorite error – “Internet explorer cannot display the web page”.

I had used normal asp.net file upload control in my web role and a very standard code to upload file to Azure Blob storage. After tweaking some code segments it finally worked. Following is the code segment used for uploading big files on a button click to Azure Blob storage from web role or web site page.

First read all the configuration values from project. I am here using web role project therefore my code to read configuration values will be as follows. If you are using simple asp.net web site project then your code to read configuration values will be different.

protected void btnUpload_Click(object sender, EventArgs e)

{

//using some local variables

string packageFilePath = string.Empty;

string packageFileName = string.Empty;

string referenceFileName = string.Empty;

string packageFileURL = string.Empty;

#region read required configuration values

            Container = RoleEnvironment.GetConfigurationSettingValue("ContainerName");

            AccountName = RoleEnvironment.GetConfigurationSettingValue("AccountName");

            SharedKey = RoleEnvironment.GetConfigurationSettingValue("SharedKey");

            blobEndPoint = RoleEnvironment.GetConfigurationSettingValue("BlobStorageEndpoint");

            tableEndpint = RoleEnvironment.GetConfigurationSettingValue("TableStorageEndpoint");

            #endregion

//create cloud blob client

CloudBlobClient blobClient = new CloudBlobClient(blobEndPoint, new StorageCredentialsAccountAndKey(AccountName, SharedKey));

// For large file copies you need to set up a custom timeout period and using parallel settings appears to spread the copy across multiple threads

// if you have big bandwidth you can increase the thread number below because Azure accepts blobs broken into blocks in any order of arrival.

blobClient.Timeout = new System.TimeSpan(1, 0, 0);

blobClient.ParallelOperationThreadCount = 2;

//get container reference           

CloudBlobContainer container = blobClient.GetContainerReference(Container);

//setup permission on container to be public

var permissions = new BlobContainerPermissions();

permissions.PublicAccess = BlobContainerPublicAccessType.Container;

container.SetPermissions(permissions);

//retrieving file details. uploadCSPKG is the name of my file upload control              

if (uploadCSPKG.HasFile)

{

try

{

                  packageFileName = Path.GetFileName(uploadCSPKG.FileName);

//attaching application name before file name so as to create hierarchy in blob. txtApplicationName is my text box where in user provides sname for logical folder in blob under which file will be uploaded.

referenceFileName = txtApplicationName.Text + "/" + packageFileName;

                  uploadCSPKG.SaveAs(Server.MapPath("~/") + packageFileName);

                  packageFilePath = Server.MapPath("~/") + packageFileName;

            }

            catch (Exception ex)

            {

                throw ex;

            }

      }

//open the stream and upload package file to blob

using (FileStream fs = File.OpenRead(packageFilePath))

{

CloudBlob blob = container.GetBlobReference(referenceFileName);

      blob.UploadFromStream(fs);

      packageFileURL = blob.Uri.AbsoluteUri;

}

}

The above is the complete code required to perform a file upload to Azure blob. Apart from this, you will need to do one more configuration related to ASP.NET runtime setting using httpRuntime tag in web.config of your application.

Add following line in web.config under <system.web> -

<httpRuntime maxRequestLength="51200" enable ="true" executionTimeout="45"/>

Here I assume that, my file size will never exceed the value provided in maxRequestLength attribute. If you feel the size of file to be uploaded can exceed above value then you can change the value as per your need. So with the above setting of maxRequestLength, you should be able to upload file upto sizes of 50MB. If you wish to upload file of size more than 50MB then increase the value under maxRequestLength attribute.

maxRequestLength  - Indicates the maximum file upload size supported by ASP.NET. The size specified is in kilobytes. The default is 4096 KB (4 MB).

Therefore I was able to add 4MB file previously without any difficulty whereas file sizes more than 4MB was failing with various errors. The above code segment resolved the issue.

Hope this helps.

Cheers…

Happy Programming!!!