The Dark Side of File Uploads

Posted on January 12, 2006

26 comments

I saw a December MSDN article, entitled Uploading Files in ASP.NET 2.0, and wanted to offer my comments on some gotchas with uploading files. I’ve spent countless hours and tried numerous hacks to tame file uploading and have enough bruses from hitting my head against the wall (figuratively speaking).

ASP.NET 1.x shipped with the HtmlInputFile control, while 2.0 has a brand new, FileUpload control, although its HTML counterpart is still there. As a quick recap, you declare an upload control as follows:

[ASP.NET 1.x]
Select File To Upload to Server:
<input id="MyFile" type="file" runat="server" />

[ASP.NET 2.0]
Select File To Upload to Server: 
<asp:FileUpload id="FileUpload1" runat="server" />

Uploading files in ASP.NET is very inefficient. To be fair, IIS is a bigger offender than ASP.NET itself. When you pick a file and submit your form, IIS needs to suck it all in and only then you have access to the properties of uploaded file(s). IIS 5 does it this way. IIS 6 does it this way. IIS 7 promises to be more like Apache in this respect. Until then, there’s not much you can do about the fact that you have to sit through a long upload and wait. Neither can you display a meaningful progress bar because there’s no way to know how much is transmitted at any given time.

Once IIS buffers your upload, ASP.NET takes it from there. By default, you can upload no more than 4096 KB (4 MB). To raise this limit, you need to adjust maxRequestLength in the <httpRuntime> config section.

The larger the file, the longer it takes to upload. ASP.NET kills requests that take too long; consequently you also need to increase executionTimeout. In 1.0 and 1.1, the default is 90 seconds, in 2.0—110 seconds.

There’s also a new shutdownTimeout attribute, but I don’t understand its purpose yet.

Files That Are Too Large

It gets really interesting if someone uploads a file that is too large. Regardless of what your maxRequestLength setting mandates, IIS has to guzzle it (remember?), and then ASP.NET checks its size against your size limit. At this point it throws an exception. Peek inside the GetEntireRawContent() method of HttpRequest and you see this:

HttpRuntimeConfig config1 = 
(HttpRuntimeConfig) this._context.GetConfig("system.web/httpRuntime");

int num1 = (config1 != null) ? config1.MaxRequestLength : 0x400000;
if (this.ContentLength > num1)
{
  this.Response.CloseConnectionAfterError();
  throw new HttpException (400, 
   HttpRuntime.FormatResourceString("Max_request_length_exceeded"));
}

The rest of this method assembles the file piece by piece in case the file was preloaded only in part, and then checks if its size exceeds the imposed limit. In either case, if an end-user uploads an oversized file, he/she will see a timeout “white page of death”. I put together a sample project with a custom error page, but I always get a white page instead.

Since it is theoretically possible that the file isn’t loaded in full, I’d like to know if one can configure IIS to read it in chunks. I haven’t seen any guidance on this, and I’ve never seen articles that explain it. If somebody out there knows, please share.

How Do I Save Face?

It’s difficult to explain to an end-user that it’s not their fault that the file happened to be too large or the page took too long to upload a file and timed out. Is there a way to tap into this process early and save face by failing gracefully? I have a couple of ideas, none of them perfect.

You may override Page.OnError (see how) and inspect the HTTP code, which should be 400, if the exception happens to be of type HttpException. This is kludgy.

You may also implement an HttpModule, set up its BeginRequest handler and compare Request.ContentLength with the size limit (which you can read straight from web.config). If ContentLength is too high, redirect to a page with a meaningful error message. I believe ContentLength may or may not reflect the total size of the uploaded file, so this approach is not 100% accurate.

Custom HttpModule to Track Progress

I think the best and most accurate solution would be to implement an HttpModule whose sole purpose would be to read a file in chunks and keep the page alive. This way it won’t time out, and you’ll be able to track progress and cancel an upload. Telerik has such a server control + HttpModule combo for sure. Other vendors should have similar offerings.

Uploading Multiple Files

The samples I’ve seen demonstrate 3–5 file field controls, all statically declared. Why 3? Why 5? What’s the magic number? There’s none. Since none of them show how to add file fields on the fly, I decided to write a sample that does.

When you upload several files from the same page, you can access them all via the Request.Files collection:

HttpFileCollection uploads = HttpContext.Current.Request.Files;

Let’s declare one file field which you can treat as an instance of HtmlInputFile thanks to the runat="server" attribute.

<p id="upload-area">
 <input type="file" runat="server" size="60" />
</p>
<p>
 <a href="#" onclick="addFileUploadBox(); return false;">Add file</a>
</p>
<p>
 <asp:Button ID="btnSubmit" runat="server" 
   Text="Upload" OnClick="btnSubmit_Click" />
</p>

This snippet has a link which ads a file field on the fly when clicked. addFileUploadBox is a JavaScript function that performs some DOM manipulation. I noticed that as long as you have at least one HtmlInputFile or FileUpload contol on the page, you can add as many other file fields as you want, and ASP.NET will nicely package them into the Request.Files collection. Go figure.

By clicking the Add file link you add multiple file fields, assign their id and name attributes (otherwise corresponding files won’t be thrown into Request.Files), and add them to the download area.

All this with only a single server-side upload control! The server-side code on the bottom shows how to process all uploaded files. Remember to check for zero file size in case someone added an upload box but didn’t pick a file. Feel free to copy and paste sample code and play with it.

Conclusion

File uploading in ASP.NET is a very imprecise and imperfect science. This is one area I’d love to see improved in the future. If you find yourself struggling with it, don’t worry—you’re in good company. Stick with stock server controls for rudimentary uploads. Otherwise look around for third-party products.

26 comments

Dean Brettle
on January 21, 2006

It is possible to use an HttpModule to provide a progress bar while the upload occurs. For example, my open source NeatUpload package does that. See:

http://www.brettle.com/neatupload

--Dean


scottgu
on January 21, 2006

ASP.NET 2.0 now spools uploaded files to disk, so that it doesn't chew up memory as you are doing a file-upload (whereas ASP.NET V1.1 always loaded it in memory). This should help a lot with memory pressure.

Hope this helps,

Scott


Milan Negovan
on January 25, 2006

Thanks, Scott. Any improvements in file uploads are appreciated!


Jesse Gavin
on February 08, 2006

I have been using Chris Hynes' SlickUpload for the last few projects and it seems to be a pretty good tool. - It handles really large files and allows you to query progress.

http://www.assistedsolutions.com/components/SlickUpload/


Jesse Gavin
on February 08, 2006

I have been using Chris Hynes' SlickUpload for the last few projects and it seems to be a pretty good tool. - It handles really large files and allows you to query progress.

http://www.assistedsolutions.com/components/SlickUpload/


Travis Musika
on March 16, 2006

I created my own solution about a year and a half ago using the following thread as guidance:
http://forums.asp.net/1/55127/ShowPost.aspx

It's a lengthy thread with lots of details (and has probably changed a lot since I first rolled my own version), but I believe it's similar to what SlickUpload and the other solutions do. It works because IIS preloads about 64 K of the entity body, but then it gives ASP.NET a chance to handle it - at which point you use an HttpModule at BeginRequest to intercept the request.
Then, basically, you use reflection to get the HttpWorkerRequest object from the HttpContext, tap into some of the private fields for the content type and size, and use the ReadEntityBody method of the worker request to loop through the http entity body and parse it into form fields and uploaded files. Stream the files to the filesystem and push the form fields and other headers/boundaries back into the preloaded content field. This will prevent ASP.NET from automatically storing the files in its HttpFileCollection object, so you'll need to track them separately.
While parsing the entity body, you can record your stats (current filename, bytes received, time taken, bytes remaining, etc) and access them from a separate thread. I used a popup window with AJAX-style javascript to retrieve the stats and display a progress bar + details (it downgrades to using a meta refresh for incapable or js-disabled browsers).
It works on ASP.NET 1.1 - IIS5.1/6 (at least I haven't tested it more widely), but since we control our server environment it's good enough for the time being. I haven't used any other solutions out there for .NET, but we used to use ASPUpload in our ASP Classic apps.

Travis


Joe Audette
on March 24, 2006

Hi Milan,

My friend Dean Brettle has built an amazing free lgpl licensed upload control for .NET named NeatUpload. It does uploads with a nice proggress bar and works both on IIS and on Mono

http://www.brettle.com/neatupload

I'm using it in my mojoportal project and it really works great.

Cheers,

Joe


McCulloch
on March 24, 2006

Good information. Thank you.


Jesse
on April 06, 2006

Code examples are deprecated for .Net 2.0

try this:

HttpRuntimeSection config1 = (HttpRuntimeSection)this.Context.GetSection("system.web/httpRuntime");


tejaswini das
on May 01, 2006

How to select multiple files from a single HtmlInputFile control to upload multiple files at a time


kufu
on May 09, 2006

ASP.NET 2.0 allows buffering to disk to avoid the memory problem. Just add this in the web.config or machine.config:

executionTimeout="10000"
maxRequestLength="512000"
requestLengthDiskThreshold="80"
useFullyQualifiedRedirectUrl="false"
minFreeThreads="8"
minLocalRequestFreeThreads="4"
appRequestQueueLimit="5000"
enableKernelOutputCache="true"
enableVersionHeader="true"
requireRootedSaveAsPath="true"
enable="true"
shutdownTimeout="90"
delayNotificationTimeout="5"
waitChangeNotification="0"
maxWaitChangeNotification="0"
enableHeaderChecking="true"
sendCacheControlHeader="true"
apartmentThreading="false" />

Note: you're only interested in the first 3 lines - max request timeout (s), max file size (kb) and max memory buffer size (kb). So uploading a file of any size will only take 89kb of memory, after which it is buffered to disk in a temporary folder until upload is complete.


jil
on July 18, 2006

what is spools and why to use spools?
as above information to write some code in web.config but where to write this code in wich tag....?


ganu
on September 21, 2006

where the upload files are saving..if want to save it in my desired folder..where i should had the string..

anyone plz..


Scott Perham
on October 10, 2006

Hi,

I have written an AJAX upload/progress HttpModule, now the files are uploaded fine, but the browser always hangs until the file has completed uploading, so the progress bar only appears once the upload is finished (not really much use!)... i've been doing a LOT of research on this and it seems that im not the only person with this problem, but people do solve it (but dont seem to want to explain how)

If anyone knows how to solve this... pleeeeease help, its driving me crazy!!!

Thanks


harinath
on January 16, 2007

Is there any way to get rid of httpRequsetLength limit of 2GB.

I am not able to set the request size to 3GB or so. because its not letting me set anything greater than 2 GB.


cisco kid
on March 27, 2007

Hi - My problem relates to viewing picture files before uploading it. In order to do this I've tried to implement some javascript which fetches the id of the input control which has the type set to 'file'. however the path is contained in the form c:\folder\filename.ext where as the img control requires an Internet URL. Most people who want to upload picture files hold them locally. I've converting the path into file:///c:/...etc but this does not seem consistant. Any ideas?


cisco kid
on March 27, 2007

Hi - My problem relates to viewing picture files before uploading it. In order to do this I've tried to implement some javascript which fetches the id of the input control which has the type set to 'file'. however the path is contained in the form c:\folder\filename.ext where as the img control requires an Internet URL. Most people who want to upload picture files hold them locally. I've converting the path into file:///c:/...etc but this does not seem consistant. Any ideas?


Gagan
on May 25, 2007

Hi
I am using FileUpload control in asp.net 2.0. Even after writing above mentioned entry in web.config, I am getting the error message "Page can not be displayed" / "DNS Error".
How can i avoid it or atleast tap it and show some error message. This is happening when i try to upload files larger than the maxRequestLength value.
Thanks is advance
Gagan


Yanela Somdaka
on July 18, 2007

Hi All

I just want to say thank you to all in this forum, i have been strugling to get a good file uploader, but now with all the information you have provided i am going to develop my own control and will post it here so you guys can have a look and advice on it.

Thanks a mill guys (From South Africa)


Pimpl
on October 13, 2007

Hi.
Do U know another ways to read file in chunks but HttpWorkerRequest ?..
Does ASP.NET have standart things for this ? (
Using HttpWorkerRequest isn't so handy for that purpose I think...


Milan Negovan
on October 15, 2007

There's no "standard thing" in ASP.NET. However, there was a rather lengthy discussion at ASP.NET Forums. Take a look. Maybe it'll give you some ideas.


Zetix
on February 07, 2008

Hi all!

I have been using RADactive I-Load for the last few projects:

http://www.radactive.com/en/Products/ILoad/Image_Resize_Crop_Upload.aspx

it manage only Image uploads but seems to be a very good tool!

Cheers,

Zetix


venu maddineni
on March 12, 2008

Hi
I have a problem i m trying to download an excell size of 570kb
but the system throwing Max length exceed exception
as of my know ledge i know the default behaviour will be upto 4MB
but my site is not allowing after 500KB
can any body please help me out in this

only if i increse the size more than 4MB then only its allowing to download
but comparitively this is happening even the file size is 570 KB(is very small figure)
so what would be the problem in this


rumata
on June 18, 2008

http://www.15seconds.com/issue/071025.htm

>> Summary
>>
>>
>> Uploading large files can have a tendency to swamp your web server.
>> This solution allows for many uploads of large files because each upload is only handled a chunk at a time.
>> As each chunk is read in, it is immediately put in the database.
>> The database keeps track of the chunk count and size of the content.


Milan Negovan
on June 18, 2008

That's an interesting approach. Thank you!


Hanna
on September 30, 2008

Hi,
First of all thanks to all guys for such a nice discuss I have need the file upload control for my site but now after reading your guide and information posted by you guys I am going create you own file upload control thanks

Regards
Hanna