AJAX Uploads in Django (with a little help from jQuery)

*** April 4, 2010: This post is a bit outdated and does not work with Django 1.3 final's stricter CSRF enforcement. I have a new post that is much more up-to-date, cleaner, and easier to follow, especially because it uses the github repo I created that holds the changes that need to be made to the file uploader's javascript.***

Part of my current project is creating an area where files need to be uploaded in a snazzier way that the normal "browse for a single file" sort of forms. The idea is to have a download button with the same OS chrome file dialog, but one that allows multiple files to be selected. Additionally, the even snappier part, is that HTML5's drag-and-drop functionality should also be available where supported.

After some searching I found a couple of sites that pointed me in the right direction.

• AJAX Upload handles the client-side end of the upload process, including the required multiple-select and drag-and-drop (for browsers that support it). It even handles graceful fallback to iframe uploads for browsers that do not support those advanced features (Opera, IE, etc.).
• On the Django side, I found this site, which gives some pointers to get it working with Django 1.2's CSRF token. The problem I encountered is simply passing the CSRF token to Ajax Upload via its params will not work, because Ajax Upload sends it in the querystring and Django expects it as POST data.

Because neither of those gave me the whole picture, I had to piece things together on my own. Here is the straight-skinny on my findings for getting Ajax Upload to work with Django.

Overview

Ajax Upload handles the client-side very seamlessly and only gives one challenge to the programmer: it passes the file either as the raw request, for the "advanced" mode, or as the traditional form file for the "basic" mode. Thus, on the Django side, the receiving function must be written to process both cases. In the raw request version, reading the request without Django blowing memory was a bit of a challenge. I at first tried reading the data into Django's SimpleFileUpload. It is an in-memory class though, so it runs into issues with large files. Next I tried reading/writing the data through Python functions, which had similar problems. The "works all the time" solution requires Django 1.3, which is to use its new "http request file-like interface" to read from the request. If you're using Django 1.2 and figure out another way to read the request data for all file sizes please comment! I discuss these solutions a little more in depth at this Stack Overflow question of mine.

Setup

First is to get AJAX Upload installed by getting the JS and CSS files wherever is appropriate and linking to them in your Django templates. Also make sure you have set up Django's file upload handlers. Next is setting it up on the site.

The Web (Client) Side

HTML

This is the HTML code that will house the upload button/drag area so place it appropriately.

<div id="file-uploader">       
    <noscript>          
        <p>Please enable JavaScript to use file uploader.</p>
        <!-- or put a simple form for upload here -->
    </noscript>         
</div>

Javascript

You probably want to dump this in the same HTML/template file, but it is up to you.

function generate_ajax_uploader( $url, $csrf_token, $success_func )
{
  var uploader = new qq.FileUploader( {
    action: $url,
    element: $('#file-uploader')[0],
    onComplete: function( id, fileName, responseJSON ) {
      /* you probably want to handle the case when responseJSON.success is false,
         which happens when the Django view could not save the file */
      if( responseJSON.success )
        $success_func( responseJSON ) ;
    },
    params: {
      'csrfmiddlewaretoken': $csrf_token, /* MUST call it csrfmiddlewaretoken to work with my later changes to Ajax Upload */
    },
  } ) ;
}

A little explanation is probably needed here.

• I have wrapped the code inside of a function that generates the uploader because I use it on a couple different pages that require different URLs. You can easily strip off the function part and simply place it in a regular <script> block for use on a single page.
• It is probably simplest to use the url template tag to fill in the action, which is the URL that gets the ajax data and does the server-side processing. I use the url template tag to construct the $url parameter to this function, but if you are removing the function part put the tag directly with action: or you can hard-code the URL you want as a string.
• I use the onComplete callback to pass returned json to another "success function" that parses the json and add information to a table on the page. Again, this is not necessary, but I thought it would be useful to show how this could work. The upload plugin itself will say whether the file upload was a success based on the returned json.
• jQuery is used to grab the appropriate part of the div. If you are not using jQuery use whatever method is appropriate for your system to get the file-uploader DOM element. Using regular Javascript you could do document.getElementById('file-uploader'), as Valum uses in the examples on his site.

Ajax Upload Modifications

I found the easiest way to get the CSRF token piece going was to modify Ajax Upload itself, unfortunately (I hate editing libraries since they need to be re-updated at each new release). Around line 1100 of fileuploader.js you will find the line "var form = ..." within UploadHandlerForm's _createForm method. Replace this line with the following:

var form = null ; 
if( params.csrfmiddlewaretoken )
{
  var csrf = '<div style="display:none"><input type="hidden" name="csrfmiddlewaretoken" value="' + params.csrfmiddlewaretoken + '" /></div>' ;
  form = qq.toElement('<form method="post" enctype="multipart/form-data">' + csrf + '</form>');
  delete params.csrfmiddlewaretoken
}
else
  form = qq.toElement('<form method="post" enctype="multipart/form-data"></form>');

All this code does is search for the CSRF token, and if it is present insert it into the form in the way Django expects to receive it.

The Server (Django) Side

Django URLs

It is best to have two views for this setup to work: one to display the upload page and one to process the upload file. First, the URLs

url( r'/project/ajax_upload/$', ajax_upload, name="ajax_upload" ),
url( r'/project/$', upload_page, name="upload_page" ),

Views

First is the upload_page view, which is going to display the page with which the user interacts. This is a simple skeleton, add whatever your template needs.

from django.middleware.csrf import get_token
def upload_page( request ):
  ctx = RequestContext( request, {
    'csrf_token': get_token( request ),
  } )
  return render_to_response( 'upload_page.html', ctx )

Next is the view to handle the upload. Remember that this code must handle two situations: the case of an AJAX-style upload for the "advanced" mode and a form upload for the "basic" mode.

def save_upload( uploaded, filename, raw_data ):
  ''' raw_data: if True, upfile is a HttpRequest object with raw post data
      as the file, rather than a Django UploadedFile from request.FILES '''
  try:
    from io import FileIO, BufferedWriter
    with BufferedWriter( FileIO( filename, "wb" ) ) as dest:
      # if the "advanced" upload, read directly from the HTTP request 
      # with the Django 1.3 functionality
      if raw_data:
        foo = uploaded.read( 1024 )
        while foo:
          dest.write( foo )
          foo = uploaded.read( 1024 ) 
      # if not raw, it was a form upload so read in the normal Django chunks fashion
      else:
        for c in uploaded.chunks( ):
          dest.write( c )
  except IOError:
    # could not open the file most likely
    return False

def ajax_upload( request ):
  if request.method == "POST":    
    # AJAX Upload will pass the filename in the querystring if it is the "advanced" ajax upload
    if request.is_ajax( ):
      # the file is stored raw in the request
      upload = request
      is_raw = True
      try:
        filename = request.GET[ 'qqfile' ]
      except KeyError: 
        return HttpResponseBadRequest( "AJAX request not valid" )
    # not an ajax upload, so it was the "basic" iframe version with submission via form
    else:
      is_raw = False
      if len( request.FILES ) == 1:
        # FILES is a dictionary in Django but Ajax Upload gives the uploaded file an
        # ID based on a random number, so it cannot be guessed here in the code.
        # Rather than editing Ajax Upload to pass the ID in the querystring, note that
        # each upload is a separate request so FILES should only have one entry.
        # Thus, we can just grab the first (and only) value in the dict.
        upload = request.FILES.values( )[ 0 ]
      else:
        raise Http404( "Bad Upload" )
      filename = upload.name
    
    # save the file
    success = save_upload( upload, filename, is_raw )

    # let Ajax Upload know whether we saved it or not
    import json
    ret_json = { 'success': success, }
    return HttpResponse( json.dumps( ret_json ) )

And that's it, go have some fun!

***Edit: A few errors in the source have been fixed, thank you for your comments!