Ajax file uploads and CSRF (in Django 1.3, or possibly other frameworks)

To begin, this is an update of my old post AJAX Uploads in Django (with a little help from jQuery). This guide is specific to Django, but my version of the file uploader can (theoretically, it is untested) be used with other web frameworks that use CSRF, like Ruby on Rails. You should be able to follow along with the guide and make adjustments as appropriate for your framework.

Required Software

• My version of Valum's file upload
• Python 2.6+
• Django 1.3+

If you are on an older version of Python and/or Django, reading the prior version of this post and especially this Stack Overflow question of mine may provide some help in adjusting the code. The only part that requires updated Python and Django is the save_upload function. The code uses buffered readers/writers and the 'with' keyword from Python 2.6+ (these parts can easily be changed I suspect) and reads from the raw HttpRequest, which comes with Django 1.3+. The Stack Overflow question has code I tried before moving up to requiring these newer software versions. It worked for small uploads below CD ISO size (700MB) and can probably be fixed to work with all uploads, I just found the Django 1.3+ solution easier and quicker at the time.

Overview

Ajax Upload handles the client-side very seamlessly and only gives one challenge to the programmer: it passes the file either as the raw request, for the "advanced" mode, or as the traditional form file for the "basic" mode. Thus, on the Django side, the receiving function must be written to process both cases. As the old post discusses, reading this raw request was a bit of trouble, and that is why I went with Django 1.3 as a requirement for my code.

Setup and Settings

First is to get AJAX Upload installed by downloading the latest version from my Github repo. This fork of Valum's original includes my changes as well as improvements from other forks that I need. As of this writing, I have added correct awareness in FileUploader of FileUploaderBasic's 'multiple' parameter and included David Palm's onAllComplete trigger. Once downloaded, grab fileuploader.js and fileuploader.css out of the client folder and place them wherever is appropriate for your setup. Finally, link them in your HTML via your Django templates.

The Web (Client) Side

HTML

This is the HTML code that will house the upload button/drag area so place it appropriately.

<div id="file-uploader">       
    <noscript>          
        <p>Please enable JavaScript to use file uploader.</p>
    </noscript>         
</div>

Javascript

You probably want to dump this in the same HTML/template file as the above, but it is up to you of course.

var uploader = new qq.FileUploader( {
    action: "{% url ajax_upload %}",
    element: $('#file-uploader')[0],
    multiple: true,
    onComplete: function( id, fileName, responseJSON ) {
      if( responseJSON.success )
        alert( "success!" ) ;
      else
        alert( "upload failed!" ) ;
    },
    onAllComplete: function( uploads ) {
      // uploads is an array of maps
      // the maps look like this: { file: FileObject, response: JSONServerResponse }
      alert( "All complete!" ) ;
    },
    params: {
      'csrf_token': '{{ csrf_token }}',
      'csrf_name': 'csrfmiddlewaretoken',
      'csrf_xname': 'X-CSRFToken',
    },
  } ) ;
}

Now, let's make some sense of that.

• It is probably simplest to use the url template tag to fill in the action as I did above, but it could also be a hard-coded URL as a string. It is set here to match the URL config covered later in this guide.
• The multiple option is not something that is not discussed in Valum's documentation that I found. Its purpose is to limit the uploader to allow you to determine whether it supports selecting/dragging multiple files for upload at a time. A value of true allows multiples, false will let it only do one at a time. In Valum's, this option is available to FileUploaderBasic, but not FileUploader, which is the class most people use. For my repo I chose to update FileUploader to be aware of the multiple option.
• The onAllComplete callback is something added to my repo over Valum's that I got from David Palm's fork. It is called whenever the queue of uploads becomes empty. For example, if you drag/select 4 uploads, this will fire once all 4 have finished. If you then drag/select 2 more files for upload, this will fire again when those 2 are completed.
• The params are set up so the uploader can interact with Django's CSRF framework properly. csrf_token is obviously the token itself, while csrf_name is the name of the input expected by Django for form submissions and csrf_xname is the HTTP header parameter it reads for AJAX requests. Why did I bother with making these last two parameters? Well, theoretically my version of the file uploader should work with other frameworks, which may expect different names for these. For example, Ruby on Rails will expect 'X-CSRF-Token' for AJAX requests and 'authenticity_token' for forms (I think).
• jQuery is used to grab the appropriate part of the div. If you are not using jQuery use whatever method is appropriate for your system to get the file-uploader DOM element. Using regular Javascript you could do document.getElementById('file-uploader'), as Valum uses in the examples on his site.

The Server (Django) Side

Django URLs

It is best to have two views for this setup to work: one to display the upload page and one to process the upload file. The URLs need to be set in urls.py of course.

url( r'/project/ajax_upload/$', ajax_upload, name="ajax_upload" ),
url( r'/project/$', upload_page, name="upload_page" ),

Note that these may require some adjustments depending on how your urls.py is coded.

Views

First is the upload_page view, which is going to display the page with which the user interacts. This is a simple skeleton, add whatever your template needs.

from django.middleware.csrf import get_token
def upload_page( request ):
  ctx = RequestContext( request, {
    'csrf_token': get_token( request ),
  } )
  return render_to_response( 'upload_page.html', ctx )

Including the csrf_token in the context is very important, as earlier code depends on having this variable available. For some reason Django does not give you access to the token automatically in templates.

Next is the view to handle the upload. Remember that this code must handle two situations: the case of an AJAX-style upload for the "advanced" mode and a form upload for the "basic" mode. I split this code up into two functions: one to actually save the upload and the other the view.

def save_upload( uploaded, filename, raw_data ):
  ''' 
  raw_data: if True, uploaded is an HttpRequest object with the file being
            the raw post data 
            if False, uploaded has been submitted via the basic form
            submission and is a regular Django UploadedFile in request.FILES
  '''
  try:
    from io import FileIO, BufferedWriter
    with BufferedWriter( FileIO( filename, "wb" ) ) as dest:
      # if the "advanced" upload, read directly from the HTTP request 
      # with the Django 1.3 functionality
      if raw_data:
        foo = uploaded.read( 1024 )
        while foo:
          dest.write( foo )
          foo = uploaded.read( 1024 ) 
      # if not raw, it was a form upload so read in the normal Django chunks fashion
      else:
        for c in uploaded.chunks( ):
          dest.write( c )
      # got through saving the upload, report success
      return True
  except IOError:
    # could not open the file most likely
    pass
  return False

def ajax_upload( request ):
  if request.method == "POST":    
    if request.is_ajax( ):
      # the file is stored raw in the request
      upload = request
      is_raw = True
      # AJAX Upload will pass the filename in the querystring if it is the "advanced" ajax upload
      try:
        filename = request.GET[ 'qqfile' ]
      except KeyError: 
        return HttpResponseBadRequest( "AJAX request not valid" )
    # not an ajax upload, so it was the "basic" iframe version with submission via form
    else:
      is_raw = False
      if len( request.FILES ) == 1:
        # FILES is a dictionary in Django but Ajax Upload gives the uploaded file an
        # ID based on a random number, so it cannot be guessed here in the code.
        # Rather than editing Ajax Upload to pass the ID in the querystring,
        # observer that each upload is a separate request,
        # so FILES should only have one entry.
        # Thus, we can just grab the first (and only) value in the dict.
        upload = request.FILES.values( )[ 0 ]
      else:
        raise Http404( "Bad Upload" )
      filename = upload.name
    
    # save the file
    success = save_upload( upload, filename, is_raw )

    # let Ajax Upload know whether we saved it or not
    import json
    ret_json = { 'success': success, }
    return HttpResponse( json.dumps( ret_json ) )

The first thing you probably want to edit here is the use of filename in either ajax_upload or save_upload. The saving function as it stands assumes filename is a path. In my actual usage, I combine filename with a constant from settings.py that represents the path to where uploads should be saved. So, at the beginning of save_upload you could have something like filename = settings.UPLOAD_STORAGE_DIR + filename where UPLOAD_STORAGE_DIR it set to something like "/data/uploads/". Or, of course, you could skip the constant and hard code your path string, but that's bad right?

And that's it, go have some fun!

I have had many people ask me via comments or email about providing a demo of this system. From a user standpoint it looks/works no different from the demo on Valum's site. As of right now I cannot provide my own demo because my web host does not provide a Django environment. I'm trying to work with them on getting it available though. I will also work on getting my code in as the Django example with the uploader code in my github repo. When either of those happen I will update this post.

Thanks to everyone who commented on the last post, they helped immensely in creating my github repo and in fixing bugs on the post itself. If you find any mistakes here please comment or contact me directly (contact info can be found on my site).