Upload to Django with progress bar using Ajax and jQuery

September 30, 2010

In this article, I am going to describe how I implemented upload to Django + progress bar with Ajax and jQuery. I needed this feature so users could post their dish pictures on Gourmious and follow the upload’s progress.

Client Side

We need a form so the user can select a file to upload.

<form id="form_upload" action="/upload" method="POST">
  <input type="file" name="picture" id="picture" />
  <input type="hidden" id="X-Progress-ID" name="X-Progress-ID" value=""/>
  <input type="hidden" id="id" name="id" value=""/>
  <input id="form_submit_button" class="tp-button" type="submit" value="Submit" />
  </form>

We added 2 hidden inputs, first one is ‘X-Progress-ID’ which is the upload ID so we can support simultaneous uploads on the server side. We will see later how that value is handled by the server.

We also have the hidden input ‘id’ representing the dish ID in our case.

We want to use Ajax to send the POST request so it can be integrated nicely in a modern web interface along with a progress bar. To support that, we are going to use the jQuery Form plugin.

The function ajaxSubmit() is going to take care of everything for us.

We generate a random string for this upload ID and set the input value to that string.
We need to specify a URL to be called for the upload and 2 callback functions: one to be called before the request and 1 after.

$('#X-Progress-ID').val('random string');
var options = {
  dataType: 'xml',
  url: '/upload?X-Progress-ID='+$('#X-Progress-ID').val(),
  beforeSubmit: showRequest,
  success: showResponse
}
$('#form_upload').ajaxSubmit(options);

The showRequest callback can be as simple as:

function showRequest(formData, jqForm, options) {
    // do something with formData
    return True;
}

In the showResponse function, you need to process the response and act on it. In my case, I process some xml returned by the server with a status value.

function showResponse(response) {
    // do something with response
}

When the user presses submit, we want to display a progress bar so we use the following JS code to add a progress bar to the form. The progressBar() method is part of the jQuery progress bar plugin.

$('#form_upload').find('#form_submit_input').append('&lt;span id="uploadprogressbar"&gt;&lt;/span&lt;');
$('#form_upload').find('#uploadprogressbar').progressBar();

Now, we need to add a function running every few seconds to get the upload progress from the server and update the progress bar accordingly.

To do that, we use setInterval() to issue a GET request to the server to get the progress value using the JSON format. We pass the upload ID to the server. When the value null is returned, we know that the upload is finished.

function startProgressBarUpdate(upload_id) {
  $("#uploadprogressbar").fadeIn();
  if(g_progress_intv != 0)
    clearInterval(g_progress_intv);
  g_progress_intv = setInterval(function() {
    $.getJSON("/get_upload_progress?X-Progress-ID="
+ upload_id, function(data) {
      if (data == null) {
        $("#uploadprogressbar").progressBar(100);
        clearInterval(g_progress_intv);
        g_progress_intv = 0;
        return;
      }
      var percentage = Math.floor(100 * parseInt(data.uploaded) / parseInt(data.length));
      $("#uploadprogressbar").progressBar(percentage);
    });
  }, 5000);
}

Server side

First, we need a function in views.py to handle the upload. This function handles the request: “/upload?X-Progress-ID=xxxx”. We are reading the file chunk by chunk to not use too much RAM. In my case, I return some xml containing the status value: upload OK or not.

def upload(request):
  id = request.POST['id']
  path = '/var/www/pictures/%s' % id
  f = request.FILES['picture']
  destination = open(path, 'wb+')
  for chunk in f.chunks():
    destination.write(chunk)
  destination.close()
  # return status to client
  ...

How do we keep track of the upload progress? We need to use a different file upload handler. We are going to use UploadProgressCachedHandler. We just need the class from this snippet, not the view function which we are going to write ourself. You can add the class to a file named uploadprogresscachedhandler in your project.

This handler saves the upload progress to the cache so it can be retrieved easily when we receive the requests from the client.

To enable this handler, we need to add the following to settings.py:

from django.conf import global_settings
FILE_UPLOAD_HANDLERS = ('uploadprogresscachedhandler.UploadProgressCachedHandler', ) \
+ global_settings.FILE_UPLOAD_HANDLERS

We also need to enable the cache system. We are going to use memcached. This goes inside settings.py too.

CACHE_BACKEND = 'memcached://127.0.0.1:11211/'

You need to make sure memcached and the python bindings are installed on your server.

We need to add a function in views.py to return the upload progress asked by the client every few seconds during the upload. This function handles the request “/get_upload_progress?X-Progress-ID=xxxx”. The progress value is stored using the key “remoteaddress_uploadid”.

from django.utils import simplejson
from django.core.cache import cache
def get_upload_progress(request):
  cache_key = "%s_%s" % (request.META['REMOTE_ADDR'], request.GET['X-Progress-ID'])
  data = cache.get(cache_key)
  return HttpResponse(simplejson.dumps(data))

That’s it for now. Don’t hesitate to add comments to discuss more.

If you like this article, it would be cool if you can share your favorite restaurant dishes on Gourmious.

tags:
posted in Uncategorized by Laurent Luce

Follow comments via the RSS Feed | Leave a comment | Trackback URL

16 Comments to "Upload to Django with progress bar using Ajax and jQuery"

  1. Ben wrote:

    great post, but I’m missing something. Why is the UploadProgressCachedHandler.handle_raw_input() checking for the X-Progress-ID in the request.GET…I see the value in the request.POST instead…

    thanks

  2. laurent wrote:

    @Ben

    It is passed as a POST parameter because it is present in the form but it is also passed as a GET parameter.

    If you look at the ajaxSubmit options, you will see that X-Progress-ID is passed in the URL so it will be a GET parameter.
    url: ‘/upload?X-Progress-ID=’+$(‘#X-Progress-ID’).val()

    X-Progress-ID is also a hidden value in the form so calling ajaxSubmit will result in this parameter being part of the POST data.

    This is a bit ugly to mix POST and GET parameters in the same request but I prefer that than modifying UploadProgressCachedHandler because it is a third-party piece of code I am using.

  3. Roberto wrote:

    Hey man, i have a little dude, i hope you can solve me.

    in this part

    $.getJSON(“/web_modules/process.py?request=get_upload_progress&X-Progress-ID=”

    My question is what i found in process.py

    because i can see the method get_upload_progress is found in uploadprogresscachedhandler, but i’m not understand why process.py i’m have less one month work with django and i have many doubts.

    thank you for your help :-)

  4. laurent wrote:

    @Roberto: You need to define a relation in urls.py between the URL request and the function in views.py. Let say you are using $.getJSON(“/get_upload_progress?X-Progress-ID=xxxx”) to retrieve the upload progress value then you can add the following in urls.py to define the relation:

    urlpatterns = patterns(”,
    (r’^/get_upload_progress?.*$’, ‘app.views.get_upload_progress’),
    )

    uploadprogresscachehandler is used internally by Django during the upload to update the upload progress value stored in memcached. It is different than the get_upload_progress() function in views.py used by the browser to retrieve the upload progress value from memcached.

  5. Rajasekar wrote:

    Can anybody tell when to call the startprogressupdate() javascript method. My code first uploads the file then calls the startprogressupdate method. I am confused

  6. Laurent Luce wrote:

    Yes, you first call ajaxSubmit(options) to start the file upload and then you call startProgressBarUpdate() to launch the progress bar update requests.

  7. Callebe wrote:

    hello.

    in def get_upload_progress(request):
    where is defined object ‘cache’ ?

    I need import something?

    thank you.

  8. Laurent Luce wrote:

    @Callebe: You need to import django.core.cache.cache. I updated the code snippet. Thanks.

  9. Ryan wrote:

    This is great. I think the line:

    (uploadprogresscachedhandler.UploadProgressCachedHandler’, ) \

    needs an apostrophe after the opening parenthesis.

  10. Laurent Luce wrote:

    @Ryan: Thanks for noticing it. I fixed it.

  11. Aamir Adnan wrote:

    Hi, i am trying to follow your instructions. But i am stuck at one place, what does “def upload” return when it receives request??

  12. Laurent Luce wrote:

    @Aamir I left out the returned value because it is app dependent. In my case, I am returning some xml containing a status code and I have some javascript code handling it in showResponse.

  13. Josh wrote:

    Thanks for this basic intro, I have learned a lot already. I’m trying to implement this (just a test for now), but I am missing some key bits of info. The main problems I’m having:
    - I know that I will need to supply a function/handler for ‘showRequest’ and ‘showResponse’ but I don’t know what should go into these. Could you update your examples above with something simple?

    - I know you said that it is app dependent, but for purposes of filling out this tutorial, could you post more of your ‘upload’ view routine, specifically the return?

    - could you discuss more about how to map the ‘upload’ and ‘get_upload_progress’ in urls.py, and how that relates to the view containing the upload form? Do I need 3 entries in urls.py? (one for the form, one for ‘upload’, and one for ‘get_upload_progress’?

    - any idea how this might be modified for handling multiple files in a Django-friendly manner?

  14. Laurent Luce wrote:

    @Josh. I added examples for showRequest() and showResponse(). In my case, I return some xml in upload() containing a status value which is parsed in showResponse().

    You are correct that you need to add url patterns in urls.py for upload(), get_upload_progress() and the upload form. I build the upload form using javascript and display it as a dialog box to avoid page navigation so I don’t have a form url.

    Regarding handling multiple files, HttpRequest.FILES in Django contains the list of files posted. upload() should be modified to handle multiple files. The rest can stay the same.

  15. Sohaib Iftikhar wrote:

    I am pretty new to ajax.
    What I wish to know could I extend the same for uploading to amazon s3?
    That cache CACHE_BACKEND
    What is that used for?

  16. Laurent Luce wrote:

    @Sohaib: Regarding S3, it depends if their API gives feedback on the upload progress. The cache backend is used to store the upload progress value.

Leave Your Comment

 
Powered by Wordpress and MySQL. Theme by Shlomi Noach, openark.org