Trusted answers to developer questions
Trusted Answers to Developer Questions

Related Tags

python
django
ajax
communitycreator

How to implement AJAX file upload in chunks in Django

Shubham Singh Kshatriya

Django allows you to create a server where you can upload any type of file. However, Django will upload the file as a whole and reload the page after it performs an upload, which sometimes forces a user to do only singular tasks. This won’t be bothersome if the file size is small, but things get tricky when the file size increases.

The key point that makes a website flexible is when it allows users to do multiple tasks simultaneously. Imagine you upload a video with a size of 1 GB, and until the video is uploaded, you can’t do anything. How painful!

The idea to upload files as a whole turns out to be really bad when we deal with files in GB size. The concept of uploading files in chunks can be very handy here. A chunk is an instance of a file at a particular time. When you upload files in chunks, this requires breaking your file into smaller chunks to upload each of them synchronously.

In this tutorial, we will see how can we upload a file in chunks to a Django server, through the use of the AJAX request and response cycle.

Set up the project

You can skip this part if you already have a project set up already.

Create a new Django project

django-admin startproject fileUploader
cd fileUploader

Run the project

python manage.py runserver

Create a new app inside the project

python manage.py startapp uploader

Let’s quickly configure our code to include the uploader app in our project.

  • Create the urls.py file in uploader.
  • Configure the project level urls.py file to include the urls of this file
from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('admin/', admin.site.urls),
    path('fileUploader/', include('uploader.urls')),
]
  • Create three folders, namely static, media, and templates under fileUploader.
  • Create two folders, namely css and js inside the static folder.
  • Create a file named index.html inside the templates folder
  • Create a file named app.css inside the css folder
  • Create a file named app.js inside the js folder
  • Configure project level settings.py file to include these changes:
INSTALLED_APPS = [
    ...
    'uploader',
]

TEMPLATES = [
    {
        ...
        'DIRS': [os.path.join(BASE_DIR,'templates')],
        ...
    }
]

STATIC_URL = '/static/'
STATICFILES_DIRS = [
    os.path.join(BASE_DIR, 'static')
]
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
MEDIA_URL = '/media/'

With this we are done with the setup. Let’s move to the actual implementation.

Set up the UI

index.html

This is the HTML file which will contain the UI. We have kept this really simple, and you can style the way you like. As you may have noticed, we will use bootstrap four components and some custom CSS. Make sure to include csrf_token in the form.

{% load static %}
<!doctype html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css"
        integrity="sha384-JcKb8q3iqJ61gNV9KGb8thSsNjpSL0n8PARn9HuZOnIxN0hoP+VmmDGMN5t9UJ0Z" crossorigin="anonymous">
    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css"
        crossorigin="anonymous">
    <title>AJAX + DJANGO File Uploader</title>
    <link rel="stylesheet" href="{% static 'css/app.css' %}">
</head>
<body>
    <div class="col-lg-6 col-md-6" style="margin: 0 auto; display: block; margin-top: 100px;">
        <form enctype="multipart/form-data" method="POST" action="">
            {% csrf_token %}
            <div class="form-group">
                <label>Select file to upload.</label>
                <input type="file" class="form-control" id="fileupload" placeholder="Select file">
            </div>
            <input type="submit" value="Upload" id="submit" class="btn btn-success">     
        </form>
        <div id="uploaded_files"></div>
    </div>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
    <script src="{% static 'js/app.js' %}"></script>
</body>
</html>

app.css

Let’s add some of our own CSS to make our UI more aligned. We also add styles for the progress bar, which will appear dynamically while uploading the file.

#myProgress {
    width: 100%;
}
#uploaded_files {
    margin-top: 25px;
    display: flex;
}
label {
    font-weight: bold;
}
.file-icon i {
    font-size: 60px;
    color: rgb(0, 0, 0);
}
.file-details {
    margin-top: -2px;
    padding-left: 10px;
    width: 100%;
}
.file-details p {
    margin-bottom: -7px;
}
small {
    margin-top: 0;
    color: black;
}

This is how our UI will appear:

app.js

This is the heart of our code. We have used an objected-oriented approach here, but the functional approach would work fine as well with little changes.

There is an attribute called max_length which indicates the maximum size of the chunk that can be uploaded at a time. There is also a method, upload(), which will be called when the click event of button is fired.

class FileUpload {
    constructor(input) {
        this.input = input
        this.max_length = 1024 * 1024 * 10; // 10 mb
    }

    upload() {
        this.create_progress_bar();
        this.initFileUpload();
    }


(function ($) {
    $('#submit').on('click', (event) => {
        event.preventDefault();
        var uploader = new FileUpload(document.querySelector('#fileupload'))
        uploader.upload();
    });
})(jQuery);

nitFileUpload() method

The following is the list of variables that are used:

  • existingPath: Will be null if the file can be uploaded as a whole or contains the path at which the previous chunk was uploaded.
  • nextChunk: The next part of file, if it exists.
  • currentChunk: The current part of file.
  • uploadedChunk: The aggregation of all chunks uploaded so far.
  • formData: n object to hold the data that will be sent to server.
  • end: Whether an upload has ended or not.

First, we create an instance of FormData and append all the values into it that we want to send to the server. Then, we create an instance of AJAX using $.ajax(), which comes with a lot of properties. Here we have used:

  • xhr(): To compute the amount of file that has been uploaded.
  • error(): Called when an error occurs while doing some action.
  • success(): Called when action is successfully completed.
  • url: The url at which the request will be made.
  • type: The request method.
  • dataType: The type in which we pass the data.
  • data: Actual data that will be passed.
upload_file(start, path) {
        var end;
        var self = this;
        var existingPath = path;
        var formData = new FormData();
        var nextChunk = start + this.max_length + 1;
        var currentChunk = this.file.slice(start, nextChunk);
        var uploadedChunk = start + currentChunk.size
        if (uploadedChunk >= this.file.size) {
            end = 1;
        } else {
            end = 0;
        }
        formData.append('file', currentChunk);
        formData.append('filename', this.file.name);
        formData.append('end', end);
        formData.append('existingPath', existingPath);
        formData.append('nextSlice', nextChunk);
        $('.filename').text(this.file.name)
        $('.textbox').text("Uploading file")
        $.ajaxSetup({
        // make sure to send the header
            headers: {
                "X-CSRFToken": document.querySelector('[name=csrfmiddlewaretoken]').value,
            }
        });
        $.ajax({
            xhr: function () {
                var xhr = new XMLHttpRequest();
                xhr.upload.addEventListener('progress', function (e) {
                    if (e.lengthComputable) {
                        if (self.file.size < self.max_length) {
                            var percent = Math.round((e.loaded / e.total) * 100);
                        } else {
                            var percent = Math.round((uploadedChunk / self.file.size) * 100);
                        }
                        $('.progress-bar').css('width', percent + '%')
                        $('.progress-bar').text(percent + '%')
                    }
                });
                return xhr;
            },

            url: '/fileUploader/',
            type: 'POST',
            dataType: 'json',
            cache: false,
            processData: false,
            contentType: false,
            data: formData,
            error: function (xhr) {
                alert(xhr.statusText);
            },
            success: function (res) {
                if (nextChunk < self.file.size) {
                    // upload file in chunks
                    existingPath = res.existingPath
                    self.upload_file(nextChunk, existingPath);
                } else {
                    // upload complete
                    $('.textbox').text(res.data);
                    alert(res.data)
                }
            }
        });
    };

create_progress_bar() method

Here, we create a bootstrap progress bar, which will be shown while uploading a file. It’s always good for users to be able to visualize how much progress has been made.

create_progress_bar() {
        var progress = `<div class="file-icon">
                            <i class="fa fa-file-o" aria-hidden="true"></i>
                        </div>
                        <div class="file-details">
                            <p class="filename"></p>
                            <small class="textbox"></small>
                            <div class="progress" style="margin-top: 5px;">
                                <div class="progress-bar bg-success" role="progressbar" aria-valuenow="0" aria-valuemin="0" aria-valuemax="100" style="width: 0%">
                                </div>
                            </div>
                        </div>`
        document.getElementById('uploaded_files').innerHTML = progress
    }

With this, we are done with the front end. Now let’s build a model and a server to try this code.

Setting up the server (uploader)

urls.py

urls.py contains the url where the request will be made.

urlpatterns = [
    path('', views.index, name='index'),
]

views.py

The requests made to the server are handled by function defined in views.py. When we get a POST request, we retrieve the data and create a new file or append to an existing file inside the media folder and send the path to which the file was stored as a response. Notice that we are storing files in a binary mode.

from django.shortcuts import render
from django.http import JsonResponse
import os
from .models import File

def index(request):
    if request.method == 'POST':  
        file = request.FILES['file'].read()
        fileName= request.POST['filename']
        existingPath = request.POST['existingPath']
        end = request.POST['end']
        nextSlice = request.POST['nextSlice']

        if file=="" or fileName=="" or existingPath=="" or end=="" or nextSlice=="":
            res = JsonResponse({'data':'Invalid Request'})
            return res
        else:
            if existingPath == 'null':
                path = 'media/' + fileName
                with open(path, 'wb+') as destination: 
                    destination.write(file)
                FileFolder = File()
                FileFolder.existingPath = fileName
                FileFolder.eof = end
                FileFolder.name = fileName
                FileFolder.save()
                if int(end):
                    res = JsonResponse({'data':'Uploaded Successfully','existingPath': fileName})
                else:
                    res = JsonResponse({'existingPath': fileName})
                return res

            else:
                path = 'media/' + existingPath
                model_id = File.objects.get(existingPath=existingPath)
                if model_id.name == fileName:
                    if not model_id.eof:
                        with open(path, 'ab+') as destination: 
                            destination.write(file)
                        if int(end):
                            model_id.eof = int(end)
                            model_id.save()
                            res = JsonResponse({'data':'Uploaded Successfully','existingPath':model_id.existingPath})
                        else:
                            res = JsonResponse({'existingPath':model_id.existingPath})    
                        return res
                    else:
                        res = JsonResponse({'data':'EOF found. Invalid request'})
                        return res
                else:
                    res = JsonResponse({'data':'No such file exists in the existingPath'})
                    return res
    return render(request, 'index.html')

models.py

We cannot store data until we have a model. So, here is how we can make one for this uploader:

class File(models.Model):
    existingPath = models.CharField(unique=True, max_length=100)
    name = models.CharField(max_length=50)
    eof = models.BooleanField()

Run these commands in the terminal to migrate your model:

python manage.py makemigrations
python manage.py  migrate

Now we are all set to test our application. Go to your browser and run the serving URL, select a file, and click on upload. You can see the beautiful thing that you just built. Probably, the progress bar filled very quickly, so this time try a larger file (any size you want, it won’t collapse) and see how the file gets uploaded in chunks.

This are few snapshots of the output.

Here is the GitHub repository link for this code.

RELATED TAGS

python
django
ajax
communitycreator

CONTRIBUTOR

Shubham Singh Kshatriya
Copyright ©2022 Educative, Inc. All rights reserved
RELATED COURSES

View all Courses

Keep Exploring