Offline File Upload with FormData and FileReader

29 April 2020

I've been doing an increasing amount of work recently with Progressive Web Applications as browser support becomes more widespread. Much has been written elsewhere on topics around the subject, but one requirement I faced recently, that seems little documented, is that of allowing offline file uploads with a form submittal. In this case the platform was a recruitment site on which I wanted to allow job applications to be submitted without a network connection. I've seen many references to offline form submittals but not with an accompanying file upload. So, in case it's of use, here is my solution based around the FormData and FileReader objects.

My application form has a standard file input as follows:

<input id="userfile" name="userfile"  type="file"> 

 

If no file input was required you could simply store a 'stringified' representation of a FormData object in local storage, then when a network connection is restored, reconstruct the FormData object and submit it asynchronously in the background. That approach needs modification if looking to submit a file; in order to make use of local storage it becomes necessary to represent the file as a string representation rather than the blob that FormData will automatically use when a file element is present. That is where FileReader comes in.

 

File Validation and String Representation

I have a change event listener attached to the file input whch calls the function below. It performs a check on allowable file size and a superficial check of file type based on file extension (I always provide additional validation on the server). If both checks are passed then the readAsDataURL method is used to assign to a variable a base64 encoded string representation of the file, as follows:

var fileData = null,
fileName = null;

const statusField = document.getElementById('formStatus');
const fileInput = document.getElementById('userfile');

fileInput.addEventListener("change", fileValidation);

function fileValidation(){

    let filePath = fileInput.value;
    const allowedExtensions = /(\.pdf|\.rtf|\.doc|\.docx)$/i;

    if(!allowedExtensions.exec(filePath)) {
        statusField.innerHTML = 'Only files of type pdf, rtf, doc, and docx can be accepted';
        fileInput.value = '';
        fileData  = null;
        fileName = null;
    }
    else {
        if (fileInput.files && fileInput.files[0]) {
        	var size = fileInput.files[0].size;
        	if (size > 2e+6) { /*applying a 2Mb limit*/
        		statusField.innerHTML = 'Only files up to 2Mb in size can be accepted.';
        		fileInput.value = '';
        		fileData  = null;
        		fileName = null;
		    }
        	else{
	        	var reader = new FileReader();
	            	reader.onload = function(e) {
		            	fileData = e.target.result;
		            	fileName = fileInput.files[0].name;
	            };
	            reader.readAsDataURL(fileInput.files[0]);
        	}   
        }
    }
}

 

The Form

Before moving on to the mechanism for handling the form submittal itself, a means is required of detecting the presence of a functioning network connection. Much as been written elsewhere about how best to do that so there seems little point in going over that here. Suffice to say that variable browser implementations for Navigator.onLine means you'll probably end up with a hybrid solution of that, plus perhaps an asynchronous ping to a known url  (your favicon for example).. or you could save yourself some work and just use a library such as https://github.com/ryvan-js/wiremonkey.

Before including the code the basic process is as follows:

a) If any form validation checks are passed, and the connection is offline, and the browser supports local storage and FormData then,

b) If the connection is online and form validation checks passed, then the form just submits as normal.

The formdata object is iterable so it's possible to use Array.from() to create an array representation of key / value pairs that can subsequently be converted to a json string and stored.

Here's the code for that, with comments:

const appForm = document.getElementById('applicationForm');

appForm.addEventListener('submit', function(e) {
	  e.preventDefault();

	  if( /*form validation checks passed*/ ) {

	  	/* connOnline is updated, by whatever method you choose, whenever the network state changes */

	  	if (connOnline === false) {

	  		/* make sure the browser supports local storage and the FromData object. Could also include a check on available storage space if expecting to be storing large items, most browers make between 5Mb and 10Mb available */

  			if (typeof Storage !== "undefined" && typeof window.FormData !== "undefined") {  
    		
        		var postData = new FormData(appForm);

        		/* local storage can only accept string data so unset the userfile */
        		postData.delete('userfile');

        		/* and if the dataurl is available append that to the formData object together with the filename */
        		if(fileData){
        			 postData.append('filedata',fileData);
        			 postData.append('filename',fileName);
        		}
        		
        		/* the formData object is iterable so use Array.from to create an array which can then be stored as a json string */
        		var arr  = Array.from(postData)
			    localStorage.setItem('appformData', JSON.stringify(arr));

			    /* store the action URL too .. */
			    localStorage.setItem('appformUrl', appForm.action);

			    statusField.innerHTML = 'Thank you. Your connection appears to be offline at the moment so we have stored your data on your device and will submit it automatically once the connection is restored.';

			   appForm.reset(); 
        	}
        	else{
        		/* the scenario on which a browser supports service workers but not formData or local Storage doesn't exist as far as I know.. but I can't help myself with accommodating this scenario */

        		 statusField.innerHTML = 'Your connection appears to be offline and we were unable to store your data for later submittal. Please check your network connection and try again.';
        	} 

  		}
  		else{ /* the device is online so just submit the form as normal. In this case the form was not being submitted asynchronously but no reason it couldn't be. */

  			appForm.submit();
  		}
	  }
});

 

That's it for handling the form itself.

 

Sending to the server

Once a network connection is restored then the process for sending the stored form to the server looks like this:

Rather than reinvent the wheel, a useful utility function for converting a dataURL to blob can be found here: https://gist.github.com/aermolaev/a96ba40cd68fe357195f. Thanks Alexander, saved me some time.

Here's the code for doing all of that:

function sendOfflinePost()
{
	if (localStorage.getItem('appformData') && localStorage.getItem('appformUrl') ){

		/* get hold of  the form data and parse back into a json object */

		const appformData = JSON.parse(localStorage.getItem('appformData'));

		/* new FormData object */
		let form_data = new FormData();
		let fileBlob = null;
		let fileName = null;


		/* iterate over the data object and append name / value pairs to the FromData object, except for the file data and the filename 
			which will be captured, if encountered, to allow both filedata and original filename to appended to the filedata object */

		for (let i = 0; i < appformData.length; ++i) {
			const item = appformData[i];
			const fieldname = item[0];
			const fieldval = item[1];
				
			if(fieldname != 'filedata' && fieldname != 'filename')
			{
				 form_data.append(fieldname, fieldval);
			}
			
			if(fieldname == 'filedata' ){
				fileBlob = fieldval;
				
			}
			if(fieldname == 'filename' ){
				fileName = fieldval;
			}
			
		}

		/* if fileblob and filename variables are not empty then make a new blob, and append to
		the formData object using the original filename and the expected fieldname */

		if( fileBlob ){

			const blob = dataURLToBlob(fileBlob);

			form_data.append('userfile',blob, fileName);
		}

		/* send to the server, set these parameters to submit a formData object with jQuery */
		$.ajaxSetup({
		  	processData: false,
			contentType: false,
		});


		var request = $.post( localStorage.getItem('appformUrl'), form_data, function( data ){

			/* clear the storage if the server indicates successful receipt */
			if(!data.error){
				localStorage.removeItem('appformData');
				localStorage.removeItem('appformUrl');
			}
			else {
				/* do something here to let the user know what what went wrong at the server end */
			}

		}, "json");

		request.fail(function(jqXHR){
        /* let the user know the request failed if necessary */
        });
	}
}

function dataURLToBlob(dataURL) {  //https://gist.github.com/aermolaev/a96ba40cd68fe357195f
    var BASE64_MARKER = ';base64,';

    if (dataURL.indexOf(BASE64_MARKER) == -1) {
        var parts = dataURL.split(',');
        var contentType = parts[0].split(':')[1];
        var raw = decodeURIComponent(parts[1]);

        return new Blob([raw], {type: contentType});
    }

    var parts = dataURL.split(BASE64_MARKER);
    var contentType = parts[0].split(':')[1];
    var raw = window.atob(parts[1]);
    var rawLength = raw.length;

    var uInt8Array = new Uint8Array(rawLength);

    for (var i = 0; i < rawLength; ++i) {
        uInt8Array[i] = raw.charCodeAt(i);
    }

    return new Blob([uInt8Array], {type: contentType});
}

 

All that's required then is to look out for a network connection and send. This project was using the aforementioned wiremonkey.js so I just did:

window.onload = function(){

		WireMonkey.init();

		WireMonkey.on('connected', function(){
		  connOnline = true;
		  sendOfflinePost();
		});
		
		WireMonkey.on('disconnected', function(){
		  connOnline = false;
		});
}

 

This is a first pass so I'm sure the above could be improved.. let me know if you do, and if this works for you, or it doesn't.