Building an HTML5 Drag & Drop File Uploader Using Sinatra and jQuery: Part 1

Upon setting out to build a Drag & Drop uploader, I first trawled the web for prior art. Blog posts, examples, any information that would lead me down a well-trodden path. As it turns out, such a path doesn’t really exist yet.

FileReader: A Dead End

Any cursory research into HTML5 file handling will undoubtedly turn up mention of the FileReader class which has made an appearance in the most recent versions of Chrome and Firefox. FileReader is useful, in that it allows a browser to read data from files, either all at once, or in chunks, however it quickly becomes unworkable when you try to use it to read large files for uploading to a server. This is due to the fact that you need to read the entire file in one fell swoop to pass off to an XMLHttpRequest, and buffering this much data (this much being 50MB or more) will cause every browser I tested to become completely unresponsive.

XMLHttpRequest Level 2: The Missing Piece

Why haven’t more drag and drop multi-file uploaders shown up on the web before now? Certainly lack of support for gathering file information from drop events was one reason, but another was the fact that until recently, XMLHttpRequest didn’t provide any mechanism for sending file data to the server. Some clever hacks abound, such as using an IFrame or Flash, but none of these would really work for our purposes.

It just so happens that the XMLHttpRequest spec has recently been revised by the W3C: enter XMLHttpRequest Level 2. Here’s the description from the aforelinked working draft:

The XMLHttpRequest Level 2 specification enhances the XMLHttpRequest object with new features, such as cross-origin requests, progress events, and the handling of byte streams for both sending and receiving.

Wow, that sounds perfect! So in supported browsers, an XHR will have an upload attribute, which is an instance of XMLHttpRequestUpload. You can even get progress events from this object, allowing you to build progress tracking into your uploader. This appears to be supported in Firefox 3.5+, Safari 5+, and recent versions of Chrome stable (6.0.472.63 as of writing, but it’s probably supported in earlier versions).

There is one caveat to this approach: there’s currently no good way to perform your XHR file upload as a multipart form post. Experimental support for this has been added in recent builds of Firefox 4, using the FormData interface, however currently the only way to reliably submit a file in a multipart form using XHR in existing browsers is to use FileReader, read the entire thing into memory, then manually build a multipart post. So in other words, it’s a non-starter. Instead, a XMLHttpRequestUpload can be sent with a reference to a file, which the browser will stream from the filesystem as the body of the request. This may require some extra work on the server side, if you want to avoid buffering the entire file inside the process handling the request. For our part, we’ve made some modifications to the venerable nginx upload module, allowing it to also accept PUT requests with raw file contents in the body.

Drag & Drop: Just Enough to Get By

People with far more knowledge and tenacity than I have written eloquent sonnets declaring their undying love for the HTML5 drag & drop API. Unfortunately, it’s the only game in town if you want to do interesting things with files dropped on the browser window. For the remainder of this post, and in subsequent posts, I’ll be building an uploader from the ground up, adding features and complexity as I go. For now, the simplest thing that can possibly work: a drop target on the page, which listens for drop events and, if any files are dropped, creates XMLHttpRequests to send the files to a URL. Things get a lot more dicey when you want to make a complex element (like a table) into a drop target, highlight the drop area, or show information underneath the mouse, so we’ll go for progressive enhancement and add that stuff in later.

Where The Rubber Meets Brass Tacks

This example micro-app will use Sinatra, and have two endpoints, one that displays a page with a drop target, and one that accepts a file upload. Follow the code on github. We’re implementing the front-end code as a jQuery plugin, but most of the concepts can be adapted to Prototype or even plain Javascript.

First, we create a simple page with a drop target div on it. We also include a CSS file and links to both jQuery and our uploader Javascript file. Lastly, we attach an uploader to the drop target:

  <html>
    <head>
      <title>Drag &amp; Drop Tacos</title>
      <link rel="stylesheet" href="/css/master.css" type="text/css" media="screen" title="no title" charset="utf-8">
    </head>

    <body>
      <div id="drop_target">
      </div>
    </body>

    <script type="text/javascript" charset="utf-8" src="/javascripts/jquery-1.4.3.js"></script>
    <script type="text/javascript" charset="utf-8" src="/javascripts/jquery.dnduploader.js"></script>
    <script type="text/javascript" charset="utf-8">
      $("#drop_target").dndUploader({
        url : "/"
      });
    </script>
  </html>

The Ruby code to support this is super simple:

  require 'rubygems'
  require 'sinatra'
  require 'erb'

  get '/' do
    erb :"index.html"
  end

To run the app, make sure you have the Sinatra gem installed (gem install sinatra … depending on your system, you might have to sudo), drop into the example app directory, and type ruby app.rb. WEBrick should start up, and you should be able to access the application at localhost:4567.

Now, let’s take a look at the jQuery plugin. I’ll dispense with the boilerplate plugin code, but if you’d like a refresher, the jQuery documentation will get you up to speed. This is a first pass at the plugin– it won’t really do any uploading yet, but it’ll allow you to drag and drop files onto the drop target and prevent the browser from trying to open them:

(function( $ ){

  var methods = {
    init : function( options ) {

    return this.each(function(){

       var $this = $(this);

       $this.bind('dragenter.dndUploader', methods.dragEnter);
       $this.bind('dragover.dndUploader', methods.dragOver);
       $this.bind('drop.dndUploader', methods.drop);
     });
    },

    dragEnter : function ( event ) {
      event.stopPropagation();
      event.preventDefault();

      return false;
    },

    dragOver : function ( event ) {
      event.stopPropagation();
      event.preventDefault();

      return false;
    },

    drop : function( event ) {
      event.stopPropagation();
      event.preventDefault();

      return false;
    }
  };

  $.fn.dndUploader = function( method ) {
    if ( methods[method] ) {
      return methods[method].apply( this, Array.prototype.slice.call( arguments, 1 ));
    } else if ( typeof method === 'object' || ! method ) {
      return methods.init.apply( this, arguments );
    } else {
      $.error( 'Method ' +  method + ' does not exist on jQuery.dndUploader' );
    }
  };
})( jQuery );

The first order of business is to intercept three drag & drop events: dragenter, dragover, and drop. In order to define your own behavior for drag & drop, the default behavior and event bubbling must be cancelled in your event handler. This bears repeating- if you don’t cancel event propagation, the browser’s default drop handling behavior will take over, and your code won’t work at all. So, we write three identical functions to handle these events:

  dragEnter : function ( event ) {
    event.stopPropagation();
    event.preventDefault();

    return false;
  },

  dragOver : function ( event ) {
    event.stopPropagation();
    event.preventDefault();

    return false;
  },

  drop : function( event ) {
    event.stopPropagation();
    event.preventDefault();

    return false;
  },

Then, in our init function, we bind our event handlers to the appropriate events:

  init : function( options ) {

  return this.each(function() {

     var $this = $(this);

     $this.bind('dragenter', methods.dragEnter);
     $this.bind('dragover', methods.dragOver);
     $this.bind('drop', methods.drop);
   });
  }

So, with all of that, we should now have a drop target primed to accept items dragged onto it.

Getting information out of the MouseEvent

When files are dropped into our target, the event carries with it important information about what items were dropped. This information can be found in the dataTransfer property of the MouseEvent. The following modification will allow us to list the contents:

  drop : function( event ) {
    event.stopPropagation();
    event.preventDefault();

    console.log( event.originalEvent.dataTransfer.files );

    return false;
  }

If you’ve been following along, when you drop a couple of files onto the target, you should see something like this:

  FileList
    0: File
      fileName: "Scan 2.pdf"
      fileSize: 4999673
      name: "Scan 2.pdf"
      size: 4999673
      type: "application/pdf"
      webkitRelativePath: ""
      __proto__: File
    1: File
      fileName: "Scan.jpeg"
      fileSize: 943332
      name: "Scan.jpeg"
      size: 943332
      type: "image/jpeg"
      webkitRelativePath: ""
      __proto__: File
      length: 2
      __proto__: FileList

So, we can easily get the filename, size, and type from the objects in the dataTransfer. Next, we’ll loop through and upload each file. Since this only works with XMLHttpRequest Level 2, we won’t bother with interfaces that abstract away the XMLHttpRequest object- we’ll create one and manipulate it directly. First though, we should make sure to store the url, and any other options, on the uploader node:

  return this.each( function () {

    var $this = $(this);

    $.each(options, function( label, setting ) {
      $this.data(label, setting);
    });

    $this.bind('dragenter.dndUploader', methods.dragEnter);

So, now that a url and method can be passed in, we’ll handle the upload once the files have been dropped:

  drop : function( event ) {
    event.stopPropagation();
    event.preventDefault();

    var $this = $(this);
    var dataTransfer = event.originalEvent.dataTransfer;

    if (dataTransfer.files.length > 0) {
      $.each(dataTransfer.files, function ( i, file ) {
        var xhr    = new XMLHttpRequest();
        var upload = xhr.upload;

        xhr.open($this.data('method') || 'POST', $this.data('url'), true);
        xhr.setRequestHeader('X-Filename', file.fileName);

        xhr.send(file);
      });
    };

    return false;
  }

Next, make sure to edit index.html so that it specifies a PUT:

  $("#drop_target").dndUploader({
    url : "/",
    method : "PUT"
  });

Finally, we’ll add rudimentary handling of the file on the server. For now, this just prints the name of the file and its length. From here, reading the contents or writing to the filesystem isn’t much of a stretch, but for the time being, I’ll leave that as an exercise to the reader.

  get '/' do
    erb :"index.html"
  end

  put '/' do
    puts "uploaded #{env['HTTP_X_FILENAME']} - #{request.body.read.size} bytes"
  end

What’s Next?

There are some nice features we should probably add to this uploader, such as the ability to show/hide an overlay when the mouse hovers over the drop target, an upload progress bar, and feedback when the files have successfully (or unsuccessfully) uploaded. In the next post in this series, I’ll work on adding these features.

Firesheep: What It Is And Why You Should Be Scared

Recently, there has been a lot of discussion surrounding the Firefox add-on Firesheep. Firesheep is a tool that makes it easy for its user to assume (hijack) the identity of other users on a number of popular websites. I want to take a moment to explain not only how Firesheep works, but how when you are using the Onehub site, we protect you from the vulnerabilities that Firesheep exposes.

The Internet, and the protocol it is primarily built on, HTTP, are inherently stateless, meaning that by default no information is kept on either the client or server between requests. In other words, if you request the homepage from a website, then the product page, the website has no way of knowing that these two requests came from the same person, thereby making things like user logins and shopping carts impossible to track. In order to work around this limitation, and provide the seamless experience we have all come to expect, websites place a small piece of data on each user’s computer: a cookie. These cookies (specific to each website) are sent back to the website with every browser request, and they can use the information contained therein to recognize that all of your requests are coming from you, and you alone. In essence, once you are signed-in, you are your cookie. Cookies have a bad reputation amongst some users, however they are an indespensible piece of technology, and without them the modern Internet would not function.

An Example

Let us use a simple story. Charles would like to visit onehub.com to check the latest activity in his Workspaces.

Charles opens his web browser, Mozilla Firefox, and types onehub.com into the location bar. On his behalf, Firefox sends a request to the onehub.com servers for the homepage. In a few milliseconds the server replies with the HTML, CSS, JavaScript and images that make-up the onehub.com homepage. Firefox takes this data and renders onehub.com in Charles’ browser. So far so good. In order to see the latest activity, Charles will need to sign-in, and then visit the activity tab. So he clicks the “Sign In” button on the top-right of page and again, Firefox sends a request for a webpage to the onehub.com servers, but this time for the sign-in screen. This is where it gets interesting.

The sign-in screen is secured via SSL/TLS (indicated by the S in the HTTPS in Charles’ browser). This ensures that the onehub.com server really is who it claims to be, and encrypts all the data exchanged between Charles’ computer and the server. Confident that his email and password will not be exposed to the world, Charles fills out the form and presses enter. With the email and password in hand, the server does a few checks. It first looks in the database to see if a user with Charles’ email exists, if this is the case, it checks the password for that account against the password Charles typed in earlier. Provided both of these pieces of data match, the server replies back with the requested data, Charles’ home screen. In addition, the server provides a small piece of data in a cookie, a session identifier. This is a randomly generated string of characters, something like “a74822fdbce86b1541fec9c1f92d366f”. In addition to sending this identifier to Charles’ browser in a cookie, it is also stored by the onehub.com servers. In fact, if you are signed in to onehub.com right now you may look in your cookies for _onehub_session_id which will have your unique session identifier.

The session identifier functions as a stand-in for the user’s credentials, it saves Charles from having to provide this information on every single request.

Now that Charles is on the homepage, he clicks on the activity tab. Firefox again requests a specific webpage from the onehub.com servers, but this time it also provides the data in the cookie, the session identifier.

The server accepts this request, and looks up the session identifier in the database. The database provides two pieces of information to the server about this particular identifier. First, if this identifier was created recently enough (you may extend the length a session identifier is valid by choosing ‘Keep Me Signed In’), and second, that it belongs to Charles. The server can now reply with the data for Firefox to render Charles’ activity page.

This type of data exchange happens millions of times per-second across the Internet and it is this same technique that lets you do everything from sharing a photo on Flickr to keeping a shopping cart on Amazon.com.

So how does Firesheep work?

It is actually relatively simple. Most websites (onehub.com included) secure their sign-in forms with SSL/TLS. This means your email or username and password are encrypted when sent to the server. However, once a user is signed-in these sites do not encrypt the data exchanged between the browser and the server (onehub.com does). Firesheep watches all the traffic flying around on the network it is running on, like the coffeeshop which Charles is working from this morning. It listens specifically for the session identifiers for many popular websites including Facebook and Twitter. Firesheep then provides an easy way to change Charles’ session identifier to another one that Firesheep has found on the network. Charles is transparently able to assume (hijack) someone else’s identity.

How is Onehub secure?

We do two things differently than the majority of sites. First, we encrypt all traffic, all the time. This makes the session identifiers opaque to tools like Firesheep, provided the user visits the SSL/TLS secured webpage. Second, when providing the cookie to the browser, onehub.com marks the cookie as ‘Secure’. The implications of this second measure are subtle. Suppose Charles visits http://onehub.com (without the S). Without the cookie marked as ‘Secure’, Firefox would transmit the cookie to the onehub.com servers unencrypted. While onehub.com could redirect his browser to visit https://onehub.com (with the S), the damage is already done: Charles has broadcast his session identifer in an insecure fasion. Firesheep and other tools would be able to intercept it before the server can instruct the browser to try again via the secure connection.

Internet Security is a tough problem, it requires education of both end-users (Charles) and system administrators (Onehub) to ensure data is not leaked. The team here at Onehub works very hard to ensure your data is safe. We will continue to monitor new threats as they arrive, and adjust our systems and policies where appropriate.

The Most Awesome HTML5 Uploader Ever Built

HTML5, the next major revision of the HTML standard, has given us the ability to support file uploads via the web interface using drag and drop. You can now simply drag and drop files from your desktop to your Workspaces. The simplicity of this new feature makes file sharing and online collaboration a breeze.

Dragging Files

Right now, this feature is available in Chrome, Firefox, and newer versions of Safari. We have heard rumors that Internet Explorer 9 (which is currently in Beta) will support HTML5 including drag and drop.  However, if you want to try this out now, download one of the modern browsers I mentioned above.   We can’t wait to hear your thoughts on this cool feature.  We hope you are as excited as we are.  Tell us what you think by commenting on this post.

Happy dragging and dropping!