Introducing Folder Uploads in Google Chrome

The ability to upload entire folders from the browser is an incredibly common feature request. While we have supported folder uploads via FTP, and Onehub Sync for some time now, technical limitations have prevented us from handling folder uploads through the browser. Fortunately, that situation is starting to change.

Google Chrome recently added support for folder uploads, and we wasted no time adding it to Onehub! So, if you are using the most recent version of Chrome, you can now upload entire folders to your Workspaces – directly from your browser.

Folder Upload Menu

The change is virtually seamless, as Chrome users will simply be presented with two options when clicking “Upload”. All you have to do is click “Upload a Folder”, choose the folder you want to upload, and click “Select”. The entire folder structure and all files will be uploaded through the browser.

We hope you’re as excited about this as we are, and we will continue to activate this feature in other browsers as they begin supporting it.

Introducing Advanced Permissions

Onehub allows you to organize files into Workspaces and folders, and control access by inviting people to certain items. However, many of our customers have requested additional flexibility in the way that they share their sensitive data with other people.

Imagine that as the marketing director of Coffee House Co., I would like to solicit bids for a logo redesign. Thomas from Whiz Bang Studios and Alice from Acme Design Collective are both going to be submitting bids for this job, and they both need access to the same folder containing visual assets and style guides. They also each need their own folder where they can upload their proposals, works in progress, and receive additional documents not intended for the other designer.

Since I’m the Administrator, I need to be able to see everything in the Workspace, and it would also be great to see some indication when folders are hidden by default.

As the administrator of the Workspace, all folders are visible

Thomas and Alice, on the other hand, only see their respective folders.

Thomas's view is on top, Alice's is shown below.

Notice that both Thomas and Alice see a helpful icon to alert them to the fact that they have increased privileges in their folders.

In addition to being able to hide folders after they’ve been created, we also added a switch to hide them right when they’re created, so that Creators and below won’t see any activity pertaining to the folder, until they’re given explicit access.

We’ve worked hard on these new features, and hope that you find them useful. Click here to head to our support site for more detailed information, as well as examples of some common security scenarios.

Building an HTML5 Drag & Drop File Uploader Using Sinatra and jQuery: Part 1

Upon setting out to build a Drag & Drop uploader, I first trawled the web for prior art. Blog posts, examples, any information that would lead me down a well-trodden path. As it turns out, such a path doesn’t really exist yet.

FileReader: A Dead End

Any cursory research into HTML5 file handling will undoubtedly turn up mention of the FileReader class which has made an appearance in the most recent versions of Chrome and Firefox. FileReader is useful, in that it allows a browser to read data from files, either all at once, or in chunks, however it quickly becomes unworkable when you try to use it to read large files for uploading to a server. This is due to the fact that you need to read the entire file in one fell swoop to pass off to an XMLHttpRequest, and buffering this much data (this much being 50MB or more) will cause every browser I tested to become completely unresponsive.

XMLHttpRequest Level 2: The Missing Piece

Why haven’t more drag and drop multi-file uploaders shown up on the web before now? Certainly lack of support for gathering file information from drop events was one reason, but another was the fact that until recently, XMLHttpRequest didn’t provide any mechanism for sending file data to the server. Some clever hacks abound, such as using an IFrame or Flash, but none of these would really work for our purposes.

It just so happens that the XMLHttpRequest spec has recently been revised by the W3C: enter XMLHttpRequest Level 2. Here’s the description from the aforelinked working draft:

The XMLHttpRequest Level 2 specification enhances the XMLHttpRequest object with new features, such as cross-origin requests, progress events, and the handling of byte streams for both sending and receiving.

Wow, that sounds perfect! So in supported browsers, an XHR will have an upload attribute, which is an instance of XMLHttpRequestUpload. You can even get progress events from this object, allowing you to build progress tracking into your uploader. This appears to be supported in Firefox 3.5+, Safari 5+, and recent versions of Chrome stable (6.0.472.63 as of writing, but it’s probably supported in earlier versions).

There is one caveat to this approach: there’s currently no good way to perform your XHR file upload as a multipart form post. Experimental support for this has been added in recent builds of Firefox 4, using the FormData interface, however currently the only way to reliably submit a file in a multipart form using XHR in existing browsers is to use FileReader, read the entire thing into memory, then manually build a multipart post. So in other words, it’s a non-starter. Instead, a XMLHttpRequestUpload can be sent with a reference to a file, which the browser will stream from the filesystem as the body of the request. This may require some extra work on the server side, if you want to avoid buffering the entire file inside the process handling the request. For our part, we’ve made some modifications to the venerable nginx upload module, allowing it to also accept PUT requests with raw file contents in the body.

Drag & Drop: Just Enough to Get By

People with far more knowledge and tenacity than I have written eloquent sonnets declaring their undying love for the HTML5 drag & drop API. Unfortunately, it’s the only game in town if you want to do interesting things with files dropped on the browser window. For the remainder of this post, and in subsequent posts, I’ll be building an uploader from the ground up, adding features and complexity as I go. For now, the simplest thing that can possibly work: a drop target on the page, which listens for drop events and, if any files are dropped, creates XMLHttpRequests to send the files to a URL. Things get a lot more dicey when you want to make a complex element (like a table) into a drop target, highlight the drop area, or show information underneath the mouse, so we’ll go for progressive enhancement and add that stuff in later.

Where The Rubber Meets Brass Tacks

This example micro-app will use Sinatra, and have two endpoints, one that displays a page with a drop target, and one that accepts a file upload. Follow the code on github. We’re implementing the front-end code as a jQuery plugin, but most of the concepts can be adapted to Prototype or even plain Javascript.

First, we create a simple page with a drop target div on it. We also include a CSS file and links to both jQuery and our uploader Javascript file. Lastly, we attach an uploader to the drop target:

  <html>
    <head>
      <title>Drag &amp; Drop Tacos</title>
      <link rel="stylesheet" href="/css/master.css" type="text/css" media="screen" title="no title" charset="utf-8">
    </head>

    <body>
      <div id="drop_target">
      </div>
    </body>

    <script type="text/javascript" charset="utf-8" src="/javascripts/jquery-1.4.3.js"></script>
    <script type="text/javascript" charset="utf-8" src="/javascripts/jquery.dnduploader.js"></script>
    <script type="text/javascript" charset="utf-8">
      $("#drop_target").dndUploader({
        url : "/"
      });
    </script>
  </html>

The Ruby code to support this is super simple:

  require 'rubygems'
  require 'sinatra'
  require 'erb'

  get '/' do
    erb :"index.html"
  end

To run the app, make sure you have the Sinatra gem installed (gem install sinatra … depending on your system, you might have to sudo), drop into the example app directory, and type ruby app.rb. WEBrick should start up, and you should be able to access the application at localhost:4567.

Now, let’s take a look at the jQuery plugin. I’ll dispense with the boilerplate plugin code, but if you’d like a refresher, the jQuery documentation will get you up to speed. This is a first pass at the plugin– it won’t really do any uploading yet, but it’ll allow you to drag and drop files onto the drop target and prevent the browser from trying to open them:

(function( $ ){

  var methods = {
    init : function( options ) {

    return this.each(function(){

       var $this = $(this);

       $this.bind('dragenter.dndUploader', methods.dragEnter);
       $this.bind('dragover.dndUploader', methods.dragOver);
       $this.bind('drop.dndUploader', methods.drop);
     });
    },

    dragEnter : function ( event ) {
      event.stopPropagation();
      event.preventDefault();

      return false;
    },

    dragOver : function ( event ) {
      event.stopPropagation();
      event.preventDefault();

      return false;
    },

    drop : function( event ) {
      event.stopPropagation();
      event.preventDefault();

      return false;
    }
  };

  $.fn.dndUploader = function( method ) {
    if ( methods[method] ) {
      return methods[method].apply( this, Array.prototype.slice.call( arguments, 1 ));
    } else if ( typeof method === 'object' || ! method ) {
      return methods.init.apply( this, arguments );
    } else {
      $.error( 'Method ' +  method + ' does not exist on jQuery.dndUploader' );
    }
  };
})( jQuery );

The first order of business is to intercept three drag & drop events: dragenter, dragover, and drop. In order to define your own behavior for drag & drop, the default behavior and event bubbling must be cancelled in your event handler. This bears repeating- if you don’t cancel event propagation, the browser’s default drop handling behavior will take over, and your code won’t work at all. So, we write three identical functions to handle these events:

  dragEnter : function ( event ) {
    event.stopPropagation();
    event.preventDefault();

    return false;
  },

  dragOver : function ( event ) {
    event.stopPropagation();
    event.preventDefault();

    return false;
  },

  drop : function( event ) {
    event.stopPropagation();
    event.preventDefault();

    return false;
  },

Then, in our init function, we bind our event handlers to the appropriate events:

  init : function( options ) {

  return this.each(function() {

     var $this = $(this);

     $this.bind('dragenter', methods.dragEnter);
     $this.bind('dragover', methods.dragOver);
     $this.bind('drop', methods.drop);
   });
  }

So, with all of that, we should now have a drop target primed to accept items dragged onto it.

Getting information out of the MouseEvent

When files are dropped into our target, the event carries with it important information about what items were dropped. This information can be found in the dataTransfer property of the MouseEvent. The following modification will allow us to list the contents:

  drop : function( event ) {
    event.stopPropagation();
    event.preventDefault();

    console.log( event.originalEvent.dataTransfer.files );

    return false;
  }

If you’ve been following along, when you drop a couple of files onto the target, you should see something like this:

  FileList
    0: File
      fileName: "Scan 2.pdf"
      fileSize: 4999673
      name: "Scan 2.pdf"
      size: 4999673
      type: "application/pdf"
      webkitRelativePath: ""
      __proto__: File
    1: File
      fileName: "Scan.jpeg"
      fileSize: 943332
      name: "Scan.jpeg"
      size: 943332
      type: "image/jpeg"
      webkitRelativePath: ""
      __proto__: File
      length: 2
      __proto__: FileList

So, we can easily get the filename, size, and type from the objects in the dataTransfer. Next, we’ll loop through and upload each file. Since this only works with XMLHttpRequest Level 2, we won’t bother with interfaces that abstract away the XMLHttpRequest object- we’ll create one and manipulate it directly. First though, we should make sure to store the url, and any other options, on the uploader node:

  return this.each( function () {

    var $this = $(this);

    $.each(options, function( label, setting ) {
      $this.data(label, setting);
    });

    $this.bind('dragenter.dndUploader', methods.dragEnter);

So, now that a url and method can be passed in, we’ll handle the upload once the files have been dropped:

  drop : function( event ) {
    event.stopPropagation();
    event.preventDefault();

    var $this = $(this);
    var dataTransfer = event.originalEvent.dataTransfer;

    if (dataTransfer.files.length > 0) {
      $.each(dataTransfer.files, function ( i, file ) {
        var xhr    = new XMLHttpRequest();
        var upload = xhr.upload;

        xhr.open($this.data('method') || 'POST', $this.data('url'), true);
        xhr.setRequestHeader('X-Filename', file.fileName);

        xhr.send(file);
      });
    };

    return false;
  }

Next, make sure to edit index.html so that it specifies a PUT:

  $("#drop_target").dndUploader({
    url : "/",
    method : "PUT"
  });

Finally, we’ll add rudimentary handling of the file on the server. For now, this just prints the name of the file and its length. From here, reading the contents or writing to the filesystem isn’t much of a stretch, but for the time being, I’ll leave that as an exercise to the reader.

  get '/' do
    erb :"index.html"
  end

  put '/' do
    puts "uploaded #{env['HTTP_X_FILENAME']} - #{request.body.read.size} bytes"
  end

What’s Next?

There are some nice features we should probably add to this uploader, such as the ability to show/hide an overlay when the mouse hovers over the drop target, an upload progress bar, and feedback when the files have successfully (or unsuccessfully) uploaded. In the next post in this series, I’ll work on adding these features.

Decorating ActiveRecord Accessor Methods

Recently, I was faced with a quandary. I needed to impart some of our ActiveRecord accessor methods with special functionality, but I also needed to maintain the default behavior for those fields. This was done to support a new input type which we added to the venerable CalendarDateSelect plugin which allows a user to enter a date, time, and meridian, all in separate fields.

calendar_date_select_fields

Though I could parse each of the fields in the controller and pass the resulting time to the model, this is messy, error-prone, and unnecessary.

What I really want to be able to do is something like this:

>> CalendarEvent.new(
:start_time => {:date => '6/23/2009', :time => '12:30', :meridian => 'pm'})
=> #<CalendarEvent start_time: "2009-06-23 19:30:00">

My first thought was to simply override the accessor methods like so:

CalendarEvent < ActiveRecord::Base
...
def start_time=(new_time)
self[:start_time] = parse_time_from_hash(new_time)
end
end

However, as I alluded to before, there’s a problem with this approach. Rails already does some magic behind the scenes to ensure that time values stored to your ActiveRecord objects are automatically cast to and from UTC. The code above will completely overwrite these default accessors, and time zone casting will cease to function for those fields. What I really want to do is decorate these accessor methods, so that I can add to the existing functionality without completely circumventing it.

One way we might think about accomplishing this would be to redefine our accessor method and add behavior to it using alias_method_chain.

def start_time_with_hash_parsing=(new_time)
new_time = parse_time_from_hash(new_time)
start_time_without_hash_parsing = new_time
end
alias_method_chain :start_time=, :hash_parsing

Try that, though, and you’ll get this fun little error

`alias_method':NameError: undefined method `start_time=' for class `CalendarEvent'

It doesn’t work because there is no original start_time= method to alias; ActiveRecord creates those methods only when the class is loaded, because it has to read the columns from the database at runtime. So, how do you decorate methods which haven’t even been created yet? Well, it turns out you have to decorate ActiveRecord::Base.define_attribute_methods, which is responsible for creating accessor methods for each model.

There are a couple of ways you can go about doing this, depending on your goals. I wanted to be able to use these enhanced date/time inputs with any model in our application and trigger it from an initializer, so my approach uses alias_method_chain to modify ActiveRecord::Base directly.

module TimeAttributeExtensions
module ClassMethods

def define_attribute_methods_with_time_parsing
if define_attribute_methods_without_time_parsing

columns_hash.each do |name, column|
if [:datetime, :timestamp].include?(column.type)
unless method_defined?(:"#{name}_without_time_parsing=")

define_method("#{name}_with_time_parsing=") do |time|
send(:"#{name}_without_time_parsing=", parse_time_from_hash(time))
end

alias_method_chain :"#{name}=", :time_parsing
end
end
end

return true
end
end
end

module InstanceMethods
def parse_time_from_hash(time)
# Do time parsing here
end
end

def self.included(receiver)
receiver.send(:include, InstanceMethods)
receiver.extend ClassMethods

unless receiver.respond_to?(:define_attribute_methods_without_time_parsing)
class << receiver
alias_method_chain :define_attribute_methods, :time_parsing
end
end
end
end

Ok, so what exactly does that code do? First, we call the original version of define_attribute_methods. If that returns true, then we know that the attribute accessor methods have been generated. We loop over each column in the model, checking to see whether it stores either a datetime or a timestamp. If so, this is an accessor we’re interested in decorating. We do a quick check to make sure we haven’t already decorated the accessor (which can bite us in development mode), then we redefine the accessor with time parsing, and use the magic of alias_method_chain to alias our original method but still maintain a reference to it.

At the bottom, we use the Module#included callback to add the methods contained in our module, and we once again use alias_method_chain to redefine our version of define_attribute_methods.

Then, in the initializer, you would write something like

ActiveRecord::Base.send(:include, TimeAttributeExtensions)

At this point, any time-based field in any of your models will be able to accept a hash as a value, and properly parse it.

This works well, but one thing I don’t like about this solution is its rampant use of alias_method_chain. There has been some debate in the Rails community over its validity as a design choice, though there really are no other options if you need to modify all subclasses of ActiveRecord::Base (as you would to support a plugin like CalendarDateSelect).

Another Approach

If modifying every subclass isn’t a design requirement, you can clean up the code significantly:

module TimeAttributeExtensions
module ClassMethods

def define_attribute_methods
if super
columns_hash.each do |name, column|
if [:datetime, :timestamp].include?(column.type)
define_method("#{name}=") do |time|
super parse_time_from_hash(time)
end
end
end

return true
end
end
end

module InstanceMethods
def parse_time_from_hash(time)
# Do time parsing here
end
end

def self.included(receiver)
receiver.send(:include, InstanceMethods)
receiver.extend ClassMethods
end
end

Notice here we’re just defining define_attribute_methods and calling super when we want access to the previous implementation. The same goes for decorating the actual accessor methods.

The downside of this approach is that you have to explicitly include the module in your ActiveRecord models:

CalendarEvent < ActiveRecord::Base
include TimeAttributeExtensions
...

As with any decision, the approach you take will depend on your requirements, and tradeoffs you’re willing to make between explicitness and ease of use.