Monday, December 27, 2010

A Groovy way to track HttpClient Uploads

Recently, I had to develop a client-server application that leverage Commons HttpClient to perform file uploads.  With the upload process, I wanted to provide the user a nice progress bar that showed how much of the data (up to 2 GB) was processed.  There are two obvious approaches:
  1. Spawn a thread on the client that constantly checks the server to see how many bytes have been written on the server using something RESTful in nature
  2. Register a call-back mechanism that allows tracking the upload progress
 I chose the latter, and leveraging Groovy I/O stuff makes it a snap:

1) Create an implementation of Observable
/**
 * A utility that contains all of the client functionality needed to perform the upload tasks.  
 * This class is observable on the upload progress.
 *
 * @author Brock Heinz
 */
class ClientUtility extends Observable {

  /**
   * Wrapper around observable
   *
   * @param totalBytesWritten - number of bytes written to a stream to the server
   */
  void fireUploadProgress(int totalBytesWritten) {
    setChanged()
    notifyObservers(totalBytesWritten)
  }

  def doUpload(File f) {
    //we'll come back to this later in post
  }
}

2) Create an implementation of FilePart that overrides the sendData() method
/**
 * A wrapper around the commons httpclient FilePart object.  This class overrides the 
 * streaming portion so that we can track upload progress as we write the file to the 
 * server stream
 *
 * @author Brock Heinz
 */
class ProgressFilePart extends org.apache.commons.httpclient.methods.multipart.FilePart {
  private ClientUtility observable
  private final int uploadBufferSize

  /**
   * @param name the name of the part
   * @param file the file to upload
   * @param observable class that wants to be notified of change during upload
   * @param uploadBufferSize optional parameter that sets the number of bytes read before writing
   */
  ProgressFilePart(String name, File file, ClientUtility observable, int uploadBufferSize = 16384) 
    throws FileNotFoundException {
    super(name, file)
    this.observable = observable
    this.uploadBufferSize = uploadBufferSize
  }

  /**
   * Proxy the method that streams data from the file to the underlying socket stream to server.  
   * This method fires a change on each buffered read to the class' observer.
   *
   * @param os the stream to the server
   */
  protected void sendData(OutputStream os) {
    if (lengthOfData() == 0) return
    int totalread = 0
    getSource().createInputStream().withStream {InputStream is ->
      is.eachByte(uploadBufferSize) {byte[] buffer, int read ->
        os.write(buffer, 0, read)
        totalread += read
        this.observable.fireUploadProgress(totalread)
      }
    }
  }
}

3) Register an Observer, and handle observed callbacks
class UploadDataPanel implements Observer {
  UploadDataPanel(def clientUtility) {
    clientUtility.addObserver(this)
  }
  void update(Observable observable, Object o) {
    //update the UI's progress bar
    swingBuilder.doLater { swingBuilder.uploadProgressBar.setValue(o) }
  }
}

4) Put it all together
  //leverage the progress part
  def doUpload(File f, String postUrl) {
    def httpClient = new HttpClient()
    def post = new PostMethod(postUrl)
    def partArray = [new ProgressFilePart("myfile.zip", f, this)] as Part[]
    post.setRequestEntity(new MultipartRequestEntity(partArray, post.getParams()))
    try {
      httpClient.executeMethod(post)
    } finally {
      post.releaseConnection()
    }
  }

1 comments:

Federico said...

Very nice and useful post! thank you!