thoughts.each { println it }

A Groovy Way to Write SAS XPT

2011-07-26T11:11:00.006-05:00

Recently our development team was posed with somewhat of a challenge. Besides having various requests from our sponsors, our data management group had a desire to obtain SAS XPT clinical extracts from our EDC system without requiring SAS tools (and expensive SAS licenses) to do so.

First off, why SAS XPT? SAS XPT is the format that the FDA requires for New Drug Applications as part of the drug development process. It's the least common denominator for representing and transferring clinical data. And, because of US law (and the fact that it's just the right thing to do), the FDA must remain vendor neutral. As part of that neutrality, SAS XPT must be an open format. From the SAS FDA FAQ:

Q. What do you mean, the XPORT format is "open?"
A. Specifications for the XPORT transport format are in the public domain. Data can be translated to and from the XPORT transport format to other commonly used formats without the use of programs from SAS Institute or any specific vendor

The specification can be found here. My initial thinking was, there's gotta be some sort of open source implementation with this stuff; and apparently I wasn't the only one thinking that. I had seen some things with R where one could read SAS XPT, but nothing to write it. Why not? I'm guessing there isn't much out there because it's hard to do, and once people crack it, they don't want to share :)

In my opinion, the hardest part of the specification is the fact that, "All integers are stored using IBM-style integer format, and all floating-point numbers are stored using the IBM-style double." Really, instead of using the common IEEE format, the FDA mandates something as archaic and ancient as IBM Floating Point? Fortunately, one of my developers is good with C programming, and I had just enough experience on the mainframe to come up with a solution.

So yeah... our reporting engine runs on a JVM, and Groovy was a natural fit to develop this with. Why Groovy instead of just Java?

Groovy I/O - leveraging the dynamic left shift operator is just a natural way to write to an java.io.OutputStream, and build the XPT file. Furthermore, with Groovy's management of Exceptions, I don't have to constantly handle java.io.IOException.
Duck Typing - as we write out each data point, if the object acts like a duck, quacks like a duck... it's a duck. With SAS XPT, you define a type for each column. And since Groovy has a dynamic toDouble() method on java.lang.String and all implementations of java.lang.Number, the code doesn't care about the underlying object type passed in... it's just converted to a double.
Because Groovy is still Java - even using the Groovy programming language, we can still do the bit banging necessary to convert IEEE to IBM's Floating Point.
Incredibly useful dynamic methods for implementing some of the subtleties of the specification. A few gems are java.lang.String's pad methods, and java.util.Date's getAt method.

Obviously all of this is possible in just Java (or other programming languages, for that matter), but it's just better using Groovy :)

World, meet iQ. iQ, meet world.

2011-06-23T16:38:00.002-05:00

This week in Chicago we formally introduced the world to the iQ - a truly disruptive technology that will revolutionize the way drugs are developed and tested for cardiac safety.

I've talked about this project in the past from a technical perspective, but for non-geeks: at a high level this thing makes it dead simple for pharmaceutical companies to ensure that ALL of their later-phase drug trials have board certified cardiologists looking things over.

Here's a picture from the DIA trade show. Over three days, we demonstrated nearly 60 ECG acquisitions. In each case we were able to correctly identify Andrew (the poor guy with no shirt) from our subject database using voice biometrics 100% of the time.

Best. Groovy. Method. Ever.

2011-06-23T16:09:00.001-05:00

Well, maybe not the best method ever... but it's gotta be damn close :) In just a few lines of code I'm doing a lot of really Groovy stuff!

I was looking for a quick and dirty way of converting JSON into some simple groovy objects that I have. The method recursively walks the object hierarchy and then populates corresponding values that are present in the JSON. Enjoy!

  /**
   * Slick way of using reflection to populate value 
   * objects with JSON.  Performs a DEEP copy in order 
   * to populate all of the dependent pogo properties
   *
   * @param obj object to populate - pogo
   * @param json grails.converters.deep.JSON
   * @return the obj param
   */
  protected def populateFromJson(def obj, def json) {
    obj.metaClass.properties.each {metaBeanProperty ->
      if (metaBeanProperty.modifiers == Modifier.PUBLIC) {
        def val = null
        String jsonVal = json[metaBeanProperty.name]
        if (metaBeanProperty.type == String) {
          val = jsonVal
        } else if (Number.isAssignableFrom(metaBeanProperty.type)) {
          if (jsonVal?.isNumber()) val = jsonVal.invokeMethod("to${metaBeanProperty.type.simpleName}", null)
        } else if (metaBeanProperty.type == Date) {
          if (jsonVal) val = Date.parse("yyyy-MM-dd'T'HH:mm:ss'Z'", jsonVal)
        } else if (metaBeanProperty.type == Boolean) {
          if (jsonVal) val = jsonVal.toBoolean()
        } else if (metaBeanProperty.type.name.startsWith("com.mycomp")) { 
          val = populateFromJson(metaBeanProperty.type.newInstance(), json[metaBeanProperty.name])
        }
        obj[metaBeanProperty.name] = val
      }
    }
    return obj
  }

Inter JVM Communication

2011-04-15T11:09:00.003-05:00

Ever have an application that should only have one instance of it running? Well, I do... I have a Swing application, and it integrates with a device via USB. I want to ensure that there's only one player in the game at a given time.

How can one Java process know if another with the same application is running? There seems to be a lot of suggestions out there: RMI, Cajo, Terracotta, etc, but since I am using Groovy, I figured there'd be an easier way. Turns out (for my use case), there is. Some of the dynamic I/O methods added to java.net.Socket and java.net.ServerSocket make this a snap.

My basic strategy goes something like this:

Implement a single method to determine if the process can run. If it can, fire up a server process, and then store and encrypt the server's port number in a File and return true. If it can't, return false.
On invocation, if there is no file, or an invalid port number (non-Integer), it's safe to say that no other Java process for the given application is running.
If there is a valid port, create a socket connection and issue a simple 'ping' following an incredibly simple protocol. If the server process is not available, or if that process doesn't respond properly to the ping, it's safe to say the application in question is not running.
If, however, the ping returns as expected, we know that this application is running already, and we want to return false and then shut down.
If it's determined that no other process is running, start up a server socket in another thread, encrypt the port created, and store it in the port file.

The obvious weakness in this strategy is that a user could potentially modify the encrypted file, but for my use case, we can live with that risk.

Also, this solution is certainly doable with just Java, but it's just a lot damn easier with some of the slick dynamic Groovy I/O methods. Here's the code:

  /**
   * By examining the file, determine if a a socket thread is holding on to the specified port.  If we can 
   * communicate on that port, it's safe to say that another instance of this app is running, and this 
   * method returns false
   *
   * @param portFile file that contains encrypted port.  If null, or invalid, this method will assume 
   *        we can run
   * @return true if this app is launchable
   */
  static boolean isLaunchable(File portFile) {
    boolean ret = true
    if (portFile.exists()) {
      Socket client = null
      try {
        String decrypted = new String(MyUtils.decryptAndDecode(portFile.text))
        if (decrypted.isInteger()) {
          String ping = "ping"
          client = new Socket((String) null, decrypted.toInteger())
          client.setSoTimeout(1000)
          client << ping //send ping to the server
          client.inputStream.withStream {InputStream sis ->
            sis.eachByte(256) {byte[] buff, int read ->
              ret = (ping != new String(buff, 0, read)) //response same? we are not launchable!
            }
          }
        }
      } catch (Exception e) {
        log.debug("Couldn't determine server running. ${e}")
      } finally {
        if (client) client.close()
      }
    }

    if (ret) { //we're launchable, now grab hold of a server connection and encrypt port to file
      Thread.startDaemon("isLaunchableServerSocket") {
        ServerSocket serverSocket = new ServerSocket()
        serverSocket.bind(null) //creates a dynamic port
        portFile.text = MyUtils.encryptAndEncode(serverSocket.localPort.toString().getBytes())
        while (!serverSocket.isClosed()) {
          Socket socketConn = null
          try {
            socketConn = serverSocket.accept() 
            socketConn.inputStream.withStream {InputStream sis ->
              sis.eachByte(256) {byte[] buff, int read ->
                String msg = new String(buff, 0, read)
                if ("close" != msg) {
                  socketConn.outputStream << msg //just ping it right back
                } else {
                  serverSocket.close()
                }
              }
            }
          } catch (java.net.SocketException e) {
            log.debug("Shutting down local socket.")
            portFile.text = "" //wipe out the text of the port file
          } finally {
            if (socketConn && !socketConn.isClosed()) socketConn.close()
          }
        }
      }
    }
    return ret
  }

Exciting News - Groovy / Grails Project Getting Some Press

2011-03-21T09:37:00.001-05:00

As I've mentioned in previous blog posts, the company I work for has been working quite diligently on developing a device and software system that will be used in clinical trials to collect and process ECG data. The ultimate goal is to help ensure safety of drugs during the development process; we believe that the device and process we've created will provide much higher quality data in a much shorter amount of time. At its core, the device records 5 minutes of ECG data, and 30 seconds of voice data. The voice data is used to perform biometric matches to ensure demographic data integrity, and to prevent fraud. Data transmission is done via HTTPS, and is processed asynchronously on the server. Once processed on the server, the data is available in real-time for review by selected cardiologists. Upon performing the review, the analyzed data is available via web interface for real time review by study sponsors and regulatory authorities.

Speaking of regulatory authorities, we've recently received 510K approval from the FDA that we can move forward with this project!

With all of the marketing / general news, one thing that probably won't be communicated is the geek stuff happening behind the scenes. This project heavily leverages Groovy / Grails throughout the entire system work-flow. Simply put, the following tools have greatly contributed to the success of this project:

Groovy Swing Builder for the richer UI portions
Groovy Scriptom for integration with Windows COM dependencies (low level ECG processing with software developed by our Italian friends)
Grails for the server side stuff, heavily leveraging the Quartz plugin for asynchronous processing

A Groovy way to track HttpClient Uploads

2010-12-27T11:45:00.011-06:00

Recently, I had to develop a client-server application that leverage Commons HttpClient to perform file uploads. With the upload process, I wanted to provide the user a nice progress bar that showed how much of the data (up to 2 GB) was processed. There are two obvious approaches:

Spawn a thread on the client that constantly checks the server to see how many bytes have been written on the server using something RESTful in nature
Register a call-back mechanism that allows tracking the upload progress

I chose the latter, and leveraging Groovy I/O stuff makes it a snap:

1) Create an implementation of Observable

/**
 * A utility that contains all of the client functionality needed to perform the upload tasks.  
 * This class is observable on the upload progress.
 *
 * @author Brock Heinz
 */
class ClientUtility extends Observable {

  /**
   * Wrapper around observable
   *
   * @param totalBytesWritten - number of bytes written to a stream to the server
   */
  void fireUploadProgress(int totalBytesWritten) {
    setChanged()
    notifyObservers(totalBytesWritten)
  }

  def doUpload(File f) {
    //we'll come back to this later in post
  }
}

2) Create an implementation of FilePart that overrides the sendData() method

/**
 * A wrapper around the commons httpclient FilePart object.  This class overrides the 
 * streaming portion so that we can track upload progress as we write the file to the 
 * server stream
 *
 * @author Brock Heinz
 */
class ProgressFilePart extends org.apache.commons.httpclient.methods.multipart.FilePart {
  private ClientUtility observable
  private final int uploadBufferSize

  /**
   * @param name the name of the part
   * @param file the file to upload
   * @param observable class that wants to be notified of change during upload
   * @param uploadBufferSize optional parameter that sets the number of bytes read before writing
   */
  ProgressFilePart(String name, File file, ClientUtility observable, int uploadBufferSize = 16384) 
    throws FileNotFoundException {
    super(name, file)
    this.observable = observable
    this.uploadBufferSize = uploadBufferSize
  }

  /**
   * Proxy the method that streams data from the file to the underlying socket stream to server.  
   * This method fires a change on each buffered read to the class' observer.
   *
   * @param os the stream to the server
   */
  protected void sendData(OutputStream os) {
    if (lengthOfData() == 0) return
    int totalread = 0
    getSource().createInputStream().withStream {InputStream is ->
      is.eachByte(uploadBufferSize) {byte[] buffer, int read ->
        os.write(buffer, 0, read)
        totalread += read
        this.observable.fireUploadProgress(totalread)
      }
    }
  }
}

3) Register an Observer, and handle observed callbacks

class UploadDataPanel implements Observer {
  UploadDataPanel(def clientUtility) {
    clientUtility.addObserver(this)
  }
  void update(Observable observable, Object o) {
    //update the UI's progress bar
    swingBuilder.doLater { swingBuilder.uploadProgressBar.setValue(o) }
  }
}

4) Put it all together

  //leverage the progress part
  def doUpload(File f, String postUrl) {
    def httpClient = new HttpClient()
    def post = new PostMethod(postUrl)
    def partArray = [new ProgressFilePart("myfile.zip", f, this)] as Part[]
    post.setRequestEntity(new MultipartRequestEntity(partArray, post.getParams()))
    try {
      httpClient.executeMethod(post)
    } finally {
      post.releaseConnection()
    }
  }

Java WebStart 'offline-allowed'

2010-12-17T14:31:00.012-06:00

Wow... been going crazy on this one for a few days. I have an application that is being delivered via JNLP to run under WebStart. As a requirement, the application has to run online or offline. When offline, the data that is to later be transmitted will sit in a local 'queue'. Once the system comes back online, all of the data in the queue will be uploaded.

I just, for the life of me, could not get this to work! I checked, double-checked, and tripled-checked the format of my JNLP code. Nothing seemed to work. Finally, I decided to look at the JNLP returned from my server (Grails controller generated), and then I saw it! Cache control headers were being set, telling clients NOT to cache. It turns out the Java process that downloads the JNLP (javaws) respects that cache setting! The problem wasn't with the JNLP, but rather the HTTP headers. Problem solved :)

Here's the trivial script I wrote using Groovy and Commons HttpClient to snoop the headers:

1:  import org.apache.commons.httpclient.HttpClient  
2:  import org.apache.commons.httpclient.methods.GetMethod  
3:  def client = new HttpClient()  
4:  def get = new GetMethod("http://localhost:8080/myapp/jnlp/test")  
5:  try {  
6:   println client.executeMethod(get)  
7:   println get.responseBodyAsStream.text  
8:   get.responseHeaders.each { println it }  
9:  } finally {  
10:   get.releaseConnection()  
11: }

... one final thought. If you're generating your JNLP dynamically, make sure you set a Last-Modified response header. WebStart seems to use that then when determining if the JNLP file has changed, and should thus check for JAR updates. My suggestion, use the most recently updated JAR that is specified in your JNLP as the value for the Last-Modified header:

def jarFile = findFromExpandedWar(servletContext)
def gmt = TimeZone.getTimeZone("GMT")
def modified = new Date(jarFile.lastModified())
def pattern = "EEE, dd MMM yyyy HH:mm:ss zzz"
response.setHeader("Last-Modified", DateFormatUtils.format(modified, pattern, gmt))

You can find the client JAR files by using ServletContext's getRealPath() method.

Data Driven Award

2010-10-14T15:18:00.011-05:00

More exciting news! Spaulding Clinical was recently named one of five finalists for the SCDM Data Driven awards. SCDM is the Society for Clinical Data Management, and Data Driven is their annual global award, and this year's entrants will be judged on Monday October 18th, 2010 in Minneapolis, MN. The award is described as:

... honors the year’s best advancement in clinical data management, and recognizes innovation that improves technology or processes within clinical research studies.

and the Annual Conference co-chair Charlene Dark says,

"Our finalists have addressed many areas of clinical data management and how we can contribute to the speed and accuracy of getting products to market. The entrants are expanding the reach of clinical data management to improve collaboration among cross-functional groups within clinical research. We are encouraged to see the importance of clinical data management demonstrated through these value-added concepts and strategies."

The application we're presenting on is named SCi ECG (pronounced Sky ECG :) So what is SCi ECG, and why do other people think it's important?

From a geeky tech perspective: SCi ECG bridges the gap between an electronic data capture system (EDC) and an ECG management system. Transcription of ECG data is completely eliminated. SCi ECG auto-imports subject demographic data from the EDC system into the ECG system. In addition to auto-importing the demographics, time-point programming is also done automatically. In clinical trials, the timing of events relative to dosing is very very important, and the timings must be precise. Assuming that a study schedule has been built in the EDC system, that schedule can be automagically propagated into the ECG system. Finally, SCi ECG handles data transcription seamlessly. As ECGs are processed, the results of the ECG are automatically transcribed into the EDC system. In real time :)

From a practical perspective, SCi ECG empowers Phase 1 organizations to also act as ECG core labs. Traditionally, this type of data handling is entirely separate from the Phase 1 clinical conduct. So who cares? Pharmaceutical sponsors and investigators do! With a highly integrated system, sponsors, investigators, and cardiologists have access to critical ECG data as it is being collected. An integrated model leads to higher quality data in a fraction of the time!

(it's a good thing we're not being judged on our architectural diagramming skills!)

Next Monday, we'll be judged on the following:

Does the project solution significantly impact the quality and/or efficiency of the clinical data manager’s work?
Does the project represent an innovative application of a process and/or technology?
Does the project demonstrate an active collaboration between a service or technology provider and a study sponsor?

Spaulding Clinical Becomes World’s Largest Telemetry Provider

2010-08-24T07:31:00.005-05:00

Exciting times! Today it has been announced, that the company I work for, has become the largest provider of of telemetry monitoring for a Phase 1 research unit.

Why are ECGs in clinical trials important? What are the other guys doing to collect ECGs? What is telemetry, and why does the fact that we're the world's largest provider of it make us special?

First, as mentioned before, an ECG is a measurement of heart activity. From Wikipedia:

.... interpretation of the electrical activity of the heart over time captured and externally recorded by skin electrodes.

Collecting ECGs in clinical trials is important for two reasons: a) immediate safety of subjects during the trial and b) it measures cardiac changes as a result of receiving a drug; this change could impact long-term safety. The change that everyone looks out for is what is called QT prolongation. QT prolongation can lead to some nasty things if not properly addressed. You may remember a few drugs being pulled from the market over the years for this exact cause. Don't worry, the FDA's got your back... From the FDA (E 14 Guidelines),

"Drugs are expected to receive a clinical electrocardiographic evaluation, beginning early in clinical development, typically including a single trial dedicated to evaluating their effect on cardiac repolarization (“thorough QT/QTc study”). "

Ok, if it's important to collect ECGs in clinical trials because it helps keep people safe, what are other research units using to collect ECGs? Typically, there are two methods of ECG acquisition employed: electrocardiographs (I mentioned what that is in a previous blog post) and Holter recorders. While Holter recorders definitely have an advantage over electrocardiographs, both are less than optimal when it comes to drug research. With traditional electrocardiographs, clinical personnel roll up to a subject with a giant machine on wheels, they have to enter in a bunch of data using tiny keys, they then have to hook the subject up to electrodes, and finally they click 'print' to get a paper version of ten seconds of ECG data. Lots of opportunities for error, lots of action in getting the subject ready. With all of that commotion, there are definitely going to be effects on the heart. With Holter recorders, you have to program the device to specify who you're collecting data from, you start recording, and then you forget about it for a while. You hook subjects up once, and you can collect A LOT of ECG data. When the collection period is over, the ECG data corresponding to the proper time-points is extracted from flash memory cards (just like in your digital cameras). A couple of problems here though:

If there is some sort of impedance to collecting the data, you're not going to know it until the data is analyzed.
Cardiologists aren't going to be able to review the data in real time - if there are immediate safety issues, it's going to take a bit to have that known.
Extracting the proper time-points one by one is cumbersome and introduces opportunities for error. In a recent study, we had 256 subjects where approximately10 ECGs were collected each day over 8 days: yeah, that's over 20,000 ECGs!

Ok, old_school_cardiographs = bad, Holters = (not as) bad. What about telemetry, what makes telemetry good? Telemetry (according to Wikipedia), " ... is a technology that allows remote measurement and reporting of information". Think of the TV show ER: in the background of most hospital scenes, you get to see some squiggly green lines, and hear some beeping. While although, I don't know for sure if that is telemetry, that's the same idea; you're seeing cardiac data as it is being collected. The big bonus with telemetry is that people get to see how things are working in real time! You can collect the same amount (and more) data as Holters, but all necessary parties know what is happening. That obviously leads to higher quality.

Here's the software part: by integrating with the telemetry system, Spaulding Clinical has developed a system that really keeps the focus on the subjects in the trial. By leveraging an interface we've named SCi ECG (I indirectly blogged about this a few years back), clinical personnel focus on making sure subjects lay still at the specified times when an ECG must be acquired. With clinical trials, timing is important. Regulatory bodies mandate that ECGs are taken at very precise times... typically relative to a dosing time. With this interface, personnel don't have to do anything other than ensure that the subjects are all very still, the automation does the rest.

Spaulding Clinical's automated and integrated telemetry model means no transcription, no rushing around with giant carts, no blindly hoping the Holter recorders are still recording, no big variables. And donchaknow, less variability is better science. Better science leads to safer drugs!

What I've Been up to

2010-08-20T12:05:00.012-05:00

So here's that proverbial wow-it's-been-a-long-time-since-my-last-post post :)

The last year or so has been quite exciting both personally and professionally. Since last posting, I've been blessed with a beautiful little boy, and I've had the opportunity to do some traveling throughout the US.

Professionally, things have been going quite well too. The company I work for, Spaulding Clinical, is really making a few splashes in the pharmaceutical world. I think we've finally outgrown the 'startup' moniker, and we're quickly becoming more established as a force to be reckoned with. Interestingly enough, this isn't just our own observation. Earlier this year, we achieved Technical Partner status with Medidata solutions. This partnership grew out of an interesting integration project where we implemented a secure interface that bridged data we collect using our EDC environment with Medidata's Rave Web Service's API. From a geeky perspective, we used the Grails Quartz plugin to poll our environment for data changes on a configurable cycle. Once changes are detected, we leverage a lot of the Groovy XML magic, and combined that with Commons HTTP Client to execute the web services calls. Using GORM, we were able to persist life-cycle events, and provide some decent metrics on timing, etc. While this implementation isn't exactly real time, it's damn near close.

The technology relationship with Medidata has also lead to a number of interesting speaking engagements as well. In June, I presented at the Drug Information Association's 46th annual meeting in Washing D.C. The topic was, Electronic Data Capture in Phase 1: Do the Pros Outweigh the Cons? In September, I will be presenting in a couple of webinars on a similar topic. In October, I'll be in Cologne, Germany talking to a user group about some nuts and bolts of the integration.

The most exciting thing I've been working on was also debuted at the same meeting in DC. This project is truly going to be considered a disruptive technology in the ECG core lab world. First a bit about what a core lab is, then a bit about some of the problems they face, and finally our solutions to those problems.

An ECG core lab is an entity that supports clinical trials by offering ECG (ECG and EKG can be used interchangeably) domain expertise to pharmaceutical companies performing drug trials. With later phase studies, pharmaceutical companies contract with 'sites' (think clinics, hospitals, etc) all over the world. These sites offer their patients the option to participate in an experimental drug trial. The sites dispense the drug, collect safety information, etc. But, one of the big tasks required of them is to collect an ECG per subject visit: typically just a ten second sample of what is going on in the subject's heart. This is where core labs come in...

ECG core labs support sites by shipping ECG acquisition hardware, providing phone support, and data services to these sites (lots of trials are still done just by faxing ECG printouts too!). On top of all of that, ECG core labs must provide a means for allowing cardiologists to assess and measure individual ECGs. Once ECGs are processed accordingly, the data is prepared for reporting to pharmaceutical companies conducting the clinical trial, and to the proper regulatory authorities (think FDA). So, that's what core labs do.... but what are the problems?

The problems that ECG core labs face are things that most software people are familiar with, and are things that developers are constantly looking to eliminate: unnecessary transcription, multiple sources of identical data, manual processes that can be automated, etc. Aside from the finer grained problems, the biggest issue is that the sites participating in a clinical trial don't necessarily have their primary focus on collecting ECGs in these clinical trials. Their focus is 1) on the health and well-being of their patients 2) Conducting the overall trial and finally 3) Getting the ECG stuff done. With the current state of ECG acquisition in later phase trials, things are kinda hard. Sites must know how to operate kludgy equipment to acquire ECGs, they have to do data entry by duplicating subject demographics on each visit, and they have to know how to transmit the data to the core lab... most of the time via fax.

So how did we build a better mouse trap? Well, we've done a few things... First we designed our own hardware device. The device we've developed is smaller than your average smart phone; tiny compared to traditional electrocardiographs (acquisition machines). Next, we think that more is better; our device can capture five minutes of ECG data. While although the industry may not be ready for more than 10 seconds, we know they eventually will be. But, in the mean time, we've prepared for that by teaming up with AMPS, and we can leverage their validated Antares algorithm to search the 300 seconds for the best 10. Then, we've added some voice biometrics into our work-flow. The device we've created can also record up to 30 seconds of audio. Using another validated algorithm, we can get rid of all of the messiness of duplicating subject demographics; ECGs are matched to a subject programmatically simply by using the distinct characteristics in the human voice. Less key presses means less errors! And finally, we've made the transmission dead simple - it's just a matter of hooking the device up to a PC, and letting the application do the rest! For all you geeks out there, we're using Java Web Start to download a very thin client to a site's desktop. That client is written primarily in Groovy, heavily leveraging the Groovy Swing Builder, and the server side stuff is all Grails.

Here's the teaser we released at the DIA:

Hudson and Grails (update)

2010-08-20T11:58:00.002-05:00

In a previous post I had outlined a mechanism on how one could hook into Grails events to tag a Hudson build numbers into a GSP layout. Well, that was done using Grails 1.0.3. Below is the implementation that I use when building with Grails 1.3.2.

In a comment box, I've been informed that this may not work with Grails 1.3.4. The guts of the script I have outlined will probably still work, but you may need to hook into a different event.

eventCompilegspStart = {arg1 ->
//move up the original layouts
mystaging = new File((File) grailsSettings.projectTargetDir, "mystaging")
String layoutPath = "${grailsSettings.baseDir}/grails-app/views/layouts"
ant.mkdir(dir: mystaging)
ant.move(todir: mystaging) {
ant.fileset(dir: layoutPath, includes: "*.gsp")
}

//transform them in
def hudsonBuild = ant.project.properties["environment.BUILD_NUMBER"] ?: "Custom Build"
ant.copy(todir: layoutPath, overwrite: true) {
 ant.fileset(dir: mystaging, includes: "*.gsp")
 ant.filterset() {
   ant.filter(token: "HUDSON_BUILD", value: hudsonBuild)
 }
}
}

eventCompilegspEnd = {arg1 ->
//GSP compiliation is done, move original layouts back
mystaging = new File((File) grailsSettings.projectTargetDir, "mystaging")
String layoutPath = "${grailsSettings.baseDir}/grails-app/views/layouts"
ant.move(todir: layoutPath) {
 ant.fileset(dir: mystaging, includes: "*.gsp")
}
ant.delete(dir: mystaging)
}

Grails and the Wild Wild West

2008-12-04T16:00:00.007-06:00

Every so often while doing some Grails development, I feel like I'm in the 'Old West.' Just as in the days of our great-great-grandfathers things can get really exciting when embarking on an adventure without a map.

After recently coming across a closed JIRA, I wanted to explore how I could leverage Grails' paging mechanism with complex criteria queries. In particular, I wanted the collection returned by my query to tell me how many results I'd have to page over. Enter PagedResultList.

How and why is this geekily adventuresome? Good question, thanks for asking.

First off, there is no map. Second, leveraging the PagedResultList query result doesn't quite follow the pattern of dynamic persistence domain methods that I've seen and become accustomed to.

I'll spare you the details of the entire adventure, but I will share what I got burned on so that it might help someone else out there.

In order to obtain a PagedResultList, you have to first create a criteria object (whis is of type HibernateCriteriaBuilder) from a domain, and then call a dynamic 'list' method with two arguments. After lots of metaClass inspection and debugging, I finally found that the dynamism can actually be found in the 'invokeMethod' method of the criteria builder, and not on the metaClass. From there the implementation is insanity and not for the faint of heart... I'll stay away from the details :)

This won't work:

MyDomain.createCriteria().list(offset: 0) {
  eq("parent", parent)
  order("dateCreated")
}

This will:

MyDomain.createCriteria().list(offset: 0, sort: "dateCreated") {
  eq("parent", parent)
}

The difference? The sorting is defined in the map that is passed to the 'list()' method, and not in the body of the closure.

While I'm sort of complaining about a lack of documentation (yes I realize Grails is open source, yes I realize I didn't pay for it, and yes I realize I can contribute code and documentation to the project :), my real point is to illustrate just how darn hard it will be to come up with an automated documentation mechanism (like Javadoc) for a dynamic language. Not only would an automated tool have to be aware of all of the dynamic methods added to the MetaClass, but it would somehow also have to take into consideration classes that override 'invokeMethod.'

How did the Rails folks solve this one?

Hudson and Grails

2008-11-17T16:17:00.008-06:00

I debated on whether or not to blog about my very positive experience with the Hudson CI server. I think the buzz pretty much speaks for itself, so I'll stick to how I implemented build number tagging with my Grails build.

My goal for my Hudson builds was to 'tag' the Hudson build number directly into the footer of my Grails GSP layouts. Since the applications that I'm applying this strategy to are internal facing, it suits us really well to be able to see the build number directly in the user interface.

Here's how I went about accomplishing it:

1) I added a filterset token in my layout ($PROJECT_HOME/grails-app/views/layouts/main.gsp):


<html>
 <body>
  ...
  <div class="footer">Build Version: @HUDSON_BUILD@</div>
 </body>
</html>

2) I hooked into Grails' WAR target by defining a closure in $PROJECT_HOME/scripts/Events.groovy

eventWarStart = {
  def hudsonBuild = Ant.antProject.properties."env.BUILD_NUMBER" ?: "Custom Build"
  String relLayoutPath = "grails-app/views/layouts"
  Ant.copy(todir: "$stagingDir/WEB-INF/$relLayoutPath", overwrite: true) {
    fileset(dir: "${basedir}/$relLayoutPath", includes: "*.gsp")
    filterset() {
      filter(token: "HUDSON_BUILD", value: hudsonBuild)
    }
  }
}

The reason this works is because Hudson exposes a number of environment variables at build time. As you can see from the script above, I use the 'BUILD_NUMBER' variable as the token's replacement value.

Kudos to Jeff Brown and the Grails folks for releasing the Hudson plugin which enables Hudson builds using Gant.

Unfavorable Grails Review Response

2008-10-07T08:48:00.010-05:00

After being away from the Grails mail lists for the past few weeks, I came across a thread that discusses an unfavorable review of Grails on ohloh.net.

The reviewer has five qualms with the framework... which probably leads to the 1/5 rating. Over the past few months I've deployed four Grails applications to production. I'd like to use my experience to discuss the user's qualms.

"1. The biggest problem is definitely the lack of support for advanced features, specially in queries. If you want to do any kind of advanced querying then you will be writing SQL code. The framework simplifies SQL to an ... [More] unrealistic point. If you are using associations (something so basic) then good luck doing any kind of advanced filtering, sorting, etc... with the built in tools."

I'm not sure this one is even worth arguing. This qualm is against Hibernate - not Grails. The arguments FOR Hibernate have been made countless times... and that horse has been beaten to death.

"2. Debugging is a mess. If you have an error during your bootstrap or other start up problems then good luck if you can even read the stacktrace in your command prompt (They have over 500 calls in the stacktrace). I had to pipe mine out to a file then read the file in notepad.

If your bug is in your controller/view then you get a nice webpage with the exception. The only problem is that the error message on the website is 95% not your actual problem. It will tell you the problem is on line 150 while the problem is most likely down another 50 lines."

I would agree in part with this point... Groovy stacktraces can be tough to make sense of. Grails has helped the issue a bit by introducing a stack sanitizer in the GrailsUtil class. This is a class that is employed by framework code, and typically won't be leveraged by Joe-Developer.

Addressing this point in full would require asking the developer on how they employ TDD in their development environment. Without writing tests first (yes, even with BootStraps), coding in a dynamic language can be EXTREMELY painful.

"3. The documentation is poor. I used Grails to do a semi-advanced project and outside of the basics, the documentation will not get you far. I found myself often searching on google for answers and through trial and error."

IMHO, the user guide is pretty good, but overall the documentation is pretty weak. In fact, I would go as far as to say the Grails website is an embarrassment. Ever since G2One 'ate their own dog food' and powered the Grails website using Grails, it's been a bit chaotic.

As development shops continue to explore Grails, there will likely be many developers who urge the 'powers that be' to take a look at Grails. What a shame it would be for upper management to disregard the power of Grails (and the paradigm shift towards convention, for that matter) because the Grails website isn't readable.

Over the past few months - things have gotten better. Some Good Samaritans in the Grails community have taken it upon themselves to clean up some of the formatting on Grails.org. I hope this continues to improve.

"4. Memory consumption and performance. Grails is a memory hog. My website when doing absolutely nothing consumes nearly 80 megs. One of the main problems that may increase your memory consumption and/or CPU is because of problem #1 on this list. You will so often be doing advanced things manually that should be done by the database and/or the framework. Many times I had to duplicate/triplicate objects to accomplish what I needed."

... not sure how to answer this one either. It's tough to comment without knowing what the user has to work with. What are their JVM settings, etc?

Without a sound understanding of the underpinnings of Hibernate - things can get painful. Memory can be an issue.

In my experience, Grails memory consumption hasn't been as bad as I've seen reported. I've deployed production Grails applications to both Windows and *nix JVMs. Oh - and two of the four applications I've deployed use the Quartz plugin. For those unfamiliar with the plugin - it's essentially user managed threads that execute on a timed fashion.

"5. Buggy. The framework randomly throws Hibernate errors (concurrent session problems etc..) when I am browsing my website. You refresh the page and the problem goes away."

What errors? Randomly? Again, a sound understanding of Hibernate is required to be really effective with Grails. 'Random' errors can occur with Hibernate if you have an application that does A LOT of concurrent updates. If that is the case, the user is likely seeing Optimistic Locking Exceptions. This is not 'random' and is, in fact, a feature of Hibernate. More on optimistic / pessimistic locking can be seen in the Grails documentation.

Outside of the Hibernate stuff, I have encountered a few bugs. No show-stoppers though. Here's one I found related to 404 mappings and another related to declarative 500 error code processing.

Finally, the poster suggests that Grails is not ready for 'real world' scenarios. I disagree... as do these folks.

Successful Grails Deployment

2008-09-07T01:05:00.003-05:00

I haven't been very actively blogging or posting on the Grails' list for the last few weeks - and for good reason. On Friday I deployed a major integration application for my current employer: a phase one pharmaceutical trial unit. The application integrates our cardiac system with our clinical trial management system. The application associates ECG (Electro Cardio Graph) data with the right trial subject, for the right event, at the right time. The deployment so far has been a huge success.

While this application doesn't really fit into Grails' sweet spot (web 2.0 applications, IMHO), there were some major reasons I chose Grails:

1) Grails is built on industry-strength, proven technologies. The availability of Spring / Hibernate for managing transactions was an absolute necessity.

2) The dynamic I/O methods added to the Groovy JDK were perfect for the large amounts of data streaming that the application required.

3) I needed the ability to quickly develop administration screens to the user interface. The development boost of edit/refresh in conjunction with scaffolding was perfect.

4) The quartz plugin. The backbone of the application is built on five quartz jobs.

Now that this deployment has been shipped, I should have some time be a bit more active in the Groovy / Grails community :)

Grails' JAR build time check

2008-08-19T14:22:00.007-05:00

A while back, there was a discussion on the grails-user mail list regarding using a different version of commons-lang than the one provided by Grails. The common answer on the thread was, "just try it out...".

Ok, great. With that advice I went into $GRAILS_HOME/lib and I replaced commons-lang-2.1.jar with commons-lang-2.4.jar. My reason for doing so was that I wanted to use some methods on the DateUtils class that were introduced in commons-lang 2.2. My only question was this: 'how could I ensure that I would always have at least commons-lang 2.2 in my project when deploying as a war?'

Having very good test coverage would be one answer :) or another would be checking that when a WAR file is built that the commons-lang version is checked at build time. With my implementation, I ensure that I only have one version of commons-lang in my project and that the JAR's manifest file details the version expected. Here's the script.

$PROJECT_HOME/scripts/Events.groovy:

import java.util.jar.JarFile

eventWarStart = {
  String nm = "commons-lang"
  def min = 2.2
  def jars = new File("$stagingDir/WEB-INF/lib").listFiles() as List
  def commLang = jars.findAll { return it.name.startsWith(nm) }
  if (!commLang || commLang.size() != 1) throw new RuntimeException("[fail] Commons lang problem")
  def manifest = new JarFile(commLang[0]).manifest
  if (!manifest) throw new RuntimeException("No manifest in ${commLang[0].file}")
  def vers = manifest?.mainAttributes?.getValue("Implementation-Version")?.toBigDecimal()
  if (!vers || vers < min) throw new RuntimeException("[fail] Illegal $nm version: ${vers}. Minimum $min")
}

More on Grails Bootstrapping

2008-07-28T15:48:00.016-05:00

In a previous post, I shared a simple Groovy script that exhibits a mechanism for executing scripts within a bootstrapped Grails environment (basically it shows how to use GORM outside of a running web application or integration test).

The script I originally posted is great for simple scenarios, but what about if you are dealing with GORM associations? Well, it becomes a big-time PITA due to the default 'lazy' nature of GORM associations. Should you attempt to load an association, you'll be greeted with the infamous LazyInitializationException.

So... instead of changing the default nature of all of your associations using the ORM DSL, you can use the following updated script :)


import org.codehaus.groovy.grails.support.PersistenceContextInterceptor

Ant.property(environment: "env")
grailsHome = Ant.antProject.properties."env.GRAILS_HOME"

includeTargets << new File("${grailsHome}/scripts/Bootstrap.groovy")

target('default': "Runs scripts in the test/local directory") {
    //we need one arg, the script to run.  Follow a convention here, the argument 
    //is the name of the script to run minus the file suffix and 'Script' 
    //naming convention.  For example, running:
    //>grails ScriptRunner Merge
    //will run $PROJECT_ROOT/test/local/MergeScript.groovy with the fully 
    //bootstrapped environment
    if (!args) {
        throw new RuntimeException("[fail] This script requires an argument - the script to run.")
    }
    depends(configureProxy, packageApp, classpath)
    classLoader = new URLClassLoader([classesDir.toURI().toURL()] as URL[], rootLoader)
    Thread.currentThread().setContextClassLoader(classLoader)
    loadApp()
    configureApp()

    def interceptor = null
    def beanNames = appCtx.getBeanNamesForType(PersistenceContextInterceptor)
    if (beanNames && beanNames.size() == 1) {
        interceptor = appCtx.getBean(beanNames[0])
    }
    try {
        interceptor?.init()
        new GroovyScriptEngine(Ant.antProject.properties."base.dir", classLoader)
            .run("test/local/${args}Script.groovy", null)
        interceptor?.flush()
    } catch (Exception e) {
        e.printStackTrace()
        interceptor?.clear()
    } finally {
        interceptor?.destroy()
    }
}

The key to the script is 'wrapping' the destination script invocation with an instance of a PersistenceContextInterceptor. This mechanism is similar to how Grails controllers work in relation to the OSIV Interceptor.

{Update}: if you are using Hypersonic, you'll probably want to shut down the DB in the 'finally' block. You can use the following to do so:


...
} finally {
  interceptor?.destroy()
  new groovy.sql.Sql(appCtx.getBean("dataSource")).call("SHUTDOWN")
}

More on Grails declarative error handling

2008-07-10T16:09:00.020-05:00

In a previous post, I outlined a mechanism for declaratively handling a Grails NoSuchFlowExecutionException from a webflow. It turns out that the post will only be useful if you are using Jetty with Grails 1.0.3. If you are using another application server (I've tested with Tomcat and Glassfish) - you're out of luck. Grails' ability to handle URL Mappings to HTTP Response codes defined in their documentation, does not work as expected. The reason isn't due to a bug in the Grails source code (IMHO), but rather not enough clarity in the servlet specification. The issue being: how should a servlet filter chain operate when an error occurs AND an 'error-page' mapping is configured in the web.xml? Should the entire chain be invoked again, or should the container just dispatch to the error handler from the current position in the chain?

Once Grails packages an application, it generates a web.xml file that maps a root filter (the first one in the filter chain to be executed) and an error page that refers to an error handling servlet.

The root filter looks something like this:

...
try {
  bindRequestAttributesToThread()
  chain.doFilter()
} finally {
  removeAttributesFromThread()
}
...

And the error handling servlet like:

...
//following throws exception if no attributes
getAttributesFromThread()
if (mappedHandler) {
 renderResponseWithMappedHandler()
} else {
 renderBasicErrorResponse()
}
..

In Jetty - the error handling servlet works great... and just as described in the Grails documentation. But... when running in Tomcat / Glassfish, an IllegalStateException is thrown in the error handler when the servlet tries to obtain the attributes from the thread. What gives? Why the difference between containers?

After an embarrassingly large amount of debug time, I finally saw what was happening. Take a look at the following:

Jetty:
Start root filter
- bind attributes to thread
- chain
- error handler invoked (404, 500, etc)
- remove attributes from thread
End root filter

Tomcat:
Start root filter
- bind attributes to thread
- chain
- remove attributes from thread
End root filter
Error handler invoked (404, 500, etc)

In Jetty, the error handler is invoked in the life of the filter chain. With Tomcat, the error handler is invoked after the chain completes. I double checked the servlet specification to see who is at fault... and from what I can gather it's just a case of ambiguity. There is no clear instruction on what should happen. So what to do? I wanted a solution where I did not have to tamper with the Grails source code... and I think I came up with something viable.

Section SRV 9.9.3 of the 2.4 servlet specification defines error filters:

The error page mechanism operates on the original unwrapped/unfiltered request and response objects created by the container. The mechanism described in SectionSRV.6.2.5, "Filters and the RequestDispatcher" may be used to specify filters that are applied before an error response is generated.

By having the root filter be invoked upon errors in the chain, I get something similar to what happens with Jetty:

Start root filter
- bind attributes to thread
- chain
- remove attributes from thread
End root filter
Start root filter
- bind attributes to thread
- error handler invoked (404, 500, etc)
- remove attributes from thread
End root filter

In order to have the filter be invoked upon error, the mapping should look like:

<filter-mapping>
  <filter-name>grailsWebRequest</filter-name>
  <url-pattern>/*</url-pattern>
  <dispatcher>ERROR</dispatcher>
  <dispatcher>REQUEST</dispatcher>
</filter-mapping>

Note: this doesn't cause a new request by way of a redirect, but it just ensures that the root filter is invoked before the error handler servlet is executed.

Tampering with Grails' generated web.xml turns out to be pretty easy due the exposed event hooks in the build system. Here's the script I used to ensure the error mapping above (my script is located at $PROJECT_HOME/scripts/Events.groovy):

import groovy.xml.StreamingMarkupBuilder

//modify the generated web.xml so that it supports being mapped to 'request' and 'errror'
eventWebXmlEnd = {String tmpfile ->
    //find the filter mapping to change
    String filterNm = "grailsWebRequest"
    def root = new XmlSlurper().parse(webXmlFile)
    def gwr = root."filter-mapping".find { it."filter-name" == filterNm }
    if (!gwr.size()) throw new RuntimeException("[fail] No Filter named $filterNm")
    if (gwr.dispatcher.size()) throw new RuntimeException("[fail] Dispatchers exist for $filterNm")

    //xml is as expected, now modify it and write it back out
    gwr.appendNode {
        dispatcher("ERROR")
        dispatcher("REQUEST")
    }
    //webXmlFile is an implicit variable created before event is invoked
    webXmlFile.text = new StreamingMarkupBuilder().bind {
        mkp.declareNamespace("": "http://java.sun.com/xml/ns/j2ee")
        mkp.yield(root)
    }
}

For an even more exhausting explanation, take a look at the mail list thread, or the newly opened JIRA.

Grails Bootstrapping

2008-07-03T14:50:00.025-05:00

So outside of the project I've been describing in various posts thus far, I have a Grails application that I've developed for my real job. The application manages personnel data related to the industry I'm in. The data can originate from our website, or through a phone screening partner. The phone screening partner manages an Excel spreadsheet for all of the data that they collect, and it's my job to merge that data back into our central database.

Seems simple enough:

Create an SSH tunnel to our production database
Configure a Grails DataSource with an arbitrary environment configuration ('prodtunnel')
Develop a script that parses the spreadsheet, binds each row to a Grails domain object and then persists the object to the production database by leveraging Grails' dynamic GORM save() methods

Sounds easy enough, right? Well - it turns out that getting GORM to work outside of a running application (and an integration test) was tougher than I planned.

My first swipe went something like this:

Create a GANT script.
Follow the documentation here
Writing a one-liner just to ensure that Grails was properly bootstrapped and that I would be able to use GORM for persistence.

Here's that initial script:


Ant.property(environment: "env")
grailsHome = Ant.antProject.properties."env.GRAILS_HOME"
includeTargets << new File("${grailsHome}/scripts/Bootstrap.groovy")

target('default': "First try") {
    //copy and paste from $GRAILS_HOME/scripts/Shell.groovy
    depends(configureProxy, packageApp, classpath)
    classLoader = new URLClassLoader([classesDir.toURI().toURL()] as URL[], rootLoader)
    Thread.currentThread().setContextClassLoader(classLoader)
    loadApp()
    configureApp()
    println "Subject count: ${Subject.count()}"
}

When I ran the script, here's what I was greeted with


No signature of method: static Subject.count() is applicable for 
argument types: () values: {}

So what's wrong? Well, Groovy scripts are not interpreted line by line (and as a whole, Groovy IS NOT an interpreted language). From GinA:

Groovy syntax is line orientated, but the execution of Groovy code is not. Unlike other scripting languages, Groovy code is not processed line-by-line in the sense that each line is interpreted separately.

So what does that mean in the context of the problem I've presented? Well, even though I've bootstrapped the the Grails environment and THEN called my the dynamic GORM method on my domain class - it doesn't matter. The JVM has already loaded the Subject class because the Java byte code has already been generated from the script by the time the script runs. That's the problem, and now here's my solution:


Ant.property(environment: "env")
grailsHome = Ant.antProject.properties."env.GRAILS_HOME"

includeTargets << new File("${grailsHome}/scripts/Bootstrap.groovy")


target('default': "Working edition") {
    //we need one arg, the script to run.  Follow a convention here, 
    //the argument is the name of the script to run minus the file 
    //suffix and 'Script' naming convention.  For example, running:
    //>grails ScriptRunner Merge
    //will run $PROJECT_ROOT/test/local/MergeScript.groovy with the 
    //fully bootstrapped environment
    if (!args) {
        throw new RuntimeException("[fail] This script requires an argument to the script to run.")
    }
    //copy and paste from $GRAILS_HOME/scripts/Shell.groovy
    depends(configureProxy, packageApp, classpath)
    classLoader = new URLClassLoader([classesDir.toURI().toURL()] as URL[], rootLoader)
    Thread.currentThread().setContextClassLoader(classLoader)
    loadApp()
    configureApp()
    new GroovyScriptEngine(Ant.antProject.properties."base.dir", classLoader)
        .run("test/local/${args}Script.groovy", null)
}

MergeScript.groovy:


println "Subject count: ${Subject.count()}"

Console after running the script:


Subject count: 505

The key to the solution is using the class loader created by the Grails bootstrapping process and then passing that to my target script. I hope this saves someone some time out there :) If anyone's got something more elegant or cleaner - please post a comment!

Update: here's a bit more information on the same subject.

Rendering Images in a Webflow

2008-07-02T06:59:00.006-05:00

Ok... so I've ditched the 4 AJAX calls from Webflow that I've mentioned before: not due to complexity, but because I took an honest look at my user interface and I realize it just didn't quite make sense.

This exercise has brought me to an interesting realization... developers may create their own bottlenecks based on a desire to really dig into a technology. For example, since Grails takes the tedious and complex nature out a lot of server-side stuff, developers can now spend their time spinning wheels in different parts of application development. In my case, my wheels have been spinning in the mud of the the user interface because I've gotten AJAX-crazy.

Speaking of Webflows... I've got an interesting problem I'm now trying to solve. Within the flow, I have a state that processes uploaded images. After I validate the content type and size, I want to store the images in flow scope so that I can write them to disk when the flow ends (and have Grails 'clean up' should the user abandon the flow). But, before ending the flow, I want to present a 'preview' page where users can optionally edit data already entered and see the images that have been uploaded. Therein lies the problem... how do I render the images that are stored in flow scope? Since rendering an image within a flow state would violate the transitions semantics, I can't take that route... Also, I don't have access to flow scope from other controller actions, so that won't work either. I've got a couple of ideas on how to solve this, but first I want to see if there are any Grails users out there that have any better ideas.

... more to come on this later.

Webflow Woes

2008-06-20T06:59:00.009-05:00

So as I began to really dig into webflow, I realize I didn't quite 'get it'. I was under the incorrect assumption that a flow had zero dependency on HttpSession. It turns out that during the life of a flow, an instance of org.springframework.webflow.conversation.impl.ConversationContainer (a package protected class) is stored in session scope for the duration of the flow under the session key, 'webflow.conversationContainer'.

So what happens if the session expires? Or if the user tampers with the '_flowExecutionKey' request parameter? So far I've just seen that Grails throws a NoSuchFlowExecutionException, and you're on your own with how to process it. It would be great if the Grails webflow DSL allowed a declarative mechanism for handling invalid states. Some convention like:


def myEventFlow {
  startOfFlow {
    ...
  }
  nosuchflow {
    action {
      log.debug("Flow not found..")
      flash.message = "Flow expired, start over"
    }
    on("success").to "startOfFlow"
  }
}

As a work around, I've configured an error handler controller (SystemController):


class UrlMappings {
  static mappings = {
    "/$controller/$action?/$id?"
    {
        constraints {
            // apply constraints here
        }
    }
    "500"(controller: "system", action: "error")
  }
}

And then put a bit of logic in the SystemController's 'error' action to detect the invalid flows:


def error = {
  if (isWebFlowException(request)) {
   def uri = getWebflowUri(request)
   flash.message = message(code: "webflow.illegal.${uri.controller}.${uri.action}",
           default: "Illegal flow detected.")
   render(view: "/${uri['controller']}/${action:uri['action']}/error")
   return
  }

  render(view: "/error")
}

/**
 * @return true if the exception associated with the
 *         request is a 'NoSuchFlowExecutionException'
 */
boolean isWebFlowException(request) {
  def exception = request.exception
  return exception instanceof NoSuchFlowExecutionException ||
       exception?.cause instanceof NoSuchFlowExecutionException
}

/**
 * @return the URI in the webflow that caused the
 *         'NoSuchFlowExecutionException'.  The map will contain the
 *          controller and the action
 */
def getWebflowUri(request) {
  //get the original servlet path where the error occurred
  String servletPath = request['javax.servlet.forward.servlet_path']
  //remove '/grails/' at the beginning and the dispatch matching
  String[] split = servletPath.substring(8, servletPath.indexOf(".dispatch")).split("/")
  return [controller: split[0], action: split[1]]
}

This solution seems a bit fragile... especially since I'm depending on the internal Grails dispatch mechanism to be mapped to *.dispatch with a '/grails/' prefix, but it works.

Now, my next big challenge is to be able to handle 4 AJAX operations in one of my webflow states. I've been taking a look at Glen Smith's solution, but my scenario is not going to require any polling. I'll essentially have forms that can be submitted through a single 'state' (from the UI perspective). More on that later...

Project Overview

2008-06-16T07:02:00.006-05:00

As I mentioned in my first post, I'm putting together a web-based 'event management' system. The goal of the project is to allow non-profit organizations promote 'events' and manage data related to hosting events: registrants, volunteers, vendors, etc.

Some of the challenges in the Grails world are as follows:

Two (at least) webflows for supporting the registration wizard and the event creation wizard. One of the webflows will have a number of AJAX calls that will need to interact with flow scope.
Managing varying levels of volunteer access to an administration area of the application.
File uploads
Dynamic image processing (re-sizing to a 'least common denominator' dimension)
PDF ticket generation
Allow organizations to manage an RSS feed
Payment gateway integration (most likely with authorize.net)
Optionally creating a plug-in so that others can leverage the payment gateway interaction.

A bit about the data model:

Organizations are the root of the application.
Organizations can define a number of events
Organizations can have a number of volunteers
There are four levels of volunteer access: administrator, regional leader, captain and miscellaneous.
Events have many promotion codes, registrations and vendors
A registration can have many ticket assignments

Current status:
I'm currently working on the administration section of the application... and in particular I'm working towards completing one of the more challenging parts (from my current perspective) of the application: the creation of events.

Hello World

2008-06-12T09:37:00.000-05:00

First blog post... the pressure is on to write something inspiring, intuitive, etc. Whatever - I just want to get something posted.

Over the coming months, I'll be blogging mostly about my adventures in Grails programming. I've got an October deadline to deliver an event management web application. I've got about ten hours per week to program, and in order to complete this project, I truly need to leverage a framework that will greatly reduce development time. I'm gambling that I'll be able to pull it off with the help of Grails and the Grails community.

I've really enjoyed two things about the Grails community thus far: 1) Live coding demonstrations and 2) People who blog about their joys, frustrations and general geeky conquests. That's the inspiration behind this.

... to be continued.

Q.	What do you mean, the XPORT format is "open?"
A.	Specifications for the XPORT transport format are in the public domain. Data can be translated to and from the XPORT transport format to other commonly used formats without the use of programs from SAS Institute or any specific vendor