Hot-pluggable extensions in Grails – adding and changing your application behavi

One of the biggest challenges in developing web applications is the ability to alter the behaviour of your program once the application has been deployed.

If you are building a highly extensible application like a content management system or an e-commerce engine, you don’t necessarily want to restart or redeploy every time a change has been made. You also want the ability to define business rules, add custom scripts, and turn functionality on and off.

In this post, I will show you how to externalize application functionality using closures, the Groovy configSlurper and the pluggable behaviour pattern.

How it works.

The idea is pretty simple.

Within our main application, we define extension points that load programming logic from an outside source. The extensions can live in a database, the file system or within any other means of storage.

We treat them as any other extenalized resource. When we need to change the behaviour of our application or add new functionality, we simply add new extensions or modify our existing ones.

The Pluggable Behaviour Pattern.

One of the aha! moments for this exploration came when I was looking at the slides of Venkat Subramaniam’s presentations on Design Patterns in Java and Groovy. In it, he describes how to define the pluggable behaviour pattern in Groovy. Here are the relevant slides.

At the core of this pattern is the idea that you can encapsulate functionality you want to inject using Groovy closures.

All we have to do to make these closures into hot-deployable extensions is simply create a mechanism by which we can load them into our application.

How to implement this in Groovy and Grails.

Here is a proof of concept of this functionality. ( note: since I’m a lazy typer, the app runs at http://localhost:8080/ ).

In this sample, I’ve created a simple Payment Gateway service.

Extension files

Each of my payment providers is defined in an external file located within the extension/groovy folder and look like this:

  name = "Not Authorize.net"
  key = "authorizenet"
  authenticate = { status ->
    System.out.println( "Payment Provider 3 authenticate for ${status}" )
    return 'authenticated'
  }
  processCreditCard = { cardNumber ->
    System.out.println( "Payment Provider 3 --- process credit card ${ cardNumber }"  )
    return 'accepted'
  }

---

As you can see, each extension is just a ConfigSlurper property file with String values and a few simple closures defined.

Loading extension files

The Payment Gateway Service class simply keeps an internal map of providers.

This is accessible to other parts of the system that need to use it via paymentGateway.providers.

To load or update the list of providers, the application simply reads the externally defined extension files and loads them into a ConfigSlurper object:

  def refreshProviders = {
    providers.clear()
    new File( grailsApplication.config.externalFiles ).eachFile{ file ->
      def config = new ConfigSlurper().parse( file.toURL() )
      providers.put( config.key, config )
      log.debug( "loaded provider ${config.name} from file ${ file }")
    }
  }
---

To make reloading easy, I added an action in my controller to reload my payment providers, this is found in CreditCardController ( grails-app/controller/CreditCardController ) and can be accessed via http://localhost:8080/refreshProviders.

Additionally, in grails-app/conf/Config.groovy, I specify the location of my extensions directory via:

  externalFiles = "c:/external/groovy"
---

Finally, so we don’t start up with a list of empty providers, I added this to my bootstrap file ( grails-app/conf/bootstrap )

  def paymentGatewayService
  def init = { servletContext ->
    paymentGatewayService.refreshProviders()
  }
---

Creating a extensible method

  def provider = providers[ providerKey ]
---

Once it knows which provider to use, calling the methods within the external extension file is simply a function call to the Config object.

  def authenticationResult = provider.authenticate( status )
---

Here, the authenticate method is the one defined within the externalized extension file. As you can see, we can pass in parameters in and out of the script and chain different parameters. I can have many extension points within my methods. So if I was to write a shipPackage method for a provider like FedEx, DHL or Canada Post, for example, I could write it as

  def calculateShippingRate = { key, item, addressObject ->
    def provider = resolveProvider( key )
    def address = provider.resolveAddress( addressObject )
    def taxRate = provider.calculateTaxRate( item, address )
    def packageDimension = provider.resolvePackageTier( item )
    def shippingCost = provider.calculateShippingCost( packageDimension, taxRate, item )
    return shippingCost
  }
---

and then externalize each of these methods into closures.

Seeing it in action:

1. Change your config.groovy file so that the extension directory points to the right location.
2. Start the application under app with grails run-app
3. Navigate to http://localhost:8080/creditCard/You will see three payment aggregators available under the provider dropdown. Add a dummy credit number and you will see each provider giving you a different behaviour.
4. Go to your extensions/groovy directory and move two providers out of the folder.
5. Navigate to http://localhost:8080/creditCard/refreshProviders , and then back to http://localhost:8080/creditCard/You will see that only one provider is available.
   
6. Restore the providers and write your own, repeat step 6 for fresh, chewy providers.

Benefits of this approach.

I see three benefits to this approach – it simplifies your code, allows you to change it on runtime and lets you build manageable customizations.

1. Simplifying code

One of the nice things about this approach is that it forces you to write code in a generic way.

Let’s say I’m writing a generic payment gateway. Instead of worrying about Paypal or WorldPay or Authorize.net, I just need to write out bits of my application that assume this functionality works. It allows me to abstract away my implementation details and keeps my deployed code simpler.

2. Change behaviour at runtime

Another benefit of this is the ability to add new behaviours at runtime.

Let’s say that I have a new business requirement to support the WorldPay interface to provide payments. Traditionally, I would write a new class that supports WorldPay, add it to my application and deploy a new War file. By using extension points, I can simply write an extension and add it to my extensions folder. After I reload my list of extensions, the new behaviour is ready for use.

3. Manageable customizations

Grails plugins are great, but a lot of them have a ‘all or nothing’ approach to adding functionality.

With extensions, you can choose to deploy a small part of existing functionality. In our payment gateways example, we would have to add all supported gateways at deploy time to enable or disable them if we were not using extensions. If you look at the Magento Commerce site, you can see they have over 300 payment gateways. Loading them all at application startup would create unnecessary memory use. With extensions, you get to pick and choose which ones to use.

Unresolved questions about this stuff.

Loading / unloading of classes: I can imagine adding an evict / install closure and passing a classLoader into the actual extension file, but haven’t really tested this and don’t have a clear answer to how this would impact things like the PermGenSpace or cause exceptions.

Plugin interactions: It is still unclear to me how an extension could add a plugin it needs or modify the Spring or ApplicationContext on the fly without a restart. The Grails roadmap suggests that there would be a hot-deployable plugin infrastructure in place soon, which might solve this question.

Ideally, this technique would be used in the creation of new plugins. Instead of creating a separate plugin for Paypal or Google checkout, we would have one top level plugin that abstracts all the difficult parts of the implementation and use extensions for implementation details like web service definitions, etc.

Maybe it makes sense to make a plugin of this stuff with a nice little GUI for managing extensions and a domain object to properly store them.