The lite at the end of webservice tunnels

In the last post, I alluded to how shifting complexity from one area to another can actually improve perceived simplicity. Here is a concrete example of how we got rid of scores of wsdl-client generated jaxb classes and brought in a fresh wave of simplicity to our Grails applications.

Problem

One or more of our Grails application uses several our own plugins and each of them talk to several web services that share a lot of xsd schemas, the java client generation of which end up proliferating duplicated jaxb classes in each of our plugins and also in the applications which use them, thus resulting in multiple classpath conflicts.

Also we dont like the date conversions from Date to Calendar to GregorianCalendar to XmlGregorianCalendar or other conversions from Byte to Int to Shorts. Also jaxb does some non-intuitive magic with some of the xsd:any types in the schemas. If xsd schemas want strict type checking, fine. We from the “def” world, just look everything as plain data. And finally the jaxb classes look ugly.

Solution

With a combination of groovy-wslite, grails wslite plugin and MarkupBuilder and Config, the code become pretty straight-forward, almost.

From the grails command prompt, run install-plugin wslite

def grailsApplication

public def amazonBookListMania() {
  WsliteConnector.instance.withSoap(serviceURL: grailsApplication.config.amazonBookListManiaServiceUrl) {
    def soapResponse = send(SOAPAction: grailsApplication.config.amazonBookListManiaSoapAction) {
      //Get the namespaces, which is a simple Groovy map
      //['xmlns:amz':'http://www.amazon.com/ListMania']
      envelopeAttributes grailsApplication.config.amazonBookListManiaNamespaces
      body {
        //Get the raw xml and slap it into soap body
        mkp.yieldUnescaped(createXmlRequest())
      }
    }
    //Wslite automatically xmlslurps the result and gives you the closure
    def response = soapResponse.body.bookListManiaResponseRoot
    println response.Items.count()
  }
}

The “withSoap” is a DSL, that is supposed to be injected automatically into controllers, but on other classes it didnt seem to work. I tried using WsliteEnhancer, but that didnt help either. So the direct call to withSoap using WsliteConnector singleton.

The example shows all params injected from grailsApplication config, to allow webservice urls for different environments.

The interesting aspect is the envelopeAttributes, which is a map of namespaces. Since we are not using generated client classes, whcih does the work for us, the namespace prefixes must be specified somehow in the xml itself as part of soap request. Adding namespaces to envelopeAttributes will append the namespaces to the soap-env tag. No need to include the namespace for soap-env itself. The wslite plugin does that.

public String createXmlRequest() {
    def writer = new StringWriter()
    def builder = new MarkupBuilder(writer)
    builder.'amz:ListMania' {
      //tag attributes
      'amz:credentials'(username: 'vasya10', password: 'yeahright')
      //tag value
      'amz:dateRequested'(new SimpleDateFormat('yyyy/MM/dd').format(new Date()))
    }
    //Returns <amz:ListMania><amz:credentials username='vasya10' password='yeahright'></amz:credentials><amz:dateRequested>2013/01/01</amz:dateRequested></amz:ListMania>
    return writer.toString()
}

With the complexity of creating the xml shifted into MarkupBuilder, creating soap requests become trivial, even more so with the withSoap DSL.

Wsdl import generated classes
Pros
No need to worry about namespaces
Set and Get of values use Java-like syntax
Setting values does not matter as xml marshalling happens at a later time than setting the values
Setting tag attribute or tag value is no different (ie code doesn’t care)

Cons
Every minor change in wsdl forces wsdl client generation
Duplicated client generated classes could cause classpath conflicts
The hierarchy of the xml elements is not apparent in the code, the only way to know the hierarchy is to look at the xml samples

WSLite + MarkupBuilder
Pros
No confusion about which client to use: there are scores of them and each have some unique issue (jax-ws, jax-b, cxf and so on)
No wsdl import client generated sources and jars
No crazy xerces, xalan and other marshalling runtime errors, especially if deploying to different app servers (tomcat, weblogic etc)
If there are trivial changes to elements (saying adding a new optional element), will still work without changing client code (if that element is not used)
Creation of xml is very straight-forward
Code documents the hierarchy of the xml!
Response returns XmlSlurped data, so dot-notation can directly be applied on the objects

Cons
Code must be namespace-prefix aware
Must know tag attributes and tag values to create the xml correctly (this must be a Pro)
Code must generate xml in the same sequence of elements; blocks of elements can still be called out-of-order, but the final xml body must be created in a sequence
Unvalidated namespace prefixes increases testing time
Extra testing time for un-type-checked variables

Compared to our previous jaxb code (with ObjectFactory, creation of new Objects etc.), the code is now 40% smaller, and additionally got rid of the client sources too.

Extensions via metaclass in Grails

Even after several releases and wrapper utilities, representation of time in Java seems to be an unconquerable mess that it originally was. At the same time, surprisingly, web services have been there for a long time now, still there is no magic wand to create webservices and clients.

Webservices is something Groovy/Grails should have made it ridiculously easy to work with, by default. When xml, json, http and so on have been outrageously simplified, the natural expectation is that the webservice call would be like:

WebServiceObject ws = new WebServiceSlurper().parse('http://someweathersite.com/weather.wsdl')
WeatherObject w = ws.getWeatherObject()
Temperature[] temperatures = w.findTemperature(zipcode: '11505', days: 5, unit: 'farenheit')

There are so many wrapper api-s, every webservice client generation is lacking in someway, making a new comer to Grails to try and find what works. axis, axis2, groovyws, xfire, cxf, jws, jaxb – there are so many acronyms and some of them are just wrappers over others, the choice may be overwhelming. Forums do provide lot of answers, but most times what works for one goose, does not work for even for its goose twin.

So after my friend pointed out the pot-holes of webservices in Grails, I started using the jws. All good until I hit the xs:date object in our wsdl, which translates to XMLGregorianCalendar. Like we all wanted a new class to represent date and time. I seriously hope Java/Groovy will deprecate and delete all its Date and Calendar classes and introduce just one class with a philosophical outlook:

/**
* Dead yesterday, unborn tomorrow, why fret about them, if today be sweet?
*/
class StarDate{
public double day;
public double increment(double unit) { day += unit }
public String toString() { return "Captain's log, stardate $day"  }
}

Conversion between a date in String to XMLGregorianCalendar goes like this:

SimpleDateFormat sdf = new SimpleDateFormat("MM/dd/yyyy")
Date d = sdf.parse(dateString)
GregorianCalendar g = new GregorianCalendar()
g.setTime(d)
def date = DatatypeFactory.newInstance().newXMLGregorianCalendar(g)

So much for a date? I tried to generate the client classes with a custom binding to Date instead of XMLGregorianCalendar. No matter what I tried, the value inside the webservice always ended up as long format “Sun Dec 4 …”. Fed up, I changed it back to XMLGregorianCalendar and now I saw the date as 1995-03-28T06:00:00. So where did the timezone come from and how do I avoid it? Stackoverflow had an answer:

dob.setTimezone(DatatypeConstants.FIELD_UNDEFINED)

Seriously, DatatypeConstants? Minimum integer value to specify undefined? Anyway, with that undefining of the timezone, the webservice was correctly seeing the date as just 1995-03-28. But I still dont like the fact that I had to convert this in the service class or that I had to put this in an utility class.

And all this while I was developing a Grails plugin. I wanted to make the conversion as an extension to String, so that my call is like:

dateString.toXmlGregorianCalendar()

Recall that Grails plugin will load the *BootStrap.groovy when it starts up. We need to inject a metaclass method into java.lang.String so Grails runtime knows about it. (Only when run standalone, not after installing within an application).

public class MyPluginBootStrap {
def init = { servletContext ->
println "booting the plugin"
new GroovyShell(this.getClass().getClassLoader()).evaluate(new File("grails-app/conf/MetaFunctions.groovy"))
}
}

And define the MetaFunctions.groovy:

String.metaClass.toXmlGregorianCalendar = {
SimpleDateFormat sdf = new SimpleDateFormat("MM/dd/yyyy")
Date d = sdf.parse(delegate)
GregorianCalendar gcal = new GregorianCalendar()
gcal.setTime(d)
def dob = DatatypeFactory.newInstance().newXMLGregorianCalendar(gcal)
dob.setTimezone(DatatypeConstants.FIELD_UNDEFINED)
return dob
}

In .Net doing this may be simpler – just define StringExtensions. But they are static methods and cannot override existing methods. But in Groovy that is entirely possible, which makes a powerful feature, because if you don’t like the behaviour of a plugin class, just override it via metaclass using the above technique.

SOAP UI 3.0 vs WSI Tools

The SOAPUI 3.0 by default comes with wsi-test-tools. When a wsdl is checked against for WS-I compliance it throws a “Premature end of file” error from sax parser. This is apparently a bug in soap ui 3.0. To fix this, follow these steps:

  • Download the wsi-test-tools from http://www.ws-i.org/deliverables/workinggroup.aspx?wg=testingtools (Interoperability Testing Tools 1.1); Expand the zip
  • Goto SOAP UI , Preferences menu > WS-I settings > Tools , set the value to the directory upto “wsi-test-tools”.
  • Run the ws-i compliance (Ctrl + Alt + W) again and the report should be generated fine.
  • If the wsdl is hosted on https, dont forget to import the keys into the jre/lib/security/certs which is located under $eviware_home directory.