CF Groovy 1.0RC (With Hibernate!)

I've just released 1.0RC of CF Groovy, including Hibernate support.  You can download it, or view the demo app.  The download includes both the demo and the runtime engine.

The big new feature is Hibernate support, of course.  Here are a couple snippets from the demo app.  First, the entity class:

package com.barneyb

import javax.persistence.*

@Entity
class User extends AbstractEntity {

    String firstName
    String lastName
    String email

    @Column(unique = true)
    String username
}

Then the engine setup (in code here, ColdSpring would be better):

<cfset application.cfhibernate = createObject("component", "HibernatePlugin").init() />
<cfset application.cfhibernate.setEntityPath(
    getDirectoryFromPath(getCurrentTemplatePath()) & "entities"
) />
<cfset application.cfhibernate.setColdFusionDsn("cfgroovydemo") />
<cfset application.cfhibernate.setAnnotatedClasses(listToArray("com.barneyb.User")) />
<cfset application.cfgroovy = createObject("component", "cfgroovy").init() />
<cfset application.cfgroovy.addPlugin(application.cfhibernate) />

Finally, doing some persistence operations:

<cfset request.sess = application.cfhibernate.getSessionForRequest() />
<g:script>
import com.barneyb.*

variables.user = new User([
    firstName: attributes.firstName,
    lastName: attributes.lastName,
    username: attributes.username,
    email: attributes.email
])
request.sess.save(variables.user)
request.sess.flush()
</g:script>
<cfset userList = request.sess.createQuery(
    "from User order by lastName, firstName, username"
).list() />

As you can see, very minimal fuss to do some pretty powerful things.  This only scratches the surface of what Hibernate can do; the full power of the tool is available (custom annotations, the full XML mapping capabilities, caching, etc.).

Another significant change is that all non-inline Groovy is precompiled into Java classes and loaded dynamically.  This gives you compiler checks on your Groovy code and engine startup, more helpful message for code errors, and some performance improvements.

Finally, as illustrated above, the <g:setPath> tag has been superceded by an explicitly managed CF Groovy runtime.  This gives you complete control over the lifecycle of the engine, which lets you target performance in production and development speed in development.  You can still use <g:setPath> instead of managing the runtime yourself, but that should be considered a deprecated practice.

Thread.setContextClassLoader Doesn't Work on CF 8.0.0

I know the number of CF 8.0.0 installs is probably pretty minimal compared to CF 8.0.1, but thought this was worth pointing out.  If you use Thread.setContextClassLoader on CF 8.0.0, it raises no exception, but it doesn't actually set the ClassLoader for the thread.

I ran into this today using my CF Groovy/Hibernate integration on one of my existing CF installs that is still 8.0.0.  It totally bombs, because the classloading trickery fails if it can't set up a classloader.  After spending a bunch of time on it, I just gave up, as nothing I tried had any effect.

Both CF 8.0.1 and Railo 3.0b2 correctly allow overriding the context ClassLoader, and can therefore use the Hibernate via CF Groovy.  I haven't tried in a while, but OBD exhibited similar symptoms as CF 8.0.0 when trying to run Hibernate, so it may suffer from the same problem.  This only affects the Hibernate stuff I'm doing, the base Groovy integration requires no ClassLoader trickery, so it works on all four without issue (and probably CF 7 too).

I should also note that this is NOT the underlying JVM.  I use the same JVM (a 1.6 series from Sun) for all my stuff, both development and production.

Junction, from Sysinternals

I discovered Junction today, while looking for a way to easily test my Groovy/Hibernate integration on all three major CFML runtimes.  It's part of Sysinternals, which is a suite of tools aimed at making Windows not suck so much.  It's got all the low-level utilities you need to efficiently manage and monitor your Windows machines that Microsoft figured people would be happy to write and compile some C++ to use.

Junction, as you might guess, is a utility for managing NTFS junction nodes, which are the equivalent of *NIX symlinks.  With symlinks, I can have a single working copy "live" inside three different web roots, one for Railo, one for CF, and one for OBD.  Any changes made to that working copy are instantly available in all three places without any need to sync anything.

As you can imagine, this makes testing a breeze: just save and go refresh the tab for the runtime in question, and when it's set, go refresh the other tabs to ensure it still works everywhere else.  Until this point, I'd always used separate VMs, but that's way overkill for a standalone CFML app.  If you've got database and web server integration and all that, VMs are a lifesaver, but I just wanted to run the same 30 files in three web contexts and Junction is perfect.

Watch Out for CFQUERYPARAM and Garbage Collection

In the past few months I've spent a fair amount of time doing large data transformations for a couple projects, in to and out of MSSQL databases. In both projects, I've ended up refactoring my code to eliminate CFQUERYPARAM on oft-run queries, because it seems to leak memory (yes, I have debugging disabled), as well as be less performant.

In both cases, the primary offender was repetitive INSERT statements, ranging from 10,000 executions per request up to around 50,000. Stripping the CFQUERYPARAM tags gave about a 40% improvement in performance (batches in ~4 seconds instead of 7-8 seconds), and a huge improvement in memory consumption.

I've also noticed that while CF does do garbage collection within requests (thank god that got fixed), it doesn't do aggressive collection. A few well placed System.gc() calls have been a lifesaver for the larger processes, particularly when combined with judicious use of structDelete to eliminate non-essential references. Just using structDelete makes a difference, but the effect is magnified with the explicit collections.

NB: neither of these are solutions to be taken lightly; both have potentially serious side effects, and go against generally accepted best practices. Not using CFQUERYPARAM exposes you to SQL injection attacks among other things, and garbage collection is expensive in terms of processor utilization.

NB: these results are for ColdFusion 8 (not 8.0.1) against 64-bit MSSQL 9.0.3054 Developer Edition, using the stock CF driver. Other CF runtimes and other databases (and possibly even other MSSQL database drivers) will undoubtedly yield different results.

CF Groovy w/ Hibernate Sample App

I want to put together a little sample app leveraging CF Groovy and it's Hibernate integration, but I don't know what. From a high level, the architecture will look like this:

  • CFML for the view layer (the HTML templates)
  • CFML for the web controller (probably FB3Lite)
  • CFML and Groovy for the service layer (ColdSpring-managed CFC's with Groovy delegation where appropriate)
  • Groovy w/ Hibernate for the data layer

The question is what the application should do. The default choice of a simple blog engine is, well, boring. I considered implementing the BlogCFC API as a drop-in replacement for BlogCFC backed by Hibernate, but that means mapping the legacy schema, which I don't really want to get into for a simple demo.

My next choice is a simple survey application, since that at least has the potential to be real-world useful. It's also sufficiently simple and would provide appropriate illustration of the concepts I need to demo. The data model for survey construction is simple, but it gets a bit nastier with response storage, and I'd like to avoid too much nasty.

I've implemented a basic timeclock as a sample app a few times (CFML, Spring+Hibernate, Grails) as well, but there's a lot of "application" in there, compared to the amount of persistence, which is what I want to be highlighting.

So, my all-knowing readers, what simple app do you want me to implement? Preferably something that will be useful to people, either as it is, or as a starting point for building a richer version of what I produce. I've just about got the Hibernate APIs nailed down, but I've got a couple more hours on that task.  I'm thinking that Wednesday night I'll pick something and start coding, so speak up (in the comments) with what you want to see.

Fun with Classloading in Java

With my CF Groovy project, I've been doing a lot of Java coding of late. It's all been written in CFML, but it's mostly Java code in CFML syntax. As an aside, doing that sucks. CFML is horrible for writing anything more than the simplest Java code. But on to the topic at hand….

To this point in my career, basically all of the Java code I've written has been application code. Most of the CFML, JavaScript, and ActionScript I write is also application code, but I've contributed significantly to Fusebox and built a number of micro-frameworks in all three languages. However, none of that touches on the stuff I've done in the past couple weeks. The primary source of my issues getting Groovy and Hibernate to play together was classloading.

Unlike Grails (GORM, really), any of the various examples on the web, and I'm guessing Joe Rinehart's Hiberailooving project, three of the inviolate framework objectives were zero dependency on an IDE beyond text editing, no manual compilation, and no container restarts. Obviously, Hibernate provides a set of IDE tooling that can be brought to bear, but I'm talking about just the CF integration portion. I want to be productive with emacs and a web browser.

This presented a number of issues with classloading, because Groovy can only be loaded by the GroovyClassLoader (and subclasses). Basically this forced my hand to using compiled Groovy code, and ensuring it and Hibernate are loaded by the same ClassLoader. No GroovyClassLoader at all.

Now all that's well and good, but you have to be a bit careful, because two of the goals where no manual compilation and no container restarts. Doing compilation from CF is fairly simple (instantiate FileSystemCompiler and call 'main' with the right args), though again, getting all the right stuff on the classpath in the right order proved a nightmare. This is further complicated by the while JEE classloader situation, which isn't straightforward at all.

The details of this intricate dance completely escaped me for a week and a half. Then one night, while lying in bed, it all clicked together (thank you, Heather, for going to England). The solutions seem fairly obvious now, but they took a lot of fussing and fighting to deduce. I'd never even instantiated a ClassLoader before, aside from a couple one-off URLClassLoaders in CFML to lazy-load some JARs. Now I've got stacks of ClassLoaders, in some places two or three deep.

After my initial post-epiphany implementation was complete, I realized I can do some streamlining, but that's for after the release. I think I can even remove the need to drop the Groovy JAR in your /WEB-INF/lib folder, without adding any code complexity or runtime overhead, which would a nice bonus for ease of deployment. I.e., unzip the engine, write an entity, save it to the DB. No JARs, no restart the container, nothing.

It's been a very interesting (and pleasurable) experience, since it's so different from my day job of banging out boring little presentation apps all day. I've learned a lot, and even just the past few days of thinking about how other Java apps/frameworks work, I've realized how applicable that knowledge really is.

CF Groovy Redux

I just got through significantly revamping CF Groovy so that it's CFC based, instead of all in the custom tags.  The custom tags remain, but now you can create and manage a CF Groovy runtime instance manually, rather than letting the tags do it for you.  This will greatly assist in production performance as the tag-based version recreated everything each request.  It's also a prerequisite for Hibernate support, as Hibernate spinup is non-trivial and needs to be avoided unless needed, even in development.

These changes are still in the Hibernate branch in Subversion, not the main trunk.  I don't plan on moving the new code back to the trunk until the Hibernate integration is complete.

CF Groovy Takes a Nap

After close to two weeks of struggling, I finally managed to deploy pure source to a CFML runtime (Railo, in this case), and get Groovy entities in and out of the database with Hibernate.  No compliation, no IDE, no development-mode server, just my Groovy source along with a hacked up CF Groovy.  This is very exciting for me, because while Groovy is cool and all, CF's dynamic nature has become a must have.

I don't have source to download yet, just a horribly hacky proof of concept, but here's what the user of the "framework" does.  First, create your entity:

package com.barneyb
import javax.persistence.*

@Entity
class Person {

  @Id
  @GeneratedValue
  Long id
  Long version
  String name
  Date dob

  def getAgeInSeconds() {
    (new Date().getTime() - dob.getTime()) / 1000
  }

  def getAgeInDays() {
    ageInSeconds / 86400
  }

  def getAgeInYears() {
    ageInDays / 365.249
  }

  String toString() {
    "$name is $ageInYears years ($ageInDays days) old"
  }

}

That's the same class as my initial CF Groovy demo used, except with the of JPA annotations added.  Because I'm binding to Hibernate, you can use any Hibernate-supported annotations, but I opted to stick with vanilla JPA.  Now we need our hibernate.cfg.xml, which is totally stock:

<?xml version='1.0' encoding='utf-8'?>
<!DOCTYPE hibernate-configuration PUBLIC
  "-//Hibernate/Hibernate Configuration DTD//EN"
  "http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
<hibernate-configuration>

  <session-factory>

    <property name="current_session_context_class">thread</property>
    <property name="cache.provider_class">org.hibernate.cache.NoCacheProvider</property>
    <property name="show_sql">true</property>
    <property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>
    <property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
    <property name="hibernate.connection.url">jdbc:mysql://localhost:3306/dbname</property>
    <property name="hibernate.connection.username">username</property>
    <property name="hibernate.connection.password">password</property>
    <property name="hibernate.hbm2ddl.auto">update</property>

    <mapping class="com.barneyb.Person" />

  </session-factory>

</hibernate-configuration>

You could also map your classes manually with XML, if you wanted, but for the sake of simplicitly, I've done it all with annotations.  You'll also notice the database connection information embedded directly in the file. This is far from ideal.  I'm hoping to extract it from a CFML DSN, but that might only be possible on Adobe ColdFusion (via the Admin API).  The last thing is the test script:

<g:script>
  import java.text.SimpleDateFormat

  import org.hibernate.*
  import org.hibernate.cfg.*

  import com.barneyb.Person

  sdf = new java.text.SimpleDateFormat("yyy/MM/dd")
  sdf.lenient = true

  def sf = new AnnotationConfiguration()
      .configure().buildSessionFactory()
  def sess = sf.openSession()
  def tx  = sess.beginTransaction()
  sess.createQuery("delete Person").executeUpdate()
  sess.save(new Person(
    name: attributes.name ?: "Barney",
    dob: sdf.parse(attributes.dob ?: "1980/06/10")
  ))
  tx.commit()
  sess.close()
  sf.close()
</g:script>

No need to compile anything, no need to drop JARs all over your server, nothing.  Write, refresh, persist.  That includes adding new entities, new properties to existing entities, etc.

It's not ready for public consumption at the moment, but that's my top "free time" priority.  For the incredibly impatient, there's a branch in my SVN repository with the current source.  Don't expect anything pretty.  ;)

Railo 3 Beta 2

Railo dropped a second beta last night. Get it here. They've also added a WAR download, though it only works when installed into an existing "express" installation. Hopefully that'll get addressed for the next release.

A couple shining points about the runtime:

  • Array and Struct literals (the [...] and {…} notation) are expressions, exactly as they should be, unlike other CFML runtimes (cough…Coldfusion…cough).
  • Arrays and Structs now implement java.util.List and java.util.Map respectively, greatly easing Java integration. This also allows the Comparators CF Groovy example I published to work correctly on Railo.

Closures for Java

Wanting some light reading for this evening, I decided to dig into the Closures for Java draft spec (homepage).

Oh. My. God.

I'm a huge fan of closures. They're elegant, simple, and easy to use. Their semantics are subtle, but not confusing. They allow for very concise implementation of otherwise complicated algorithms, trimming a vast number of "normal" control structures, particularly various looping constructs.

Java already has (anonymous) inner classes which provide similar semantics, though with a marginally clunkier interface. This mechanism is basically how closures in Groovy are implemented, with some compiler magic to do the heavy lifting.

If you go look through the Closures for Java spec, however, what you see is appalling. Closures are about simplicity and elegance, and the spec outlines exactly the opposite. Closures themselves are exactly what you'd expect, using the Groovy (or Ruby) syntax, though types are required for the arguments. But what you don't expect is the function type syntax, the closure conversion rules, generics support, etc. It's a mess. Check this example:

String name;
{int => long} transform;

The first line declares a variable of type "java.lang.String". The second declares a variable of type "function that takes one int argument and returns a long". Note that that is different from a variable declared as "{int => int}", of course. Even better, any closure that can be assigned to the 'transform' variable implicitly implements every single-method interface whose method accepts an int and returns a long. It's unclear to me whether the converse is true – I couldn't take any more dry spec-speak.

The spec is pretty short (less than nine pages), and if you do any Java, it's definitely worth perusing. After reading, you'll realize that anonymous inner classes are just fine, thank you very much, even with the requirement of having to declare the interfaces explicitly. With the amount of baggage they seem to be lugging into Java (mostly because of it's typing), all I can say is "yuck." Closures require a bit more dynamic an environment to be effective, I think, and Java is static. Totally static.

ActionScript 3 provides an interesting middle ground that provides the benefits but culls a lot of the garbage. In AS3 you declare functions with typed arguments and an explicit return type. However, every function is of type function, which means if I have an algorithm that requires an {T, T => int} closure (that's the Comparator interface, by the way), I can happily pass in any old function I want and the compiler won't care. I'll get an error at runtime, of course.

Much the dancing the spec details is to protect developers from this potential situation, and the rest is to deal with backwards compatibility an interoperability with older libraries. I understand that Java is a strongly typed language, and that the compiler is omnipotent, but c'mon. Backwards compatibility is also very useful, but again, to what extent. You can already thrash a generics-based library with reflection, because it's absolved from compiler checks. It seems like there should be a point where you recognize the fact that the language has very deliberately poked holes in it's own security structures, and realize that that means trying to create airtight compiler checking is pointless.

I guess that's why Groovy got created, though. Backwards compatibility is important for Groovy as well, but the compiler (javac, mind you, not groovyc) and all it's type checking wonder has basically been given the finger since everything is just an Object and duck-typed as needed. Quack! I'm a developer that likes transparency in my compiler. Compiler writers are supposed to be smart, they can figure out what I'm doing and tell me if it'll work or not. I shouldn't have to give them explicit instructions, at least not that explicit.

NB: Function pointers (a la CFML) are NOT closures. Closures bind to their definition scope, function pointers just point at a function with no bound scope. I've heard the argument that function pointers are equivalent/sufficient. They're not.