Wednesday, December 17, 2008

Impala 1.0M4 released

I am pleased to announce the release of Impala 1.0M4, with preliminary support for OSGi.

Impala 1.0M4 now includes the ability to arrange Impala modules and their corresponding class loaders in a graph, instead of in a hierarchy as in previous releases. This feature offers application developers much more flexibility in choosing the appropriate module structures for applications.

1.0M4 is the first release of the project to include OSGi support. All Impala jars are now OSGi-compliant bundles. It is now possible to run Impala and its application modules in an OSGi container (currently Eclipse Equinox). Further improvements to Impala's support for OSGi are expected in subsequent releases.

At this stage Impala support for OSGi is still experimental. It is not recommended at this stage to use it in a production environment.

Impala promises to become the first and only project to offer a seamless transition to (and from) OSGi. There is no intention to make OSGi the default deployment model for Impala. However, OSGi is likely to become a good choice in the future for projects which will benefit from more rigorous management of third-party libraries than is possible using the traditional model.

More information on Impala's position with respect to OSGi can be found in this blog entry: http://impalablog.blogspot.com/2008/12/announcing-osgi-support-in-impala.html

For more information on this release see: http://code.google.com/p/impala/wiki/Release1_0M4Announcement

Phil Zoio

Impala Home: http://impala.googlecode.com

Friday, December 5, 2008

Announcing OSGi support in Impala

The soon to be released version of Impala will contain OSGi support. Does this mean that I've abandoned my mission to make Impala the simplest, most lightweight, test-friendly Java dynamic module framework. Absolutely not. The idea is simply to translate the benefits of Impala into an OSGi environment.

In the future, you will no longer need to choose between OSGi and Impala. You can have the best of both worlds. As long as you have some appreciation for the value and benefits of dynamic modules in Java, then Impala can work for you, no matter what your stance on OSGi is.

If you have not looked at OSGi and have no intention of doing so, then you're already well covered by Impala. If you are likely to look at OSGi in the future, then you can take advantage now of a framework which already does most of what you'd want, and move towards OSGi when you're ready. If you're already an OSGi enthusiast, I'd encourage you to take a look now at the Impala OSGi sample. Just point your Eclipse SVN repository explorer to http://impala.googlecode.com/svn/trunk/osgi-sample, check out the contained projects, and run any of the unit tests. You'll see something like the following, for MessageServiceTest.





















Note that the unit test is actually run on the Equinox OSGi container using the Spring Dynamic Modules test infrastructure. If this seems very easy, it's because it is - you won't even need to run any build scripts to get the example working!

If you're familiar with Impala you will notice that your OSGi-based application is configured in an identical manner to regular Impala applications, allowing a seamless transition from a plain Impala to an OSGi-based runtime (and back again). Impala's abstraction for representing relationships between modules provides a convenient mechanism for expressing and reloading module subgraphs, something you need to do continually during development.

Impala's OSGi support leverages much of the ground work covered by the Spring DM project. Impala also automatically generates OSGi compliant jars using Peter Kriens' BND tool when using Impala's build support, and provides a convenience target to generate an OSGi manifest so that you don't need to continually create a new set of OSGi jars each time you modify your application.

So is Impala's OSGi support production ready? Not at this point. Apart from the lack of any serious road testing, the big blocker is there is no explicit web support (yet). Additionally, Impala applications should have the option to use OSGi's service registry. I'd also like to better test framework integration so that Impala's interactive test runner and integration test suites can work equally well in an OSGi environment.

There are still some interesting technical challenges to overcome, and plenty of work to do to bring this project to its full potential. So if there is anyone out there with an interest in dynamic modules, Spring, OSGi , etc. and would like to help out, I'd love to hear from you.

So why add OSGi support to Impala?

The sweet spots in Impala are the ease in writing and running integration tests, ability to reload application modules, great web support, and a really simple, no-nonsense development environment. The sweet spots with OSGi is the way in which it gives you control over loading of third party libraries. OSGi allows you to dynamically reload third party libraries as well as concurrently run different versions of the same third party library classes, two things that you cannot do with Impala on its own.

I might stick my neck out a bit by saying that the features that I described above that are available in Impala on its own are much more valuable for the typical Java application than the extra features provided by OSGi. It is for this reason that Impala is built from the ground up to work in the simplest, most developer-friendly way possible without any reliance on third party runtime environments, tools or build systems.

That being said, there are clearly valid and potentially important use cases for the unique OSGi features. This, together with the growing industry momentum behind the technology, make OSGi support pretty much mandatory for any Java dynamic module system which demands to be taken seriously.

As I made clear at a talk in London last week, OSGi is the most powerful and sophisticated Java dynamic module system, particularly in its use of class loaders. But for many projects a simpler technology would suffice. Impala is the only dynamic module framework that offers the flexibility to choose easily between alternatives.

It is worth noting that in the upcoming version 1.0M4, Impala introduces support for expressing module dependencies via a graph rather than simply as a hierarchy. This allows individual modules to reuse functionality from multiple dependencies, rather than a single parent. This feature is particularly important both because it narrows the gap between Impala's built-in module support and that provided by OSGi, allowing for a more seamless and less constrained transition between the technologies.

Monday, October 13, 2008

Impala 1.0 M3 released with major web enhancements

I am pleased to announce the release of Impala 1.0M3, with major web enhancements.

Impala is a dynamic module framework for Java enterprise application development, based on the Spring framework. With a focus on simplicity and productivity, Impala radically transforms application development using Spring and related technologies.

The Impala 1.0M3 release includes a range of improvements aimed at more robust and powerful support for web applications consisting of multiple dynamic modules. This release provides the underlying capabilities required for building dynamically reloadable web applications using frameworks other than Spring MVC, such as Struts, Wicket, Tapestry and others.

Impala 1.0M3 also includes a dynamic properties framework, as well as various other feature improvements and bug fixes. A more detailed release announcement is here: http://code.google.com/p/impala/wiki/Release1_0M3Announcement

With Impala, you can take your Spring application development to the next level without having to grapple with unfamiliar technologies or tool sets. It requires no additional third party libraries beyond those required for vanilla Spring applications. It works within existing Java deployment environments, and requires no complex runtime infrastructure or tool support.

Impala Home: http://impala.googlecode.com

Friday, October 3, 2008

Using the Thread's context class loader in a multi-module environment

When you working in a multi-module environment, a problem you bump up against is how to handle the thread's context class loader.

In case you haven't seen this before, you can access the thread's context class loader using the code

Thread.currentThread().getContextClassLoader();

and you can set it using

Thread.currentThread().setContextClassLoader(classLoader);

Why it exists

So why does the context class loader exist, and how is it used? The thread context class loader was introduced in Java 1.2 to enable frameworks running in application servers to access the "right" application class loader for loading application classes.

Lets take a framework like Struts, which uses the thread's context class loader (as do many other similar frameworks). Struts allow you to declare action classes in an XML configuration file. Once the framework has read the file and figured out what class to load (at this point only identified name), the framework needs a classloader to load the class.

One naive approach to this problem would be to use the code

this.getClass().getClassLoader()

This would return the the class loader responsible for loading the framework class. There is an obvious limitation here. The framework jar would need to be packaged in a location visible only to the application class loader, and not to the system class loader or to some shared or server class loader. Again, coming back to the example of Struts, say running in Tomcat, the framework jar would need to be placed in WEB-INF/lib and not on the system class path or in a server class path such as TOMCAT_HOME/common/lib. This restriction might hold in many cases, but is not one which can be relied upon.

To solve this problem, the context class loader was invented, to give framework code a mechanism to find the "correct" class loader to load application classes. In the case of the web application, the server typically applies the web application class loader as the context class loader.

So what's the problem?

This mechanism has worked well enough until now, but is a bit of a hack, something which is certainly exposed by multi-module frameworks such as OSGi.

There is an interesting article on the Equinox wiki page on this problem and how it addresses. To save some typing (and a bit of thinking), I've picked out a quote:

Many existing java libraries are designed to run inside a container (J2EE container, Applet container etc). Such containers explicitly define execution boundaries between the various components running within the container. The container controls the execution boundaries and knows when a boundary is being crossed from one component to the next.

This level of boundary control allows a container to switch the context of a thread when a component boundary is crossed. Typically when a container detects a context switch it will set the context class loader on the thread to a class loader associated with the component which is being entered. When the component is exited then the container will switch the context class loader back to the previous context class loader.

The OSGi Framework specification does not define what the context class loader should be set to and does not define when it should be switched. Part of the problem is the Framework is not always aware of when a component boundary is crossed.


Some Solutions

Eventually, the OSGi specification will presumably come up with a standard way of addressing this problem. Until then, it is up to vendors to implement their own solutions. Equinox does so through Buddy Class Loader and Context Finder mechanisms, which you can also read about here.

Rob Harrop describes his solution here for the Spring Application Platform:
Each bundle in OSGi has it's own ClassLoader, so therefore, only one bundle can be exposed as the thread context ClassLoader at any time. This means that if a third-party library needs to see types that are distributed across multiple bundles, it isn't going to work as expected.

The Platform fixes this by creating a ClassLoader that imports all the exported packages of every module in your application. This ClassLoader is then exposed as the thread context ClassLoader, enabling third-party libraries to see all the exported types in your application.


So this will work as long as you export the types that the third party libraries needs to load.

Impala's Solution

Impala uses proxies to call across module boundaries. The proxy interceptor has access to a service reference for the target object being invoked. This service reference, as well as containing a reference to the target instance, also has a reference to the class loader responsible for loading the module which exported the target bean. Before invoking the instance, it sets the thread context loader to this class loader.

That's quite a mouthful - if code makes more sense that this English then see this souce.

Additionallly, Impala Servlets - either extensions of the Spring DispatcherServlet or those responsible for integrating with other servlets - perform a similar operation.

There are one or two gotchas. It is possible to call beans defined in parent modules without going through a proxy. In this case, the described mechanism will not be applied. I'm pretty confident though that this particular scenario won't cause any serious problems.

I'd certainly be interested in finding out over the course of time whether there are other strategies that Impala's solution won't cover.

Monday, September 22, 2008

Is the case for forking Spring building?

SpringSource recently announced a new maintenance policy which I, like many in the developer community, find quite disturbing. The basic idea is that SpringSource will only publish releases to non-paying customers of SpringSource for three months after the initial release. After that, it will be up to the rest of the community to do its own releases, with community releases only coming out every so often.

Reactions to this on The Server Side and other places have been understandably very negative. Personally, I have been less than enthusiastic about some of the developments that have been coming from SpringSource since it got VC funding. See a previous entry.

This time, it has gone to far. By biting the hand that feeds it, it has risked alienating its most loyal users and putting many people off the project altogether. It also really undermines the increasing number of other community projects which are built around Spring, projects like Impala.

As the author of a number of open source projects which despite obvious (to me) technical merits never attracted a very large number of users, I know only too well that users are by far the most precious feature of an open source project, to be valued above all else.

SpringSource has not only begun to ignore it's users, it has shown the willingness to outrage them. This shows an unbelievable arrogance and complacency.

All of this is quite apart from reservations I increasingly have about the direction that SpringSource is taking Spring.
  • The project is no longer open
    if you look at the history of SVN commits in Spring, you will notice that the project has become much less open. Around early 2005 there were plenty of commits going into Spring from individuals who are not SpringSource employees. Now commits to Spring core (the part that most people use) are only being made by SpringSource employees.

    Also, strategically important projects are increasingly developed outside of the public domain - including Spring 3.0, which will be the next major release of the core framework.

  • We should be nervous about future licencing moves
    So far the official line is that none of the none of the licences will be changed for any existing project. It would probably impossible to do this anyway. However, many new projects from SpringSource are no longer coming out with friendly open source licences (specifically ASF 2). How do we know, for example, that Spring 3.0 won't be released as a "new" project with a less friendly licence.

  • Obsession with OSGi
    SpringSource has become obsessed with dragging the entire Java community into using OSGi. Don't get me wrong, I do like OSGi, and hope to support it with Impala, and at some point I am sure that I will be using it in real applications. But it shouldn't be at the exclusion of other things which could be done to make Spring more usable without OSGi. It's exactly for this reason that I started Impala. I wanted modules, reloadability, isolation, etc. but not necessarily with the baggage of OSGi at this point.

  • Key man dependencies
    Every project has it's major contributors, the individuals who do most of the heavy lifting while others go around doing the talking. Spring is no different - as the commit graph shows, this is Juergen Hoeller, and it is very much down to his excellence that Spring is so well regarded. If Juergen decided to go off to pastures new, Spring core would be in a very bad place. For a project which is relied upon by so many around the world, this is not a good thing.
In short, it's hard to like the way that Spring is going. But it's not too late to change. One of two things need to happen:
  • SpringSource needs to take Spring back to where it came from. This doesn't mean winding up the company and going back to working for free. But it needs to reaffirm and re-demonstrate it's commitment to the spirit of open source - openness, transparency and a desire to serve it's users. This probably means lowering it's financial objectives. There's still plenty of money to be made from consulting, training, and from providing value added products and tools. But not when it contravenes the essence of what made Spring popular in the first place.
  • If SpringSource cannot get it's act together, the project should be forked. Because Spring runs on an ASF licence, this is quite feasible. But there are lots of question to answer. Who would be willing to do this? Would they be the right people? And where would they take the project?

    Forking a project could be a dangerous move for the community, but history shows that it won't necessarily fail. Perhaps it's better for the project to take the pain now than to have death by a thousand cuts.
Judging by the strength of ill-feeling regarding SpringSource's latest announcement, I'm sure there must be other people in the wider Spring community thinking the same thing. I wonder if there are people working within SpringSource who feel the same way, or is the gravy train now just moving too fast for them to hop off?

Friday, September 12, 2008

Generic support for dynamically reloadable web frameworks

One little feature that I think we're close to pulling off with Impala is the ability to embed any Servlet-based web application and have it dynamically reloadable as a module. I'm thinking of the likes of Tapestry, Struts, etc.

Up to version 1.0M2, this capability has only really existed for Spring MVC. However, to make Impala really attractive to users of other frameworks, you'd want to support these as well. After all, not every Spring user is also using Spring MVC as the web framework.

More recent changes now allow you to embed any Servlet into a the application context XML belonging to an Impala web module, using code such as:

<bean id = "delegateServlet" class="org.impalaframework.web.integration.ServletFactoryBean">
<property name = "servletName" value = "delegateServlet"/>
<property name = "servletClass" value = "servlet.SomeFrameworkServlet"/>
<property name = "initParameters">
<map>
<entry key="controllerClassName" value = "servlet.ServletControllerDelegate"/>
</map>
</property>
</bean>

Note how the init parameters, the servlet name and the servlet class have been specified, as they would be for entries in web.xml.

The InternalFrameworkIntegrationServlet, whose definition is shown below, is what allows the servlet to "live" within an Impala module. It contains the glue code
that ties an invocation from outside of a module (within the context of a servlet container) to the servlet within the module.

<bean class="org.impalaframework.web.integration.InternalFrameworkIntegrationServletFactoryBean">
<property name = "servletName" value = "myservlet"/>
<property name = "servletClass"
value = "org.impalaframework.web.integration.InternalFrameworkIntegrationServlet"/>
<property name = "delegateServlet" ref = "delegateServlet"/>
</bean>

Finally, to communicate with the module itself, there is the ModuleRedirectingServlet, which simply delegates to the a servlet which is registered in the ServletContext using a name which corresponds with the name of the module (and happens to correspond with the first part of the URL's servlet path).

Here's an example configuration in web.xml:

<servlet>
<servlet-name>module-redirector</servlet-name>
<servlet-class>org.impalaframework.web.integration.ModuleRedirectingServlet</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>module-redirector</servlet-name>
<url-pattern>*.htm</url-pattern>
</servlet-mapping>

Together, these offer the capability dynamically embedding applications using arbitrary web frameworks. The idea is that you can register and load a Struts, Webwork, Tapestry or other module after the JVM has started, without impacting any existing code or without having to restart the JVM or reload the entire web application within the application server. A powerful concept.

There are definitely some gotchas to take care of. For one, it relies on object instances being loaded using the classloader obtained via Thread.currentThread().getContextClassLoader(). Also, how can we make sure that objects saved to sessions don't cause ClassCastExceptions when the module is reloaded? Also, how can we make sure that the same result does not occur because of one module attempting to access objects saved to the session by another module? And of course, we'd need to prove it working with real applications based on these frameworks, not example code assuming noddy pseudo frameworks. These kinds of problems will need to be taken care of before the problem can be considered to be fully solved. It should be fun figuring it out, though.

Impala talk at Spring User Group UK

After quite a few weeks of being very busy at work, followed by a few weeks of taking it pretty easy and enjoying the summer, I'm now back to work, and trying to ramp things up a bit with Impala. On Wednesday night I did a talk at the Spring User Group UK on Impala. You can view the talk here - it's a bit grainy, but you can still hear everything. You can view the slides here.

I was given half an hour for the presentation, and I managed to squeeze it into 27 minutes, including a demo. I was pretty happy with the talk - my little example demonstrating creating a new Hibernate entity within a Spring application went without a hitch.

Hope to do a few more talks in the coming months.