Dealing with library version clashes in Impala

One of the problems that occasionally crops up in Java application development is library class version clashes. For some reason, you have to use a particular version of some library, but that library, or some of dependencies, clashes with other libraries that you are using for different parts of the application. I encountered an example of this occurring on our project, where we were forced to use the old Axis 1.4 library to connect with an integration partner. (For those who are interested, the reason is that the our partner is using the rpc/encoded SOAP API configuration, which is generally regarded as obsolete, and is not supported by any of the modern SOAP libraries, from CXF to Metro, as well as the Axis successor, Axis 2. See here and here for more on this problem.)

The ability to deal with these kinds of problems is often cited as one of the main reasons for using OSGi. In my development of Impala, I have steered clear from tackling these kinds of problems, on the basis that while they are theoretically quite valid to consider, in practice they occur quite rarely, and there generally is a workaround. Unfortunately, in my recent project, the workarounds seemed quite unpalatable. One would have been to resort a plain text based client for this SOAP API. For a small API, this would have been satisfactory, but for a large API it would have involved quite a bit of extra work. Ultimately, it seemed like a sensible point to add to Impala some capability for dealing with library version clashes.

The new mechanism simply involves the following:

• add any module-specific library in the module's lib directory. For our project, this involved adding axis-1.4.jar and a couple of its dependencies
• add the setting supports.module.libraries=true to impala.properties

This feature is currently in the code base and will be available in the next release, 1.0.2, which should be available soon.

How does it work?

In the normal Impala application, a class is loaded first by using the web application or system class loader (typically to load third party libraries), then using a topologically sorted list of the dependent module class loaders (typically to find application classes).

With module-specific libraries, the scheme is a bit more complex. For any module which has module-specific libraries, the module-specific library locations are searched first. Suppose we are loading module A, and we need to load the class Foo is in both the shared library area (WEB-INF/lib) and in a module-specific location for module A. In this case, the class Foo will be found in and loaded from the module-specific location.

Caveats

The main caveat is that the class Foo should never be 'published' from module A, for example as the return type in a method which might be called from another module, otherwise ClassCastExceptions or LinkageErrors can result. Instead, the interfaces to module A should ensure that appropriate abstractions are in place to ensure that this does not occur. In our Axis example, we need to ensure that instances of classes which hold references to Axis libraries are not passed around between modules.

More details on how the class versioning feature works, and how it can be configured, will be added to the documentation wiki.

The solution is by no means the solution to any kind of class versioning requirement which may crop up. Multiple versions for particular classes is still seen as the exception rather than the rule, and the scope as well as the complexity of the solution reflects this. That being said, there is at least a workable practical solution when you are stuck in a corner on this problem.

Impala 1.0.1 released

I am pleased to announce the release of Impala 1.0.1. This release is basically a bug fix release. For more details on the bugs fixed, see this list.

While the pace of Impala's development has slowed in recent months, it is still in active development. I am still using Impala every day in my day job. If provides the foundation for the next generation e-commerce fulfillment software we are developing at Realtime Despatch. The reason I have been spending less time developing Impala is not because I have less use for it. On the contrary, it has really proved it's worth in the last year, making what could otherwise be a monster project quite comfortably manageable. The modularity and productivity features provided by Impala are not only fundamental for rapid development of our software, they also important in supporting our business model.

That being said, there are a few features that are backing up which I am hoping to develop in the next few months, and hope to pick up the pace again.

Impala Code Jam in London in London on May 26th

Next Wednesday I will be holding an Impala Code Jam at Skills Matter in London, organised through the Java Web User's Group (JavaWUG). For the people who come along, this will be a hands-on opportunity to spend some time working on a simple Impala application and to get a better understanding of the concepts involved, and for most, to get a first hand flavour of how Impala can help in writing better, more flexible Spring based applications.

More details on the Code Jam are available from the JavaWUG announcement.

Some further comments on the 1.0 release

I'd like to make some comments about the timing of the 1.0 final release of Impala. As I've mentioned elsewhere, the aim of Impala is to extract every last ounce of productivity and flexibility out of Spring-based development, while at the same time remaining true to the principles that made Spring popular in the first place - simplicity and testability. I believe that with this release I can confidently claim that these aims are being achieved, and that Impala provides one of the most productive and flexible environments for building Spring-based applications.

Impala has been used in real world projects for almost three years. Over this period it has undergone extensive refactoring to ensure that the architecture is right - flexible, maintainable, and open to the kinds of extensions which are likely to be introduced in the coming months and years. After over 5500 subversion commits, I am pleased to say that I am comfortable that this goal has been achieved.

While Impala has allowed me to achieve unprecedented levels of productivity in projects in which I have used it, I have also been mindful that more widespread adoption requires adding support to commonly requested features, even ones not necessary in my own projects. Much of the effort put into the project in recent months supports these objectives. Impala now has decent support for Maven, and also allows for fully modularised web applications, including their constituent classes, JSPs and resources.

This work has gone a long way to extending the capabilities of Impala and offering a greater choice in the technology combinations that will work out of the box with Impala. This work is continuing. Nevertheless, I am now comfortable that the feature set in Impala is suitable for a 1.0 release.

I have really enjoyed developing Impala and look back on the work with a lot of pride. It has been a fun project, full of technical challenges. However, there is still plenty of work to do, and plenty of exciting areas to explore. To this end I would very much welcome more direct involvement from the community in helping the project to achieve it's potential. This involvement can take any form you might imagine, from testing the framework in different environments, to building a web site for the project, to adding tool support, through to working on new samples and even new feature set.

Impala 1.0 final released

I am pleased to announce the 1.0 final release of Impala.

Impala is a dynamic, modular productivity framework built around Spring. The aim of Impala is to extract every last ounce of productivity and flexibility out of your Spring development environment, while at the same time remaining true to the principles that made Spring popular in the first place - simplicity and testability.

Impala offers instantaneous redeployment of parts of your application, dramatically improving build/deploy/test cycle times. Impala supports on the fly 'drop-in' of new application functionality - including web modules - without requiring an application restart and without requiring any changes to existing application code or configuration.

With Impala you can achieve the productivity benefits of a modular approach to application development. When using Impala, you no longer need to work with application contexts which contain hundreds of bean definitions. Instead, you break your application down into smaller, manageable chunks. You no longer need to put up with the ever-increasing complexity and tangled interdependencies which come with a monolithic approach to application development. Instead, you develop your application in a loosely coupled way, with a clean distinction between application's interfaces and implementation components.

With Impala it is much easier to mix and match features for particular environments, which is particular useful for applications which may need to be deployed with different features and configurations on a per-environment basis. Writing integration tests with Impala is a doddle, because setting these up is simply a matter of selecting the modules you want to include in your tests.

Impala works with all the technologies you would expect in the Spring world - Hibernate, Quartz, JMX, etc. - without requiring any special plugins to be written for those technologies. It also supports - in most cases without modification - web applications using a variety of web frameworks, from Struts to JSF to Tapestry and others. A lot of work has been put into making Impala elegantly support modular web applications, allowing for modular vertical web application 'slices', rather than just a modular back end.

Impala works straight out of the box using a plain Eclipse installation (no fancy plugins required). It has a built-in build system which you can optionally use, based on Ant, and also integrates nicely with Maven.

With Impala, you can take your Spring-based application development to a new level of ease and sophistication, with remarkably few changes to the way you write applications.

See the full list of issues covered in this release.

With Impala 1.0 released artifacts are also available in the Maven central repository. See http://repo1.maven.org/maven2/org/impalaframework/.

If you like Impala and would like to support the project, please take a look at this page: http://code.google.com/p/impala/wiki/GetInvolved.

Impala 1.0 RC4 released

I am pleased to announce the release of Impala 1.0 RC4.

Impala 1.0 RC4 was originally supposed to be the 1.0 final release. However, due to a few more than expected changes in this release, it was decided to have one last release candidate prior to 1.0 final.
The main changes in this release are:

• simplifications and enhancements in the mechanism for mapping web requests to modules, allowing for more intuitive arrangements of JSPs and resources within modules.
• modification of the starter application to use the multi-web module mechanism out of the box, as this is the recommended approach for building Impala web applications.
• the addition of Javadocs to the build artifacts (this is now required for artifacts deployed to the Maven central repository via Sonatype's infrastructure).

For more information on this release see: http://code.google.com/p/impala/wiki/Release1_0RC4Announcement.

Phil Zoio

Impala Home: http://impala.googlecode.com

Seven ways in which Impala makes Spring development more productive

Impala is a modular, dynamic productivity framework. The principal aim of Impala is to help developers extract every ounce of productivity out of the Spring development environment, while remaining true to the original Spring premise of simplicity and testability. The result is a much more productive Spring application development than you might think possible.

Here's how Impala lets you be more productive when working with Spring.

1. Virtually instantaneous build/deploy/test cycles

For typical Java applications, redeploying a change requires redeploying the whole application. Not with Impala. Only the affected parts of the application need to be deployed. This leads to virtually instantaneous build/deploy/test cycles, as the granularity of redeployment is typically very small relative to that of the application as a whole. You don't need to sit around for ages waiting for your whole application to load.

2. Automatic modification detection and redeployment

When you make a change to an Impala web application running in Eclipse, Impala can automatically pick up which parts of the application have changed, and redeploy those parts of your application without any user intervention. This removes the need for a manual step to perform this task. Impala gives you control over what resources are monitored for changes, how frequently resources are checked for modification, etc.

3. No explicit build step required

When writing typical Java web applications using Eclipse, in many projects there is an extra build step required to package your applications in order to deploy them. Not with Impala. Eclipse automatically compiles your classes for you incrementally in the background. The project structure conventions for Impala mean that this sufficient, and that no extra build steps are required during development.

Of course, you can do a regular WAR build when required for packaging for a production environment.

4. You can write your integration tests interactively

Writing Spring integration tests can be slow, particularly for monolithic single module applications. Impala allows you to write integration tests interactively. You can write and run integration tests using an interactive console within Eclipse. If making changes to your application between test runs, you simply redeploy the changed modules. If you're simply changing your test, you don't need to do anything special as changes to your test class will automatically be picked up on the fly. All of this leads to a much more rapid build/deploy/test cycle when writing integration tests than would be possible otherwise.

5. Suites including integration tests run super-efficiently

When running a suite of integration tests, modules needed by successive tests are loaded incrementally and remain available for subsequent tests run as part of the same suite. There is no need for continually starting and stop Spring application contexts, which can really slow down the execution of integration tests.

All of this makes test suite execution really fast, making Impala very friendly to use in environments where developers are expected (as they should be) to run the full project test suite before each commit to the version control system.

6. Setup is very simple

For a developer getting started on an existing project using Impala, it is really as simple as "check out and go" - simply check out the project files and starting writing and deploying the application within the IDE.

If you're starting a new project, this too is easy, as Impala provides a script which allows you to set up a simple, working application by executing a couple of commands.

7. Easy to use build and dependency management support if you need it

Impala works with Maven, but if you're not a fan of Maven, Impala also features a reusable build system which saves you having to write create your own build from scratch. Impala also gives you a simple way to pull third party library jars and source from Maven repositories, even if you aren't using Maven as your build system.