Plans for project migration on GitHub

Just to let you know what is happening with this project since the Google Code since the hosting repository shutdown was announced.

Impala is was previously hosted on Google Code, which went into only mode in August and was due to be shut down in January 2016.

We have recently migrated the project to GitHub. The most recent fork on which future changes will be made is https://github.com/realtimedespatch/impala until further notice are.

The project has not seen a lot of active development, but it is a core part of the software infrastructure at Realtime Despatch, where I am a part owner and have been Technical Director since 2009.

It has been fantastically effective at Realtime Despatch in creating a development and deployment platform which both aligns with our commercial objectives and suits our agile approach to software development.

We have no plans to 'market' or promote the project to a wider audience in a way that will increase the scope of the software we need to develop and support within the project. However, we will potentially add features as required to meet our own business needs.

The main focus of changes made in the GitHub fork are as follows:

• bringing the code and examples up to date with the latest version of Spring (currently 4.2.4).
• setting the compile compatibility flag to Java 1.6.
• stripping out examples for older frameworks.

Going a bit further forward, my plans for the project are:

• a project rename. There are too many other projects called 'Impala' (although there weren't when I chose the name originally). I need to decide how far to take this in terms of things like renaming files, and core packages, for example.
• further slimming down the project by stripping out features that we no longer intend to support. However, no changes to the structure of core interfaces are planned.
• porting and refining the documentation, focusing on more simple documentation based on the mode of operation that we are currently using at Realtime Despatch.

Impala 1.0.3 released

I am pleased to announce the 1.0.3 release of Impala. Partly a maintenance release to bring Impala fully up to date to latest versions of the Spring Framework.

You can download Impala 1.0.3 from the project site. Modules have also been published to the Maven central repository.

While the core of Impala is very stable, the new release contains a few useful new features to help users of the project in development and in the field. I plan to add separate posts regarding a couple of these features. Below is a quick overview.

Selectively reloadable modules

With Impala 1.0.3 it is now possible to turn on module reloadability selectively.
In other words, some but not all of the modules in an application can be set to reloadable.

The advantage of this feature is that it opens up the possibility of supporting safer reloadability of certain modules in a production environment. Modules that may require reloading while an application is running can be set to reloadable, while core modules that should remain static can be set to non-reloadable. This allows for partial update of selective elements of a running application without disruption to users.

File system module resources

Impala web application resources are typically packaged for production in module-specific jars. On occasions, a requirement exists to change resources (such as web templates) packaged as part of the application without doing a full software build and release. Prior to 1.0.3, replacing any module-specific resource would require the to be rebuilt, which is a little awkward.

With Impala 1.0.3, the resource loading mechanism has been modified to first look in a module-specific file system location, if such a location is present. This supports a simpler mechanism for updating or 'hot fixing' resources in an application, without the overhead of a new software release.

Spring 3.2.2 support

Impala has been updated to work out of the box with the latest version of Spring, currently 3.2.2. Impala is now also compiled using Spring 3.2.

Java Generics compliance

Changes have been made in the Impala source code to support Java 5 generic compliance where this support wasn't present.

Comment syntax

With 1.0.3, the Impala module definition file (in which modules to be deployed are listed) supports a comment syntax, making it easier to comment against entries in this file.

Planned updates to Impala in the near term

I am planning to do some overdue work on Impala in the short term to bring it up to date. The next release I am planning to do will be version 1.1. It will essentially be a maintenance release.

Apart from a couple of minor features still to be determined, the main focus will be on getting Impala working nicely with Spring 3.1 and Spring 3.2.

Right now, Impala won't run Spring 3.1 because of changes to the ApplicationContext interface, which introduced a getEnvironment() method. The changes to Impala to support this are very small, and will be made in this release. I also plan to use Spring 3.2 as the compile version.

I also intend to introduce changes to make it easier to add module-specific resources to a running application, which can be useful in providing on-the-fly template fixes in production.

Apart from this, I also intend to slim down the project significantly:

• removing features that are no longer necessary, have been superseded, or are no longer supported.
• removing old examples projects that I no longer intend to maintain.

I am also planning to all projects relating to OSGi support in Impala. As some will know, I experimented with introducing OSGi support to Impala, with some success. However, I also reached the conclusion it would take a great deal of work to make OSGi really usable in an Impala environment, work which I was certainly not interested in committing to. I'm quite happy to maintain Impala's status as a simpler alternative to OSGi, rather than a project which is designed to support OSGi directly.

Impala 1.0.2 released

I am pleased to announce the release of Impala 1.0.2. This is primarily a bug fix release, but also contains some minor as well as more significant feature enhancements.

It's downloadable from the Impala downloads page. The jars will also be available shortly on the Maven central repository. 

Optional modules
It is now possible to express dependencies on optional modules. Module B may declare an optional dependency on module A, which means that if A is present, it's classes, resources and beans will be visible to module B. However, if A is not loaded, the application will not fail.

This feature makes it possible to allow optional functionality to be included in an application.

Module-specific library jars
This feature, described an earlier blog entry, allows a module to use third party library classes which are specific to that module. In other words, it allows a module to load it's own version of a particular third party library class, potentially one which is different from or conflicts with the class visible to other modules in the application.

In addition, this release covers a number of other issues. See the list of issues in this release.

Dealing with library version clashes in Impala

One of the problems that occasionally crops up in Java application development is library class version clashes. For some reason, you have to use a particular version of some library, but that library, or some of dependencies, clashes with other libraries that you are using for different parts of the application. I encountered an example of this occurring on our project, where we were forced to use the old Axis 1.4 library to connect with an integration partner. (For those who are interested, the reason is that the our partner is using the rpc/encoded SOAP API configuration, which is generally regarded as obsolete, and is not supported by any of the modern SOAP libraries, from CXF to Metro, as well as the Axis successor, Axis 2. See here and here for more on this problem.)

The ability to deal with these kinds of problems is often cited as one of the main reasons for using OSGi. In my development of Impala, I have steered clear from tackling these kinds of problems, on the basis that while they are theoretically quite valid to consider, in practice they occur quite rarely, and there generally is a workaround. Unfortunately, in my recent project, the workarounds seemed quite unpalatable. One would have been to resort a plain text based client for this SOAP API. For a small API, this would have been satisfactory, but for a large API it would have involved quite a bit of extra work. Ultimately, it seemed like a sensible point to add to Impala some capability for dealing with library version clashes.

The new mechanism simply involves the following:

• add any module-specific library in the module's lib directory. For our project, this involved adding axis-1.4.jar and a couple of its dependencies
• add the setting supports.module.libraries=true to impala.properties

This feature is currently in the code base and will be available in the next release, 1.0.2, which should be available soon.

How does it work?

In the normal Impala application, a class is loaded first by using the web application or system class loader (typically to load third party libraries), then using a topologically sorted list of the dependent module class loaders (typically to find application classes).

With module-specific libraries, the scheme is a bit more complex. For any module which has module-specific libraries, the module-specific library locations are searched first. Suppose we are loading module A, and we need to load the class Foo is in both the shared library area (WEB-INF/lib) and in a module-specific location for module A. In this case, the class Foo will be found in and loaded from the module-specific location.

Caveats

The main caveat is that the class Foo should never be 'published' from module A, for example as the return type in a method which might be called from another module, otherwise ClassCastExceptions or LinkageErrors can result. Instead, the interfaces to module A should ensure that appropriate abstractions are in place to ensure that this does not occur. In our Axis example, we need to ensure that instances of classes which hold references to Axis libraries are not passed around between modules.

More details on how the class versioning feature works, and how it can be configured, will be added to the documentation wiki.

The solution is by no means the solution to any kind of class versioning requirement which may crop up. Multiple versions for particular classes is still seen as the exception rather than the rule, and the scope as well as the complexity of the solution reflects this. That being said, there is at least a workable practical solution when you are stuck in a corner on this problem.

Impala 1.0.1 released

I am pleased to announce the release of Impala 1.0.1. This release is basically a bug fix release. For more details on the bugs fixed, see this list.

While the pace of Impala's development has slowed in recent months, it is still in active development. I am still using Impala every day in my day job. If provides the foundation for the next generation e-commerce fulfillment software we are developing at Realtime Despatch. The reason I have been spending less time developing Impala is not because I have less use for it. On the contrary, it has really proved it's worth in the last year, making what could otherwise be a monster project quite comfortably manageable. The modularity and productivity features provided by Impala are not only fundamental for rapid development of our software, they also important in supporting our business model.

That being said, there are a few features that are backing up which I am hoping to develop in the next few months, and hope to pick up the pace again.

Impala Code Jam in London in London on May 26th

Next Wednesday I will be holding an Impala Code Jam at Skills Matter in London, organised through the Java Web User's Group (JavaWUG). For the people who come along, this will be a hands-on opportunity to spend some time working on a simple Impala application and to get a better understanding of the concepts involved, and for most, to get a first hand flavour of how Impala can help in writing better, more flexible Spring based applications.

More details on the Code Jam are available from the JavaWUG announcement.

Some further comments on the 1.0 release

I'd like to make some comments about the timing of the 1.0 final release of Impala. As I've mentioned elsewhere, the aim of Impala is to extract every last ounce of productivity and flexibility out of Spring-based development, while at the same time remaining true to the principles that made Spring popular in the first place - simplicity and testability. I believe that with this release I can confidently claim that these aims are being achieved, and that Impala provides one of the most productive and flexible environments for building Spring-based applications.

Impala has been used in real world projects for almost three years. Over this period it has undergone extensive refactoring to ensure that the architecture is right - flexible, maintainable, and open to the kinds of extensions which are likely to be introduced in the coming months and years. After over 5500 subversion commits, I am pleased to say that I am comfortable that this goal has been achieved.

While Impala has allowed me to achieve unprecedented levels of productivity in projects in which I have used it, I have also been mindful that more widespread adoption requires adding support to commonly requested features, even ones not necessary in my own projects. Much of the effort put into the project in recent months supports these objectives. Impala now has decent support for Maven, and also allows for fully modularised web applications, including their constituent classes, JSPs and resources.

This work has gone a long way to extending the capabilities of Impala and offering a greater choice in the technology combinations that will work out of the box with Impala. This work is continuing. Nevertheless, I am now comfortable that the feature set in Impala is suitable for a 1.0 release.

I have really enjoyed developing Impala and look back on the work with a lot of pride. It has been a fun project, full of technical challenges. However, there is still plenty of work to do, and plenty of exciting areas to explore. To this end I would very much welcome more direct involvement from the community in helping the project to achieve it's potential. This involvement can take any form you might imagine, from testing the framework in different environments, to building a web site for the project, to adding tool support, through to working on new samples and even new feature set.

Impala 1.0 final released

I am pleased to announce the 1.0 final release of Impala.

Impala is a dynamic, modular productivity framework built around Spring. The aim of Impala is to extract every last ounce of productivity and flexibility out of your Spring development environment, while at the same time remaining true to the principles that made Spring popular in the first place - simplicity and testability.

Impala offers instantaneous redeployment of parts of your application, dramatically improving build/deploy/test cycle times. Impala supports on the fly 'drop-in' of new application functionality - including web modules - without requiring an application restart and without requiring any changes to existing application code or configuration.

With Impala you can achieve the productivity benefits of a modular approach to application development. When using Impala, you no longer need to work with application contexts which contain hundreds of bean definitions. Instead, you break your application down into smaller, manageable chunks. You no longer need to put up with the ever-increasing complexity and tangled interdependencies which come with a monolithic approach to application development. Instead, you develop your application in a loosely coupled way, with a clean distinction between application's interfaces and implementation components.

With Impala it is much easier to mix and match features for particular environments, which is particular useful for applications which may need to be deployed with different features and configurations on a per-environment basis. Writing integration tests with Impala is a doddle, because setting these up is simply a matter of selecting the modules you want to include in your tests.

Impala works with all the technologies you would expect in the Spring world - Hibernate, Quartz, JMX, etc. - without requiring any special plugins to be written for those technologies. It also supports - in most cases without modification - web applications using a variety of web frameworks, from Struts to JSF to Tapestry and others. A lot of work has been put into making Impala elegantly support modular web applications, allowing for modular vertical web application 'slices', rather than just a modular back end.

Impala works straight out of the box using a plain Eclipse installation (no fancy plugins required). It has a built-in build system which you can optionally use, based on Ant, and also integrates nicely with Maven.

With Impala, you can take your Spring-based application development to a new level of ease and sophistication, with remarkably few changes to the way you write applications.

See the full list of issues covered in this release.

With Impala 1.0 released artifacts are also available in the Maven central repository. See http://repo1.maven.org/maven2/org/impalaframework/.

If you like Impala and would like to support the project, please take a look at this page: http://code.google.com/p/impala/wiki/GetInvolved.

Impala 1.0 RC4 released

I am pleased to announce the release of Impala 1.0 RC4.

Impala 1.0 RC4 was originally supposed to be the 1.0 final release. However, due to a few more than expected changes in this release, it was decided to have one last release candidate prior to 1.0 final.
The main changes in this release are:

• simplifications and enhancements in the mechanism for mapping web requests to modules, allowing for more intuitive arrangements of JSPs and resources within modules.
• modification of the starter application to use the multi-web module mechanism out of the box, as this is the recommended approach for building Impala web applications.
• the addition of Javadocs to the build artifacts (this is now required for artifacts deployed to the Maven central repository via Sonatype's infrastructure).

For more information on this release see: http://code.google.com/p/impala/wiki/Release1_0RC4Announcement.

Phil Zoio

Impala Home: http://impala.googlecode.com

Seven ways in which Impala makes Spring development more productive

Impala is a modular, dynamic productivity framework. The principal aim of Impala is to help developers extract every ounce of productivity out of the Spring development environment, while remaining true to the original Spring premise of simplicity and testability. The result is a much more productive Spring application development than you might think possible.

Here's how Impala lets you be more productive when working with Spring.

1. Virtually instantaneous build/deploy/test cycles

For typical Java applications, redeploying a change requires redeploying the whole application. Not with Impala. Only the affected parts of the application need to be deployed. This leads to virtually instantaneous build/deploy/test cycles, as the granularity of redeployment is typically very small relative to that of the application as a whole. You don't need to sit around for ages waiting for your whole application to load.

2. Automatic modification detection and redeployment

When you make a change to an Impala web application running in Eclipse, Impala can automatically pick up which parts of the application have changed, and redeploy those parts of your application without any user intervention. This removes the need for a manual step to perform this task. Impala gives you control over what resources are monitored for changes, how frequently resources are checked for modification, etc.

3. No explicit build step required

When writing typical Java web applications using Eclipse, in many projects there is an extra build step required to package your applications in order to deploy them. Not with Impala. Eclipse automatically compiles your classes for you incrementally in the background. The project structure conventions for Impala mean that this sufficient, and that no extra build steps are required during development.

Of course, you can do a regular WAR build when required for packaging for a production environment.

4. You can write your integration tests interactively

Writing Spring integration tests can be slow, particularly for monolithic single module applications. Impala allows you to write integration tests interactively. You can write and run integration tests using an interactive console within Eclipse. If making changes to your application between test runs, you simply redeploy the changed modules. If you're simply changing your test, you don't need to do anything special as changes to your test class will automatically be picked up on the fly. All of this leads to a much more rapid build/deploy/test cycle when writing integration tests than would be possible otherwise.

5. Suites including integration tests run super-efficiently

When running a suite of integration tests, modules needed by successive tests are loaded incrementally and remain available for subsequent tests run as part of the same suite. There is no need for continually starting and stop Spring application contexts, which can really slow down the execution of integration tests.

All of this makes test suite execution really fast, making Impala very friendly to use in environments where developers are expected (as they should be) to run the full project test suite before each commit to the version control system.

6. Setup is very simple

For a developer getting started on an existing project using Impala, it is really as simple as "check out and go" - simply check out the project files and starting writing and deploying the application within the IDE.

If you're starting a new project, this too is easy, as Impala provides a script which allows you to set up a simple, working application by executing a couple of commands.

7. Easy to use build and dependency management support if you need it

Impala works with Maven, but if you're not a fan of Maven, Impala also features a reusable build system which saves you having to write create your own build from scratch. Impala also gives you a simple way to pull third party library jars and source from Maven repositories, even if you aren't using Maven as your build system.

Impala 1.0 RC3 released

I am pleased to announce the release of Impala 1.0 RC3. Impala 1.0 RC3 is the final release candidate prior to the official 1.0 release. If no significant issues are raised against this released, it is envisaged that there will be virtually no changes in the subsequent 1.0 final release.

This release contains a number of enhancements:

• simplifications making it easier to migrate existing Spring-based projects to use Impala. It is now possible to run Impala as part of an existing virtually unchanged traditional Spring web application, which should significantly simplify the task of migrating an existing Spring application to use Impala.
• support for JSPs deployed within individual web modules. Prior to 1.0 RC3, JSPs need to be contained within the web application folder. With 1.0 RC3 it is now possible to completely modularise the web elements of a JSP-based web application.
• support load time weaving of aspects created using AspectJ.
• various other minor features and bug fixes.

Also, Impala 1.0 RC3 is available in the Maven central repository for the first time.

For more information on this release see: http://code.google.com/p/impala/wiki/Release1_0RC3Announcement.

In the next couple of weeks, I plan to add more documentation on how to get Impala working nicely with Maven, and how to migrate an existing Spring-based project to use Impala.

Please don't hesitate to report any issues you may encounter, which you can do on the Impala issue tracker: http://code.google.com/p/impala/issues/list.

Phil Zoio

http://impala.googlecode.com

OSGi and Enterprise Java dynamic modularity after Spring dm Server

The key message to be taken from SpringSource's decision to hand last month their Spring OSGi projects (Spring DM and dm Server) to the Eclipse foundation is the following: that OSGi is not ready for adoption by the mainstream of enterprise Java. Reality has finally dawned!

According the announcement from Adrian Colyer, CTO of SpringSource:

For a mainstream development team though, who just want to build an enterprise application as quickly as possible, and with as little hassle as possible, the costs currently associated with adopting enterprise OSGi can outweigh the short-term benefits. This situation needs to be addressed before enterprise OSGi can become the de-facto approach for mainstream enterprise application development.

While SpringSource will no doubt continue to support OSGi, the extent to which it will continue to provide resources to the donated projects is far from certain. What is more clear, though, is it will no longer be wielding it's own considerable influence within the industry to shepherd its user base en masse towards OSGi.

This is a good thing. Firstly, the obsession with OSGi has distracted the community away from practical efforts which could work more comfortably within the existing enterprise Java model. Hopefully that obsession will go away, or at least be limited to those already converted to the OSGi way. Secondly, from a purely selfish, egotistical point of view, I feel vindicated. For years, I have been saying that OSGi is too complex for the Java mainstream. For years, I have been beaten over the head by a small group of hardcore OSGi evangelists who have persistently used the argument that if SpringSource is betting on OSGi, then it must be a good thing for everyone else, and that any alternative approach does not warrant any consideration. The bet has failed, and this argument has now fallen rather flat. If SpringSource, with all the credibility it has generated through the Spring Framework, all its intellectual and marketing muscle, and all the effort and expense it has incurred, cannot make work OSGi for mainstream enterprise Java, then no one else can.

But all of this is also points to a real missed opportunity. The fact remains that Spring users still need a modularity solution. They need a way of being able to take advantage of the best of what dynamic modularity has to offer - dynamically composable software with genuine decoupling between parts of the application, and rapid build/deploy/test cycles during development - without having to take on all the complexity of the full-blown OSGi approach. This need still exists, and will not go away. As Adrian suggests, this need will not be served by OSGi without some fundamental simplifications, something I could not imagine happening any time soon.

In the meantime, I have invited SpringSource to engage more with Impala. While they have flirted with the idea in the last few months, perhaps even seriously, they have backed away from it for now. In view of recent history, it is not that surprising. Imagine the uproar that would ensue among the OSGi zealots if SpringSource were to go down that route. It doesn't bear thinking about.

Impala 1.0 RC2 released

I am pleased to announce the release of Impala 1.0 RC2. With this release, support for Maven is introduced for the first time, along with various other enhancements, refactorings and bug fixes, including:

• simplifications to the default project structure, making it more intuitive and Maven-friendly
• a dynamic properties Spring namespace, making it easier to support properties which can be injected and dynamically updated without requiring module or application reloads.
• further refinements to the mechanisms for supporting multi-module web applications.
• initial support for Spring 3.0.

For more information on this release see: http://code.google.com/p/impala/wiki/Release1_0RC2Announcement.

Cheers,
Phil Zoio

Don't let OSGi put you off modularity

If you are interested in the ideas of modularity, there is quite a good chance you have looked at OSGi. But there is also quite a good chance that you have been put off by the extra development overhead required to set up and maintain OSGi-based applications.

If this is the case for you, my message is this: don't let fact that you are put off by OSGi put you off the ideas of modularity. Modularity is a bigger, more important and valuable idea than any particular technology. The ability to modularise an application can massively improve your application's ability to absorb complexity as new features and components are added, which in turn means that you remain much more productive as your application grows. It reduces the rate of entropy in your system, extending its potential life span, which of course if great for business.

This kind of modularity is one of the fundamental benefits of Impala. Also, because Impala's modules are dynamically reloadable, and you also get dynamic redeployment of small parts of your application, allowing you to greatly accelerate application development, maintain high levels of productivity, and retain high levels of code and functionality reuse as you application grows.

Impala does not solve all of the problems solved by OSGi. Notably, it does not provide versioning of third party libraries. However, it solves most of the important ones relevant for day to day development. Fundamentally, it allows you to modularise your application, without having to worry about micro-managing the dependencies of third party libraries in your application. Don't get me wrong - there are times when this capability can be important, especially for very large projects with large budgets and teams. But certainly not for every project, and probably not even for the typical one.

One way to think of choice is to look at the graph below.


The basic idea behind modularity is that the growth in complexity of your application slows as your application grows in size, compared to applications without modules. This applies both for Impala and OSGi-based applications. The difference is that because the barriers to entry are lower for Impala-based applications, the benefits kick in sooner, and accumulate for longer over the duration of the project, greatly reducing the overall cost of complexity over the lifetime of a project.

So, don't shy away from modularity just because OSGi looks complex. The benefits of modularity are too valuable, and an alternative like Impala make these attainable with fewer headaches.

The granularity of change in dynamic Java web applications

When writing Java web applications, you are continually making changes to your application, and to be productive you need to be able to deploy and test these changes quickly. The kinds of changes you make are of all sorts: from changes to resources to changes to markup templates to changes in the way your application is wired to changes in the code itself.

The point of this article is that not all changes are equal, both in their frequency and in the difficulty in applying them dynamically in a running web application. Let's go through some examples, and how the changes might be applied in different types of frameworks:

1. Static resources
The simplest kinds of changes for any dynamic web framework to apply are those of static resources - images, JavaScript files, page templates etc. That's because these kinds of artifacts are inherently free of dependents.

2. Application configuration
Configuration consists of things such as settings for data sources, mail servers, etc, as well as switches in your application itself. While it is in general possible to reflect these kinds of changes dynamically, there is a cost. For example, if you are using database connection pooling, pointing to a different data source dynamically is a non-trivial exercise. Also, if your application checks particular application settings at source each time the affected functionality is used, then the system is more dynamic, but also less performant. By contrast, if you only load particular settings at startup (for example using wired in property placeholders in a Spring application context), the application is more efficient but is more likely to require reloads to reflect changes.

3. Application wiring
Application wiring is a configuration of sorts, but relates more to how parts of the system are composed or wired together to form the whole application. In a Spring application, the application wiring is simply your Spring configuration.

In general, changes to application wiring requires reloads. There are special cases where this doesn't apply. For example, you could introduce new Spring beans without reloading the application context. Changes to existing beans and their collaborators are harder to make.

4. Scripts
Scripts are programs in your application which by definition can be altered without having to reload your entire application or even significant parts of it. However, a scripting infrastructure needs to be in place to allow changes to scripts to be introduced, recognised and reflected in the system.

5. Application code
Application code in a Java application are your Java classes. Actually, considering this group as a single category is an oversimplification, especially in a dynamic module system, where making changes to core interfaces and domain classes will impose much greater requirements in terms of reloading than peripheral or implementation classes with fewer dependants.

6. Third party libraries
The libraries in your application are the jar files containing all the third party dependencies.

The challenge for frameworks

The key productivity challenge with a dynamic application framework is to make it as easy as possible to make the kinds of changes you need to make, while at the same time keeping the framework as lightweight as possible. In the next section I take a look a number of technology stacks, what they do to make different types of reloading possible, what they get wrong, and what they get right.

A. Traditional Java web application
A traditional Java web application might consist, for example, of a Struts or JSF front end, Hibernate back end, all wired together using Spring.

The traditional Java web application has no problem reloading static resources without having to reload any other part of the application. Most web containers are able to reload an entire web application, including third party libraries, Java code, application wiring etc.

The problem is they are not much good at reloading any finer grained changes. Any changes you make to your application wiring or code will normally require a full application reload.

B. Scripted applications
Scripted applications are based on scripting languages such as Groovy (Grails) and JRuby (Rails). As well as explicitly providing support for reloading capabilities, these frameworks rely on the fact that all application functionality is in scripts rather than in compiled Java code, making fine grained reloading of parts of the application possible. The downside (if you think of it this way) is that you have to work with scripted code without any of the type safety checking of a statically typed language such as Java.

C. OSGi applications
OSGi applications offer a fairly comprehensive solution to the reloading problem. All artifacts within the application, from resources to application code to libraries are contained within modules which are treated in a more or less uniform manner by the OSGi container. This is a strength, but it is also a weakness. The strength is that it does allow third party libraries to be reloaded in a fine grained way. The weakness is that in your high level view of the application, OSGi doesn't really allow you to easily distinguish between the parts of your application which should be easy to reload - e.g. resources - and the parts which are harder to reload, but are changed much less frequently during the lifetime of the application (third party libraries).

What about Impala?

Impala tries to find the right balance in the strengths of the various approaches. Resource reloading works as with traditional Java applications - nothing special needs to take place. Impala includes mechanisms which make it easier to change configurations dynamically without requiring any module reloading. For changing static configuration and application wiring, Impala allows you to reload parts of your application at exactly the right granularity. If only a single module is affected, then only that module needs to be reloaded. If the change affects core interfaces, then the reload will automatically ripple through to the right dependent modules.

Impala even allows you to dynamically change the structure of modules within the application. Unlike OSGi, it doesn't support reloading of third party libraries. For this, an entire application reload is required. However, Impala's approach does your application modules in central focus, which is important as these are the normally parts of your application which change most frequently.

Slides for Impala talk at the Server Side Europe in Prague

I'm pleased to say that my talk at The Server Side Java Symposium in Prague went well and was apparently well received. There definitely is a growing interest in the ideas of modularity and how the benefits of modularity can be achieved in practice.

I've posted a copy of my slides for the talk here in PDF format. The document contains the slides that I presented, as well as quite a few which were held in reserve but weren't actually presented during the talk.

How to make your Spring wirings more manageable

When Spring first came along it was a breath of fresh air - a clever way to wire up applications which did not rely on the use of all sorts of singletons all over the place. (I still remember what it was like working that way, and shudder at the thought.) The idea was simple: let classes in the application just do their own job, but leave the business of figuring out how to get collaborators to the IoC container, with the help of some XML configuration. No longer did your application code did not have to deal the messy business of having to resolve their own dependencies.

OK, we've solved the problem with the code, but the job isn't completely done. Actually, the problem has shifted onto managing Spring wirings.

The problem with managing Spring wirings

Applications necessarily get big. You end up having to write a lot of XML. So while your code may stay nice and clean, you end up with some tricky questions about how to manage this part of your application. Of course, here you try to be as "modular" as you can, putting DAOs together, infrastructure related beans together, etc. You try to identify vertical slices for your application and put beans relating to particular vertical slices together.

The problem is that for all the bean definitions that exist in your application, some groups are inherently coupled, while others are inherently free of coupling. In a vanilla Spring application, there is no way to express these dependencies at a system level. So it is very easy for your application wiring to become an unnecessarily fragile collection of invisible dependencies, liable to break in unexpected ways when any rearrangement takes place.

Autowiring, namespaces and class path scanners don't necessarily help

Then of course there is the drudgery of editing XML configuration files by hand. Personally, I think that is less of a problem, but Spring has gone to great lengths to free developers from some of this pain over the years, through the introduction of autowiring, class path scanners, and XML namespaces. I happily embrace all of the above as they reduce the amount of code I need to write, but they don't address the fundamental problem. They don't enhance one's ability to express dependencies between parts of your application at a system level, and where possible, to reduce these dependencies.

So how does Impala help?

Remember how easy Spring seemed when we were working with just small applications? Impala allows you to keep your applications small, or at least keep them feeling small. This is done through modules. You can think of a module as a mini-application which is able to communicate with other mini-applications in the system through well defined mechanisms and through sharing common interfaces.

The Spring configuration for each module remains pretty small. If it starts getting too big, then its a good sign that some of it's functionality needs to be split off into another module. So within the module, you only need to deal with small configurations - bite size chunks.

You can configure beans within a module however you like - through plain Spring XML, custom namespaces or through annotations. If your module needs to use functionality from other parts of the system (as most will), then you can import services directly from the shared service registry (as long as the service has been exported using an interface or class visible to the current module). If necessary, you can allow your module to depend directly on another module, either as a dependent or as a direct child.

If you need to compose your application in different ways according to environment or customer, that's easy too. Simply change which modules your deploy, or you can even vary the configuration of within a module according to requirements.

You no longer need to wait ages for integration tests to load, because you can easily create integration tests which consist simply of the modules you need to use.

And you get the benefits of much more productive, responsive development environment because each of these modules can be reloaded on the fly, either individually, in groups, or as a whole - and this applies whether you are running integration tests or running your application on a web container.

Talk at the Server Side Europe in Prague

I am doing a talk on Impala at The Server Side Europe's conference in Prague, which is taking place on October the 27th and 28th. Really looking forward to it, especially as I used to be a regular visitor of Prague in the early 90s when I was living in Germany.

Why web.xml makes it hard to write modular web applications

In a typical Java enterprise application, the web.xml is used to define servlets and filters, which are among the main entry points into your application from the outside world. Since web.xml cannot be reloaded, added to or modified without reloading the entire application, it is not a very convenient place to host application configuration and definitions in a dynamic module applications.

Another related problem is the limitations of the request mapping capability of web containers as defined by the current servlet specification. Currently, these make it possible to map requests to servlets and filters using an exact match (e.g. /myservlet/myresource) or either a prefix and wildcard match (e.g /myservlet/*) or using a suffix wildcard match (e.g. *.do). It doesn't allow you to use a combination of prefix and suffix wildcard matches. This means that you cannot, for example, use the path (/myprefix/*) to match application URLs, and at the same time allow your application's CSS files to be accessible in a resource such as /myprefix/styles.css.

Multi-module web applications in Impala

One of biggest changes in the recent 1.0 RC1 release of Impala is the ability to write web applications which are less reliant on web.xml, allowing both dynamic registration of modules containing servlets and filters, and at the same time solving the path mapping limitation described in the previous paragraph.

In an Impala application, you cannot do away with the web.xml altogether. However, you can reduce the request handlers defined in web.xml to the following:

<filter>
<filter-name>web</filter-name>
<filter-class>org.impalaframework.web.spring.integration.ModuleProxyFilter</filter-class>
<init-param>
  <param-name>modulePrefix</param-name>
  <param-value>urlmapping-web</param-value>
</init-param>
<load-on-startup>2</load-on-startup>
</filter>

<filter-mapping>
<filter-name>web</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

In the example above, the ModuleProxyFilter captures requests and routes them into Impala modules. The mapping rules which determine which modules service which requests are contained within modules. Here's an example from the URL mapping sample:

<web:mapping>
<web:to-module prefix = "/webview" setServletPath="true"/>
<web:to-handler extension = "htm" servletName="urlmapping-webview" filterNames = "characterEncodingFilter,sysoutLoggingFilter"/>
<web:to-handler extension = "css" servletName="urlmapping-resources"/>
</web:mapping>

<web:servlet id = "urlmapping-webview"
servletClass = "org.impalaframework.web.spring.servlet.InternalModuleServlet"/>

<web:servlet id = "urlmapping-resources"
servletClass = "org.springframework.js.resource.ResourceServlet"
initParameters = "cacheTimeout=10"/>

<web:filter id = "characterEncodingFilter"
filterClass = "org.springframework.web.filter.CharacterEncodingFilter"
initParameters = "forceEncoding=true,encoding=utf8">
</web:filter>

<web:filter id = "sysoutLoggingFilter"
filterClass = "org.impalaframework.urlmapping.webview.SysoutLoggingFilter">
</web:filter>

This module defines a number of servlets and filters who's life cycles are tied to that of the module, rather than that of web.xml. They can be dynamically registered and removed, and don't require an application restart. The modules can contain all the classes and resources necessary to service requests, without relying on the presence, for example, or resources such as JavaScript files on the context path (e.g. in the WEB-INF directory).

What about Servlet 3.0?

The changes described above are very much in line with the changes in the forthcoming Servlet 3 specification which allow servlets and filters to be added via web.xml fragments, and via annotations. It will also allows you to add Servlet and Filter instances programmatically. I expect that Impala will be able to take advantage of this mechanism when it becomes available, perhaps by wrapping the Servlet or Filter to ensure that it is associated with the originating module's class loader, and not the web application class loader. This will have the advantage of allowing Impala to make use of the web container's invocation infrastructure while still supporting dynamic servlet or filter registration.