On paper, the choice to use OSGi should be an easy one. After all, OSGi offers an escape from the "jar classpath hell" that Java developers have been living with for years. The ability to compose systems through modules that can be composed and loaded dynamically promises a solutions to a range of important problems that enterprise developers have been grappling with, unsuccessfully, for years.
Yet the takeup of OSGi has been quite slow. I was curious enough the other day to take a look on Jobserve on job postings requiring OSGi, and I found only a handful of positions available. While there seems to inexorable movement towards OSGi driven in particular by application server vendors (who, remember, also drove the adoption of the now infamous EJB 1 and 2), and a few evangelists, we are yet to see a groundswell of enthusiasm from the mainstream developer community, in the same way as, for example, with technologies like Grails and, before it, Spring.
This is a shame, because I believe the ideas that underpin OSGi are fundamentally important to writing flexible systems which can remain manageable as they grow in size and complexity.
I'd like to comment on some of the reasons why OSGi has still not taken off in a way which cements it's role as the foundation for Enterprise Java applications, and also to explain why I haven't used OSGi as the basis of Impala. My intention is not to spread FUD, but to identify some of the perceptions (and potentially misconceptions) held on OSGi and to give my interpretation on to what extent they are justified.
Some people just don't "get it"
Not everybody thinks that the idea of partitioning an application into modules is a good one. Some developers are happier just to lump all classes together under a single source directory, and don't see how an application can benefit from modules. Maybe they haven't worked on projects that really require this kind of partitioning, or have suffered from a botched attempt to modularise an application. Clearly, these developers are not going to be early adopters of OSGi or, for that matter, a technology like Impala.
OSGi won't really help the productivity of my development
Clearly, there is more work involved in setting up an OSGi application than a regular Java application. You need to ensure that all your jars, both for your application and for the third party libraries, are OSGi compliant. For your application's jars, you'll be responsible for the bundle manifest yourself, making sure that its content fits in with the structure and organisation of your application. You'll definitely want some tool to make this job easier. Also, you'll have to source third party libraries which are OSGi compliant, or, in the worse case, add the necesary OSGi metadata yourself.
The productivity advantages of dynamically updatable modules will probably kick in at some point, but not until you have a smooth running development environment set up. You can accelerate this process with the help of an OSGi-based framework such as SpringSource's dm Server, or ModuleFusion.
While OSGi will undoubtedly help you write better and more flexible applications, you don't get many wild claims that OSGi will allow you to build your applications dramatically faster. Developers who come to OSGi with those kinds of expectations will probably be disappointed.
OSGi requires a complex environment
Enterprise Java has a reputation for being complex. Not only do you need to know the Java language, you need to know Spring, Hibernate, web technologies, relational databases, etc. etc.. You need to know all sorts of related technologies, test frameworks, ANT or Maven, and more. And this is just to write traditional Java applications.
To write OSGi-based Enterprise applications, there is much more to know. You'll need a good conceptual understanding of how OSGi works - both in the way that it manages class loaders and the way services are exported and consumed. Not Java 101 stuff. You'll also need a practical understanding of the idiosyncracies of your own OSGi environment. There will be differences in the way you build and deploy applications, and the way you manage the tools and runtime, depending on which OSGi containers and helper frameworks you use. You won't need to be a rocket scientist to figure this all out, but you will need some time, patience and experience. The wave of books coming out on OSGi will definitely help, but don't expect the junior members of your team to be able to jump straight into an OSGi project and hit the ground running.
How do I know it will all work?
Some people might be put off OSGi because of lingering thoughts that they will run into difficulties getting their applications to work in an OSGi environment, especially those with large existing code bases.
Some of the most commonly used frameworks out there are not very OSGi-friendly, typically either because they are designed and packaged in a not very modular way, or because they use class loaders in a way which does not align with the OSGi specification, for example, by using the thread context class loader to load classes. Naive use of these libraries in an OSGi environment will lead to unexpected problems.
You'll need to find a way to work around these issues. The hard way will be to try to do it yourself. The easy way will be to rely on a packaged OSGi solution, again such as dm Server or ModuleFusion. But remember, even here, there are trade-offs. In the case of the dm Server, you'll be very closely tied in to SpringSource as a vendor, and with ModuleFusion, you may need to accept a technology stack which does not include your favourite frameworks.
OSGi applications are difficult to test
This, in my opinion, is a real achiles heel of OSGi. Because OSGi applications need to run in an OSGi container with class loading managed in a very specific way, you cannot run low level integration tests without the tests themselves running in a container. This makes testing OSGi applications particularly challenging.
The only serious attempt I am aware of to address this problem is the Spring Dynamic Modules test framework, which dynamically creates a test bundle using your application code, launches an OSGi container, deploys the test bundle to the OSGi container (as well as the bundles you need to test plus some infrastructure bundles), and runs your test code. It's not especially pretty, but there's no substitute for real integration tests as opposed to unit tests or tests using mock objects.
For me, ease of testing is of fundamental importance in choosing technologies - it certainly is a large part of the reason for the emergence of Spring. I certainly have no appetite for a return to the days of EJB 1 and 2 when applications could only be tested on a container.
Some concluding remarks
Let me make my position clear. I am not an OSGi evangelist. I prefer to think of myself as OSGi-neutral. I have deliberately chosen not to base Impala on OSGi, but I have designed it in a way which accomodates OSGi - indeed I even have a working example of Impala running on OSGi. As OSGi gains traction - and if users demand it - Impala will provide much better support for OSGi and even offer a simple migration route to OSGi which users can choose to adopt on a per project basis.
Monday, May 4, 2009
Thursday, March 26, 2009
Where are all the Groovy web frameworks?
A little while ago when I was exploring web frameworks - as I tend to do once every few months - I spent a bit of time looking into web frameworks built in Groovy.
Groovy has all the raw materials needed for a powerful web framework implementation. It's dynamic nature and metaclass facility make it perfect for layering syntactic sugar on top of common tasks. It's very easy to dynamically update Groovy-based functionality - very little special framework is required - which makes it potentially very productive. And it already has very powerful facilities for templating, string manipulation, etc. - all capabilities required for web application development.
With all of these advantages Groovy should be a very fertile breeding ground for web frameworks. After all, for all the dozens of Java web frameworks, you could at least expect a few out there harnessing the power and convenience of Groovy.
Sadly, this does not seem to be the case. You can use Groovy to embellish plenty of existing Java web frameworks, but precious few web frameworks appear to have been built from the ground up with Groovy's power features in mind.
I'd happily be proved wrong on this point, but from my searches Grails seems to be the only real show in town.
I like Grails' web framework. It is an excellent showcase for the power of Groovy. What I don't want is the full stack Grails experience. If I were to use Grails as a web framework, I would want to embed it in an Impala-based application with a modular back-end. It was originally in the Grails 1.1 roadmap to separate the web part of Grails. However, during a recent meetup, I got it from the horse's mouth that this wasn't going to be happening any time very soon.
According to Graeme Rocher, he would consider splitting the web framework "if his users demanded it". I think he is missing the point that by not providing a separately embeddable flavour of Grails he is isolating himself from a significant untapped user base - those looking for a capable web framework but not the full Grails stack. Maybe because Grails has become so popular it is a luxury he can afford!
So where does this leave Groovy? I don't think that it is healthy for a language to be too dependent on a single framework for its popularity. The language needs - and deserves - more variety, a choice to suit a wider range of appetites.
Groovy has all the raw materials needed for a powerful web framework implementation. It's dynamic nature and metaclass facility make it perfect for layering syntactic sugar on top of common tasks. It's very easy to dynamically update Groovy-based functionality - very little special framework is required - which makes it potentially very productive. And it already has very powerful facilities for templating, string manipulation, etc. - all capabilities required for web application development.
With all of these advantages Groovy should be a very fertile breeding ground for web frameworks. After all, for all the dozens of Java web frameworks, you could at least expect a few out there harnessing the power and convenience of Groovy.
Sadly, this does not seem to be the case. You can use Groovy to embellish plenty of existing Java web frameworks, but precious few web frameworks appear to have been built from the ground up with Groovy's power features in mind.
I'd happily be proved wrong on this point, but from my searches Grails seems to be the only real show in town.
I like Grails' web framework. It is an excellent showcase for the power of Groovy. What I don't want is the full stack Grails experience. If I were to use Grails as a web framework, I would want to embed it in an Impala-based application with a modular back-end. It was originally in the Grails 1.1 roadmap to separate the web part of Grails. However, during a recent meetup, I got it from the horse's mouth that this wasn't going to be happening any time very soon.
According to Graeme Rocher, he would consider splitting the web framework "if his users demanded it". I think he is missing the point that by not providing a separately embeddable flavour of Grails he is isolating himself from a significant untapped user base - those looking for a capable web framework but not the full Grails stack. Maybe because Grails has become so popular it is a luxury he can afford!
So where does this leave Groovy? I don't think that it is healthy for a language to be too dependent on a single framework for its popularity. The language needs - and deserves - more variety, a choice to suit a wider range of appetites.
Friday, March 20, 2009
A natural successor to Java?
Before I started learning Java in the late 1990s, I remember picking up a book on Java and on the back cover it was described as the "natural successor to C++". Java has now reached that stage in its evolutionary cycle. It's not that Java is not not a productive and capable language. Because of the excellent tool set that you get with Java it is still more productive (particularly for me) than anything else out there, including the likes of Groovy and Ruby. But just imagine how much more productive you could be with a language which had Java's tooling support, but was free from the annoyances and deficiencies of Java, which are now generally quite well understood but very difficult to fix without breaking backward compatibility.
So what are we looking for in a language that will succeed Java? Firstly, it has to be based on the JVM. While Java as a language is running out of steam in terms of new features added, the JVM has still got real momentum, if only to be judged by the plethora of languages that run on the JVM, both from the ranks of languages which have independent lives outside of the JVM (e.g. JRuby), through to languages which are designed to work on the JVM.
Second, it still needs static typing. That's a strong personal preference - I don't see myself ditching statically typed languages altogether. Dynamic languages like Groovy are great for certain tasks - web app development, integration, etc, but for the guts of a substantial application or application framework, I really do think static typing is a must have.
Third, it needs to have substantial value add features. The one area most missing in Java is support for a more functional style of programming. As a developer who never studied computing at university, I have a background in commercially popular languages - the trend towards more functional programming styles has been around in academia for ages but is more recently moving into the enterprise programming world. While a more functional style of programming does not come that naturally to me, it's absence in Java is quite often a barrier to effective code reuse, and there is no doubt that adopting a more functional style of programming will help me to become a better programmer.
Scala seems to tick the boxes nicely in these three areas, and the fact that it is still fundamentally OO will make it more accessible to existing Java developers.
Still absorbing Scala, so its too early to say what I don't like about it. One concern is whether it is too feature rich, too complicated. Too many features may make it more powerful but may also raise barriers to adoption enough to prevent it from becoming mainstream.
Another potential contender to watch out for is Fan. I saw Stephen Colebourne's talk earlier this week, and was impressed. It strikes me as a serious attempt to fix the problematic aspects of Java without introducing any unnecessary new complexity. It has a very simple type system, and will be much more accessible than Scala, and allows easy switching from the base static typing to duck typing.
On the down side, it has a few features which may prove to be controversial, such as the inability to share mutable state between threads, and only partial support for generics. Another thing I'm note sure about is that abstracts over .NET as well as Java. This may be of interest to those who have to work in both of these environments, but I don't see much value in this for those working primarily in Java.
While these questions rumble on in the background, I intend to add support in Impala for modules implemented in different languages. Should be very simple to do this - after all, it will be mostly just a question of adding build support.
The multi-module structure of an Impala application makes development in multiple languages a good choice, with Java for core interfaces and domain classes, a language like Scala for the more complex implementation elements, and more dynamic languages suitable for modules closer to the fringe of the application.
So what are we looking for in a language that will succeed Java? Firstly, it has to be based on the JVM. While Java as a language is running out of steam in terms of new features added, the JVM has still got real momentum, if only to be judged by the plethora of languages that run on the JVM, both from the ranks of languages which have independent lives outside of the JVM (e.g. JRuby), through to languages which are designed to work on the JVM.
Second, it still needs static typing. That's a strong personal preference - I don't see myself ditching statically typed languages altogether. Dynamic languages like Groovy are great for certain tasks - web app development, integration, etc, but for the guts of a substantial application or application framework, I really do think static typing is a must have.
Third, it needs to have substantial value add features. The one area most missing in Java is support for a more functional style of programming. As a developer who never studied computing at university, I have a background in commercially popular languages - the trend towards more functional programming styles has been around in academia for ages but is more recently moving into the enterprise programming world. While a more functional style of programming does not come that naturally to me, it's absence in Java is quite often a barrier to effective code reuse, and there is no doubt that adopting a more functional style of programming will help me to become a better programmer.
Scala seems to tick the boxes nicely in these three areas, and the fact that it is still fundamentally OO will make it more accessible to existing Java developers.
Still absorbing Scala, so its too early to say what I don't like about it. One concern is whether it is too feature rich, too complicated. Too many features may make it more powerful but may also raise barriers to adoption enough to prevent it from becoming mainstream.
Another potential contender to watch out for is Fan. I saw Stephen Colebourne's talk earlier this week, and was impressed. It strikes me as a serious attempt to fix the problematic aspects of Java without introducing any unnecessary new complexity. It has a very simple type system, and will be much more accessible than Scala, and allows easy switching from the base static typing to duck typing.
On the down side, it has a few features which may prove to be controversial, such as the inability to share mutable state between threads, and only partial support for generics. Another thing I'm note sure about is that abstracts over .NET as well as Java. This may be of interest to those who have to work in both of these environments, but I don't see much value in this for those working primarily in Java.
While these questions rumble on in the background, I intend to add support in Impala for modules implemented in different languages. Should be very simple to do this - after all, it will be mostly just a question of adding build support.
The multi-module structure of an Impala application makes development in multiple languages a good choice, with Java for core interfaces and domain classes, a language like Scala for the more complex implementation elements, and more dynamic languages suitable for modules closer to the fringe of the application.
Monday, March 9, 2009
New Impala extensions project
I've created a new Impala Extensions project, also on Google Code. The idea is to host modules, typically extensions to Spring or Impala or both, which are generically useful, but not substantial enough to warrant their own project and too far from the core of Impala to warrant inclusion in Impala itself.
So far the extensions project contains a set of extensions to Spring annotation-based MVC framework, and also a general purpose event management framework, particularly useful for managing persistent and asynchronous events (as a lightweight alternative to JMS). I expect the list to into a number of other areas over time.
The extensions project also contains an example application which can be used as testbed for individual modules.
So far the extensions project contains a set of extensions to Spring annotation-based MVC framework, and also a general purpose event management framework, particularly useful for managing persistent and asynchronous events (as a lightweight alternative to JMS). I expect the list to into a number of other areas over time.
The extensions project also contains an example application which can be used as testbed for individual modules.
Thursday, February 12, 2009
Roadmap Update
I took a look at the Impala roadmap and realised that it had got a little out of date. I've corrected this - http://code.google.com/p/impala/wiki/Roadmap.
There are some pretty cool features on there way, and plenty of work to do. The good news is that a final 1.0 is not too far away.
There are some pretty cool features on there way, and plenty of work to do. The good news is that a final 1.0 is not too far away.
Monday, February 9, 2009
Impala 1.0M5 released
I am pleased to announce the release of Impala 1.0M5, which introduces many API and configuration improvements into the framework.
Following 1.0M5, only minor changes in internal APIs are now expected prior to the 1.0 final release. The 1.0M5 release also features improvements which make it much easier to configure Impala-based applications, and to add your own extensions to the framework. While Impala is still very heavily based on the Spring framework, 1.0M5 now also makes it possible to plug in other runtime frameworks into Impala's dynamic module loading mechanism.
For more information on this release see: http://code.google.com/p/impala/wiki/Release1_0M5Announcement
Following 1.0M5, only minor changes in internal APIs are now expected prior to the 1.0 final release. The 1.0M5 release also features improvements which make it much easier to configure Impala-based applications, and to add your own extensions to the framework. While Impala is still very heavily based on the Spring framework, 1.0M5 now also makes it possible to plug in other runtime frameworks into Impala's dynamic module loading mechanism.
For more information on this release see: http://code.google.com/p/impala/wiki/Release1_0M5Announcement
Wednesday, December 17, 2008
Impala 1.0M4 released
I am pleased to announce the release of Impala 1.0M4, with preliminary support for OSGi.
Impala 1.0M4 now includes the ability to arrange Impala modules and their corresponding class loaders in a graph, instead of in a hierarchy as in previous releases. This feature offers application developers much more flexibility in choosing the appropriate module structures for applications.
1.0M4 is the first release of the project to include OSGi support. All Impala jars are now OSGi-compliant bundles. It is now possible to run Impala and its application modules in an OSGi container (currently Eclipse Equinox). Further improvements to Impala's support for OSGi are expected in subsequent releases.
At this stage Impala support for OSGi is still experimental. It is not recommended at this stage to use it in a production environment.
Impala promises to become the first and only project to offer a seamless transition to (and from) OSGi. There is no intention to make OSGi the default deployment model for Impala. However, OSGi is likely to become a good choice in the future for projects which will benefit from more rigorous management of third-party libraries than is possible using the traditional model.
More information on Impala's position with respect to OSGi can be found in this blog entry: http://impalablog.blogspot.com/2008/12/announcing-osgi-support-in-impala.html
For more information on this release see: http://code.google.com/p/impala/wiki/Release1_0M4Announcement
Phil Zoio
Impala Home: http://impala.googlecode.com
Impala 1.0M4 now includes the ability to arrange Impala modules and their corresponding class loaders in a graph, instead of in a hierarchy as in previous releases. This feature offers application developers much more flexibility in choosing the appropriate module structures for applications.
1.0M4 is the first release of the project to include OSGi support. All Impala jars are now OSGi-compliant bundles. It is now possible to run Impala and its application modules in an OSGi container (currently Eclipse Equinox). Further improvements to Impala's support for OSGi are expected in subsequent releases.
At this stage Impala support for OSGi is still experimental. It is not recommended at this stage to use it in a production environment.
Impala promises to become the first and only project to offer a seamless transition to (and from) OSGi. There is no intention to make OSGi the default deployment model for Impala. However, OSGi is likely to become a good choice in the future for projects which will benefit from more rigorous management of third-party libraries than is possible using the traditional model.
More information on Impala's position with respect to OSGi can be found in this blog entry: http://impalablog.blogspot.com/2008/12/announcing-osgi-support-in-impala.html
For more information on this release see: http://code.google.com/p/impala/wiki/Release1_0M4Announcement
Phil Zoio
Impala Home: http://impala.googlecode.com
Subscribe to:
Posts (Atom)