Everybody (pretty much) knows what they mean by unit testing, and on the whole, I would expect most people to have a broadly speaking common understanding of what this means. Unit testing of course involves testing the behaviour of a particular unit of code (e.g. Java class) in isolation.
When it comes to integration testing, it's probably not quite as clear cut. For example, does a test need to cover an entire use case to be considered an integration test? What about a database test which only covers a single JDBC-based method? Is this an integration test?
For me, the key point of distinction is the realness of the test. An integration test is such because it embodies the real behaviour of the system. The more a test relies on simulating aspects of a system's behaviour, the less real it is, and further it moves away from being an integration test.
This brings me on to the question of what is an integration test in a Spring/Impala environment. Here, an integration test will typically always involve using a real Spring application context rather than constructing the fixtures programmatically. It will involve involve connecting to a real database, rather than an in-memory test database (e.g. HSQLDB). It will make limited if any use of mock objects. All in all, integration tests in a Spring environment are closer in behaviour to that exhibited by the real system.
Another question worth addressing is this: what is the relative value of integration tests versus unit tests? A working set of integration tests gives me much more confidence than a set of unit tests which are not backed by integration tests.
One difficulty with integration tests is that they are much less direct than testing using unit tests. It's harder to establish the precise link between the tests and what is actually getting tested. On the plus side, you can get a lot of test coverage with very little code through integration tests.
The other problem with integration tests is, of course, they can be painful to write. Specifying dependencies required for integration tests can be fiddly and time consuming. Also, integration tests can be slow to run. For these reasons, some developers shy away from integration tests, preferring to rely on unit tests plus manual testing of use cases using the deployed application. The big cost is that quality of the testing regime in these circumstances is much diminished, making it much easier for bugs to slip through.
With Impala, integration tests are so easy to write and quick to run, that there really are no excuses. Integration tests are easy to write because dependencies are expressed at a high level through composition of modules. They are quick to run because of the dynamic module loading support used by the interactive test runner, and the efficient way that suites of unit tests are managed within Eclipse. See this wiki page for more details.
Wednesday, April 30, 2008
Saturday, April 12, 2008
Impala and Strecks
A few of you may about my involvement in another open source project, Strecks, which was a set of Java 5 specific extensions to Struts. Yes, remember Struts, the one that used to be so popular a few years ago. There are a few similarities between Impala and Strecks, but also a few differences, that I would like to comment on.
First, the similarities. Just like Strecks was an extension to Struts, Impala is an extension to Spring. I wrote Streck to address what I perceived to be limitations of Struts. These limitations of Struts are now fairly well understood. In a similar way, I wrote Impala to address shortcomings of Spring - in particular, the lack of first class support for modules.
Strecks allowed me to write web applications with Struts the way I believed they should be written. The changes I made with Impala have allowed me to write Spring applications in the way that I would like to write them. Some of the real headaches that came with attempting to scale Spring applications, attempting to build in sophisticated configuration options, have just gone away.
Now to the differences. The timing of Strecks was unfortunate. Just as I was in the reasonably advanced early stages of development, an announcement was made that Struts was to merge with Webwork, with future Struts 2 development using the Webwork code base. That was the death knell to Struts as we knew it. I believed at the time that Strecks was a genuine way forward for Struts, without having to throw away the code base and stick a finger at the huge Struts user community. However, once the decision had been made, any project based on old Struts had little chance of gaining any long term traction. I still took the project to a 1.0 final release, but there seemed little point in actively developing the project from then onwards, unless it served my own needs directly. Since then, I haven't been working on Struts projects more than intermittently, so such a need hasn't arisen in any real sense. That being said, it's still getting 100 to 200 downloads a month, so it hasn't disappeared completely off the map. Oddly, I am now spending a bit more time on a Struts-based project, so it might be a good opportunity to revive Strecks and put in a couple of features I had wanted to add. Also, I might focus on Strecks as a target for web framework integration with Impala.
In contrast to Struts, I don't think there is any chance that the Spring code base is going to be deprecated any time soon. It's an extremely healthy project, arguably more so than any other in the Java community. Secondly, internally, Spring is well architected. It has all the extension points I have needed for Impala to fit naturally into the existing design paradigm. I have never felt the need to correct Spring's faults. Struts, on the other hand, had flaws in its architecture that were not always easy to get around when implementing features in Strecks.
My hope is that the Spring community will share my view that Impala elegantly solves some of the real practical problems which come when developing large, complex Spring-based applications, delivering substantial productivity benefits at the same time. It has been a fascinating and enjoyable project to work on. It would be nice to see some of these benefits being shared more widely. But first I need to get the public release out!
First, the similarities. Just like Strecks was an extension to Struts, Impala is an extension to Spring. I wrote Streck to address what I perceived to be limitations of Struts. These limitations of Struts are now fairly well understood. In a similar way, I wrote Impala to address shortcomings of Spring - in particular, the lack of first class support for modules.
Strecks allowed me to write web applications with Struts the way I believed they should be written. The changes I made with Impala have allowed me to write Spring applications in the way that I would like to write them. Some of the real headaches that came with attempting to scale Spring applications, attempting to build in sophisticated configuration options, have just gone away.
Now to the differences. The timing of Strecks was unfortunate. Just as I was in the reasonably advanced early stages of development, an announcement was made that Struts was to merge with Webwork, with future Struts 2 development using the Webwork code base. That was the death knell to Struts as we knew it. I believed at the time that Strecks was a genuine way forward for Struts, without having to throw away the code base and stick a finger at the huge Struts user community. However, once the decision had been made, any project based on old Struts had little chance of gaining any long term traction. I still took the project to a 1.0 final release, but there seemed little point in actively developing the project from then onwards, unless it served my own needs directly. Since then, I haven't been working on Struts projects more than intermittently, so such a need hasn't arisen in any real sense. That being said, it's still getting 100 to 200 downloads a month, so it hasn't disappeared completely off the map. Oddly, I am now spending a bit more time on a Struts-based project, so it might be a good opportunity to revive Strecks and put in a couple of features I had wanted to add. Also, I might focus on Strecks as a target for web framework integration with Impala.
In contrast to Struts, I don't think there is any chance that the Spring code base is going to be deprecated any time soon. It's an extremely healthy project, arguably more so than any other in the Java community. Secondly, internally, Spring is well architected. It has all the extension points I have needed for Impala to fit naturally into the existing design paradigm. I have never felt the need to correct Spring's faults. Struts, on the other hand, had flaws in its architecture that were not always easy to get around when implementing features in Strecks.
My hope is that the Spring community will share my view that Impala elegantly solves some of the real practical problems which come when developing large, complex Spring-based applications, delivering substantial productivity benefits at the same time. It has been a fascinating and enjoyable project to work on. It would be nice to see some of these benefits being shared more widely. But first I need to get the public release out!
Friday, April 4, 2008
More about Impala scaffolding
It is important to appreciate the developers are busy people - if they are going to spend an hour looking at your project, then this has to be an hour well spent. For this reason, there is a real focus in Impala on making the developer experience as seamless as possible. Everything should work out the box. It's only when you start doing interesting stuff, stuff specific to your problem domain, that you should have to do any real work.
In my previous post I described the simple scaffolding system that is available for Impala. This is not supposed to compete with the scaffolding provided by Ruby on Rails, Grails and such projects. There is a fundamental difference in approach. Impala's scaffolding is only supposed to take you to the point where you have a clean slate to work on. It is not a code generation framework, and I have no intention of moving into that space with it. However, getting to a clean slate position - where you can actually start working on your application - is not a zero work task for many projects. For a project such as Impala which is designed to support very complex applications, you want to make this as simple as possible. I don't want developers who take time out to try Impala to spend their first hour creating directory structures, setting up class paths, and downloading third party libraries from various sources.
The scaffolding I've created with Impala is designed to help you transition to square one as quickly as possible. Once you get there you have a working Impala Hello World application, with working module definitions, a working web application, interactive integration tests and a JUnit suite. With a decent internet connection, running through the scaffolding steps should only take a few minutes - allowing time to play, try things out and do some fun stuff.
In my previous post I described the simple scaffolding system that is available for Impala. This is not supposed to compete with the scaffolding provided by Ruby on Rails, Grails and such projects. There is a fundamental difference in approach. Impala's scaffolding is only supposed to take you to the point where you have a clean slate to work on. It is not a code generation framework, and I have no intention of moving into that space with it. However, getting to a clean slate position - where you can actually start working on your application - is not a zero work task for many projects. For a project such as Impala which is designed to support very complex applications, you want to make this as simple as possible. I don't want developers who take time out to try Impala to spend their first hour creating directory structures, setting up class paths, and downloading third party libraries from various sources.
The scaffolding I've created with Impala is designed to help you transition to square one as quickly as possible. Once you get there you have a working Impala Hello World application, with working module definitions, a working web application, interactive integration tests and a JUnit suite. With a decent internet connection, running through the scaffolding steps should only take a few minutes - allowing time to play, try things out and do some fun stuff.
Tuesday, April 1, 2008
Scaffolding for Impala
It's been a little while since my last post, and since then, Impala has taken a few steps forward in the background. One of these is a simple scaffolding mechanism, which allows you to go from an empty empty Eclipse workspace to a working application in just a few simple steps.
First, start by downloading the latest snapshot distribution of Impala from the subversion repository.
http://impala.googlecode.com/svn/trunk/impala/impala/dist/impala-SNAPSHOT.zip
You can do this via the web browser. From Linux or Mac OSX you may find it more convenient to use curl.
cd ~
curl -o impala-SNAPSHOT.zip http://impala.googlecode.com/svn/trunk/impala/impala/dist/impala-SNAPSHOT.zip
You can then unzip the Impala distribution:
unzip impala-SNAPSHOT.zip
Once you've unzipped Impala, set the IMPALA_HOME environment property.
IMPALA_HOME=~/impala-SNAPSHOT
export IMPALA_HOME
In windows, you will probably want to use the GUI to do the same.
Now, change to IMPALA_HOME, and run the following command:
ant -f scaffold-build.xml scaffold:create -Dimpala.home=./
You will then be guided through an interactive process where you will need to specify the following information:
Before we import the Eclipse project, there are just two more steps to follow:
First, go the main newly project, and run the following two commands:
cd /Users/philzoio/workspaces/scaffold/main
ant fetch
ant get
The fetch command will copy the Impala libraries into the repository project of the new workspace, as shown by the following output.
The get command downloads the necessary third party libraries, as defined using a simple format in dependencies.txt files.
Use the menus File -> Import ... -> General -> Existing Projects Into Workspace. When prompted, set the import base directory to the workspace root directory. This should bring up a dialog box as shown below.
Select all of the projects and import them.
If you reach this point and no errors are showing in your workspace, the congratulations! You have just set up a new Impala workspace.
Let's test it out:
Running up the web application
Using CTRL-Shift + T, find the class StartServer. Right click, then select Run As ... Java Application.
This will start a Jetty Server and run up a server on port 8080.
The text shown on the console view of Eclipse will look something like this:
http://localhost:8080/web/message.htm
Note that Impala is started up to automatically detect changes in your modules, and reload modules in response to these changes. You can play with this mechanism by making changes to classes such as MessageController (in the web project) and MessageServiceImpl (in the module project).
Running the standalone interactive client
Use Eclipse to find the JUnit test class MessageIntegrationTest. Again, run this as a Java application, and execute tests, reload modules etc, using the interactive test runner. Here's some example output:
Run the suite of tests
Any of the JUnit integration tests can be run as a regular unit test in Eclipse, with green bar and all. From the tests project, find the class AllTests. This contains a suite of tests covering all the tests in the project. Run this as a regular unit test, and you will see the following:
There's plenty more to get your teeth stuck into, but having a working web application, a working suite of tests, and interactive tests which can be run out the box is a helpful start.
First, start by downloading the latest snapshot distribution of Impala from the subversion repository.
http://impala.googlecode.com/svn/trunk/impala/impala/dist/impala-SNAPSHOT.zip
You can do this via the web browser. From Linux or Mac OSX you may find it more convenient to use curl.
cd ~
curl -o impala-SNAPSHOT.zip http://impala.googlecode.com/svn/trunk/impala/impala/dist/impala-SNAPSHOT.zip
You can then unzip the Impala distribution:
unzip impala-SNAPSHOT.zip
Once you've unzipped Impala, set the IMPALA_HOME environment property.
IMPALA_HOME=~/impala-SNAPSHOT
export IMPALA_HOME
In windows, you will probably want to use the GUI to do the same.
Now, change to IMPALA_HOME, and run the following command:
ant -f scaffold-build.xml scaffold:create -Dimpala.home=./
You will then be guided through an interactive process where you will need to specify the following information:
- the name of the project containing the root Impala module. This project is a kind of a master project, and will also be the project from which you will typically run ANT build scripts when this is necessary.
- the name of the project containing a non-root module. In a real world application, non-root modules would contain implementations of DAOs, service methods - anything really. In a substantial real world application there will be several if not many non-root modules, together forming a hierarchy of modules.
- the name of a project containing a web module. Although it is possible to have multiple web modules in a single application, the simple scaffolding starts with just one web project.
- the name of the repository project. This contains the third party jars used by the different application modules.
- finally, the name of the test project. The test project is really a convenience from which you can easily run suites of tests for the entire application.
ant -f scaffold-build.xml scaffold:create -Dimpala.home=./
Buildfile: scaffold-build.xml
scaffold:input-workspace-root:
[input] Please enter name of workspace root directory:
/Users/philzoio/workspaces/scaffold
scaffold:input-main-project:
[input] Please enter main project name, to be used for root module:
main
scaffold:input-module-project:
[input] Please enter name of first non-root module:
module
scaffold:input-web-project:
[input] Please enter name of web module:
web
scaffold:input-test-project:
[input] Please enter name of tests project:
test
scaffold:input-repository-project:
[input] Please enter name of repository project:
repository
scaffold:create-confirm:
[echo] Workspace root location: /Users/philzoio/workspaces/scaffold
[echo] Main (root) project name: main
[echo] First non-root module project name: module
[echo] Web project name: web
[echo] Tests project name: test
[echo] Repository project name: repository
[input] Press return key to continue, or CTRL + C to quit ...
scaffold:create:
[mkdir] Created dir: /Users/philzoio/workspaces/scaffold
[copy] Copying 6 files to /Users/philzoio/workspaces/scaffold/main
[copy] Copied 6 empty directories to 5 empty directories under /Users/philzoio/workspaces/scaffold/main
[copy] Copying 3 files to /Users/philzoio/workspaces/scaffold/main
[copy] Copying 1 file to /Users/philzoio/workspaces/scaffold/main/spring
[copy] Copying 4 files to /Users/philzoio/workspaces/scaffold/module
[copy] Copied 5 empty directories to 4 empty directories under /Users/philzoio/workspaces/scaffold/module
[copy] Copying 2 files to /Users/philzoio/workspaces/scaffold/module
[copy] Copying 1 file to /Users/philzoio/workspaces/scaffold/module/spring
[copy] Copying 11 files to /Users/philzoio/workspaces/scaffold/web
[copy] Copied 8 empty directories to 4 empty directories under /Users/philzoio/workspaces/scaffold/web
[copy] Copying 2 files to /Users/philzoio/workspaces/scaffold/web
[copy] Copying 1 file to /Users/philzoio/workspaces/scaffold/web/spring
[copy] Copying 2 files to /Users/philzoio/workspaces/scaffold/test
[copy] Copying 1 file to /Users/philzoio/workspaces/scaffold/test
[copy] Copying 1 file to /Users/philzoio/workspaces/scaffold/repository
[copy] Copied 2 empty directories to 1 empty directory under /Users/philzoio/workspaces/scaffold/repository
BUILD SUCCESSFUL
Before we import the Eclipse project, there are just two more steps to follow:
First, go the main newly project, and run the following two commands:
cd /Users/philzoio/workspaces/scaffold/main
ant fetch
ant get
The fetch command will copy the Impala libraries into the repository project of the new workspace, as shown by the following output.
ant fetch
Buildfile: build.xml
[echo] Project using workspace.root: /Users/philzoio/workspaces/scaffold
[echo] Project using impala home: /Users/philzoio/impala-SNAPSHOT
repository:fetch-impala-from-lib:
[copy] Copying 12 files to /Users/philzoio/workspaces/scaffold/repository/main
repository:fetch-impala-from-repository:
repository:fetch-impala:
fetch:
BUILD SUCCESSFUL
The get command downloads the necessary third party libraries, as defined using a simple format in dependencies.txt files.
We're now ready to import our projects into Eclipse. Start by opening Eclipse in the newly created workspace.
ant get
Buildfile: build.xml
[echo] Project using workspace.root: /Users/philzoio/workspaces/scaffold
[echo] Project using impala home: /Users/philzoio/impala-SNAPSHOT
shared:get:
[echo] Project using workspace.root: /Users/philzoio/workspaces/scaffold
[echo] Project using impala home: /Users/philzoio/impala-SNAPSHOT
download:get:
[mkdir] Created dir: /Users/philzoio/workspaces/scaffold/repository/build
[mkdir] Created dir: /Users/philzoio/workspaces/scaffold/repository/test
[get] Getting: http://ibiblio.org/pub/packages/maven2/commons-logging/commons-logging/1.1/commons-logging-1.1.jar
[get] To: /Users/philzoio/workspaces/scaffold/repository/main/commons-logging-1.1.jar
[get] Getting: http://ibiblio.org/pub/packages/maven2/commons-logging/commons-logging/1.1/commons-logging-1.1-sources.jar
[get] To: /Users/philzoio/workspaces/scaffold/repository/main/commons-logging-1.1-sources.jar
...
[download] ******************************************************
[download]
[download] RESULTS OF DOWNLOAD OPERATION
[download]
[download] org/springframework/spring-webmvc/2.5.2/spring-webmvc-2.5.2.jar resolved from
http://ibiblio.org/pub/packages/maven2/org/springframework/spring-webmvc/2.5.2/spring-webmvc-2.5.2.jar
[download] org/springframework/spring-webmvc/2.5.2/spring-webmvc-2.5.2-sources.jar resolved from
http://ibiblio.org/pub/packages/maven2/org/springframework/spring-webmvc/2.5.2/spring-webmvc-2.5.2-sources.jar
[download]
[download] ******************************************************
get:
BUILD SUCCESSFUL
Total time: 3 minutes 6 seconds
Use the menus File -> Import ... -> General -> Existing Projects Into Workspace. When prompted, set the import base directory to the workspace root directory. This should bring up a dialog box as shown below.
Select all of the projects and import them.
If you reach this point and no errors are showing in your workspace, the congratulations! You have just set up a new Impala workspace.
Let's test it out:
Running up the web application
Using CTRL-Shift + T, find the class StartServer. Right click, then select Run As ... Java Application.
This will start a Jetty Server and run up a server on port 8080.
The text shown on the console view of Eclipse will look something like this:
You can connect to the server using the URL:
2008-04-01 20:56:10.408::INFO: Logging to STDERR via org.mortbay.log.StdErrLog
2008-04-01 20:56:10.479::INFO: jetty-6.1.1
2008-04-01 20:56:10.963:/web:INFO: Initializing Spring root WebApplicationContext
INFO : BaseImpalaContextLoader - Loading bootstrap context from locations [META-INF/impala-bootstrap.xml, META-INF/impala-web-bootstrap.xml, META-INF/impala-jmx-bootstrap.xml, META-INF/impala-web-listener-bootstrap.xml]
INFO : ScheduledModuleChangeMonitor - Starting org.impalaframework.module.monitor.ScheduledModuleChangeMonitorBean with fixed delay of 2 and interval of 10
INFO : LoadTransitionProcessor - Loading definition root-module
INFO : ScheduledModuleChangeMonitor - Monitoring for changes in module root-module: [file [/Users/philzoio/workspaces/scaffold/main/bin]]
INFO : LoadTransitionProcessor - Loading definition module
INFO : ModuleContributionPostProcessor - Contributing bean messageService from module module
INFO : ScheduledModuleChangeMonitor - Monitoring for changes in module module: [file [/Users/philzoio/workspaces/scaffold/module/bin]]
INFO : LoadTransitionProcessor - Loading definition web
INFO : ScheduledModuleChangeMonitor - Monitoring for changes in module web: [file [/Users/philzoio/workspaces/scaffold/web/bin]]
2008-04-01 20:56:12.133:/web:INFO: Initializing Spring FrameworkServlet 'web'
INFO : ExternalLoadingImpalaServlet - FrameworkServlet 'web': initialization started
INFO : ExternalLoadingImpalaServlet - FrameworkServlet 'web': initialization completed in 21 ms
2008-04-01 20:56:12.168::INFO: Started SelectChannelConnector @ 0.0.0.0:8080
DEBUG : ScheduledModuleChangeMonitor - Completed check for modified modules. No modified module contents found
DEBUG : ScheduledModuleChangeMonitor - Completed check for modified modules. No modified module contents found
http://localhost:8080/web/message.htm
Note that Impala is started up to automatically detect changes in your modules, and reload modules in response to these changes. You can play with this mechanism by making changes to classes such as MessageController (in the web project) and MessageServiceImpl (in the module project).
Running the standalone interactive client
Use Eclipse to find the JUnit test class MessageIntegrationTest. Again, run this as a Java application, and execute tests, reload modules etc, using the interactive test runner. Here's some example output:
log4j:WARN No appenders could be found for logger (org.springframework.context.support.ClassPathXmlApplicationContext).
log4j:WARN Please initialize the log4j system properly.
Test class set to test.MessageIntegrationTest
Unable to load module corresponding with directory name [not set]
Starting inactivity checker with maximum inactivity of 600 seconds
--------------------
Please enter your command text
>test
No module loaded for current directory: main
Running test testIntegration
.Hello World!
Time: 0.055
OK (1 test)
Please enter your command text
>reload
Module 'root-module' loaded in 0.096 seconds
Used memory: 1.5MB
Max available memory: 63.6MB
Please enter your command text
>module module
Module 'module' loaded in 0.035 seconds
Used memory: 2.1MB
Max available memory: 63.6MB
Please enter your command text
>t
No module loaded for current directory: main
Running test testIntegration
.Hello World!
Time: 0.012
OK (1 test)
Run the suite of tests
Any of the JUnit integration tests can be run as a regular unit test in Eclipse, with green bar and all. From the tests project, find the class AllTests. This contains a suite of tests covering all the tests in the project. Run this as a regular unit test, and you will see the following:
There's plenty more to get your teeth stuck into, but having a working web application, a working suite of tests, and interactive tests which can be run out the box is a helpful start.
Subscribe to:
Posts (Atom)