Before I started learning Java in the late 1990s, I remember picking up a book on Java and on the back cover it was described as the "natural successor to C++". Java has now reached that stage in its evolutionary cycle. It's not that Java is not not a productive and capable language. Because of the excellent tool set that you get with Java it is still more productive (particularly for me) than anything else out there, including the likes of Groovy and Ruby. But just imagine how much more productive you could be with a language which had Java's tooling support, but was free from the annoyances and deficiencies of Java, which are now generally quite well understood but very difficult to fix without breaking backward compatibility.
So what are we looking for in a language that will succeed Java? Firstly, it has to be based on the JVM. While Java as a language is running out of steam in terms of new features added, the JVM has still got real momentum, if only to be judged by the plethora of languages that run on the JVM, both from the ranks of languages which have independent lives outside of the JVM (e.g. JRuby), through to languages which are designed to work on the JVM.
Second, it still needs static typing. That's a strong personal preference - I don't see myself ditching statically typed languages altogether. Dynamic languages like Groovy are great for certain tasks - web app development, integration, etc, but for the guts of a substantial application or application framework, I really do think static typing is a must have.
Third, it needs to have substantial value add features. The one area most missing in Java is support for a more functional style of programming. As a developer who never studied computing at university, I have a background in commercially popular languages - the trend towards more functional programming styles has been around in academia for ages but is more recently moving into the enterprise programming world. While a more functional style of programming does not come that naturally to me, it's absence in Java is quite often a barrier to effective code reuse, and there is no doubt that adopting a more functional style of programming will help me to become a better programmer.
Scala seems to tick the boxes nicely in these three areas, and the fact that it is still fundamentally OO will make it more accessible to existing Java developers.
Still absorbing Scala, so its too early to say what I don't like about it. One concern is whether it is too feature rich, too complicated. Too many features may make it more powerful but may also raise barriers to adoption enough to prevent it from becoming mainstream.
Another potential contender to watch out for is Fan. I saw Stephen Colebourne's talk earlier this week, and was impressed. It strikes me as a serious attempt to fix the problematic aspects of Java without introducing any unnecessary new complexity. It has a very simple type system, and will be much more accessible than Scala, and allows easy switching from the base static typing to duck typing.
On the down side, it has a few features which may prove to be controversial, such as the inability to share mutable state between threads, and only partial support for generics. Another thing I'm note sure about is that abstracts over .NET as well as Java. This may be of interest to those who have to work in both of these environments, but I don't see much value in this for those working primarily in Java.
While these questions rumble on in the background, I intend to add support in Impala for modules implemented in different languages. Should be very simple to do this - after all, it will be mostly just a question of adding build support.
The multi-module structure of an Impala application makes development in multiple languages a good choice, with Java for core interfaces and domain classes, a language like Scala for the more complex implementation elements, and more dynamic languages suitable for modules closer to the fringe of the application.