Lambda the Ultimate

See here for more on these issues, as well as Michael Yuan's article.

I'd be interested in hearing more about what you and others think about these issues.

Your link is busted Ehud.

I'm pretty new to the J2EE world, but it looks like it's going in the right direction with annotations, less xml, and more defaults in configurations.

I took the liberty of fixing Ehud's link (a missing quote).

On the subject of annotations being the right direction, the article What Is POJO Programming? has a section (on page 3) entitled "Beware of Annotated POJOs", which suggests that annotations can be bad because they tend to couple your code to a particular framework, whereas techniques like dependency injection allow code to be kept much more framework-agnostic.

I'm working on a presentation on Service-Oriented Architecture in J2EE for work in which I make the same point (that annotations tend to result in tighter, rather than looser, coupling), but I hadn't seen that article. I'll refer to it in one of my slides. Thanks!

yeah, that's what i was trying to say.

i know paul has a lot more experience than me with this stuff, so ignore anything that disagrees with him, but here's some random opinions on ehud's link (and the one it refers to)...

both make sense to me (although the "proprietary" issues seems to be a red herring - everything is open source, afaik). i should add that we're still finding our way with persistence / database interfaces, but afaik you can use hibernate in a spring application, so i don't (at the moment) see how that forces you one way or the other as long as you keep fencing technologies away behind interfaces, with each choice cleanly isolated.

as far as xml complexity goes - this seems to be a religious thing. we've not had a problem yet, but if we do there are so many xml-related tools out there i don't see a problem with abstracting some of that away (for example, we could restrict the patterns we use, make our own simpler xml format and generate the spring config via xsl - i think that's a fair price to pay for the flexibility, but on the other hand i know people who have never used xsl and froth at the mouth when you mention xml, so i guess it's not for everyone).

another difference was the ease with which j2ee gives you clustering support. again, we don't worry about that yet, and we hope to avoid it as much as possible by keeping services small and dedicated to well-defined, simple tasks. but when if it is necessary for some service, we can either switch to j2ee for that or handle it explicitly at the esb. simple load balancing is trivial with something like mule. the biggest technical problem we see is distributed/clustered databases, not java.

our general attitude is to keep things as decoupled as possible, and as simple as possible. early iterations go with spring for simplicity but, if we do find we need something that only j2ee can give us, we can switch across (speaking mainly for the half of the project i work on - there's a geographically separate half which has traditionally favoured j2ee, hence the cross-platform experience).

[bullshit/speculation mode] i sometimes think j2ee is sold by scaring people - over-hyping the problems and making the solutions sound much more complex than they really are. if you keep the architecture simple, modular, decoupled then you can start small and grow. you don't need j2ee at the start and you don't lock yourself out of anything by not using it. at least, as far as i can see. and tools like spring and mule are largely non-invasive, flexible and, i suspect, more powerful than the j2ee salesmen would have you believe.

i sometimes think j2ee is sold by scaring people - over-hyping the problems and making the solutions sound much more complex than they really are. if you keep the architecture simple, modular, decoupled then you can start small and grow. you don't need j2ee...

I have no practical experience building j2ee apps, but I kinda got this feeling myself... But doesn't this also apply to spring/mule etc.? What do they really bring to the table that good design and good languages don't?

Note: This is a real question. I am not trolling. I am simply rather clueless about this technology.

nothing (they don't bring anything extra to the table) - but i simply don't know that much. one big reason i like spring (which includes a toolkit that covers a wide range of functionalities) is that it's something i can learn from. by writing software to exploit spring i ended up learning a lot. a lot more than i learnt by writing an earlier j2ee application.

j2ee is largely a black box; spring is more of a framework. you can learn from the framework (or more accurately - i learnt from the framework). if you already know all that, you could write the framework yourself. and if you were using a more productive language than java you might find that approach wins out (ie you write the app and the framework and still win over java). but if you're using java anyway, and particularly if you don't understand everything, then spring is a good way to go.

i'm not saying someone experienced in architecting these systems, and fluent in lisp or ocaml should switch to java + spring!

(although i'm not so sure i'd feel confident implementing the service bus. something like mule includes an awful lot of cross-system knowledge, and it's a great help knowing someone has already done the work of interfacing to messaging system X, so that you can just wire it up).

i learnt from the framework...something like mule includes an awful lot of cross-system knowledge

Can you help me understand by giving some examples of the most valuable kinds of hidden knowledge in these frameworks? Thanks!

in something like mule i'm talking about technical details. it provides interfaces to a whole pile of different messaging protocols. so there's no "deep" knowledge, but lots (and lots) of annoying technical details.

in spring it's a couple of things. first, repeated emphasis in the basics - separating concerns, modular design, etc. the kind of thing you already know and yet - at least if your me - somehow don't manage to attain as well as you should.

second, it's experience with particular problems. if you're implementing the logic for a web form there must be a handful of different ways to do it, with different trade-offs. if you follow the framework you're likely to hit one of the better ones, just because someone has been there before and made the mistakes earlier.

there's nothing really deep. but - and perhaps it's just me - help even with the obvious can be useful. i have the kind of job where i'm working on wildly different projects, which is great, but it means i'm a jack of all trades. working with a framework like this is like having someone who's done it before. i guess that's kind-of true with any toolkit; maybe it's just that spring agrees with my own prejudices.

[added a moment later] one other thing it gives you is extra clout when arguing with other programmers and management. i guess this follows the prejudices comment above, but it's useful to have an example that backs up your ideas.

This is really for both andrew and Ehud:

One of the points that I end up making in my presentation (yet to be given—we'll see how it's received by other folks here, which happens next Tuesday) is that some concepts that are being presented as somewhat "revolutionary" in the J2EE world, particularly in the Spring subset of that world, are really just a very pleasant rediscovery of very old object-oriented principles. Here I'm thinking in particular of "Inversion of Control," whether in its "Dependency Injection" incarnation or not. Way back in 1985, when MacApp came out, one of the questions we MacAppers had to contend with was: what's the big deal about another library? Well, we'd say, MacApp isn't a library; it's a framework. What's the difference between a library and a framework? We decided that a framework says: don't call us; we'll call you. Thus was born the "Hollywood Pattern," which seems to keep getting rediscovered and renamed, but it's been around for a bare minimum of 21 years, and no doubt someone from the Smalltalk or Lisp Flavors/New Flavors/LOOPS/Common LOOPS/CLOS communities could convincingly claim still prior art.

What Spring brings to the table, in my opinion, is a radically consistent application of loose coupling principles to Java. The obvious question that arises when you contemplate the Hollywood Pattern is: where do the objects that I'm constructed with/are set on my properties before my method(s) get called come from? A very flexible answer to this question is that they come from some implementation of the Abstract Factory pattern. So I tend to think of Spring Core as "Hollywood Pattern + Abstract Factory Pattern," and I think you can basically read this from the Spring documentation as well. I wholeheartedly agree that there's nothing magic about it—well, unless you start relying on the autowiring stuff, which gets into the difference between configuration and discovery and rapidly gets almost as political as static vs. dynamic typing. :-)

What Hibernate brings to the table, then, is a very nice Object-Relational mapping system that nevertheless continues to work with Plain Ol' Java Objects (POJOs). So Spring and Hibernate go a long way towards making it possible to develop a large, complex Java application that is nevertheless easy to test and refactor, with or without an actual database, because 99% of your classes don't implement anyone else's interface or extend anyone else's classes—it's all about your application domain. In my world, this is huge. Then you add in stuff that basically revolves around the clever use of dynamic proxies to do AOP on the cheap, for things like authentication, logging, transaction management, the usual AOP hobby horses, and your classes still don't have to change. "It works quite well. Quite well, indeed," with apologies to Christopher Lee.

What Mule brings to the table, then, is to provide yet another kind of container that exposes any given method on any given object via a variety of protocols, transports, invocation mechanisms, and URLs. So without changing my code, I can expose a method as the handler for messages arriving on a JMS topic, as the handler for messages arriving in a POP3 mailbox, as the handler for a SOAP method... Unfortunately, Mule is a bit more invasive than Spring or Hibernate, but in the end I don't mind having a "service layer," the classes of which all have an is-a relationship to a "Universal Messaging Object" class and can therefore be exposed merely by configuration in the way that I've described.

To attempt to bring this home, could this all be done with good design and good languages? Of course it could—I fully intend to have a slide pointing out that at least 80% of service-oriented architecture is just doing good object-oriented design, and probably another 10% of it is doing functional programming, in the sense that a good service should ideally be stateless. I'll point out that there are languages that make it vastly easier to do good service-oriented architecture than Java, such as Scala and Erlang, and that if you follow the stateless, parallelizable reasoning to its logical conclusion you end up with GoogleFS and MapReduce.

only wanted to clarify that a UMO in mule is a POJO(*). the two big restrictions are (1) single argument and (2) rmi dependencies.

(1) - an exposed method must have a single argument (the message object). so the interface layer unpacks the message object, calling the application layer in its own terms (and typically packaging the response so that it's a distinct type to make routine easier)

(2) - it simplifies rmi if you call an interface rather than an object.

so we have the following package hierarchy: a bunch of serializable objects that depend on nothing; interfaces that depend only on message objects; and finally applications. only message objects and interfaces need to be deployed in mule (if you're calling an object exposed elsewhere; obviously mule needs everything if you're using mule/spring as the container)

(*)i'm not at work, so i can't quickly check. it's possible UMO is indeed a class or interface, but you certainly don't have to subclass or implement anything to call it from mule. the only mule dependencies we have are for classes doing routing etc inside mule.

[later] sorry, seem to be monopolizing this thread, but you might find the following useful, and it also helps explain something i said earlier about us writing deployment-agnostic code for j2ee or spring.

we wrote a provider for mule that uses JNP to call stateless session beans in a J2EE server (tested only on jboss). the code has been submitted to mule. this lets you call code inside the j2ee server as easily as calling code deployed inside mule/spring. it also removes the coupling between code and queues that you get with message driven beans.

(JNP is RMI plus some kind of naming protocol. i don't claim to understand it - i just followed some example code to implement the provider - and i don't understand why it isn't already in mule, so i may be missing something obvious).

This is great. I'd like to see those slides when you're done, if you're able (and willing) to share them.

The slides contain no information proprietary to my employer. However, as they were made using a standard template belonging to my employer, I need to check with my management team before releasing them outside the company, because someone might misconstrue the presence of the template to mean that the slides somehow represent some kind of "official position" of the company. Once I ensure that that isn't the case, I'll try to come up with a reasonable public place to put them and link to them.

Can you share the slides with us/me?

I have no practical experience building j2ee apps, but I kinda got this feeling myself... But doesn't this also apply to spring/mule etc.? What do they really bring to the table that good design and good languages don't?

When it comes to Spring's most important aspect, dependency injection, I think Andrew nailed the answer in his first comment above, writing "with Spring you get a two level approach that's surprisingly similar to the old idea of using tcl or some similar lightweight language to assemble your application".

Spring uses an XML DSL for "dynamically" connecting components, i.e. without their having to be statically aware of each other via static references to each other's interfaces in their code. Achieving that dynamic capability in Java is not so trivial that every project team would want to do it for themselves.

Of course, a "good language" might give you that capability without requiring a special framework. For example, you can use a scripting language on the JVM to control Java components in a way that's similar to what Spring does. However, the degree of structure which Spring imposes would require a framework in any language. One of the things that Spring does is identify and support specific common patterns for connecting components, instead of just giving you a Turing-complete scripting language. In that sense, it's like any true "framework" in that it allows you to develop applications by filling in the blanks that are specific to your application.

The other, more mundane thing that these frameworks bring to the table is just providing support for many of the fiddly infrastructural things that typical applications have to deal with. J2EE did that too, but it did so in a way that required a great deal of commitment in your code, which is bad for maintainability over time.

Spring uses an XML DSL for "dynamically" connecting components, i.e. without their having to be statically aware of each other via static references to each other's interfaces in their code. Achieving that dynamic capability in Java is not so trivial that every project team would want to do it for themselves.

I never understood why they don't just use one of the many terse scripting languages available for the JVM to connect everything. Heck, why not just use the beanshell so you can configure everything at runtime in plain-ol' Java. What is it about XML that makes it so much more suitable?

I attempted to answer that in the paragraph after the one you quoted. All Spring needed was a restricted configuration language, as opposed to an entirely general-purpose language, for the same sorts of reasons that people use config files instead of source code to configure programs. One of the easiest and most standard ways of implementing a custom configuration language is to use XML.

Thanks, guys.

I was mainly interested what domain knowledge is embedded in the fws (the domain being ditributed transactional applications).

I think I have a clearer picture now, that will help me follow the documentation without losing the forest for the trees.

"less xml" makes sense when it's that horrible ejb deployment stuff with layers of names/indirection and undefined vendor-specific randomness. but when it's used to do something useful, it's really not so bad. so i wouldn't extrapolate the pain of ejb deployment to spring (maybe that's not what you're doing...).

sorry, feel like i'm pimping spring, which isn't my point. i agree that ejb deployment descriptors were a complete pain, but i worry that after being burnt like that people have gone too far in labelling all xml as bad.

Here's a very informal and anecdotal critique of Spring. It highlights my own concerns, not just with Spring but with this whole godless generation of workflow, injection, and rules frameworks... If there has ever been a case of Greenspun's tenth, this is it. Not that this really invalidates the approach, but at some point I have to wonder why we're still using Java at all.

Interesting. I still fail to see why you need a fw in order to do "dependency injection" (took me awhile to realize this is yet one another of those fancy names for things we all knew ages ago).

Naturally, you don't need a framework in order to do Inversion of Control/Dependency Injection. Thankfully, there's quite a bit more to Spring than that. What's clever about Spring, though, is that it really isn't invasive and is itself very loosely coupled, so you can use as much or as little of Spring as makes sense for your needs. As I wrote before, "Spring Core" is basically a combined IoC + Abstract Factory library that's done very well—well enough that it means that I don't have to roll my own such combination. Spring's Hibernate support is a totally separate piece that also, for me, is extremely useful, and many folks doing web applications seem to like Spring MVC, although I personally think there are better web app frameworks for Java, and that's fine: I can use, e.g. Stripes with Spring just as easily as I can use Spring MVC.

So for me, Spring and Hibernate are exactly what frameworks are supposed to be: something I can use as much or as little of as I want with a minimum of contorting my code to fit into the framework. They're definitely a breath of fresh air after the EJB 2 fiasco. For folks who "don't get" Spring, I seriously wonder what they think the alternatives are. The critique that Matt linked to isn't really a critique at all; it totally ignores the whole concept of loose coupling, the benefits of configuration over code, and basically has nothing to say about Java programming in the large. I would welcome an actual critique of the concepts behind POJO programming and Spring in the context of how server-side Java development should be done, but I haven't yet seen one.

I agree. These frameworks are used to do things that need to do be done one way or another, in applications that deal with databases, web servers, transactions, stateless sessions, etc. Either you're writing and maintaining your own ad-hoc mini framework(s) (been there, done that), or you use ones that are standard, in wide use, and integrate easily with other frameworks.

Greenspun's law was mentioned, but that sort of thing just gets you the necessary foundation, which looks icky in Java, sure. It's the functionality that's then implemented on that foundation that's important. As I wrote elsewhere, you could use a scripting language instead of XML as the configuration glue (possibly losing the configuration vs. code benefits), but you'd still end up with some kind of framework to provide all the necessary capabilities.

The critique that Matt linked to isn't really a critique at all; it totally ignores the whole concept of loose coupling, the benefits of configuration over code, and basically has nothing to say about Java programming in the large. I would welcome an actual critique of the concepts behind POJO programming and Spring in the context of how server-side Java development should be done, but I haven't yet seen one.

This is a good point, and for what it's worth, I agree. If the question is "What's the best way to do large-scale server-side Java programming?" then I agree that this "critique" is pretty lousy (maybe flagging it as "anecdotal" wasn't strong enough). But more and more, I think we shouldn't take it for granted that that's the right question.

I'm not experienced enough with Spring to give it an unqualified yay or nay with respect to some postulated universe of competitive frameworks, but I do work in exactly this setting (very large-scale n-tier web apps, 100% Java as a matter of policy) and the benefits of this approach are quite apparent. On the other hand, when I'm presented with a sophisticated program in the form of 10,000 lines of XML, I have to scratch my head... Between workflow engines, rules engines, IOC containers, webflow languages, and so on ad nauseum, it's really starting to feel like the best way to do server-side Java development is, by hook or by crook, not to.

So I really don't have any complaint with Spring's design or implementation quality. It just starts to feel a little funny, that's all.

...workflow engines, rules engines, IOC containers, webflow languages...

For those of us who don't follow this sort of enterprise Java development, could someone give a brief explanation of what these components do, and maybe what language features they correspond to? For instance, is a "workflow engine" a finite state machine? Is a "rules engine" something like Jess or CLIPS? What is a webflow language?

what language features they correspond to?

Unfortunately, they don't. As they are products, they tend to aggregate a lot of "features", often forcing their premature design decisions upon their users.
A typical workflow engine adds to Java (or other mainstream language) first class continuations and invocations (or their equivalent); transparent persistence of continuations and invocations (complete with their closures); distribution of "reducable contexts" (runnable logical threads = continuation + invocation) between hardware and people; security; transaction control; tools for reporting and manual intervention in the process (think distributed debuggers); and finally, some language for describing process definitions - this sometimes can look like FSM, but usually is more complicated.

What bothers me most, the workflow engines fail to make this separation between infrastructural features (think VM), and the language for process definitions. This often results in uneasy compromises between orthogonality, efficiency, and usability.
And yes, most process definition languages could benefit from a principled PL design - but too often I find that people don't realise these boxes and arrows _are_ a PL. In some way the situation is similar to not recognising XML-based configuration as a program - another popular belief.
I have to finish this post before I get angry :-)

I think these are all fair points, but I have to say that the configuration-as-language perspective tends to miss something very important (IMHO), which is that if a configuration file is a programming language, it's almost universally a declarative, stateless one, with the vital property that local changes have local effects. Obviously, there are limits to this—a local change to a Makefile can have many cascading effects; changing a DataSource declaration in a Spring ApplicationContext.xml file can mean the difference between all of your objects successfully persisting and none of them—but even these issues tend to be easily identified and rectified, compared to the scope of possible effects of a change to (imperative, stateful) code.

Having said that, I wholeheartedly agree that most "workflow engines" overreach and do a poor job of separating concerns. Also, far too many of them are totally ad-hoc, with no basis in a process calculus such as the Pi calculus or any of its cousins, with the attendant tendency towards either lack of expressive power, redundancy and bloat in an attempt to attain expressive power or, even more horrifyingly, both.

Also, far too many of them are totally ad-hoc, with no basis in a process calculus such as the Pi calculus or any of its cousins

Even worse, some of them make false claims about having this basis. For example (from jBPM whitepaper):

a business process engine should have a solid technical basis, such as a mathematical
core in pi-calculus. This branch of computer science was initially developed for the mobile
market and provides the mathematical foundation for jBPM management of similar process
complexity.

I never saw any at least semiformal mapping from jBPM to pi calculus. Also, I am intrigued, are tokens in jBPM called so to make it look related to Petri nets? Ah, I guess this is way off topic for this thread. But I believe scientific community should react to such attempts to bastardize science.

I wish I were using UnCommon Web and CL-SQL too, but unfortunately I don't get to make that choice. So given that I must work in server-side Java, at least we have some good tools available to us!

For the record: I am not trying to criticize Spring, I am sjut trying to understand exectly what it does. I am actually looking for concrete things it does for you, since as far as I can tell the patterns used by Spring can be used by your own code, without the help of a framework. By "concrete" I mean things that belon gto the problem domain, the problem domain being scalable and robust web apps.

even so, j2ee isn't as nice as spring (which is the standard pojo/dependency injection implementation/toolkit), because the annotations still appears in your java code.

In pre-3.0 EJB you use separate deployment descriptors for components, and you still can use them in 3.0 also. So you don't need to clutter your code with annotations. But your code needs still to be EJB code, so yes, EJB is a very intrusive API.

The other issue with EJB is that it couples the logical structure the components are based on with the data and deployment structures. Database structure and usage should match your components. This leads to a lot of issues. The persistance side has been rebuilt in 3.0, I don't know how much of an improvement it has been.

The component structure is also the deployment structure. If you look at books about Enterprise Java design patterns, there's a lot of trickery you have to master to make EJB apps use network efficiently.

When used correctly, EJB can be good. It does offer a nice way to build applications that employ complex transactional processes. But I don't see much point in using it to anything else..?