Maxine VM: A VM in Java

Maxine VM is an open source meta-circular JVM from Sun.

Maxine is a VM designed for and written in the Java(TM) Programming Language with an emphasis on leveraging meta-circularity, componentized design, and code reuse to achieve flexibility, configurability, and productivity for academic and industrial virtual machine researchers.

The Maxine Inspector [5min, YouTube] seems pretty neat! Bringing VM research to the masses :)
A longer presentation by Bernd Mathiske at the JVM Language Summit 2008 provides a good introduction.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Jikes RVM is also open source

The big contribution here is the Maxine Inspector, which is cool and all BUT! BUT! it is written in Java and the GUI is Swing. This is 2010, and maybe I am the only one who feels like that was a Brutalist architecture decision. Pragmatically speaking, this means coming up with richer visualizations will be difficult - Swing is painful when you have an idea for a new visualization and want to prototype it in Java. Sadly, Jikes RVM has a summer coding intern project for a student to implement the Inspector for Jikes.

I guess I can't blame either Maxine' or Jikes' project leaders; the NetBeans JavaFX branch is pretty underwhelming as well. JavaFX Composer just isn't titilating.

Anyway, check out Ben Titzer's e-mail to the Jikes mailing list; most of what Ben talked about doing is now in progress (from what I've been spying on).

Yay

Good to see Maxine getting some interest, and on the front page of LtU too.

The C1X compiler was the last thing I did before I left Sun, and I think we did a pretty good job separating the runtime system from the compiler and made some pretty big improvements to the performance of Maxine. There are some details in our VEE 2010 paper Improving compiler-runtime separation with XIR.

I noticed that Tony Hosking opened a Codehaus project to port C1X to JikesRVM. We talked at length about this a few times, and I hope he gets funding and volunteers to do it. MmTk (and perhaps the Harmony libraries) aside, there has been precious little code reuse in VM projects and I hope this idea may get some traction.

Paper

Java and Swing in the Maxine Inspector

As the developer of the Maxine Inspector since its infancy (though not from birth) I can add some background on the choices mentioned here: Java and Swing.
First Java. The early Inspector was co-designed with the VM, and was in fact essential: no other tools would support the VM's development. An (inspired, IMHO) early architectural decision made by the original development team (before I joined) was to expedite the ability of the Inspector to analyze and model VM state by sharing code with the VM: not only some of the core VM code, but also the specific implementations of various VM functions that might be bundled into a particular VM build. This, combined with Java reflection, allows the Inspector to "read the bits" from the VM process and pretty much "disassemble" the entire VM state. The part of the Inspector that does this, which I have gradually been disentangling from the presentation layer, can only be Java. Furthermore, the ability to load both the VM and Inspector code into the IDE of my choice, and to browse, refactor, and build them both in place, has greatly amplified my productivity while working on Maxine.
Now Swing. The original choice of Swing was the natural default in that time and place, and I chose not to change it for a variety of reasons. Yes, programming in Swing is an incredible PITA: one of the worst parts of the job. I dislike many things about it, and I know more than a few of the guilty parties. Enough said about that. On the other hand, it wasn't a bad choice. For one thing, the Inspector has never been about aesthetics, but rather about expediency: helping VM developers with the incredibly challenging task of diagnosing failure by looking at the bits of a broken VM state. A developer needs access to a lot of state, must be able to think about it simultaneously at many levels of abstraction (bits, words, memory addresses, data structures, language constructs, objects, etc.), and must do so in the presence of likely breakage. I submit that this is mainly a numeric/textual exercise (as is programming itself), for which I've exploited the ability of Swing to create complex, fine-grained display and interactive behavior for otherwise ordinary text displays. Once that's available, the real power of the Inspector derives from the sophistication of the analysis/modeling code. I'm not enthusiastic about rich graphics for this sort of thing, a topic I'll expand upon in a separate comment.

Sink/hoist visual representations

Text is really just one visual representation: one way to View Source for an object.

In a fully object-oriented display system, you should be able to sink/hoist visual representations. For example, Henry Baker once pointed out that the ideal way to read a linear object / linearly typed resource is graphic in nature, because text itself doesn't really communicate the linearity of the resource well. The same logic applies to graph grammars, or grammars in general, such as web grammars. There is more to languages than manipulating text.

Something I've been toying with: In a dynamically federated, dynamically distributed system, like the Internet, you can can control representation of resources. This means you can recursively model various visualizations of resources as simple transformations. For example, minification tends to obfuscate code but can produce a tighter wire format while preserving the original semantics of the code. So what you are fundamentally doing here is manipulating a shadow of a resource, not the resource itself. If you can provide a schematic mapping for your obfuscator, then prove that schematic mapping is complete and sound, then you now have a degree of freedom is how you think about text-agents. A hot button here is that most mash-up tools for the Web demonstrate a complete non-understanding of this principle and how powerful it can be, and that this problem, in my humble opinion, is greatly exacerbated by the design of modern Web browsers.

Ergo, I think it is not enough to have a meta-circular VM. You want a recursive virtual machine -- a virtual machine that is objects all the way down. Actually, representations of objects. Then all your bookkeeping becomes agents that manage those representations.

I am interested in you expanding upon your position on not being enthusiastic about rich graphics, in the separate comment you've promised.

right direction

Architecture desiderata aside, this looks like a good direction to head based on my first, shallow pass. Getting the community at large used to having such tools should create a demand-side pull for better visualizations that will motivate elimination of architectural impediments.

I'd be interested in field reports of the ease-of-use of this thing, looking in the direction of pedagogy. There have been some enthusiastic discussions lately about the problems of black box technology creating an impediment to effective autodidactism in budding programmers. In my own anecdotal experience, the greatest system builders I've worked with all got their start piecing together environments from the barest metal on up. I'd be happy to see this kind of system provide some recreation of that experience for modern environments.

There have been some

There have been some enthusiastic discussions lately about the problems of black box technology creating an impediment to effective autodidactism in budding programmers. In my own anecdotal experience, the greatest system builders I've worked with all got their start piecing together environments from the barest metal on up.

These two things -- autodidactism and bottom-up system building experience -- are not equivalent.

I am self-taught, but I tend to learn top-down and breadth-first, owing to being highly associative. I only appear to have bottom-up knowledge, because I've simply put in so much time learning and doing.

Anyway, I'm with you, one-by-one Java is finally getting everything Smalltalk had given the world in 1980. It only took thousands of Java developers ~15 years to do what it took a small team of Xerox PARC researchers ~8 years. (Side note: Code Bubbles, previously discussed on LtU, uses an Eclipse backend that communicates via Java XML-RPC to a Windows Presentation Foundation frontend.)

Java is a hard bootstrap

Java is a very hard language to bootstrap. It is even harder to bootstrap with "official" JDK and set of required VM features. Most of this is due to the lazy initialization and dynamic class loading model but a large part is also the poorly factored design of the JDK classes.

There have been half a dozen or so really serious tries at this, but unfortunately Java was never really designed for this and seems to fight you every step of the way. We had to work around this in Maxine in several ways, with some shortcuts.

That said, I think the Maxine inspector is certainly more feature rich and robust than anything that was ever available for Smalltalk, or for any other meta-circular VM project. Michael Van DeVanter put a huge amount of time and energy into making the Inspector not just a tool for controlling and using a "working" VM, but a tool for debugging a broken VM. And there have been a number of innovations there, some of which aren't published yet, including automatic relocatable watchpoints and lots of subtle tricks for continuing to provide debugging service through GC periods, etc. While the UI is ugly and full of machine-level grit, it is also thoroughly laced with both VM level and object-level information and hyperlinks (in a sense) that are automatically derived and verified robustly by the inspector itself.

No disrespect to Michael Van DeVanter

Awhile ago I read his work on language-based editing environments and so I don't doubt his attention to detail.

I think the problem with Java visualization is more systematic than a personal responsibility of one person; awt was implemented in one month and Swing made the poor decision to be backward compatible with awt, to get everyone psyched for JavaOne. Swing was also designed before HLSL was standardized, back when everyone was still writing games at a very low-level without support like OpenGL and DirectX, and its lack of retained mode design reflects it. While JavaFX has promise, it remains to be seen what Oracle will do with it; the 2010 Winter Olympics will be an interesting test (TIOBE Index never even listed it as an up and coming programming language discused on the Web, even after JavaOne, and even in April 2010 they say it is only "now approaching the top 20", whatever that means as a statement from a Java vendor), and some of NetBeans' add-on libraries for it are (IMHO) poorly designed.

This really just reflects my frustration with trying to build a new language with visualizable code fragments. When I get a cool idea for something, the amount of effort to express that idea can be daunting and I get headaches trying to implement it. As inspiring as something like Code Bubbles is, it is inherently not meta-circular. So you can really just add awt and Swing to the list of problems with building a meta-circular JVM.