Distributed Transactional Memory

Hi, I am new to this Forum. I hope this will be of interest to this community. I am working on a project that tries to combine some of the strengths of shared state and of the Actor model. How to allow developers to model the world using object orientation, while offering safety and composability when writing parallel and distributed applications? The option I am exploring is an innovative STM implementation, with very simple transactions semantics and an extension mechanism to allow custom processing of transactions' write sets. Transactions are View Isolated, i.e. take a full and stable snapshot of memory, without significantly compromising performance thanks to a new lock free algorithm. Transactions never read inconsistent data, do not abort while partially executed, and global progress is guaranteed. The extension mechanism allows processing like logging every changes occurring on transactional objects. Our most sophisticated extension sends transactions’ data over a network, and commits them identically on two machines. This enables a high-performance form of object replication. Our implementations can replicate data between Java, .NET and GWT. If you are at JavaOne next week, I am going to present the project on Monday:

Session ID: S312949
Title: Simpler and Faster Cloud Applications Using Distributed Transactional Memory
Duration: 60 min.
Schedule: Monday, September 20, 13:00 | Parc 55, Powell I / II

A paper is available here and an open-source implementation here.

Comments suggestions welcome.
Thanks.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Nice

I don't know much about JavaOne, but if possible be sure to film your presentation and stick it on YouTube or Google somewhere.

Since you are new to the forum, you might want to emulate other topics. Use <blockquote> to add an abstract of your paper to the OP. Or perhaps I'll do it for you:

Extensible Software Transactional Memory by Cyprien Noël

XSTM is a software transactional memory that can be extended by pluggable components. Extensions can access transactions read and write sets through an API, and process them e.g., for logging, change notification, state persistence or replication.

This project explores ways to make memory transactions useful beyond thread synchronization. We describe in particular an application architecture enabled by extensions which aims to combine some of the strengths of shared state and of the Actor model. Shared state offers developers the modeling power of object orientation, and avoids the overhead of copying memory between components. The Actor model offers safety and composability when writing parallel and distributed applications.

Our second design goal is to make memory transactions easy to reason about and safe to use. Opacity is achieved using a Multi Version Concurrency Control design where transactions are view-isolated, i.e., run in stable and consistent snapshots of the full memory. Transactions never encounter inconsistent data, do not abort while partially executed, and global progress is guaranteed. The programming model is further simplified by enforcing strong atomicity at the type system level, as transactional objects accessors require an ambient transaction.

Finally, our design offers interesting performance characteristics by avoiding mutable shared state. Data is either mutable but private to a thread, or shared but immutable. This allows transactions to run without synchronization (no memory fence) between start and commit, which are themselves implemented in a lock-free way using O(1) memory fences and compare-and-swaps. We describe working implementations on the JVM and CLR for the STM and some extensions.

Uh... may I humbly suggest use of paragraph breaks within your original abstract and topic OP, as well?

This looks like a very decent implementation of STM for both JVM and .NET. I am reminded of some architectural decisions of VoltDB with respect to your restrictions on shared memory. I do like your use of type-system to limit which methods may be called from within a transaction.

A challenge with distributed transactions not mentioned in your paper (AFAICT) is dealing with network partitioning (which is generally indistinguishable from node failure). What does XSTM do in case of partitioning?

Partitioning

Thanks for the abstract, my paragraphs are indeed a little long. For now, by lack of time, we are ignoring issues related to clusters like network partitioning. Our focus is to allow clients, like desktops or phones to connect in an easy way to a server, e.g., in the cloud. The model is to have clients connect to a server, retrieve a graph of objects, do something on it, and disconnect. If a client looses connectivity in the middle of an operation, it buffers transactions until connectivity comes back, and continues to work in a speculative way, like a processor waiting for some memory.

XSTM tries to fill the gap between your local device and a server, and will rely on existing cloud infrastructures for data persistence and cluster management. In the future it would be interesting to see if libraries like ZooKeeper could allow XSTM to directly manage a cluster, deal with network partitioning etc.

I read about VoltDB, and they also have the same model for threading as our NIO server, i.e. to use one thread per code and asynchronous processing. Seems like a very interesting system to read about, thanks.

It would be interesting to

It would be interesting to see whether persistence can be made more efficient in a native code setting by using memory mapped files to automatically save parts of memory to disk. Anyone have experience/ideas on how this could be integrated with STM?

Dead link

It appears the link to the publication is dead. Here is a more permanent link.

Full text

ACM has a policy where anyone can get the full text from the author home page.