Project Loom: adding fibers and continuations to Java

Just saw this on Hacker News -- Project Loom: Fibers and Continuations for the Java Virtual Machine with the following overview:

Project Loom's mission is to make it easier to write, debug, profile and maintain concurrent applications meeting today's requirements. Threads, provided by Java from its first day, are a natural and convenient concurrency construct (putting aside the separate question of communication among threads) which is being supplanted by less convenient abstractions because their current implementation as OS kernel threads is insufficient for meeting modern demands, and wasteful in computing resources that are particularly valuable in the cloud. Project Loom will introduce fibers as lightweight, efficient threads managed by the Java Virtual Machine, that let developers use the same simple abstraction but with better performance and lower footprint. We want to make concurrency simple(r) again! A fiber is made of two components — a continuation and a scheduler. As Java already has an excellent scheduler in the form of ForkJoinPool, fibers will be implemented by adding continuations to the JVM.

I'm a fan of fibers, and this has quite a bit of interesting material in it for like-minded folks.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Home page material?

Home page material?

sure (park vs block terms)

Home page works, if you want to move it. Since I also want to discuss, I did not want to seek attention. Rather than dump my impressions, I'd rather chime in, sporadically. I can add a first comment though:

I thought using verb to block for both threads and fibers causes confusion, since this makes it hard to discuss non-blocking styles ported to a fiber runtime. You end up describing fibers as both blocking and non-blocking, so it's a mess. When applied to fibers, I like to park better, which means to suspend without blocking the host thread. You can use suspend as the generic idea, with block and park specialized to threads and fibers.

Edit: I think the proposal is being actively revised; I didn't notice the park/unpark operations in my first reading. I'm cool with this, but wish I had saved a copy of an earlier draft.

Done. I thought the whole

Done. I thought the whole point of putting on home page is to discuss. I tend to put my thoughts into the first comment. A subtle form of clickbait.

Misc thoughts

The article seems to claim that JVM's today do not have "green threads" which existed in Java many years ago. https://en.wikipedia.org/wiki/Green_threads Is that true? If so, what happened to them in the interim? Presumably, the answer is quite relevant to the topic.

For improving the popularity of threaded programming in general, in languages where backward compatibility is not an issue, I favor the approach of the D-language which makes static and global variable storage thread-local by default, and shareable by request.

Java does have non-blocking I/O libraries. It doesn't have good primitives to periodically check for completion in the middle of unrelated execution in the current thread. Are fibers a/the best solution to that? I can imagine a simple kind of "Do this when I/O is ready" construct which would not be a thread at all, but simply a code generation strategy to check for completion at some interval and then call a function when the flag is set, without bothering the programmer.

control flow origami

I've been thinking about a post describing the "why" of fibers, with the problem of saying little about the Project Loom proposal in Java. While on topic, it seems too much about what I find interesting; making it short seems hard too. Also, the "why" part varies depending on one's objective. Every time someone asks how to achieve an effect, you want to ask, "What are you trying to do?" The answer is quite different when I consider things I do at work, versus things I would do for fun. Both involve "doing a lot of things at the same time", but a work context version is about a thousand times worse in scale. There's little point unless there are not enough native threads to scale to a set of activities you want to organize as concurrent, while still viewing them as call graphs for "separate" programs (for clarity in debugging).

I'll take a shot at that later. Before I respond to your comments, note I also favor the same word used in Project Loom to describe an abstraction of threads. A strand is something like a thread, which can be satisfied with thread-like behavior. This includes native threads and user space fibers (aka green threads). When thread is used to mean both a general and a specific idea at once, conversation can be confusing, because it's hard to tell which you mean. It's simpler to let thread imply native, to let fiber imply green, and to use strand as a generalization of both, especially when a system uses threads and fibers at the same time. (Points made with fewer syllables can have better clarity.)

The article seems to claim that JVM's today do not have "green threads" which existed in Java many years ago. https://en.wikipedia.org/wiki/Green_threads Is that true? If so, what happened to them in the interim? Presumably, the answer is quite relevant to the topic.

That page on Wikipedia says green threads were used early on, but were replaced with native threads, except in Squawk. I get the impression Java has a model of threads that could be native or green, but not easily both. I guess usually just one is picked. Another useful page on the topic might be https://stackoverflow.com/questions/5713142/green-threads-vs-non-green-threads. (Some comments there suffer from ambiguity when not distinguishing thread from fiber.)

For improving the popularity of threaded programming in general, in languages where backward compatibility is not an issue, I favor the approach of the D-language which makes static and global variable storage thread-local by default, and shareable by request.

As little shared mutable state as possible seems a good idea. Exceptions should be motivated by a story, and an argument explaining safety and correctness in the face of concurrently updated mutable state. (Single writers are a good idea, with one actor as the update bottleneck. When this is not feasible, a good justification is needed with clear explanation.)

Java does have non-blocking I/O libraries. It doesn't have good primitives to periodically check for completion in the middle of unrelated execution in the current thread. Are fibers a/the best solution to that? I can imagine a simple kind of "Do this when I/O is ready" construct which would not be a thread at all, but simply a code generation strategy to check for completion at some interval and then call a function when the flag is set, without bothering the programmer.

Checking for completion is a sort-of single-threaded perspective. You would only care if you intended a control flow join, where one thread needs to coordinate with some other effect that happened concurrently. I want to discuss that topic as a separate post, about the idea of "organizing cooperating strands which divide a single program into several concurrent parts". In effect, I want to run a program as a lightweight process composed of multiple strands in a group which know about each other, and probably signal each other with condition variables. Note here granularity is a motivation for fibers, because instead of one thread representing a task, I want N strands in a group, and I want them as cheap as possible. Suddenly having too many threads is a scaling failure, when you cannot scale by several orders of magnitude whenever needed.

Coordination of a non-blocking i/o API with fibers is simple when a completed i/o can unpark a fiber waiting on that. In this sense fibers are a good solution when the clarity is striking. The completion callback amounts to "awaken fiber". This only gets complex in the context of cancellation. What if that waiting fiber was killed? This is why you want the notification to go to a strand group instead, by process group id, so it awakens whatever fiber was waiting on that, if any. If the whole lightweight process is gone, the id is no longer valid and notification does nothing. (Unless, I suppose, you register surrogates to note messages to processes that are gone, for debug auditing purposes.)

A "do this when i/o is ready" callback mechanism is a typical way async interfaces are structured. I see a lot of this in C. (I do a lot of IPsec stack debugging, basically full time, involving too many native OS processes.) This sort of organization often results in what people call "callback hell" because it is difficult to see how you got someplace, any time something goes wrong. The absence of monolithic call graph for the activity as a whole makes it hard to construct a synoptic view when the past is erased except for breadcrumbs of data here and there. While it totally works, it is daunting to be smart enough to debug, when the author was just barely smart enough to wire all the necessary bits together. You can think of fibers as the same thing, but folded up differently, with a cheap stack providing context in a way analogous to normal single threaded programs, so you can debug more easily.

The article seems to

The article seems to claim that JVM's today do not have "green threads" which existed in Java many years ago. https://en.wikipedia.org/wiki/Green_threads Is that true? If so, what happened to them in the interim? Presumably, the answer is quite relevant to the topic.

That page on Wikipedia says green threads were used early on, but were replaced with native threads, except in Squawk. I get the impression Java has a model of threads that could be native or green, but not easily both. I guess usually just one is picked. Another useful page on the topic might be https://stackoverflow.com/questions/5713142/green-threads-vs-non-green-threads. (Some comments there suffer from ambiguity when not distinguishing thread from fiber.)

I recall some early VMs which claimed the ability to use both green threads and native threads. Your stack overflow link hints at one reason why green threads became less popular: when multi-core CPUs became the norm, they would not take advantage of the performance boost available to native threads, which would get scheduled on other cores.

These historical discussions suggest that green threads were dropped as a capability after being implemented suggests an overall lack of interest in the use cases where the number of simultaneous threads exceeds practical OS/memory resources. while at the same time, the single CPU/box is still adequate for overall computational throughput - as opposed to needing a rack/server farm/compute cloud. That might still be true...

Java does have non-blocking I/O libraries. It doesn't have good primitives to periodically check for completion in the middle of unrelated execution in the current thread. Are fibers a/the best solution to that? I can imagine a simple kind of "Do this when I/O is ready" construct which would not be a thread at all, but simply a code generation strategy to check for completion at some interval and then call a function when the flag is set, without bothering the programmer.

Checking for completion is a sort-of single-threaded perspective. You would only care if you intended a control flow join, where one thread needs to coordinate with some other effect that happened concurrently. I want to discuss that topic as a separate post, about the idea of "organizing cooperating strands which divide a single program into several concurrent parts". In effect, I want to run a program as a lightweight process composed of multiple strands in a group which know about each other, and probably signal each other with condition variables. Note here granularity is a motivation for fibers, because instead of one thread representing a task, I want N strands in a group, and I want them as cheap as possible. Suddenly having too many threads is a scaling failure, when you cannot scale by several orders of magnitude whenever needed.

Coordination of a non-blocking i/o API with fibers is simple when a completed i/o can unpark a fiber waiting on that. In this sense fibers are a good solution when the clarity is striking. The completion callback amounts to "awaken fiber". This only gets complex in the context of cancellation. What if that waiting fiber was killed? This is why you want the notification to go to a strand group instead, by process group id, so it awakens whatever fiber was waiting on that, if any. If the whole lightweight process is gone, the id is no longer valid and notification does nothing. (Unless, I suppose, you register surrogates to note messages to processes that are gone, for debug auditing purposes.)

This Wikipedia article on Futures and Promises is relevant, but poorly written. Lazy, non-blocking use of futures seems like a good feature, though the article makes it difficult to say where it is available and in exactly what form.

Fibers to replace ad hoc state machines with vanilla code style

Yes, people note cores are underutilized when pursuing green threads alone. You need multiple threads and/or OS processes to better spread load over cores. Performance has more than one dimension; it depends on what you measure. If you do one thing, and you measure wall clock latency, devoting as many cores to it as possible tends to reduce latency, unless you screw up via lousy locality. If you do many things, and are taking pains to spread load over cores already, you want to get as many things done as possible during a given period of time, and this measure is more like transaction throughput. Ideally, you want as many free cycles left over as possible, after servicing some load, so you can do even more.

Marketing is always going to want to advertise a bigger number of simultaneous peers, maximum number of tunnels, and tunnel setups per second. There is essentially no limit to the demand for number of devices connected at once. But once you've spent all the resources, you're done. All you can do is spend less per thing. Doing this too cleverly tends to introduce bugs. So you want simple structure that scales with little cost, where you can also tell what the heck happened. Threads become a scarce resource in high end commercial stuff, but likely not in personal projects.

Absent fibers, async conversations are typically built around finite state machines of varying informality. The next event looks up an entity involved so it performs the next step in negotiation before waiting on other events. At the same time, the system may alter due to either configuration changes or network errors that abort ongoing connections. The complexity of timing in edge conditions can be really irritating. In my environment I want fibers in C, since more of the processes I deal with are written in C. (Numerous daemons are written in several languages including C++ and Java as well. The Java processes are not as highly loaded as those in C.) I have no strong opinion about Java; fibers just seem a good idea in addition to threads.

However, I think code must be written so it knows (the coder knows) about fibers and threads, as opposed to being ignorant of them, in hypothetical pursuit of strand transparency. Locality is quite important, so you need to know if you have scheduled a task to be performed "close by" for low context switch cost. You also need to know which things must be thread-safe, and which don't. I'm not at all in favor of programmers having no idea what they are doing. So a semantic model must be visible in either language or library interfaces, for a programmer to manipulate. Adding fibers to a language yields something like a new variant of the language, where code written for it would not generally work in the stock base language.