Lambda the Ultimate

Anna: A KVS For Any Scale, by Chenggang Wu, Jose M. Faleiro, Yihan Lin, Joseph M. Hellerstein:

Modern cloud providers offer dense hardware with multiple cores and large memories, hosted in global platforms. This raises the challenge of implementing high-performance software systems that can effectively scale from a single core to multicore to the globe. Conventional wisdom says that software designed for one scale point needs to be rewritten when scaling up by 10−100× [1]. In contrast, we explore how a system can be architected to scale across many orders of magnitude by design.

We explore this challenge in the context of a new key-value store system called Anna: a partitioned, multi-mastered system that achieves high performance and elasticity via wait-free execution and coordination-free consistency. Our design rests on a simple architecture of coordination-free actors that perform state update via merge of lattice-based composite data structures. We demonstrate that a wide variety of consistency models can be elegantly implemented in this architecture with unprecedented consistency, smooth fine-grained elasticity, and performance that far exceeds the state of the art.

This isn't strictly programming language related, so I didn't post this as a story, but actors and distributed systems are popular topics around here, and this builds on the CRDT and Bloom work discussed here before. The performance numbers under contention are certainly impressive compared to existing alternatives.

See also the accompanying article where the authors discuss this further.