Lambda the Ultimate

Given that Unix won, I think it's interesting that dynamic languages make very little use of the dynamic linking and loading infrastructure provided by modern free Unixes such as Linux and the BSDs.

Most dynamic PLs opt to implement "dynamism" (i.e. redefining stuff, loading code at runtime) with application-specific data structures (e.g. Lisp: red-black trees for uniquifying symbols, function pointers and indirect function calls), and they do so solely at runtime (mostly using interpreters and JITs, although Scheme, one of the most advanced dynamic languages, is increasingly illuminating the possibilities of static, independent compilation of dynamic programs).

(Metaprogramming at runtime is perilous, as it is easy to mix up phase distinctions, something we can expect newer dynamic programming languages to discover in a decade (of course we don't know which decade.))

Link-time is mostly ignored. And yet, under Linux with its heavy use of shared objects, one cannot even start a single program without invoking the dynamic linker.

But some people, even some computer programmers, don't know how linkers work and what they do. Basically, a modern linking file format, such as ELF, is a declarative way to construct the memory image of a running process, with lots of features for dynamic customization of the image construction process (ELF even contains a customization hook called "program interpreter" in every executable!).

Likewise, modern compilers and runtime systems such as GNU C contain sophisticated features aimed squarely at dynamic languages: weak symbols for runtime redefinability (used by libc's malloc and in the Linux kernel, for example), computed gotos, nested functions, and increasingly, GC. And there is evidence that dynamic compilation and linking of C snippets is accepted and used in modern systems software.

I have collected some links to these topics, and would be interested to hear of languages and systems that you know that exploit them.

• Linkers and Loaders. A very well-written book with broad coverage of the topics discussed, that is available for free.
• The Linux Program Library HOWTO talks about dynamic loading under Linux.
• Executable and Linkable Format (ELF) specification, 60 pages of highly compressed information in best Bell Labs style.
• ELF and ABI Standards. These specifications describe the actual layouts of things such as stack frames and ELF data structures for different CPUs.
• Shared library call redirection via ELF PLT infection. This article shows a technique that could be used to implement a REPL and other important dynamic language features solely through link-time operations.
• ELF: From The Programmer's Perspective: "Combined with appropriate tools, programmers can use ELF to control the flow of execution at run time."
• How to Write Shared Libraries, an in-depth look by glibc's Ulrich Drepper (thanks, kkourt).
• Series of articles on linkers by Ian Lance Taylor, the author of the gold linker.

(Updates: added 3 ELF links; added Drepper; added Taylor)