Lambda the Ultimate

Story on Nabble here. Release note highlights:

Objective Caml 3.11.0:
----------------------

(Changes that can break existing programs are marked with a "*" )

Language features:
- Addition of lazy patterns: "lazy " matches suspensions whose values,
after forcing, match the pattern .
- Introduction of private abbreviation types "type t = private ",
for abstracting the actual manifest type in type abbreviations.

Compilers:
* The file name for a compilation unit must correspond to a valid identifier
(no more "test-me.ml" or "my file.ml".)
* Revised -output-obj: the output name must now be provided; its
extension must be one of .o/.obj, .so/.dll, or .c for the
bytecode compiler. The compilers can now produce a shared library
(with all the needed -ccopts/-ccobjs options) directly.
- With -dtypes, record (in .annot files) which function calls
are tail calls.
- All compiler error messages now include a file name and location.
- Optimized compilation of "lazy e" when the argument "e" is
already evaluated.
- Optimized compilation of equality tests with a variant constant constructor.
- The -dllib options recorded in libraries are no longer ignored when
-use_runtime or -use_prims is used (unless -no_auto_link is
explicitly used).
- Check that at most one of -pack, -a, -shared, -c, -output-obj is
given on the command line.
- Optimized compilation of private types as regular manifest types
(e.g. abbreviation to float, float array or record types with only
float fields).

Native-code compiler:
- A new option "-shared" to produce a plugin that can be dynamically
loaded with the native version of Dynlink.
- A new option "-nodynlink" to enable optimizations valid only for code
that is never dynlinked (no-op except for AMD64).
- More aggressive unboxing of floats and boxed integers.
- Can select with assembler and asm options to use at configuration time.

Run-time system:
- Changes in freelist management to reduce fragmentation.
- New implementation of the page table describing the heap (a sparse
hashtable replaces a dense bitvector), fixes issues with address
space randomization on 64-bit OS (PR#4448).
- New "generational" API for registering global memory roots with the GC,
enables faster scanning of global roots.
(The functions are caml_*_generational_global_root in .)
- New function "caml_raise_with_args" to raise an exception with several
arguments from C.
- Changes in implementation of dynamic linking of C code:
under Win32, use Alain Frisch's flexdll implementation of the dlopen
API; under MacOSX, use dlopen API instead of MacOSX bundle API.

Standard library:
- Parsing library: new function "set_trace" to programmatically turn
on or off the printing of a trace during parsing.
- Printexc library: new functions "print_backtrace" and "get_backtrace"
to obtain a stack backtrace of the most recently raised exception.
New function "record_backtrace" to turn the exception backtrace mechanism
on or off from within a program.
- Scanf library: debunking of meta format implementation;
fscanf behaviour revisited: only one input buffer is allocated for any
given input channel;
the %n conversion does not count a lookahead character as read.

Other libraries:
- Dynlink: on some platforms, the Dynlink library is now available in
native code. The boolean Dynlink.is_native allows the program to
know whether it has been compiled in bytecode or in native code.
- Bigarrays: added "unsafe_get" and "unsafe_set"
(non-bound-checking versions of "get" and "set").
- Bigarrays: removed limitation "array dimension - Labltk: added support for TK 8.5.
- Num: added conversions between big_int and int32, nativeint, int64.
More efficient implementation of Num.quo_num and Num.mod_num.
- Threads: improved efficiency of mutex and condition variable operations;
improved interaction with Unix.fork (PR#4577).
- Unix: added getsockopt_error returning type Unix.error.
Added support for TCP_NODELAY and IPV6_ONLY socket options.
- Win32 Unix: "select" now supports all kinds of file descriptors.

Tools:
- ocamldebug now supported under Windows (MSVC and Mingw ports),
but without the replay feature. (Contributed by Sylvain Le Gall
at OCamlCore with support from Lexifi.)
- ocamldoc: new option -no-module-constraint-filter to include functions
hidden by signature constraint in documentation.
- ocamlmklib and ocamldep.opt now available under Windows ports.
- ocamlmklib no longer supports the -implib option.
- ocamlnat: an experimental native toplevel (not built by default).

Bug fixes:
- Major GC and heap compaction: fixed bug involving lazy values and
out-of-heap pointers.
- PR#3915: updated some man pages.
- PR#4261: type-checking of recursive modules
- PR#4308: better stack backtraces for "spontaneous" exceptions such as
Stack_overflow, Out_of_memory, etc.
- PR#4338: Str.global_substitute, Str.global_replace and the Str.*split*
functions are now tail-recursive.
- PR#4503: fixed bug in classify_float on ARM.
- PR#4512: type-checking of recursive modules
- PR#4517: crash in ocamllex-generated lexers.
- PR#4542: problem with return value of Unix.nice.
- PR#4557: type-checking of recursive modules.
- PR#4562: strange %n semantics in scanf.
- PR#4564: add note "stack is not executable" to object files generated by
ocamlopt (Linux/x86, Linux/AMD64).
- PR#4566: bug in Ratio.approx_ratio_fix and Num.approx_num_fix.
- PR#4582: weird behaviour of String.index_from and String.rindex_from.
- PR#4583: stack overflow in "ocamlopt -g" during closure conversion pass.
- PR#4585: ocamldoc and "val virtual" declarations.
- PR#4587: ocamldoc and escaped @ characters.
- PR#4605: Buffer.add_substitute was sometime wrong when target string had backslashes.
- PR#4614: Inconsistent declaration of CamlCBCmd in LabelTk library.

Now for some unfortunate news:

Native dynlink used to work on Mac OS X

The clean solution to make natdynlink work on recent Mac OS X systems (beside convincing Apple to support the old behavior of their linker in their new implementation) is to change OCaml's x86 backend so that it produces only PIC code (this has been done for the AMD64 port). I don't think there is currently any plan to work on that.

...

Ouch, this makes it almost a dead end for us. I can offer some time to help in this effort, working in the port, or providing feedback. The native dynlink and toplevel are, at least to me, the killer features in 3.11, but adding another hole for Mac OS X intel (in addition to not supporting x86_64) does not seem like the best choice for an increasingly popular architecture.

...

Well, we'd very much like to support native dynlink on OS X 10.5, but Apple is not helping in the least by crippling their linker compared with the one in 10.4. If anyone from Apple is on this list, feel free to contact us at caml-devel@... for more information on this regression.

So it seems like OCaml needs help along two dimensions: support for the X86_64 in general, and support for Position-Independent Code (PIC) in particular. At least on the PIC side, we can steal from the AMD64 port.