Lambda the Ultimate

Didn't the Lisp machines do something similar?

ZetaLisp on Symbolics or other hardware is a systems language on the level of C (yes, you can even do pointer arithmetic and varying degrees of manual memory management, but you can't cast-in-place since the hardware won't honor it), but with the semantics and library of Common Lisp (which came later). It also had a bootstrap Lisp dialect which had extremely minimal support and was mostly equivalent to today's OpenFirmware Forth. And all Common Lisp's support assembly, compilation, and de-compilation to the level that their compiler works.

By comparison, GOAL is a cross-compiling macro-assembler (where the macros are Lisp macros and the run-time is non-trivial enough to be interesting).

It's not so unusual to support multiple levels in one language. Generally a language has a primary focus which can be inferred from its library, which may be built around hardware operations such as 32-bit integers and direct pointer manipulation, or on the other hand on abstractions such as unbounded Integers and checked containers.

To support both, a language simply includes both levels of operations, as well as generates decent code based on the low end operations. That's the theory...

The size of the libraries written in other languages are usually larger than the one that you are programming in. The success of a language depends on how easy it is to interface foreign functions, and how the foreign functions concepts map into your language's.

For instance, if your language is side effect free, you'd have a lot of trouble interfacing foreign functions that have side effects.

For instance, if your language is side effect free, you'd have a lot of trouble interfacing foreign functions that have side effects.

Haskell is referentially transparent (i.e. side effect free) but has one of the most pleasant FFI around, IMHO. The FFI doesn't preclude functions with side effects.

I think the bottom line is that users of static languages are not looking for `fast enough', they're looking for `as fast as possible.'

So, that makes me wonder. Perhaps all we need to have one-language-fits-all is a compiler whose optimizations can beat or match hand-written code in a given static language.

In practice, this is pretty difficult, since the development effort of such a compiler is fragmented over all the different dynamic language implementations that already exist.

Of course, I'm biased, because I'm trying to write a compiler for my own dynamic language that can match the performance of static languages, because you can give it hints about what parts of your program you want to make static.

To me, that's the only feasible approach. When a compiler does separate compilation (i.e. creates an object file), it cannot possibly know which portions of the output need to remain dynamic, and which can be optimized as static.

So, the development process should look something like this:

1) Implement your program in the dynamic language (syntax of your choice for each module, since the language should provide an overridable default syntax). Use a rapid edit-run-debug cycle.

2) Now it works the way you want it, so create a new module that just loads your program, specifies enough hints to the compiler to get it to work on the target platform, and invokes the compiler.

3) Test the resulting object, and optimize to your heart's content by providing more hints (such as the fact that a set of objects can't have more methods or slots added to them at runtime, or the fact that a set of functions can't ever be captured by a continuation at runtime, or that a given function can be hand-coded in assembler).

The power of such a system means that the workflow is proper... if you specify optimization hints too early in the development of your package, you have to end up removing or rewriting them when your requirements change. I would find it immensely satisfying to design software that Works Correctly, then have the luxury of optimizing bits of it for a given platform without compromising its design (i.e. the optimization hints are orthogonal to the software's core design).

The other chief advantage of such a system would be for bootstrapping itself: implement the compiler in the dynamic language. To further simplify things, remove all optimization from the dynamic language (like calling convention, data representation, etc). That would make it very easy to write translators from other languages to this new language, since a choice of optimization wouldn't ever conflict with the new language's implementation, and the rest of the source language reduces to optimization hints and runtime modules.

[Meta: This is my first post to LtU. If you're interested in my particular plan for the above-alluded-to dynamic language (Sand) or its compiler (Ocean), please don't hesitate to respond on this thread or write me at gord@fig.org (beware the aggressive spam filter that requires you to confirm your message).]

http://fig.org/gord/

Perhaps all we need to have one-language-fits-all is a compiler whose optimizations can beat or match hand-written code in a given static language.

What do you mean by static? Why such optimizations couldn't be done in a static language?

What do you mean by static?

Probably, my definition of static and dynamic languages is not conventional.

To me, a static language is one that specifies implementation details beyond what the program requires. That is, it requires a certain runtime environment. So, C is relatively static in that it specifies, say, memory layout for structs, a stack-based calling convention, and integer widths.

A dynamic language is one which frees the programmer from implementation details. So, C is partially dynamic in that its runtime model works on both RISC and CISC machines.

Perhaps I should have said `low-level' instead of `static' and `high-level' instead of `dynamic'?

Why such optimizations couldn't be done in a static language?

A low-level/static language doesn't need those kinds of optimizations because the program already assumes them. Code in a high-level/dynamic language has not been optimized for a specific runtime, so the compiler needs help in determining what the runtime's requirements should be.

Thanks for the questions,
Gord.

http://fig.org/gord/

Perhaps I should have said `low-level' instead of `static' and `high-level' instead of `dynamic'?

Probably. Static and dynamic are commonly used in typing discussions/flame-wars. Sometimes people use dynamic to talk about a certain implementation strategy (e.g. REPL instead a compiler).

Why such optimizations couldn't be done in a "static language"?

There are certain optimizations that can not be done in a statically compiled language, because they require the full source and possibly information only available at runtime.

Simple example is a boolean variable in the configuration of a program that enables/disables a simple but widespead feature. In a dynamically compiled language the compiler can throw away all the if's and their bodies.

Nowadays when recompiling a program can take only a tiny little fraction of the run time of the program, it's pointless to statically compile anything. Not to mention all the problems it brings...