Lambda the Ultimate

OK. I'm faced with a bunch of decent choices for a problem at work, and it might be a general enough problem to be of interest to others.

I have a program that needs to scan through an annoying log file to pick out and reassemble individual low-level events into "operations" -- one action taken by a user or administrator might result in a large number of low-level events, but we're really interested in what the user thought he or she was doing, hence the reassembly.

So far, I've made the log interpretation work fairly well by "pretty-printing" various event and attribute names using the terminology users prefer, rather than the system's names. Now, the customer wants to go further: I need the code to check for certain combinations of attributes within the events and then label the operation records appropriately.

I can do this in code. I could do this sort of check efficiently if I wanted to add a rule-based inference system, but I fear that this is more solution than is needed or desired. For various reasons, I don't want the customer writing code, but I need good performance (there are a *lot* of operations per day).

In rough order of increasing complexity and power:

• Simple expression language, like JXPath
• "Full" expression language, like the EL from the JSP spec.
• Object graph navigation language (OGNL), which includes a limited lambda function capability
• Inference engine, hopefully with a Rete implementation (e.g. Drools)
• Java-based little language, like Groovy or Jython
• Embedded Scheme interpreter, with some sort of user-friendly DSL.
• Java "fragment" compiler (Janino or some sort of direct byte code writer)

So, at the start of the list we have a bunch of expression languages, limited, but decent at expressing relationships between parts of an object tree (the operation and its associated attributes). In the middle, we start seeing rules as a DSL, and at the end, we get full languages shoehorned into a seemingly simple log interpretation program.

I'd like to keep this program fast and relatively simple, while providing the maximum flexibility to users creating rules or expressions to filter out the interesting events. I'm wary about full languages -- too powerful, possibility of writing code with side-effects or thread issues -- but I don't know whether ordinary expression languages will do the trick. The middle alternative of defining a limited language that can be analyzed or checked for problems without being run sounds appealing.

Thoughts? Examples? References?

The examples alone were enough to make me download and try out Frink; the fact that it is also a quite useful, and practical language helped a little.

New in this Release

This is the first stable release in the 1.9 series. It contains numerous enhancements to both the engine and library.

• The engine now uses a flat closure representation supported by a referenced variable analysis pass which makes SISC "safe-for-space" for most programs.
• I/O is now based on generic procedures, allowing extensibility from Scheme and Java.
• Support for trapping OS signals.
• Support for breaking evaluation at the REPL with Control-C.
• Support for a form of primitive inlining when directed, lending a 2x increase in speed.
• Added a call to retrieve the exports of a module as a list.
• Support for the latest SRFI 45.
• Added the "os" module, which facilitates spawning and interacting with external processes.
• Immutability of sublists in quasiquote expressions now preserved.
• Better handling of infinites and NaN values in the numeric library.
• Serialization of Scheme values is now differential against loaded libraries, resulting in smaller serialized forms.
• Better error reporting from nested loads and from the reader.
• Numerous bug fixes.

About SISC

SISC is an extensible heap-based interpreter of Scheme running on the Java VM, with an aggressively optimized, lightweight (<200k) Scheme engine. SISC outperforms all existing Java interpreters (often by more than an order of magnitude).

In addition, SISC is a complete implementation of the language. The entire R5RS Scheme standard is supported, no exceptions. This includes a full number tower including complex number support, arbitrary precision integers and floating point numbers, as well as hygienic R5RS macros, proper tail recursion, and first-class continuations (not just the escaping continuations as in many limited Scheme systems). SISC also attempts to implement the standard as correctly as possible, while still providing exceptional performance.

Finally, SISC provides many useful real-world extensions, such as networking, threading, elegant exception handling, generic procedures, an object system, SLIB and comprehensive SRFI support, a scope-friendly module system, a Scheme and Java object system with a clean foreign-function interface and more.

Downloads and More Information

Source code, binaries, and SISC documentation can be found on the web at:

http://sisc.sourceforge.net

Licensing

SISC is Free Software. It is released simultaneously under the GNU General Public License (for free-software projects), and the Mozilla Public License (for commercial entities). The documentation is available under the GPL.