Lambda the Ultimate

PowerShell is obviously related. I remember reading a very interesting document about the PowerShell design from a programming language point of view. All that I could find is this less on-topic message from a developer. Z'Bo certainly knows much more about this.

Your mention of a "general graph structure" made me think of "workflow systems". I'm quite ignorant of these and other will probably come up with better references, but I learned about them in this presentation of VisTrails, a tool meant for flexible scientific computation and visualization workflows. Note that this is using a graphic language rather than a textual language; my intuition is that, for not-too-complicated not-too-automatizable situations, representing graph visually is a better user interface than a label-based textual system, but might also be less productive for one-shot small workflows.

Some related LtU threads (some other threads may be relevant as well, but didn't generate any discussion):

• New Dataflow Programming Language
• The Design and Implementation of a Dataflow Language for Scriptable Debugging
• F3: New statically typed scripting language for java
• Computer music: a bastion of interactive visual dataflow languages

scsh

Scsh has two main components: a process notation for running programs and setting up pipelines and redirections, and a complete syscall library for low-level access to the operating system

Es: A shell with higher-order functions

If this is for writing scripts that interact with the complicated logic of a server (daemon) or simulator or some such, you can add some glue logic to make its functions callable from an scripting language (python, perl, scheme, tcl, ...). That way you don't have to reinvent everything + you can take advantage of existing libraries, dbms etc, should you need them. For example, you can write a cgi script to drive a web interface.

For interactive use a simple CLI is easier to use but that can be built in the scripting language. A friend once did this to aid hardware engineers in their debugging. He added functions to peek/poke hardware registers + other low level hacks, to mzscheme. He then wrote a simple REPL in Scheme so that users didn't have to type the outer most parentheses + a bit more for more convenient syntax. This was good enough for the engineers. And they didn't mind typing e.g. (+ x y) to add two numbers -- this was more than the CLIs they were used to provided!

Now if you are just using this project as an excuse to write the shell you always wanted, you can ignore the above!

The strength of the /bin/sh ecosystem is that it is easy to tie together most program in interesting ways since they accept and produce plain text. If you replace text with typed or structured data, you'd have to reimplement all the standard tools. Still, if using standard tools is not relevant, you can probably use s-expr as the format of communication.

Handling general graphs in a linear language is painful. If you go down that path, consider a GUI. Providing an easy to use scriptable GUI can be an interesting project by itself.

While general graphs can be difficult to use, bash goes a long way with trees. Take a look at what you can do with subshells and redirection.

I have a new shell language deep down in my TODO list as well. Personally, I think that (ba)sh got more wrongs than rights, but somehow it's still fantastically productive to work with, even for non-interactive use. The problem is to fix those flaws and still have a script language that still works well...

Here's some of the things that I want to fix:

• static typing
  • extensive type inference
  • extensive implicit type conversions
• avoid tokenizing errors (like spaces in file names)
• error handling
  • actually have error handling :)
• unit testing
  • lots of issues here
• module systems, get away from the old source / . Needs to support partial imports and type compatibility/inference on the API level.

I also have a dream to get away from VT220 terminals (but still have a terminal), and away from the monospace fixed height fonts/layout and syntax limited to US-ASCII... would still require a vtXXX emulation layer, and no matter how artificial the term is, the source files would end up as "binary" as opposed to "text"... I think I'll file this dream together with the "shift the earth axis" one...

This is a subject I love to muse about. But as of yet, very little action :) A common affliction to visitors of this place I guess.

Anyhow, one thing I seem to be missing in your list, is event oriented programming. In contrast to streams, which have been ``solved'' in unix/sh, reacting to (asynchronous) events is quite a bit harder.
A use case would be the launching multiple jobs in parallel, with graceful handling of failures.

Could you expand on ``type directed parsing''?

Are you aware of termkit? It's a really interesting attempt to improve the shell using modern UI.

Shok

A shok session is a voyage along two parallel modes: the command-line and its companion programming language.

[..]

A { at the start of a command-line swaps you into a block of code.

shok:~ : {
shok:~ >     new x = "Cool!"
shok:~ >     print("This is a code block. " ~ x)
shok:~ > }
This is a code block. Cool!

[..]

In code, use : to make a single command-line statement:

shok:~ : {
shok:~ >     : mkdir foo
shok:~ >     : echo "hello, world!"
shok:~ > }
hello, world!