Unix as programming language

It has been discussed recently somewhat in The new old..
and Revisiting AWK, but I am finding more and more treatments of Unix shell scripting (actually, not quite "shell scripting", but the combination of all of the Unix mini-languages) as a means to perform complex tasks usually reserved for real monolithic programming languages.

Here is an interesting (intro level) computational linguistics paper (original postscript) (verbatim pdf) about counting trigrams on a decent size corpus using just standard Unix utilities.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.


There's also the fact that Unix utilities (specifically the GNU/Linux versions) are far more robust about bad data than typical one-off programs are likely to be. See the Fuzz work.

What we (I) need is...

We have:

  • Bash, rc, bourne, korn, etc as the job control language
  • Make and its variants as declarative languages
  • awk, tr, sed, cut as text processing mini-languages
  • Unix pipes and sockets as the message passing facility
  • The unix filesystem and random tools for lightweight database work (see NoSQL

We (I) need:

  • A lightweight logic Unix tool (mini-language).
  • A lightweight XML manipulation Unix tool (mini-language)
  • A lightweight constraint solver as Unix tool (mini-language)
  • A lightweight FP Unix tool (mini-language)

Perl took us away from the lightweight Unix mini-languages (it was a monolithic language to replace the shell and lightweight Unix tools). And, most FP language implementations are too heavy to be seamlessly incorporated in the "unix programming language/environment". They have too much startup costs, too many loadable (peripheral) libraries and (currently) are not ubiquitous.


While I haven't actually tried it, from a quick perusal of their website it seems like the goals of Scsh are to provide the kind of FP scripting language you are looking for.

Scsh is built upon Scheme48

Scsh is nice. I first encountered it a few years ago. It's built upon Scheme48 which is VM (image) based. On today's machines it probably starts pretty quickly, but it sort of feels like firing up Emacs versus Vi. (vi = single smallish binary; emacs = large editing environment with lots of support files).

I want to do something like:

cat foo | awk -f parseit.awk | scheme -f complexstuff.scm | awk -f present_it.awk

without a lot of disk grinding or paging!

Is there a small fully R5RS scheme that installs just as a single binary (no libs, no image, no startup files)?

Scheme for scripting

Is there a small fully R5RS scheme that installs just as a single binary (no libs, no image, no startup files)?

The problem with "no libs" is that unless you build a custom binary which contains the libraries you need, you're going to end up missing lots of useful functionality. However, it ought to be possible to build a custom interpreter binary with Chicken that includes the libraries you want. Bigloo and Gambit are also possibilities.

For that kind of work, I usually use MzScheme, but I just make sure the libraries I need are installed. I don't see a problem with that - that's how Python and Perl work too, after all.

Gauche is another implementation with somewhat of a Unix scripting orientation.

ah, but I want to "useful functionality" elsewhere

I want a stripped down FP or logic language because I want Unix to supply all of the unixisms (and posixisms). Part of the brilliance of awk is that it DOESN'T do a lot outside of string processing. It can't do directory listing, file globbing and other things already provided by other unix commands.

I've got a need to do a bit of natural language parsing. I've got a pipeline of unix commands that deals with email. It downloads it from a pop3 server, splits out the mime parts, extrapolates commands from some embedded text and responds accordingly. Figuring out the commands (part of the subject line and possibly text/plain body) requires a bit of natural language processing. I'd rather do that in a programming language that is better fit for that task.

So, I have something like:

pop3download | extract_headers | split_mime | tokenize_text | figure_out_request | generate_response

I want to do the "figure_out_request" part in something like Scheme or Prolog where I can take advantage of its richer symbol manipulator capabilities. The libraries I will need (If I do need them) will be domain (NLP) specific...

Chicken as a unix command line utility?

An out of the box compiled chicken interpreter on cygwin yields faster startup time (using statically linked chicken binary) than perl. Nice. Very nice. It's actually on par with gawk. I haven't done anything interesting with it yet, but I am excited about its potential. (Maybe I can let gawk go back to what awk is good at and use chicken scheme for manipulating my tokenized text!)

Thanks for suggesting Chickenm, Anton

Brutal Scsh parody

Read this if you can handle it!

is that really a parody?!

is that really a parody?!

I think that he calls it a

I think that he calls it a parody because it doesn't look easy to work with..

A language which replace "echo hello" by "(run (echo hello))" is on a slippery slope of being unreadable as the rest of the article quite clearly demonstrated: scsh will probably only used by people who are fans of functionnal programming, the other will keep using the classic Unix tools.

Except that you would really

Except that you would really use (display "hello").

The case for scsh is not to replace the interactive shell. However, for writing more complex scripts than you would typically type at the command prompt, there are definitely benefits to scsh in terms of power & maintainability, compared to shell scripts.

An article like the one Luke linked has the disadvantage of having to teach most readers an entirely unfamiliar syntactic & programming paradigm, plus a powerful library for interacting with Unix, all at the same time. That's what leads to the "brutality".

there are definitely

there are definitely benefits to scsh in terms of power & maintainability
s/scsh/Java/ could be argued just as well, but in both cases the actual code I've seen suggests the opposite. For example, the shell code in the article is much nicer than the Scheme. The few scsh scripts that I have written (like untar) look like abominations of verbosity in hindsight (as do my Java programs).

There is an element of (CDR (ASSQ KEY ALIST)) in this, though.

P.S. I would still like to try scsh as a high-level unix system programming language instead of C.

What Abominations?

I'm curious how you think the code could be improved. I don't know Scsh very well, but in PLT Scheme there are some libraries that would improve the code. Specifically, I'd use pattern matching more, and I'd use some utilities in the file.ss module in Mzlib (such a file-name-from-path and make-temporary-file).

quick and dirty

shell scripting mean quick and dirty tasks execution. Most are one-liner pipes written in the shell prompt. I don't think, for instance, that Lisp's parenthesised syntax helps a lot here, specially as most command-line editors don't support parenthesis matching. And certainly one more level of indirection -- like in (run (cat "foo.txt")) -- will never be any popular for quick scripting...

There's a saying, "Use the right tool for the job".

Of course, more elaborate scripts would probably benefit from hygienic-macros, lambda abstractions, lexical scoping and elaborate list processing... the kind of scripting that is generally a messy plain block of repetitive loops and conditionals generated by automated tools. :)


I just ported my untar program from scsh to bash. I don't like the original but it is six years old so now it seems unfair to compare the results. I post this script anyway (it's pretty handy).

# Unpack a tarfile and make sure that it ends up in a single directory
# with a name derived from the tarball (create this if necessary).
# Example: "untar foo-1.0.tar.gz" will always put the files in foo-1.0/
# Written by Luke Gorrie <luke@synap.se> in February 2006.

# Setup

basefile=$(basename $file)
# we will make sure everything goes into this directory
wantdir=$(echo $basefile | sed 's/\.tar\.gz\|\.tgz\|\.tar.bz2//')

# check args
if [ $# != 1 ]; then
    echo "Usage: untar "
    exit 1
elif [ ! -r "$file" ]; then
    echo "abort: file does not exist or not readable"
    exit 1
elif [ -d "$wantdir" ]; then
    echo "abort: $wantdir already exists" >&2
    exit 1

# detect compression scheme
if echo $basefile | grep -q '\.tar\.gz\|\.tgz'; then
elif echo $basefile | grep -q '\.tar\.bz2'; then
    echo "Unrecognised file format" >&2
    exit 1

# Extract & move & cleanup

trap '[ -d "$tmpdir" ] && rm -rf "$tmpdir" ]' SIGINT
mkdir "$tmpdir"
tar Cxf${compression} "$tmpdir" "$file"

if [ $? != 0 ]; then
    # tar failed
    rm -rf "$tmpdir"
    exit 1;
elif [ "$(ls -1 $tmpdir)" == "$wantdir" ]; then
    # The archive unpacked the way we want
    mv "$tmpdir/$wantdir" .
    rmdir "$tmpdir"
    # "Messy" unpack. Put it under the desired directory.
    echo "untar: creating $wantdir"
    mv "$tmpdir" "$wantdir"

Bash vs Scsh (for simple scripts)...

I can see by your example how much more cumbersome scheme scripting can be... a lot of verbosity in that scsh version.

Okay, I couldn't resist. I've taken your bash untar and compressed it further. My obfuscation of your script is here.

I've grown use to writing dense shell scripts. I find that brevity makes shell scripts more readable (hint: I compressed the awkward shell syntax for if-then-else as well as used some bash-isms to drop dependence on other sed/basename/etc).

Oh, and... neat tool Luke!

Nice program!

I'm suitably impressed :-)

Transliterating bash to Scheme

All this talk amongst the Philistines about "cumbersome scheme scripting" is getting to me. I can't help thinking that Luke has just been dying for someone to convince him that he's wrong about s-expression based scripting. So here's my attempt to do that, for MzScheme:


(define (main args)
  (match args
    ((list _ file)
     (match-let* ((basefile (file-name-from-path file))
                  ((list wantdir ext) (split-filename basefile)))

       (or (file-exists? file) (die "abort: file does not exist"))
       (if (-d wantdir) (die "abort: ~a already exists" wantdir))

       (let ((compression
              (case (string->symbol ext)
                ((tar)        "")
                ((tar.gz tgz) "z")
                ((tar.bz2)    "j")
                (else (die "Unrecognized file format"))))
             (tmpdir (or (make-temporary-dir "untar.~a")
                         (die "Can't mkdir ~a" tmpdir))))
         (try-pk (exn:break?
                  (if (-d tmpdir)
                      (run (rm -rf ,tmpdir))))
           (|| (tar ,($ 'Cxf compression) ,tmpdir ,file)
               ((rm -rf ,tmpdir) (exit 1)))
             ((equal? (run (ls -1 ,tmpdir)) (list wantdir))
              (&& (mv ,($ tmpdir '/ wantdir) ".") (rmdir ,tmpdir)))
               (printf "untar: creating ~a\n" wantdir)
               (run (mv ,tmpdir ,wantdir))))))))
    (else (die "Usage: untar filename(.tar|.tar.gz|.tgz|.tar.bz2)"))))

This relies on some generic support routines, like run, to do some scsh-like things. A version complete with comments, syntax highlighting, and all the necessary support code can be found at ProcessUntarScriptExample in the Scheme Cookbook. The scsh-like features are documented (briefly!) in the source code for the support routines.

This code is more likely to win an award for obfuscation than readability, but it's not too different from the "compressed" bash version in that respect. I wanted to keep it as close to the bash version as practical, while still using Scheme features instead of shell features for everything except actually executing external programs. I even went so far as to (define -d directory-exists?), more for fun than anything else.

I even went so far as to

I even went so far as to (define -d directory-exists?), more for fun than anything else.

Yes, that was fun...

Good, but

I can't help thinking that Luke has just been dying for someone to convince him that he's wrong about s-expression based scripting.

Stranger things have certainly happened on LtU :-)

I like your example. If it were a stand-alone scsh program then I would concede the point, but really I have to count the several-page ad-hoc scsh implementation towards verbosity :-)

Notes: I don't like the need for string->symbol, that seems very low-level to me. I also just noticed a bug that stops these programs from working on the foo-1.0.tar.gz filename in the comment (just mentioning).

AST tools

I know it should prove to be pretty cool if we had mkast, asted and ast2src, little utilities to do arbitrary AST processing.

$ mkast source.c | asted -e 's/foo/bar/gi' | ast2src -pp k&r > source2.c

makes an AST from the source code, substitutes (g)lobal (i)dentifier "foo" for "bar" and converts the ast back to source code using k&r pretty printer... yes, it would leave local identifiers "foo" alone, just change the references for the global one.

$ mkast source2.c | asted -e 's/(Person).surname/$1.last_name/' | ast2src -pp k&r > source3.c

This should change the definition of the field "surname' from (.) struct "Person" to be "last_name" and accordingly to propagate this change throughout all the AST, in all references for surname. Just a mere text substitution like sed does wouldn't cut it.

of course, by now you realize asted acts like sed. ;)

how about it? Gosh, i would kill for such tools and vim and emacs integration... :)

Good idea, but...

I know it should prove to be pretty cool if we had mkast, asted and ast2src, little utilities to do arbitrary AST processing.

Unfortunately, this is where the text-based nature of the Unix pipeline starts to stretch at the seams a little. Since in order to be "well-behaved" from the traditional Unix point of view each of these tools ought to accept and emit plain text, we'd need a standard serialization format for these ASTs. Of course, such a format already exists: the C programming language. Establishing a more machine-friendly interchange format would be pretty easy, but it comes at the expense of interoperability. And since interoperability is really the whole point of the pipeline approach, that's a pretty heavy price...

This is one reason why this kind of tool typically becomes pretty monolithic in Unix. Attempts to allow typed data pipelines (as in Microsoft's new shell) are a great idea, but bringing this to Unix would probably require establishing a new data processing hegemony and re-implementing ls, ps and their many friends, which of course flies in the face of Unix culture in many respects.

I'd really like to use a shell with these capabilities, though, and as Microsoft will probably push us in this direction, I think we'll see more of this kind of thing.

By the way, for a step in this direction, you might be interested in XMLStarlet.

not necessarily

"Establishing a more machine-friendly interchange format would be pretty easy, but it comes at the expense of interoperability. And since interoperability is really the whole point of the pipeline approach, that's a pretty heavy price..."

I don't think the price would be that high as long as the interchange format remains textual. It doesn't even have to be structured xml, though i believe a W3C-sanctioned xml format for ASTs would greatly speed up adoption. It certainly shouldn't be C code in this day and age.

"bringing this to Unix would probably require establishing a new data processing hegemony and re-implementing ls, ps and their many friends, which of course flies in the face of Unix culture in many respects."

as long as its text, there should be no problem. It should be no problem if only mkast, asted and ast2src were the only utilities to truly understand the underlying format. sed or awk do not understand what is passed to them, they just do simple textual substitutions. If there are no tools for general-purpose structured processing, i think it's easier to just add new ones than making the old behave likewise.

"By the way, for a step in this direction, you might be interested in XMLStarlet."

Yes, that's a nice start as far as xml processing from the command-line goes. Now we just need a standard xml format for ASTs... the java guys have such one in one of Eclipses plugins, but they don't mind command-line processing anyway... :/

Constraint solver?

I just curious here, what would you like to use the light-weight constraint solver for? I can't really imaginge when one would use such a thing.

adhoc calendar scheduling...

Lightweight may be too strong of a word ;-) I am thinking of a very specific application that involves scheduling/rescheduling around a meeting calendar (where the primary interface is email -- but that's a different topic!)

As part of a unix pipeline workflow, I would like to be able to feed the solver meeting requests that may conflict with an already populated calendar and have it propose scheduling solutions.

I admit that this is a rather specific need ;-)

I probably should have said a I was hoping for a "lightweight" programming/scripting language wrapped around a constraint solver library. I don't want the weight (from a Unix utility perspective) of an Alice or Oz. I haven't been keeping up with Gecode, so I don't know if it has been bound to any "scripting capable" language yet.

A potentially really bad analogy would be that I want a troff, not a TeX.

Now I understand

Ok, now I get the picture.

What one would want in your application is good abstractions for scheduling requests, so a model could be written cleanly. On the other hand, it sounds like you have quite a static problem (although the instance-date is not), and then a C++ program would do the trick, I believe.

There is no lightweight wrapper for Gecode (yet), Java being the only language other than C++ that we are currently looking at.

Meaning of "Lightweight"

What do you mean by "lightweight"? I usually see that word to describe incomplete projects that don't fully implement their specifications. i.e. projects that strongly favour NJ-style (aka 'worse is better').

Searching through Freshmeat provides a lot of examples.

My definition of lightweight

I mean something that is not a world unto itself. It should have a smallish footprint and very quick startup time (as it may be invoked many times as part of a pipeline). It doesn't need networking capabilities, regular expression parsing, etc because there are other tools available that do that well.

By no means do I want incomplete!

Regarding "worse is better". I am starting to appreciate the NJ style more these days. In particular, I am more and more impressed with what NJ (Bell labs) did themselves, not the often "pale" attempts at that style by others. Bourne shell, C, Awk, troff, etc were very high quality tools that were written to fill specific needs. They were fed by a healthy dose of research with a bias toward practicality. (Plan 9 continued that tradition). I miss the quality and depth of thought that went into these Bell lab tools in a lot of the stuff produced today. Today we have a lot of bloated software that wants to do everything.

I've been scouring the net looking for a lightweight Scheme (something that doesn't feel the need to include the kitchen sink because the kitchen sink -- under Unix -- already exists!).

Chicken Scheme (the interpreter) looks really, really interesting to me because I can jettison a lot of additional stuff and add in what I need. It starts up really fast and it appears that if all I need is the interpreter binary (along with my Scheme scripts), then that is all I need to deploy.

But, what use is a lightweight Scheme (something that just implements R5RS)? Well, I've been happily passing flat streams of data through my unix pipeline with Awk and other tools. But, now I have a need to manipulate more complex data structures and relations. The end result is once again a flat stream of data, but I prefer to use something good at manipulating nested lists and trees... sounds lispy, eh?

Re: smells lispy

Hear, hear. Maybe blame XML - but even before that, if one were dealing with records in files I think the line-by-line nature of the unix utilities really limited what could be bung'd together. That's why Perl met with such success; it lifted what people already knew from shell programming into a more stateful world that could handle "ok, I'm in a record now, do stuff once the record is fully read in" that wasn't doable with piped things on the command line.

lightweight fp?

It looks like one needs to have a haskell to bootstrap it, so I don't have any idea how well it works, but this at least sounds interesting:

h4sh provides a set of Haskell List functions as normal unix shell commands. This allows us to use Haskell in shell scripts transparently.

with an example given of:

cons "str" f | take 4 | map sort.reverse | hd | i | filter '== "str"'

A lightweight XML manipulation

We (I) need:

  • A lightweight XML manipulation Unix tool (mini-language)

Have you tried XLMGawk?


i've found that i'm doing

i've found that i'm doing more and more in bash rather than python - it's more closely integrated with the various utilities i want to call. my 'blog is implemented with a dozen bash scripts (ok, it's not very complex, but it has threading, templates, user comments, an atom feed, can be rebuilt from the "database" (maildir directory) etc).

one reason for the shift is that i discovered the advanced bash scripting guide (though i seem to be using an ancient version...!). i also thought a fair bit about what the art of unix programming had to say.

is this another case of worse is better?

I had to maintain a quite

I had to maintain a quite complex program written in shell scripts, and above a certain level there are many downside of shell programming which appears: portability problems, silent errors, etc..

There's a reason why Perl has replaced the shells for programming in the large, I dislike Perl but Ruby or Python are nice, I'm a bit surprised that you prefer bash to Python..
Now it's true that script language are not really good for doing 'shell scripts' programming, unfortunately..

complex shell scripts considered harmful

Anytime I see a (bash/bourne/etc) shell script with lots of functions or lengthy switch/case statements I begin to worry. It is definitely not the strong suit of shell scripting. However, this is where the mini-languages (and utilities) come into play. Good shell scripting delegates complex code to better suited languages.

Perl's evolution is quite interesting. It could be used to replace shell scripting (it offered richer controls, richer data types, etc) and (in the early days) you would seamlessly call unix utility programs through backticks and other syntatical sugar. But, then something happened. More and more unix command functionality was being built into Perl (and more Perl libraries were being created) until it became rare to see Perl calling unix programs directly.

Python and Tcl took it further by building abstractions upon the unix calls so you couldn't tell (or care) that unix was underneath. With that, a lot of capability and lore was lost.

Now, portability, that drove the final nail in the coffin. What is the one true Unix? These days I stick to GNU utilities for scripting.

i wasn't really arguing that

i wasn't really arguing that i'd want to write big programs in bash. i think the point is more that if you go with "the unix way" - leveraging existing functionality - even quite complex programs can become fairly trivial shell scripts.

of course, it's not something i'd want to supoort if other people started using it, and it's not something i intend to extend with piles of extra fucntionality. but something else i'm trying to avoid is over-engineering and worrying about out0guessing needs for future expansion in what are, to be honest, stupid little things that only i use...

and if that souds like i shouldn't be doing bash scripting work, well, i'd like to assure you i'm not, except that (1) i just did for a one-off job (converting our build process to maven) and (2) many of the same arguments apply to agile developemnt, which we're trying to move towards.

here's an example i wrote

here's an example i wrote recently. it recursively descends a directory tree adding files to cvs.

pushd "$dir" > /dev/null
  echo -n "adding files in "
  cvs add *
  for subdir in `ls -dp * | egrep "/$" | egrep -v "CVS"`
    "$0" "$subdir"
popd > /dev/null

...and i in the context of this thread i noticed that:

  • recursive tree traversal is about as bread-and-butter as you get for functional programming. yet clunky old bash handles this with surprising elegance. would scsh help here? the main uglies i see are (1) listing directories only (i suspect there's a better way than the hack i used) and (2) the obfuscated self-call.
  • the obvious extension to the above is to extract the operation done in each directory (and perhaps the filtering to select directories). it seems to me (without trying it) that eval would work just fine.
  • the next extension, however, would be to include state - a fold across the directories (counting certain files for example). at this point i think bash fails miserably - i assume scsh would be way better, since i guess it has closures.

but does anyone shell script in this way (writing folds on directories, for example)? if not, is it because of the tools, or because it's just not worth the effort? and is there some sweet hack that gets me clousres in bash?!

Count me in

I do that sort of scripting all the time, except I write it in Scheme. Example is recursing down directories generating svn logs (svn log doesn't recurse by itself). In PLT Scheme fold-files [To use: (require (lib "file.ss"))] is your friend.

Try using "find" to unfold

Try using "find" to unfold your filesystem data, producing (but not necessarily realizing) a nice flat list of files on which to operate. For operations like "cvs add", try using "xargs" as a currying/vectorizing consumer.

If you need not even explicitly iterate, why bother recursing?

:: :: ::

and, apropos another couple of LTU threads:

jot 1000000 1 | awk '{s+=$1}END{print s}'

which, on my system at least, is far more practical than a theoretical alternative:

{ echo "0"; jot -w "%d +" 1000000 1; echo "p"; } | dc

actually, i was recursing

actually, i was recursing because somehow i got in a mess with cvs and find. i don't know if i screwed up, or if it doesn't like directories in the file name, but i ended up clearing out the repository and starting again with that script. although i guess i could have used a more sophisticated argmuent in find that combined a cd and a cvs add. hmmm.

ah. finally i understand this comment - http://lambda-the-ultimate.org/node/1231#comment-14197 - i'd forgotten exactly how awk works. thanks. i like the idea of find as unfold. it's a nice way to explain things to people new to functional programming.

can you call the shell from within awk? i guess so. hmmm....

is "jot" the editor?

ah, and http://www.network-theory.co.uk/docs/cvsmanual/cvs_67.html suggests paths are not acceptable


Jot is a BSD utility, similar to APL iota, for generating sequences -- including (now that I've reread the man page) infinite sequences, so one can have the tail of the pipeline, instead of the head, control termination: (fold with early exit).

jot 0 | awk '{s+=$1}NR==1000000{print s;exit}'

xargs as poor man's map

I haven't thought about xargs in years. It can be used as a poor man's map for infinite streams. In bash, try:

$ alias map="xargs -n 1"
$ echo /pathA/f1 /pathB/f2 | map wc -l

which will give you the number of lines in f1 and f2, like:

212 /pathA/f1
23244 /pathB/f2

Even more useful(?):

alias car="awk '{print \$1}'"
echo /pathA/f1 /pathB/f2 | map wc -l | car

Which will give you just the count for each file.

Dave, who seems to be

Dave, who seems to be posting anonymously, wrote a shell script that constructs and queries an inverted index of text files in order to do a full-text search. Very impressive.

write now, install forever...

It's really too bad there isn't a 'CPAN' or 'rubygems' for Bash. Actually, the lack of a solid package manager has kept me away from Python as well. Working from the shell is a great way to prototype, and as my Bash powers have increased I am much more efficient on a day to day basis; but I would really hesitate to write an 'application' in Bash and ship it to a client as a tarball of functionality that goes 'somewhere'.

Lisp vs. short-term memory

What's really nice about UNIX pipes is that every time you get to another pipe character, if you have in your mind what the stream of data looks like at that point, you can forget about what you've typed so far and just concentrate on the remainder of the process. That quality makes it easy to read, understand, and debug.

There may be a limit to what you can do in a strict linear form like that, but lisp-like languages are the worst in that regard. All those parens are beautiful if you're a parser, but if you're a person they wreak havoc with your short-term memory. See this wikipedia article for a little discussion of that in the context of natural language.

Obviously there is a place for languages with lots of brackets and nesting, but IMO they aren't ideal on a command line. (However, as a self-contained step in a pipe, as in Todd's examples, I have no problem with that)

Saving the mental stack

That's why my language Kogut has a particular syntactic sugar: x->F y z means F x y z, where x may contain further unparenthesized arrows itself.

This leads to pipeline-style syntax for nested function application where function names are written in the order they are executed:

NumberMoves plans->Reject ?(_, plan) {plan %Is Null}
->SortBy ?(_, _, states) {BestState states} IsWorse
->Each ?(x, oldDepth, oldStates) {

This works best when the primary object of every operation is the first parameter. Kogut indeed tries to order parameters that way, and at the same time any functional parameter tends to be the last one.

Compare with the equivalent, which assumes the order of function parameters being more typical for functional languages:

Each ?(x, oldDepth, oldStates) {
} (SortBy ?(_, _, states) {BestState states} IsWorse
   (Reject ?(_, plan) {plan %Is Null} (NumberMoves plans)));

Stacking your thoughts

I can live quite happily with

let x = f ...
    y = g x ...
    z = h x y ...
in ...

to stack my thoughts and "forget" about what I typed after I bound the result to a variable.
On the other side of the spectrum, if somebody does not want to invent names for intermediate results, there is always FORTH. We can write


and only have to remember how the stack looks like. Personally, I find meaningful names helpful when returning after some time to code I wrote.

plt vs plp

in theory, the linear nature of pipelines more directly expresses the associativity of the command line monad — nesting overspecifies.

in practice, it's probably more important that an interactive exploratory session incrementally building a pipeline generally involves no more than concatenation, or at worst a single replacement — is needing nesting a sign that what one has to say is too complex for a command line phrase, and ought to be scripted?