Lambda the Ultimate

Here is an introductory article about a programming language with extensible syntax, based on a mixture of PEGs and classic Lisp macro metaprogramming.

A preliminary version of the language compiler is also available (it needs Microsoft .NET Framework or Mono to run, syntax highlighting editor works on Windows only).

2009 Lang.NET videos are available here

Some of the speakers include:
Lars Bak (Google Chrome Javascript VM)
Gilad Bracha
Anders Hejlsberg
Erik Meijer
John Rose (JVM)
Philip Wadler

Although my motivation is financial systems, I use the example of computer games to make the discussion appeal to a wider audience.

Assume there is an on-line video game which has tens of millions of users. The game is some sort of strategy game with armies, battles, shootouts, etc. Although most players control everything manually, the game provides a way for players to script their actions. For example, a script could tell three soldiers to spread out in a triangle whenever they see a certain monster. Another script could have the workers automatically produce new soldiers as old one die out.

In this scenario, it makes sense to provide a proprietary, domain specific, scripting language that lets players cause all kinds of mayhem with virtual guns, but doesn't allow formatting the local disk drive.

As the game becomes more popular, some players demand more features in the scripting language. They wish to do monte-carlo simulation to figure out the possible outcomes of an attack. They wish to buy third-party scripts and integrate them into their own scripts, etc.

The game writers decide that they should create a general purpose language. They write their game engine in this general purpose language and accept scripts in the same general purpose language. This allows a community to grow around the language, with nice tools, libraries, compiler/interpreter optimizations, etc. However, this language must have the ability to control what these 'injected' scripts can do. Obviously these scripts can't modify the game engine, itself, they can't access the local disk, they can't create data-structures that eat up all the memory, they can't hog all the cpu, etc.

What kind of language is this? What real-world examples are there that do this or what papers describe such systems?

To partially answer this: the most obvious example is Java and its applet infrastructure. Applet writers embed their programs in web-pages, and don't generally have the ability to access random parts of the local disk, nor do they have the ability to open any tcp/ip ports. Applets, however, don't describe how they can be embedded into a general purpose server--their interaction is strictly defined for a browser and nothing else (as far as I know).

Lisp/Scheme system provide the ability to change running code. The clojure folks have an amazing (to me) example where a programmer telnets into his server and changes a servlet as it is serving pages to users. Other languages provide the ability to interact with the server: Erlang and Java (via osgi) come to mind. However, these interactions don't necessarily happen in a safe enough manner. They are certainly not designed to host little programs from millions of users whose competence and ethics may vary.

Haskell provides a way to keep pure code separate from code with side-effects. Perhaps the server only accepts pure Haskell code, thereby ensuring users don't do any I/O. However, Haskell also desn't have a good way of 'dynamically loading' code (hs-plugin is not very pleasant to use). I may wish to stop users from doing disk I/O, but they shouldn't have to jump through hoops to generate random numbers or mutate local state.

It seems to me that the right language to allow such a system would have the ability to control effects: but do so at a very granular level: permission for non-deterministic code, mutable state, I/O, etc. need to be controlled separately. How would one ensure that CPU time or memory is used fairly? How could the server writers make sure use scripts actually terminate (no infinite loops or recursions, beyond the global event loops which exists as long as the game continues)?

Can type/effect systems solve this? Is this better handled by virtual machines or run-time checks?

Other examples where such a system can be useful:
-Financial exchanges which let people submit little programs, rather than single commands to buy/sell securities.
-Car computer systems which let drivers embed their programs to enhance the dashboard
-General programming languages which let users provide their own implementation of interfaces (such as Comparable in Java), but ensure that these implementation don't have access to disk/networks, etc.

I think that this is a very well written paper about the challenges facing the computing and software worlds at the beginning of the 21st century. There are two on line versions: a word version and an HTML version.

FringeDC is a group in Washington DC interested in functional and fringe programming languages (Lisp, Haskell, Erlang, Clojure, Prolog, etc.)

www.lisperati.com/fringedc.html

In this presentation, Philip Fominykh will be presenting readtables, MOP, persistency of image, FFI, readtime/runtime separation and similar Common Lisp subjects. The presentation will be geared towards non Lispers, so don't be intimidated! The meeting is generously hosted by Clark & Parsia (tinyurl.com/6wmmbj) located at 926 N St NW rear Studio #1 Washington DC and is near the Convention Center Metro Stop.

Map: tinyurl.com/7mbc4o
(I recommend checking the Clark & Parsia website for detailed directions to the office)

Anyone is welcome to join our meetings!

An old saw: IIUC, a problem faced by programming languages is that there are multiple dimensions which ideally could all be serviced, but in the end in 2D ASCII aren't. The "expression problem" is a classic example thereof. I am wondering what the current state of the art is / new promising directions people have seen?

For me, a big part of the problem is that "it depends". When I'm writing code it doesn't bug me to have it split up, but when I'm debugging code I'd rather see things "linearized" rather than have to chase through inheritance relationships to figure out the total gestalt. What underlying representation could we have to allow us to view the code in different ways?

Subtext (RIP?), CoreTalk (ok, that might be impenetrable), what else? Are we just doomed as long as we're still using ASCII? If so, what is the least offensive approach? Generic programming a la STL? Multimethods? How can we ground the investigation: what metrics tell us which approach is best / does the least harm?

A new candidate specification for EcmaScript (formerly "3.1," now "Fifth edition") has been completed.

See http://www.ecma-international.org/news/PressReleases/PR_Ecma_finalises_major_revision_of_ECMAScript.htm

or

http://www.ecma-international.org/publications/files/drafts/tc39-2009-025.pdf for the draft spec itself.

*quick note: I don't have much formal training in PL's, so if I'm wrong, please correct me. For reference, I know python, C++, C, JAVA, OCAML, LISP, prolog (in that order of familiarity).

The quick one liner:
Why are objects that we use in programming so vastly different from real-world objects?

The slow multi-liner:
Human languages have the ability to talk about the state of the world, and all of them contain nouns, which rather than describing a particular thing, they are supposed to indicate a set of things that contain a certain property. Cars can be driven; boxes are containers; speakers produce sound. People from different cultures that have different languages, even with a difference in nomenclature, typically have common words that describe sets of things the same way. In addition, modern science has found mathematical relationships between classes of things with precision. All this to say there is such a thing as "real-world objects," it is not arbitrary (and suggests that there is a right way and wrong way of describing things.)

Most programming languages have a concept of objects, and allow programmers to define them and describe relationships between objects. The idea of objects helped programming with type-checking, encapsulation, code reduction, etc. In fact, an argument can be made that the job of the programmer is to figure out a representation (real-world object -> code -> binary data) of a problem and a process to solve the said problem.

Yet even with the amount of programs being written and problems about objects being solved, rather than converging on a complete representation of real-world objects, objects in programs seem to diverge, where an object from one project is different from an object from another project and is typically different from how a normal person thinks about the real world object.

When the representation in code is the same as a person's understanding about a real world object ("common sense" or common understanding), the person can process and reason and be productive with the code with ease (like.. integer object types). On the other hand, if the representation is not the same (unintuitive), then the person has to go through the documentation/look over each line to match up the person's internal representation with the representation in code, making programming difficult (any large scale programming project).

Working Definition:
Class of Real World Objects: A set of things that has the same property. ex: a cardboard box (made from paper, rectangular, contains stuff)
Relation: A relationship between objects. A relation between object A and object B describes a constraint on some properties of A and B. ex. a ball is "in a" box (constraint on the location of the ball)
Inheritance: A particular relation between object A and object B where object A has a subset of the properties of object B. (an ISA relation) ex. a paper box "is a" paper object. a paper object "is a" physical object
Functions: a relationship between the parameter and the return value

Objects in OOPL:
Objects in object-oriented programming languages (the most straightforward implementation of objects) are defined by the name of the class, with a list of attributes and functions. Relationships between objects are implemented through inheritance, templates, functions, or attributes. Many problems come from the use of inheritance, where the base class has to contain a subset of the derived class, so there is no implicit way of sharing attributes between derived classes if it isn't in the parent class -- unless you restructure the inheritance tree to no longer resemble any real objects (which require constant refactoring with each additional class). Note, functions in the class definitions are relationships between the parameter, the self object, and the return value.

Objects in logic programming:
Logic programming (the most straightfoward implementation of relations) defines objects in terms of relations. Atoms can be objects, and classes may be declared with single term predicates.

brother(A,B) :- isHuman(A),isHuman(B),father(C,A), father(C,B).
Defines the relationship "brother" for objects of class human

ownsPet(A,B) :- isHuman(A),isAnimal(B),has(A,B).
Another example of a relationship. you can find who owns a pet named kitty (?-ownsPet(X,kitty).) and you can find the pets that a person John owns (?-ownsPet(john,X).) (think about how to program this simple relationship in OOPL -- adding owner to class animal and pet to class human would require two fxns and messed up classes).

Some logic programming languages are object oriented, and with clear definition of objects and relations, come the closest to the way we think about objects. However, logic programming is (typically) done on horn clauses, and uses backtracking to solve the query (run the program). Therefore, you can't put all human knowledge into a prolog program, and still expect it to solve your query in a reasonable amount of time.

Conclusion
Object-oriented programming give very good benefits to the programming environment (encapsulation, abstraction, code reduction, code reusability, type correctness), but common approaches limit the definition of objects, creating objects that no longer describe real world objects and sometimes decrease productivity in the process. Rather than thinking "what objects there are, and how to use the relations between them to solve the problem," a programmer has to think about "what objects can I implement that will give me all the benefits without any of the problems."
Logic programming approaches, on the other hand have a particular control process, which make intuitive implementations of real world objects unfeasible to solve.

So here's the discussion: How do you improve object and relation definitions in a way that is "natural" and useful to programmers? Is there a provable reason why objects in programming languages are so unintuitive? will they ever be?
[and bonus question: benefits/pitfalls of object-oriented functional langages?]

Thought this might be of interest to the Scala (and Twitter and Ruby on Rails) folks here. The Twitter dev team has switched to Scala because RoR couldn't handle the server side traffic. To me this is really interesting since most shops are usually hesitant to switch to "academic" languages, but here we are. And I'm pretty sure its not an April Fool's prank either!

Hi,

I have been recently trying to read the book algebra of programming which discusses a point free style of functional programming which can be quite appealing (via fold functions)- but there are some details which I dont quite understand yet (so far I have only gotten through the first few chapters since I had to go find a book on category theory to understand some of the terminology). The question I have is how do you do things like zip in the point free style especially if you have different fold functions for the two things you wish to zip?

Thanks in advance,

Carter.