Ivory, an experimental declarative programming system

It has probably all been done before, but for a number of years I have been attempting to construct a "data" driven programming system.

There is a scrappy website under construction at:
www.ivorysystem.com
or
www.ivoryscript.com
where I am trying to present what has been done so far.

Apart from some useful work done by some staff at Southampton University, it has been something of a solitary exercise. As the system includes a new programming language derivative, this is far from ideal, and I would appreciate any serious peer review and the opportunity to discuss whether or not anything of worth has been achieved.

The website has a .zip archive containing an executable file. I am reasonably sure that it doesn't contain any viruses, but I can't take any responsibility if it becomes infected.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Sparks

Does the term "spark"--meaning very short lived thread--have prior meaning in computer science or is that your own term? I ask mostly because I used the same term for a language design I came up with a few years ago.

Sparks

The term "spark" in FP seems to be well established to describe the action of creating a new parallel thread. My intention (and yours?) was to refer to a fairly short lived thread which may not need cell level garbage collection. I haven't done enough research to see if this has prior meaning.

Rationale?

Thanks for posting this, it is interesting to read. I am curious: how would you summarize the benefits of the system? For example, how does having a data store and separate sparks with events solve general software development problems (state, concurrency, side effects, etc.)?

Reading about how sparks are meant to be short lived so it is easy to clean up after them, I wonder (I haven't seen this yet in my initial reading, apologies if I missed it!) if there is any ability by the system to guarantee things will be sufficiently short lived?

The business of being easy to clean up, or perhaps just easy to get back to some know state as far as the spark is concerned - although of course the data store would have changed - is perhaps related to microreboots, (perhaps at least if one drinks a lot and squints the eyes).

Rationale

There is a lot to answer here.

Rationale/Benefits

I will be including a section on this on the website www.ivorysystem.com.

For now, some of the key ones for me are:

Space and time efficient handling of dynamic structures

Property sets are widely in use for data modelling. Ivory has a particularly efficient implementation using a vector of Name/Expression pairs.

Firstly, memory efficiency.

As any name is represented by an integer, and many expressions just require 32 bits, the storage requirement for many properties is just 8 bytes each.

Secondly, performance.

The main loop of the class method to access a property takes 5 instructions on a PC architecture. This is extremely important in supporting high speed data storage access.

Component based modelling

The Ivory data types have been designed so that the main relationships between data are in a data store and not embedded in code. The reverse reference mechanism may be considered by some to be excessively CPU intensive, but it is a very powerful tool.

What happens in practice is that it is possible to build a common growing repository of core functionality, with different applications determined solely by their configuration scripts.

Rule based scalability

Data change sensitive rules eliminate the need for almost all direct forward function calls for application state changes. This enables applications to scale with less risk of breaking existing functionality. Built-in primitives are rarely changed, just new ones added.

Transparent Functional Properties

Ivory fully supports lazy evaluated properties similarly to a spreadsheet. This extends to the byte code where lambda sub-expressions sections can be stripped out of larger segments as data is copied.

Dynamic programming

Ivory is "theoretically" type safe, but not statically. Support for run-time type checking and class dispatch is at the heart of the system. The system also carries out sufficient "environment" references so that data stores can be fully independent.

General software development problems

I don't claim that Ivory solves or even addresses all of these problems, but the ability to monitor every state change does tend to improve reliability over sequentially based applications of a similar size.

Spark lifetimes

The system does not guarantee that the lifetime of a spark is short. Here it really comes down to programming discipline.

In future a conventional garbage collector will certainly be added; it just hasn't been necessary up to now.

microreboots

I will get back to you on this one when I have read the paper, thanks.



Finally, may I encourage you (and others) to trust the .zip file and try out winIvory. The socket proxy example probably best illustrates many of the currently implemented features.

POSIX version

By any chance, is the Linux implementation mentioned on the website compatible with OS X? I'd love to try this out. Seems very interesting, especially once a conventional GC is added (which may just be a personal preference). Without a "conventional" GC, how does the system deal with the lifetimes of sparks?

POSIX version

re. OS X, no is probably the short accurate answer. I do intend to pursue the open source route at some point. For now what is holding me back is the potential for embarrassment; there are just too many aspects of the current design and implementation which expose my mathematical and programming limitations!

As for spark lifetimes, there is fairly standard custom allocator for cells etc. This keeps track of blocks allocated from the O/S during spark execution. When finished, they can be freed up very quickly.

If you can run the Windows version, the Store Info button provides details of data store memory allocation; using the same allocator.

It probably comes down to the sort of applications that an Ivory system might address. It has been deployed successfully in an communications gateway similar to an SMS host. Whilst it would probably not be suitable for a high speed IP router, it is very easy to specify script level criteria for routing/interface decisions. For the gateway, sparks are mostly triggered by timers and the presence of socket and serial port I/O.

two cents encouraging going open

I'd encourage you to open source it, but be the tzar when it comes to the whole project - people can contribute, but you should keep driving the integration and vision.

Believe me, no matter what your system looks like internally, there are more open source systems that are worse! Would it not be an opportunity to get to work with like-minded folks to trade ideas and learn from one another? And if someone is a jerk, kick 'em off :-)

You know what it is like when the cleaner is due

There is the rush to clean everything up first.

Thanks for the encouragement to go open source, I essentially agree. Does anyone have any recommendations about the best route for this?

What it comes down to is the old simple question: Would anyone want to program in this way?
If not, it becomes a pointless exercise!

I know that the web site is weak - one advantage of open source is that someone might offer to document it properly - but I would still appreciate opinions from this site. So far, most effort has probably gone into the Introduction (updated after raould's earlier comments) and Examples sections.

Finally, believe me, I never intended all this. It just grew. Should it have been killed at birth (around about 1992)?

Linux version

Can you please post the linux version on the site.
I do not use windows.

Linux version

I intend to do this soon - it is just a question of when I can find the time to bring the Linux version up to date. I agree that it is important to be as O/S independent as possible.

The Linux version is a bit more restrictive in that it currently has no GUI, but it should run the demo scripts. I may decide to wait until putting in place a planned DHTML version with a server side page object model (naturally) and Live connect for the user interface.

A Linux version can now be do

A Linux version can now be downloaded from the web site.

It is a bit more restrictive in that it has no GUI. However, scripts may be configured to support remote evaluation, and I have provided some examples. This can be accessed via telnet on port 2152, but it is easy to use any other port.

Through this interface, there is support for single line scripts such as:

"Hello, world"
"This string" = "This" ++ " string"
(atan 1.0) * (360.0 / (2.0 * 3.14159))

Probably more useful are data store queries. The following one rather incestuously returns the name of a connecting host:

(getAt (.commsStore.scriptListenerSocket.connections) 0).peerName

or to shut an ivory session down:

exit 0

Scripts extending over multiple lines may be entered by bracketing them with :{ and :}

:{ (\!n ->
   let !fact2 !acc !n =
      if n = 0 then acc
               else fact2 (acc * n) (n - 1)::Int
   in
      fact2 1 n) 5 :}

For the observant, the Int type constraint is needed because if alternatives are not required to be the same type.


The documentation is still very limited, but I have included an informal explanation - mainly concentrating on one of the example scripts.

I would still welcome feedback as to whether this project is worth pursuing further.

Re: worth pursuing further?

Perhaps aim for more people to see it so that the 1% of viewers that actually try it out becomes a larger absolute quantity? Try bunging it up on Sourceforge, and post on Freshmeat (and maybe even /., shudder) and see what happens :-)

I was wondering those "!" is

I was wondering those "!" is those for introducing new variables? Because in my language thats what "!" mean, and it looks similar to yours.

Sorry, more and improved docu

Sorry, more and improved documentation will come. For some unknown reason, patterns were omitted from the HTML syntax.

! has two syntactic forms in IvoryScript:

  • To signify a strict (or evaluated) variable as per Haskell etc.;
  • To force evaluation of an expression where its context is lazy.

The implementation of recursive let definitions is incomplete, but !fact2 declared to be strict currently makes it easier for the compiler to perform the now standard tricks of generating code similar to that from a procedural language; the Linux version on the web site lists the byte code of submitted expressions to show this.