Lambda the Ultimate

inactiveTopic dealing with Scheme (or Lisp, etc)
started 6/8/2002; 2:18:27 PM - last post 6/11/2002; 10:39:57 AM
Zach Garner - dealing with Scheme (or Lisp, etc)  blueArrow
6/8/2002; 2:18:27 PM (reads: 1148, responses: 6)
First, let me say that I really really want to like Scheme. I greatly enjoy what programming I have done with it. I have worked with it off and on for the past four years and have created a simple Genetic Programming application with it.

As I've graduated from a public university focusing on Mainstream languages, my primary programming experience is with OO languages (Java and C++).

I feel lost when writing any large program in Scheme. Software Engineering, as I have been taught, seems to require an OO langauge. What are Scheme's Best Practices or common mistakes (Design Patterns are an OO thing)? How do I conceptualize my programming problem (i.e. what is analogous to UML)? Do scheme people do unit testing?

My biggest problem is with Reading scheme code. Because of Guile a number of applications use Scheme as a language for writing plugins. Its terrible. I miss having an interface. I miss encapsulation. I hate dealing (with-function-names:like-this). I especially hate not knowing that I have to call function foo before function bar will work on some data. (this applies to procedural languages and any library that lacks documentation)

Even more, I hate Scheme books that teach me how to Program pieces of code but not how to Develop Software.

Most of my complaints probably extend to Lisp and other languages. I've learned too much OO that I have a hard time with other types of languages. I want to break that. If anyone has any Book or URL references that would help me, I would greately appreciate it. Tips are very welcome too.

andrew cooke - Re: dealing with Scheme (or Lisp, etc)  blueArrow
6/9/2002; 9:05:15 AM (reads: 1103, responses: 0)
[this is a bit long, maybe it would be better deleted and replaced by a link to ?]

before writing anything about software engineering in scheme i want to say that i have used scheme hardly at all, lisp only a little, ml only in small (home) projects, and have only recently had to worry about the issues of software engineering (java) in a team with mixed abilities. so i have no authority.

also, the following points are in the rather random order they occurred to me while out running just half an hour ago. so this is more to provoke discussion than present any deep truths...


i doubt that patterns are only for architects and java/c++ programmers. patterns are, imho, nothing more than guides to "good practice". every discipline has good practice, even dadaism.

maybe you are referring to paul graham's comment about patterns. if so, i think you need to re-read his article with a rather more sceptical eye - there's a lot of snide and bitter point-scoring there, imho, that deserved the aggressive response it received on lambda. it would have been much better to say that in lisp the power of macros allows you to adapt the language to better suit the patterns that occur in particular application areas. from that point of view his excellent "on lisp" can be seen as partly a catalogue of lisp patterns and partly guidance on how to handle patterns in lisp.

so i'd suggest asking in a scheme related ng for advice on appropriate scheme patterns. there was a thread a while back on clf about patterns in functional languages, too.

jack of all trades

one of the big arguments for lisp (and scheme to a lesser extent) is that it supports all styles of programming. the advantages ascribed to this seems to contradict one of the big arguments for great art - that it requires boundaries. certainly i enjoy working in c (yes!), ml or python more than lisp or perl because they are simpler. maybe i'm just stupid, but i find that a simpler language helps focus the mind. perhaps more importantly, having just one way to do it suggests that libraries and code are easier to understand (although maybe more tedious, less succinct, elegant etc to use).

anyway, one of my problems when using lisp, and i suspect one of yours in using scheme, is that you are dithering between approaches. i suggest you choose one and stick with it. so what are the options?

functional programming

state is the Big Problem. so get rid of it. you can use scheme to write purely functional programs. in my experience that means that you design programs by providing ways to iterate across data structures (to both read and construct values). the modularity in the program comes from the series of intermediate structures needed to get from where you begin to where you want to arrive. the "interfaces" tend to be folds, maps, etc.

for an example of good engineering within the functional paradigm, imho, look at the papers describing erwig's functional graph library.

one of the useful tools that makes this approach simpler is types. some automated way of making sure that you are applying the right function to the right datum is invaluable. since scheme and lisp do not, usually, use static typing, but do have strong dynamic typing, that suggests tests are the way to go. and so yes, i'm sure scheme programmers do have unit tests - it's either that or prayer, as far as i can see. again, ask on a scheme ng.

maybe one alternative to prayer is to introduce some kind of static typing "yourself". in the lisp world, cmucl is always provided as the example of a lisp where you can add static type declarations if you want. there's also a commercial (but possibly freely available?) lisp-like language out there that has static typing of some kind. i'm pretty sure it's been mentioned on lambda before (sorry, can't remember the name). and for scheme, there are also options - search google (or lambda) for scheme and "soft typing".

something i should have mentioned earlier - avoid lists (at least in public, if you see what i mean). obviously, lists are wonderful things and when you start with functional languages you just can't get enough of them. but you end up with every damn function working on lists and, without static typing to describe what the list contains, big possibilities for confusion.

so when you divide your problem up into data structures, with appropriate functions for iterating over them, stick those structures inside something hard and spiky. i don't know what's available in scheme, but lisp certainly has named structures. it makes code easier to read and even if you still get a runtime error at least it's located where the problem is, rather than several function calls away when you finally decide to process the value that you pulled of the front of "some list".

one of the things java somehow managed to get at least half-right, imho, is the separation between interface and implementation. it's often a sign of lazy java code when an object is an interface. this was hammered home last week when i lost half a day or more after getting the signature wrong when implementing an "interface" - because the interface was a (non-abstract) class, the compiler didn't complain and my method wasn't called.

unfortunately, without static typing, scheme and lisp do seem to be somewhat screwed in this area.


if you can't banish state, stick it in a closure. this is the approach that sicp advocates and it's effectively oop (without inheritance, but who needs that if you don't have interfaces? ;-).

lisp comes with support for oop, although it doesn't enforce data hiding. i don't know what you do in a production environment to cure that apart from code reviews and documentation. if you're paul graham, of course, (or, in my experience, any long-term member of cll) you work with the best programmers on god's green earth and it's simply Not A Problem. but if you're condemned to working with mere mortals, well....

for scheme, oo support has also been implemented by various people. maybe this sounds like a cop-out, but the whole point of the language is something simple and extensible, so objecting to extensions seems a bit contradictory. again, i suspect they don't hide data.

i was going to write more, but seem to have suddenly run out of ideas. i may add to this later. suspect all of the above is obvious, and know some of the above includes flippancy, over simplification and frustration that may offend more delicate lambda readers, but hope it is of some help (in places) or, at least, confirms (in other places) that the problems are inherent in the language.

Zach Garner - Re: dealing with Scheme (or Lisp, etc)  blueArrow
6/9/2002; 10:17:53 AM (reads: 1083, responses: 0)
Thank you very much for your time and comments. That gave me a lot to think about.

Ehud Lamm - Re: dealing with Scheme (or Lisp, etc)  blueArrow
6/10/2002; 2:33:03 AM (reads: 1076, responses: 0)
First let me say that Common Lisp and Scheme are really quite different when thinking about modules, OOP etc. Since I am not really into Common Lisp, I will leave it to others to write abou how you structure systems in this language.

Scheme is a very thin language. It doesn't support full fledged sturucturing elements (modules, functors etc.). There are many Scheme dialects that try to solve this problem. See for example the module system of DrScheme.

If you use plain Scheme, you may want to check out how people structure their code. I like the examples in Dybvig's book.

Oleg - Re: dealing with Scheme (or Lisp, etc)  blueArrow
6/10/2002; 12:23:28 PM (reads: 1222, responses: 1)
Software Engineering, as I have been taught, seems to require an OO langauge.

What you probably meant is the separation of interface and implementation and the structuring of large code as relatively independent and abstract components. If this is the case, then you need a module system. OOP is a restricted module system. Many Scheme implementations have module systems -- PLT Scheme, Chicken, Bigloo, Scheme48, Chez Scheme -- to name a few. Many of these module system are quite advanced and leave OOP in the dust. BTW, behavioral inheritance in OOP impedes rather than promotes the separation of interface and implementation. Thinking that everything is an object with a reactive behavior may also lead to suboptimal designs.

Do scheme people do unit testing?

Yes, they do. A recent thread on comp.lang.scheme has discussed this very topic.

You may find the following article
and the recent thread on comp.lang.scheme

Ehud Lamm - Re: dealing with Scheme (or Lisp, etc)  blueArrow
6/10/2002; 1:09:04 PM (reads: 1191, responses: 0)
BTW, behavioral inheritance in OOP impedes rather than promotes the separation of interface and implementation.

I tend to agree with this statement, but I must emphasize that it requires proof.

One should always keep in mind that inheritance works on interfaces and on the implementation details. The simplest inheritance relation is between an abstract class (only pure virtual functions, if you prefer this terminology) and an implementation. The abstract class is an interface, and the derived calss an implementation of the abstraction.

The power (and problems) of inheritance come from the possibility to inherit partial implementation details, and (also) to extend the interface of a class.

Oleg - Re: dealing with Scheme (or Lisp, etc)  blueArrow
6/11/2002; 10:39:57 AM (reads: 1076, responses: 0)
BTW, behavioral inheritance in OOP impedes rather than promotes the separation of interface and implementation.

I tend to agree with this statement, but I must emphasize that it requires proof.

For completeness, here is the reference to the proof of the assertion that OOP (behavioral inheritance) is detrimental to encapsulation.
The referenced paper at the Monterey 2001 worskshop may be particularly relevant. The paper also shows that not everything is an object and should be an object.