Lambda the Ultimate

However, I find the "philosophy" behind it just as fuzzy as the pseudo-linguistics that Larry Wall frequently cites.

Having given it a fair amount of thought, I believe the characterstics that make Ruby better (at least for me) are these:

- "light lambda" blocks - most collections have a built-in iterator that use these blocks - object-orientation as an organizing principle

The first two work together as a surprisingly power and succinct idiom for doing a very common operation in scripting: performing a repeated task over a set of data.

The third requires a bit of explanation: I don't mean object-orientation as a semantic feature of the language, but rather the fact that I it offers a hint of where to find the information when I forget the "vocabulary".

To give a concrete example, the other day I had to write a script to massage some file names and save some info about them to a DB.

I went looking for the method that allows search and replace on strings using regexps. My first guess was "replace", which exists but does something different (destructive update of the whole string). At this point I could easily have blown away quite some time looking for the write method. (Many Perl tasks have gone this way.)

But because I knew this had to be a method for the String class, I quickly went through the methods for String in the docs and found the right one, "sub" (Which seems make me think of subtraction, but I guess intuitive is in the eye of the beholder).

Actually, it strikes me that this mnemonic organization of the standard library is one of the powerful (and little discussed) features of OO.

mnemonic organization of the standard library
That's true of Smalltalk.
Seems like this was undermined in Java (get/set).
Maybe you just put more effort into learning Ruby libraries ;-)

What about Python?

I've taken a run at Python many times over the years, I always run away screaming when I remember that the following will run without complaint:

classExample:
	x = 10

ex = Example()

ex.y = 10

class Example:

Scratch that previous badly formatted comment. Seems like you had posting problems as well.

As far as you're full example goes, I'm not sure I see the problem in a language designed to be as dynamic as Python.

How does Ruby handle this case?

http://www.python.org/doc/Humor.html#zen

I just thought that this discussion should contain a reference to these pythonic principles, as listed by Tim Peters (channelling Guido van Rossum).

Well, that is the problem itself. I'm not quite prepared to be that dynamic. ;-)

In Ruby, the equivalent code fails, since y is undefined in class Example.

(You can extend a class in a similar way, but you have to do it explicitly at least)

To get a sense why this might be bad, try to maintain code that someone else wrote using such an idiom, or even to maintain your own code from 6 months ago.

If your program is bigger than a "script", this might also result in hours of debugging entertainment 5 minutes after you wrote it. ;-)

One sane response to this is: "Well don't use that then.", which is reasonable enough, but hardly convinces me that I shouldn't take the same attitude to the language. ;-)

Should languages be this dynamic? I think so. What keeps programs and programmers sane is the use of unit tests.

I've been doing a fair bit of programming with Python and XML. What struck me before too long is that a DOM isn't really necessary because Python's object model is so dynamic.

And so the Python or Ruby object model serves as a useful DOM. In fact for Jython (Python that compiles to JVM byte codes), I wrote an extension to JXPath that maps XPath, including update, to Jython objects.

class Foo(object):
    __slots__ = ["bar", "baz"]


    def __init__(self):
        self.bar = 1
        self.baz = 2
 
>>> f = Foo()
>>> f.bar
1
>>> f.baz
2
>>> f.boom = 3
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
AttributeError: 'Foo' object has no attribute 'boom'

Hmmm, I can't decide if I think this a step in the right direction, or the wrong one. ;-)

Written-down thoughts become programs.

We just want to be "influenced". (not brainwashed)

while you program to solve your problem, I program your brain to work better.

Could this be evidence that dynamic languages are not as dynamic as claimed?

Dynamic is simply a nice way of saying "Sorry, no static guarantees here".

It just struck me that this sort of "make the world a better place" zeal may actually be a meaningful phenonmenon for PLs:

it may be what it takes to motivate a Matz or Guido or Larry Wall (and those around them) to devote the kind of time and energy it takes to make a language fly (and stay flying).

Only some sense of mission could keep you going through that I imagine; it doesn't sound like there is much financial incentive.

A second dimension to this is the motivation of afficionados. Do you want to program in an evil language designed by a corporation? Why would you when you can program in a language that will "improve the way you think", "make you more productive", "make programming fun again".

Sounds a bit like a self-help infomercial, doesn't it? ;-)

Also, that Onlamp.com article is amazingly inaccurate. I shudder to think of how many people will read that and say, "Strings don't have methods? I can only raise strings as exceptions? Python doesn't handle unicode? There's no *way* I'm using Python before 3.0."

Ugh.

The folks at twistedmatrix.com write a decent amount of code that uses such an idiom, and don't seem to have much of a problem maintaining their code. I've used such an idiom in my own code and never had any trouble with it.

When I need to instantiate two mutually referent data structures at the same time, I create the instances and then add the attributes they need to refer to each other. It's fundamentally the same thing as having the class initailize the attribute to None and then later setting it to the necessary value. I'll get an AttributeError exception either way (whether it's by the class not having the attribute or the None object not having some attribute). Personally, I don't see a need to bother setting the attribute to a None object.

Without knowing anything about what them I can't comment, but I have seen applications that were written in monolithic spaghetti code that the maintainers claimed was no problem to maintain, since the original developers were still there. However, this doesn't inspire me to rely on monolithic spaghetti code. ;-)

Let me give a more concrete example of how this could be bad. Let's say I define a class like:

class Example:
   column = "id"

Later on I'm coding using an instance "ex" of class "Example", and I falsely remember my variable name and use:

ex.col = "pk"

I then write, say DB access routines that use that as the column name of the primary key for a table, using "col".

My tests pass (the name is locally correct) and everything is keen until I pass the object elsewhere where the original "column" is referenced, and whammo! Things stop working, but I don't spot the cause right away.

I now spend "happy" hours debugging that could have been spent developing new functionality.

I now spend "happy" hours debugging that could have been spent developing new functionality.

From a more general sense, though, I think two questions arise out of the process. First, tests should be designed to capture errors before they occur. For whatever reason you are changing the field, there should have been some test to authenticate that the modification had the intended effect. Test first is the order of the day for any project using a dynamic language that wishes to scale. Not saying it would necessarily capture this particular instance, just that freedom of dynamic languages almost makes testing frameworks a requirement.

Second, from an OOP perspective, it probably makes more sense to isolate such behavior in methods (e.g. accessors). In the example, you are not encapsulating the behavior of the database access - the intermediate class becomes little more than a repository or global variable. The question of how the database is accessed should be isolated to the class. You may make suggestions about how you'd like to access the table via messages (aka functions), but in the end the actual access to the resource is the responsibility of the encapsulating class.

People also managed to program in C for 20 years and yet there are many misfeatures and infelicities in C which have been eliminated in newer languages.

Let's express that differently: Can syntax prime the way we think about a problem?

"It turns out that Hong Kong citizens can be encouraged to think in either an Eastern or a Western way by presenting them with images that suggest one culture or the other."
p118 The Geography of Thought

Can syntax move our attention to different programming concerns - C, optimize the code; Ada, someone will read this ...

Syntax alone? It may have some effect, but I would bet fairly little, at least in the positive sense. (Inconsitent and imprecise syntax, or overly complex or dense notation, on the other hand, can have a negative impact.)

As I've said before, the combination of the semantic/syntactic primitives, the standard library and the sub-culture of the language probably does influence the way you solve problems by offering a path of least resistence.

There is a big leap from this to saying my "brain will work better", unless you start with the a priori belief that those types of solutions are necessarily the best ones.

An interesting use of the word "power".

A nuclear explosion is very powerful, but it destroys everything indiscriminately.

A laser beam is powerful in a different way, since it can very precisely remove, say, a harmful growth, while not affecting anything you want to keep.

The latter type of power is the only kind that I cultivate. ;-)

First, tests should be designed to capture errors before they occur.

That is the kind of "should" wishful thinking that Agile is trying to replace. ;-)

Even the best test suite will miss some things, and you can't anticipate everything. Maybe that is a worthwhile trade-off for you. My personal taste is that I want a little more help from my language catching my own dumb mistakes.

Second, from an OOP perspective, it probably makes more sense to isolate such behavior in methods

Of course; it was an example.

Given this, isn't it better for a dynamic PL to be dynamic "all the way"? Ehud once asked about the benefits of dynamic types (types whose structure can be modified after they've been defined) and I provided some examples.

So, I do like the ability to add properties to an existing object in a dynamic PL. I also think that such an operation should be explicitly annotated, for error detection but even more so to convey programmer intent.

One premise of the discussion was that you were using a test suite to verify the program. I can get my test to fail with Ruby, but not with Python. (In this sense, well-designed automated testing is practically equivalent to static type-checking since testing is done with each build)

Given this, isn't it better for a dynamic PL to be dynamic "all the way"?

Hmmm. Isn't this like saying "a bowl of ice cream is good, so eating 2 tubs in a sitting should be even better"?

There are things for which dynamic typing is very handy, but it is strong medicine

The whole dynamic vs. static debate reminds me of when I was a kid and they started legislating seat-belt use in cars.

Many peopled balked at the idea, saying things like: "but I've driven without a seat-belt all my life and there haven't been any problems" or "but if there is an accident it is better to be thrown clear."

Only people who have known someone (or a project) that was seriously hurt without seat-belts (or static checking), or who can easily imagine this happening, are convinced. The others say "what is the point?".

Only people who have known someone (or a project) that was seriously hurt without seat-belts (or static checking), or who can easily imagine this happening, are convinced. The others say "what is the point?".

X = X + 1

I'm personally of two minds. I want the software I write in dynamic languages to be written in the most wide-open, no-holds barred, everything is dynamic a programming language as possible. On the other hand, when I'm programming in a static programming language, I want it to be rigorous in it's type checking, guaranteeing as much correctness as possible. Working in various projects, I have a need for both types of languages, so I really don't take sides in the debate.

Why is that? Granted, as I myself pointed out, getting the Ruby version to fail a test is easier than getting the Python version to fail. But if you are rigorous about your use of testing to verify correctness, you should also be able to catch the Python error.

My point was that for a dynamic PL, the correctness guaranteed by Ruby in not much better than that guaranteed by Python (as Frank has pointed out often, and with much greater expressiveness and zeal than I can muster). And in exchange you loose quit a lot of flexibility, which for me is the main reason to use a dynamic PL.

Chris: Spoken like a true Static Typing enthusiast.

BTW I'm also a "Static Typing enthusiast" (provided the PL provides competent type inference). But this does not detract from the usefulness and expressiveness I find in dynamic PLs. One of these useful and expressive features is the ability to dynamically define and modify types.

Marc: Many peopled balked at the idea, saying things like: "but I've driven without a seat-belt all my life and there haven't been any problems" or "but if there is an accident it is better to be thrown clear."

Given my previous statements I think you can see that I agree drivers should wear seat-belts (both metaphorically and in reality).

Chris: Working in various projects, I have a need for both types of languages, so I really don't take sides in the debate.

Which is exactly the approach I have. My solution is to use both statically typed and dynamically typed PLs in the same project. The core will be written using a statically typed PL and above that I would use a layer of a dynamic PL for flexibility and customization. If I find that a bit of dynamic code is becoming rigidly defined or frequently used, I will migrate it to the static "core".

I realize that a purist like Frank may also scoff at this approach, but it has worked really well for me.

Actually, it doesn't defy the laws of algebra, if you pick the right algebra. :) In fact, this is the archetypal example of a recursive domain equation and says that X is (isomorphic to) the set of natural numbers.

The ideal level of protection provided by the language is seemingly a subjective criteria.

Yes, but most people would not balk at the idea of very basic forms of protection such as checking that an n-ary function is always supplied with exactly n arguments. If you accept that premise, then (I think) one can show that it's inconsistent to reject static typing in general because more stringent forms of typing reduce to such simpler guarantees.

I'm personally of two minds. I want the software I write in dynamic languages to be written in the most wide-open, no-holds barred, everything is dynamic a programming language as possible. On the other hand, when I'm programming in a static programming language, I want it to be rigorous in it's type checking, guaranteeing as much correctness as possible. Working in various projects, I have a need for both types of languages, so I really don't take sides in the debate.

Fortunately for you, then, you are not as schizophrenic as you think: what you want is a statically typed language. As I've said before (and even demonstrated), a dynamically typed language L is just a module in a statically typed language which defines one, universal type U, where every type of L is injected into U.

data U
  = Int Integer
  | Chr Char
  | Str String
  | Uni ()
  | Par (U, U)
  | Fun (U->U)

If you want call-by-value side effects, you just use the IO monad:

data U
  = Int Integer
  | Chr Char
  | Str String
  | Uni ()
  | Par (U, U)
  | Fun (U -> IO U)

If you want call-by-value continuations, you use the Cont monad:

data U
  = Int Integer
  | Chr Char
  | Str String
  | Uni ()
  | Par (U, U)
  | Fun (U -> Cont U)

The static part of the language does not even need to support effects or continuations.

What, you want objects? OK:

data U
  = Int Integer
  | Chr Char
  | Str String
  | Uni ()
  | Par (U, U)
  | Fun (U -> IO U)
  | Obj (U -> String -> IO U)

Complex numbers?

data U
  = Int Integer
  | Chr Char
  | Str String
  | Uni ()
  | Par (U, U)
  | Fun (U -> IO U)
  | Com (Double, Double)

Built-in sets, arrays, dictionaries?

data U
  = Int Integer
  | Chr Char
  | Str String
  | Uni ()
  | Par (U, U)
  | Fun (U -> IO U)
  | Set (Set U)
  | Arr (Array U) -- or (Int, Int -> U)
  | Dic (FiniteMap String U)

Maybe you would like documentation strings attached to every function?

data U
  = Int Integer
  | Chr Char
  | Str String
  | Uni ()
  | Par (U, U)
  | Fun (String, U -> IO U)

data U
  = Int Integer
  | Chr Char
  | Str String
  | Uni ()
  | Par (U', U')
  | Fun (U' -> IO U')
data U' = (Meta, U)
data Meta = Meta
  { doc :: String,
    sourceLoc :: (Filename, Col, Line),
    author :: String }

What else do you want? First-class modules? Quaternions? Logic variables? Backwards state transformers? Exceptions? It's all trivial to implement and none of it needs to exist in the base language.

In conclusion, the whole dynamic-static debate is a red herring. There is a question of syntax which is easily handled by something like macros.

ST languages are superior to DT languages in every way.

OK, the analogy may have been a bit dramatic. ;-)

Your algebraic example only proves that someone coming to programming naively from a math background may be shocked, but this is a well-defined expression to most programmers (though it's exact interpretation might vary slightly from language to languge.) and not likely to produce gotchas.

Working in various projects, I have a need for both types of languages, so I really don't take sides in the debate.

This is a different position than the one I was arguing against. I myself have used "dynamic" languages for various purposes at various times. For certain uses, I don't think they are "bad". I just have reservations about certain features for certain purposes.

I am also inclined to believe that certain features, such as the violation of the Open-Closed Principle I originally pointed out, are just a bad idea, much more likely to produce hard to find bugs than offer any extra "power".

The position that is akin to the seat-belt situation are the people who say "why would I want any restriction on what I can do in my programming language?", who see no reason to place restrictions on any aspect of the language. To my ears they sound the same as the no-seat-belt people from the 70s.

You're right, you could find a way to test that, but you would have to think of it first. Test-driven is a wonderful technique, but it isn't magic.

In this instance, you probably wouldn't have thought of it, because you would have tested your class (and it passed) and you tested your DB function and it passed. The problem arises when you create a third function that assumes that the two are using the same data. You aren't likely to test every combination of all of your classes and all of your functions: too many permutations most of the time.

So from my point of view this kind of thing is a mistake waiting to happen, regardless of what other good practices you are using.

If I find that a bit of dynamic code is becoming rigidly defined or frequently used, I will migrate it to the static "core".

I actually have thought about a language that would allow you to do that: prototype "type-free", but if you add type information it enforces it. This could be used to increase and refine the checking as you go. (As near as I can tell, this is different from "soft-typing")

This strikes me as a livable compromise between the strength of both styles of typing. (Though others will disagree ;-) )

In general, you cannot `add' type information, you can only remove it, because the type tells you what the available operations are. Systems where you can meaningfully `add' type information must use equality for type equivalence or some partial order for subtyping, because otherwise there is no choice of which type to assign: it could be inferred. But any system which compares types in this way suffers from a modularity problem because you cannot change the representation of a type without changing its interface.

The way around this is to wrap the type with operators and use existential quantification to make the type opaque. And now you are back to the original situation again, because each value must have a unique type.

Let me be more precise.

Imagine a language where the type U from your type demo (disjoint union of all available types) was an implicit default type. As you refined your thinking about the program, you could express these refinements by giving a more restricted type (a proper subset of U) to constructs that were more "settled".

This would provide the "freedom to experiment" early on, but provide the benefits of more tightly constrained typing as you converged on the "right" solution.

Anyway, using the technique I showed, you can do almost what you want without any modularity problems.

Suppose you've written a program and there is a function f in it which you happen to know always takes and returns an integer. So what you do is you write a function f' :: Integer -> Integer and replace f with

Fun (check f' (int --> int))

where:

-- embedding-projection pairs (retracts)
data EP a = EP (a -> U, U -> a)


int :: EP Integer
int = EP embed proj
  where embed = Int
        proj (Int i) = i
        proj _ = error "type error"


(-->) :: EP a -> EP b -> EP (a -> b)
(EP ea pa) --> (EP eb pb) = EP (embed proj)
  where embed f = Fun (eb . f . pa)
        proj (Fun f) = (pb . f . ea)


check :: a -> EP a -> U
check x (EP e p) = e x

(Note: I haven't tested this code.) This is a little combinator language which makes it easy to use an ST value in the DT sublanguage. Note that the syntax:

Fun (check f' (int --> int))

is isomorphic to what you wanted:

f : int -> int

except that f has become f'.

So now you have a function which is usable in the DT sublanguage but implemented in the ST language.

I read that explanation again, and I think I misunderstood it. Let me rephrase what I understand you are saying the problem is and you can set me straight.

So the problem arises if the "refined" types are allowed to be arbitrary subsets of the "U" type because some mechanism must exist to work out equivalence classes of those subsets, and that this will horribly confuse the type inference mechanism, as well as any external callers of the typed entities? (I hope that made sense. ;-))

Would the problem be fixed somewhat if instead of arbitrary subsets, we have the discrete partial order with U as top, i.e. either the thing is type U and anything goes, or it is a specific type?

This version of U would be similar to [alpha], except it would be a type instead of a type variable.

This is a little combinator language which makes it easy to use an ST value in the DT sublanguage

Though my theoretical side liked this example, I must confess that I spent quite a long time working out how it worked for a relatively small sample of code.

As I've said before, I never properly learned Haskell, and I've done more OO programming recently than FP, but nonetheless that seemed like a lot of cognitive work for the "simple" task of embedding a DT language in an ST language.

Though it may be more "correct" and elegant, do you disagree that it might be require a disproportionate amount of effort, even for someone with a good theoretical background?