Lambda the Ultimate

The problem is interactions between whitespace-sensitive syntax and anonymous functions. A lambda is an expression, so it's expected to be used within another expression. Then you need to close off the other expression, so you get code like this:

    map(lambda point:
        print "Point.x = " + point.x
        print "Point.y = " + point.y
    , list_of_points)

The dangling comma and further arguments are just plain ugly, and would be confusing to any beginners. Plus, if you dedent too far you break the indentation of the containing block and end up with code you didn't really mean (but which might be valid anyway!) Goodbye Python, hello Fortran.

This is always a problem with whitespace-significant syntax, but without lambda you don't have cases where the next character is a single comma all alone, which is easier to miss than an identifier.

By contrast, a def is always a statement, so it doesn't break up any expression. When you dedent, you start a new statement. Nesting is natural, there're no ambiguity problems, you don't have to break your train of thought to wonder "What is this block doing?", and you don't get dangling tokens.

You don't get these problems with Scheme (or C or Java or Ruby) because they have explicit terminators for the lambda. Even then, it's still somewhat awkward: I hate looking at anonymous inner classes in Java and seeing }, value).methodCall() at the end.

As much as I like lambda expressions, Guido has a point. That is, syntax and semantics of a language has a lot of influence over what constructs are reasonable in that language. If you think about it, this is similar to what the Lisp/Scheme people have been saying all along: Scheme and Lisp are so flexible, simply because they have essentially no syntax. This has both positives and negatives. The negatives are obvious: harder to get use to, everything looks the same (hard to tell different constructs from each other). The positives are: everything looks the same (don't have to remember 10 different syntaxes for 10 different constructs), meta-programming is easy, and as a result, the syntax is extremely flexible.

When I say that the syntax is flexible, I mean that it easily allows new constructs without damaging readability, from a syntactic point of view. Every new construct looks the same in Lisp/Scheme, unlike in Python, or even Java, where there are different methods of determining block structure (python has parenthesis and whitespace, Java has parenthesis and curly braces).

One of the lessons learned is, if you make syntax a major part of your language, you have to design all constructs into you're language very carefully. In the case of Python, lambdas were added later because programmers wanted them, but the syntax of lambdas wasn't designed into the language very well. Guido could design lambdas to be more Pythonic if he wanted to, but I get the feeling that he doesn't really want to.

In the end, it's not a big loss for Python if Guido keeps/improves its support of the creation of lexical closures (unfortunately, code locality will suffer).

"When I say that the syntax is flexible, I mean that it easily allows new constructs without damaging readability, from a syntactic point of view." In other words, damage may never take syntactic readability below 0. :-)

"If you make syntax a major part of your language, you have to design all constructs into your language very carefully." For sure. In Nickle, our anonymous λ looks like e.g.

(int func(int x){return x + 1;})

where func is a keyword. This syntax is verbose, ugly, and fits reasonably into a language that is supposed to look syntactically like C. It took us several turns to get there.

I'm not sure if you're joking or not, but I don't find this syntax that ugly, quite simple actually: even a beginner can understand it thanks to your use of func instead of lamba as keyword.

Maybe I'm too used to C though :-)

Actually I was sort of half-joking, as usual. I think func makes way more sense than lambda as a keyword in Nickle. But with the return keyword and all that syntactic noise, it makes for pretty verbose anonymous functions. Of course, I was writing in the explicitly static-typed variant of Nickle. If you omit the types (which also suppresses static typechecking—no type inference), you get the simpler-looking

(func(x) { return x + 1; })

All in all, though, let me just say that if that syntax looked natural to you as a C programmer, I'm proud: that was one of the explicit Nickle goals. Check out our first-class continuations sometime, or our threads, or our twixt(;) statement. We're trying really hard to help C programmers learn how to get at some of the cool stuff from the last 20 years. So thank you for the positive feedback!

It is almost Javascript, with func instead of function.

The runtime, and, ultimately, the operating system?

I don't think so... Try again!

How about processes, because programming is ultimately about us (programmers) expressing and creating the processes in we create in our minds (the machine and/or OS is, ultimately, irrelevant).

"part of the language's user interface"

I never thought about that, but i think it's well put: a programmer (user) interfaces with a language via its keywords and syntatic constructs. A user interface, indeed.

"This is also the reason why Python will never have continuations, and even why I'm uninterested in optimizing tail recursion."

so, no true lambda, no real proper lexical scoping, no continuations and no proper recursion... well, except for lexical scoping, C++ doesn't feature any of this either, and seems to be doing fine. But, then again, C++ is a far more flexible language than Python...

at least, Ruby features them...

Since when Python has no lexical scoping?

it's broken - the following code (to generate a "counters" / "generators") doesn't work:

def make_counter():
  x = 0
  def counter():
    x = x + 1
    return x
  return counter

i understand that is because you only have write access to the outermost (global) and innermost scope (in this case x = x+1 fails because it cannot update the x in the enclosing scope).

the above may seem obscure if you're not used to using closures, but it's a basic pattern in, say, scheme. as someone has said already, i think, this doesn't matter in that objects encapsulate state in a very similar way to closures, and python has objects, but if you try to do "functional" programming in python you get bitten by the above very quickly.

this doesn't work:

a = 5
def foo(): print a
a = 10
foo() # prints 10!

_lexical_ scoping means the scoping is the textual area where things were first defined! foo was defined when a was 5, thus it should print the value of a when it got defined.

it's hands down broken...

I guess Scheme is broken too then:

(define a 5)
(define (foo) (display a))
(define a 10)
(foo) ; -> prints 10

Arguably, the Scheme version is worse as it looks like defining a new variable, rather than assigning to an existing one.

Sorry, but this is nonsense. The global variable a does not turn into a constant just because you access it in foo. It remains a name in the global scope and you can rebind it freely there. The binding of a to foo is a name binding not a value binding. It is also irrelevant for the compiler if a is already defined somewhere. Since a is not defined in foo's scope the compiler concludes that he has to load a global. This can be shown easily analysing the disassembled code:

import dis
dis.dis(foo)
  2           0 LOAD_GLOBAL              0 (a)
              3 PRINT_ITEM          
              4 PRINT_NEWLINE       
              5 LOAD_CONST               0 (None)
              8 RETURN_VALUE        

I never really gave much thought to the subtleties of static lexical scoping and i guess haskell's staticness and imutability has affected me somewhat as of late...

Prelude> let add2 x y = x + y + z
:1:23: Not in scope: `z'
Prelude> let z = 10
Prelude> z
10
Prelude> let add2 x y  = x + y + z
Prelude> add2 1 2
13
Prelude> let z = 20
Prelude> add2 1 2
13

The definition of add2 closes over z (i.e. it has its own environment with z=10 in it). It's independent of any other bindings to z. The definition of add2 is not in the scope of let z = 20 so it should not be affected by it. Just in case this is also an issue, the let statements bind variables; they do not assign them so the second z is a new fresh variable.

To put it another way, in let z = 20 in add2 1 2 you should be able to alpha-rename z to something else (replacing all, in this case, no) occurrences in the body. Thus this code should behave the same as let quux = 20 in add2 1 2. This all makes sense; you don't want to have to know what variable names a function used so that you don't accidentally rebind them. This is why dynamic scoping (which would have resulted in 23 in the above example) is usually considered a very bad idea for a default scoping convention.

Sorry, I should have been more clear about the fact that I actually really like the behaviour I posted about :-) but am still always confused about how there are examples which, to me, appear to contradict lexical scoping. Like what is going on with the Scheme example above? (I manage to confuse myself easily - slow loading link-with-anchor.)

Assuming you are referring to "Mutability v. Scope", the issue there is that Scheme defines 'define' to essentially be set! (although the first use is binding). It's becoming more and more clear to everybody that having an explicit reference type (or just being more explicit on what is a variable v. a reference) helps clarify this and many other issues (e.g. co- and contra-variance with subtyping).

There is some confusion between the behavior of define vs let in Scheme—let always binds, define (which should only be used once) binds only on first use. My personal opinion is that define should throw an error if used twice on the same name as its intended use is to bind global values (procedures or global vars), for example:

(define a 5)
(define (foo) (display a))
(define a 10) ; <- throws an error
; but these are okay:
(let (a 10) (foo))
(define (bar a) (display a))

The problem, of course, is that define is binding a value in the global environment. When redefineing a value, the meaning is unclear. Since the old value isn't simply masked by a closer environment the Scheme example makes a little more sence. Redefineing doesn't mask the old value: it replaces it.

Perhaps there should be a thread explaining scope, binding, and assingment and their implementation in various languages?

That would truly be a god-send, assuming one could eventually get all the bickering down to a concise set of explanations / definitions :-) :-)

I think the programming community is pretty well agreed on these issues.

If the commuinty is so agreed, then why are there such long threads on things like LtU? I'm just saying that while there might be a core truth, it appears to me to very often be obfuscated. And, actually, who can even give me a solid definition of referential transparency that will meet with only the chirping of crickets?

I, for one, believe that argument on a forum such as LtU, although sometimes frustrating, is generally a good thing. If you can wade through all of the discussion, you will end up with a broader understanding of the topic than if you only heard one side of the issue. You see both the positives and negatives of different viewpoints, instead of getting the one-sidded viewpoint you sometimes see in a textbook. (Not that textbooks are intentionally one-sided.)

Obviously argument, for the sake of argument isn't good, but argument for the sake of realizing a better definition or description is good.

I'd agree; I always learn a lot from the discussions (although sometimes they go on way past what I can muster parsing). Sometimes I even learn that I'm not the only one who is - apparently - easily confused by terms like 'referential transparency' and 'scoping'. :-)

Section 4.5.2 of CTM points out three moments of a variables life:

1. Declare a variable in the store
2. Specify the function to calculate the variable's value
3. Evaluate the function and bind the variable

Doesn't address the idea of scope, but it's probably useful for a discussion on the subject of binding and assignment. The upshot on Oz (and Alice) is that the three moments can be separated.

I would think the reason for Schemes behavior in this situation is incremental development. Its nice to just be able to rebind an errornous function then you discover it, and not have to recompile the whole program. I'm not sure if this is possible in Scheme but in Common Lisp, then you encounter an error you can sometimes just rebind some variables and continue the program from the point of the error, this could save a lot of time if you have to run the program for a while before the error shows upp. Note also that neither SML nor Haskell has an incremental development enviorment, and therfore wouldn't benefitt from this functionality. (I wonder what happens in Ocaml?)

If you are talking about a REPL, then there are several SML implementations that provide one. Probably the best known is SML/NJ. I usually draft new SML code incrementally in XEmacs sending individual definitions to the inferior SML process running SML/NJ.

You can find some of my comments about O'Caml and Tuareg Mode for EMACS here. Briefly, O'Caml is quite unusual among statically-typed languages inasmuch as it has a "toploop" (REPL), a bytecode compiler with a time-travel debugger, and a native-code compiler. So it's an everyday experience for an O'Caml programmer to have the same "feel" of trying out snippets of code in real time, as with Python or Ruby or Lisp or Smalltalk, without the nasty edit-compile-link-test-crash-debug cycle commonly associated with statically-typed languages. However, the compilers are still there waiting for you if/when you want them, and as I pointed out in the other thread, you can even use omake -P to do continuous builds as you save source files.

After having put far too much energy and verbiage into far too many static/dynamic flame wars, I've finally become convinced that interactive development vs. batch is the real killer app with respect to productivity in all but the most stringent "assured software" environments. Although there are other statically-typed languages with interactive modes (e.g. GHC's ghci), O'Caml's toolset seems to be unusually consistent in its treatment of modules, separate compilation, scoping and binding rules, etc. so that moving from the interactive environment to the compiled environment is effectively trivial. For example, it would be hard for me to overstate the extent to which O'Caml's separate compilation system being based on the module system is a win: signatures and modules in the toploop really do simply become the contents of .mli and .ml files, without change. Couple that with omake's automatic dependency tracking, and it's just ridiculously easy to turn your off-the-cuff exploration into a standalone bytecode or native binary. Great stuff and highly recommended.

I agree that interactive development does boost productivity. The basic reason for this is that you get feedback much faster. Unit tests have the same effect. Without unit tests, it can take days, weeks or even months before you get feedback (bug reports from daily builds, smoke tests, internal QA or customers) on the unintended changes to the behavior of the (legacy) program you are working on. Perhaps even more importantly, unit tests serve as a formal documentation on how parts of the program are supposed to work. My recent experience, working on a largish legacy application written in a language that does indeed support interactive development, suggests that interactive development and interactive testing is no substitute for thorough automated unit testing. It probably works (in the sense that new features can be developed quickly) as long as the original developers are working on the project. After they are gone, the knowledge they should have put into the unit tests, but decided to test only interactively, is also gone, and the pace of development slows down dramatically. I'm not implying that you would have necessarily suggested otherwise. I'm also not implying that you couldn't use both interactive development and unit tests. However, interactive testing is, in my experience, like debugging, a time sink. It probably feels productive, because it keeps your mind busy, but that is a false belief.

I would think the reason for Schemes behavior in this situation is incremental development.

Oh, I'm sure it is. If Scheme started throwing errors on redefinition of a term (like I suggested), I'm sure it wouldn't take long for everybody—including me :-)—to want the old behavior back. There's no question about it: for development purposes, using define twice if perfectly reasonable, but it shouldn't be misused as a source of side effects.

It's one of those things: when you know what your doing, it's nice to redefine a value when developing, however, when you make a mistake, and redefine over a value by accedent, you wish it had given you an error.

It's one of those things: when you know what your doing, it's nice to redefine a value when developing, however, when you make a mistake, and redefine over a value by accedent, you wish it had given you an error.

The way some Schemes deal with this is to allow duplicate defines at the "top level", but not inside modules. This means that you can rely on the relaxed behavior when working in a REPL or in scripts, but when you reach the point of dividing a program into well-defined modules, the compiler will complain about duplicate definitions. This works very well in practice.

The R6RS library proposal codifies this behavior, saying that within a library, "No identifier can be imported multiple times, defined multiple times, or both defined and imported."

It's good to know that there are people out there who have already solved the problem. :-) I haven't used Scheme modules yet, but to me it looks like this is a pretty compelling reason to use them.

"haven't used Scheme modules yet"

It's because modules are not an standardized feature of Scheme, and each implementation has its own way to do it. R6RS should fix that.

In SML the code you gave would be:

- val z = 10;
val z : int = 10
- fun add2 x y = x + y + z;
val add2 : int -> int -> int = _fn
- add2 1 2;
val it : int = 13
- val z = 20;
val z : int = 20
- add2 1 2;
val it : int = 13

To get the behavior that most non-FP languages have, you'd have to use a reference (which IIRC don't exist in Haskell):

- val z = ref 10;
val z : int ref = ref 10
- fun add2 x y = x + y + !z;
val add2 : int -> int -> int = _fn
- add2 1 2;
val it : int = 13
- z := 20;
val it : unit = ()
- add2 1 2;
val it : int = 23

In the first case, there are two values that were named z - though the first z value becomes shadowed after the second one is declared. In the second case, there is one value z that is a reference.

I took a stab at this whole topic in my paper Purely functional structured programming. I based my new programming language Babel-17 on something I call "linear scope" and which is based on the assumption that

val a = 1
val f = x => a*x
a = 2
f a

should evaluate to 2, not 4.

By the way Babel-17 allows multi-statement lambdas :-)

Furthermore, I do not agree that language design is not like solving a puzzle. It is, if you take into account that you are able to change the picture the puzzle represents while you are solving it.

Furthermore, I do not agree that language design is not like solving a puzzle. It is, if you take into account that you are able to change the picture the puzzle represents while you are solving it.

Guido is very driven by the user. He wants to give programmers a great tool in which it is just easy and fun to state programs.

Others, like Haskell and Clean experiment with conciseness and correctness.

For me, and I think a lot of small language developers, it is the intellectual challenge of puzzling together a minimal language with a minimal implementation.

For me, and I think a lot of small language developers, it is the intellectual challenge of puzzling together a minimal language with a minimal implementation.

Yep. The background and experience of the user drive the problem the PL designer is solving, where defining this is as important as or more important than the puzzling process itself. Designing a language in a problem vacuum might be intellectually stimulating, but the result is not that useful. Sometimes I wonder about Haskell, but then perhaps their target users are just more formal than I'm used to.

Much of the work that designers in all disciplines do is related to understanding and mastering the problems of their users. That obviously applies to PL design also.

It took the mathematical community two thousand years to reduce the proof of the theorem of Pythagoras to a one-liner. Was that useful?

I like to think of the algorithms used in a functional compiler in that manner. I have looked at the implementations of Helium, Clean, Haskell and a dozen or so more languages, and I am baffled by the behemoths produced. Especially in comparison to small languages. I just wonder a lot about how to implement stuff in a such manner that you can do without all that complexity and still arrive at ok-ish running time and memory consumption.

Will it produce a useful or fast language? No idea. It beats playing Sokoban.

Minimal design is not really my concern. I was maybe somewhat interested in that 5 to 10 years ago, but not anymore. Today I have several problem spaces that I am looking at that I would like to encompass with one language, among them server-side and client-side and mobile computing. I like beauty and elegance in a language, but minimality is not the only (nor a sufficient, nor a necessary) way to achieve this.

As an example, my language Babel-17 has two constructs for defining things: val and def. You could do with val alone because the language is dynamically typed, and therefore you could define recursive functions via the Y-combinator, so having def in the language for mutually recursive functions is not minimal, but adds to the elegance and beauty of the language.

Looking at the implementation size of a language is probably misleading. Because any time you invent a language that raises the level of abstraction, and you care about performance, there are certain things you need to implement in a lower and less concise language. Do not get fooled by the size of the implementation; often the idea of the implementation fits on a sheet of paper, but to make the idea into something nice and useable, it takes time and effort, and this normally translates into lines of code. So if your implementation only consists of 10 lines of code, then it probably is only worth these 10 lines of code. Because then what you are doing is pretty close to what has been achieved already.

Looking at the implementation size of a language is probably misleading. Because any time you invent a language that raises the level of abstraction, and you care about performance, there are certain things you need to implement in a lower and less concise language.

A lot of people do care about minimalist performant implementations. It is the essential argument of the 'Yacc is Death' discussion in the other thread. Their argument is: 'Look at this minimal design which performs well and even is more general.'

Some people like minimalism, some don't. To each his own.

That discussion is somewhat useless in my eyes :-) Of course yacc is dead, use ANTLR. ANTLR embodies the same idea (top-bottom parsing instead of bottom-top parsing) as parser combinators do. People who use an editor which does real-time parsing of your syntax will thank you.

Searching for minimalist solutions is mathematics. Thats OK. But most of the time when doing mathematics only, you are not solving the problem at hand, but the problem you like.

It is an egocentric self-indulgence mathematics game for me. ;)

I like beauty and elegance in a language, but minimality is not the only (nor a sufficient, nor a necessary) way to achieve this.

The appropriate quote should be "as simple as possible, but not any simpler."

Will it produce a useful or fast language? No idea. It beats playing Sokoban.

One of the points of designing a language is to see it be used, hopefully by more people than the designer. If that isn't your goal, Guido's advice doesn't apply.

Of course, I want it to be used. But, as an exercise, I made a lot of decisions to trivialize the compiler (most compiler steps are transformations on the AST, no symbol table, no local or global variables, interfaces on values instead of classes, factoring out native types in the runtime, using a simple scheme for an operational semantics, etc.). Most of them payed off, some of them didn't. So, I am looking at how to maintain the triviality and still arrive at a usable tool.

Ah? Interesting, I'm designing a language with similar restrictions right now: no symbol table, no binding or variables, everything is applied directly. In other words, all operations operate on tangible values rather than on abstract bindings.

I would argue that what you are doing is rethinking language design. If you are successful, you will still have users eventually, and you should think about them. But then don't worry about their preconceived notions of what a language should be.

Furthermore, I do not agree that language design is not like solving a puzzle. It is, if you take into account that you are able to change the picture the puzzle represents while you are solving it.

Doesn't sound like much fun.

What you are doing with "linear scope" is conceptual work in the first place and I believe it can be deepened and studied separately. I would usually expect a treatment of this in some toy language or "calculus" or even drop that layer and move towards control flow graphs and then turn back to concrete syntax. The educational value of your paper is obvious to me though: when things should be as simple as possible but not simpler, SSA might fall into the latter category. But how do I know that "linear scope" doesn't fall there as well?

Maybe it does but then you could argue that more sophisticated scoping rules complicate the design. So I can solve a more advanced puzzle but this doesn't look too good, confuses the programmer, or leads to architectural frictions. That's a totally different kind of argument. Design is social, technical, aesthetical and the final reason why not two nerds can live together in peace.

Doesn't sound like much fun.

From where I stand, it is rather the other way around: Solving a static puzzle is not that much fun (and also not very realistic, and generally better done by computers than by humans) :-)

Linear scope is just something that developed naturally from thinking about how to make purely functional syntax more convenient. I recently translated linear scope into purely functional continuation passing style, so there are two interpretations for linear scope now, one purely functional, one imperative, and both are very natural.

Also the scope rules of linear scope are not complicated. If you say to a programmer, just treat "val" like "var" in an imperative language, she will do everything right, except when it comes to declaring local functions.

In short, I think linear scope is just an obvious conclusion that one comes to after years both of purely functional academic and day-to-day industrial programming.

The gotcha here is that you can't modify bindings of names defined in outer scopes. Therefore you actually create a new local variable x in the scope of counter. This is dissimilar to Scheme where such things are possible. But this is unrelated to the question whether the scope is fixed at compile-time or not.

import dis
counter = make_counter()
dis.dis(counter)
 55           0 LOAD_FAST                0 (x)
              3 LOAD_CONST               1 (1)
              6 BINARY_ADD          
              7 STORE_FAST               0 (x)

 56          10 LOAD_FAST                0 (x)
             13 RETURN_VALUE        

As someone has said already, i think, this doesn't matter in that objects encapsulate state in a very similar way to closures, and python has objects, but if you try to do "functional" programming in python you get bitten by the above very quickly.

Right. Python is very asymetric in this respect. Since it promotes objects as 'native' types it feels no responsibility in supporting modifiable encapsulated state by means of closures. The obvious equivalent solution of your example is creating a generator:

def counter():
    x = 0
    while 1: 
        yield x

Kay

your suggestion:

def counter():
    x = 0
    while 1: 
        yield x

can only be used like so:

for x in counter():
    dostuff(x)

No idea what use the grandfather poster had in mind, though.

This could be fatal because counter produces a generator that hangs up in an endless-loop if not treated carefully. You will most likely use the generic next-method that each generator obtains.

First I have to correct counter in order to produce non 0 values :)

def counter(start=0):
    x = start
    while 1:
        yield x
        x+=1

>>> inc = counter()
>>> inc.next.__doc__
'x.next() -> the next value, or raise StopIteration'
>>> inc.next()
0
>>> inc.next()
1

I think this is covered by the "There should be one-- and preferably only one --obvious way to do it." bit of the Zen of python[1]. As you said, it can be done with objects, so providing two ways of doing things isn't really needed and is a "bad thing" in scripting languages. I would be interested in seeing an example that doesn't map well into objects, if one exists.

class count:
    def __init__(this): this.x=0
    def inc(this): 
        this.x=this.x+1
        return this.x
    def finc(this): return this.inc
def make_counter(): return count.finc()

Calling make_counter() makes you a counter, calling the result of make_counter increments the counter and returns the new value. I'm not sure what this proves, apart from that you can abuse any language if you try hard enough. Discussing the readability of the code above vs your example is a interesting topic by itself, but yours is probably more readable.

[1] The Zen of python is a hint of the thinking behind python, try running python -c "import this" at a command prompt.

I think this is covered by the "There should be one-- and preferably only one --obvious way to do it." bit of the Zen of python.

Could be, but, well, here's another way -- the one that strikes me as being the closest Python way to "fix" andrew's code (i.e., it's the code that looks the most like andrew's, but does what he was expecting:

def make_counter():
  x = [ 0 ]
  def counter():
    x[ 0 ] += 1
    return x[ 0 ]
  return counter

It's not an uncommon idiom in Python, and its use seems to me like a straightforward consequence of the way assignment (rebinding) works in Python. But maybe that just means that I've been using Python for too long.

You can't do this in haskell either, or any purely functional language (for the same reason you can't change any other variable). I don't see how you can say this is a hindrance to functional programming when the whole point of the above code is to trigger a side effect from the function call!

This works in GHC, though you might need -fglasgow-exts to use STRefs:

type Counter s = ST s Integer

makeCounter :: ST s Counter
makeCounter = do ref <- newSTRef 0
                 return $ modifySTRef ref (+ 1) >> readSTRef ref 

Also see the Haskell entry on C2

And its certainly leaving a purely functional style. If you don't require that you should be able to rebind an actual variable in the enclosing scope, python can do this too:

def makeCounter():
    count = [0]
    def f():
        count[0] += 1
        return count[0]
    return f

However, neither has anything to do with functional programming (after all, we're creating non-idempotent functions).

You can't do this in Haskell either, or any purely functional language (for the same reason you can't change any other variable).

Python is not a purely functional language. Purely functional languages make certain tradeoffs, assignments for laziness is one of them. Python has made no such tradeoff. Assignments are perfectly acceptable in Python.

I don't see how you can say this is a hindrance to functional programming when the whole point of the above code is to trigger a side effect from the function call!

The discussion wasn't concerned with purely functional programming, but Lexical scoping. Haskell captures the essence of Lexical scoping properly. It treats captured variables exactly the same as other variables, just like Scheme, ML, etc. It's simply that Haskell, by default, forbids assignment to any variable. Python is broken, because, unlike Haskell, ML, and Scheme, it treats captured variables differently than other variables. That is the complaint raised here.

Is Python broken? Possibly. Weird? Certainly. Different from Scheme, ML, Haskell, and how lexical scoping is normally done? Yep. I think the problem is that Python is conflating assignment and variable introduction. That always results in strange scoping behavior.

Python's original scoping rules were very weird, and like a lot of Perl, work tolerably well only as long as you didn't think about them too much. Along about version 2.2, as I recall, the current behavior was introduced that acts more-or-less lexically for reads, but continues to introduce a new variable on the first assignment in a scope. Since assignment and variable introduction are the same operation, you simply can't spell writing to a captured variable.

I was too harsh in concluding that Python was broken, when it's simply confusing.

Since assignment and variable introduction are the same operation, you simply can't spell writing to a captured variable.

Yes, this is the essence of the problem (and confusion). Variable binding should be made explicit, this would result in much less confusion. (Weird and seemingly arbitrary scoping rules confuse new users who expect lexical scoping.)

About rebinding of names in enclosing scopes. From PEP 227:

There are technical issues that make it difficult to support
rebinding of names in enclosing scopes, but the primary reason
that it is not allowed in the current proposal is that Guido is
opposed to it. His motivation: it is difficult to support,
because it would require a new mechanism that would allow the
programmer to specify that an assignment in a block is supposed to
rebind the name in an enclosing block; presumably a keyword or
special syntax (x := 3) would make this possible. Given that this
would encourage the use of local variables to hold state that is
better stored in a class instance, it's not worth adding new
syntax to make this possible (in Guido's opinion).

Regards,
Kay

I was objecting to the phrase "but if you try to do "functional" programming in python you get bitten by the above very quickly." in the comment I originally replied to. Claiming this is a hindrance to functional programming is clearly wrong.

I'd agree that it is a wart that you can't rebind variables in enclosing scopes, though I think "broken" is a bit strong. The reason is that python has no difference between assignment and declaration: (define x 5) and (set! x 5) are both written "x=5" in python, hence "x=5" in an inner scope creates a new variable that shadows the one in the enclosing scope, rather than rebinding it. This is not an inconsistency in lexical scoping, its just a lacking feature - python lacks an operator exactly equivalent to scheme's set!.

The bigger problem here is not whether there's a distinction between a named or anonymous function. The real question is whether Python fully supports the concept of higher order functions. For example, can I treat the definition of a function as data construction? Can I Curry functions? Can I combine or compose functions? Can I get closures?

Python has gone a way towards higher order functions but Guido's attempt to eliminate lambda was more concerning to those who appreciate functional programming for the power that it gives us. To those who view functions as data, it would be like saying that you couldn't use a constant value in a statement unless it was defined:

   s = "Hello World"

versus:

   def helloWorld = "Hello World"
   s = helloWorld

Now which one of these is clearer? By the logic used by the Pythonistas, the second is much preferred.

If I said the constant was 3.14 (pi), then I'd agree that giving that constant a name would be good programming practice. Yet, it would be a real pain to name every constant used in every program that you write. Well, in the same way, functions that are reused frequently should be named. But for those used to functional programming, naming every function that you encounter is about as much pain as naming every constant.

Naming things is one of the harder things about programming, as we try to avoid name space pollution. The more meaningful names we have to come up with, the harder it can be to write. Of course, this idea of minimizing names can lead to hard to read code - just as having too many names can make it hard to read. Writing elegant code is not something that can be had by forbidding certain programming styles - though it can be nudged along by such things as offside rules.

The real question is whether Python fully supports the concept of higher order functions. For example, can I treat the definition of a function as data construction? Can I Curry functions? Can I combine or compose functions? Can I get closures?

If that's what you care about, the answer is yes to all of the above. The syntax (and even the existence) of the lambda expression has no bearing on the availability of any of this functionality. It will always be there.

But it isn't the only "real question".

I disagree. The second is less clear, because of code locality. I have no idea what do-stuff-and-add means unless it either does something very simple and is well named, or I have access to the definition, so I can see directly what it does. The easiest way to have access to the definition is to put it right there, i.e. use a lambda. If I am maintaining this code, I'll have to go find the definition of do-stuff-and-add to understand what the map is doing. With the lambda this isn't the case. Now, obviously, if do-stuff-and-add is defined as a function local to the function in which the map is being used, then it isn't a huge issue.

In the end it all comes down to syntax and semantics. The Lisp example is rather silly. It's silly partly because the code didn't get indented properly making it harder to read that it would actually be, but mostly because lambdas work really well in Lisp and Scheme, but don't in Python. So Python programmers would prefer the second, not because they can't parse the first (I'm quite confident they could—it's really not that difficult of a concept), but rather because they are programming in Python, and Python's lambdas are syntactically broken.

And everyone with any sense would prefer some saner indentation! It seems this is beyond Python's capabilities?

(flet ((do-stuff-and-add (x y)
          (a-very-long-procedure x)
          (another-very-long-procedure y)))
   (map 'list #'do-stuff-and-add a b))

That seems to address both of the concerns of naming the function, and keeping the definition of the function local.

Rules, events, backward chaining? Sounds interesting. Could you explain in more detail?

My idea of what the interface of a language should be is a cognative model of ORDINARY human activity. This leads to a backward chaining control structure with multiple types of rules.

That's odd. Most cognitive models that I know of (Soar, ACT-R, EPIC, etc etc) that are based on rules seem to prefer forward chaining for most things. What is "ordinary" human activity?

Actually our error handling is so smooth and natural we are never aware of it.

I'm painfully aware of mine on occassion.

I do not see forward and backward chaining as separate competing methods. But I do think that backward chaining is more general and expressive. Forward systems consist of two languages: an IF clause language and a scripting language used in THEN clauses. When the IF clause is true the script in the THEN clause is executed. A single more general purpose language can do both at once. For instance we can write a forward system in Prolog and use it as a part of the overall system. I show how to do this using my own proposal here .

But the issue is more general than this. Forward, backward, and most other programming models are all closely related. The well known CTM (Concepts, Techniques, and Models of Computer Programming) has made this very clear.

The issue of this thread is interfaces. I like my proposal, but as always it is all there for you to choose from.

But I do think that backward chaining is more general and expressive. Forward systems consist of two languages: an IF clause language and a scripting language used in THEN clauses.

This doesn't have to be the case. You can represent rules as e.g. Horn clauses and then do forward chaining or backward chaining over them. There is nothing about forward chaining which ties it to having (imperative) actions as the consequence of rules. I don't think it really makes much sense to say that some inference procedure is more expressive than another — the underlying language may be identical. You can make distinctions on soundness, completeness, tractability, etc.

For instance we can write a forward system in Prolog and use it as a part of the overall system.

Yes, and you can also do goal-directed processing in a forward-chaining system. Which to choose depends on what you are intending to do. You can even separate out the language (representation of rules, facts) from the inference procedure and have some meta-level system decide which to apply. My point was that appeals to intuition about human cognition don't really justify a choice of backward chaining rules as a general mechanism for computation.

My point was that appeals to intuition about human cognition don't really justify a choice of backward chaining rules as a general mechanism for computation.

I wish I had a better justification! For me it really does come down to intuition. When I try to explain behavior it usually takes a backward form. The same thing happens in ordinary functional programming. A set of nested functions is really part of a backward chain except that we can't backup if it fails. A lot of pragmatic liturature gets into discussion that sounds like backtracking.

Forward chaining usually seems to belong at a very high level. If the system is idle and something happens where do we start. What is the first goal. In general I find goal representation very difficult in forward chaining. At the same time a goal representation in backward chaining may be all that is needed.

Forward chaining usually seems to belong at a very high level.

I usually see things the other way round. Looking from an "intelligent agents" design perspective, forward chaining gives a natural way to do reactive style behaviours where you want to respond to events occurring in some environment. Backward chaining is more usually associated with higher level deliberative behaviour, pursuing long term goals. Of course, that's a crude characterisation, as few systems are purely reactive or purely deliberative.

In general I find goal representation very difficult in forward chaining.

There are various ways to represent goals in a forward-chaining system. One simple way it to just assert the current goal as a fact in working memory and then have rules match against the current goal in their antecedents. That's just a simple way to prune the currently applicable rules to just those relevant to the current goal, though you are still doing forward chaining. You can go further and write a backward chaining rule interpreter in terms of forward chaining rules (not too unlike doing the reverse in Prolog) but my personal preference would probably be to write both forward and backward chaining (and the whole rule system) in a language like Scheme (or Haskell, ML, etc).

Yes, I like forward systems but what about the forward system as an interface for general purpose programming. How does it make programming more accessable and understandable for more people even including non-programmers. Can we agree in general that rules are a way to do this? Rules are already widely used by non-programmers in business and as a type of systems analysis tool. It is not too much of a stretch to come up with a system that works for everybody.

My idea of what the interface of a language should be is a cognative model of ORDINARY human activity.

The relationship between PL's and human cognition is very interesting, but also very complex. I think there are two major reasons why mimicking cognition with a PL migh fail, or should be implemented with great care.

First, the relationship is bidirectional. Humans are very flexible and our environment, including PL's we use, have an impact on how we think. If you were to design a PL for people of stone age, would you use their none/one/many way of calculation, or instead teach them to use numbers? I could argue that if there is such a thing as a perfect programming language, we are not yet clever enough to use it.

Second, programming is not only about thinking. If you look at modern programming environments, like J2EE, the design largely boils down to social and economical issues: avoiding vendor lock-in, dividing responsibilities between different developer roles, allowing for integration even when it is not technically motivated (company mergers etc.), preventing uninformed programmers from doing stupid things etc.

I agree that cognition is too complex to use directly. My strategy is to use theory common in engineering for studying systems as a part of their environment. Now days this seems to be called systems theory or formerly Cybernetics. If you try to computerize these ideas you get a form of representation almost equivalent to ordinary rules and facts as commonly used. The difference is that you also need a concept of ongoing change or situation and named action. Taken together I like to call this activity theory. Activity theory is an implied foundation of a theory of cognition because it is fundamental. Systems theory is thought to apply to systems of any kind whether biological or mechanical. A closely related way of looking at things would be pragmatic philosophy.

Other programmers, of course.

Bingo! Give that man a large stuffed python...

Dave, are you a time-traveller from C2 Wiki of the past? :-)

To me a programming language is the user-interface of a compiler and the language spec is a manual for the compiler. A good language description is one that tells me how to make the compiler do what I want. The UI goof that annoys me most is when lots of things are specified as "undefined" and you're prevented from trusting your observations an experiments. That is bad UI. My $0.02.

s/compiler/compiler or interpreter/g

There seems to be a strong focus in your mind on programming as a solitary endeavor. You see this a lot in the scripting community. To each his own, I suppose. I personally am not interested in problems small enough to be coded by a single developer, even myself. I wish to build cathedrals, with the trained teams of architects, engineers, craftsmen, and managers that such enormous endeavors imply. Such teams need languages to communicate their designs in, so that they can be clearly, quickly, and accurately understood, even as teams and technologies change. The only practical way we have found to make such languages sufficiently rich and our communications sufficiently rigorous is to use languages that are actually runnable by a computer, but that's somewhat incidental, and has both good and bad side effects.

Saying that a programming language is an interface to a compiler sounds like saying that typewriter is an interface to a piece of paper. It's true, in it's way, but only at the cost of eliminating all sense of purpose or context from the discussion.

I would be curious to know which software cathedrals you admire as paradigms of good design and successful implementation.

(As a show of good faith, two of my favourites are Unix and Emacs.)

Favorites?

GCC - amazing insights into the limits of portability

Boost libraries - Pushing a horrible language far beyond what anyone thought were it's limits, and arriving somewhere beautiful

The J2SE class libraries - Breadth and quality far beyond the expectations of a least-common-denominator standard library. It's sheer completeness managed to create a programming system with hardly any "gotchas".

IntelliJ IDEA - architecturally nothing amazing, but the exuburant blossomings of amazing functionality in unexpected places constantly enthrall

Opera, BeOS - commercially insane, but with enough technological sweetness to make you simply want to give them money to continue

BSD - the first programmer's OS

SABRE - uptime at all costs

ITunes - architecture perfectly tuned to create rabid customers.

I think we're now sufficiently off topic...

the original Smalltalk environment - far beyond anything anyone would have believed was technologically possible for the time.

Doom - In order to build this breakthrough game, first we need to build a multi-tasking OS for it to run on...

Google - The first software to actually change the way that people think

"Cathedrals" seems appropriate for these

Your taste seems excellent to me, but I don't see many of these as cathedrals. Weren't gcc, BSD, Smalltalk, Doom, and Google (web search engine) all designed by small groups of largely self-directed programmers who were focused on making something run?

In a substantial project, more of the developers' effort is with the program than with the machine/OS/etc., and that's where the language provides an interface: between the team and the program they're developing or maintaining.

a language that can both explicate the architecture design decisions of a 20-person, 5-year development effort, and still run

Experience casts doubt on the achievability of this admirable goal. In current practice other, non-programming languages are used for software architecture. Therefore, architectural communications among programmers is not particularly relevant to the design of a language like Python with no pretentions to being cutting-edge for 100 person-year projects.

In current practice other, non-programming languages are used for software architecture.

Which is one of the primary reasons for the sorry state of the software industry.

Hi Ehud, are you sure about the "sorry state of SW industry"?

You might checkout this (unfortunately not free) article that was published on IEEE Software May/June 2005.

Regards,
Kay

The link doesn't work, try

Robert L. Glass. "IT Failure Rates--70% or 10-15%?," IEEE Software, vol. 22, no. 3, pp. 112, 110-111, May/June, 2005.

In this editorial the author questions the oft cited failure rate of 70% for software projects. This figure goes back to the 1994 Standish group Chaos report. Apparently more recent reports show some improvement, but Glass remains sceptical of these figures as do others. The average 189% cost overrun is also put in doubt.

Yes, I am aware of these publications. They are not enough to make me change my mind...

Once can argue against the specific statistics, but I think it is enough to visit most software development shops, and see their practices and the quality of the product.

Try to read about the state of the art as regards "software development methodologies" and tell me if you still think talking about the "sorry state of se" is an overstatement...

tell me if you still think talking about the "sorry state of se" is an overstatement...

Not sure if you're addressing me, as I didn't express an opinion. From what little exposure I have to real world (as some call it) Software Engineering, it still seems an immature discipline to me.

Well, Kay was they first to mention these studies not you...

But, anyway, I wasn't addressing anyone, just ranting...

Glass remains sceptical of these figures as do others.

Actually, the Infoworld article ends by saying: Probably the news with the most damaging implications for IT projects is not the number of those that were abandoned, rather it’s those that were completed but offer fewer features and functions than originally specified...

This is pacifying to listen. I work ( partially ) against specs that are written by ETSI and it needs a lawyer to interpret them. Many features are just there for political reasons i.e. company agreements. Not to talk about historical concessions. Others bloat the system and are excessively complicated. Some features are never used. To blame software developers for creating bad products is quite convenient and part of their folklore - I guess that's because they have to read the sourcecode of other people, not because their Windows system crashes one time a week ( that latter did not happen to me for years - Kudos to Microsoft ). Nevertheless most products ( in the small section of my own experience ) are on budget simply because they do not reinvent the wheel and suffer from the "second system syndrome". When people do everything from scratch and do not acquire skilled architects, developers, supporters and project managers from the beginning they will likely going to fail - but this social fact is quite trivial and not anyhow distinct from other engineering domains. Once Alistair Cockburn had something interesting to say about this with regard to processes. I'm to busy in the moment to look for this on the Web.

I am not sure I understand what you are trying to say...

To blame software developers for creating bad products is quite convenient and part of their folklore

I, for one, didn't blame developers. In fact, I think talking about developers as if such a beast exists and is well defined, is a symptom of the general problem I was referring to.

So you believe once the role X becomes well-defined the "software-crises" has the chance to evapourate? This is too close to magical thinking for me to agree. On the other hand some people argued here on LtU that the "software-crises" would be solved once nobody writes sofware anymore but just specware, designware and docuware. With this I agree of course ;)

Kay

No, it's the other way around. Once the "software-crises" is "solved", the role X will will also be clarified...

I took this as an example of the natural human tendency to always desire more than we can deliver. I don't think it's necessarily a bad thing that most software projects eventually scale back their ambitions; after all, isn't that something that most of is have to go through when growing up? Perhaps it's a sign that the field is "growing up" as well.

There's probably also an element of "Software gives me God-like powers, so I can do anything!" in this, without realizing that doing "anything" requires a whole lot of work. As mentioned above, most features in software have very little utility for the average user. Sometimes, the finished project is better off without them.

Come to think of it, I think that's exactly what Guido is referring to in the original post. Just because we can solve the lambda problem doesn't mean we should...it would take away resources needed in other areas and add complexity that would further bog down development.

That's what I found interesting about the "agile revolution". Instead of talking about magic technologies that will solve all our problems once everyone uses them its proponents focussed on communication. How can one complex social system ( a SW development team ) practically realize conviviality with its environment i.e. users and customers. As long as we believe that this is a command-control process guided by some input-logic software will inevitably fail to fit the "requirements". The requirements, the design and the software are a social relationship that emerges.

Come to think of it, I think that's exactly what Guido is referring to in the original post. Just because we can solve the lambda problem doesn't mean we should...it would take away resources needed in other areas and add complexity that would further bog down development.

Guido, Larry, Mats etc. are community leaders. So they mediate forces and channeling requirements according to their design ideas that attract the masses. There is some deep reciprocity. I can imagine to liberate Python from the singularity of a continously active chief designer such as happened with Lisp. But this causes the dual problem of standardisation and preserving "character". As a developer on my own I have to defend sometimes my SW architecture from the wishes of other people - for their own sake. Three wishes are free only. If you as a developer have a mature user/customer you are a lucky person and vice versa. Ripeness is all, as Shakespeare said.

The rates of cost overrun and time overrun have little to do with the sorry state of software development. They might just mean that upper management is getting used to just how incredibly expensive and time-consuming most software is to build, and how often quality problems arise, and factored them into their expections. The underlying suckiness of development outputs may not have changed in the slightest.

Out of curiosity, what do you advocate for describing software architecture ? If you hate/despise UML, well, count me in. But what is the alternative ? Are you saying that a programming language (which one ?) is appropriate for describing at a sufficiently high level the architecture of an application ? Or more globally, the architecture of an information system ? Or did you use the word "architecture" while thinking more of the detailed design of an application ?

Out of curiosity, what do you advocate for describing software architecture? If you hate/despise UML, well, count me in.

The basic problem with separate architecture description languages is simply that they are separate from the implementation language. That means that architecturally useful information is unavailable to the programmer, his peers, the compiler, and static analysis tools. Because of that, artifacts in architectural description languages are basically always out-of-synch with the implementation, making them useful only for historical purposes. Most devlopers don't even bother to read them.

Think of it this way: Why do we have build scripts? Why do we have installer creation scripts? There's no reason that all of the architectural and design information necessary to drive build and deployment (dependencies, versioning, modularization, configuration) couldn't be declaratively exposed in the implementation language. There, you get all the good stuff we're used to when we're working with advanced implementation language: error checking, high-quality navigation tools, code assistance. Except in this case the error checking would be showing you places where your implementation violates your architectural declarations, navigation would help you move seamlessly between architecture and implementation, and code assistance would help you either infer architectural annotations from implementation or generate implementation stubs from architecture.

Are you saying that a programming language (which one ?) is appropriate for describing at a sufficiently high level the architecture of an application ?

Current programming languages are inadequate for the task. That said, there's no reason that they couldn't grow to include the sorts of architectural information necessary. You could consider object-orientation, design-by-contract, and ML-like module systems all to be baby steps in the direction I'm describing. All assist in taking implicit design information and making it explicit, within the context of an implementation language.

The basic problem with separate architecture description languages is simply that they are separate from the implementation language. That means that architecturally useful information is unavailable to the programmer, his peers, the compiler, and static analysis tools. Because of that, artifacts in architectural description languages are basically always out-of-synch with the implementation, making them useful only for historical purposes. Most devlopers don't even bother to read them.

I think you are making the assumption that we are talking about a single software application, implemented in a single programming language. In my post, I was trying to stress the difference of perspective between the small-scale (single application) and the large- to very-large scale (say, global systems & software architecture of an IT department, numerous applications and languages). That being said, what you describe is a common and well-known problem with modeling tools and there have been various attempts at keeping the code and modeling language in sync, although much/most remains to be done. That's what MDA is tackling in its own way, but don't count me in the believers.

Current programming languages are inadequate for the task. That said, there's no reason that they couldn't grow to include the sorts of architectural information necessary. You could consider object-orientation, design-by-contract, and ML-like module systems all to be baby steps in the direction I'm describing. All assist in taking implicit design information and making it explicit, within the context of an implementation language.

That's true, but that leaves us at the "baby steps" stage and with the same question: how can (future) programming languages describe the architecture of a software application at the right conceptual level ? And is that going to help us in expressing / figuring out the global architecture of the 10000 applications used by a large IT shop ?

You have to learn to walk before you can run. Improvement is a lot slower than you might like because it has to move through a complex social consensus system rather than simply work by idealised mandate, but that doesn't mean it isn't happening. Object Oriented design was a (small) step in the general direction of allowing design and architecture to be made more explicit at the level of implementation language. It also helps at the larger conceptual level by providing at least some framework for understanding how different OO based applications might fit together. Having a common VM language, be it the JVM, Microsoft's CLR, or something like Parrot, is another (small) step that provides the ability for a wider array of applications, potentially in different languages, to all share a common object/design/architecture description making the higher level multi-application description a little easier.

We're still a long way from what could be considered "good" - I am personally ambivalent as to whether OO is the right long term answer - but over the last 30 or 40 years the general consensus view has shifted and we now have better methods for explicating design information, and are currently in the process of slowly migrating further in that direction. We still need people at the vanguard yelling for everyone to catch up, but we also need people back in the pack cajoling the stragglers and helping to move the bulk of the group in the right direction. The latter are the baby steps, and they're quite necessary if the mob is ever going to make it to the lofty heights you're shouting from.

You might take a look on ADLs ( architecture description languages ). I don't know if they were yet on topic here on LtU? Ehud!

You might take a look on ADLs ( architecture description languages ). I don't know if they were yet on topic here on LtU? Ehud!

Yep, I know about ADLs but last time I looked, it reminded me of a graveyard i.e. mostly dead projects or projects with very low traction. Sad. It's like everyone has moved to UML or given up on the idea (whichever is worse).

check out the ArchJava project. It's a (fairly) small and (mostly) understandable Java extension with a lot of interesting architectural and design informations embedded in it.

UML as terminator of ADLs would be definitely a "sorry state in software architecture". To make it clear: I expect from ADLs that I can communicate visual representations with product managers that use powerpoint for such issues! I want graphically distinct different type of hardware and software entities. I want to pronounce that certain communication is wireless and encrypted that they are client- or serverside etc. I also want to describe certain workflow and have a visual environment that animates it. It should be possible to extract data from production system and feed them into ADL scripts ( in realtime? ) so it is possible to monitor certain aspects of the system. The ADL should be powerfull enough to write testscripts without pain. I want it to be way cool not a waste-product of OO detail design.

Experience casts doubt on the achievability of this admirable goal.

I would disagree, mostly because there's so much that manifestly could be done at the language level, but isn't.

Some examples

• Real, first class module systems, with explicit dependency and version management
• Bullet-proof object lifecycle management
• Full Design-by-contract, including resource usage contracts and exception specifications
• Type systems that include rich aliasing, ownership, and effect annotations

Now those prone to complain about verbosity are probably polishing their pitchforks, as a language that supported all of that would end up with fifteen lines of specification and structural code for every line of algorithmic code. Count in unit and functional tests, and you're probably talking about thirty lines of support code for every line of product algorithm.

Now those prone to complain about verbosity are probably polishing their pitchforks, as a language that supported all of that would end up with fifteen lines of specification and structural code for every line of algorithmic code.

I think these sorts of arguments come down to an issue of "the right tool for the job". There are plenty of projects where the extra verbosity you suggest is paid back many times over by the increased maintainability. If you don't spend much time working on such projects then it looks like a lot of wasted effort (and may well be for the projects you work on). As long as people are looking through the lens of the sort of work they personally do there will continue to be stupid debates. I tried to address some of this in a post about static and dynamic typing - another perennial debate, and to me pretty much the same one: decent specification is simply the next step up from static types. It provides much of the same sorts of benefits for correctness and maintainability for much the same sorts of costs in terms of flexibility and potential verbosity. When people can start seeing dynamic types, static types, design by contract, and full specification as a continuous scale, have a decent understanding of the costs and benefits of various points on the scale, and select the level most suitable for the project at hand... then I think we'll finally see some progress.

Treehouse vs. house vs. bridge is a perfect analogy for what I'm talking about, and the sliding scale you talk about with DBC between static typing and formal specification fits perfectly. I will be reusing those. Thanks.

Actually, I was thinking about Wittgenstein, though Laozi would fit the profile as well. :-)

Ah, now I get it. But I'm sure the semantic calculus is incomplete so that we can make sense also with those entities that are not addressable. Interface design issues might refer to aesthetic values that we can't talk about in clear terms according to the Tractatus. The interface-description is hypothetical: I can imagine a super-language of Python that has no connection to the real but feels nevertheless "Pythonic". The underlying calculus/machinery is just a machinery as-if. In the end we can try to separate the style, the surface and the vocabulary from the referent. The language becomes ghostly i.e. an idea.

The interface is totalitarian. It encloses the user within a fixed mindest and does not leave any space for criticism. That's why we will always have several incommensurable languages (cultures, religions, social systems etc. ). Strange enough but from this perspective the rationalist Guido van Rossum and the postmodernist Larry Wall achieve exactly the same despite their philosophical differences. I'm not sure who failed more? The postmodernist who tries to enclose people in a non-design design that liberates them from design but therefore encloses them in several awkward conventions and practices. The modernist who limits himself by his wish to create a consistent whole? The failure of the postmodernist is paradox and ironic. The failure of the modernist is tragic.

Kay

Well, I find this point weird, but if you look at failure, you'll notice that the post modernist have been trying for a long time to recreate Perl in Perl6 without much success or interest of many I might say: those who want a "better Perl 5" can go to Ruby now, whereas Python backers don't really feel the need to reinvent Python..

Now given their number of users you could say exactly the opposite: both have won (and Perl more than Python).