Udell: A strategic vision for dynamic languages

We're going to need a fabric of pervasive intermediation, and the TCP/IP of Web services -- that is, SOAP -- will enable that. But we're also going to need a whole lot of agility woven into that fabric. I want middleware that works like Indigo, but I want to program it in a language that works like Python.

Jon Udell is one of the champions of so-called dynamic languages. This item is yet another attempt to explain the importance of dynamic languages and the central role they should play in building complex networked applications.

I am sure all you guys debating static typing in the forum are going to find this worth reading (and worth debating).

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

that's rude

I really don't like his characterization of TCP/IP as a bloated, insecure, ad hoc mess.

/snark

p.s I did like hearing that there was a 40% chance that soap might still fail though.

Desirable properties of dynamic languages

I suspect that all the genuinely desirable properties of dynamic languages will turn out to be properties of some as yet uncreated statically-typed language X, but while we're waiting for X to arrive perhaps it would be a good idea to try to outline what some of those desirable properties are.

(Microsoft once did a survey to find out what extra features people wanted in the next edition of MS Word. Apparently some large proportion of the responses were requests for features that already existed, that the users simply didn't know how to access. It's always worth checking: we might find that we already have language X after all (and it's O'Caml)).

A lot of the puff I've heard for dynamic languages is a slightly queasy-making mish-mash of vague market-speak, pseudo-zen pontification and breathless personal testimony. Python is agile. It has flow. And oh my gosh, it's just so fun to use and has made me twenty times more productive!. Well, I happen to like Python a lot. But I'd like to have better reasons - or better articulated reasons - than these for liking it.

I think that the largest benefit of dynamic languages in the context Udell's discussing is that they accommodate a staged software construction process. The big deal is not that decisions about typing can be deferred until run-time, but that they can be deferred until a later stage of development - which might additionally take place while the software is still running, on a server somewhere. To say that dynamic languages accommodate staged development is not to say that they support it, of course: you need a significant scaffolding of unit tests in order to feel confident that stage N+1 has not broken something in stage N, and ensuring adequate (let alone complete) test coverage is a difficult problem in itself. But I think it can be argued that Python is at least more accommodating than Java in this regard.

Python

it's just so fun to use and has made me twenty times more productive!. Well, I happen to like Python a lot.

I like Python. It is my "scripting language" of choice. In fact, I am working on a little Python tool as we speak.

For what it's worth some of the things that made me productive were that I could use lists, map over them (and apply anonymous functions) and use list comprehensions.

Shouldn't the functional languages heritage be acknowledged more often?

Sure

A lot of my favourite things about Python are taken from functional languages.

The other thing I like a lot is the degree of control you have over what happens when something tries to invoke a method or set/get an attribute on an object. Support for metaprogramming is one of Python's stronger points.

I think Frank emitted a jibe in passing about how all "scripting languages" were just Scheme with added syntax. Actually, I think that's about right. Scheme with added syntax is pretty much what I want, and Python, Ruby and Javascript are all reasonably acceptable variations on that theme.

functional python

will python 3000 (the next big change) be less functional?
IIRC GvR is going to kill the various lambda, map,filter & reduce leaving just generator expressions/list comprehension. Ok, I know LC are a functional heritage too, but I kind of liked map & lambda :/

re: functional python

How could they get rid of reduce? I'm fine with getting rid of map and filter--list comprehensions replace them pretty well--but reduce still hasn't been replaced. Anyone who's used true lazy lists (aka streams, as Scheme has them) doesn't really like generators; they're like lazy lists' evil, incompitent twin. When accessing the next "element", you loose the past one. With function decorators (which I assume will also be generator decorators), you could solve this (I guess you already could today, it'd just be ugly) by making a class to memoize it, but then GvR doesn't seem to like it too much when we make abstractions....

Waiting for future type systems

I suspect that all the genuinely desirable properties of dynamic languages will turn out to be properties of some as yet uncreated statically-typed language X, but while we're waiting for X to arrive

I often wonder if that lack of waiting--the let's make something useful right now, theory be damned, philosophy--is what makes dynamic typing so appealing. So many functional language arguments include statements like "You can get that kind of type system right now in language X" where X is some not-anywhere-near-production-ready research project. So while it may be true that a future statically typed language will encompass the properties of currently popular dynamically typed languages, waiting for it isn't an option.

Agreed

James Hague: So while it may be true that a future statically typed language will encompass the properties of currently popular dynamically typed languages, waiting for it isn't an option.

I want to make clear that I agree with this.

My points of concern—not disagreement, just concern—are:

  • Correctness concerns. Most of the debate to this point has been about these, and perhaps that wasn't an effective focus.
  • Ossification concerns ("Well, sure, this new statically type-inferred does what Python does more safely and faster, but we already have 100,000 lines of Python.")
  • Mindset lock-in concerns ("Hey, our 100,000 lines of Python works most of the time, nevermind that it look five years to iron out all the priority-one bugs. Why should we change?")
I guess, having spent decades around lots of communities (the Lisp community, the Perl community, the Java community, the Pascal community, the C community, the C++ community...) I end up worrying a lot about the latter. Every day I'm shocked at the lack of recognition, or even acceptance, of our industry's failures, and the lack of questioning of whether very different tool choices might not make a significant difference. Yes, I'm aware that there are other issues too: education, methodology, documentation, etc. But as "Code Complete" points out, writing code is the one activity that's guaranteed to happen on a project, so I tend to focus my attentions there. The bottom line for me is that I agree with Alan Kay:
A twentieth century problem is that technology has become too "easy". When it was hard to do anything whether good or bad, enough time was taken so that the result was usually good. Now we can make things almost trivially, especially in software, but most of the designs are trivial as well. This is inverse vandalism: the making of things because you can. Couple this to even less sophisticated buyers and you have generated an exploitation marketplace similar to that set up for teenagers.
I also agree with Kay's assertion that software engineering doesn't exist, so until it does let's use really-late-bound languages. That's the POV I take when I think about Oz, for example. But I also end up devoting an increasing amount of my time to exploring where languages with a formal type-theoretic basis are going, and given some of the results we're seeing now, I find that very exciting.

Still, in the final analysis, I do have to agree that the expressive/convenience difference between, say, O'Caml and Python is epsilon, so the only concrete reason I can think of to choose O'Caml over Python is speed, and heaven knows there are other DLs that are faster than Python (pick any compiled Lisp or Scheme, for example).

Inconsistency?

Paul Snively wrote:

James Hague: "So while it may be true that a future statically typed language will encompass the properties of currently popular dynamically typed languages, waiting for it isn't an option."

I want to make clear that I agree with this.

If you agree with James Hague (good call!), then you're implicitly acknowledging something about the richness or at least flexibility of the latent type systems in dynamically checked languages. That is, unless you have some other explanation for the relevant DL properties.

It doesn't seem consistent to simultaneously hold that position along with what you more recently wrote in another post, "there are no grounds for even suggesting, let alone claiming, that DLs are richer than SLs".

Consistently Inconsistent?

Anton van Straaten; If you agree with James Hague (good call!), then you're implicitly acknowledging something about the richness or at least flexibility of the latent type systems in dynamically checked languages. That is, unless you have some other explanation for the relevant DL properties.

In this particular case, all I'm agreeing with is James' observation that in industry we don't have the luxury of waiting for, e.g. Epigram to show up on job listings.

Now, having said that, I do happen to know that many DLs do have quite rich latent type systems. What I also know is that they don't have an extensive body of literature describing foundational support for the definition of those systems. As you've no doubt noticed, this bothers me a lot.

Anton: It doesn't seem consistent to simultaneously hold that position along with what you more recently wrote in another post, "there are no grounds for even suggesting, let alone claiming, that DLs are richer than SLs".

I'll get there on the other thread, but let me just note here that I have what feels to me like exactly the same "er" problem that you had with the untyped/unityped error that Frank and I made: we now seem to have shifted back to the (perceived) claim that DLs are somehow richer than SLs.

Exactly!

Anton van Straaten: "If you agree with James Hague (good call!), then you're implicitly acknowledging something about the richness or at least flexibility of the latent type systems in dynamically checked languages. That is, unless you have some other explanation for the relevant DL properties."

In this particular case, all I'm agreeing with is James' observation that in industry we don't have the luxury of waiting for, e.g. Epigram to show up on job listings.

Exactly! What is it that makes ordinary DL's capable of doing something that is cutting edge in a more static context?
What I also know is that they don't have an extensive body of literature describing foundational support for the definition of those systems. As you've no doubt noticed, this bothers me a lot.
It's not as bad as you think. Ordinary type theory applies just as much to dynamically-checked languages as it does to statically-checked languages. If you want to see this for yourself, since you're familiar with Python, go take a Python program you're familiar with and annotate it with its types. I think it'll be a lot easier to discuss this if we each have some knowledge of specifics.
we now seem to have shifted back to the (perceived) claim that DLs are somehow richer than SLs.
Richer in a certain dimension, yes. Again, this shouldn't be a controversial claim.

Soap is not a protocol

Soap is packaging. I can do Soap over smtp, http, ...
Comparing it to tcp/ip is only valid in the sense of something that's everywhere.

TCP/IP is not a protocol

I can do it over SLIP, Ethernet, ...

Seriously, SOAP is a link in a big foodchain of abstractions, it preys on transports, and is eaten by more high-level (non-)protocols like WebServices.

Uhhh

Ofcourse you can run TCP/IP over multiple transports, it's a protocol. SOAP doesn't tell you anything about garantueed delivery (or not), idem potency (or not) etc. TCP/IP does, best as I remember.
(I am a little out of my sweetspot here, so I could be wrong)

Half-agree

I agree it was wrong to compare SOAP to TCP/IP, as WS use SOAP just as one of possible bindings (at least in theory), which is usually not true about TCP/IP users.

And SOAP is positioned as a protocol by its creators (and that P in the name has something to do with protocols).

But our discussion goes away from PLT. I suggest we just agree that it was too poetic to compare SOAP for WS to TCP/IP for whatever was implied. Can we allow artistic license for programmers time to time? :)

Deal :-)

Back to closures!

The whitepaper

While I think the ActiveState whitepaper contains some useful ideas and starting points, I think it really calls for detailed debunking. Vague terms ("Technical purity", "Advanced technologies" etc.) are used throughout, but I suggest starting with the definition:

For the purposes of this paper, the term dynamic languages refers to high-level, grassroots, open source programming languages with dynamic typing, including but not limited to Perl, PHP, Python, Tcl, and Ruby.

I must admit that I think that the fact that the definition combines sociological ("community") issues and technological properties (limited, alas, to "dynamic typing") is A Good Thing. At first I was appalled that the definition isn't purely concerned with language features, but since I think the community issues are important, I think putting them up front is a good idea.

However, the technical discussion is flawed. I think the subject is better served by Ousterhout's paper (which isn't devoid of problems itself).

One question: Provided we pick an open source Scheme implementation (I can think of a couple), shouldn't Scheme qualify? If it does, why isn't it mentioned given that it is a well established language with at least some name-recognition. Or is it regard as a "legacy language"? The way the whitepaper is written you'd think being a "dynamic language" is simply about being fashionable.

I think the whitepaper doesn't do the subject justice. I have my reservations about the term "dynamic languages", and I admit I like type systems, but I think Python and ilk deserve to be studied, and do form an interesting category. It's more than just being "pragmatic" and "open source". Or is it?

Nits with the paper

Technical purity:
Dynamic languages were designed to solve the technical problems faced by their inventors, not to address specific goals identified as part of a 'strategic plan' to influence buyers of IT solutions..

The same could be said of C and other non-dynamic lab born languages. This seems to be a direct swipe against C# and VB.


Optimizing person-time, not computer-time:
The driving forces for the creation of each major dynamic language centered on making tasks easier for people, with raw computer performance a secondary concern...

One big gotcha for using "dynamic languages" is that it places much more runtime checking onto the programmer's shoulders. Hence, my unit tests have to cover much more than intent of code, but deal with the dynamicism introduced by the language.

I've done production systems and applications in Perl and Tcl (and Python too).

Even with perl warnings turned on and "use strict" pragma engaged, I have still found a few problems (years later) in my perl code regarding variable use (doh! which context am I executing in?).

Some of my Tcl systems have crashed weeks later due to variable interpolation errors.

In the end, I am still glad I built these systems using dynamic languages (they allowed me to get something working quicker and shortened the turnaround on feedback), but in the long run I don't think they made me more productive. I was able to get something running quickly, but if I were to follow the "old school" approach: Prototype in a scripting language and then implement in a "production" (stricter) language, would I get the same benefits?

scheme with

it was scheme with different syntax, not added, and without closures IIRC.

FWIW

I think Ruby manages closures just fine. Python not so fine - you can do

def adderOfNToX(n):
   def addNToX(x):
      return n+x
   return addNToX

but not

def incrementer(start):
   def increment():
      start += 1
      return start
   return increment

This can be worked around, but it's still a blemish. Continations are another thing that Ruby does and Python doesn't (that is, Ruby has call/cc; obviously you can write Python in CPS if you want to).

I wouldn't miss Python's lambda much

It's occasionally useful in a tight spot, but not nearly as useful as it could be.

List/generator comprehensions make map and filter fairly redundant, but there isn't an equivalent comprehension syntax for reduce. Perhaps Python 3000 will have one. OTOH, Python's reduce isn't really that fold-like anyway (there's no reducel and reducer). Programming with folds isn't really all that idiomatic in an imperative language like Python.

itertools

itertools already has a function for building a list from a generator as it is accessed, so that you can run through the same list of values more than once.

Drat

Parent should be the comment about functional Python, not root.