Lambda the Ultimate

inactiveTopic Mixing paradigms (Python)
started 3/14/2003; 1:53:49 PM - last post 9/12/2003; 6:50:05 AM
Ehud Lamm - Mixing paradigms (Python)  blueArrow
3/14/2003; 1:53:49 PM (reads: 759, responses: 13)
Short story.

We often hear comments about how Python and other new age scripting languages incorporate functional programming constructs and idioms. This is true, as far as it goes. But mixing paradigms can be dangerous.

It has been a while since I used Python (I am doing very little coding these days), but today I hacked a short program (in fact, I am building a very small DSEL). Anyway, Python is a nice enough language, and the basic constructs are familiar, so I didn't expect much trouble, and indeed things went smoothly.

However, at one point I wanted to sort a list of events. Events are, obviously, objects so I has to supply a comparison routine, but I remembered Python now has lambdas so I was ready to go.

Python provides a ".sort([cmpfunc])" method for "inplace sorting" of lists, so I figured that I simply need to supply the very simple comparison routine, that compare the time fields from two events.

My code looked like this: e1,e2: e1.time>e2.time or -1)

(events being the event list argument given to the class constructor).

Alas, when I added this simple line, the program stopped working. In fact, the line itself didn't raise any error, but on one of the next lines I got an "iteration over non-sequence" error.

Time to debug, you say.

But no. I was fairly certain that .sort() works, so I concluded that the error must come from inline comparison function. I tested it, and it seemed to return the expected results. So what gives?

I left for dinner, and when I came back I knew where the problem was. I didn't think about this over dinner. At least not consciously.

The mistake (did you find it already?) is that .sort doesn't return a sorted list. It modifies the object it belongs to.

So events.sort(lambda e1,e2: e1.time>e2.time or -1) sorted the events just fine. But this expression has no return value, so by assigning it to I wasn't initializing the field I was trying to initialize.

Changing the statement to e1,e2: e1.time>e2.time or -1)

solved the problem.

So lambda or not, Python isn't a functional language (nor does it claim to be). Getting carried away with programming Python in a functional style, you can forget that the language isn't built this way to the core, and builtin operators will not always do what you expect them to do. God forbid, they can even mutate something...

Bryn Keller - Re: Mixing paradigms (Python)  blueArrow
3/14/2003; 3:48:55 PM (reads: 770, responses: 0)
Back when I had a thing called "spare time", I used to work on a toolkit that would give Python more of a functional feel. First I added support for curry other useful HOFs, and closures (this was in 1.52 days, when closures were for people with pointy heads ;-) ), and then added some stuff for lazy sequences, and I was all set to add in a bunch of little extensions for __builtins__ to make functional programming easier. Little things, like:

def xprint(thing): print thing return thing

A replacement for list.sort() was high on my list also. :-)

Then two things happened: I ran out of spare time, and I realized that Python people (by and large, there are exceptions) couldn't care less about functional programming, and consider map(), filter(), and reduce() to be warts on the language. Oh well.

Tj - Re: Mixing paradigms (Python)  blueArrow
3/15/2003; 12:21:21 AM (reads: 757, responses: 0)
That's my most common python bug. In either Norvig's or David Mertz' examples, care is taken to redefine these functions so they are functional.

Things like this is why I would repeatedly look at the java api docs before using something I hadn't for a while. Maybe python's real problem is programming's so fast and carefree, that you don't notice the speedbumps until it's too late.

Isaac Gouy - Re: Mixing paradigms (Python)  blueArrow
3/15/2003; 10:48:25 AM (reads: 739, responses: 1)
But mixing paradigms can be dangerous
I'm interested in hearing more about experiences with multi-paradigm and hybrid languages. They are becoming almost commonplace but how do people approach their work given the choice of styles.

Do they identify with a single paradigm and work mostly within it, even when others are available? How does that work out in larger projects, now that it's possible for people to use the same language but an entirely different programming paradigm? Do you need mastery of each paradigm before you can come to terms with choosing when to use each one?

Are there really advantages to having more than one paradigm available within the same language? Are there really advantages in using more than one paradigm within the same project?

you don't notice the speedbumps until it's too late
I think some people would say that language designers put intentional speedbumps into a language when they believe a particular approach is undesirable.
To stretch the analogy, this discussion seems to have been about an unfilled-construction trench ;-)

Dan Shappir - Re: Mixing paradigms (Python)  blueArrow
3/15/2003; 12:11:13 PM (reads: 730, responses: 1)
It actually makes sense for list.sort() not to return the (sorted) list given that it's sorted in place. If anything, the problem is that in Python you can assign the "nothing" returned by this function to a variable (the same is true for JavaScript BTW). In C/C++ such a function would have a void 'return value'. If you tried to assigned that to a variable you would get a compiler error (shades of TD vs. ST again).

Speaking of C/C++ this reminds me of an excerise I got in C during my Uni days. It was something like "write a function that takes a string and upercases it" or somesuch. The prototype to the function I wrote was:

char* strUpperCase(char*)

I actually got a lower grade because the function returned a copy of its argument. Geek that I was I photocopied the page from K&R describing strcpy() and brought it to the grader.

Ehud Lamm - Re: Mixing paradigms (Python)  blueArrow
3/15/2003; 12:12:28 PM (reads: 759, responses: 0)
Let me jsut add that I think I am at least as much to blame as the language.

My expectations come from criss-crossing between paradigms and languages. If I didn't develop functional programming myopia, I would have been less likley to overlook the documentation which clearly states that l.sort() only modifies l.

Ehud Lamm - Re: Mixing paradigms (Python)  blueArrow
3/15/2003; 12:17:14 PM (reads: 759, responses: 0)
I still believe that the semantic and syntactic distinction between functions and procedures, as found in Pascal and Ada for example, is A Good Thing.

Michael Vanier - Re: Mixing paradigms (Python)  blueArrow
3/15/2003; 4:04:16 PM (reads: 714, responses: 1)
I actually think that the functional paradigm has a lot to offer even hard-core OO languages like java. I've been doing a fair bit of java coding lately, and I've found that there is a temptation to have all methods manipulate object state, even when there isn't a real need to (e.g. the state is just some trivial data that can easily be passed in as an argument). I've started writing more "referentially transparent" methods i.e. methods that don't manipulate state. One advantage is that the code becomes easier to understand since the inputs and outputs are obvious. Another is that you can reuse the code in completely different contexts using (*gasp*) cut-and-paste, or in static methods if you're more righteous.

Ehud Lamm - Re: Mixing paradigms (Python)  blueArrow
3/16/2003; 1:49:45 AM (reads: 745, responses: 0)
Sure. Since this is the general view I wanted to highlight the potential pitfalls, that's all.

Dan Shappir - Re: Mixing paradigms (Python)  blueArrow
3/16/2003; 11:40:25 AM (reads: 713, responses: 1)
I still believe that the semantic and syntactic distinction between functions and procedures, as found in Pascal and Ada for example, is A Good Thing.

This is an interesting comment coming from a self proclaimed FP aficionado. While I generally believe that giving the compiler (and other programmers reading the code) more hints as to how it's to be used is a Good Thing, I don't see the advantages in this case. Indeed, I see several benefits in not creating this distinction:

  1. Functions can return one value or several values (via touples or objects). Returning zero values seems to be a special case of this.
  2. During development I find that functions are changed to procedures and back to reflect design changes. Many programmers I've worked with often see this change as technical (e.g. a subroutine returning two values in a PL that doesn't support touples - sometimes it's a function that also has an out parameter and sometime its a procedure with two out parameters).
  3. Exceptions can be viewed, to an extent, as alternate return types and values. In this context a procedure that can throw and exception is also a function.
  4. Considering void a type, and defining procedures as functions returning this type can have some advantages, at least for C++. I can think of two situations where this is so, see if you can guess. Here are clues:
    A. One appears in every C++ and C programs.
    B. The other is the only situation in which C++ functions with void return type can return a 'value'.
  5. Both semantically and syntactically functions and procedures are very similar. And the implementation by the compiler is almost identical

As I mentioned in my previous post, the problem as I see it is that in Python a function returning void type can participate in an expression, and that a 'value' of this type can be assigned to a variable.

The main advantage I see to the Pascal and Ada way, where procedure calls to constitute an expression is that it allows adding goal-oriented extensions to the PL. I seem to recall an LtU posting about such extensions that were added to Modula-2.

Ehud Lamm - Re: Mixing paradigms (Python)  blueArrow
3/16/2003; 11:55:44 PM (reads: 720, responses: 0)
I am not so much an FP aficionado, as someone who appreciates the useful notions of the different programming paradigms.

Since imperative programming is something we do, I prefer to be aware of side effects and their impact on my software.

Functions should have side effects only in exceptional situations. Procedures, by definition, have side effects.

This distinction helps readability, and helps reason about code. It also forces the programmer -- using an imperative language -- to think about side effects. That's why I find the distinction helpful.

Come to think of it, this is in some respects similar to the use of monad classes to mark code with side effects in Haskell (the difference is that in Haskell the type system is responsible for the distinction).

Frank Atanassow - Re: Mixing paradigms (Python)  blueArrow
3/24/2003; 3:39:46 AM (reads: 638, responses: 1)
Come to think of it, this is in some respects similar to the use of monad classes to mark code with side effects in Haskell (the difference is that in Haskell the type system is responsible for the distinction).

Yes, I was thinking when I read your post that in Haskell functions and procedures are indeed distinguished---by type. Your original erroneous Python program would also have been caught by the type checker, of course, even in ML, which is impure.

However, I suspect that simply distinguishing between functions and procedures syntactically does indeed cause problems. Once you commit to distinguishing between pure-functional and impure code, you really need something like a type system to propagate that information along. For example, the language should be able to determine that composing a procedure with a function yields a procedure, etc.

Ehud Lamm - Re: Mixing paradigms (Python)  blueArrow
3/24/2003; 4:18:03 AM (reads: 663, responses: 0)
You are, of course, quite right. However, it is a question of tradeoffs. In Pascal/Ada the programmer can write functions that have side effects (e.g., IO is allowed). The language doesn't enforce purity, but the syntactic distinction is there to help programmers.

In a sense this is a nice example of the tradeoffs that can happen in language design. The language designers didn't want to enforce purity (I suspect they would still think monads and high order type systems introduce complexity they'd rather live without even today, and one must keep in mind the languages were originally designed quite awhile ago). However, they decided that with little overhead (i.e., adding a keyword, and some simple static semantic checks) they can encourage programmers to consider this issue, and documents it in their code.

Ehud Lamm - Re: Mixing paradigms (Python)  blueArrow
9/12/2003; 6:50:05 AM (reads: 367, responses: 0)
Some interesting techniques for walking around the problem that originated this thread are discussed here. Elegant or ugly? You decide.