Lambda the Ultimate

Python doesn't support full continutations or even coroutines; instead it supports "generator" functions which create a kind of limited coroutine.

This has changed for version 2.5, see the PEP. Of course, continuations are still more general. Also, the new "send" method is unlikely to please PL aficionados; it really seems like a hack. I expect that the average python programmer won't use it very much. It will probably be used by library authors when they really need coroutines.

cheers,
Jess

...and lambda expressions in Python are restricted to a single expression...

I don't believe this is quite right. I believe python can have plural expressions in lambda, however it can have no expressions that allow for blocks of code (for, while, etc). The restriction is not an arbitrary one. It has to do with the lexical structure of python itself. Due to the block-delimited scope, and the lack of any explicit scope tags, Python cannot represent unambigous lambda expressions. When Guido first went to implement lambda, he ran up against this basic constraint, and was probably ill advised to attempt lambda at all. From having tried it myself, in a custom python parser of my own, I believe there is simply no way whatsoever of doing it without introducing an explicit scope token.

it must be said that in python you can have nested functions, so even if they are not inlined as arguments you may still get some of the advantages.

But there is a small issue which I think is related to parsing: assigning to an external variable will create a new local one instead of updating the name/value binding, so you can access them like this :

>>> def f(n):
...  def g(i):
...   return n+i
...  return g
...
>>> zero=f(0)
>>> zero(1)
1

but you could not write the usual "accumulator":

>>> def f(n):
...  def g(i):
...   n+=i
...   return n
...  return g
...
>>> zero=f(0)
>>> zero(1)
Traceback (most recent call last):
  File "", line 1, in ?
  File "", line 3, in g
UnboundLocalError: local variable 'n' referenced before assignment

In ruby you can't really have nested functions but if you use a lambda in a method it will work as expected.

You can write an accumulator, it just involves a little stickyness due to the way that python syntax works:

>>> def f(n):
...  nn = [n]
...  def g(i):
...   nn[0] += i
...   return nn[0]
...  return g
...
>>> zero = f(0)
>>> zero(1)
1
>>> zero(1)
2

Because of the weird necessity ot package up the variable in a container people often assume python can't do these things. It can, it just does them strangely.

This is not really the same thing, imo, in the sense that while the classical closure example changes the name/value binding this example just accesses a mutable object.
Well, possibly using frame objects that could be done anyway, but that would be cheating :)

Oh, and I even know the trick to do that with a lambda ;)

They're fixing this in python 3.0 (and possibly 2.6):

Python 3.0a2 (r30a2:59382, Jan  4 2008, 13:31:57) 
[GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(n):
...   def g(i):
...     nonlocal n
...     n += i
...     return n
...   return g
... 
>>> zero = f(0)
>>> zero(1)
1
>>> zero(1)
2

When you talk about code semantics, yes, you can have concurrency in Python. But you will never get anything close to true concurrency running on any real machine. This is due to the notorious Global Interpreter Lock (GIL) of Python, which basically prohibits parallel execution - even on multicore. I use Python every day, but I think for real concurrency it is a dead end (you can tell by the fact that the most popular Python module to achieve real parallelism is multi-processing which maps a threading API unto OS processes (sic!)). Unless, of course, something dramatically happens (the Unladen Swallow guys are pushing in that direction). As of today, you can use concurrency constructs to better organize your code, and maybe exploit a little bit of WaitIO parallelism, but that's about it; the general approach in Python is flawed (which is another case of the problem of concurrency as an afterthought, which Joe Armstrong once pointed out). If you want to learn more about the limitations of Python's threads and the GIL, check out this awesome presentation by David Beazley (video, slides). If you want to explore concurrent programming, Erlang and Oz are far more suited. I cannot comment on the Ruby side.

Ruby is Objects all the way down, python isn't.

Which Python object is not an object? Seems like Rubys still don't know what they are talking about when they consider other languages. Does make a bad impression on me. But enough about language communities for the moment.

...an impression as made by Pythonistas that don't read their own docs...

2.3 Built-in Types

The following sections describe the standard types that are built into the interpreter. Historically, Python's built-in types have differed from user-defined types because it was not possible to use the built-in types as the basis for object-oriented inheritance. With the 2.2 release this situation has started to change, although the intended unification of user-defined and built-in types is as yet far from complete.

I apologize for blaming Rubys in this affair.

I guess this sentence hasn't been updated since Python 2.2 ( for 5 yrs? ) which ironically already completed unification ( i.e. "new-style classes" ). Swollen programmer egos with great future plans are almost always ludicrous. When they start to pin down their ToDo lists in beginners tutorials right at the moment where they implement the features to come, it's going to be unhealthy.

Types were still first class objects, they just weren't classes suitable for inheritance. Ruby people like to insist that it is better because the language is 'purely' object-oriented, like java. Don't get me wrong, I don't have a problem with Ruby, but why does 'multi-paradigm' have to be a dirty word?

C:\Python21>python
Python 2.1.3 (#35, Apr  8 2002, 17:47:50) [MSC 32 bit (Intel)] on win32
Type "copyright", "credits" or "license" for more information.
>>> def foo(*a):
...     for b in a:
...             c = b(1)
...             print c
...
>>> type(str)
<type 'builtin_function_or_method'>
>>> type(int)
<type 'builtin_function_or_method'>
>>> type(float)
<type 'builtin_function_or_method'>
>>> foo(str,int,float)
1
1
1.0
>>>

In Java int / char / ... are NOT objects and have to boxed and unboxed into objects to be treated as such. (Especially before you can use them in the container classes.)

Tell me, my Python has got a bit rusty, and this whole unification thing has progress majorly since I last looked at Python closely.. Can you do this? (I find this so beautiful...)

class Integer
  def factorial
    return 1 if self <= 1
    self * (self-1).factorial
  end
end

puts 10.factorial

or nice and consistent, but less useful than the above example,
perhaps inherit from a Builtin...

  class Beads < Integer
    
  end

These were things you couldn't do in earlier versions of Python. It may have changed since then.

As of python 2.4, there still is the difference between builtin types and user-defined. You can't add attributes, nor modify in any way the builtin behaviour of builtin types, unless you explicitely inherit from them, like in your second example. But then, some would argue this is saner -- since someone can't turn an int.add into int.multiply in the middle of a program -- and allows for better optimizations...

I believe this Ruby feature allows for some sweet DSL-handling, but i guess that's not something python folks are interested into, since "there should be one way to do it" and DSLs clearly allow for a lot more...

I like python and i like ruby. But i'm still waiting for Perl 6. :)

You address two issues. One is subclassing, another is open classes.
int is not an open class so you can't add methods to it but create a custum subclass:

class Integer(int):
    def factorial(self):
        if self: 
            return self
        return self*Integer(self-1).factorial()
>>> Integer(8).factorial()
40320

The Integer class is indeed open but it would be a redefinition ( rebinding ) of the name Integer if you tried to add another method with a new class statement later.

>>> def iseven(self): return not bool(self%2)
>>> Integer.iseven = iseven  # define new Integer attribute 'iseven'

If you don't want to refuse Ruby style open class definitions as shown in your Integer example you have to define a custom metaclass ( e.g. a metaclass "partial" which is inspired by the language functionality in C# ) that registers your definitions and fit them together. The syntax
would be like this:

class Integer(int):
    __metaclass__ = partial
    def factorial(self):
        return 1 if self else self*Integer(self-1).factorial()

class Integer(int):
    __metaclass__ = partial
    def iseven(self):
        return not bool(self%2)

That is roughly the current state.

Maybe you can help me with something. I've never seen a good use case for open classes. Take your example. In Python I would write:

def factorial(n):
    if n <= 1:
        return 1
    return n * factorial(n-1)

...which is shorter, and more general, and won't clobber anyone else's implementation of factorial.

You can subclass "int" in Python, but I've yet to see a good use case of that either.

...a sense of right place for it.

Suppose you found the Integer class had a '+' method, but not a '-'. (I'm not saying Ruby's base classes are so deficient, I'm merely "what iffing") You could put in a non-class method as you did for factorial. But it would irritate, it would grate, it would hurt (mine at least) the aesthetic senses. You would strongly desire to put the inverse method with the forward method _on the class_.

At least this way, you can...
* Keep your code clean.
* Create an elegant proof of concept.
* Having done so, create an RCR (Ruby Change Request) where you submit your working implementation.
* And when the change is accepted, odds on it will be very close to what you designed.
* Thus the next version of ruby will be improved, and your code will pretty much "work out the box" with it.

I choose Integer class purely for tutorial reasons. Everyone knows what it is what it does. Where this is far more likely to be useful is on the far more sophisticated standard library classes available in Ruby. eg. Networking / XML / ... classes.

So what if you submit your implementation and Matz decides to change the method order. Now your code is broken when you upgrade Ruby.

So on the one hand, open classes allow for a "pleasing symmetry". But at the same time, relying on it will usually reduce the maintainability and long-term reliability of your code.
Also: class objects are global variables. In addition to upgrade risks, you have all of the usual issues with mutating globals.

As a software developer (more engineer than artist) I'm more interested in code that works and will continue to work than in code that looks pretty. When I program in Ruby I avoid adding methods to existing classes like the plague. (In Scala, they've thought things through a bit more so that "open classes" are lexically scoped)

Monkey patching is a bit different: Sometimes that's the only way to fix a bug and get your code working without physically patching someone else's code (which has its own maintainability headaches). Both Python and Ruby allow monkey patching in pretty much all situations where it is necessary. You can't change the way that integer add works (without subclassing) but then you aren't likely to find a bug in Python's implementation of integer add, so why would you want to replace it globally.

So what if you submit your implementation and Matz decides to change the method order. Now your code is broken when you upgrade Ruby.

So on the one hand, open classes allow for a "pleasing symmetry". But at the same time, relying on it will usually reduce the maintainability and long-term reliability of your code.
Also: class objects are global variables. In addition to upgrade risks, you have all of the usual issues with mutating globals.

What if GvR implements your factorial function (that you pulled into the global scope from your math lib with a from math_utils import *), but changes the parameter order?

Ah-ha, you shouldn't be polluting the namespace like that--you should be importing it like import math_utils as mu. But guess what? It's considered best practice in ruby to put the class re-opening code in a separate module and then explicitly require that module--you always know when you get the modified version of the class. You could go a step further (if you've changed a method, which is unusual, normally you're just extending the class) and add a class variable which flags that the class has been modified.

If that then opens up the possibility of bugs because somebody has changed the behavior of a method of the base class, inheriting from the base class and changing the behavior of an inherited method (or just adding new methods) opens up the possibility of bugs because you forgot to wrap a built-in type in your inherited class or you called a method that returns a built-in type that you forgot to wrap. E.g:

class Int(int):
  def print_add(self, n):
    y = self + n
    print "%d + %d = %d" % (self, n, y)
    return y

x = 5
x.print_add(5) # oops, x is an int
Int(x).print_add(5).print_add(5) # oops print_add returns an int

Neither the python nor ruby way is idiot-proof, but neither will bite you very often if you follow best practices and know what the functions and methods you call are doing.

One of the best things about having open classes is that a lot of static methods disappear. A lot of the static methods I write in Java or C# (before extensions), for example, are utilities that operate on a parameter of built-in type.

To implement the same function in an open system I would just add that method to (what used to be) the parameter's class. For example, let's say you want to repeat a string. In Java I might declare a method Repeat in a new StringUtils class. Then to call it I have to say StringUtils.repeat(someString, 4).

In a language like Ruby or Smalltalk you would just add the repeat(n) method to the String class. In Ruby at least you can put those class extensions in a module and mix it in at your leisure.

The other benefit is the ability to remove/nullify dangerous methods in an instance of the runtime and let users run code in a sandbox.

P.S. Ruby already has String.*

I think the lesson to take from the awkwardness of class extensions in closed sysetms is not that open classes are good, but that functions and namespaces are more natural, since the invocation of calls bundled with the abstraction, and invocation of calls in an extension look the same.

Or you could take C#'s approach and create so-called "extension methods", but from what I understand extension methods have a somewhat complicated resolution strategy.

a nutjob

Please refrain from this sort of language on LtU.

Hi Doug, thanks for your friendly words. See you on RuPy in April next year ;)

Cheers

I've been programming C++ multithreaded applications for many years and I have never had any problems with it. Of course I never attempted, for example, to synchronize access to bitfields. I think that the "gotcha's" mentioned in literature are abnormally overemphasized.

There are well-established techniques to code multithreaded applications using hardware threads. I am not saying it is necessary (i.e. software threads may be Ok in the majority of cases), but it is not that difficult as everybody says.

...if you have a good architecture and/or don't care about the odd sporadic unreproducable bug.
Both approaches work and are used extensively.

Failure to be anal about the Good Archiecture guarantees the presence of races. Too many companies have bad threading architecture and then "test it into the shape of a product". Still has thousands of defects, just not reproducible on the test bench.

Things like failure to use "volatile" correctly, failure to obey the strict aliasing rules, etc also make these bad threading architectures fragile and sensitive to changes in optimization parameters and/or versions of the compiler.

These problems are not 'first class' problems of threading. All the problems you mention are problems of the C/C++ programming language that leaves many things unspecified, rather than the problem of hardware threads. ADA has hardware threads but none of these problems, from what I have seen so far.

While processors have a difference between "register" and "external memory" volatile and the exact meaning of that is "first class".

While processor are pipelined and a good optimizing compiler _has_ to reorder instructions, the exact meaning of a variable at each point in the code will be precise from a single threaded point of view, but fuzzy from a multithreaded point of view.

Partly we have been shielded from these issues by...
* 99% of our code only runs on single CPU. In future more and more cpus will be dual core or hyperthreaded.
* Compilers are starting to push the boundary on pointer optimizations. More an more optimizations assume "strict aliasing rules" apply. Is your code strict aliasing safe? How many of your colleagues even know what that means?
* Inlining and partial evaluation becomes more aggressive with each generation of compiler.
* Pipelines become longer with each generation of CPU, and compilers have to reorder deeper to gained the advertized performance benefit.

But multithreaded programming is not affected by the issues you mention. No optimization can mess up the simple 'lock->operate->unlock' protocol needed for most, if not all, of cases.

Exactly what I'm talking about. Excellent Reference! Thanks.

Of course, the compiler needs to know if it should put memory barriers around some instructions.

What I meant was that C, as it is implemented right now with specific optimizations for threads support, is just as good for threaded applications as ruby or python is and that you do not need a VM to do threading properly; threading can be hardware-driven.

Optimizations that current compilers offer can not mess up threading, even if it is hardware based.

Your last statement doesn't hold, there are existing compilers that include optimisations that will mess threading up.

And while the techniques for shared state concurrency are known, they're still error-prone enough in practice that I've frequently seen the advice "don't unless there's no other way, and there probably is". In terms of Joe Moron up the corridor's ability to make your code behave badly, it's far worse than pointer arithmetic.

Achilleas wrote:

There are well-established techniques to code multithreaded applications using hardware threads. [...] it is not that difficult as everybody says.

Could you list what you had in mind, or at least point to a website? I've never run across such a definitive list of "well-established techniques". I know there's simple advice, like reducing the scope of code that is threaded and using objects that are immutable, but the hard part is enforcing these techniques. The compiler doesn't tell you when you made a mistake; instead you get nasty bugs.

Philippa wrote:

In terms of Joe Moron up the corridor's ability to make your code behave badly, it's far worse than pointer arithmetic.

I'd just like to add that it doesn't take a "moron" to screw up threading.

I figure we're all Joe Moron for an hour or two sooner or later.

1) use RAII.
2) lock operations, not data.
3) don't lock immutable objects.
4) share resources between threads, only if needed; otherwise, use synchronized queues.

It is not very difficult to enforce the above. The most important is #1, which makes synchronization very simple and effective, and the synchronization primitive will be unlocked in any case.

I fail to see how you're going to enforce this in most languages in a case where someone's just plain forgotten to include the locks.

And if someone does a 'memset(0, 0, 0xFFFFFFFF)' the memory of the process will be wiped out.

The argument here is that a lowlevel programming language allows efficient problem-free multithreading based on hardware, and there is no need to simulate threads using software.

And not everyone agrees with you about the problem-free part. There's a reason we're not all writing code using macro assemblers any more.

...is patently ludicrous, and as I've pointed out before, one of the reasons you tend not to get responses from people is your obstinate refusal to learn a single solitary thing from anything that's said to you. You continue to engage in proof by repeated assertion, show no interest in actually learning anything about programming language issues, and simply browbeat everyone who's gullible enough to attempt to engage you.

Regretfully, at this point, I'm afraid that I have to suggest that you be the second person to be banned from LtU for trolling.

Achilleas, a problem with this discussion is that we don't have sufficient information to evaluate what you're saying. Your sketch of an approach to concurrency raises more questions than it answers — Jeff and Neil have raised some of them, but we don't have detailed answers.

LtU depends heavily on links to papers or articles. Your argument would be much easier to evaluate if we had a link to a writeup of the approach you're thinking of. From what you've said so far, it sounds as though you're willing to accept certain tradeoffs in exchange for a simple programming model. As an example of a tradeoff, it sounds as though you're willing to accept reduced concurrency on multi-CPU machines, or perhaps you aren't working with multi-CPU machines. That's fine in some situations, but not every programming problem is similar to the ones you work on.

For example, the threading issues in a typical GUI application don't compare to the kinds of issues that come up in a heavily-loaded multithreaded server application. In the former, unnecessarily blocking due to overzealous locking is virtually a non-issue; in the latter, it can mean the difference between success and failure of an application. When discussing programming languages, we can't just restrict ourselves to one kind of application, or one approach to writing code — we have to acknowledge that there's a bigger picture, even if it may not be of interest to us personally.

One huge benefit of PL and CS theory is that it allows us to understand issues that we might never have personally encountered. Along those lines, I would highly recommend the treatment of concurrent programming in CTM, if you haven't already read it. That ought to help in understanding where some of the other respondents in this thread are coming from.

Also, please bear in mind that many, perhaps most LtU readers have an interest in ways to improve the state of the art in programming languages; indeed, that's one of the purposes of LtU. Arguing for the status quo in languages like C and C++ is not usually very interesting. It might be, if there really were a case for saying that there's absolutely no benefit to competing approaches. However, both the practical evidence and the theory is against that being the case.

I agree with Anton. Achilleas, it seems like you are either uninterested in programming languages (as languages), and thus uniterested in the expressiveness they have to offer, or that you don't appreciate what they can do to help regarding concurrency.

Both these issues were discussed here (with plenty of references) many times, including in a paper posted yesterday to the homapage. I suggest you look at these discussions, study the references, and comment on specific things you find objectionable.

Note that from the perspective of LtU the expressiveness of the language constructs is the main thing, implementing them as efficiently as possible comes second: efficient implementation should be possible, but might still not exist in the marketplace.

Thus, it is more fruitful to discuss the advantages (or lack thereof) of the language constructs, and move the discussions about the speed of hardware threads elsewhere (note that seening these as mutually exclusive is certainly not the only option).

• How do you distinguish code that is meant for concurrency from non-concurrent code?
• Why are operations locked and not data?
• What operations should be locked?
• How do you know what objects are immutable?
• How do you prevent deadlocks?
• How likely are you to find bugs in testing?
• Are the bugs reproducible?
• How easy is it to change the code and not violate previous assumptions?

How do you distinguish code that is meant for concurrency from non-concurrent code?

Do you mean from a compiler's perspective? because from the programmer's perspective concurrency is in the design. In case you are talking about the compiler, there is no compiler than can automatically identify which resources should be locked and which should not be, and there isn't going to be one in the near future.

Why are operations locked and not data?

Because locking an operation prevents from making difficult-to-find problems. For example, if I have a synchronized double-linked list, I have to lock the operations 'insert' and 'remove' and not the 'first ptr', 'next ptr' members etc.

What operations should be locked?

The operations accessed from more than one thread.

How do you know what objects are immutable?

Immutable objects are 'const' (for those languages that have 'const', that is :-) ).

How do you prevent deadlocks?

By locking multiple resources with the multi-lock function the host provides.

How likely are you to find bugs in testing?

As likely as in single-threaded apps.

How easy is it to change the code and not violate previous assumptions?

As easy as your design makes it to be.

Because locking an operation prevents from making difficult-to-find problems. For example, if I have a synchronized double-linked list, I have to lock the operations 'insert' and 'remove' and not the 'first ptr', 'next ptr' members etc.

If you use separate locks for operations that alter the same data structure, then you have not got a correct solution. Concurrent access to shared mutable state requires synchronisation not just between multiple threads performing the same operation, but also between multiple threads performing different operations on the same data structure. If 'insert' and 'delete' have separate locks, then how do you ensure that a thread attempting an insert won't interfere with another thread attempting a delete at the same time?

If, on the other hand, you mean that you encapsulate all lock handling within some set of operations rather than allowing anybody access to the data structure, then this of course seems like a sane thing to do. However, it's not that simple as concurrent operations are usually not composeable: e.g. if you want to do something like:

if (!container.contains(key)) {
    container.insert(key, value);
}

then you run into problems as 'contains' will release its lock before 'insert' acquires its lock, leading to interleaving. So you need to either provide external access to the locks:

container.lock();
if (!container.contains(key)) {
    container.insert(key, value);
}
container.release();

Or provide some form of transactions or other way of performing compound updates. This stuff is really hard to get right. I'm currently marking undergraduate concurrency coursework submissions (in Java, which is at least easier and more consistent that C/pthreads), as I have done for the past three years. In that time, I've only seen a handful of absolutely correct solutions. This leads me to believe that the current mainstream approach to shared-state concurrency is just too difficult.

If you use separate locks for operations that alter the same data structure, then you have not got a correct solution.

Indeed. I think it is obvious, so I did not mention it.

Or provide some form of transactions or other way of performing compound updates. This stuff is really hard to get right.

It is not hard at all. Just use RAII. It has worked beautifully for me and I never had a synchronization problem at all.

EDIT:

The example you mention is a perfect one of a operation composition and it really proves my point. The composition of operations 'contains' and 'insert' are combined to form another operation, and it is this new operation is locked.

no matter what the ASCII looks like, no matter if you use POJava, or Erlang Actors, or your combined operations, or even STM, the point is that some human has to identify what the critical sections are, and also how to merge them.

seriously, if the developer knew the answer to that in every case, then the rest is just syntax.

now, i'm not saying that syntax doesn't matter, and i'm not saying that there isn't a spectrum of betterness, but i don't believe (from what i hope i understand, apologies if i'm still just confused!) that the locking operations you talk about really are going to be the magic pixie dust i really want. :-)

I agree with you that is what needs to be done....

But I have on my desk 200000 lines of real world C and a nifty Ruby script that follows call graphs from thread starts to static resources that will show you clearly how uncommon that sense is in practice..

You also forgot to mention things like "volatile" and thread contexts (interrupt service routine vs deferred service routine / timer context vs types of locks like interrupt masks vs disable all interrupts vs scheduler lock vs mutex vs condition variable vs semaphore vs bit semaphore vs counting semaphore vs .... sigh. Believe me, I have ample proof sitting on my desk right now that native threading and the sense of the common man don't go very well together.

But I have on my desk 200000 lines of real world C

First of all, you have 200,000 of C code. I have 3 C++ apps, one is 120,000 lines of code, the other 45,000 lines of code, and the last one 10,000 of code, all heavily multithreaded, but I do not have a single problem of handling them all (and I am the solo maintainer!).

I suspect that the reason is that RAII saves my day.

You also forgot to mention things like "volatile" and thread contexts (interrupt service routine vs deferred service routine / timer context vs types of locks like interrupt masks vs disable all interrupts vs scheduler lock

You can not prove that native threads are evil compared to non-native threads using operating system-level examples. Interrupt service routines, deferred servicing, interrupt masks are all things that belong in the domain of kernels/drivers and a non-native thread implementation has nothing to do with them.

vs mutex vs condition variable vs semaphore vs bit semaphore vs counting semaphore vs .... sigh.

All these things are handled transparently using RAII. The apps I mentioned above contain mutexes, semaphores and events, and I never had any problem handling them with RAII.

It may be boring to repeat it, but RAII is very important.

(and I am the solo maintainer!).

Why are you implying that being a solo developer is harder? It is much, *much* easier.

i think it is also that the ASCII syntax is not over-bearing when you have stack-allocated things because the destructors are called w/out having to have noisy ugly evil bad java style try/catch/finally. i.e. even IDisposable is an ugly hack to my mind. maybe D's scope-exit is the most reasonable way out from t/c/f?

Philippa Cowderoy: In terms of Joe Moron up the corridor's ability to make your code behave badly, it's far worse than pointer arithmetic.

I often work with Joe's cousin, Moe Optimistic, who doesn't think the problem will come up, but with pretty much the same results as code written by Joe. Moe gets it, kinda, but thinks the issue of correct multi thread protection is academic, and not a real world problem.

Then I have to prove my crashes were caused by them updating non-locked shared state, which makes me look unproductive (because maybe those crashes are just my mistake) up until I show everyone's recent unstable system experiences are directly attributable to Moe's refusal to heed my forewarning.

I hate watching glibc abort in malloc() when memory is corrupt due to memory management under completely unprotected races, resulting in updates to deleted memory and freed but reallocated memory. It's too late to pin blame on the code which made the error.

Many folks believe its fine to write code wrong at first, and slowly run the problems to ground later, through incremental refinement and defect removal. But, making your code correct under threading is not something easy to do when you reach the fit and finish stage of development, with looming deadlines.

The well-known techniques for writing correct multi threaded code only have nice results when everyone actually uses these techniques with some diligence. :-) Joe and Moe make life hard for the rest of us. I can't figure out how to put the fear of god into Moe; but maybe the lesson will be learned.

Before I reach the head-shaking stage, I usually ask Moe the following series of questions, after I see he has some refcounted objects shared widely in the code, which are casually modified by methods without benefit of any kind of mutual exclusion. I already know we have lots of threads.

Q. Are these objects used by more than one thread? A. I don't think so. Q. We have lots of threads -- are these objects used by this thread here and that thread there? A. Yes, they are. Q. Why don't you use a lock for this state you change? A. That state won't change when more than one thread is involved. Q. How do you know that? A. It won't be a problem.

Q. Have you thought about making the objects readonly once they become shared? A. That's too complex. Q. No, it would be easy to have these objects copy-on-write once you tell the object to 'freeze' when you're done building them and you actually start to share. Want to do that? A. Too much architecture. I just know I won't update those objects when more than one thread is involved. Q. And you don't want me to add something to check, just so we'll know there's a problem if you're wrong? A. No, speed is very important.

Q. Objects used by multiple threads should either never change, or have some kind of mutex. This machine you're running on has multiple CPUs, and this operation happens very many times a second. It's going to fail if you read and write concurrently without a lock. How do you know this update here won't affect an instance that's been handed to another thread? It's refcounted and you're casual about passing around references. Are you sure? A. Yes, I just know it will be okay.

As near as I can tell, folks want to apply the YAGTNI (you are going to need it) principle to multithread safety, which seems like lousy procedure to me. They're afraid you'll lock unnecessarily and lose performance, and would rather risk instability until a need to lock is later proved. If this increases the cost of development for everyone at all times up until the proof is found, then so be it.

...believes we live in the best of all possible worlds.

The pessimist fears that he is right.

I'm depressed now.

I had some vague optimistic hope that Joe & Moe were purely local and perhaps there was another place where I wouldn't have to endlessly explain and re-explain these issues.

You have destroyed that thin ray of hope.

you are all hired, trust me!

;-)

Unfortunately, what's needed is the ability to say "convince us it works - your word isn't good enough". Which tends not to work so well on a social or political level, it upsets people and leaves room for stupid games ("I don't like you so I'll make you do twice as much work as everyone else proving every last detail!"). .

I guess this probably has something to do with why I'm a lot more disciplined about side-effects these days. Of course, I have the luxury that I'm pretty much always working on single-woman projects - despite which Joe still manages to sneak some code in once in a while. Thankfully at least Moe's being kept out.

I frequenty use mutexes and critical regions in my apps. The code is very simple:

Lock<CriticalSection> lock(cr1);
...

or

Lock<Mutex> lock(mutex1);
...

I do not see how there is a problem with that, and what optimizations can break the synchronization. I have never had a nasty problem with C++ from 1998 that I have been doing this.

If a compiler breaks the above due to optimizations, then it is the compiler that should be blamed and not hardware threads. And certainly doing software threading (ala ruby or python) does not buy you anything.

Any sufficiently aggressive non-lock-aware dead code elimination could decide that a lock isn't actually doing useful work. Any rescheduling that isn't lock-aware could move things out of the resulting critical section. If we accept that locks can be implemented by libraries, there needs to be a language-level mechanism for indicating that no, the sequencing really does matter and so do the side-effects even if the end result is the same state you started in.

Post-'98 C++ is not somewhere you'll encounter this. But someone writing a compiler will sooner or later trip over it or else consciously avoid the problem from the outset. Not everybody's working from your viewpoint.

My argument here is that you do not need to simulate threading with a VM and a software model, because hardware threading works just as fine. If the compiler needs to be aware, so be it.

It goes without saying that a sufficient lowlevel programming language like C/C++ (which hits the bare metal) must provide ways to the programmer to say "here do not do this optimization" or "here do this optimization in this specific way". That's what #pragmas are for.

If these changes are hidden from from the programmer inside a library, then the programmer need not be aware of them. Still, no problem for me.

The issue is simply that, if the language specification itself doesn't take hardware threading into account (the C and C++ specifications presently don't), any facilities you may attempt to use to ensure correct hardware threading aren't likely to be portable, are unlikely to have carefully-defined semantics, and may therefore be incomplete.

Also, as a practical matter, if you're reduced to using #pragmas to disable specific optimizations, it's very difficult to be exhaustive in explicitly disabling all the unsafe optimizations (especially since they and their safety may change across compiler versions). You also can't necessarily control the #pragmas with which third-party libraries were compiled, and sometimes that can matter.

I don't disagree about the usefulness of RAII for defining critical sections, and the importance of having good libraries for this stuff, but in the case of C and C++, clarification of the language specifications is required for such libraries to be robust. Broadly speaking, you want to do the right things, but in the absence of well-defined hardware threading in the language specification I think your unquestioning faith in your compiler vendor is misplaced.

How do you find the concurrency semantics (and indeed, the sequentiality semantics, if any) of ToonTalk (as compared to say of Ruby)? Ignore "sensors"; consider only a subset of ToonTalk where birds and trucks are the only communication. I ask in this context because it seems that in the comparison of C++ to Ruby concurrency, issues of safety are on the table, and message semantics seems to me very safe; I can't think how even Moe could mess it up.

Message-passing is desirable in that simple things are safe by default, but for more complex things it's certainly possible to create all the same bugs (deadlocks, races, etc.) which you can with locks and shared state (they are duals).

Ruby is painful in that it doesn't have well-defined concurrency semantics, although I am one of the people working on changing that. Presently one must rely on the semantics of the platform underlying the Ruby implementation (Java for JRuby, for example).

These statements are largely content-free:

• Rubies yield is just so cute.
• Rubies blocks make decoupling layers so easy and obvious, it's hard to spot that you have done it.
• Articles on double dispatch in C++/Java magazines make me giggle. It's all just so much harder than it needs to be. Ruby is the counter example.
• Mixin's are really really nice. Duck typing is really really a Good Idea.

What is it about Ruby that makes people talk this way? (Though I note you missed out “simple and intuitive”)

I'm not saying you couldn't formulate a decent argument using the intent behind these statements, but as they are a decent argument they do not make.

Ruby's culture tends to be very "touchy-feely" compared to most languages. I actually like that for the most part, but in cases like this it can admittedly be counterproductive.

I'm not sure that most of the statements here are quite correct either: I would not dare mention double/multiple dispatch as a selling point for Ruby (it sucks in that respect).

These statements are largely content-free:
....
What is it about Ruby that makes people talk this way? (Though I note you missed out “simple and intuitive”)

It's not the fault of Ruby-the-language. I've seen that kind of "marketing speak" associated with several languages as they became popular - in fact, to go with Ehud's recent comment, there's probably a reverse correlation between a language's popularity and the substantiative content in the claims of its adherents.

So one might instead ask what it is about "the herd" that reduces them to saying the equivalent of "my language rocks, your language sucks." For the most part the designers of these popular languages have seemed to be solid, smart people with enough understanding of alternatives to be aware of some of the trade-offs they've made in their design. But as the masses descend and the community grows from 1 to 10,000, the thoughtfulness disappears in the noise. Maybe it's just the standard bell shaped distribution curve in action.

Certainly it should be enough to give any language designer pause in his/her desire to create The Next Big Language.

This is only a syntax taste issue, so isn't so important in a sense as the semantic discussions in this thread, but I want to react about Ruby's yield, which is part of the overly complex way that Ruby treats parameters and arguments in its syntax. I don't like it. In Lisp and Smalltalk, you can pass some kind of block or lambda expression or procedure or function (not getting into the subtle differences among these things semantically) as an ordinary argument to an ordinary parameter and the callee can decide whether to invoke the passed-in block zero times or however many times and/or to communicate it on to someone else who then has the same range of decisions. This is also true in Ruby. But Ruby adds this special syntax for passing one block, outside the ordinary list of arguments. Accordingly there are special syntaxes for the parameter (yield, and the & notation) corresponding to this special argument. So every time I write a method that is going to accept a block as an argument, I have to decide whether it should be passed as an ordinary argument or as the special block argument. And every time I write a call on a method someone else wrote that accepts a block as an argument, I have to know which way to pass it. And if I am going to design a method that wants two block arguments, at least one of them has to be an ordinary argument because there can be at most one special argument, and I have to decide whether one of them should be the special and if so, which one. For example, I could implement Smalltalk's ifTrue:ifFalse: (Self's ifTrue:false:) in Ruby; I suppose the false branch should go in the special argument position.

I don't know where Python comes down in this regard.

Is this really such a problem? If your programming style lends towards passing lambdas as arguments to methods, your method definitions should reflect that. If on the other hand you want your block definitions to be coupled with the function that yields them, you should define your methods using the '&' notation.

In [link], I respond to this.

Python's designers very strongly discourage such Smalltalk-esque "flow control macros"; see the "motivation" section of PEP 343.

Interesting. The PEP (Python Enhancement Proposal) refers to Raymond Chen's rant about flow-control macros. Not getting into responding to that too much, I find it nonetheless hard to avoid noting that COMMENT the snippet of the Bourne shell that Chen presents makes it obvious that just as the Bourne shell implements Algol 68, it is also written in Algol 68 TNEMMOC.

Getting back to the discussion about lambdas, how in Python do you write an expression for the result of taking some collection and applying a function to each element to produce a new collection ("map" in Lisp)?

In Ruby, this is

    a_collection.map { | each_element | each_element + 1 }

for example if the function to be applied is the successor function on integers.

map(funct, a_list) returns a new list that is a_list with funct applied to each element
[funct(x) for x in a_list] does the same thing

Or do you mean other collections than lists?

Or do you mean other collections than lists?

This shall work for all iterables i.e. for all classes that implement __iter__ and next() appropriately.

. . . in Python.

Not to defend Python or anything, but you should take a look athe python department (especially in the "classic" archive). We pointed to many projects that use Python for cool language experimentations.

Well you can consider RubyOnRails as a DSL and it seems to have many positive reports.

For the ruby vs python part, I remember looking at both language and thinking that they are very similar, I'd say that Python is more adequate for beginners and ruby for more advanced people (especially if you know shell|perl).

If you want to write DSL's, Tcl might be another one to look at, as you can really do a lot in that direction, to the point of defining your own control structures.

To answer the original question, I agree with the other poster that compared to the more 'accademic' languages, Python and Ruby are very similar. Lately I like Ruby a bit more, but I'll come clean and admit that there's some 'bandwagon effects' there.

I've written several DSLs in Ruby, all of them embedded. My experience has been that for the user, they work fairly well, but for the implementor they're not such a happy experience. Ruby, unfortunately, has a lot of syntatic and semantic oddities that tend to make doing more sophisticated things difficult. This was a major thing influencing me to switch most of my work to Haskell.

I'm not so familiar with Python, but my impression is that there's not really much to pick from between the two languages: each has minor things it does better than the other, and both suffer heavily from having grown out of a very casual design process.

I was recently examining Python for an internal DSL and because Python lacks overloading the assignment operator makes some things difficult; consider working with Pythons Decimal class. If the user is working with a Decimal object then assigns a float, the decimal object is lost:

d = decimal("1000.001")

d = d + .001 # this works because + is overloaded

d = .001 # d is now not decimal and this behavior cannot change

That's Python semantics obviously, but if I have Decimal like object wrapping arithmetic on numbers I'd like to allow users to work only with my objects when that's clearly what they're doing.

Will you be here all week?

Every month or three I throw up my hands and say, Ruby or (...language of the day...) is all way too complicated, I'm going to use Scheme.

Scheme is so simple, it's so clean.

So I get stuck in, do stuff for a day or two and then throw up my hands and say who cares about syntax whether it's clean or not, it's the semantics that really matters. Schemes semantics are not simple and clean enough for me.

So off I go an use Joy for a day or two.

Then I throw up my hands and say it's expressiveness and power that matter, so I go back to Ruby.

After piling ruby code on ruby code for a day or twenty, I start to say, I wish I could trivially parse this stuff so I could visualise, analyse, instrument, measure, refactor it, ....

At which stage I started working my own language that has the simplicity of syntax of Scheme, the algebra of Joy, and the expressive semantics of Ruby, but a cleaner more analysable semantics than any of them.

So after I day of hard thinking on that I find it is actually a very very hard problem I have set myself and start using Ruby again...

This sounds way way way to familiar.

"Why We Fight"

When programmers fight about programming languages it's the fight they're interested in, not the programming languages.

How do you find Haskell? It seems to me that the semantics're clearer than Scheme's and probably comparable to Joy's, and the syntax is a nice middle ground between Scheme and Ruby.

yeah, wasn't it for monads and we'd have a winner. :P

Am I weird for liking the things monads offer? Parsec springs to mind pretty quickly, but the many variants on the State monad (Reader and Writer being more obvious examples) also come to mind as something that offers more than I could have just by having mutable storage in the language.

For parsing, modularity and stuff... Now if I could only have mutation as well in the language, you'd have a winner...

Or to put it another way, ST and IO aren't good enough?

Once in a while I give some thought to a language where everything's in some kind of monad-like structure and so there's no cost to suddenly needing another monad-supplied feature...

...but all of the recent work on language-level approaches to concurrency, partial/subcontinuations, Tim Sweeney's comments on "The Next Mainstream Programming Language," etc. have certainly gone a long way toward convincing me that a language that did indeed syntactically "look like" it was doing mutation, but under the covers was ST-like, and could have the syntactic sugar "stripped away" to expose more expressive power, would be a wonderful thing. Essentially you'd have an onion-peel language design, where the surface syntax was firmly in the Algol/Pascal/C tradition, but inner syntax, and the underlying semantics, would be rather more strongly ML/Haskell/Epigram/Ontic-like. The more you got stuff like Oleg's Zipper "for free," the better.

But now I'm just reiterating what Tim has already said explicitly here several times.

You get used to it with some practice, but I agree it's less than perfect. That said, I'm not entirely sure what you mean by "they're not easy to put together"? If you mean that writing new monads from scratch isn't easy, I'm inclined to agree in Haskell98 - it's a lot easier with GADTs, you can treat the monad as a language, build an AST type and write an interpreter (and you can get the more conventional style back by fusing the AST constructors and the interpreter, eliminating the AST type in the process and given you H98 code again)

Is there a nice example of this technique (using GADTs+monads) somewhere that I can look at? It sounds interesting, in line with how I sometimes try to use monads anyways, and a good way to learn how to use GADTs.

Also, as a general question -- how good is the Haskell compiler (or rather, GHC) at spotting examples of code written to "interpret" computations in some monad, and emitting code which simply performs this computation fairly directly? (Or is this not a sensible question?) Perhaps this is not really on-topic, in which case apologies...

I talked about it at Anglohaskell last year - here're the slides: http://flippac.org/talks/Fusion.pdf . Never did do a nice version of the parsing example, sadly.

I know GHC can in theory pick it up with inlining and case/ctor, but I don't know how aggressively it does so and recursion can get in the way some. The slides do show you how to translate fairly mechanically into a 'shallow' style where there's no interpretation happening though.

Mutation is basically the same thing as merging streams of messages indeterministically, right? Let's say there's a process consuming a stream of messages. When it processes a message, it determines the state for processing the next message to arrive on the stream, etc. And let's suppose that whoever composes a message can send some logical variable that can be told by the state at the receiving end, and asked by the sender of the message or her delegate. If the stream being processed by the process resulted from merging two or more streams from different sources, the sources can cause state changes and can observe changes caused by the other source, so you have in effect shared mutable state, right?

It took me quite some time to start building them comfortably (though probably less time than it took me to learn to use OO well), but I find now that monads provide significant Good Things in my life. Monadic parsers are a classic example, but there are plenty of other things where a custom monad can significantly simplify the expression of a problem to the point where it's pretty much a DSL. I use one for some fairly complex message generation within a trading system I'm writing, and assemblers for various machine languages are another classic example.

Speaking as someone who switched from Ruby to Haskell, I find it an absolute joy. Going to back to Ruby now feels to me almost as bad as going back to Java.

If you think Scheme's semantics aren't simple enough, I see two broad possibilities: one is that the semantics of the lambda calculus aren't simple enough for you, and the other is that Scheme's semantic extensions to lambda calculus (mainly some data types and mutable variables) aren't simple enough for you.

If it's the former, then you're in trouble, since anything simpler than the lambda calculus tends to be either quite restrictive, or low-level (think SK combinators), or both. Of course, there are some common LC features that many languages avoid, like higher-order functions — you can do that, and pay the price in terms of expressivity/power. But other than that, LC is about as basic and simple as it gets — you can't get much simpler without paying a hefty price.

OTOH, if you think Scheme's extensions to the lambda calculus complicate the semantics too much, that may help figure out what you're really looking for. For example, mutable varibles are a big, practical extension with notoriously messy consequences, notably their effect on the algebraic properties of code. If you want to avoid that, then you need to look at a more purely functional language (although nothing stops you from using a purely functional subset of Scheme, btw). But to live without mutable variables in practical code, you're likely to need to look towards something like monads, linear naming, or linear typing. By comparions to these options, IMO, in practical contexts mutable variables aren't so bad, especially when used with discipline in mostly functional code.

You mentioned the algebra of Joy: don't forget that LC-based languages have well-understood algebra-friendly intermediate representations such as CPS and ANF.

In general, I think that you should first figure out what semantics you're interested in rather than how those semantics happen to be manifested or combined in particular languages. If it turns out that what you want is some combination of existing semantics, you might be able to make that happen in a language like Scheme. But if you need an entirely new semantics, then as you say, you've set yourself a hard problem.

...many a time.

Basically it seems to me mutable does horrible things to the algebra.

The very very good thing about Lambda Calculus from the Algebra point of view is the lazy evaluation of arguments. Lazy Eval + Partial Eval + Constant Folding + inlining == Good Very Things

One of design goals of Joy apparently was to get rid of argument names as being irrelavent complications to the algebra.

Part of my problems in life is my day job is shepherding symbols in big commercial industrial size chunks of code. Thus I'm very uncomfortable with anything I feel won't scale to that size.

One of the things that makes me very uncomfortable about Joy is it is "halting problem difficult" to decide how deeply and what the stack effect of an arbitary chunk of Joy code will be.

A huge benefit (from my perspective) of Lambdalike languages is the stack effect is (relative to Joy) very very simple. Furthermore the argument name binding shields client code from implementation changes.

I also feel at the large code base scale OOP is very important.

I'll admit I haven't managed to grok how immutable & lazy eval fits with OOP.

I'll admit I haven't managed to grok how immutable & lazy eval fits with OOP.

Immutability might be achieved using immutable prototypes. I'm not sure about lazyness either but I guess it is orthogonal to the compositional mechanisms provided by OO. JavaScript would be a good target language to start with. Maybe some cooperation between JavaScript and Haskell developers could be interesting. Web 3.0 is not very far away ;)

Basically it seems to me mutable does horrible things to the algebra.

Yes! Especially the way letrec has assignments in it, and can accumulate some arbitrary amount of state that isn't in lexical variables and therefore has to be carried around in continuations... blaaaaagh.

So if you like pure, lazy, lambdalike languages, why isn't Haskell, or perhaps Clean, on the shortlist of languages you try to switch to every few months?

I think Ruby wins here. You can build a constraint "framework" with continuations, and Ruby relies on functions that use blocks (lambdas) more than python (so it is more functional).

A Ruby Quiz about constraint processing:

http://www.rubyquiz.com/quiz70.html

The problem of constraint programming is three fold:

• Declare the Domains
• Establish the Constraints
• Search for the Solutions

writing "amb" in ruby is some kind of rite of passage (I did it too, and I'm stupid :) but I don't think it qualifies ruby as a better environment for logic/constraint programming, especially if you use Kernel#callcc which is very slow.
OTOH you could write a CSP library or bindings for an existing one (i.e. jcl via jruby or gecode) but that won't be better than in python.

Sadly both languages miss something like Smalltalk's #become, which forces developers to explicitly request the value of a computation and thus the integration of this libraries with the rest of the environment would always be less than perfect.

evil.rb has an implementation of #become if you really want it.

AFAIK evil.rb's implementation is not perfect, for example a generic Object can't become a Fixnum, cause they have different representations at the interpreter level.

not necessarily the langauge itself. On the other hand, the entirety of evil.rb is an artifact of the implementation so I guess you win some, you lose some.

What functional-like constructs are being removed?

The state of discussion here is somewhat anachronistic ( from 2006 ) and doesn't reflect the actual Python 3.0 design decisions. The only noticable change in Python 3.0 a2 is that reduce has been moved from the builtins to the functools module i.e. a tiny refactoring. It was all just a storm in the teacup.

It is common in Ruby (at least, when using the Rails framework for web programming) to utter some macro call in a class definition. I know that in theory Smalltalk could do anything if you monkey with the class browsers, etc., but in practice, Smalltalkers don't tend to use macros of this sort the way the Ruby practitioners do.

One of the uses for such a macro, for example, is to establish accessor methods for state. Smalltalkers in developing a class may invoke some code to generate access methods to save keyboarding, but once generated, the methods are ordinary methods maintained in the usual way. If a Ruby macro generates methods, code maintainers see the macro call, not the generated code.

Macros provided by the Rails framework let you economically express the relationships among entities, validation of input data, and much more.

Smalltalkers ... the methods are ordinary methods maintained in the usual way
Yes.

economically express the relationships among entities, validation of input data, and much more
And that's where Smalltalkers would do similar stuff.

OK, I have started a topic on Ruby vs. Smalltalk. Would you go over there and tell me about the "similar stuff"? How are modern Smalltalkers writing macro calls similar to those Rubyists are writing? Where in ones Smalltalk code base do the calls go, in some class methods, for example? Is there an initialize method on the class side, where the class can talk to itself, and say hey, my instances are related to those of that other class, or validiate these fields, or whatever?

PyPy was discussed here quite a few times. Do search the archives!

Peter, I do think these general questions about Pythons design are better addressed at comp.lang.python if the Python FAQ doesn't suffice to answer them.

Coincidentally, I just came across this.

This is starting to get pretty off topic for LtU... I don't especially object to it at this point, but web framework advocacy can quickly get out of hand. And I think there's already plenty of it elsewhere?

Yikes! My toe nails start curling.