Overloading - Syntactic Heroin?

Ehud, theoretically on vacation, emailed me a link to an article from the June ACM Queue, Syntactic Heroin:

User-defined overloading is a syntactic drug that has seduced all too many language designers, programmers, and project managers.

It's focused on overloading in C++, and in particular the problems that can arise with implicit conversions, both built-in and programmer-defined. The author concludes that this is all a very bad idea, although to my disappointment, he doesn't explore more disciplined approaches to "overloading" in languages that are less conversion-happy, such as parameterized higher-order modules in functional languages, or Haskell's typeclasses.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Huh?

How is a lack of proper typing logic a fault of "overloading"?

I think there are real problems with C++ that need to be fixed.

Non-problems on many real-world projects

I thought they stopped publishing "X considered harmful" flames. :-) But I might as well take the bait.

First, many languages handle overloading gracefully. In particular, I'm quite fond of Dylan-style generic functions. ML-style polymorphism is just fine, too.

Second, even in C++, this kind of thing isn't a problem on well-managed projects. There's plenty of things about C++ which annoy me, but 'operator+(Matrix,Matrix)' is not one of them. It's simple: All overloads must obey a common contract. And don't define automatic conversion operators without extensive peer review.

Third, his argument is based on the fact that overloading "means that a function name or an operator has two or more distinct meanings." Isn't this also an argument against overriding member functions? The whole point of polymorphic abstraction is to group operations obeying a single contract under a single interface.

Theoritically, the problem yo

Theoritically, the problem you say exists. But in reality, noone will abuse C++ operators to do a different thing than the one that is supposed to do.

Furthermore, C++ has true functions: procedures that are suffixed with 'const' and have all their by-reference arguments as 'const'.

Of course, all this is at the discretion of the programmer. Stroustrup's moto is 'don't make the programmer pay for what he does not want'. This means some discipline is in order.

Err...

Theoritically, the problem you say exists. But in reality, noone will abuse C++ operators to do a different thing than the one that is supposed to do.

...such as overloading the shift-left operator to do file I/O, you mean?

Bit shift and file I/O

The overloading of to mean both bit-shifts and file I/O at first, strikes one as absurd. But given that it's unlike that one would ever want to bitshift an iostream, or conversely extract data from an unsigned long--'s use as homonyms doesn't bother me in the slightest.

Plus, it's such an established C++ idiom that it scarcely makes sense to complain about it anymore...

Note that nobody has reused t

Note that nobody has reused this idiom in other language, showing that it was probably not a very good idea.
But a part from this, I agree that it isn't a problem.

Ruby has reused that idiom

See http://www.rubycentral.com/book/tut_io.html. Not that that makes it a good idea...

Whoops

You're right, I've forgot this.
I'm not a ruby export but in the few code I've looked this is not used: 'puts' is used instead.

Especially since you can write: puts "hello #{my_var}" which is so much nicer to read than to have to separate variables and string as in

[

Any ruby developer to confirm/reject?

people: readers vs writers

I think people talking about programming language features often sort themselves into two piles - readers and writers. Readers read code, and want features that help making reading easy. Writers write code, and want features that help make writing easy. (Of course we all do both reading and writing, but an individual's emphasis leans more one way or the other, wrt PL features.)

The problem with overloading (speaking as a reader) is that it makes my life as a reader, harder. With overloaded operators, I have to keep more stuff "in core" as I am reading, to interpret the context for the current operator correctly.

From an SE perspective, we read code a lot more often than we write it, so it would make sense to optimize or choose PL features to make the common case (reading) easier.

The argument of "just use discipline" is a weak one, in practice. My co-writers have to use discipline too, as did the writers no longer working for my organization, whose code I now maintain. The whole point of PLT evolution (in my view) is that PLs should enable me to use my limited brain space on high-level domain-oriented concerns, not with grokking how some (@#$%!) earlier bozo overloaded '+' to mean something that is neither associative, nor commutative, nor addition-like.

Write once, Read many

Yes, all software should be developed for maintainability. But frankly, I find it much more difficult to read this:

totalBalance = balance.add(interest).multiply(rate).divide(time);

Than this:

totalBalance = (balance + interest) * rate / time;

How about you?

Mixing metaphors

more typical is

   divide(multiply(add(balance, interest), rate), time)
or
   (divide (multiply (add balance interest) rate) time)
or
   (balance `add` interest) `multiply` rate `divide` time

Actually...

I was talking about Java. I really don't see the point in operator overloading in prefix languages, since it's not going to look natural no matter how you spell the operator. That's why I'm quite unimpressed when someone says: "But Lisp has had operator overloading for years!" I just wonder: "Yeah, but why bother?"
Operator overloading only makes sense when you can use the natural fixity.

fixity is not fixed

http://www.cliki.net/infix

The article is only about com

The article is only about compile-time overloading (based on type declarations and implicit conversions), not polymorphism. The real complaint seems to be about the hidden complexity of implicit conversions in C++, not about overloading itself.

Yeah, implicit conversions are pretty hosed

On our C++ projects, we generally only use implicit conversions from 'char*' to std::string. But this is also old news--I think Effective C++ argued against conversion operators back in 1995 or so.

Then again, I think that Haskell's fromInteger is broken for much the same reasons. It's hard to build good libraries when everything is auto-converting at random.

The lack of a common contract is the problem

I have with operator overloading. For every time I've seen overloading used correctly, I've seen it used incorrectly at least three times. And that's not counting the more egregious violation of contract that's encoded in the standard: I/O streams. What's the contract of the operators? The base language defines them as bit shift operators, and then the library overloads them as I/O operators. So what's the common contract? And if the standard violates the contract, why can't I?

The common contract is that o

The common contract is that operators are used either for bit-shifting or for insert-extract. If you don't like it, change it, but nobody will want to use your library.

I see lots of complaints about the freedom C++ gives to the programmer. What wrong with that? can't you be disciplined enough so you need the language to restrict you?

Speaking of overloaded operators doing different things...

The one that's always annoyed me is the use of + for both numeric addition and string concatenation--especially when combined with automatic conversions. And this extends well beyond C++; it's a well-known trait of Javascript that "2" * "3" is 6 but "2" + "3" is "23".

Ugh.

And on a related gripe; I hate the term "operator overloading" anyway. If I define exactly one version of Foo::operator+ for some class Foo; just ''what'' am I overloading? Without my definition (and assuming no strange type conversions come into play), "aFoo + " is illegal and won't compile. Just because + happens to also be defined on ints and doubles and such, does that make it overloading? Is having the same member function name in two unrelated classes overloading?

rant rant rant...

How about...

Just because + happens to also be defined on ints and doubles and such, does that make it overloading? Is having the same member function name in two unrelated classes overloading?

Matrix operator+(Matrix const& lhs, Matrix const& rhs);
Tensor operator+(Tensor const& lhs, Tensor const& rhs);

template
T sum(T values[], unsigned n)
{
T const* const end = values + n;
T total;
for (T const* i = values; i != end; ++i)
{
total = total + *i;
}
return total;
}

Are you saying that you can parse this code without using overload resolution? Operators don't always need to be member functions (except for a certain set).

And on a related gripe; I hat

And on a related gripe; I hate the term "operator overloading" anyway.

I think you place too much importance in very unimportant matters. It's 'operator overloading' because you overload operators for other types beyond the primitives.

By this logic

seat belts and air bags are unnecessary- just be disiciplined enough to never get into an accident!

The reality is that it's not just me I have to worry about. I'm also effected by the discipline, or lack there of, of anyone I might work with. And I'm not super human. I make mistakes too. So thank you, I'll go on wearing my seatbelt, and driving cars with air bags.

seat belts and air bags are u

seat belts and air bags are unnecessary- just be disiciplined enough to never get into an accident!

You equate driving and flying with programming...but does it make sense to do so? In my opinion, it does not. With flying and driving, one is exposed into circumstances you have no control over, like the weather, the other drivers, etc. In programming, the developers control everything.

Actually, no: A programmer is

Actually, no: A programmer is exposed to other programmers, within his own project, and in external libraries, and to unknowable execution environments. Any of these may break one's assumptions, and hence one's code.

I'm calling it

This paper is a worthless troll. Move along folks, move along.

Psychology

I do think that operator overloading can cause a lot of grief for programmers trying to use badly designed APIs. But of course, the same can be said about virtually any other language feature that is used without any consideration of the consequences.

As another poster already pointed out, it's all about contracts. I think the Python approach is interesting: To override the + operator, you have to write a method called __add__. This does not, of course, prevent you from abusing it in any way. But it does make you ask yourself an important question before doing so: Do I really add anything here?

That's one of the things I dislike about Ruby, too. Python encourages you to write good code without forcing you.

I think the Python approach

I think the Python approach is interesting: To override the + operator, you have to write a method called __add__.

How come using 'operator +' is any different???

add vs. +

I think he's saying that the word "add" has a more specific connotation than the symbol "+", so it's less likely to be used for conceptually conflicting operations.

I never associated "+" with a

I never associated "+" with anything but addition...

Never appended a string in Ja

Never appended a string in Java or C++ with '+'?

Used <> associated with.

Used <> associated with.

I've used sheets of paper as funnels before, but I only associate them with writing. I've used a butter knife as a screwdriver, but I only associate them with spreading tasty stuff on toast. I've used my dog to warm my feet, but I only associate him with being my pet. And so on...

-30-

I've never used a butter knif

I've never used a butter knife as a screwdriver, but I do associate dogs with spreading the tasty stuff on bread.

seems rather C++ specific

The limited method overloading available in a language such as Java isn't much of a problem. That shouldn't be lumped in with true operator overloading, which does have readability problems.

Whenever you're reading an unfamiliar code listing, there are always going to be user-defined operations that you have to look up. The question is whether you can syntactically distinguish between user-defined operations and built-in operations, and what can you safely deduce without looking up the user-defined operations. In Java, there is a clear distinction: operators are built-in and method calls are user-defined. The line could be drawn differently, so long as everyone is clear which is which. A language in which nearly every operation is potentially user-defined (such as C++) leaves you with precious little to reason about without looking everything up.

Builtin vs. User Defined?

Seems somewhat arbitrary to me. I like my languages to make as little distinction as possible. The only difference between a builtin and user defined should be (a). which functions are standard across all users and domains; and (b). where can I find the documentation. Beyond that, I don't like having my encapsulated functions stick out like a sore thumb.

Proper operator overloading

The reason operator overloading is critical to the success of programming languages is that it allows the solution to be written in the language of the problem domain. The closer you can get to that target language, the easier the solution will be to understand by domain experts (which makes it easier for domain experts themselves to participate in development, rather than leaving it all to programmers).

They key to using and understanding operator overloading properly is to recognize that it should only occur as the definition of a DSEL. Operators should only be overloaded with a specific problem domain in mind, and it should be clear that the solution at hand is written for and in that domain. So when using, say, Blitz++, it is the reader's responsibility to understand that she is looking at Blitz++ code. Only then can the operators be correctly understood. It is *ad hoc overloading* that is intrinsically evil and which leads to torment and destruction. And it is exactly ad-hoc overloading which most novice programmers engage in when they create unreadable code.

You know you're looking at spirit code when you see #include at the top of the source file. Yes, you need to know the context to parse a C++ file. But that is exactly what makes it a powerful language. There's nothing more annoying than having to call .add() and .multiply() on BigDecimals in Java.

So C++ produces readable code for non-programmers?

Somewhat like Cobol producing readable code for business managers? Perhaps you mean that in small contexts, the code manages to mimic some mathematical notation that an engineer might be more comfortable with.

My experience is that operator overloading can indeed help the cause of producing more digestible code. But I wouldn't say that it necessarily makes the code read like a formal mathematical specification. Indeed, for every time that it makes for a more elegant solution, there's more times that it actually makes the code harder to comprehend - either by the most proficient C++ in the world or the engineers.

Yes, it can

I would say that 99% of the bad rap operator overloading has gotten has come from the painful learning experience of how to use it. I don't mean how to use it syntactically, but rather what the best practices are for using it. It's a screwdriver that can be applied to a lot of uses, and for a long time, programmers were using it as a hammer, a crowbar, a chisel, and a shovel. Other programmers look at them and go: "What are you doing?!?" But now that we have good libraries that use operators effectively, we can see how and when to use them, and how not to abuse them.

I predict that in the future, operator overloading will only become more popular, not less, and that it will eventually spread to compile-time code as well.

domain-specific syntax

Sounds like a good argument for infix macros as in Dylan or the Java Syntactic Extender.

Let's Go All The Way

Why limit syntax customization to infix? In the most general case, operators should be allowed with any positive arity and the operator appearing anywhere amongst its arguments. Of course, having no constraints leads to a parsing nightmare, so the challenge is how to give people some rope, but not so much that you can hang yourself.

OBJ3

OBJ3 had pretty ambitious operator defining capabilities.

CamlP4, anyone?

CamlP4, anyone?

To make this comment more contentful: CamlP4's powerful and relatively elegant dynamically extensible parsers do come with some costs, though — they take a noticeable amount of time to run, it can (or it least it did when I tried) take no small amount of care to avoid having your exention disrupt the core language's productions (I'm not sure if the lack of backtracking makes this worse or better), and it's not the easiest thing to learn how to use.

And I'm still not sure whether or not to be disturbed by the fact that CamlP4 syntax extensions are parsed by other CamlP4 syntax extensions, and that those latter are metacircular.

Bah humbug

Oh come on, calling add and multiply is really not a big deal. It seems to me that "critical to the success of programming languages" is rather stretching it, considering that there are popular languages without operator overloading and we get along okay.

Also, we already have too many domain-specific languages. On my current contract, I have used Java, HTML, CSS, SQL, regexp, xpath, xslt, ant, JSP, perl, and bash. Other projects use C++, JavaScript, PL/SQL, ASP, and python. And it's not a very big company! This is a tower of babel, not programming nirvana. And you're saying we should make up our own languages that nobody outside the company will have heard of? No thank you, we don't need any more languages.

I am all for domain-specific classes and methods, but not for domain-specific grammars. A human language like English is very extensible using new nouns, verbs, and adjectives, but there is a core grammar that doesn't change much. Computer languages should work the same way.

I am all for domain-specific

I am all for domain-specific classes and methods, but not for domain-specific grammars. A human language like English is very extensible using new nouns, verbs, and adjectives, but there is a core grammar that doesn't change much. Computer languages should work the same way.

I don't know...I don't see anything wrong with allowing the programmer to define '+' to mean "add these two elements of this ring." In most languages, '+' works for both floating point values and integer values, even though the underlying operations are different. Why should adding two elements, a and b, of some ring be 'a + b' sometimes and 'sum(a,b)' (or something of that nature) other times? So, just because the language designers didn't want to add matrix addition (or the operation of addition over some other ring) to the language now means I have to abandon the syntax of addition? If we want to create a programing language that abstracts concepts and separates them from the underlying machine, lets be consistent in our abstractions, otherwise we are just making our language less readable.

focusing on usability

Do your customers care about matrices and rings, or even know what they are? Is it really worth making a general-purpose language less readable to cater to such a specialized domain? Consider how difficult mathematics is to read for most people. Mathematics is optimized for communication on a whiteboard between specialists. If the goal is communication, maybe a more application-specific domain makes more sense - for example, an API for a 3D graphics library might better be written in terms of camera and object configuration and positioning rather than bare matrix multiplication.

But actually, I don't think there's any particularly wrong with defining +, -, *, / so long as the language consistently treats them as type-defined operators that you have to look up (just like methods), and doesn't allow types to change the precedence rules. But in C++ you can also override and, or, not, assignment, and member access. That's clearly going too far.

I think you've been sidetrack

I think you've been sidetracked on this issue - domain specific languages can be a very good idea. The key is that operator overloading, on its own, is a laughable way to produce such a domain specific language. Languages which have extensible type systems, various sorts of polymorphism, and the ability to define custom control structures provide much nicer extensibility. In those cases, the domain specific language makes easier to understand, if it is well designed.

Your problem with a tower of babel is that some of the DSLs suck - like XSLT, and none of them are embedded in your main language, with no underlying unity. By contrast, with embedded DSLs, there often is a unity that helps comprehension, and because the DSL code is embedded, it can be easier to see how it functions in the wider context of the system.

The lord says...

Even if you like the sugar high, the cost of programmer-defined overloading is exorbitant. Language designers, compiler writers, developers, and users all suffer.

Hey man, in the end Jesus will redeem all users as well as poor widows and orphans from suffering about operator overloading! But before this happens chaos, fear and darkness will be with all illuminated C- and Assembly- and COBOL programmers that were working in the soft light of god 30 years ago. Building abstractions and creating syntax sugar for reducing boiler-plate is the way the devil seduces innocent souls. So beware of the evil and find your way back to clumsy but honest languages. Amen.

Overloading is free

On dynamicly typed VMs, overloading on objects comes for free since most of the time you are already doing some checks on integers/floats and bignums (for automatic conversions purposes). Checking for objects adds a test in the case previous tests failed, and doesn't impact overall performances.

However, if you want to generate speedy native code, you might end up inlining additions checks and that will increase a little more the code size.

You might design a dynamicly-typed language where + works on only one time (integers or having only floats) but then you're loosing polymorphism, and you'll have to check types at runtime anyway.

On staticly-typed languages, you know the types at compile-time, so there might not be any speed change either.

So it's not about speed, the author here is just focusing on issues with C++ way of overloading and not other languages.