Lambda the Ultimate

Yet another unwitting plea for DrScheme! :)

1. The perfect programming language will not need a (separate) IDE.
2. The perfect IDE will be language neutral (or close to language neutral).
3. IDEs and languages are complementary. But in unanticipated ways.
4. Good programming languages are designed with IDEs in mind.
5. Good IDEs are designed with a (specific) programming language/family of related languages in mind.

I bet on #3. #4 being a good candidate, too.

1. Humans are not able to produce PL artefacts directly, we need some tool anyway. 2. Depends on a definition of neutral. Are plugins allowed? If not - no can do. 3. This one has conditional clause - I vote for it :) 4. Yes, because they are designed with programmer in mind. No conditional, though, so... 5. No, good IDEs usually provide for language-specific extensions.

BTW, I am doing an IDE for my living :) Not very successful at the moment :(

1. Some of these things seem beyond the scope of programming languages, which should mean that programmers of the perfect language could still benefit from tool integration.

2. Maybe within a paradigm, or when used with perfect programming languages

3. In anticipated and unanticipated ways (feeling generous)

4. Hmmmm maybe they are designed with a variety of toolsets in mind. Seems some folk would rather give up their programming language than their text-editor.

5. In part. Analogy with multi-language retargetable compilers?

Maybe it also depends on what kind of software development we are discussing (and for that reason, Mr Raskin's example isn't really to-the-point: wouldn't 20 people writing lots of code benefit from a different kind of tool than one person learning to program?).

Smalltalk itself is simple and elegant. Using it in a practical environment is complex and messy. And, as is usual these days, there is a dearth of documentation. The IDEs for languages such as Smalltalk and Java are conventional GUIs, with their inhumane over-reliance on the mouse. None of this is necessary. It is merely customary.

Back when I used Smalltalk intensively I suffered serious tendon pain and cut-back to 9 hour days. Not enough tricky syntax to slow-things down ;-)

Cincom Smalltalk product manager agrees there was an over-reliance on mouse interaction.

(Of course the real problem with using a mouse is that it isn't integrated into an ergonomically designed keyboard.)

Nevermind that the complexity of, for example, a VB program is in part the result of the ubiquitous GUI libraries that he was partly responsible for creating. ;-)

Ehud's questions on the relationship between IDE and language are interesting, but strike me as a packaging issue. The core issue is what kinds of functionality do we want or need from an IDE, whether it is conjoined with the language per se or not.

For me the essentials are these: 1) smart reference work on the language e.g. pop-ups with info on parameters. 2) some decent facility for organizing all the source code and resources involved in my project

"Pretty printing", debugger, tool integration, automated refactorings etc. are all nice to haves.

Now do I really care if these are "built-in" to the language? Not really, though it's fine if they are. Clearly there are some languages that have more "tool infrastructure" built in to make some of these functions easier.

Eclipse is showing promise as the "new Emacs", a universal IDE, but unfortunately the only language I've found so far with really decent support is Java.

The fifith relates to the second which is the most important. I disagree with the idea of neutrality. What does it even mean in this context? A text editor is langauge neutral. But an IDE implies something more than a text editor. Something more than a point an click GUI. If that all it is it is not an IDE. I think a better word is polyglot. The perfect IDE will be a polyglot.

I chose polyglot on purpose. Its more than merely multilingual.I want an IDE which allows me to switch between langauges as I write and handle the dirty work of weaving them together. The ability to write a program in number of languages without being exposed to the ugly glue.

For example, Java native methods. A perfect IDE would allow me to write a java method tagged as a native method then inside the body write in C++. It would handle the task of breaking apart the body, calling javah, compiling and linking the C++ shared library, etc.

Perl does have a module which allows you to write C code inline and do this for you.

I liked this essay and all of Raskin's work. I agree with most of what he is doing. Also this essay hit right into the heart of my research: the border between user interfaces and programming languages.

and/or a programming language.

I agree, of course. My questions were meant to assist in exploring these issues, that's all.

The perfect programming language will not need a (separate) IDE.

This is vaccuous because there is no such thing as a `perfect' language for interesting notions of `perfect'. But, I would definitely agree that better programming languages requires less external support including IDE's but also debuggers, profilers, parser generators and so on.

The perfect IDE will be language neutral (or close to language neutral).

No. That is a text editor not an IDE.

IDEs and languages are complementary. But in unanticipated ways.

Eh?

Good programming languages are designed with IDEs in mind.

That's pretty funny.

Good IDEs are designed with a (specific) programming language/family of related languages in mind.

Yes. The best IDE for a particular language is one designed specifically for that language. Ideally it would allow you to exploit the metatheory of the language, for example to do refactoring in ways which really respect the language syntax and semantics.

A thing that bugs me, for example, is that to rename a variable that is widely used, I need an IDE, or resort to find-and-replace. To my mind, this is a bug: the system obviously violates the OnceAndOnlyOnce principle. A variable name should exist only once, however, because the underlying representation is text-based, it has to be repeated over and over.

I have recently seen a post in a Java newsgroup about escaping the '\' character in a regular expression. Because '\' is an escape character both in Java source code and in regexps, something along the lines of "\\\\\\\\" had to be used to get one '\' in the regexp. Escaping is another area that shows how bad plain text as a user-medium is.

Also, in plain-text, transformations of source code are very hard, no in fact they are not possible unless we build an Abstract Syntax Tree (AST.) But transformations (like Lisp macros) would be a day-to-day tool for much more programmers, if they could just describe transformations on the AST, for example using simple forward inference. I hope that Gosling's new project is something along these lines.

> Although never will more than a small percentage of users learn programming ...

I am much more optimistic on this one. I believe that in the future an ever growing number of people will program. Of course, grandma will never write a device driver, but she will program to customize and extend her work environment, just like Unix users do today.

Hmmm. This sounds like you want to program directly in ASTs with references. I suspect that for many situations that would be MORE cumbersome than using plain text as an intermediary.

I believe that in the future an ever growing number of people will program.

I think sometimes we underestimate how specialized a skill programming is. I would imagine most of us here come to it naturally, so we probably take that it is "easy" for granted. But I think that expecting "ordinary" people to program is like expecting that in the future everyone will get medical degrees.

Compared to bygone days, we are all doctors now. When you wash your vegetables before eating them you are making use of medical knowledge that the most advanced physicians in ancient Egypt lacked.

Might not the same thing happen with programming?
To a large extent it is happening. Think about programming a VCR, for example...

Here is my rough definition: the skills and knowledge sufficient to make a plausible attempt at solving an arbitrary programming problem.

Compared to bygone days, we are all doctors now.

If you take a definition of doctor parallel to my definition of programming, substituting "arbitrary malady" for "arbitrary problem" and you see this is fallacious.

Think about programming a VCR, for example...

Many people cannot handle that either. ;-) But if you think about it anyway, some programmer had to figure out how to "really" program the VCR so that a naive user could use it in a limited way to achieve a fixed range of functionality.

If we allow that fixed range to be considered programming, then the users of ANY software or electronic device would have to be considered a "programmer".

Alan Kay agrees:

[T]he true personal computing revolution will happen when all people, especially children, become fluent with the deeper powers of expression that only the computer makes possible, and when these powers of expression bring forth a new way to discuss, think, and argue about important ideas.

So do the authors of HTDP:

[D]esigning programs teaches a variety of skills that are important in all kinds of professions: critical reading, analytical thinking, creative synthesis, and attention to detail.

We therefore believe that the study of program design deserves the same central role in general education as mathematics and English. Or, put more succinctly, everyone should learn how to design programs.

There is a big difference between studying math in school and being a mathematician.

There is likewise a big difference between studying some programming in school and being a programmer.

Think how many people just get by in the "fundamental" subjects of language and math.

Yes, there are general problem solving skills that can be learned that relate to programming, but there is a lot of fairly complex knowledge in computational fundamentals that a good professional programmer should have, not even to mention the specialized knowledge of specific languages, platforms and libraries.

Why do people want to devalue what programmers do? Given all the griping I hear from other programmers about how most programmers suck, you'd think that it would be well established that programming is a very challenging profession. ;-)

Comment from OOPSLA 2002: "Next up was John Duimovich, demonstrating some work he's done on building a Smalltalk IDE on Eclipse. It was a very early prototype, but pretty cool. Interspersed with his demo were frequent complaints that the main problem integrating Smalltalk with Eclipse was that Smalltalk really shouldn't be using source files."

A variable name should exist only once, however, because the underlying representation is text-based, it has to be repeated over and over.

After going thru releases, their usage had stablilized to following: each module is a graph, so all intramodule references are by ref, not by name; while all intermodule references are by name. There were several reasons for by-name decision - inheritance, templating, version control...

Very important is the fact that developers communicate among themselves using natural language, which is biased towards names, not refs :)

By-ref also has it merits, as it allows you create anonymous features - remember lambda :)

So, briefly: I believe in names for intermodule (-component, -package, etc.) references, while using anonymous references inside modules.

How about: The perfect programming language needs a shell. on edit: maybe the perfect programming language is such that a shell seems to be the most suitable environment to develop in it?

btw: "When you wash your vegetables..." damn, my lazy style of environment interaction has turned me into Dr. Death.

It's called Unix.

(No, I'm not really serious... But I guess I'm not really joking either...)

Manuel:

I think a problem with today's standards is that they rely solely on plain text (byte sequences) for the transmission of information.

I think there are at least two reasons for this. (I have the feeling I posted something like this once before, but anyway...)

First, most programs are broken. If important data is kept in a non-trivial format which requires a program to access it (rather than a simple text editor), then you are likely to lose data. This is, I think, the biggest reason most configuration files and whatnot on UNIX are stored as text: people, even programmers, don't trust software. This is one of the myriad invisible ways in which faulty software limits progress.

Second, there are many tools for text-editing, but few tools for editing more structured kinds of data. One of the reasons for this is, I believe, in most programming languages there is little code you can share between algorithms which process, say, strings and algorithms which process, say, trees. (This is called genericity.) So you have to start largely from scratch, and there is not much reuse.

Of course, on some level byte sequences need be used, but it's not neccessary that this is the medium the user works with.

One thing I have often thought is that programming languages (or IDE's) might benefit from having a distinct `input syntax' and `output syntax'.

For example, consider operator precedences in a typical language. People often forget them, and it gets much worse when, as in Haskell and ML, you can define your own operators with their own precedences.

Now, when a programmer is writing code, he always has in mind a particular semantics and thus a particular parenthesization of an expression. For example, he always knows if he wants to write x*(y+z) or (x*y)+z. But when a programmer is reading code, he is trying to figure out the semantics, and this depends on knowing the parenthesization. Often the precedences which the writer knows by heart are not the same as the ones the reader knows by heart.

Here is some ways to take advantage of these facts. Require the writer to always fully parenthesize expressions, but display these expressions to the reader using an minimal parenthesization. Or, conversely, allow the writer to use as much or as little parenthesization as he wants, but always fully parenthesize an expression when you display it. Note that the writer and the reader could be the same person at the same time, as when you are working in an IDE: input and display need not be the same. You could have a compromise too; for example, IDE's could support a toggle which displays code fully parenthesized. Indeed, you could even have completely different operator precedences for input and output/syntaxes, or indeed completely different syntaxes, period.

A thing that bugs me, for example, is that to rename a variable that is widely used, I need an IDE, or resort to find-and-replace. To my mind, this is a bug: the system obviously violates the OnceAndOnlyOnce principle. A variable name should exist only once, however, because the underlying representation is text-based, it has to be repeated over and over.

Well, one semantic role of a variable is precisely to enforce the `OnceAndOnlyOnce principle': a variable may be used in many places to stand in for its value. So variables are already actually doing what you want.

On the other hand, you're right: one semantics for a program which supports local binding is actually a kind of graph, more specifically a tree augmented with back edges. (There is lots of info on this on CiteSeer: search for Wadsworth, Huet, Levy and term graph.) Variables are just placeholders for those backedges.

There is a very cool system called HOPS which represents Haskell programs this way. In fact, it also represent types that way and has an edge from each node to its type. In this system it is actually impossible to construct an ill-typed program.

There are some problems with using this idea in practice, though. For medium-size terms you get lots of crossing edges and it's hard to read. Unfortunately optimal graph layout is a very difficult problem computation-wise (NP-complete I think).

The deeper problem here is actually that we are trying to project a discrete structure (a graph) onto a sort of 3D linear space (the display: dimension 3 is the color), and there are too many choices. There is a radical way to address this, if you are willing to change the programming language.

First, you can design a language whose semantics is not just a graph but in fact a planar graph. Languages based on non-commutative linear logic have this property. Second, design a programming language whose semantics is not discrete (some sort of topological structure, say). This is much more challenging and interesting: you can imagine programs whose semantics are actually regions or maybe manifolds in some sort of 3-space, like the sort of thing William Gibson wrote about in Neuromancer. Of course, there are lots more problems too: the regions might not be smooth or connected.

The cool thing is that if, say, you have a program which is a region X in 2-space, which reduces (via evaluation) to another region (the value) in 2-space Y, then that computation would itself be a region in 3-space, bounded on both ends X and Y. Or something like that. I'm not clear on the details, but I know for a fact the idea is sensible. But this is all blue-sky stuff: don't expect to see that, even in research circles, for another fifteen or twenty years.

Another way to eliminate variables, BTW, is to use point-free notation, about which a you can find two recent topics on LtU.

Also, in plain-text, transformations of source code are very hard, no in fact they are not possible unless we build an Abstract Syntax Tree (AST.)

Of course you have to build an AST (and a lot more) if you are going to do anything non-trivial with a program. But that has nothing to do with whether you represent programs as plain text when you store them.

I am much more optimistic on this one. I believe that in the future an ever growing number of people will program.

In the future? It's already happened! Depending on your notion of programming, as Ehud pointed out. But I think it's very true. Just consider the archetypal WIMP (Windows-Icon-Menus-Pointer) interface. It is a kind of programming syntax. HCI people make a big deal about metaphors with the real world, but really there is nothing like a WIMP in the real world: it's a programming language that everybody who uses a (conventional desktop) computer has to learn, and they all do, and quite quickly at that. Now, of course, it's a very simple sort of language, without common programming features, but one can imagine extending it with them. For example, editors like Word and Emacs have mechanisms for `recording' macros, which is a lot like defining a procedure in a PL.

Marc wrote:

Why do people want to devalue what programmers do? Given all the griping I hear from other programmers about how most programmers suck, you'd think that it would be well established that programming is a very challenging profession. ;-)

Marc, it's interesting: I had to read this remark several times to see what you found `devaluing' about programming. I concluded that you must think specialized skills are more valuable than widely applicable ones.

Well, I disagree with that. The value of a skill is inversely proportional to its specificity, and proportional to one's proficiency in it. So the more specialized a skill is, the more important it is that the practitioner can abstract away from the specifics and be able to reapply it in other domains. Among programmers, I frequently encounter people who exhibit an encyclopedic knowledge of some super-specialized domain like POSIX and XML standards, but are stymied when confronted with analagous problems in different domains.

Andris wrote:

By-ref also has it merits, as it allows you create anonymous features - remember lambda :)

Lambda-abstraction and higher-order functions have nothing to do with anonymity; that is just a choice of syntax. C does not have higher-order functions, yet you can pass around pointers to functions: they are effectively anonymous. (You cannot define a function without a name in C, but that's not an essential restriction. For example, think of passing a function defined in another language to a C program via a foreign function interface: such a function has no C name, only a pointer to some code.)

They have uniformly written monolithic spaghetti code. ;-)

An interesting take on this, Frank.

I will give you a bit of context first, and then try to clarify.

In certain circles I often hear "soon programmers will be obsolete; applications will be generated automatically from simple specifications". I hear a bit of this in "everybody will be a programmer" too.

The devaluation is the implication that the skill (at a professional level) is so mechanical or simple that it could be either automated or "learned by everybody".

To perform almost any skill to a professional level requires time and effort spent on learning the subtleties, and programming involves subtleties that I believe need to be handled by specially-trained (even if self-trained) humans for the forseeable future.

Among programmers, I frequently encounter people who exhibit an encyclopedic knowledge of some super-specialized domain like POSIX and XML standards, but are stymied when confronted with analagous problems in different domains.

If you take a look at my range of interests(even just those I have mentioned on LtU), you will notice that I tend to be in favour of a more generalized understanding too. ;-)

However, in terms of strict economic value, the "geek with no life" is often viewed as the better asset by business people, who want to hire for specialized skills. They already have a generalist... themselves. ;-)

I want to link this back to PLs (for Ehud's sake, but also because I think this applies ;-) ).

There is a stream of research in programming theory that looks for a new paradigm of "easy" programming: executable UML, graphical programming, plain English programming, etc. (One could even argue that Python belongs on this list).

The idea is that "you won't need programmers anymore" if that is found, or "anybody will be able to do the programming".

My contention is that this is simply not true. There may be benefits to the new language, but you will still need programmers, i.e. people who have the aptitude and have spent the time to learn the semantics of programming and the skills needed to translate arbitrary real (often fuzzy) problems into precise and meaningful programs.

There may be very restricted domains (such as the VCR) where non-professionals will perform an activity that we CALL programming. But only in the same way we call both adding up numbers and proving new theorems "doing math".

Now that is a task we all do daily. For example, if I have to go to the market, I have to decide whether to ride a bike, drive a car or take a train. I also have to decide which route to take, when to go, etc. This sort of activity planning and organization is not so dissimilar from programming at its heart.

You can argue that the algorithms programmers need to deal with are much more complex, and I would certainly agree (although OTOH I think even a trivial planning task in every day life probably involves millions of details which we never consciously consider). And the need to formalize that algorithm so a computer can understand it is also something that unquestionably requires training, although see below.

But every computer user learns to do these things to some degree. `I want to write a letter to my sister. What do I have to do? I have to start Word. How? Open the application. How? Open this folder. How? Click on it...' And so on. My point is just that this sort of procedural thinking is a kind of algorithm-planning and formalization task, yet non-programmers manage to pick it up.

Some are better than others. OK, but some programmers are better than others too. Who's to say that if we give non-programmers better tools, and consequently more possibilities to automate things, that many more people won't use them?

30-35 years ago no one ever thought that a `personal computer' was feasible, not just because of the hardware problems, but because, after all, who would have the skills to use them? People may surprise us again.

most programs are broken ... This is, I think, the biggest reason most configuration files and whatnot on UNIX are stored as text: people, even programmers, don't trust software.

I never thought about it this way, and it is for me an eye-opener.

One thing I have often thought is that programming languages (or IDE's) might benefit from having a distinct `input syntax' and `output syntax'. ... Indeed, you could even have completely different operator precedences for input and output/syntaxes, or indeed completely different syntaxes, period.

I think this is what Charles Simonyi's Intentional Programming work is about. It will happen (soon) - I see a bright future :)

There is a very cool system called HOPS ... For medium-size terms you get lots of crossing edges and it's hard to read.

I haven't looked closely at HOPS, but I don't think that graphs, as used in HOPS, are very useful as a coding tool. Changing from plain text to more structured tools should preserve some of the comfort of plain text. For example Eclipse offers one-spot variable renaming in a plain text environment.

Another way to eliminate variables, BTW, is to use point-free notation

To me it seems that point-free style is aesthetically pleasing to implementors, but a burden for readers. But this remark is pointless, because future tools will make it possible to switch between the styles.

For example, editors like Word and Emacs have mechanisms for `recording' macros, which is a lot like defining a procedure in a PL.

But one of the most important aspect of programming is parametrization, and I don't know any system that makes it possible to record macros that have parameters.

Marc:

In certain circles I often hear "soon programmers will be obsolete; applications will be generated automatically from simple specifications". I hear a bit of this in "everybody will be a programmer" too.

I did not mean to imply this absurd statement "soon programmers will be obsolete" in any way. On the opposite, I think that in the future more problems will be solvable by non-specialists, and specialists will be able to solve even more complex problems than today.

(Excuse the word 'problem'. I do not like it, because it seems to imply negativity ("uh I have to solve that dull problem".) Maybe it should be "challenge" instead of "problem".)

True, but thinking up an algorithm for a person to follow and thinking up an algorithm for a machine to follow are quite different things.

I can expect a person to "just know" how to go to the market, to use your example, but I have to understand what that means much more precisely to get a machine to do it, and there are all sorts of unexpected problems that can crop up ( the market is closed, the street to the market is blocked off, etc. ) that I can expect a person to just handle, but that I have to solve for the machine.

30-35 years ago no one ever thought that a `personal computer' was feasible, not just because of the hardware problems, but because, after all, who would have the skills to use them? People may surprise us again.

In part, though, this "success" came about by finding the restricted set of problems that people wanted to solve on a regular basis (email, websurfing, running games, etc.) and making those easy. Your average computer user is still helpless when they move off the well-worn path of what they know how to do by rote.

In a PL context, (hey! I'm getting good at this ;-) ), this is like the standard library. I could have a standard library that has a "go_to_the_market" function that has "pre-solved" that problem for me. But to handle my other problems, or to handle unexpected changes in the problem ( the market has just been moved across town) I will need to know how to program the language more generally, perhaps even understand how that function was implemented in the first place.

What makes me sad is that people use the very format that they use to store programs on disk when editing those programs.

Sorry Manuel, I guess I've gotten overly sensitive about it. ;-)

On the opposite, I think that in the future more problems will be solvable by non-specialists, and specialists will be able to solve even more complex problems than today.

I have a moderate disagreement with this, and luckily it links back to the original topic of this thread. ;-)

Raskin is essentially saying "How come after all these years we don't have the 'perfect' IDE?" He assumes that if people just used his wisdom from many years back, we would be living in IDE paradise.

But perhaps this "problem" cannot be solved once and for all.

At different times and for different purposes, the programmer wants certain things to be "made easy", but this tends to make other things correspondingly harder. What happens when those needs change? (New methodologies, new problem domains, changing environment, etc.)

I guess I'm arguing that some problems are too general to have a final fixed solution. We still need engineers to build bridges even though we've been building bridges for a very long time. Each bridge needs to be rethought in terms of its unique requirements.

if you separate programming from all the details
Spoken like a mathematician ;-)
If we discard the details then we should probably be talking about George Pólya How to Solve It, and Michael Polanyi and...

Your average computer user is still helpless when they move off the well-worn path of what they know how to do by rote
To my surprise, I can't figure out what it would mean to be an average computer user. Voice recognition systems are common, embedded devices are everywhere, there's a book catalog kiosk in the bookstore ...
Maybe the average-computer-user problem-solves their way through a Playstation adventure game, or programs the alarm tones on their mobile phone?

Given all the griping I hear from other programmers about how most programmers suck
Seems like the dark-side of the above average effect.

I guess you've never done tech support. ;-)

Seems like the dark-side of the above average effect.

I think that is part of it, but I think there are other profession-specific factors at work too. (the social myth of the "computer whiz", the appeal of the profession to people whose self-worth is based on being knowledgeable relative to others, etc.)

I would go a step further and say that even among those respective groups, different programming styles or perhaps even different domains might require different IDE features.

I think I agree. Can you elaborate on what you have in mind?

Haven't I posted enough today? ;-)

To give a concrete example, as part of my professional activities, I practice Test-Driven Development with Java using Eclipse. Eclipse has several features that assist the developer by performing mechanical refactorings. Now these features may help someone not using TDD, but it seems to me they might just as easily be disastrous or confusing for that person.

Another example might be a low-level debugger (registers and memory dump). This might be an indispensible element of an IDE used for system-level development work, but probably useless at best for someone working in a higher-level language or with higher-level domain.

To take a very language specific example, being able to configure and automate parenthesis formatting is probably more of a necessity than a luxury in an IDE for a Lisp language. ;-) (Though I can't remember if DrScheme had that or not.)

I would like to point out that the lack of a good IDE can be a real problem of even a good PL. For example, consider doing LISP in an editor that doesn't do parenthesis matching. Another example for me is the Nice language. The fact that it's not natively supported by an IDE or a debugger is a problem for me (BTW if it is, I would really like to know about it). Another example is HTA (HTML Applications) on Windows. As I've previously explained, this allows you to create Windows apps using HTML and JavaScript (or VBScript). While this capability is available out-of-the-box for Windows, the only IDE for it is VS.NET, which places it well out of the range for the people who would benefit from it most.

Frank: One thing I have often thought is that programming languages (or IDE's) might benefit from having a distinct `input syntax' and `output syntax'.

Speaking of VS.NET it has this cool feature (which I'm sure is not unique) that allows it to fold code blocks, like an outliner. I find it very useful for C++ coding. And it matches the input vs. output syntax comment in that you view the code in a different way than how it was written.

For me the most important facets of choosing an IDE for a specific PL are:
1. Responsiveness.
2. How it assists me in automatically generating correct code.
3. How much typing it saves me (or more precisely keyboard clicks and mouse clicks/movements).
4. Refactoring tools
5. Integration with external tools such as code revision systems.

Item #1 is the reason I find so many Java IDEs disagreeable. That and the lack of proper implementation of native widget behavior, e.g. JBuilder/JDeveloper use tabs but don't support ctrl-tab as a means of jumping between them.

Items #2 and #3 are the reason I like InteliSense (code completion) so much (and for anyone doing C++ dev using VS v6 I would highly recommend getting Visual Assist, takes that IDE to a whole new level).

I find it interesting that no one mentioned UML.

The representation is sequential text even though the order in which methods and variables and classes are declared has no meaning for the programming language. The representation is flat-text even though the language nests variable and method declarations within classes. So I have to do make-work to manage the presentation because VS.Net doesn't provide a more specific representation.

In short I'm agreeing with Manuel ;-)

IDEs that want to be everything to everybody, also tend to be mediocre in everything they do. Just as one example, take regular expression searches. Grep, SED, and AWK have been around forever and are wonderful tools for searching text. Yet, the regular expression syntax in most IDE's barely gives you any of the expressivity that we've come to expect - invariably being dumbed down.

Personally, I prefer an IDE that can be used when I need it and gets out of the way when I don't. I don't want a monolithic IDE to be my sole view into the software. I want different perspectives with different tools. One-stop shopping is for the lazy.

Good point!

Speaking of VS.NET it has this cool feature (which I'm sure is not unique) that allows it to fold code blocks, like an outliner. I find it very useful for C++ coding. And it matches the input vs. output syntax comment in that you view the code in a different way than how it was written.

Hm, that's not really what I had in mind. I've seen block-folding before and used it in Emacs once or twice; it hides parts of the program, so it doesn't preserve the semantics. Of course, I understand no information is actually lost when you fold, but what I had in mind was rather something more like two isomorphic syntaxes, IYSWIM.

Items #2 and #3 are the reason I like InteliSense (code completion) so much

I haven't used MSV Studio for years, but I've seen this sort of pop-up completion in Java IDE's and frankly I hated it. It was always getting in my way and occluding the program context. Maybe I needed more time to figure out the behavior, though. I also don't like HTML pages and applications with lots of tooltips, though, so maybe I'm just weird.

The representation is sequential text even though the order in which methods and variables and classes are declared has no meaning for the programming language. The representation is flat-text even though the language nests variable and method declarations within classes.

Just to nitpick, that is not the representation; it's the presentation.

I think I do see what you mean, but I'm not sure how such a feature should be implemented. Would you have two views that you toggle between, or would the text displayed be different than what you type? Both options can lead to a somewhat weird behavior.

I've seen this sort of pop-up completion in Java IDE's and frankly I hated it

Good completion is great, bad completion is awful and mediocre completion is, well, mediocre. I also hate bad completion because you end up typing more than you would have if it wasn't there in the first place. It also leads to frustration. An example, in both JBuilder and JDeveloper when you want to use println you would type:

System.out.

and see a list of all possible functions in that package. Including some 20 versions of println, overloaded on the arg type. But the thing is, it doesn't matter which one you choose, they all have a single arg you must type in.

I think a good IDE helps you crank out a greater volume of code.

And that's why I love Visual Assist. It upgrades the completion in VS to a whole new level. I find that it saves me about 20% of total keystrokes, which, if you think about it, is quit a lot. Also, it saves me a lot of lookups in the MSDN.

Oh, and there is a version for VS.NET

the regular expression syntax in most IDE's barely gives you any of the expressivity that we've come to expect

Which is why I really like Epsilon (an Emacs derivative that runs on Windows (has its own C-like PL BTW). But I would take it a step further. I would claim that regular expressions for program text should not be the same as regular expressions for generic text. The regular expressions in IDEs should make it easier to locate various code constructs.

Sounds like what both you and Frank want is the canonical form of the program to be an AST, and the environment to provide tools for interacting with the AST (pattern-based searching, automated code transformation, printing and parsing several syntaxes, perhaps extensible via new customized syntaxes).

This is a neat idea, but it's not all that new. There is, of course, Intentional Programming, and I almost dare not mention Lisp. Java IDEs are moving in this direction as well, I think, after the very useful insight that the desire to have the canonical meaning of the program (and the form that the programmer interacts with) be an AST does not preclude a flat-file serialized form (*.java files).

Why not extend to an arbitrary number of input syntaxes, an arbitrary number of output syntaxes, and an unrelated storage syntax (or possibly several, an ASCII syntax, a Unicode syntax, a binary syntax)?

For Java, isn't that just the byte code?

Sounds like what both you and Frank want is the canonical form of the program to be an AST, and the environment to provide tools for interacting with the AST

If it's good enough for XML (XPath XQuery) why not Java or C++ (Lisp is just too easy ;-)

I would just like to be able to write an expression that lists all the functions receiving an integer variable called 'id', for example, without having to open a manual and getting it wrong twice regardless. Doing it right would probably require working at the AST level.

a good IDE helps you crank out a greater volume of code
Maybe a better IDE would help you understand the code better and result in higher quality?

don't like HTML pages / saves me a lot of lookups in the MSDN
"There are good reasons for having paper manuals" (from the article).

None of the other programming languages that I've used have IDEs that divide-up source-code by scope. Why is that?

(IBM VA Java probably did. The Smalltalk IDEs let you work with classes and methods, rather than blocks of text.)

Yes and no... The byte code loses information from the source code (local variable names, comments) that I consider to be important in any reasonable concrete syntax...

I would just like to be able to write an expression that lists all the functions receiving an integer variable called 'id', for example, without having to open a manual and getting it wrong twice regardless. Doing it right would probably require working at the AST level.

This is interesting, because in the case of third-party libraries, working at the AST level probably isn't quite right either, since you don't necessarily have the code itself. In that case, you'd want the ability to programmatically interact with docs and interface specs. But naturally, it's related. And of course, it's not really a reflection issue, since AFAICT you don't need this ability at run-time, just within your devel environment.

My comment was tongue-in-cheek. Maybe I should also get Ehud's bell. But, as I recall, the byte code doesn't loose as much information as you'd expect (not even as much as you'd want).

"Using the facilities in Smalltalk Refactoring Browser we defined our own program transformation rules and then applied them. This allowed us to systematically make 16,000 changes almost bug free." (less than 30 bugs)

"This technique could be applied back to other languages to produce a 'Rewrite Compiler' whose output is rewritten source code, not machine level code. The key point is that a parse tree is essential to achieving the correct transformation of complex expressions, and a compiler seems the appropriate place for meta knowledge about the target language and system."

Transformation of an Application Data Layer
Will Loew-Blosser

I sort of guessed you were joking, but since you're absolutely correct that it preserves almost everything, I couldn't be sure!

The library for C++ is already so huge that it wouldn't add much weight, and it would be well worth the trouble. Then, all IDEs would have hooks for doing these things, doing them in the same ways, and if your IDE didn't (or, like me, you weren't using one), it would be much, much easier to write your own code transformers.

Doesn't this sound like a job for Aspect Oriented Programming?

(Maybe not. I have not personally been too wow'ed by AOP, but Dan's particular wish here sounds exactly like one of the motivating examples for AOP from the talks I've seen.)

AOP would be great. But can I really retrofit AOP into an existing huge blob of C++ code? (some of it is actually C code)

I can also provide another example of the type of functionality I would like: find all occurrences of a particular local variable in a specific function, discarding occurrences inside comments. Extension: find all assignments to said variable.

For my operator precedence example and a few others I can think of it would be the latter. But the former option is possible too, and there undoubtedly other ways. I haven't given it a lot of deep thought.

Matt: Sounds like what both you and Frank want is the canonical form of the program to be an AST, and the environment to provide tools for interacting with the AST ...

That would be a step in the right direction. An AST is not good enough, though. An IDE should also account for scoping and binding, so it needs to be a term graph. There are probably other details also.

Ideally what I want, though, is that the `canonical form' is the denotation of the program in some deep semantics which accounts for the dynamic semantics (the reductions), so what one is actually manipulating is a semantic object. And, no, I cannot give you any examples at the moment. :) But I know it is what I want. (I can dream, can't I?)

This is a neat idea, but it's not all that new. There is, of course, Intentional Programming, and I almost dare not mention Lisp.

No, indeed, it is most emphatically not new. There have been structural editors for years and years, and this is why Intentional Programming and Gosling's current project don't interest me. As for Lisp, that is a language, not an IDE, but Lisp's syntactic structure is so trivial that it's really not so interesting to have the IDE account for it's AST. Emacs' ability to jump through sexp's is about all you need.

Why not extend to an arbitrary number of input syntaxes, an arbitrary number of output syntaxes, and an unrelated storage syntax (or possibly several, an ASCII syntax, a Unicode syntax, a binary syntax)?

Yes, but who's going to do the work? You can write a little language for expressing parsers to make the IDE extensible, but someone still needs to write a parser for each language.

[Dan's trace example] I would like an IDE that makes this type of code manipulation easy, fast, and safe (after all, the compiler does know how to identify a valid function definition).

Yes, this is one sort of thing I want also.

I really strongly believe that a fundamental part of the standard library for every programming language should be a parser/pretty-printer for the language and standard data types for the AST.

Yes, but that need not be an alternative. Actually the way I think about it is that an IDE is a graphical wrapper for such a library.

(Most of William Opdyke's 1992 thesis was about refactoring C++ frameworks)

I was massaging some problem tracking records from different sources in Excel, using lookups and stuff, and I thought - eh, if we forget about VBA, is Excel with its (first-order?) functional features a language or IDE? Ain't it integrated (coupled?) so tightly you cannot tell a difference? It's so limited, without any refactoring support, and still so many people use it! Where is that magic residing? I could have done the transformations I needed with perl instead, but for some reason I did not... Why - is beyond me.

Any similar experiences?

Whether you want to think about it as a language, an IDE or just an application, it does offer a simple object lesson in the power of ease of use within a limited domain.

Though I have tried a lot of different solutions for the kinds of problems I solve with Excel (laying out data in a table and performing simple manipulations of it), it is simply faster and easier for constrained quantities of data to use Excel.

My own theory of why this is so is that it offers very direct access to the spatial metaphor of a table. I don't need to translate "move that cell from here to there" into some other language to do it; I just move it.

If I want to combine these simple manipulations to do something more complex, it is generally easy and intuitive to do. I'm done before I would have even figured out how to write the test code for a script to do it.

I don't think that this is an argument for "visual programming", since most domains do not have a straight-forward spatial metaphor, but perhaps rather an argument for DSLs: constrain your solution space to a particular domain, and make it as easy as possible to quickly solve problems in that domain.

Perhaps this is similar to the "path of least resistence" subset of "general-purpose" PLs that make their advocates feel that it is "easier and more intuitive" (at least for the kinds of problems they tend to solve.).