Lambda the Ultimate

Do you know about this?

there's also a vi plugin for Eclipse. While the plugin itself is commercial, just like the VS one, at least the IDE is for free. :)

now it seems finally these IDEs have got some nice code editor component, just like emacs with viper. :)

still prefer my own IDE: bash + vim + arbitrary auxiliary scripts... :)

I absolutely love IntelliJ, I can't even fathom programming in Java without it. I don't need to go over the details because Dave just did that. I just want to say that IntilliJ (and other excellent IDEs like eclipse) have convinced me that Java is an incomplete language on its own. You need a good IDE to provide a usable interface to the language.

Sorry, but vim/Emacs just doesn't cut it. Emacs and vim are great text editors, but they fall behind modern IDEs.

The problem is that Vim/Emacs can't be extended into a real IDE because they have to be compatible with terminal-mode. Emacs is a little better than Vim in this regard, but still falls short. The vi keybindings are the most important to me, and luckily there are vi plugins for both Eclipse and VS.

Too bad the Emacs of the 21st century (Eclipse) has to be written in Java.

Emacs's hostility to GUI features cannot be explained solely by the need for compatibility with a text-only mode. Turbo Pascal's IDE had in late 1980s almost all visual elements one can see in Eclipse today -- running in 80x25 characters text mode. Emacs (and XEmacs), on the other hand, can't even get the widgets in its graphical version to look anything like the platform it's running on, much less provide a richer (and easily programmable!) model of screen elements like views, trees, drop-down boxes, containers to efficiently organize other "widgets", etc. Either Emacs developers do not think such features are necessary, or the UI code is so convoluted (all those optimizations for screen refresh on 300 baud terminals, you know) that nobody wants to touch it with a ten-foot pole.

Are you quite sure that all those bells and whistles, all those wonderful facilities of your so called modern IDEs, belong to the solution set rather than the problem set?

The best programmers I know don't use them.

Can we compose a laundry list of what IDEs do, and then decide which items contribute to the solution, and which to the problem?

I believe that universally valuable benefits of IDEs are:

1. Build/execution in a single chord of keystrokes
2. Intuitive and context-sensitive access to up-to-date documentation and source code ("how they implement this function in Prelude, I wonder?")
3. Intuitive and context-sensitive browsing of the namespace (what's visible from this point of code, what can be made visible)
4. Tools for packaging my code in a publishable format (again, in a single chord)

...and specifically not refactoring, codegeneration, and other manifestations of weak languages.

And even great languages can benefit from IDEs - I just tried programming in Haskell, and how many times I wished my editor had a backdoor to the compiler, so that it could display the inferred type of the selected code, not just the top-level names (and even those only if the type checker is happy with the whole file). I guess Epigram is something I would love were not it using emacs :-)

I would agree, for the most part. I know when I am programming in Scheme, for example, I much prefer the Dr. Scheme environment to Emacs or even VIM (my preferred editor) because of the feedback it gives me. However, Dr. Scheme can still be improved.

The less I have to focus on the details of the language I'm working with, the more I can focus on what I am actually doing. A highly interactive IDE allows me to not worry about many of the details that I would otherwise worry about. For example, when programming in Lisp I have to think about things like how to use a particular function and what functions are visible from where I currently am, because my IDE isn't that great (there is Slime, wich is better than any alternatives, but could be made so much better if it weren't for the restrictions placed on it by its environment: Emacs). In a larger project there could be literally thousands of functions that I would otherwise either have to keep track of in my head (first parameter is this, second is that...is this even visible from here?) or keep having to look up. This is another feature of a good IDE: the ability to right click on a function, and choose "go to definition." I don't care what language you are using, a good IDE will help you not have to think about details and will improve your efficiency.

I have to contest your point on refactoring. How is refactoring not a universal benefit to IDEs? Do you always get function names right the first time? I know that sometimes when I’m programming (and this may be due to inexperience), the intended use of a function changes as my understanding of the program evolves. Sometimes the original intention changes enough to make the original name misleading. Sometimes the names of parameters change as well (this isn’t as big of an issue since I try to make functions as small as I can, but refactoring (especially automatic refactoring) is still a nice thing to have. I can design upfront all I want, but one thing is guaranteed in programming: change.

when I am programming in Scheme... I much prefer Dr. Scheme

Me too. I just want to be able to write my latex inside, and produce the literate version from the same source file (kinda the reverese of what you usually do, I guess).

Any suggestions on how best to achieve this?

(Think of something like sweave+slatex inside DrScheme)

I might have used a wrong wording. I definitely refactor frequently, and not only in trivial ways (like renaming stuff). My problem with automated refactoring is that it rarely is smart enough, or specific enough. Maybe what I need in an ideal IDE is an ability to easily customize or even create new kinds of refactoring assistants. A DSL for defining refactorings, anyone?

Compared to difficulty of getting refactoring right, the points I mentioned are quite trivial, and should not require an IDE that occupies half of your virtual memory.

There has been some study in defining primitive and composite refactorings. I'm not following HaRe closely but perhaps they offer some support in this area.

I don't believe it's of good use just for "weak languages": how many times did you write some section of code and then see it'll be useful in some other place and then you have to manually crop it, write a function definition at some top level, paste the code and parametrize it? And then, go back to where you were and call it from there? It's very common, in whatever language, and having a syntax-aware tool to do it automatically for you sure is handy enough.

It's specially good when it automatically infers other sections making use of the same code and automatically transforms them into calls to the brand-new function. It's more than handy: it's a blessing!

I believe refactoring should be included in those "good arguments" for IDEs. What i don't see, however, is how a GUI makes any of it better than the "terminal" way of emacs and vim.

Are you quite sure that all those bells and whistles, all those wonderful facilities of your so called modern IDEs, belong to the solution set rather than the problem set?

I find this line of argument discouraging and ill-concieved because it seems to suggest that there is a specific methodology or toolset that is "the best" for a task that is monumentally complicated. It also evokes a romantic view of the past, which is generally misleading.

The best programmers I know don't use them.

My anecdotal evidence suggests otherwise.

Are you quite sure that all those bells and whistles, all those wonderful facilities of your so called modern IDEs, belong to the solution set rather than the problem set?

I certainly don't know of any studies demonstrating increased productivity with IDE use. (Anybody know of any studies on IDEs and programmer productivity? I'm sure they exist, however I didn't find anything good in a really quick Google search.) You could claim that such studies don't apply to languages such as Lisp or Erlang. I would have to partly agree: I believe a good IDE wouldn’t improve an Erlang programmer’s productivity as much as Visual Studio improves a C# programmer’s productivity. However, I bet a good Erlang IDE, if existed, could be tailored to help Erlang programmers focus more on the problem and less on the Erlang. That can't be bad, can it?

The best programmers I know don't use them.

Anecdotal evidence isn't very convincing. Why don't they use them? Do they claim they don’t need one? Is that claim valid? Or is the claim made in self defense of their language because there isn’t a good, compelling IDE for it? (On that note, has anybody used Erlide? How is it? Does it make Eclipse a decent Erlang IDE?)

You know how sometimes VIM and Emacs users talk about how much productivity they gain by being able to keep their hands on the keyboard while doing their text editing. Users of other editors don’t quite understand this. How absurd! The mouse is just a foot or two off to the side. How much productivity can you really loose by having to use the mouse? Of course those proficient in VIM or Emacs believe the claim isn't absurd at all. They believe an interruption, even a short one, is a distraction and can ruin your train of thought. This is especially true when programming.

As much as VIM improves my typing and text editing, the longer it takes me to look something up (a function for example), the more likely I will loose my train of thought. That is lost productivity. A good interactive IDE will give me all (or most anyway) the information I need on the fly, while I’m typing, in a little tooltip popup very close to the cursor. An IDE like that is invisible and seamless: a good tool.

By the way, I do like Slime, it convinced me to install Emacs on my Linux box. I just want to say thanks for all the work you've put into Slime! I'm also happy to see the headway in getting VIM to integrate with Slime. I just envision something better: something more invisible to the programmer. I’m just not sure I have what it takes to makes such a tool myself. :-(

Another TextMate user here.
I've created an OCaml bundle that allows me to make quick work of the common constructs and use of the standard library (I should probably add the Str and Unix modules to it as well), and to allow me to make good use of the code completion/navigation features it has available. Of course that's just an editor. For IDE-alike behavior, I just throw CocOCaml (which I should try to integrate with TextMate a bit) and omake into the mix, and it's really all I need. And when I switch to Ruby/Rails or *TeX, its pretty much everything I could want in an editor.

In addition to that, I drop back to vim on systems where TextMate isn't available. And I really want to throw together a Windows box so I can play around a bit with F#. I guess that would imply some usage of Visual Studio (unless, of course, someone far smarter than gets F# running on Mono).

I am an Emacs old-timer, but when I have to use Eclipse (usually because of external forces), I use the SchemeScript plugin for all my Scheme development.

Not meaning that I have just emerged from cave-dwelling, either. I have never found anything better than ex (I gave up Teco in favor of it) for my purposes.

Unfortunately, most ex's turn out to be vim nowadays, and vim is a very broken ex implementation: for example, the "undo" command undoes everything back to the first time the file was loaded!

One of my back-burner projects is to add just enough features to GNU or BSD ed so that it will be a satisfactory ex for me. Alas, nvi (which would otherwise be ideal) does not build or run on Cygwin.

Update: vim 7.0 is a satisfactory ex.

I'm curious as to what you don't like about Emacs's help system and Info?

Info is downright awkward to navigate in. It's also been obsoleted by html for about a decade now, though I agree with Thomas that a basic manpage should be kept to use as a quick reference.

You can only truly appreciate info from within Emacs, where it combines narrowing to a single topic with search across an entire manual. Granted, an HTML reader with similar narrowing and/or cross-page search would be both more modern and more flexible, but no one's written one yet, despite there being at least two (w3 and w3m) HTML renderers for Emacs.

btw, regarding the original question, I'm an Emacs die-hard because: (1) having to edit a wide variety of languages and formats, I place a high premium on text manipulation power; and (2) the learning/customization curve for Emacs is much smoother than that for e.g. Eclipse.

You can only argue that Info has been obsoleted by HTML if you compare reading Info pages in a text terminal with viewing HTML in a nice GUI browser. This is not really an argument about Info or HTML, but applications used to read documents in those formats.

If you want to compare the formats, consider their design goals and history first. Info was designed to use a single source texi file to produce both typesetting quality output (via TeX) and a hypertext document for online reading (info). It was a big improvement over troff and man pages, which were the standard format for programmer documentation at the time (at least on Unix). The only feature that HTML added which (AFAIK) Info doesn't have is graphics. (Assuming one uses DocBook to have both typesetting and hypermedia outputs, although I personally prefer TeX to SGML syntax.)

Now let's consider the readers for Info, and whether they really make Info pages awkward to navigate in. The canonical Info reader is Emacs of course, and as far as navigation goes, it certainly lets you jump around the document by clicking the mouse on embedded links. There is also a command to go back in the navigation stack. This is no different from an HTML browser, is it? (Oh, and you can jump directly to any named page by entering its name.) A more serious problem with Emacs's Info reader is that it's single-window, so you can't read two Info pages at once. Unfortunately, I think it's pretty unlikely that someone will add this feature any time soon. But Emacs is not the only option for reading Info pages. Gnome's standard help application, Yelp, can render them in a nice GUI environment (as well as HTML and man pages, incidentally), and there is always the alternative of converting them to HTML using texi2info.

And just one more comment on man pages. Yes, they're great to look something up quickly from the command line, but really, their main advantage is in their "standard" organization, which is ideal for a quick-reference type lookup. The source format from which they were produced is secondary, although it sure would be nice to have hyperlinks for the "See also:" section, wouldn't it? Which is available in a number of separate applications (going at least as far back as TkMan), but somehow using those is not quite as convenient as just typing in "man foo" in the terminal. :-)

The only feature that HTML added which (AFAIK) Info doesn't have is graphics.

Recent versions do.

A more serious problem with Emacs's Info reader is that it's single-window, so you can't read two Info pages at once.

Not true.

Unfortunately, I think it's pretty unlikely that someone will add this feature any time soon.

It already exists.

A more serious problem with Emacs's Info reader is that it's single-window, so you can't read two Info pages at once.

Not true.

Using XEmacs 21.4.13 on Windows (the current stable branch), I can open multiple frames with Info in them, it's true, but if I navigate to a different page in one of them, all the views change together. So how exactly do you read two Info pages at once in Emacs?

So how exactly do you read two Info pages at once in Emacs?

You can fork a new Info buffer with `M-n' (clone-buffer), or by using the `m' or `g' commands with a prefix argument (e.g.)

XEmacs (I didn't see that one coming) is a different matter. `C-h i' will reuse any existing *info* buffer, but you can keep an old Info session around by renaming its buffer. If you run Info again it will generate a fresh *info* buffer.

Anyway, I was talking about Emacs, and it has actual support for this.

I find navigating Info to be a complete pain. It's a great disservice that you can't have multiple Info buffers open; you often lose the context of what you are doing. Secondly, the search is based only on regular expression matches and finding abstract concepts is next to impossible. Worse yet, searching takes you right to the point where the regular expression matches instead of pointing out where it would take you, so you end up jumping around a lot (again losing context easily). It's not too bad if you want to sit down and read something, but it's annoying when you are looking for specific information ("What node was that in again?").

To top it off, Info is just plain ugly. It's the esthetic equivalent of sweaty tube socks.

As for the rest of the Emacs help system, it's far too specific. If I know the function or variable that I'm looking for, C-h f and C-h v are very nice (much like documentation in Common Lisp). However, C-h a (apropos-command) is just as useless as apropos on the command line: far too much information comes spewing forth with nothing to tie it together. If you go for symbols that seem to be related to working with X, you'll end up with some useful stuff surrounded in a sea of internal functions that have no meaning and no documentation. It's confusing, to say the least.

Given the introspection capabilities of Lisp, it seems as though something much better should be the default. Unfortunately, it seems that speed trumps usefulness with respect to the help system.

"you often lose the context of what you are doing."

does the "l" keyboard command help? (hint: l(ast) takes you to the last node you were reading.) Info features a full location stack, like most help systems. I fail to see how you can lose context...

"To top it off, Info is just plain ugly."

i fail to see how textual information can be ugly...

"If you go for symbols that seem to be related to working with X, you'll end up with some useful stuff surrounded in a sea of internal functions that have no meaning and no documentation. It's confusing, to say the least."

but it is consistent in that it lists everything even remotely related to X.

"Given the introspection capabilities of Lisp, it seems as though something much better should be the default."

AFAIK, Info is a C program, not implemented in ELisp... it is even an standalone program in most Linux distributions...

"you really don't need much more than insert, delete, find, replace, undo and save for 90% of the work you do"

That can only come from someone who never edited large documents or source code and can't even dream of the large benefits macros, hierarquical navigation, bookmarks quick repetition of last command, numbered prefix commands and many others can improve on just the basic cut-and-paste...

notepad is easy to use for very simple tasks, i agree. Its ease means nothing in face of difficult tasks, though...

I can't imagine using something like vi for editing LaTeX after having used Kile for quite a while. Autocompletion of LaTeX commands is nice, but understanding TeX enough to add new commands in the header to autocompletion is even better. Autocompletion of label names when using \ref is even better - now I can have long descriptive label names and not worry, I cna just narrow down then select from the autocomplete list. Then there are all the many other nice things, like a well organised complete symbol table available in the sidebar with drag and drop - forgotten how to get a particular obscure math symbol? Not to worry, browse the table and drop it into your document. Combined with code folding, structure views, support for multi-file projects etc. it simply makes writing large LaTeX documents remarkably less painful.

I've never used Kile before, but vim does have a rather nice LaTeX package called VIM-LaTeX. Again, as I implied earlier, although I really like the vim keybindings, it doesn't quite cut it any more. As Dave Lopez said, I'm sure this is mostly due to vim (and Emacs) having to maintain compatibility with terminal mode. Too bad really because otherwise one could write GUI driven extensions like autocomplete and refactoring. At least some of of the more advanced editors and IDEs have vi like extensions.

In much the same way I used to be devoted to Emacs (and AucTeX for LaTeX editing). Eventually though the clunkiness of the interface compared to modern devoted IDEs like Kile, Eclipse, etc. (as you and Dave note, largely due to attempting to keep backwards compatability with a console interface) slowly seemed less and less appealing. I made the jump to more modern IDEs and while I miss some of the richer editing features of Emacs for raw text, the additional features have more than made up for it - I couldn't go back now. I can't comment on vi like extensions for Kile being an Emacs person, but it just uses the KDE Katepart for the editor, so if you can convert that to vi keybindings you should be okay.

For example, I just type ":Make A B Foo" and projects A, B and Foo are being compiled in the background and errors are reported back to VIM; in VS I have to fire up three instances of the program to compile three projects at once.

You should really ask someone who knows how to configure VS. It's not that hard. But if you are an amateur programmer, not working in a team and doing pair-programming and project synchronisation with your co-workers occasionally at their workplace, it doesn't really matter what you are working with.

I consider learning programs/programming languages/libraries that are locked to one OS a waste of time so I avoid it whenever possible.

Of course I'm not completely against universalist positions but sometimes I prefer a more adapted/situated treatment. I'm an environmentalist/pluralist/pagan so to say and adopt a completely different mindset when working with Linux or Windows. That's also the reason why I used to learn more than one programming language although it is a waste of time from a universalists point of view: Java should be enough!

You should really ask someone who knows how to configure VS.

I'd be very interested to hear how you can have several projects open and compile them in a single VS window. Should be easy to explain if it's not that hard. If you mean to put several projects in a single solution, then that's not really feasible as the whole product is way too large with way too many projects and parts and bits and pieces to make this manageable.

That's also the reason why I used to learn more than one programming language although it is a waste of time from a universalists point of view: Java should be enough!

Sorry, I don't quite see the connection to what you quoted.

I'd be very interested to hear how you can have several projects open and compile them in a single VS window. Should be easy to explain if it's not that hard. If you mean to put several projects in a single solution, then that's not really feasible as the whole product is way too large with way too many projects and parts and bits and pieces to make this manageable.

I don't understand the latter requirement? I'm working with a dozend or more projects in a single workspace and define dependencies among projects using the project|dependencies menu. If you fix a project as the main project dependent projects will be compiled as well ( a mostly trivial compilation when nothing has changed ). Alternatively you can compile several projects in batch-mode using Build|Batch Build. That works fine for VS6 as well as VS 2005. If there is a natural bound of the amount of projects that can be handled this way I'm curious to know it. But if there is one I suspect that it will help much running 3 or more VS boxes at the same time...

I'm working with a dozend or more projects in a single workspace and define dependencies among projects using the project|dependencies menu. If you fix a project as the main project dependent projects will be compiled as well ( a mostly trivial compilation when nothing has changed ).

This is getting too much OT now for my taste so this is my last word on it, considering that I don't even care much. It's just a real problem I have here, and one in which I have no power to change it and that's it.

There may not be a max. number of projects a solution can hold. But we're talking about more than just a dozen or so solutions (a quick search gives me 172 solution files), each of which already contains between two and 16 (most I am aware of, but I don't know them all) individual projects, and as I said it's not my decision to make, but I'm quite glad so far that I don't have hundreds of projects in a single solution. Esp. since I can't find a way right now to hierarchically organize projects in a solution which I would consider a must for that many projects to reflect which software layer/subsystem they belong to.

Semantically-driven navigation to usages, definitions, overrides, implementations. Single-keystroke name-driven navigation that understands abbreviations (if I want to go to class MyMouseEventListener it's "Ctrl-NMMEL"). Single-keystroke popups for any symbol's definition and documentation. Semantically-aware syntax highlighting (variables that are reassigned colored differently than single-assignment variables, for instance). And if you need the big guns, a full-featured structural query language with both search and replace capabilities.

No new navigation functionality planned for the next release though. I think they ran out.

No you can't download it anywhere.

It's a small script I reimplement on every new project.

It understands the project structure intimately and knows exactly where all the files are allows me to do very rapid regex searches [& optionally replaces] across the whole code base.

Hmm. That reminds me, I need to contemplate using inotify to automagically update a grope cache...

I trust you're not coding C++ in Visual Studio 6. Otherwise your approval of 2005 would be much more glowing.

I will second that Emacs itself is actually quite small and is shipped with a vast suite of Lisp programs. Then, of course, the problem becomes finding the Lisp program that does what you are looking for.

It is commendable, however, that Emacs gives you the ability to write such a feature yourself, but doing so is often too time-consuming.

Following my totally gratuitous editor history, I'll go on to my actual point, which is my interest in using emacs as a widely deployed lisp platform.

Let's skip my 70's punched card experiences. (Summary: lousy UI.) Not nearly as bad was using a line printer remote terminal over a phone modem to access Dartmouth Basic at a local college. I caught the bug and wrote a lot of Basic (quite a bit more than all the other sophomore math students combined).

Way back when (1983) I used vi all the time under 4.2BSD when teaching myself C and Unix. But my favorite editor was MPW at Apple, partly because I loved being able to select test in a worksheet and hit enter to send the text as input to the currently executing tool, or the shell if no tool was running. I used CodeWarrior a lot after MPW; what I liked best was instant navigation to where a symbol was defined. (I can do this fast enough now under emacs.) That continued through Netscape.

I used Visual SlickEdit for two or three years when writing Linux HTTP server code, which had an uncanny similarity to MS Visual C++ studio, which I used only rarely.

These days I use emacs on Linux, and I'm editing this very text under emacs, which I only started using again (after brief trials in the 80's) at Akamai a couple years ago. I can do most of the things in an emacs window (with enormously better navigation and editing ability) that I'd otherwise do on a shell command line. Grep and cvs diffs, for example, are much more convenient under emacs. (Yes I use tags for searching, I don't need help learning how to do that better.)

A couple years ago it occurred to me I might use emacs as a widely deployed Lisp platform, even if I don't like ways elisp is different from Scheme (dynamic scope is creepy).

What I'd really like is a nice elisp reference that tells me how to program in elisp, written to a audience of coders who already know how to program, and who already understand lisp. All I want to know is elisp quirks and practical issues involved in writing code and invoking elisp programs; I'd love to write elisp filter processes with the same command line interface as any other Unix tool. How do I do that? Or if that's not possible, what's the closest approximation in a window under emacs?

When I started studying emacs and elisp in the past for this purpose, I quickly stopped to work on higher priority things, because I couldn't predict how much drivel I had to wade through to even begin to find out if I could do what I had in mind.

What I have in mind is interpreting a mini-language using elisp that I can write other interpreters for in other Lisp implementations elsewhere (including mine if I do another), where I can substitute elisp and any of the others for each other, depending on context. I want to write Lisp code I'll be able to execute on any Linux machine whether or not someone has installed a particular Lisp environment.

Corollary: Would more people program in lisp if there was a mainstream editor that supported lisp well?

One of the biggest complaints that I hear regarding Lisp is the lack of a good IDE (and I admit myself that this is indeed one of my complaints). Emacs (with slime) is okay, except for the horrid navigation. (My pinky still hurts from pressing ctrl so much. ;-) I admit I'm biased, since I'm hooked on the vi/vim key bindings.) Anyway, there are still some issues I have with Emacs (and Vim) as an IDE. Really a modern IDE needs to be closely integrated with the language and give nice visual cues and feedback to the programmer ("intellisense", live parsing and syntax error underlining). Currently, languages are judged as much (if not more) by the tools available for that language as the capabilities of the language itself.

I agree that the environment of a language prejudices potential users for or against it. Some users (myself included) have just never drank the emacs Kool-aid and won't bother struggling with some potentially useful languages like Mozart/Oz because of its "classic" interface. OTOH same users (like me) will waste time with an interactive fiction development environment like Inform 7 just because it has such a pretty little interface.

Please, functional/symbolic/constraint programming language implementors -- think of the left brain, and write good IDEs! :)

"Please, functional/symbolic/constraint programming language implementors -- think of the left brain, and write good IDEs! :)"

Paraphrasing SICP, I believe the purpose of good language design is exactly to overcome the limitations and weaknesses that call for heavy-duty IDEs by making languages more expressive and capable, thus naturally following shorter codes and easier source navigation...

And let's not forget an IDE is an Integrated Development Environment, but everyone has their particular tastes for editors, command shells and other IDE components... the best IDE is the one you assemble by yourself...

Corollary: Would more people program in lisp if there was a mainstream editor that supported lisp well?

In times past when Lisp environment technology was no worse than any other language's environment technology, Lisp didn't acheive any large-scale penetration. Today's high-end IDE technology relies heavily on static typing, and interacts poorly with macro systems and metaprogramming, meaning that it's going to be very difficult for any Lisp IDE to match the flexibility and suppleness of the best modern Java or C# IDEs. Put those together, and it doesn't bode well.

I'm not sure that there's a technical reason why dynamic languages can't match the IDEs of static languages. Many of the things pointed to in modern IDEs, like object and refactory browsers, first appeared in languages like Smalltalk. Indeed, a case could be made that the Smalltalk image provides a very rich source to tap for maximizing the development environment experience. One could also make the case that the development environment, where the code is in a state a flux, basically mimics an interpretive (read dynamic) environment. As a small example, the Smalltalk Transcript in combination with images means that I can do testing of things in the blink of an eye. It's probably not by accident that TDD comes from the dynamic language world.

Ok, so intellisense is harder to implement because you don't necessarily know the type of a variable at every turn. But aside from this, are there other things that you have in mind?

On a more general level, I remain a skeptic of whether or not sophisticated IDEs really represent magnitudes in measurable productivity boost. I'm currently working on two middle size projects in C#: One uses no IDE at all, while the other is heavily dependent on Visual Studio. What I find is that I waste time in different ways, and that the development environment has subtle and not-so-subtle influences on the way the code gets constructed - not always for the better. For example, a typical Lisper will start writing the top level routines first, deferring the definition of the functions needed by the top level to a later time (i.e., they are named at the place where they are called first). In static languages, I notice that it's usually the other way around - define all the worker bee routines first and then go write the routines that use them.

On the whole, I do believe that tools can help in the process, but I find that ultimately the act of constructing software is an intellectial exercise that transcends the immediate acts of banging out or testing the code within an environment. Of course, that could just be me and the particular projects that I stumble into.

Ok, so intellisense is harder to implement because you don't necessarily know the type of a variable at every turn. But aside from this, are there other things that you have in mind?

Navigation to usages/definition/documentation becomes much more difficult to implement.

Automated refactorings become more difficult to implement, but more importantly they are more likely to induce errors when usages can't be reliably found. That makes them less likely to be used, and thus less likely to be well implemented.

Opportunities for static analysis are vastly reduced. This is the big one. I expect a modern IDE to help me read code, help me intuit design from existing code, help me find errors in code at edit-time, and assist me in making explicit the large scale architecture of my code. All of these require precise dependency and dataflow analyses, at interactive time-scales, and that mostly means static typing.

Fundamentally, static typing (and to a lesser extent lack of macros/metaprogramming) makes code reliably machine-navigable. Reliable machine-navigability means that pretty much any interesting analysis or transformation can be automated. My gut feel is that any language that doesn't support reliable machine-navigability will be relegated to niche markets as IDEs become more powerful and pleasurable to use.

In static languages, I notice that it's usually the other way around - define all the worker bee routines first and then go write the routines that use them.

That's odd, since modern IDEs are perfectly happy to generate stubs from unknown routine calls.

On the whole, I do believe that tools can help in the process, but I find that ultimately the act of constructing software is an intellectial exercise that transcends the immediate acts of banging out or testing the code within an environment.

Agreed, but I would assert that modern IDEs are only secondarily about writing code, and primarily about assisting in the understanding and maintainence of code. In fact, the best way to quickly evaluate and IDE sales pitch is to see how much they emphasize code editing and generation capabilities, over analysis, maintenance, and refactoring. If an IDE is mostly about code creation, it's probably not worth the effort to download.

Automated refactorings become more difficult to implement, but more importantly they are more likely to induce errors when usages can't be reliably found. That makes them less likely to be used, and thus less likely to be well implemented.

*cough*Refactoring Browser*cough*.

A dilemma between static or dynamic -- only one or the other -- is not strictly necessary. So an argument claiming we must choose one or the other is a false dilemma.

I almost brought this up in the recent Code Generation Network story, because I had been thinking about a way to generate the missing type information in languages that don't require explicit types for variable names. But the idea now seems more applicable in this context, if some folks assume type information can only be written by hand. First some background.

In professional work I'm asked to change old C++ code a lot, and it could easily be a nightmare if not for static type checking to reveal unintended error. Systemic change is very error prone, so I want to preserve more protection when programming in the large, even when using a dynamic language with very little explicit static type declaration.

The next idea undoubtedly sounds like something you've heard before, in the context of using past runtime data to affect on the fly recompilation of code to be more efficient.

Imagine a large body of code with types declared rarely or only seldom. How can you infuse this body of code with some additional helpful type information without putting in by hand? Approach it as a problem.

I was already planning to capture a great deal of runtime data in a new dynamic language, because I wanted real information about system behavior and usage. So it would be trivial to assume, say, convenient capture of actual types passed dynamically at runtime when dispatching methods.

You see where I'm going with this already. The set of actual types passed to each method dispatch at runtime can be captured in a log, and this can be used to generate an additional compiler input with variable type info. This would cause the least chaos on source code control if the type information was declared out of line, in separate files. Such external type annotations amount to static typing when used in subsequent compilation. You need merely decide whether to use it or not.

The use of IDEs doesn't make it harder to have static typing in dynamic languages. It makes it easier, since you can assume it's easy to capture, maintain, and use static type information gathered from runtime logs. (One might even write test code with an eye to feeding the initial types one wants to see folded into the static type information.)

Anyway, an IDE could show static type information inline, merged with your code, even if you didn't put it there yourself manually. When you do add manual static type info, you might arrange to have this provisionally widened or narrowed by the out of line static type info stored elsewhere.

If you assume an IDE, it's also easy to assume static type info is easy to suppress or extend or update. In a dynamic language where static typing is not required, you need not be obligated to add it by hand. But it doesn't mean you have to live without the benefit of static types. And it doesn't mean you can't have all the machine navigability and support associated with static types, even when a language is very dynamic and loose with types.

You'll note that I didn't say that static analysis or automated refactoring of a dynamically typed language was impossible. (That would be a very odd thing for me to say, having worked on a commercial static analysis tool for JavaScript.) What I said was that such analyses would be much weaker, and the automated refactoring tools much less usable. The system you describe, while certainly feasible, is unquestionably going to be less usable that current refactoring/analysis tools for Java or C#. Those "Just Work", and don't need annotations, test-case feedback, or user assistance. The example was brought up of the Smalltalk refactoring browser fits this as well. It's an impressive technical achievement, and I've used it to do some exciting things. That said, it's usability and scope lag greatly behind modern Java refactoring and analysis tools, and that's pretty much directly due to Smalltalk being dynamically typed.

This may seem obvious, but if you're talking about tools that actually get used by developers, usability matters quite a bit. There's an enormous usability gulf between "tools that work 99% of the time, and you have to manually check that they did what you expected them to do" and "tools that work so well that you stop checking on them". The first set gets pulled out periodically, if you really need it. The second set gets used zealously. It's very impressive that one can build analysis and refactoring tools for dynamic languages good enough to make it to working 99% of the time, but doing that provides considerably less than half the value of tools that work 100% of the time.

I would say that 100% is not exactly correct number for Java, as it is not 100% statically typed. OTOH, with advent of generics it became much easier to use a static subset of Java (though I suspect the magical 100% can only be achieved on really straightforward code - or with really simple refactoring requirements).

with the caveat that modern IDEs are getting pretty good at finding possible incorrect casts at edit-time. Coupled with generics, I find I'm seeing fewer and fewer ClassCastExceptions, and a lot fewer explicit casts.

Don't forget about reflection, class loading and the dynamic side of Java in general, which adds most of its flexibility, and most of bugs (not the least because this stuff is not statically analyzable).

Navigation to usages/definition/documentation becomes much more difficult to implement [in a dynamic language].

Is this really true? I've always thought it's object-orientation that make definition-lookup difficult because of the indirection that it encourages. I also don't understand how static analysis can fully recover what is lost to indirection, but I would be interested to hear.

I would like to illustrate my point about object-orientation by contrasting the "find function definition" algorithm for Erlang compared with Smalltalk. (Common Lisp would have made a more interesting example of both, but that's a longer story.)

In Erlang a function call looks like this:

mymodule:myfunction(Arg1, ..., ArgN)

A good algorithm to jump from function call to function definition is:

1. Parse function call into module, function, number of args. This is straightforward.
2. Find the source file that implements the module. A good way is to first lookup the object file in your code-path and then use a utility function to find out where it was compiled from.
3. Find the definition of myfunction with the right number of arguments. This is straightforward because the syntax is unambiguous.

This is the implementation that I have been using for years and it works very well. The hardest part of the simple implementation is parsing the function call to see how many arguments there are.

Compare with Smalltalk where a message send looks like this:

anObject do: aBlock with: anArgument

Now we don't know what class anObject belongs to and there needn't be any one answer. This means we can never implement the same command as for Erlang, so we settle for the editor command "Find all classes that implement the do:with: message" instead. This is easily implemented by looping through all the class objects in the system and checking them for the right method. However, it is far enough away from my "Find the single implementation of this function" in Erlang to be a bother.

I would also think that Java IDEs would have trouble in situations like this:

MyInterface obj = ...;
obj.foo(myArgument);

How do you step through the indirection of the interface to show the method that will actually be executed?

P.S. It would be interesting for someone to post a new discussion topic where they explain the best features of these fancy IDEs. The demos that I've seen of e.g. IntelliJ have always emphasised solution to problems that are critical to Java programmers but simply don't exist for me, like using type inference to generate type declarations. How about a summary for the dynamic languages crowd?

goertzenator: I've tried to penetrate emacs a few times for the purpose of programming in lisp, but have given up in frustration every time.

For most folks, emacs seems to have a long ramp time to learn. The number of things you can do is enormous, but the number of things you need and want to do all the time is much smaller, and hard to find in the deluge of options unless someone shows you. Most ways I use emacs were shown to me by someone who asked me, "Why don't you just do X?" (And every time I sighed and thanked them for a lesson that followed.)

An ideal intro to emacs, which doesn't exist as far as I know, would be a guide showing you how to do operation X for each thing typically done in other classes of well known editors. But once using emacs, one typically does a few things under emacs that other folks don't do in their editors (like using gdb, for example), and a direct comparison is hard.

goertzenator: Would people use emacs as much as they do if there was a mainstream editor that supported lisp well?

I never had a single coworker who used emacs for lisp. They all used it for C++. So the lisp connection is irrelevant to the value in using emacs, for everyone I've worked with. In an economic demand and competition sense, another editor supporting lisp well would have zero effect on folks I know using emacs.

goertzenator: Corollary: Would more people program in lisp if there was a mainstream editor that supported lisp well?

If it helped them solve a problem they perceived, and this was apparent to potential users (a subtle but key point), and cost to get involved was small.

The key to answering most such questions is an analysis of network effects that might be touched by a tool you want to insert into the environment. And there are a pretty large number of things that can affect a balance of values folks are weighing in costs and benefits. If you want a new lisp tool to succeed, try to make sure you:

• simplify doing some new things cheaply at low risk
• fit into the current ecology so you reap network effects
• be clear, funny, clever, and very succinct in your docs
• avoid making a new tool that can't scale and evolve
• avoid asking folks to change how they do things (they won't)
• avoid making old tools and approaches more expensive
• avoid blundering into other tools' future growth path
• avoid competing with wired folks who hate competition
• avoid getting burned at the stake for actionable heresy

I'm sure other things will come to mind -- this is just a start. But the first one is the hook: can you show some cool thing a new tool can do that's not simpler and cheaper done some other way? Or if it's just as easily done another way, is your new tool a way to pick up some new side benefits, or to reduce costs somewhere else?

Best is a new tool that works with old tools and has the ability to augment or amplify the value of those old tools. You'd have to sell them on the idea that Lisp made the old things better, instead of replacing them.

Except for profiling, everything you list is present in Eclipse and basically all the other Java IDEs out there, and has been for at least 7 years. Modern Java IDEs have all that as a baseline, and tend to differentiate based on automated refactoring tools, code analyses, visual editors, usability, cost, and about a bezillion tool and framework integrations.

Other than NetBeans, profiling isn't native to any Java IDE I'm aware of, but the IDE integrations with commercial profiling tools have gotten very slick. I recommend either JProfiler or YourKit, both of which have seamless integrations with all the major IDEs, are extremely full featured, and reasonably priced.

DrScheme provides a nice visual feedback for errors by actually drawing in the editor screen arrows and circles delineating the errors provided by a stack trace.

I've never seen any IDE doing something like that and although it may look like a minor gimmick, i believe it truly shines among students and beginners in programming languages. Heck, even I was amazed the first time I saw it in action, as it provided a spot-on feedback of exactly where the error originated.

sweet.

Is that a crutch?

Are all those plugins for Emacs and Vim that try to bring IDE-like functionality to those editors a crutch?

Vi is a crutch....real men program in notepad.

Please see the policy document (linked from the faq) for the LtU policy on posts of this nature.

Note that hot button issues like IDEs and types were discussed here many many times (search the archives!). Many people share your sentiments, but many others disagree. Simply expressing your feelings does not advance the discussion of these subjects, so it is better to try and present arguments to support them, that haven't been discussed here before.

Welcome to LtU.