Lambda the Ultimate

haxe < version 3 did not support .* and that drives me nuts daily. haxe 3 is, i think, now supporting .* :-)

That is an excellent idea for brittle, unmaintainable, unevolvable, creaky and downright broken software. At Google we have an absurd amount of code--nearly all of it open, shared, and reusable--and have learned very painfully that unstructured reuse of code is extremely bad. It leads to long build times, huge binaries, frequent breakages, and a considerable drag on productivity and evolution. We have learned the hard way that modules really do help programming in the large.

Modules are compatible with Sean's vision. They'll be first class, subject to functional processing, transform, inclusion in lists, etc..

First class modules are very difficult to make static guarantees about. Dealing with a codebase in the many millions of lines of code requires static reasoning, and I am skeptical that first class modules can be designed that don't defeat large-scale tools.

Static reasoning in first-class staged programming systems does require alternative mechanisms - e.g. dependent types, or evidence-carrying code that allows reconstructing a proof (cf. Microsoft Research's F*). I'd say some such techniques already beat traditional large-scale tools, at least for expressiveness and utility in open or long-running systems.

Static local reasoning can similarly be supported if imports themselves are described or implicitly constrained in terms of proofs and contracts (e.g. import by type, constraint models).

I don't really expect these techniques to take off this decade, though.

If we integrate modules seamlessly into "the language's own abstraction capabilities," then I think the "Internet-like global namespace" will not be unstructured, due to its own self-imposed structure. When Sean McDirmid talks about search and versioning, I hear flexible dependency resolution and immunity to (or at least awareness of) breaking changes. (Maybe I hear too much? I might be coloring over this with my own vision.)

Nevertheless, maybe I'm not telling you anything you didn't expect; your concerns would still be valid whenever the module developers didn't impose an effective structure on themselves. Programmers would need to use semantically honest and open-minded search terms, or else they'd have brittle one-to-one dependencies. Programmers would need to do frequent deprecation-related refactoring, or else they'd drag down build times and binary sizes with several API-incompatible versions of a "single" dependency. It would be nice to minimize these responsibilities too, but how? I expect cultural conventions to become more supportive than formalism at this point.

I think language-provided modules are best, and there are certainly opportunities along the ones you describe. However real systems for the foreseeable future are going to continue to be multi-module and multi-language. Thus some kind of language-agnostic or at least inter-language module system is needed.

That is an excellent idea for brittle, unmaintainable, unevolvable, creaky and downright broken software.

I'm sure someone said this about the web once, then we fixed the problem without going the Yahoo directory approach. The module problem with programming seems to be mostly one of legacy, but I don't think it's an intrinsic one. I really see no reasons for the multiple layers of modularity that we have now, just look at how awkward the classes vs. module debates were 15 or so years ago. I bet we can solve this problem with one top program organization abstraction (like classes or traits, or functions if you are from that world) + much better tooling to handle macro issues (search, versioning).

I played the module game before, and this is just my feeling on where we should have been going instead.

I'm sure someone said this about the web once, then we fixed the problem without going the Yahoo directory approach.

Well, humans tend to handle 404s, syntax errors and grammar ambiguities much better than your average programming language do.

I don't disagree with your feeling that both modules and classes/traits/functions carries some level of redundancy.

Oh, the problem is much more difficult than that. It's versioning and trust that are the biggest issues with not having deployment modules. There is also cost to consider, but then we fly into the problem of component markets.

Wouldn't it suffice to replace the harsh "module X not found" error message on a failed import with a search result? In a search-as-you-type editor this wouldn't possibly make much of a difference with your more radical suggestion UX wise. It is also no law by nature that autocompletion must work from the root of a package to the leaf. I'm annoyed by long namespace paths as well but wouldn't bother if I could type "Token" after "import" and get a list of all packages containing "Token" in the path ordered by certain preferences.

In Python caching modules has always been quirky. The language uses a flat cache which associates a dot-separated module path "M1.M2.." as used in import statements together with a (module) object. This is really fast when it comes to (module) look-ups but it can happen that the same physical module is accessed from different import paths and yields different (module) objects. In a sense the "physics" is not coordinate invariant. Changing the perspective means changing the world.

Usually, for us, "changing the perspective" means that we use different sets of paths e.g. a classpath which changes from device to device. Having the wrong classpath means that the world is just broken: galaxy clusters remain black and vanish from our universe if not the whole universe collapses. Although the collapse may not be entirely avoidable it happens so frequently that I wonder if the number of occurrences cannot be somewhat reduced?

It sounds as if you know something about this, but it is fun to post simpe answers to monumental problems. The basic problem of regulation is to see, and identify. A classic in this area is this paper by Conant and Ashby. Here are a fiew more references. The principles are universal. An observable thing has bounded complexity. Gurevich's postulate two.