What will programming look like in 2020?

Things are too quiet lately, so maybe its time to start a fun thread. I used this question as a lead in to a presentation on live programming I recently gave at a working group, but I thought it would be fun to gather a bunch of predictions if anyone is willing.

What will programming look like in 2020? Keep in mind that programming in 2012 mostly resembles programming in 2004, so could we even expect any significant changes 8 years from now in the programmer experience? Consider the entire programming stack of language, environment, process, libraries, search technology, and so on.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

App Servers

I predict that application servers will be the common way of developing apps. You'll abstract most applications with a DSL, structured of applets or formlets operating on an abstract client API. The same app can then be 'compiled' and served to multiple mediums - as a web-page, an android app, a Chrome app, or even a desktop app. This technique will enable portability, efficiency. With careful design of the DSL, it can also support occasionally-offline apps, runtime upgrade (and live programming), efficient caching, implicit synchronization of client data, and secure mashups (including ads).

The DSLs in question could be embedded or external, and there will probably be many competing variations in many host languages.

I don't really see this

I don't really see this happening; there is just no easy way to express design tweaks and layout in a client-independent way. To be honest, I don't see abstraction as getting much better in the future unless we are talking about something like machine learning/AI.

Not client-independent

Actually, my vision was that the exact layout and styling could be parameterized and compiled per client and device, or even per session (if the medium allows). There would be default policies and templates for a variety of common use cases, of course. But developers could hand tweak rules as needed.

(Or a good DSL could even enable clients to tweak layouts and share them.)

Even if no code at all is

Even if no code at all is shared between different platforms (unlikely) then it's still a big advantage to have a single language and standard library that targets multiple platforms. Currently if you want to develop for ios, android, desktop, web you have to learn 4 different sets of languages, tools and libraries. That is a huge time investment to make and it would be valuable to get the ability to write for the other platforms for free if you've mastered one.

Heroku

Is the closest thing to this today Heroku-style PaaS?

Haxe, Curl, Ur/Web, and GWT

Haxe, Curl, Ur/Web, and GWT are close on the languages side. (And there are probably more I don't know or don't recall.)

And I'm currently thinking about how to support it from Haskell - albeit, mostly in context of my Sirea project. I think I could get a lot more bang for each line of code if I focus on app servers rather than direct use of keyboard/video/mouse/sound - and that, with a little careful design and abstraction, I could still effectively support the 'local canvas' (wx, local audio, OpenGL).

I wouldn't be surprised to see such idioms in other apps - i.e. if you can 'serve' a web page, why not also 'serve' a wx form? A sort of parametric cross-compilation to different client models need not be very difficult if you have a good abstraction for API (and client parameters and extensions).

Today, the main limiting factor seems to be that most languages don't have effective support for partitioning or code distribution - i.e. to allow some parts of a program to run on the client, and other parts on the server. The secondary limiting factor is that our code often is too tightly bound to a particular GUI model (e.g. containing fragments of HTML rather than semantic descriptors, or deep integration with Win32 peculiarities). The third limiting factor is that HTML was historically bad at a lot of things GUI shells were good at, and vice versa - but that distinction is diminishing with the help of libraries, plugins, new HTML 2D and 3D canvases.

PaaS isn't at all what I was describing, though it also has value and may serve as a useful distribution mechanism. (I wouldn't be surprised to begin seeing 'meta-clouds' that run on multiple providers, switching based on time-variations in price and risk-management policies.)

in a similar vein

I'm not sure about a DSL, but application servers is a decent call. We already have plenty of software "appliances," apps running on application servers, and virtualization uses for management & security. I think these can be further expanded to create well-managed app instances with small TCB's. The Mirage project is trying to do that in Ocaml and Barracuda inadvertantly does it for Lua app server for embedded use. (That they run their whole web site on their platform and a plug computer is pretty cool.)

Then, there's the MILS offerings and microkernel projects that let you choose isolated apps on microkernel, standards compliant runtimes (e.g. POSIX) or VM for legacy OS/apps. Great for TCB and gradual decomposition. I could imagine taking say a DNS server, putting it in a VM on one of these platforms, and moving its functionality into isolated protection domains piece by piece. Eventually, invisible to user, you have greatly improved modularity and assurance of the software without having to rewrite it all at once.

I will buy the DSL thing

I will buy the DSL thing too. Furthermore, I predict that the DSL will be derived from a vendor-designed hack on top of JSON (or whatever the 2020 equivalent is; it could be anything but XML), and by 2020 we will still be hacking around implementation-specific quirks despite having a W3C or ISO standard for it.

(Incidentally: The implementation-specific quirks probably won't be around multitouch interaction. A model for multitouch interaction will be standardised by 2020, much like the model for mouse interaction is standardised today, and implemented in all the major UI toolkits, DOM and the application DSL. The same will be true of 3D displays, assuming they are deemed sufficiently useful for general-purpose user interfaces. There will, however, be a new fad in input or display technology which the standard can't handle, and this will result in multiple competing extensions.)

I also predict that by 2020, NoSQL databases will have converged on something close to a de facto standard (which will also have implementation-specific quirks that must be hacked around), but there there will be native support from all major vendors for this DSL, enabling tighter integration with application servers, at least in theory.

I think you're right about virtualization, too. By 2020, hardware virtualization will be ubiquitous to the point that it's done for each application; spawning a new virtual machine will be as easy as spawning a thread is today, and far easier than MPI. Moreover, there will be programming support for QoS in these virtual machines, so that (for example) the OS could slow down a core or relax the hardware exception model to conserve power if the program doesn't need it.

Not really

so could we even expect any significant changes 8 years from now in the programmer experience?

I wouldn't expect many. We'll probably be doing more work for mobile devices including discovery and communication with more 'personal devices', but how we do that I don't think is going to change.

hardware first

I think that to think about programming you must first look about the hardware: how will it be different in 8 years?
I think that we know more or less how the hardware will evolve: several (4-16) heterogeneous(not all with the same performance, some with specialized function unit) CPUs (due to the power wall).

Now will programming will evolve to use a "new" language better suited to program this kind of hardware or will it be similar to the current weird mix of languages (C/C++/Java/OpenCL etc)?

I'd say the latter (mostly).

HCI-ware (and wear)

While increasing heterogeneous and concurrent architectures will certainly impact languages and frameworks (and there is much progress there already, e.g. to leverage clouds or GPUs), I'm more interested in how things like Project Glass, LEAP motion, Emotiv, and Touché might impact programming.

Project Glass could provide pervasive access to eye-cams, voice, and HUD feedback. Something like that would be a suitable basis for pen and paper programming, or card-based programming, or various alternatives that involve laying out objects in the real world and clarifying with voice (perhaps non-speech voice).

Well, GPUs should become

Well, GPUs should become more powerful and (carefully) more general purpose. But to keep their edge over CPUs, they'll need to maintain lower hardware abstraction (less caching, less virtualization). CPUs should remain similar to how they are today but less relevant as GPUs continue to provide an order of magnitude better performance in many data-parallel applications spaces, which will just continue to expand. Heterogeneous hardware will probably mean a mix of CPU vs. GPU nodes (this is beginning to happen now).

I disagree slightly on what

I disagree slightly on what multicore CPUs will look like. I think they'll be mostly homogeneous (some specialised functional units may be different, I agree), but the power use of each won't be static, it will be dynamic. Power can be altered by the OS by altering the clock speed, or dynamically controlling the cache size, or switching between out-of-order and in-order execution, or something else not yet thought of.

Of course, the inevitable result may be a virus which destroys your CPU by overheating it.

Predictably…

I predict that functional programming will continue to gain ground as people discover the benefits of immutability and easy parallelism—but the functional language will not be Haskell, nor Scala, nor Clojure. At a guess, people will use something with:

  • Strong tooling and libraries
  • An accessible type system
  • Deterministic memory behaviour
  • By-default strict evaluation
  • Commercial backing

Every mainstream functional language is lacking in at least one of these areas.

So...

So, basically, C++11, then? ;)

Sure, except for the type

Sure, except for the type system, tooling, and library bits. C++11 is a good step forward from C++03 to be sure, but as an “expert” in C++ I’m rather strongly biased against the language as a whole, cool new features be damned. If you already know it, then it’s a powerful tool; but the effort required to adequately learn it could, I think, be better spent elsewhere. And to be perfectly honest, I’m also biased toward the features I mention because they’re essentially the principles of my own language-in-progress.

Of course

I agree completely, and my tongue was firmly in cheek. C++ remains, unfortunately, extremely complicated, and all attempts at simplification-via-augmentation have so far had exactly the expected outcome.

I'm not actually sure I agree about tooling and libraries, though. I've had surprisingly good luck with Eclipse for the past couple of years, and I've gotten a lot of mileage out of just the standard library and Boost. But of course it's still a far cry from the situation with, say, Java.

The somewhat serious point behind my joke, though, is that the particular combination of features you named has yet to be gotten into a comprehensible form. In particular, a "heavily functional style" and "deterministic memory management" are pretty hard to reconcile, and even harder when you still want an "accessible type system." (Compare, for example, ATS, which is quite impressive but very far from accessible.) If you can achieve something in this area, I will be your biggest fan.

Yeah, the STL is great. The strength of C++ is really in its generic programming facilities—though iterators leave something to be desired in the way of composability. D-style ranges help, but have problems of their own. That aside, the real problem I have with C++ is plain usability. Even with good error messages and code navigation tools, it’s remarkable how often work gets interrupted by the need for careful thought about how to navigate some trap or another.

Thanks for the link to ATS. I hadn’t heard of it before, but at first glance I agree with you: it’s impressive—half because it seems genuinely useful, and half because it spares no complexity. As for my own language, there’s plenty yet to be done in the way of polish, so don’t hold your breath. ;)

Some safe and some bold predictions

Things from academia that will (start to go in the direction of) mainstream by 2020, though perhaps not entirely in its current form:

  • Dependent types
  • Staging
  • Type classes
  • Functional reactive programming
  • Modules

Things from the mainstream that academia will start to pay attention to by 2020:

  • Value types
  • Library design, in particular of the standard library
  • IDE design (e.g. more direct feedback, structured editing, library as UI, tools for collaboration, UI aspects of types)

Though another option is that academia will focus solely on the mathematically provable or clearly measurable, and then the last two will not enter academia. This would be sad in my opinion, because I'm sure that academia has a lot to contribute to this.

Things that are somewhere in between mainstream and academia that will be important:

  • Distributed partitioning of a single program (mostly but not entirely client/server)
  • Databases more directly integrated with languages
  • Machine learning, including language & library support (e.g. automatic differentiation and probability models)
  • Domain specific compiler optimizations
  • New hardware, in particular C will eventually lose its reputation of being a fast language
  • Realizing some of the goals of AOP but this time by attaching functionality to data types instead of to procedures

Things ranging from academia to mainstream that are overrated and will (start to) go out of fashion (but some still valuable in a niche context):

  • Lazy evaluation
  • STM for parallelism
  • Compiler based nested data parallelism
  • Monads
  • Functional data structures everywhere
  • Heterogeneous computing in the form of a CPU/GPU split might be popular in 2020 but will eventually die out in favor of hardware that can take good advantage of both small scale instruction level parallelism (CPU) and data parallelism (GPU)
  • Javascript (and later the web in general)
  • Text based formats (of course a lot of legacy formats will be out there by then)
  • The command line
  • Huge programs

Debates that both sides will win because they will be combined into a single superior solution:

  • Dynamic vs static
  • OOP vs FP
  • Testing vs types

Debates that both sides will lose because the debate will be made irrelevant by a third option:

  • Emacs vs Vim
  • Windows vs Linux
  • Git vs Mercurial

Lazy evaluation

I'm curious: you mention you expect lazy evaluation to be overrated. In what contexts is lazy evaluation frequently used today where strict evaluation may be a generally better choice?

Laziness is a means to an end

Even the defenders of specific lazy functional programming languages seem to think that laziness is mostly a means to a end, and the end is purity. Me, I'd rather be able to write mostly-pure code in Scheme or Pure, introduce very careful doses of impurity, and have the intuitive advantages provided by strict reasoning.

Dependent types would be one example

Also, giving the power to the programmer to determine the evaluation strategy seems like a natural modularity need for larger and larger pure functional systems.

Thanks! Playing with the

Thanks! Playing with the evaluation strategy for a system certainly sounds like great fun. I'll have to look into dependent types.

In lazy functional

In lazy functional programming languages ;-) Here are a couple of reasons why I think lazy evaluation will not be victorious:

  • Most programs do not make much use of laziness but laziness still makes reasoning about performance and memory hard.
  • If a program does make essential use of laziness for efficiency or termination then it's clearer to have that marked explicitly in the code and types, which is exactly what strictness by default + explicit lazyness gives you. It's telling that Okasaki does this in his work on functional data structures. This is similar to explicit effects with monads or an effect system vs implicit effects.
  • Laziness doesn't play well with parallelism.
  • Laziness gives you a uniform slowdown on your program which even very smart compilers (GHC) can only partially undo.

And, as the people above have mentioned:

  • Even Simon Peyton Jones has said that laziness mostly mattered for keeping him honest about purity. If he could wave a magic wand to make Haskell strict he'd probably do it.
  • No laziness by default doesn't necessarily mean strict by default, e.g. total languages and separation of data/codata in dependently typed languages

Heh

Fair enough =X

Thanks for the detailed response. It looks like I have a lot of reading to do.

lazyness

Laziness doesn't play well with parallelism.

I'm curious about this point. It would seem to me that programs written with no particular order of evaluation would be a huge advantage for parallelism. Would you be willing to expand on what you meant?

wild guess with concrete example

I nodded when Jules said, "Laziness doesn't play well with parallelism." So I'll take a stab at what it means, though Jules might mean something else. I read it as meaning, "Every kind of ambiguity can cause problems." When you combine subtle code A with subtle code B, you can get subtle squared, which can be slow when code only barely manages to figure out the wrong state is currently in hand. I can give you a concrete example, which even involves laziness as a factor.

Suppose you write a page cache, which I prefer as i/o over memory mapping (specifically so you can schedule i/o without being at the mercy of a VM flush daemon). When a page cache has dirty pages, you can write them eagerly or lazily. But an extreme of either can cause higher latency. Only so many iops (i/o operations per second) can be had, and when the pipe is full you don't get any more. Roughly speaking, any time you schedule a spike of i/o greater than iops available, you get latency. So the best time to do lazy i/o is when you're not busy, so work backlog is lower when you have a rush of activity. But scheduling that's a pain, because it doesn't correspond to specific events observed in code during normal operation. You actually have to schedule, ick, instead of responding passively to outside events.

Consider one extreme. Suppose you never write a page until no clean pages remain and a new page must be read. So you write a dirty page, then fill that newly clean page with the read you wanted. Result: two i/o operations (write then read) every time you just wanted to read, with less than ideal latency. That happens because recognizing need to write occurs at the last possible instant, when it's too late to take advantage of earlier free iops. Planning ahead can make resource scheduling more efficient.

In general, the sooner you know something will be necessary, the better job you can do satisfying need with available resources. If you can't find out in advance, because laziness defeats ability to predict, you can get caught on the raw side of costs. Thus anything that delays finding out the score can result in (possibly much) higher costs. Code can keep saying, "Wow, I wish I had known this before." That's what I guess was meant.

paging

Good example, lets work it some more.

So assume we have a paging algorithm (P) I'm assuming the system will be something like:

1) Store pending reads from threads as soon as they come in.
2) If there is no memory available write prioritizing LRU (least recently used)
3) If there is memory available and pending reads, drop written LRU data and read.
4) If there is memory available no pending reads, then write prioritizing LRU. That way these pages can be dropped if needed.

(4) is different than your example but I think is more realistic. Now we'll let PL be a lazy version of P and PE be an eager version. I'm hard pressed to see how PE and PL will be any different. You obviously can't drop memory until is is written: so 3 must come after 2. Any system is going to want (1) to go first. And 4 is lowest priority in both.

______

OK so assuming you agree PL and PE are the same (both P) now assuming L is a lazy program using P and E is an eager program using P I'm still having trouble seeing the problem.

For L the situation is going to be something like

L1) Grab a computation
L2) If you need data from memory get it.
L3) If the memory isn't available (paged out) request the page from P (indirectly) suspend the thread and move to a new one.

Which is going to look exactly like the eager system's E1, E2, E3. As long as the lazy system keeps enough threads going it will be more efficient.

So I'm stuck seeing the problem. I may be being stupid here but I'm not getting it.

slight elaboration plus minor conjecture at end

I don't mind paging algorithm analysis, but I only meant to illustrate a specific concrete example where maximum laziness can increase latency by not using ephemeral resources when available earlier. (Since I haven't said anything at all yet to predict how programming might change over the next seven years, I should try to squeeze in some. Note I think it will change only slightly, in directions where obvious pressure leads.) I think laziness can make it harder to schedule tasks on available parallel compute capacity.

In some ways paging isn't the best example of a problem under laziness, because write backlog is clearly visible pending work, and the best example would be something you were unsure needed to be done until demanded. I only picked that example because I saw both those extremes happen in a prototype, showing a time-based scheduler was needed to get optimal i/o in a caching-oriented app I won't describe much.

Here's an aside on memory mapping. A few years ago we shipped product using it, then QA folks found interesting latency blips under heavy load. At first we thought user space locks were at fault, because we could see—shudder—locks held up to several consecutive seconds. But further analysis revealed the OS flush daemon got behind, and the OS decided not to schedule a user space process for several seconds (exclamation points go here) causing all network traffic to back off because our TCP stack was in user space. The phrase "dead in the water" comes to mind. Online searches revealed that was a common fate for anyone (e.g. databases) doing heavy i/o with memory mapping. So I said: with a page cache we can do i/o when it suits us.

A first page cache prototype I wrote to get numbers justifying further pursuit had a problem of being either too eager or too lazy. Unless I made an effort to write dirty pages, they tended to accumulate and later cause the double i/o scenario. But simplistic efforts to write sooner caused eager writing — far earlier than necessary — with similar high i/o latency. But numbers were great when write backlog was not in the way, so folks loved the demo, even given effects of my dumb first-pass i/o scheduling. (Load balancing tasks gave another dev the storage daemon problem, so I don't have fresher full analysis than that.)

If you want better examples with parallel lazy tasks, it might be better to construct cases with realistic layering and need-to-know barriers from abstraction, impeding optimization when global visibility of facts is obscured. It's easy to create situations where the left hand in code doesn't know what the right hand is doing. A centralized page cache is not hard to optimize because you can see the score. A better hard problem example would be one where layers don't disclose what they're being lazy about.

Note it's important to try things and see what happens, to validate theory. (And theory is always needed to explain what's happening, since otherwise it's hard to construct and test a hypothesis.) A rapid feedback cycle is very helpful when getting empirical data about ideas. It would be nice if folks conducted more arguments by modifying code and seeing what happens, instead of counting angels on heads of pins. So I hope there's progress in live coding efforts that help devs try things more quickly.

I'd love to see folks write more prototypes in daemons you can run experiments on by twiddling a proc-file-system style api via browser. That would be a way of getting more technical benefit out of browsers. If I predicted that was going to happen (and I have no idea really), it would be a change in development process that only indirectly has an effect on programming. I guess I expect changes in programming to be driven by changes in context of development, such as hardware. For example, the last several years saw a lot of 32-bit apps and servers become 64-bit instead, with subtle changes in what one can do given a bigger address space (and maybe haphazard system support). We're not going to 128-bit in the next several years, but something else in hardware will change.

pending work

I agree with you on the testing issues.... "In theory there is no difference between theory and practice, in practice there is".

Getting the to the example though For layers to work each layer has to make sure the next layer has work to do. Lazy or eager not keeping sufficient work at all layers is bad design.

A better hard problem example would be one where layers don't disclose what they're being lazy about.

Which is what I mean. I don't see how laziness vs. eager makes this any worse. An eager function that doesn't say what it is going to need next is going to suffer from high latency and worse is going to have a harder time keeping the lower layers flooded.

(should have given up already)

Just out of curiosity, are you arguing eager and lazy algorithms always have the same effect, so there's no reason to distinguish? I suspect you don't mean that, but I have trouble pulling it out of your subtext. If I define eager as "try to do it sooner" and lazy as "try to do it later", then an eager algorithm submits work to be done early while a lazy approach only submits it later -- in some cases as late as feasible. If that's what makes load known and fills parallel pipes, then eager does a better job of that.

I strongly disagree that verb "test" is a synonym with verb "experiment", so your summary "testing issues" makes me wince. I don't want to fight about it though, unless you do. :-) I have noticed devs sometimes prefer to struggle along in ignorance rather than allow themselves to get testing cooties, out of fear of lower social status, which can lead to broken systems. Being effective means getting your hands dirty a lot.

lazy vs. eager

I see where we are disagreeing now. I'd define more this way:

eager code = order of evaluation is intrinsic to the code. Unless specified otherwise all values are computed fully by first use.

lazy languages = most code just defines a chain of dependencies, how to resolve the chain is left up to the system.

So parallelism in lazy languages works like connection pooling in database. That is lifting lazy code into an evaluation monad which litters the code with "sparks" that threads can just grab sparks and resolve the vast bulk of the language easily.

In the case of layered situation or any other kind of constraint you simply change the evaluation strategy in the evaluation monad. I don't see how having code that is flexible about reordering can ever present complexities for parallelism over code that is inflexible about reordering. I can easily see how for example memory usage can become much much worse but I can't see how CPU can be assuming the programmers aren't just being stupid.

___

As for testing, yeah I meant testing as "experiment". Testing in the QA sense I don't see as particularly useful for making these sorts of judgements. Either the performance is hitting Amdahl's law targets, or the algorithm is wrong, or the parallelism model is. I don't see a need for QA trickery helping for the stuff we are talking about. Once you are hitting those targets and want to make sure nothing can knock you off them then QA trickery is to be desired.

I don't think that's how

I don't think that's how most people use lazy and eager. Lazy means 'as needed'. Eager means 'as soon as possible'. Order of evaluation is a separate issue. You could express something that has no particular order and the compiler might still implement it eagerly.

compiler

I agree the compiler in a lazy language is entitled to execute any code it wants eagerly. That's what happens in Haskell, which is the archtype for a lazy language. Similarly I think "as soon as possible" is something of a misnomer. In eager languages many computations are possible at any given time. Which ones get done when is the programmer's call in an eager language.

So I guess we do disagree I see the two as very related. But I do appear to be in the minority on the definition of lazy. What are lazy languages that guarantee all computation are only done when needed?

"Lazy" usually denotes the

"Lazy" usually denotes the call-by-need evaluation strategy, whose implementation involve allocating thunks for delayed computations and, at access time, either checking if the thunk has already been forced and returning its result, or forcing it and mutating it (atomically) to put the evaluated result.

This is an implementation that is inherently based on side-effects, and if you want to avoid duplication of computation (as it is specified in the semantic), in a concurrent setting you would need to synchronize on thunk update. This make it a relatively bad evaluation model for parallelism.

Haskell, for example, emphasizes a *non-strict* evaluation strategy, meaning that they don't commit to call-by-need and reserve the right to duplicate computations when it helps on other aspects. In practice the implementations are call-by-need, with optimizations to call-by-value when they can prove it won't introduce non-termination. The future vagueness is fine in Haskell as its a pure language (you couldn't observe duplicated computations), but would be problematic as soon as you add implicit side effects.

duplication

Agree on thunk. This strategy implies that the author of the code can't determine when his functions are invoked or, excluding dependencies, in which order. And that's where the advantages lie.

Agree on Haskell. As far as side effect, let me just point to another example that is widely used. Excel is lazy, cells are recomputed when the Excel engine determines a cell needs to be and in whatever order the engine wants to recompute. There are no guarantees that duplication will be avoided. OLE embedded Excel is a side effect (it is essentially a monad on the spreadsheet). The policy on OLE is that duplication may occur, and is perfectly legitimate. The invoking application is entitled getting a fully resolved current state when they make the call. The invoking application is not entitled to any promises of state being maintained. So for example if Word documents A and B both make on OLE call against spreadsheet C, A can induce side effects which impact B's next read, and B can be unaware of this. And this setup is something that even fairly low skill end users seem to be comfortable with and understand.

I would agree thought that a lazy system that does a strict call by need with a promise of never duplicating side effects would create a whole other layer of issues about parallelism. If that was the point OK, but I'm hard pressed to even come up with an example of such a system.

While thunk synchronization

While thunk synchronization is another concern, I actually meant something else.

When you try to optimize a program by making it parallel it is important that you subdivide it into parts that execute in parallel, and these parts can't be too small (then the overhead of executing in parallel it too great) and these parts can't be too large (not enough parallelism). Therefore it is important for the programmer to be able to predict and control how much time a certain piece takes.

It is generally accepted that mixing implicit effects (e.g. mutable state) with lazy evaluation is a bad idea because it becomes very hard to predict when effects happen. The time it takes to complete a computation is an implicit effect. In most cases that doesn't matter because we don't care when a computation happens, but for parallel programming we do care. Laziness makes predicting and controlling how much time the subdivided computation takes harder.

In parallel Haskell code you often have to explicitly control evaluation order, and there are even various tools for this. A common situation is that you tried to compute something in parallel, but what actually happened is that you only computed a thunk, and the real computation will happen serially when somebody accesses that thunk. In practice it seems that the easiest way to write parallel Haskell code is to use deepseq to essentially make the whole computation strict. When you make essential use of laziness you have to be more careful and write a custom evaluation strategy. This is one of the reasons why I think it's better to make strict the default with laziness when you need it, rather than the other way around.

dividing into parts

I see. OK that is a more serious problem. I agree that using deepseq makes the code eager and if you have all over the place laziness is just an annoyance. Either the machine is picking the order of execution or the programmer is. If the programmer is then it makes sense to use eager. I agree with you completely if the code breaks naturally into parts, then deepseq works fine, laziness doesn't buy you much and may be harmful.

But I don't think that is often the case. Rather what I think is much more common is that the code has lots and lots of places to fork over meaningful work. Generally this works by no explicit breaks but pooling. So say I can break a computation into 10,000 unequal but reasonably large parts and spread them across 25 execution threads. This was the case I think is more common because it allows for all sorts of complex dependency resolutions at runtime. Especially those that are dependent on external factors I can't time, like getting data from external sources; or when I'm not entirely sure if I have 10, 20, or 100 execution threads. And here laziness helps a ton. And that's because the author didn't write the program assuming order of execution, I'm going to be able to find those many thousands of large parts all throughout the code and easily and safely mark those.

. When you make essential use of laziness you have to be more careful and write a custom evaluation strategy.

I agree with you here. I'm talking about casual use of laziness. Essential use of laziness in lots of parts of the code makes things very complex. But in general, as I mentioned above I don't know of too many examples of lazy code that makes essential use; other than things like moving around infinite data structures as streams.

And here laziness helps a

And here laziness helps a ton. And that's because the author didn't write the program assuming order of execution, I'm going to be able to find those many thousands of large parts all throughout the code and easily and safely mark those.

I think you mean pure rather than lazy. In fact if your language guarantees a lazy evaluation order you can't just execute it in another order because it might cause nontermination. A strict evaluation order gives you strictly more flexibility, since any code that works under strict evaluation also works under lazy evaluation.

either I'm being dumb or

Yes, I'm talking about the advantages of pure. I get that in theory laziness has nothing to do with purity but in practice they go together. Evaluation order is really really import for side effects. So you can't have non-obvious ordering of side effects too much, which I think prevents there being much impurity mixed with lazy. Or to take the contrapositive: lazy implies mostly pure.

As I'm talking through this issue I'm realizing that there is a definite difference between
lazy algorithms which don't do anything as far as parallelism that's helpful
as opposed to lazy languages which create an entire ecosystems and that ecosystem tends to lead towards almost full β-reduction capable programs

... As I'm talking through

...
As I'm talking through this issue I'm realizing that there is a definite difference between
lazy algorithms which don't do anything as far as parallelism that's helpful
as opposed to lazy languages
...

This is pretty much it. The confusion in this thread arises because (roughly speaking) your 'lazy algorithm' is another person's 'lazy language', and your 'lazy language' is 'does not force more or less ordering than is inherent in a problem'.

I agree that a language that does not force over-specification of things (such as ordering) will have more flexibility in implementation. I think that this is another form of accidental vs essential complexity.

agreement

Good nice to see we may be coming to consensus on the matters of fact and just a minor issue on the matter of language. And I can see the other side's position on language pretty clearly.

When you try to optimize a

When you try to optimize a program by making it parallel it is important that you subdivide it into parts that execute in parallel, and these parts can't be too small (then the overhead of executing in parallel it too great) and these parts can't be too large (not enough parallelism). Therefore it is important for the programmer to be able to predict and control how much time a certain piece takes.

This seems like something a computer ought to be able to do for us to a certain extent. I know there's been some work on this in nested data parallelism, but I'm thinking of a more type-driven analysis based on sized types.

For instance, a function which operates on a recursive data structure would optimally be split into multiple tasks as long as the data structure is above a certain size. The size of the data structure is in the type with a sized type system. Where the size is static, the decision is made at compile-time. Where the size is dynamic, it's translated into runtime evidence that the language runtime would use to decide whether to fork a parallel computation. I haven't seen any papers on something like this though.

There is something called

There is something called auto tuning. What you do is you write your algorithm like this:

parameter int n

let foo x = 
  if size x < n then ...serial base case...
  else let x1,x2 = split x
       let y1,y2 = par (foo x1) (foo x2)
       combine y1 y2

And then an external process searches for the best n. Note that often you don't want to follow the same splitting recursion for the serial base case, since there is often a more efficient way. And you might want to split differently at multiple different levels to optimize for cache line size, and L1, L2, L3 cache size. I doubt that this can be fully automated, but it would be interesting to be able to specify a spectrum of splitting patterns and the compiler searches for the best combination. There is already a lot of work on this for specialized domains like FFT and matrix multiply.

Cache-oblivious algorithms

Cache-oblivious algorithms may have something to say here.

Autuning generally does

Autuning generally does better. The cost is deploy ability.

Yup, it's really hard to

Yup, it's really hard to manually create an algorithm that does well for all cache sizes and architectures. Caches are really complicated, for example see this: Why is transposing a matrix of 512x512 much slower than transposing a matrix of 513x513?. The answer is basically that the cache is like a hash table with a really bad hash function. If you're unlucky that can lead to lots of hash collisions and reduce the effective cache size by a huge factor. And auto-tuning by its empirical nature of course optimizes all kinds of things, not just cache behavior. Better tools can probably solve the deploy-ability problem. JITs are quite pervasive and have pretty much the same deploy-ability problems.

99 problems

I've noticed three big issues:

1. Specifying many variants. To handle many architectures, many algorithms and data reps are used. Coding hits an engineering barrier.

2. Tuning time. Now multiply by discrete settings. Machine learning algs are starting to be used, but the test space still takes awhile.

3. Runtime setting. Outside of HPC, we should assume co-resident kernels; not all mem, CPU, etc. is available. You don't want just one binary specialized for your arch.

Why is transposing a matrix

Why is transposing a matrix of 512x512 much slower than transposing a matrix of 513x513?

That's one of the many cases where the simple recursive (cache-oblivious) algorithm with specialised leaf routines works very well. There's even some research on generating such leaf routines from naïve recursive formulations. It's actually not that much work to reorder a trace and linearise access patterns: it was my term project in a graduate compiler course. Disabling the loop-recognition pass (and converting the few register-register movsd to movaps) lead to output on par with GCC on loopy code (with small known iteration count) at O3, in 2008.

apples to oranges

... but did you beat the perf of an autotuner?

Far from it. But the coding

Far from it. But the coding effort to get a specialised kernel was much more reasonable ;) It also seems interesting to generate autotuners from recursive formulations. In fact, it would probably be easier than trying to determine a very good synthetic model and then reorder things according to that model.

Stencil autotuning languages

Stencil autotuning languages seem just like that, with the extra hint of the type of recurrence, right?

no need for papers

As far as I understand this request it already exists mostly except you have to have a parallel strategy (parMap). GCD by Apple also makes something like this very possible, and with just very small changes you get parallel strategies.

lazy != non-strict

Non-strict semantics is good for parallelism, but laziness in principle forces everything to be evaluated at the last moment possible, which makes the program miss a lot of oportunities for parallel evaluation of expressions (e.g., you can't just start the evaluation of every argument for a function in parallel, because one of the arguments may turn out to be an infinite data structure that will start eating up all the memory, so you must wait until parts of the structure are needed to compute them, but that means you can't take advantage of parallelism). I guess a combination of non-strict semantics and eager evaluation (so-called "lenient evaluation"), as in Id and pH, can make most of parallelism.

lenient != non-strict

Under the definitions I'm accustomed to, laziness is a memoizing strategy for non-strict evaluation order. Lenient evaluation is another reduction order entirely. It results in reasonably parallelizable code for some common cases, but I find it to be a pretty big hack. A better approach, IMO, is to separate evaluation order from semantics.

For parallel evaluation,

For parallel evaluation, lazy vs. lenient vs. whatever is a very tiny piece of the picture. But we've had that discussion several times now..

lazy = non-strict (lenient)

I don't think laziness should require compute last just compute in any order the system wants. The infinite data structure should start sucking up large amounts of memory, but not all of it because the evaluator finds other parts of the thread that can go first. Memory usage should be part of how the system chooses to pick which threads to focus on.

Laziness doesn't play well

The problem I see with controlling evaluation strategies (at least lazy vs. eager) is when/how the programmer expresses this - i.e., is it at declaration time, at funcall time, at de-referencing time? The problem with declaring laziness early is that the referential transparency of treating lazy and eager values the same is both a boon (don't need to change change referring code) and a PITA (if you don't know that a value is lazy at some point in your code, you could get unexpected complexities). Explicit laziness (like Lazy(T) ) is nice because it's explicit and can be implemented in any language, but forces a getValue fn/monad to be used - but then code gets littered w/extra cruft. I think tooling and visualization of explicit and inferred evaluation strategies and some notion of meta-strategy based on usage domains would probably help, and even the JIT running various parts of the code w/different evaluation strategies and re-coding as necessary.

I would hope that we are

I would hope that we are already working on IDE and library design issues. I don't think FRP will catch on, but maybe something similar to it (like immediate mode UIs). I'm also not so sure about modules either, but my thoughts are much more controversial here (modules are bad, long live the one big code wiki!).

Your prediction about CPU/GPU merging is interesting. I'm going to counter-predict that we will better understand CPU/GPU workloads by 2020, and will have optimized each more for their loads, but we will have failed to unify them. Maybe by 2030 "data threads" will exist beside "control threads," hmm...

I agree that it won't be

I agree that it won't be exactly FRP (which is why I added the caveat that it may not be in its current form). For example the continuous aspects of FRP will most likely be thrown out, and some impure updates added back in. You can already clearly see this trend in modern JS frameworks like Backbone, Angular, Knockout, Meteor, etc.

On modules, you can see dependency injection being quite popular in the Java world (and to a lesser extent in C#). It is basically a big workaround for the lack of proper modules. Scala has quite nice language support here. Perhaps something less structured like a big code wiki will win, but won't that make programming against a declared interface and linking components via their interfaces even more important? A literally global big ball of mud wiki where all internal details of everything are exposed doesn't seem very nice but perhaps you're thinking of solving that in some other way perhaps with tool support?

Yes, GPU/CPU will probably be popular by 2020 but the merging is likely in the long term. We already have technologies that can take advantage of both small and large scale parallelism, like FPGAs and stream processors, so we know that it's possible. I think that technology trends will make it profitable. As chip densities increase, CPUs will hit a wall because 2x transistors will only give you a couple of percent extra serial performance due to better designs (this has already happened). GPUs will hit a wall because if you just use those transistors to put more and more GPU cores on a chip, you'll run out of large scale parallelism in the problems you want to solve (this has already happened for some problems that do not have much parallelism in them, and will happen for more and more problems as GPUs get bigger).

Of course this could easily be hogwash, and this goes for all of those predictions. Although there is some reasoning behind each of them, it's so easy to overlook a crucial piece of information that will only become obvious as we actually go into the future.

I actually think the

I actually think the continuous bits of FRP will survive in the form of freaky evaluation models and/or databinding. Its the event composition parts that will fall away and become less relevant (like how Rx doesn't really solve UI programming problems).

I don't see dependency injection has related to modules, that is more AOP. My continuing point about modules is that search is just so much better; with search modules become as irrelevant as web hosts (they still exist, people just stop caring about them as concerns). Encapsulation is still a good idea, but that is encapsulated to the user of the module.

I don't think we've really scratched the surface of data parallelism yet to scratch the surface about where we are going with CPU/GPUs. The fact that we (the PL community) got this wrong and the systems and graphics communities got this right should serve as a warning that we might not know what we are talking about.

Agreed that the event

Agreed that the event composition part of FRP may go away and be replaced with imperative event handlers. With continuous bits I meant stuff like time integrals of real valued functions, not the data binding (which is very important).

Dependency injection AFAIK works like this. You have a component Foo that uses component Bar, and now instead of hard coding that reference to Bar in Foo, you let Foo take something with interface IBar as a parameter and pass in Bar at run time. This way you can also pass in a different IBar. Dependency injection frameworks attempt to keep the code somewhat reasonable in a language like Java and try to avoid passing everything around manually and instead you write a recipe for linking the components together. A module system or a powerful object system as in Scala is a much better solution for separating a program into interfaces and components, and linking those together.

I don't see how search is a replacement for interfaces and modules but this may just stem from different terminology since you say you still want encapsulation. Perhaps you mean search as a replacement for (nested) namespaces?

Hardware that can exploit both small and large scale parallelism seems inevitable at least in the long term, but yeah it could easily be wrong due to unknown unknowns.

About IDEs and libraries: perhaps I'm just not aware of it. Is there academic work on general purpose languages that explicitly considers the design of libraries and IDE together with the language?

Ah, yes, the pure continuous

Ah, yes, the pure continuous stuff in FRP is interesting but not very pragmatic. I was just referring to continuous timeless abstractions (like continuous binding). It took me a while to sort this out from the rest of the FRP work.

Module is such an overloaded term, its no wonder we aren't talking on the same page :). I see what you are saying now and I don't disagree. Modules to me represent units of linking, encapsulation, deployment, management, namespace management, etc... I just don't see a need for them to play active roles there, that somehow all the code should just live in a giant (inter) net where they are forced to be compatible.

Smalltalk: libraries and environment all came up together and it shows. Self also has Morphic and its (limited) dev environment (Maloney's UIST 95 paper is very good in this regards). I'm sure we could dig around from all the Smalltalk-related papers in the early/mid 90s and find something.

A more modern example would be Racket, though the term "language design" is sort of weird there (since libraries are often languages). Nothing extremely aggressive yet.

What?

I actually think the continuous bits of FRP will survive in the form of freaky evaluation models and/or databinding.

What is a freaky evalution model, and will I be sorry I even asked...

Before we judge FRP, can we at least set a criteria FRP implementations must achieve before being worthy of being judged:

  • Handling non-rectilinear layouts
  • Constructive area geometry for shape compositing
  • Vector-level clipping (a key point of FRP is to make reasoning about UI effects easier, but that becomes trickier if clipping takes place as part of rasterization)
  • Vector-representation of text in a particular font.
  • Handling dynamic collections (e.g. opening/closing windows, adding/removing elements from a DOM, etc.)

...and at the same time, not shaming research prototypes for failing to deliver on all these needs. For example, until mid 2000s, Cairo was the only major open source, cross-platform rasterizer available, and it basically only supported rasterization. Now, it supports some more things, but as its feature set has grown, it has become harder to think of pleasant ways for researchers to map their FRP interfaces to low-level interfaces. So you kind of need to think about what is needed to make FRP work-able on a large scale before criticizing it. And it is still really valuable in some scenarios. For example, for fun, I am writing a non-Euclidean Geometry drawing application, where you can draw lines and points on a sphere, etc. It's much easier without having to deal with state.

As per Cairo, the same is

As per Cairo, the same is becoming true with FRP and GPUs. Conal's work on Vertigo, for example, makes sense only at the point of pixel and vertex shading, but everything becomes sketchy past that, and with GPGPU, imperative programming is the norm again.

Hi Sean

Take a look at my latest draft on FRP.

I found that with the right, fairly unobtrusive, type system, it's possible to write FRP programs with a simple, predictable cost/evaluation model (call-by-value, with call-by-need only for dataflow cells). The really nice thing is that typing eliminates the need for explicit dependency tracking, which means that one of the most performance-hostile parts of dataflow programming (its bad interactions with GC) goes away.

Value types?

What is academically interesting about "value types"?

Academically it is a minor

In itself it is a minor thing not a major research area, but practically important for making other academic achievements count more in areas such as generic programming, dependent types, better GCs and parallelism. In combination with dependent types they allow you to express data structures that normally require unsafe C/C++. For example a variable length array is a dependent pair Array a = (n : Int, Vec n a) where Vec is the usual inductive type but with each pair represented as a flat pair instead of a pair of pointers. As far as I know, data representation is the major thing that keeps the efficiency of e.g. idiomatic OCaml or Haskell away from C and Fortran. Combined with staging on type class dictionaries this can make all those wonderful abstractions and generic techniques perform without overhead. You can take advantage of techniques that are not practical in other languages, e.g. generic radix sort beating C++'s built-in comparison sort, and problem specific matrix representations beating Fortran matrix representation for the common case where you have a semi sparse symmetric matrix. You could probably exploit that in Fortran for a >2x performance gain if you were willing to re-implement all matrix operations for that representation. Another example is automatic differentiation, which really sucks performance wise unless your values are unboxed. Less memory allocation and fewer pointers in the heap lets GC work shine. General efficiency lets the work on parallelism get absolute speedups instead of just relative to serial, and less memory traffic lets it scale better.

It could certainly be that all of this already exists somewhere; I didn't think of it when I wrote that but ATS perhaps? It hasn't gone "mainstream" in academia in any case.

You can take advantage of

You can take advantage of techniques that are not practical in other languages, e.g. generic radix sort beating C++'s built-in comparison sort,

Are you willing to explain better? Maybe there are ways to do that, even without ATS language.

To do radix sort on data

To do radix sort on data other than character strings you need datatype generic programming. That means that you do something different depending on the data type.

See http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.154.2018 and Generalizing Generalized Tries.

The thing is that this is supposed to improve performance because it's asymptotically better, but when you try to sort a list of (Maybe Bool, Either Bool Char) and you represent the pair as a pair of pointers, the Maybe and Either with pointers, you already lost big time to naive C++ simply because following all those pointers is so much slower than using value types. When you write a radix sort for a particular data type in C++ it *is* faster than comparison sort (e.g. for integer arrays it is >2x faster).

"Value types" seems to be a

"Value types" seems to be a particular mechanism or methodology for representation control.

Types are just a tool to me, but the underlying issue of programming over multiple representations is fascinating. For example, when designing 'clustered data parallel' algorithms, I found it profitable to switch the data representation for a tree (a webpage) based on the access pattern (what stage of processing it is in). Today, systems like NESL and Data Parallel Haskell assume you use one baked in representation, and that's wrong. There is a lot of space in designing constructs for easily, correctly, and effectively (in terms of performance) to do these conversions.

As for academic interest, I'd argue that the last two PLDI best paper awards were about this fundamental issue of language support for controlling data representation. Their domain of essentially implementing SQL is better trod in the DB literature, but designing constructs for controlling representation is the interesting part.

Some things reasonable

Some things reasonable people may disagree on. But I am certain emacs will still be the tool of choice for the discerning hacker.

I hope not

Language and IDEs should be tightly related, like what we are seeing happening with LightTable. I believe this merging could be one of the significant advances that we will start to realize by 2020, if not very popular already.

Emacs is already such an environment for LISP, but support for other languages seems tacked on at best (code completion through ctags!). In a language where great code completion is possible, using Emacs feels like turning off your targeting computer and using the force.

Best list so far

Great list of programming language issues that look imminent. In theory, these should be the most clear of anything on an 8 year horizon. The things everyone will be doing in 8 years, are the things that look good in some toy language right now.

I mostly agree with your list but would name a few exceptions.

I really don't think automatic client/server program splitting will go anywhere; I'd move it to your list of things that will dwindle. Not all abstraction is a good thing, and data sent between client and server is a bad thing to abstract. For example, you really don't want the optimizer to send over the password map to the client for efficiency purposes.

On IDEs and library design, I think the existing practical systems are way better than most of what is being done by academics. IDEs require an army to make a good one. Library design requires a kind of taste that I don't see possible to get without actually working on a large program, and thus not being a theoretician any longer.

Something called value types are already present in C#.

I'm surprised about the assertion against text files. They seem good in every way except the number of bytes, and the number of bytes is usually not that important for interchange issues. In computer software, usually complexity is a bigger problem than machine resources.

On testing versus types, I suspect scripting languages are here forever for writing scripts. However, I suspect they will dwindle as a way to write anything large enough to be worth regression testing.

Its been awhile Lex, hope

Its been awhile Lex, hope all is well.

It doesn't take an army to write a decent development environment, just people with skills that are not typically found in PL implementors. But its integrating post priori with a legacy ecosystem that really kills things. E.g. the visual studio core team is not really that large, but all the infrastructure and features around it require a lot of resources.

There are some essential principles to some library designs that we are just getting a handle on right now. Integrating those principles into our language design process can be very useful, but ya, we are still only going to be able to address domain N+1 in an ad hoc post priori manner.

The problem with text files is not efficiency, but addressibility and meta-data. In many ways, Smalltalk was just a bit early in its ideas, but I think we'll start moving to code databases soon enough, along with projected code editors that are able to present/edit richer code formats (e.g., including GUIDs next to names to track how names are resolved historically).

I really don't think

I really don't think automatic client/server program splitting will go anywhere;

I predicted it would go nowhere back when we had the discussion on LtU a few years back, and Joel even chimed in to give his perspective on GWT's code splitting, and I pushed back saying he was wrong. Back then, I was actively e-mailing researchers of various projects, letting them know my perspective and why their approach was doomed to fail, and what they should do about it.

That said, I think the approach I outlined back then is as good now as it was then, and better than automatic client/server programing splitting. The problem is academics didn't understand common fallacies in mobile, ad-hoc networks, and what kinds of APIs should be designed for them. So academics gave us really bad tools, and they are still not doing anything about it.

There are also some things I would like to be able to express more cleanly.

For example, data binding is clumsy today. Here is a simple example.

I have an (int) and want to specify a refinement type that says, Integer is in 1 .. 10. In most data binding APIs today, there is no clean way to utilize blame to guide control flow. I believe fixing this problem once and for all will lead to much cleaner APIs. We also have reasonable theories of refinement types and contracts, thanks to Contracts Made Manifest by Pierce et al. Using blame for control flow can only be truly possible if the blame can be an exception.

I actually don't even like the idea of directly tying blame to type casts or dynamic contract checking, but it's so much closer to what I really want in order to remove redundant logic. For example, inputting data into a text box is best done by first making everything a string. Then there are two different kinds of ways to alter control flow possibilities.

  1. Block/filter out input, e.g. given a string, if non-integer characters are input, don't accept them.
  2. Accept input, and ask the type system if the string can be cast to an int. This is actually kind of tricky from a usability perspective, because what do you do when the user types "-".

Accept input, and ask the

Accept input, and ask the type system if the string can be cast to an int. This is actually kind of tricky from a usability perspective, because what do you do when the user types "-".

That's an interesting problem, calling for interleaving parsing/validation with dataflow. Naively linking reactive data binding to a parser for integers would yield errors while typing in "-10". You could introduce a timeout after which the parse is attempted, but that seems crude. Seems the simplest answer would be to discriminate on whether a parse was partially successful, and if the parse ended because of the end of input, then don't display an error.

Basically, make parsing incremental where a partial parse "error" simply returns "need more input".

Reactive Error Models

Yielding errors during parse seems quite acceptable in a reactive data-binding model. E.g. you could display near the textbox a red mark while invalid or a green mark while valid. The main constraint is that, in a reactive language, error models should be equally reactive - no exceptions or control flow that bypasses the model.

Monads will evolve, not disappear

I think we're going to eventually settle on effects/effect-handlers and formalized effect systems. Not sure how first-class they'll be (via explicit instances, as in Scala if at all), but given that instances being second-class language citizens hasn't hurt Haskell, I think we'll find a good way to combine effect handlers and static effect checking.

The equivalences with Haskell-style monads already exist, but the syntax and intuition will be a lot easier for the common programmer. We already successfully teach people how to write exception handlers, after all.

I think Koka is a good

I think Koka is a good language to look at in this regard.

For me, problably not alot will change.

I've got a fairly accurate picture of what programming currently looks like for me. This year I've written

Prolog~15000 LOC
C~15000 LOC
Scheme~5000 LOC
Assembly~500 LOC

(The numbers are very high due to me working solo on a single project. This is not my typical work rate!)

Back in the early 1990s I was writing code in Prolog, C, Assembly, Scheme, and SML. I don't have the numbers but I reckon it was:

Prolog~30%
C~30%
SML~30%
Scheme~9%
Assembly~1%

All I've done in the last 20 years is simplify things by dropping a language. My current Prolog and Scheme environments were written by myself so it'll take a paradigm shift in processor architecture for me not be able to port them and continue business as usual.

Boring eh?

Watt based efficiency

Our low level languages came from a world of counting clock cycles for execution. As mobility becomes more important our low level languages are going to make the wattage needed for execution explicit. Right now there aren't easy ways to use execution strategies based on how much power its going to take to execute various algorithms. We are starting to see this in hardware microcode. My 2020 prediction is it crosses over into the next low level language that replaces assembly and maybe even C.

For higher level languages, I'm going with functional for parallelism. Everything else is driven by being able to split tasks among more cores. Once you move to functional, the organization of the data objects is going to move away from object hierarchies towards data intrinsic hierarchies that intermix better with the functional / relational paradigm. So relational database objects will be first class objects in the next mainstream programming language. In other words the next mainstream language is going to be a cross between Haskell an PL/SQL.

This leads to parallel applications with a heavy synchronous engine and a mostly asynchronous interface code layer. I don't see any reason today's OO languages don't do well for the interface layer so once they are only responsible for passing and receiving messages from the engine I suspect they remain more or less the same.

Kindred spirit

Jeff,

There was a paper by Philip Stanley-Marbell about this 2 years ago. http://doppelganger.gemusehaken.org:8180/thepub/abstract.jsp?id=50

The title and abstract are very misleading, as Philip's real goal is to have a different kind of assembly language for compute clusters where the computation might be slower but cheapeer per watt to execute.

makes sense

Sorry I missed this comment earlier. Yes, something like that. I don't have anything intelligent to say on whether that's the right approach or not, sorry for not being able to hold up my end.. But I'm glad to see some work on it, because some assembly that allows one to count watts needs to exist.

Challenges with the approach

It's unclear how well the approach outlined in the paper I linked would hold up to various non-linear, convex optimization problems. I don't even know where to start to understand such difficult program reasoning challenges.

Power will be the dominant problem

In 2020, Intel will produce the first massively parallel chips where the whole processor cannot be powered at a single time.

new language from apple

Hopefully apple will have introduced a new language and tools which will better the lives of thousands of despairing ios developer.. I work in the industry and the demand for ios developers is very high. I've considered becoming fluent in the platform myself but considering the headaches my coworkers are suffering through on a daily basis. I put my health and sanity before my career..

Previous Apple languages:

Previous Apple languages: NewtonScript, Dylan, SK8, Hypercard, Applescript, Quartz Composer, Fabrik (I'm sure I've forgotten something).

I'm not sure if Apple is going to get back into the language game again, it didn't seem to help them much out before.

depends on the competition

I think they will have too if the windows platform is hugely succesfull in the mobile / tablet space and if the momentum can be partially credited to developer productivity. This increased productivity must of course be acknowledged by industry in order to put some pressure on apple leading to an increased focus on tooling and languages.

Microsoft's (my employer)

Microsoft's (my employer) and Google's language talent is really a huge differentiator compared to Apple. However, developer productivity rarely seems to move market needles; not many iOS developers loved or love Objective C! For language to become a factor, we really need to squeeze out a magnitude increase in productivity, which is a tough goal that no one has really achieved (at best, we can say our FP languages are 25-40% better, and even these claims are dubious).

Competitive avantage

Almost nobody claims their used language is a competitive advantage. One exception is Eiffel and their customers. Or actually the customers seem to be quiet.

Jane Street

Actually, Jane Street do, too.

So does Adacore

Robert Dewar was always emphatic about that year after year sponsoring the ACMGNYR ICPC contests, at least when I was there a few years ago.

C++ to C#

> need to squeeze out a magnitude increase in productivity, which is a tough goal that no one has really achieved

As primarily a Windows programmer switching from C++ to C# as my primary development language has given me easily a 3x-5x boost to my productivity.

I doubt it was that much,

I doubt it was that much, maybe 2x? The problem is that once C++ overhead is eliminated, there is still the fundamental task of programming and debugging that remains constant. We also tend to magnify observed productivity gains because they are so satisfying, and going back to the "slower" language is really hard (those C# programmers will never want to go back to C++).

I stand by it.

> I doubt it was that much, maybe 2x?

I stand by 3x-5x. Not immediately of course, but once I became experienced in C# definitely.

I also need to make the caveat that its not just the language in isolation, but the tools and libraries that are available for the language. I believe the sophistication of tools and libraries are a direct result of the language design.

> There is still the fundamental task of programming and debugging that remains constant.

That's not true. Debugging C# programs is much easier (very little chance of undefined behavior, entire classes of errors are virtually impossible) and many programming patterns are much faster to implement (hint: anything with reflection).

I totally believe that you

I totally believe that you believe this, because I feel the same way. I just wish we could have some objective/formal experiment to prove this, and I suspect the benefit is probably not as much as we feel it is, but I totally accept that I could be wrong.

Reified generic types is definitely a big win for C#. I write all my code in C# these days.

Edit and Continue

I do most of my winders programming C#. Just the ability to edit code on a running program is worth 2X by itself. Which bring to mind there is a low of low hanging fruit left in just tool design and interface.

For instance, I remember back in the bad old days assembly programmers patching their programs while debugging, in C you can't do it. Not because it's impossible but because the compiler, linker, and debuggers weren't designed to support it. I wish I had that in C.

And yet, edit and continue

And yet, edit and continue for C# is not as robust as edit and continue for Java. I've barely been able to use edit and continue in C# because of my addiction to lambdas, where VS is just too conservative about that.

Also, I think real live programming will blow edit and continue away as the must have feature by 2020.

I bet against that

I'm willing to bet against this and predict that, perhaps with the exception of web interfaces, neither live editing nor mudball-style programming environments in general will become relevant for the vast majority of developers. Nor should they.

worse will still be better

No idea what you mean by web interfaces, maybe you mean interactive programs? If I can demonstrate and prove this concept for say...compiler writing, would you be more willing to believe?

Programmers will not continue to carefully design, plan, and validate their programs before they write one line of code because...they don't even do that today. The style that most programmers follow will continue to be somewhat less than ideal and benefit from a "heads up display" to more quickly understand where they are actually going vs. where they thought they were going.

dumbfoounded

Hypercard was incredibly successful and popular. I'm hard pressed to see that as a failure.

Quartz composer's graphics language is used all over the place with Cocoa. I'd say it is the core reason that Apple was able to get the animations to work in 2007 on the hardware they had.

Apple mismanaged Hypercard

Apple mismanaged Hypercard royally.

Quartz Composer has only been successful in the art crowd, I haven't seen any products that actually contain QC code. As far as I can tell, its already deprecated.

Hypercard

My point was it wasn't a failure. I'm of two minds on whether Apple could have saved Hypercard. On the one hand I look at Powerpoint, the Access / Foxpro / Filemaker, Visual Basic and then the web as having eaten their market niche. On the other hand there is still I believe a huge niche still left untapped for a language easy enough for even non programmers to write good simple applications in that is widely used. Apple had it and who knows what could have been if they hadn't tried to monazite it.

BTW a modern Hypercard does still exist LiveCode. I've heard great things about it, and Apple has been supportive of LiveCode, but supportative is far short of buying the company and bundling it with every OSX system sold. A more amateur HyperCard which is free is PythonCard.

Granted. My point is that

Granted. My point is that Apple hasn't been good historically in marketing and pushing their own PLs. I think at this point they might be a bit apprehensive about their experience, but who knows, they might wade back into the PL innovation market.

apple has never been a languages company

Apple view itself primarily as a hardware company and so sees its competition as Dell and Samsung while preferring to partner when possible with Microsoft and Adobe. Because their stock and trade has been GUI design their products have generally leaned away from text based scriptability. Moreover Steve Jobs has generally leaned towards a focus on professional development because he wants the software to be of high quality so Apple languages have had features but they've had features aimed squarely at the professional crowd.

So if you are saying they don't do it much, I'd agree. I don't agree they aren't successful at marketing though. When they have I don't see them being particularly bad at it. Hypercard was successfully marketed. People were unhappy with Apple's handling of Hypercard because it was immensely popular and they let it die, not because it wasn't popular and unsuccessfully marketed.

AppleScript is as successful as say Powershell and possibly more so. But again Apple doesn't see strategic advantage here so they are letting Applescript wither. I wish they didn't but I can't say they are wrong that AppleScript doesn't sell computers.

Objective-C is also an obvious counter example to the "Apple doesn't market their own languages". That language has become hugely successful due entirely to NeXT/Apple. MacApp before that. On the Apple 2 6502 Assembly and its ties to Apple Basic if you want to go back further were all successful. Dylan was a failure but Newton was a failure.

MacRuby, as I mentioned above is their big innovative language. They are trying to tie a classic IDE / development framework to a dynamic language with both OO and functional features. I'd consider that language innovation. If they can make MacRuby successful while at the same time Objective-C is still viable then they will have successfully migrated.

As an aside, I know you are a big smalltalk fan btw and F-Script which uses the MacRuby components is potentially a rather good Smalltalk with full access to Cocoa.

I think it is fair to say that in comparison to say: IBM, Microsoft, Adobe... Apple has been mostly disinterested in programming languages. I don't think it is fair to accuse them of marketing failures.

Apple is focused on making money

Apple has done a lot of great language work in the past, but they almost lost it in the late 90s, laid off all their researchers, and decided to razor focus on products and where they could add value/make money. So now we have an extremely focused, efficient company, why would they bother to get involved in language design when the pay off is so abstract? Its as much as they can do to make Objective C a bit better with reference counting and focus on making their tool set as good as they can.

The problem is, language design is not really that profitable for a company (Sun, Apple, Microsoft, Google, ...). The benefits are abstract and actually raise everyone together, it really is charity work for the industry vs. some special advantage. Apple is just too focused right now, they would have to accept being less efficient to do this kind of work.

I like F-script BTW, I totally enjoyed the OOPAL paper presented at OOPSLA a few years back. Its interesting to see Apple get involved with MacRuby, but it seems to be more of a port than something new.

MacRuby

I don't know of any other mainstream language designed to offer:

a) syntactical abstraction
b) native access to a full featured OS widget set
c) direct access to C calls
d) 2-5x slower than C (i.e. Java like performance)

If MacRuby works it will be a huge huge step forward for mainstream development. You could see by 2020 (and maybe by 2016) a language which might be consider an "acceptable LISP" being the primary development language for the most used computer platform (iOS) on the planet. That is a very very big deal. Just think about what happens when meta-programming is standard, something that every developer knows and uses on a regular basis.

And this also plays out in terms of dynamic languages. We really don't have dynamic languages with powerful IDE's and interface kits like what exist for static languages. With MacRuby, Ruby developers will have Interface Builder to tie graphical onscreen objects to code written in XCode. So this is also a very big step up for the dynamic language crowd.
____

I do agree that Apple has been very disinterested in language design and focused. No argument. Though I suspect a great deal of their behavior lately had had to do with the NeXT acquisition. When Steve Jobs came back he came back with an Object Oriented widget set (Cocoa) tied to Objective-C. The research was done at NeXT in the late 80s to the early 90s. They had a language solution, so they didn't need to research one.

OTOH Apple unquestionably did a lot of work with getting PowerPC compilation to be more efficient, that's not exciting from a LtU standpoint but it is real research (or at least real engineering), and research that Microsoft, IBM and Sony have benefited from. They are working on LLVM today.

MacRuby

They've already indicated the new language: Mac Ruby. The Mac Ruby api is already included on the iOS devices though right now they discourage 3rd party applications from using it. But they've been very open they see this as an import direction. There is a commercial version Ruby Motion which has lost of apps (sample Ruby Motion apps).

Apple knows that Objective-C is a problem for their 10 year plan. OTOH right now it is a competitive edge vs. Android and other systems.

Objective-C is a competitive edge?

"(Objective-C) is a competitive edge vs. Android and other systems"
Over Java/Android? Can you extrapolate? (In the NeXT days, obj-C was pretty intersting.)

Objective-C is a competitive edge?

Objectively smart phones are a little over 10 years behind laptops in terms of hardware performance and they were further behind in 2007 when the iPhone came out. If you think about the dominant GUIs on desktops it was Windows 3.1 which was not a smooth experience at all. Most other GUIs at the time the iPhone was released were terribly jerky. By creating a very narrow class of animations which were highly optimized Apple was able to achieve an illusion of hardware much more advanced than what they were really running on.

If you think about the core design ideas of the iPhone in 2007 were:

a) capacitive touchscreen as the primary or sole means of input
b) animation based interaction
c) high speed web rendering

(b) is all about the advantages of Objective-C. iOS developers are directly targeting one specific animation library (Core Animation) running on one specific CPU which is tweaked for these functions in a language which constantly exposes them performance related information. Animated interfaces that feel like they are running on a real time kernel are very attractive to humans. Just talk to regular people and they tell you how much faster their iPad is than their laptop, when the opposite is obviously the case.

When you contrast this with Android/Java which is:

a) Slower
b) Running on a wider range of hardware
c) Running on hardware that is not as tweaked for specific aspects of mobile GUI
d) Written in a higher level language
e) Often written with abstraction layers that write inefficient code.

Applications don't feel as good. The result is that people perceive these phones as worse.
As hardware has gotten better rapidly the advantages of C over Java are fading. The disadvantages of C are becoming more apparent. But in the end, games are still hugely important mobile applications, a huge chunk of sales and gaming is all about performance and consistent hardware.

____

A second major reasons is acculturation of new developers. Apple has a relationship with its developers which allows the entire platform to move quickly in line with Apple's strategic goals. By directly creating the toolsets that developers use they maintain frequent and regular communication with the iOS developer community which assists greatly in this. Google / Samsung have barely any relationship at all with their developer community. They can't meaningfully direct the platform in anyway. The result is that for iOS the application base is anticipatory of hardware / OS / strategic changes while for Android it is reactive.

_____

The third major reason is that Apple owns the most successful app market right now. Most mobile applications being written are being written in Objective-C / Cocoa. If that continues that creates huge lock-in for the platform similar to what Microsoft enjoys on the desktop. Conversely we already see on Sailfish and BB10 that other mobile operating systems can create an Android compatibility layer and run Android applications.

I predict that Apple will

I predict that Apple will not be a major force in driving technology by 2020. It may still be driving device design.

Assuming that iOS is even still around in anything resembling its current form, the distinction between Android, WinRT (though it won't be called that in 2020), BlackBerry OS and iOS won't be relevant for most app developers, given the requirements of most apps (e.g. UI, network, filesystem). Third-party software tools and DSLs will make it all go away.

Well, except for that little bit. Oh, and that little bit. And that incompatible change introduced by one of the vendors in 2019.

But even these will be (mostly) virtualised by third party libraries. There may even be an open source de facto standard.

Apple and 3rd party tools

the distinction between Android, WinRT, BlackBerry OS and iOS won't be relevant for most app developers, given the requirements of most apps (e.g. UI, network, filesystem). Third-party software tools and DSLs will make it all go away.

No way does Apple let that happen. If they were able to crush abstraction layers in 2009 in iOS you don't think in 2010's when they are in a much stronger position they will crush it? Apple is going to need to assure that the people who are building the heavily used applications on their platform are acculturated to their platform. They will never tolerate a situation of being a generic commodity platform. That would destroy their margins and their entire business model. Apple will tolerate 3rd party tools that abstract in so far as those tools enhance user experience and bring value to the platform. The moment they begin taking it away they will ban associated application from their distribution model.

The only time that Apple isn't driving technology to create differentiators is when they have exited a market.

Really?

I thought that the adoption of Unix was a fairly "open" thing to do.

Unix

Apple more or less adopted NeXTStep which was based on a UNIX. In today's OSX, Darwin the BSD underlying OSX is open and an active participant in the open source Unix world. The layers above that though are entirely proprietary. The result is that Apple's customers get the advantages of an Open Source platform in terms of being able to use the wide range of hardware, while experiencing the advantages of proprietary platforms in terms of offering a unified vision.

For iOS though, Apple acts in the role of distributor, which makes allowing for GPL code complex. The result is that unless you are running the Developer SDK, which requires an OSX machine, iOS is closed to lots of the open source community. While there is still Darwin underneath iOS it exists as a service layer accessed via. Cocoa. If Apple tomorrow decided to replace Darwin with the NT kernel running Microsoft Services for UNIX I'd bet 80% of all iOS software wouldn't even notice.

_____

Getting back to your post above though, Apple has been crystal clear about what they think about cross platform development tools and libraries:

We know from painful experience that letting a third party layer of software come between the platform and the developer ultimately results in sub-standard apps and hinders the enhancement and progress of the platform. If developers grow dependent on third party development libraries and tools, they can only take advantage of platform enhancements if and when the third party chooses to adopt the new features. We cannot be at the mercy of a third party deciding if and when they will make our enhancements available to our developers.

This becomes even worse if the third party is supplying a cross platform development tool. The third party may not adopt enhancements from one platform unless they are available on all of their supported platforms. Hence developers only have access to the lowest common denominator set of features. Again, we cannot accept an outcome where developers are blocked from using our innovations and enhancements because they are not available on our competitor’s platforms. (Steve Jobs)

Programming in seven years

2020 starts in seven years, within our margin of prediction accuracy. We can expect a similar level of incremental improvement as from then end of 2005.

It is unlikely we'll get any completely new languages that are dominant; a new language takes more than seven years to get strong traction.

We can expect to see tweaks to some languages, and some new directions that is starting to take hold, but not that have taken over the mainstream.

I'd guess that we'll have a larger use of Clojure and Scala, Java being in a downwards trajectory, that we'll have some replacement for R and Matlab that's starting to take off, and that we'll start to see some DSL for "noSQL" databases take off but direct access is still relatively common. If we're lucky, the latter will include a formalism that is good enough that SQL is becoming a legacy technology (and that the new DSL also works in that space), but I don't think we're going to be that lucky.

Software liability like Basel III

In 2020 we will have legislation in place that will make corporations financially liable for their faulty software, with one exception: legacy software (before 2020) is excused. However, changing legacy software is deemed new software.

But may be not: Basel III legislation has been postponed.

AI

Several of the ideas above suggest one area which I have been expecting to become important for some time. Maybe it will become more important in the near future though I am somewhat skeptical. This is about languages and type systems designed to enable automatically extracting meaningful properties from software systems. Two examples: privacy policies, data protocols.

Since no one commented on

Since no one commented on this: AI is "abstract interpretation".

Interaction Pattern Composition Language

That's what I'd like to see, at a high level. A data model exhibits certain characteristics and cardinalities. A backwards-chaining language can fit interaction patterns to the data model, with easy specialization/refinement. You might even be able to infer the data model from the selected set of interaction patterns.

There are parallels between CSS and interaction pattern selection, as well. CSS applies multiple factors to arrive at a computed style, and an interaction pattern engine might apply multiple factors to arrive at a computed interaction.

Roughly...abstract interpretation, at the level of user intent.

Cynical view

In 20 years the prevalent market will be the mobile platforms. The primary language used for development will be JavaScript or some variant, with some of the following characteristics:

* ALGOL derived syntax
* A misguided spin on lexical scope
* Some watered down version of Clojure's reference types
* Class-based with a mirror-based reflection model

People will fight over their favorite browser-based IDEs especially over the good old days when a spunky upstart called Light Table showed us all how it should be done (before being bought by Microsoft). Racket devs and their futuristic IDE will still be ignored. Crazies on the Internet will scream about something called EMACS (or maybe even ancient programming environments).

Nano-tech & distributed systems

The advances we're seeing in nano-technology are going to start being a factor. We'll be able to fabricate microscopic, purpose-specific machines, each hardware-programmed to perform its task. Compilers will produce things, not bits. Yes, I'm thinking of Neal Stephenson's Diamond Age, which seems to become closer to reality with every press release from MIT or Stanford that I read, but also of the work that is currently being done on RNA programming to create medical nano-devices, e.g. to target cancerous cells and polymerize them.

Computing devices will be wearable, superimposing context-sensitive information over the user's view of the world. High-speed, pervasive internet connectivity will have changed our view of how to architect systems. Local storage and "client-side" processing will no longer be a requirement. We'll have stopped thinking of servers in terms of hardware with CPU cores and RAM, and started taking for granted an effectively infinite set of processing resource. Almost like we've come full-circle to the mainframe time-rental of yore.

In terms of languages, some will evolve to embrace this new technology, some will wither and die, and some new languages with new semantics for describing physical machines with mechanical aspects will have been created, or derived from existing languages. Distributed processing will be the order of the day, with hundreds of small, task-oriented application modules collaborating to interpret, contextualize and understand the world, and deliver relevant information to billions of users simultaneously.

UI frameworks like Cocoa, Qt or any of Microsoft's diverse offerings will be a thing of the past, gone the way of mice, keyboards and touch-screens. We'll see solid user interfaces, apparently projected in 3D right in front of us, actually delivered via our eye-wear or (probably a few years after 2020) active contact lenses or implants. We'll interact with these 3D projections by touching them, moving them with our hands, tracked by motion sensors.

Probably.

A bit soon, don't you think?

I used to work in nanotech. I don't think that nanotech will be up to this point anywhere near 2020. Nor will quantum computers, for that matter.

What may happen by 2020 is a new type of FPGA (or a new technology like FPGAs) which makes reconfigurable hardware an almost-practical option for general-purpose programs. Need to do some crypto? Code up an AES2 coprocessor!

If this happens by 2020, it will be about as useful by then as GPU programming is today. In particular, there won't be anything closely resembling cross-vendor or reasonable OS support for it by then.

Humans matter, not machines

Any technology that can make programming

  • more accessible to large numbers of people, even in limited "Turing non-complete" form, and
  • makes it possible to organize large and complex systems that are robust

will spread more rapidly than hard-to-use math-in-the-clouds technology. Only a mathematical mind imagines a utopia in which all programs are proved correct before they are released into the wild.

Such a technology must eliminate aspects of programming at which humans suck:

  • The archaic idea that humans should type source code into computers. As speech recognition and computer vision make progress, fewer people will find it necessary to know how to type. Programming will adapt, and it sure won't be in the form of dictating ASCII symbols to Siri.
  • The idea that programming is like writing proofs cannot succeed on large scale for the simple reason that humans are more like monkeys and less like mathematicians (except for those mathematicians who are monkeys). Therefore, programming will take on a much more "organic" feel: programming by example, programming with help of machine learning, visual programming, recombining existing of-the-shelf blocks, etc.
  • Programs have to be made robust. They have to become more like auto-parts that can be thrown around, and less like delicate snow-flakes that melt when humans touch them. Javascript is a bit like that, try finding a major web page which does not give any Javascript errors, yet it still "works".
  • Humans are good at high-level organization and bad at attending to technical details that can go wrong. Programming has to done at high level, which does not involve technicalities such as interfacing with other systems, speech recognition, computer vision, user interfaces, data storage and retrieval, and communication. Primitive attempts in this directions are at present known as "SDKs", "standard libraries" and "operating systems". These concepts will still be around, but they will be somewhere in the background and the typical programmer will not have to access such low-level programming.

Of course, there will be low-level programming of the kind we know today, but the vast majority of programmers will not know the first thing about it. Typing discipline, computational effects, parallelism, and other topics discussed by the current PL community will be considered arcane. Future programmers will not know about them any more than today's programmers know about the details of CPU registers or the details of the Java Virtual Machine.

I have no idea how we are going to deal with complexity, but I think network analysis, machine learning and ecology will govern large-scale organization of software, not today's software engineers because their methods are not organic enough for the kind of environment we are heading to.

Let me end with a simile. Even just 15 years ago machine learning and data mining used to be a specialist areas in computer science. They were so successful that today we have something called "Data Science", with conferences and people using of-the-shelf tools to analyze gobs of data, even though they are not computer scientits or statisticians. They are "specialists" who have been trained to "analyze data", of course, but they are coming out of economics and humanities departments. They do not want to know how clustering works, they just do clustering. They are soon going to dwarf the "real" machine learning community, and that is going to change the way machine learning is percieved.

The same is going to happen to programming, because there is no limit on demand for new software. Programming "specalists" who are not "true" programmers by today's standards will crop up. They will just get things done with computers, they won't "write code". They will dictate trends and fashions in programming, not those few specialists who know how to use keyboards and who actually understand what is under the hood.

Okay, now what about 2020?

Okay, now what about 2020?

.

Oh that? Javascript, what else?

...programming is like writing proofs...

The idea that programming is like writing proofs cannot succeed on large scale

This reminds me of something I saw, many years ago, in a book on number theory. The author was expounding on the theme that the distinguishing feature of number theory is that it treats each number as an individual with its own unique personality. In doing so, they remarked that armies don't treat soldiers like numbers, rather arithmetic treats numbers like soldiers.

My point is that programming isn't difficult because we're mistakenly thinking it's like writing proofs. The problem is that we don't have good tools to make it easy to write proofs.

I've been frustrated for decades that we don't have good tools to help people do real mathematics (i.e., write proofs). AI isn't the answer; computers aren't good at thinking, just as people aren't good at computing. Tools should allow an ideal collaboration between computer and human, with each doing what they're best at. Type systems just shift the burden of proof around without getting at the essence of its difficulty. I've long had some preliminary notions of how real math tools might work, but doubt my ability to explain them to someone else except by demonstration, so really I'd have to build the tools myself. To program something that big myself I'd need a programming language with awesomely powerful abstraction. And if such a programming language did not exist I would then find it necessary to create one. And if I didn't know how, I'd have to make a systematic study of abstractive power. Of course, all this is purely hypothetical... oh, wait. Not so hypothetical, then.

We already have lay people

We already have lay people programming at a high level by natural, organically developed (well, human) methods of communication.

Here's how they do it:
1) find a programmer
2) talk to them

I'm interested in your prognostication...can you elaborate:

On the whole, what you seem to be proposing (to me anyway) is a form of functional programming that is abstracted several levels above where a programmer is typically involved today. I understand you to mean that you are going to be seeing commodity "functions" as the primary means of developing an application rather than a general purpose, 3-4GL (such as C, or Java to name a couple).

I'll refer to this idea as "cellular" programming, because in effect that is a decent metaphor for what you are doing. You are generating "cells" (the commodity function) that when combined together form a large scale "organ" (application in my metaphor) which satisfies the functional requirements for its continued existence.

I think what you are saying in the section where you suggest such a new technology "must eliminate aspects of programming at which humans suck" [sic], is that what would constitute as programming would require a less "logical" mindset than what is currently required to write code. Instead, a user could get away with a more "functional" mindset. As example, you give the "auto parts" analogy. You can build a motor using auto-parts, with a decent knowledge about how a motor operates and what it needs in terms of component parts. You do not need the knowledge of how to derive the component parts from elementary materials (i.e. steel, plastic, lubricants, etc...). Or, more simply, you "know you need a camshaft" or some component similar in function, to regulate the valve timing. This can be several levels of abstraction above knowing how to derive the component part.

I really like the idea, and find it an attractive premise. However, I 'm not confident that (if what I describe is what you intended to communicate) we can ever reach a programming model that would approach that level of abstraction. Possibly because at that level of abstraction, the model wouldn't be useful to any other domain except for the domain in which you were working. The tools in that domain would be explicit to that domain.

I think we will likely see an increase in "pattern" based programming, to the point that maybe class templates (if you are using OOP and classes), for example, could be generated by selecting which pattern you are attempting to implement. It is along this train of thought that I could see improvements to IDEs in that they could suggest to you what logic structures to employ and alert you to the potential "anti-pattern" that you are constructing.

Of course, on the other hand, if what the future holds is a marked increased in personal, specialized computers, then your idea could bear fruit. Since the specialized computers, could be isolated to a specific domain.

What will programming look

What will programming look like in 2020?

It will be a complete shit.

So...

Nothing changed, then.

I'll be stunned if

I'll be stunned if programming is appreciably different than it is today, the past 33 years have yielded a great deal of "change" that seems primarily to be running in-place.

Writing PL/1, JCL, and assembler in XEdit isn't much different than writing Java or Ruby in Eclipse. (Along the way there was C, C++, Smalltalk, Scheme, Prolog, SQL, Forth, OCaml, Haskell, sockets, RPC, CORBA, EJB, J2EE, and Rails to name a few of the usual suspects.)

We still tell spend enormous effort telling software how to accomplish the task, rather than focusing on what to accomplish. Too much data, precious little information, no knowledge.

Code quality hasn't improved, arguably it's worse. And we still miss schedule estimates badly.

Different day, same old software.

33 years

Of course we tell the computer how to accomplish a task. Merely telling a computer what to do without "how" is using software not authoring it. The huge number of pre-written programs that exist today that 33 years ago required their own special code, or wouldn't have been possible at all is the evidence of progress. In particular the level of detail needed in the telling has gone down tremendously and the number of things the computer already knows how to do has gone way up.

33 years ago basic communication over a network socket was complex in practice. Nothing like Ruby on Rails could have existed. Networking 33 years ago was about trying to have single small binary files successfully transfer in over an error prone circuit while maintaining sessions. Today a web programmer casually uploads a binary graphics file and it loads on the client without the slightest bit of programming by the web programmer. Ruby on Rails is a step even above that and assumes that page to page to page transitions each involving multiple graphics with database reads and writes can be maintain a state. 33 years ago that wasn't true of files on a local floppy drive much less a remote application.

100 years from now it still going to be about telling computers how to do things. When it is merely telling If it is merely what, that's operations not programming.

Grasshopper, you have not

Grasshopper, you have not snatched the pebble from my hand.

isn't much different? i beg to differ

"writing PL/1, JCL and assembler in XEdit isn't much different than writing Java or Ruby in Eclipse"

It's VERY different. For one, you worry about fewer memory management issues and some bugs are impossible (less debugging). You're IDE's do plenty of work for you from giving you important info to automatically refactoring code to automated testing. The programming process in imperative languages might be conceptually similar, but I think "practical" difference is more important than "conceptual." Just compare even Genera LISP development to your PL/1 or assembler:

http://www.symbolics-dks.com/Genera-why-1.htm

I think a tremendous improvement would be to have these capabilities in modern IDEs. Slowly moving that direction. A Java guy told me Netbeans does plenty of this stuff or provides alternatives he thinks are better. Python is ridiculously easy to learn and use for all kinds of apps. Model-driven engineering like Sun's DASL and iMatix's technologies were quite a change. And we have little kids learning to piece programs together like lego's a la Scratch project. Correct by Construction approaches also improved from requiring entire teams of highly specialized individuals to being usable by undergrads with a specialist consulting (or w/ lightweight techniques just programmers + no specialist).

I'd say things have changed for software development quite a bit since the past. If productivity, usability, and reliability keep improving at the same pace until 2020 I can only imagine how much easier it will be to crank out software. It's not the ideal direction or pace, but it will be hard to say "nothing different."

Note: I totally agree with you regarding your how vs what, poor knowledge, bad quality, and bad estimates. This seems to stem from the economics and sociology of management of software projects. I've seen enough high quality software teams to know there's little technical reason for it: just bad management or uninformed developers.

Working backwards

I've had to think about this recently, and did it by considering the emerging niches that want filling:

  • commodity robotics (e.g., for bio labs)
  • commodity neural devices (the new touch screen)
  • augmented reality / mapping
  • data science
  • lightweight distributed (fancy shared tasklists)
  • programming at social scale (e.g., libraries and stackoverflow)

Some of these will strongly benefit from new types of languages (the first three), such as programming by demonstration. Others will be big on compiler-driven tricks (e.g., data science) -- we're seeing this now with the merger of classic cluster, database, and compiler techniques for modern 'big X' systems. Others, at the bottom, will probably be big on IDEs and frameworks.

My thoughts longer term are quite different, but for short term, a conservative approach is thinking of what we will be running that is a new market and then what is the easiest way to pump out code for it :)

Implicitly, I do not expect great changes for old markets except for two things. First, a growing notion of the 'social' programmer. (Again, think stackoverflow). Second, smart fuzzing etc. tools will make it into the mainstream, especially as building/testing becomes a service outside of your personal dev machine. (This is already happening at big companies, and I expect it to trickle out as more languages get targeted and the price of compute drops even further).

Social programming languages

I'm totally into the idea that programming language design can act as a enhancer for social programming. For example, type systems that make relevant abstractions easier to find and discover, allowing the crowdsourcing of a huge library fairly quickly. Creating a more forgiving usability gradient is also useful here (easy to start using, high ceiling to keep more experienced/skillful programmers interested).

Continuous and universal testing are also ideas that are taking hold already. What if I could change some code (in a code wiki perhaps) and then get feedback about whose code I broke with my change?

As for your application areas, I see the need for more PL people to get involved in machine learning, since all of these fields are heavily dependent on that. How do I code up a recurrent neural network quickly (if you check out Hinton's coursera, there is a lot of Matlab code involved), how do I debug that since the resulting network is opaque and I can only observe it's final behavior? How do we combine subsumption architectures (for robotics) with machine learned systems? We are kind of out of our league in many of these cases, but the areas could benefit from a good PL perspective, and, more importantly, vice versa!

ML today and tomrrow

For the machine learning bit, I'm see two things in the short term. First, more data conversion infrastructure for feeding these systems. Pads/ml and mashup style systems have been going under heavy reinvention in the startup and cloud space, and the need is growing. Second, a lot of the optimization work showing up in VLDB/SIGMOD/SOCC/OSDI for ML systems etc. are rehashes of classic compiler and query optimization ideas, and there's a lot more mileage to get out of it. (Heck, I've been tempted to show how to do a few things via incremental query REPLs!).

I agree with you about integrating ML with PL being important in the longer term. There are already projects like Bayesian model DSLs, but that is for the short term. I found the work of Pedro Domingos to be particularly exciting in terms of a framework chasing the big vision.

We should let machines make all the choices they're better at.

Predictions. Hmmm. Okay.

We will start to abandon our pathetic attempts to understand the complexity of the systems we're targeting, and the complexity of interaction of the fine-scale structuring of data in our standard libraries (much less our user-defined code) with those systems.

We will allow judgements we now consider "simple", but which profilers show we often get wrong, to be made by the profilers in the first place. Actual testing and profiling information will drive compilation and interpretation, allowing standard libraries to choose among many possible implementations of the results we specify.

New programming languages, accordingly, will need to specify less of specific methods and more of desired results and performance requirements. When and as they do specify specific methods, they will do so behind a "choice point," where there are many possible implementations to choose from and the system is required to make an automated choice.

I predict that infrastructure for testing programs will get incrementally better. I predict that profilers and profile-awareness will get built into (some) development tools at a more fundamental level. This will allow profiling information (from earlier runs, from earlier versions, from earlier in the same run, or even possibly from live performance testing of prospective code snippets during compilation) to drive many of the tradeoffs and representation decisions made when interpreting the code or preparing it for execution.

These systems will produce code optimized for any specific architecture even though the developers of the systems do not understand in detail the specifics of all those architectures. Instead, the tools will automatically create profiles by testing the performance of benchmarks on the target architecture, and optimize according to those profiles.

This has been made more valuable both by the increasing ubiquity of virtualization and by the increasing complexity and variety of CPU architectures; the compiler can no longer assume that the code it is generating will run on a particular hardware or OS, nor on any particular implementation of an instruction set or the target virtual machine. Compiler writers can no longer be expected to apply detailed human understanding to all of the increasingly diverse and increasingly complex systems on offer, let alone how those systems will interact with particular implementations of the code that their programmers write.

The ability to optimize for the system, processor type, or VM that is actually there now and the way it actually performs, as opposed to the system, processor, or VM someone expected to see or the performance they expected that inevitably-different system to have when and where the development tools were written, has become more valuable than ever.

I also expect that in some experimental languages software development systems will be making simple representation decisions that we now consider to be of algorithmic importance for us. Human expectations of the performance of a given data structure are often wrong in the first place, subject to our ignorance of the specific cache line lengths, parallelism capabilities, memory latency, or task-switching overhead of the processor in question. In subsequent uses of our code, when code is reused in another project, these structuring decisions can *become* wrong with no change in the code, simply due to changed usage patterns. In subsequent deployments of the code where it is run on a different OS, processor, or VM implementation, or when code is prepared for execution on a different processor type with different capabilites (CPU, FPGA, GPU, DSP, etc) structure choices can again become wrong due to different underlying capabilities making some operations more and less expensive, again when the code has not changed.

Therefore the software development systems should "correct" these decisions to reflect actual use -- which amounts to wiping out the distinctions among the fine-scale structure decisions that can be "corrected" in the first place. And therefore we need a way to specify algorithms as vaguely as possible (don't worry, we'll call it "abstraction" instead of "vagueness"), both to cater to our pathetic human understanding of how our code actually runs on the machine, and to allow the tools maximum latitude in deciding how best to achieve the results our code specifies given the current computing substrate.

Therefore I predict a continuing increase in algorithmic and structural abstraction, with a concomitant continuing loss of specificity in the details of data structures. We will reach a point when we won't be able to tell, even by tracing our definitions all the way down to the limits of what our languages can express, whether something is implemented in terms of pointers or in terms of indexing; that kind of transitive beta-reduction on definitions will be able to tell us only that those were among the choices from which a profiler-driven decision was made.

In seven years, there will be at least a few emerging languages embracing this strategy, in which "collections" make pretty much every possible operation available in the programming environment, and then it's left to the compiler or runtime environment to decide whether a particular collection is best represented as linked list, or an array, or a doubly linked list, or a heap, or an associative array or a hash table or how it's indexed, or as a database tuple, or whatever, making a decision based on what representation best optimizes runtime, storage, and responsiveness rather than because the programmer told it specifically how to represent a given collection.

Replacing the loss of specificity in data structures, we will see two new kinds of information encoded in computer programs, and likely abstracted to have scopes that tell what specific subparts of code they apply to, the same way variables and constants and procedure names have scopes that tell where they are visible. First, programmers will optionally write code specifying decision parameters, to inform the system of what performance characteristics are considered desirable in making decisions about representation and how to optimize.

Second, programmers will encode optional information which the system might or might not be able to prove, both for the system's benefit in making optimization decisions and for their own in assessing the code's correctness, knowing at the earliest possible instant when and where it has gone wrong. This includes the kind of information that we now regard as type information, for example that numbers need be represented only to a given level of precision or within a given range, in order to get the intent of the code right. But this information will not be (correctly) seen as specifying a particular binary format. Instead it will specify minimum requirements that *restrict* the choice of binary representations used, permitting but not requiring greater range or precision.

It also includes a kind of information now included in a much broader notion of types not shared by most programmers, which we mostly now regard as programming invariants, constraints, security requirements, or contracts. These are crosscutting concerns that we currently usually implement by means of painstaking individual assertions in many places. For example, a programmer might be able to state (once!) that, within a particular scope, an arbitrary boolean expression will NEVER become true, and rely on the development system to test the expression any time it touches any of the variables of that expression and immediately halt the program and display the current values of those variables if it ever does become true.

We're going to see more languages with "selectable safety levels" where the programmer can optionally add type declarations to address undecidables, and the development environment can be commanded to do anything from "RUN IT AND CRASH IF YOU HIT A TYPE ERROR" all the way up to "EITHER PROVE THIS ENTIRE PROGRAM TYPE-CORRECT, OR TELL ME WHAT YOU CAN'T DECIDE."

Some of them will support even more esoteric commands, such as "EITHER PROVE THIS PROGRAM WILL HALT, OR TELL ME WHY IT DEFINITELY WON'T, OR TELL ME WHAT YOU PROVED YOU CAN'T DECIDE, OR TELL ME WHAT YOU WERE TRYING TO PROVE WHEN YOU RAN OUT OF TIME."

None of this will be mainstream within seven years. But I expect that we'll see (indeed, in some cases, we are already seeing) its beginnings.

Ray Dillinger

The sufficiently smart VM?

I can conceive of a VM which includes a profile for the real system on which it happens to be running—either handmade or more likely derived automatically from a suite of benchmarks. So the VM can know that, for example, using a vector now but moving the data to a linked list later will reduce overall runtime—under some set of assumptions. Choosing a representation statically could miss dynamic optimisations, and isn’t terribly interesting—it’s easy to infer that this or that interface is needed, and have a set of defaults in the case of ambiguity. That’s exactly what Haskell does with typeclasses for its numeric types.

The problem with dynamic representations is predictability: just as a GC may decide to run at an inopportune moment from your perspective, a representation manager might decide to do an expensive operation to save time later, causing a noticeable performance drop now. Of course, you could get clever and change representations lazily or incrementally. And at the cost of an extra indirection, you can hot-swap representations in constant time.

For example, one of the big reasons that people use C++ is for its deterministic resource management. I’d wager that “deterministic” is actually better than “fast on average” for realtime applications.

And I’m sure a “sufficiently smart VM” is as possible as a “sufficiently smart compiler”—the question is not whether it can be done, but whether such a costly thing will ever be.

Going to butcher your post a

Going to butcher your post a bit, but hopefully not too much so it's out of context:

...
will reduce overall runtime
...
The problem with dynamic representations is predictability: just as a GC may decide to run at an inopportune moment from your perspective, a representation manager might decide to do an expensive operation to save time later, causing a noticeable performance drop now.

The VM could be set to meet realtime requirements rather than reducing overall runtime.

In principle, such a VM could determine that removing itself from the running application (on a specific platform) was the best thing to do.

Of course you have to tell it what the requirements are.

You're expecting the system to make incorrect choices. If it does so, then that's because it's making incorrect assumptions about the requirements it needs to fulfil. It follows that your source code has failed to express the correct requirements and is therefore erroneous or incomplete.

The problem with dynamic representations is predictability: just as a GC may decide to run at an inopportune moment from your perspective, a representation manager might decide to do an expensive operation to save time later, causing a noticeable performance drop now.

That happens only when you're trying to apply dynamic representations to a language that doesn't support them with a way of expressing the requirements to be met.

My point is that if you have a responsiveness requirement about how the program runs, then your program is *incorrect* unless your source code expresses the responsiveness requirement! The system needs to make choices to meet the requirements of your source code, rather than having you take over and mangle its abstractions and types by doing low-level bit-groveling.

A responsiveness requirement isn't an indication that you can't allow the runtime (or the compiler) to make choices about how and when to do an expensive operation. Instead, it's just one of the constraints that your program needs to express so that the tools can make choices appropriate to your requirements.

Ray Dillinger

Watsons in our IDE's

The most important advance in programming by 2020 will be the beginnings of intelligent (enough) computer assistants in our IDEs. The assistants will be specialize in specific types of development like C/C++, Java, Mobile. They will have intimate knowledge of common APIs and have been trained on tens of thousands of code projects pulled from the open repositories across the web (google code, github, bitbucket,...). In addition to having 'read' more orders of magnitude more code then any human could in a lifetime, they will also have rudimentary ability to extract programmer intent, and organizational patterns from code. These assistants will be paired with human programmers in much the same ways that computer chess programs have been paired with human players to both teach and to enhance performance.
Since the assistant will have rudimentary understanding of intent. Programming will become a "pair" programming session between human and computer. The computer automatically bringing up example code snippets, suggesting references to existing functionality that could be reused. In some cases it may suggest refactorings, simplifications, or abstractions to implement. The human, computer pair, will also interactively suggest, confirm and fine tune specifications of mathematical properties and invariants at points in the the program. This will help the computer assistant to not only better understand the program but also to to generate real time, verification test runs using SAT technology, much like Alloy does today.
Leveraging the strengths of the computer and human will lead to an order of magnitude improvement in programmer productivity. Interactive testing will eliminate whole classes of logic bugs, making most non ui code correct by construction.

a "pair" programming session

a "pair" programming session between human and computer

Existing interactive programming environments such as Smalltalk and APL have this feel, which many developers seem to dislike. Finding out whether that’s a language issue or a deeper one of (perceived) control seems to be the first step to making such an environment popular. I think it’s a combination—not everybody loves REPLs, and most devs take no pleasure in “weird” languages.

Depends I think...

For high-level programming, it will look like a Lisp with a whitespace-based associativity rather than parens ad nauseum, along with polymorphism a la cart, and contracts.

For application programming, it will look like an ML with contracts and type classes.

For systems programming... unless we can get a mainstream GC without impractical arbitrary pause issues, it will remain C++... Otherwise, it will look the the forementioned ML.

About the human end

If I may add an opinion on the human end.

It is my view that programming is not hard to learn. That is to say, it is rather easy.

Software engineering on the other hand, is very difficult.

But the dominant language for programming today is English. The English language, in it's current form, is very close to a natural language, and shares all the problems of natural languages: inflexibility, imprecision and verbosity just to name a few.

So this language, poses serious difficulties for people to imprint their thought into code and what is more, to remain clear in the face of change, by their hands or other people hands.

The single one most important defect of it, is the inflexibility of the ordering of words. If i take random permutations of a sentence of 10 words, chances are that the meaning will be lost, in most of them. The technology that permits this kind of feature is called inflection. It is only natural that natural languages cannot have this, it has to be created by design.

Now, one of the best languages at that respect is Greek, but it is not the best one. As far as my research is going, there is only one better, and it's called Sanskrit. It seems to have certain innovations that Greek does not have, but it is very much a dead language by now, and one cannot really be sure.

I consider that an integration of lambda calculus with a language that people can speak, is not impossible.

Now how about 2020? Well, if a formalization like that happens, and people actually start speaking such a language, it's adoption may happen quicker than someone expects. It is just a matter of people actually realizing that they need a language like that, for automating their everyday life rather than just posting photos in the internet. 2030 sounds more like it really.

Not dead yet

Sanskrit is still used in parts of India, apparently. It's even apparently an official language in Uttarakhand, which is a apparently a place. If Sanskrit sentences are commutative, associative and idempotent, I wonder if this would be a good language for thinking about Reactive Demand Programming. David?

A language where sentences

A language where sentences are commutative, associative, and idempotent could be quite declarative. But there are many formal languages with those properties (e.g. temporal logic programming, concurrent constraint), yet not many are good for speaking of data dependencies and observer influence (and thus for thinking about RDP).

If (as I interpreted) Pramatias spoke of words within a sentence, not full sentences, that would greatly undermine the expressiveness and scalability of Sanskrit as a declarative language. After all, a sentence is maybe 10-20 words. An essay or program is maybe 100-20000 'sentences'. If we're reducing complexity from permutations to combinations, it's the composition of sentences that matters.

My apologies

We agree on what Pramatias meant but that's also what I meant when I wrote "sentences are commutative, associative, and idempotent." Those terms apply to operations, not operands, and I was viewing sentences as a concatenation of words.

Concatenation is the

Concatenation is the operator. A question is whether 'words' or 'sentences' are the operands; the latter enables much more declarative expression. I think your meaning is not obvious in what you wrote.

I find your topic line quite insulting. Please change it.

My meaning wasn't clear,

My meaning wasn't clear, which is why I was explaining it. My topic line was just silliness (along with pretty much everything else I've written in this thread). No offense intended. I've changed it. Sorry! I hope you and the rest of LtU have a great holiday!

Thanks.

Thanks.

Vulcans vs. humans

Humans will continue to think in terms of analogy and metaphors, with a tendency to anthropomorphise entities in a program, and hence OO programming will remain popular and we should even see some significant advances on object systems. As for imprecision and fuzziness, we should expect our programming environments to support the way we think (and solve problems), not the other way around! Its not that our spoken language is imprecise, but we are.

But then there is a class of people that I barely understand, who think and communicate in radically alien ways from myself and the people I understand, they heavily use math and logic, they talk about elegant constructions where I see only complexity. Perhaps we will understand each other a bit better by 2020, where today we actually know what lambdas are now, and even use them (albeit lightly), but we still don't know what they mean by "monads."

Object oriented programming

Object oriented programming tends to freeze in a particular 'view' or model, whereas humans casually shift views to reify whatever model is most convenient. OO may remain popular, but that is not a good thing. Unless OO changes considerably to address the issue of heterogeneous views, it will continue to hamstring every human that uses it.

we should expect our programming environments to support the way we think (and solve problems), not the other way around!

Do you believe we should "support the way we think" even when the way we think is commonly short-sighted, close-minded, inconsistent, incoherent, inflexible, inefficient, and so on?

Effective thinking is a skill, not something that comes naturally to humans. I seek effective ways of thinking and problem solving that humans can easily intuit and adopt with exposure and a little education. Our programming environments can help educate and shape the way we think, such that we think better and find better solutions to problems. I would prefer shaping be subtle (via controlling the 'path of least resistance').

Its not that our spoken language is imprecise, but we are.

I think both are imprecise.

Object oriented programming

Object oriented programming tends to freeze in a particular 'view' or model

Good point. The future of OO programming should involve better support for model evolution and shifts.

Unless OO changes considerably to address the issue of heterogeneous views, it will continue to hamstring every human that uses it.

Its not like the FP crowd are any better at this, I don't see it as a disadvantage or advantage for OO.

Do you believe we should "support the way we think" even when the way we think is commonly short-sighted, close-minded, inconsistent, incoherent, inflexible, inefficient, and so on?

Are you quoting a TOS episode? Yes I do, Mr. Spock.

Effective thinking is a skill, not something that comes naturally to humans. I seek effective ways of thinking and problem solving that humans can easily intuit and adopt with exposure and a little education.

We'd all like the ability and luxury of reaching an enlightened state, but until then there is code to be written! What makes us so versatile is that we are willing to tackle problems that we barely understand or barely have the skills to solve. We don't need to understand everything before we dive in, we can learn by doing. Planning and knowledge are nice luxuries.

There is also a danger in becoming addicted to too much elegance and perfection. You freeze up and can't accomplish anything because you become afraid of problems you still don't understand. A little bit of ignorance can go a long way.

I think both are imprecise.

Language is imprecise because we are, not the other way around.

The FP crowd has begun to

The FP crowd has begun to address heterogenous views via systematic uses of lenses, and is aided much by purity. Other paradigms - especially logic programming or functional-relational programming are relatively more effective than OOP for heterogenous views. I would not characterize the problem as FP vs. OO, but I think the FP crowd is "better at this".

I never advocated waiting on an "enlightened state". Effective thinking does not require big design up front. It just requires a better framework than humans have naturally - e.g. logic instead of lore, science instead of superstition. I do favor exploratory programming, but I think it's a mostly orthogonal issue.

To "support the way we think" - without shaping or directing how we think - suggests we support expression of ineffective, inefficient, inconsistent, incoherent, fragile mudballs of models, with all the intermediate bumbling and fumbling and fallacy and bikeshed coloring. Because that's often "the way we think", and it is not good or great.

I would not characterize the

I would not characterize the problem as FP vs. OO, but I think the FP crowd is "better at this".

Lenses and FRP are interesting but so far have not been shown as very pragmatic. Turns out the world is very impure.

It just requires a better framework than humans have naturally - e.g. logic instead of lore, science instead of superstition.

Pure logic and math implies more understanding than we often have; that's not to say it isn't useful but it often isn't applicable! Programmers are more like experimentalists/scientists, forming and testing hypotheses about the world, but then science and math are distinct activities. Imprecision and ignorance are expected.

To "support the way we think" - without shaping or directing how we think - suggests we support expression of ineffective, inefficient, inconsistent, incoherent, fragile mudballs of models, with all the intermediate bumbling and fumbling and fallacy and bikeshed coloring. Because that's often "the way we think", and it is not good or great.

But isn't that the point? For better or worse, we solve problems and accumulate knowledge in a very bumbling manner, probably related to the way our creative fuzzy neurons are firing around in our brain vs. the cool deterministic logic of the computer. I'm sure we can improve ourselves, but I don't see much point to becoming more like the computer that I can program and instruct anyways. Rather, let's just make the computer more like us?

Lenses have proven pragmatic

Lenses have proven pragmatic in real apps and even a GUI architecture (lamdu's bottlelib), and are used to manipulate state through views. Functional relational programming (which I do not like to call FRP, which refers more often to functional reactive programming) has been used successefully in various forms (e.g. Ur/Web); it's also about managing state. The world is impure, and that is addressed.

Whether a logic requires more information than you have depends on the logic. There are many logics suitable for open or partial systems. That's the sort of property you should consider when seeking effective tools for thought.

I'm sure we can improve ourselves, but I don't see much point to becoming more like the computer that I can program and instruct anyways.

You seem to treat "improve ourselves" as wasted time. But I see it as an opportunity for time we spend anyway. People will gain skill with whatever tools they use, and exploration is a good way to gain experience. There is much opportunity to teach users to better comprehend computation and automation.

To "make the computer more like us" probably means to hide from users the differences between man and machine. How much computational overhead are you willing to expend toward deceiving your users? And are the results any better? (As far as I can tell, humans also find humans difficult to manage.)

I'm not really up on current

I'm not really up on current research in lenses, but the last time I looked they didn't really address state transitions (induced by events). FRP has a similar problem, though I really appreciated the bi-directional nature of lenses (vs. FRP), which is something I'm designing to a more limited extent into the language I'm working on now (LPL).

The point about computers...we have found great benefits in making computers more like us via machine learning and neural networks. They begin to display all of our drawbacks (ignorance, tending to generalize, fuzziness) but then can do things like process spoken language, which is impossible to do logically (indeed, what is learned is mostly opaque to us). I see us doing more of these kinds of things, not less, and why should humans try to go in the opposite direction by changing us to be more logical when we are trying to get our computers to be less so?

Lenses address heterogeneous

Lenses address heterogeneous views, but it is easy to use the for eventful state transforms: on event read state through lens then write modified state through lens. Done. Boomerang language was built on that pattern.

Humans being illogical causes many problems even outside of computing. That will be true for illogical computers. I agree we'll see a lot more fuzzy aspects in programming, but I think that is no substitute for effective, provable reasoning. Meeting in the middle will better serve both man and machine.

Streams *are* state transition systems

Streams are isomorphic to state machines (possibly infinite-state). As a coinductive datatype, S(A) = να. A × α. However, coinductive types can be encoded in System F as να. F(α) = ∃α. α × (α → F(α)). So we have:

S(A) = να. A × α
     = ∃α. α × (α → A × α)

So a stream is essentially a way of saying "I have a hidden state, plus a state transition function to generate updated states and elements from this hidden state".

IMO, FRP has historically had trouble with events because streams are the wrong datatype for them. An event is something that may happen, so they are better modelled with the type E(A) = να. A + α. This is something that either finishes and produces an event, or tells you nothing has happened now, and you can wait and try again later.

In terms of temporal logic, a stream S(A) is "always A", and an event E(A) is "eventually A".

More operationally, event handling is like programming with promises or futures -- precisely the abstraction that modern libraries in JS and Scala provide! (Furthermore, since the eventually type constructor forms a monad, monadic notation gives very natural syntax for programming with them.)

Events

My New Year's resolution for 2013 is to figure out events. I don't think your above understanding is very useful for the events I'm interested in dealing with for live programming. Consider:

  • Events in a Paxos-style replicated distributed system (think withdraw or deposit bank actions). We need some notion of temporal ordering and history to ensure consistency between the replicas. For performance, we want to also process multiple (ordered) events at the same time.
  • We want to debug a Kinect program that is basically a stateful transformer of a stream of video input frames (events), where the output is a stream of video output frames. Most computer vision algorithms are stateful; e.g. recognizing a multi frame gesture.
  • Each time step in a physics engine can be seen as an event if we take events = state transition, which is quite convenient. Declaratively, certain forces are asserted that are then applied when they are true on the time step, causing other values to change and force applications to change. Discrete interactions like collisions can also be treated as events. Combine with the first point to replicate an RTS like Starcraft to multiple multi-player machines.

So I don't really see a promising future for promises and futures with respect to event handling :) But treating events as streams, or better yet strips (like a film strip used in AfterEffects), is quite useful in modelling event problems. You can even checkpoint your program along points in the strip to time travel (ala Bret Victor's video)

I'm not married...

...to this kind of event. The primitives I think are more fundamental are a next-step operator and recursive types. Basically I'm suggesting using a variant of the modal mu-calculus as the type system for a temporal languages.

Your examples 2 and 3 both fall out nicely when you decompose temporal datatypes using next plus recursive types. The same goes for strips, too -- they are basically temporally lazy, possibly-unbounded lists. That is, Strip(A) = να. 1 + (A × next(α)). So they yield a sequence of frames, possibly finishing at some point. I find them useful for modelling things like animated transitions.

I am dubious that off-the-rack temporal logic applies as naturally to reasoning about Paxos, though, since distributed applications do not have a global notion of time -- nodes may run at arbitrarily-varying rates, and message delivery is similarly unpredictable. Next-step is not a natural logical operation in this setting.

You need to be super-careful with time travel, though. Even something as simple as permitting arbitrary values to be used a different time step than when they were created leads to horrendous space leaks. E.g., if you permit using a stream value on a later tick than when it was created, then you have to buffer the stream's values. Checkpoints don't help you avoid buffering, if the stream values happen to come from the network or user.

I partition types into time-varying (e.g., streams) and non-time-varying (e.g., booleans and integers) types, and only permit non-varying values to be used at different ticks. If a programmer wants to use it, they have to do the buffering by hand, so that the cost is evident in the source code.

Global notion of time

I find UTC quite effective as a global notion of time in open or distributed systems. It's also better for modeling things like animated transitions, even in a closed system. And it is easy to understand buffering - e.g. exactly how long a value hangs around - if you make static latencies part of the type system.

Objects as reactive elements

So a stream is essentially a way of saying "I have a hidden state, plus a state transition function to generate updated states and elements from this hidden state".

That sounds a lot like an object, which is how I'm doing "reactive programming". Events are mutating method calls on objects. Reactive values are const methods, supported by a memoizing implementation. Similarly to what you're describing, any state that you want to accumulate over time must be captured explicitly in an object. There is no notion of 'time' built in to this model; only event ordering. If you have a nice object system (e.g. allows easily packaging subgraphs of objects), then this supports higher order FRP.

But I'm having trouble understanding your comment about event handling being similar to promises or futures. Can you elaborate? I'm guessing this is explained in your latest paper, which I haven't figured out yet. (Also, if you understand the preceding paragraph and can point out any pain points or limitations you see, that would also be very helpful to me.)

This sounds interesting.

This sounds interesting. What's the secret sauce that gives you reactive programming rather than just an object system?

The incrementalization /

The incrementalization / memoizing implementation. For example, if you implemented a GUI in this style with a typical OOP language and compiler, you'd end up rebuilding the scene graph for the entire widget hierarchy every time you rendered a frame.

How does this work? Does

How does this work? Does this memoization have a semantic meaning, or is it purely for efficiency? Could you give a small example?

One problem with traditional FRP that I've mentioned before is that "GUI = pure function of time varying application state" doesn't work as smoothly as you'd think at first because widgets often have internal state that you don't want to expose in the application state. So logically a GUI is not a pure function of the state of the application. For example a textbox needs to keep track of where the cursor is, and a sortable table needs to keep track of which column it is sorted by, etc. How do you deal with this?

Memos are an (essential)

Memos are an (essential) optimization only. I'll try to give an example tonight. The fact that GUI widgets are stateful is one of the motivating examples for this approach. I think I prompted this question by misstating that the widget hierarchy would be recreated each frame. See my edit.

Edit: I looked back at my notes on this and I'm forgetting an important aspect in dealing with change notifications. I also think a decent example is going to need me to explain my object system a little bit more (which I call Processes, as I've mentioned a few times in the past). I don't have time to get into this tonight, but maybe by this weekend I can put something together.

Cool, looking forward to it


Are you doing something like

Are you doing something like incrementalized immediate mode UI? Sounds like we have similar ideas.

This is very pretty but I'm

This is very pretty but I'm having a bit of trouble imagining how this translates to GUI programming. How would you connect this to a GUI toolkit? This model seems entirely pull based, whereas for practical GUI programming you need a push based model?

Grounding

First, my understanding is that Ur/Web doesn't use functional relational programming. Its type system supports static computation over record rows, and has a FRP layer whose API seems to focus on cells (state) more than normal FRP, so I don't know what you're referring to there.

Second, intrigued by your comment that lenses are now pragmatic, I couldn't find "lambdu's bottlelib" -- do you have a link? I'm struggling to understand what you mean by "proven pragmatic in real apps" for lenses. Is your statement grounded on experience reports somewhere or personal use, or are you treating the existence of some paper or library as proof of usability? The lenses work is interesting in theory, but I was not aware of non-theoretical connections to software engineering. (The roots in distributed file systems sounded interesting, as did trying to apply them to security, but neither projects seemed to have gone anywhere).

There isn't much to

There isn't much to functional relational programming - just a design where state is kept in external relations and modified atomically by functionally specified transforms. On the server side, that's pretty much what Ur/Web does. It even has a type system designed for it. I say it's using functional relational programming because the description fits.

Lambdu is a live programming development for Haskell. (I'm on phone. Linking painful.) I've not used it, but read on reddit from a dev that he is extremely satisfied with the use of lenses. [Addendum: Oops! I mispelled it; this is 'lamdu'. I can't remember where I heard it uses lenses. Reddit, I think.]

IMO, a single app can serve for 'proven pragmatic' if it covers enough bases. Papers aren't practice. Wide adoption is a different issue. I've seen many apps based on lenses, for editing XML, for editable views of code, and for one web business app. IIRC, Lamdu uses lenses to integrate state (text boxes, etc.) with pure, declarative widgets.

multiple inheritance

I don't agree with the basic theme that functional is somehow machine like while object oriented is human. Structural, functional and object-orientation were all attempts by humans to bridge the human-computer divide. Computers have no need for organization, abstraction nor hierarchy. Computers think in terms of large complex enumerated lists which are permanently non-understandable to all humans.

That being said... I think the major failing of object orientation in terms of moving away from human thought the last 20 years has been abandoning multiple inheritance. When people think of things they think of them as having multiple overlapping and conflicting properties that are contextual. My sister besides being in my family and having her own is a banker and a fashion aficionado. It is impossible to model her behavior without considering all three of those hierarchies and probably several dozen more.

What is so fundamentally frustrating about object orientation as it exists today is that it doesn't deliver on its promise of being able to create models which correspond to human thought at all. When confronted with the difficulties of implementing multiple inheritance the object oriented community blinked and so OO programmers inevitably have to create hierarchies that don't remotely correspond to the way they think about real objects, but instead are an artifact of their organization of functions. I think the most interesting thing going on in object orientation is multi-paradigm: Scala, Perl, Ruby. They are moving us towards a casual object orientation where every developer anytime they have hierarchical data or a complex union of data almost automatically builds an object hierarchy / structure for it. While at the same time not losing the powerful tools for manipulating data on the functional side.

I like duck-typing over multiple inheritance

My aim isn't to argue with you. Both this subthread and your points are interesting and veer close to ideas I want to discuss; so this is more free association prompted by your lead. But first I'll back up to something good Sean said. (And, uh, sorry this is so long.)

Sean McDirmid: Humans will continue to think in terms of analogy and metaphors, with a tendency to anthropomorphize entities in a program, [then associates this with OO programming]

I'm inclined to agree with George Lakoff most thought is so heavily informed by metaphor, it's hard to tell whether thought in fact consists mainly of metaphor disguised as absolute reference via decoration by concrete episodic experience detail. (Metaphor may be low-level and close to the brain's "metal" rather than being high level and abstract.) But when a developer anthropomorphizes code, sometimes it's just shorthand for, "I wanted X to happen when I wrote this code, but I'm going to say code wants X to happen, because that's brief and close enough."

Object orientation affords a chance to reify a dev's intention to seek locality in control structure while imbuing components with local responsibility for invariants code must maintain to stay organized enough for a dev to reason about. (If code actually follows these rules, system behavior will look like this.) A dev can explicitly structure code as local agents whose roles encompass responsibilities that maintain the invariants making algorithms work. And when behavior has a interface other code can mimic, this affords an "interchangeable parts" model of engineering idealized by manufacturing physical objects in the industrial revolution. (In practice, it's hard to make fully interchangeable software parts.)

Folks vary a lot in what they think the point of OO programming is. I think it's locality, to simplify reasoning and standardize interaction points of code written by different devs. If you can keep a concept local to one small section of code, you can change your mind by swapping in a new piece. But if a concept is laced through all code in a global manner, you're stuck with it. And worse, if you wonder how a change affects runtime behavior, you might have to look everywhere, which really sucks. (I've been looking at too much C code written with philosophy, "Since I'm not using an object oriented language, I can let my dependencies creep randomly into every corner and no one can berate me.")

JeffB: I think the major failing of object orientation in terms of moving away from human thought the last 20 years has been abandoning multiple inheritance.

If all you want is to satisfy api from multiple base (or super) classes, then any dynamic language with duck-typing will let any object behave as though it inherits as many ways as needed. (Actual inheritance of state and methods is considered just a convenience, and might not be supported.) An OO language with static typing without multiple interface compliance can suffer from the limiting effect you note.

JeffB: What is so fundamentally frustrating about object orientation as it exists today is that it doesn't deliver on its promise of being able to create models which correspond to human thought at all.

While someone may have promised that, I don't think object orientation did. (But I know what you mean, as you can see from my earlier comment on anthropomorphizing shorthand.) Each community is a mixture of many folks saying different things. Some folks are more vocal, and snake oil pitches can sound deceptively simple: drink this simple elixir and you're golden. A conservative and nuanced pitch is easy to ignore because it doesn't promise much, and is hedged by unsatisfying caveats that sound like lack of confidence when merely accurate. I was inclined to tell folks OO was better than following no rules at all, but saying improved locality and reasoning "sucks less" is a weak pitch completely lacking glamor.

The folks saying you could get ideal code via domain modeling that matches human thought were completely wrong, and it was clear they were completely wrong from the beginning, but you can't tell them to "shut up" and stop spoiling the story with dreamy nonsense. It wasn't clear to everyone, and calling the new fad dreamy nonsense can and does get you branded as less bright than hipsters on the bandwagon, as long as you're in the minority. Nowadays the bandwagon occasionally exaggerates how much OO sucks. Hey, it's just an organizational tactic, and all systems suffer from creeping entropy via misalignment, over-specificity, and a host of other brittle narrowing effects.

Whatever replaces OO in coming fads, it will still be good to aim for interchangeable parts, fault isolation, locality for reasoning, and means to substitute components for testing and comparison based on measurement instead of reputation alone. I won't care what it's called.

Your last paragraph notes problems with hierarchies. Yeah, deep hierarchy sucks, while shallow and flat hierarchy can lend some loose organizational structure without becoming too severe of a strait jacket. Models have diminishing return as they get more involved. Some model is better than none, but far too much model leads to a world like the movie Brazil, where your can't get anything done without filling out form 27B-6 (unless Robert De Niro as technician Tuttle arrives on a zip-line to fix the problem without going through channels).

This is a great post, and

This is a great post, and much better than what I could articulate. One of the things missing from William Cook's definition of OO was the why of it all, how OOP is meant to solve problems, the intention so to speak; otherwise its all just observed technical classifications (objects look like this, functions look like this, end of story).

Whatever replaces OO in coming fads, it will still be good to aim for interchangeable parts, fault isolation, locality for reasoning, and means to substitute components for testing and comparison based on measurement instead of reputation alone. I won't care what it's called.

Objects, functions, logic, components, procedures, modules, relations, and so on, are really fundamental concepts now and not specific enough to be fads. There are many fads that re-spin OOP specifically in one way or the other; e.g., AOP, MDD, CES, but objects themselves are too vague and general to be fadish.

Objects are great.

Objects are great (when you actually need them). Object-orientation, however, is a pox.

Duck typing

Hi MicCuster, happy new year! I think we mostly agree.

First off, excellent point about duck-typing. It will be interesting to see if duck-typing does in practice recreate complex multiple inheritance OO hierarchies. It is a very good point, duck-typing might very well be the solution for how one gets the predictability of single inheritance and the flexibility of multiple.

You don't see that much use of duck-typing though to build complex hierarchies. I think that has to do with the fact that dynamic languages are still not used much for huge projects. I was discussing MacRuby above. If we assume that by 2020 (going back to the theme of the post) MacRuby is the dominant OSX/iOS language and Apple is moving 200m+ handsets a year plus still say 5-25% of the computer market, we will have millions of Ruby developers who are building complex applications in Ruby which supports inheritance and duck-typing.

In terms of OO as one of the big strategies for controlling complexity I agree it works. The complexity of software written today is much higher than the software that was written before OO and yet the complexity of architecture is lower. OO deserves credit for complexity control.
Isolation of code has been success from the earliest days of introducing the notion of subroutines, to local variables... It is why the monadic approach of isolating state seems to be making a huge differences in terms of parallelism in practice.

In terms of interchangeable software parts I think libraries and system services have provided the models that are most used, what reuse really looks like in most practice. I don't see any evidence that OO programmers in general are able to make more use of libraries than structured or functional developers, and if anything structured programmers have an edge over either functional or OO in terms of system services. It would take us afield but in general I think the services approach, the UNIX approach, is the right one to achieve reusable code. I agree that Meyer wanted reuse to be a focus and had Eiffel rather than C++ become the first huge success story maybe it would have been but C++ did win and C++ just wasn't all that good at reuse.

There is one area or reuse that I think OO has been hugely success in where functional and structured mostly don't try and that is area of large complex language platform libraries being usable cross languages: COM, WIN32, .NET are huge success stories. Java becoming the assembler for the JVM and Java libraries being used by many dynamic languages running on the JVM is another interesting case, Qt is rather good example as it being used as a widget set for other languages though the vast majority of Qt code is still C++. While I'll give them credit for this area of success I suspect though this isn't so much because of properties of OO but rather GUI libraries are the most complex libraries people use right now and OO does a great job of modeling GUIs.

As for OO and fads, here we do disagree a bit. OO is the most used paradigm. The failures of mainstream software today are mostly the failures of OO. Anyone who is using anything other than OO needs to justify why and they do so in terms of the areas where OO has failed and why those failures are intrinsic. There is a huge difference between being the incumbent and being the challenger. When OO was being praised it was being praised relative to C. What OO promised was to fix C's shortcomings. The big shortcoming it did fix, what it was able to model effectively, was complex event handling. And so I think it is far more reasonable to say that OO does not model "the real world" any better than functional or structural. Rather all of them model some aspects of the real world.

most parts of OO are here to stay, not a fad

Season's greetings. I like your points and hope folks read them. (I never mind name typos. Personally, I blame my father for having a goofy Irish surname. This is dry humor without a smiley in this case. Call me Rys, it's shorter.)

A political platform metaphor may improve clarity in what I say next. Things like OO programming are like political platforms, composed of lots of planks (varying in how closely related) where planks come and go over time, typically shedding weirder ones when they become a liability. But summaries of an old platform linger on long after like the smell of onions. You can make a similar metaphor about religious doctrine — politics, religion, and tech agendas have elements in common.

I can see how you and Sean filled in gaps to read me as saying OO was a fad, but I only meant to imply related ideas: 1) some planks of what folks claimed about object oriented programming were a fad, and 2) some planks of new upcoming tech are typically also fads.

Over the course of twenty or thirty years, things evolve enough that you can always find some old plank that now sounds quaintly ridiculous, even if other planks are now the accepted rational standard. I sometimes use a dubious rhetorical device of adopting the perspective of critics—possibly imagined—in phrasing my own views, since I'm not really wed to my own view, which I know is weird. Some folks seem to think OO was just a fad, and say a new broom will sweep it away, proved by (insert attack on one of the old weak planks here). I wanted to note benefits we get partially from OO style code we'll want to keep no matter what new tech regime becomes popular. Just like I mostly ignore politics, I don't care about changes in tech platform popularity.

I was tempted to write a dialog illustrating the idea of platforms and planks, featuring a conspiracy theorist named Rocko who claims a long list of facts are true, bundled together in one platform that must be accepted or rejected together as a whole. One of Rocko's planks is, "This apple is red," which is manifestly true, referring to an apple sitting in front of him. But some of Rocko's planks are dubious, like his theory about who shot JFK, and others are just false, like his theory that two plus two equals one. While this dialog could be funny, it's not worth the trouble. The moral of the dialog? A true plank does not make a platform true, nor does a false plank make a platform false. Discussion is more rational when each plank is treated like a separate line item. I'd prefer to see more line-item oriented discussion of planks in tech platforms, rather than voting them in or out as omnibus indivisible batches.

That said, I'm not going to contest anything you said, even though you mentioned COM, which is like waving a red flag in front of a bull I'm not channeling just this moment. I'll just grit my teeth and nod.

I don't agree with the basic

I don't agree with the basic theme that functional is somehow machine like while object oriented is human.

My point was that OO tries to model how we think, while functional tries to get us to think mathematically and logically. There are advantages to both, of course, but OO will continue to dominate the large grains of our designs even if we use functional within.

I totally agree about multiple inheritance. I'm a big fan of traits and even virtual classes. There is just so much to explore and take advantage of in the OO design space that I'm depressed about Java/C#, while Scala has great support for OO programming (via traits) that basically get ignored by its pro-FP community.

What is so fundamentally frustrating about object orientation as it exists today is that it doesn't deliver on its promise of being able to create models which correspond to human thought at all.

I would disagree in the absoluteness of your conclusion. OOP is still very accessible, given a blank slate and some object thinking, the average novice programmer can figure out how to get started in solving their problem. The problem with OOP is that these designs just hit walls way too early and to go further one basically has to avoid classes and go with something more ad hoc (but basically still OO).

thinking

Hi Sean. Happy New Year.

but OO will continue to dominate the large grains of our designs even if we use functional within.

I would agree with you that functional doesn't seem to be winning the large grain design war at all. It really isn't even interested in it. Functional is launching an attack from below not above: first class functions, lambda functions replacing for with map, replacing accumulators with folds, chaining methods together casually i.e. (1..10).to_s.reverse

The real issue between functional and object orientation is going to be on the issues of stateful vs. stateless. As web programming and thus distribution and parallelism become much more popular the cavalier attitude that used to exist towards state is disappearing. Once state management becomes a pain that developers always need to think about, selling state isolation won't be hard. If you have that and functional techniques: purity and higher order functions then object orientation just becomes like object orientation in a functional language, a way of organizing data.

My point was that OO tries to model how we think, while functional tries to get us to think mathematically and logically.

I understood your point, I was disagreeing. I think humans do think functionally. When I get my car I take my wallet, take my keys, take my coat. There is a notion of "taking" that crosses over objects. Similarly with mathematics. From a very young age children are able to understand that there is something about two trees, two block and two pants that has nothing to do with trees, blocks and pants. Mathematics and logic aren't alien to human thought they are a product of human thought. They are universal to so much of our thinking because they are embedded deeply into the very structures of human intelligence.

Now it also quite true that humans think in terms of things and they think in terms of classifications of things. Notions of "is a" and "has a" are intrinsic to the nature of human thought just as much as mathematics. But how humans conceive of things also involves context and complex hierarchies.

OOP is still very accessible, given a blank slate and some object thinking, the average novice programmer can figure out how to get started in solving their problem.

I don't know if I agree that OO helps here. I'd argue the big features that matter for novices are:

a) The right choice of first class objects
b) A really good GUI
c) Simplicity

Again I'll use Excel as an example, a very pure out of order execution functional language with almost no ability to model datatypes that probably on any given day, ten million novices are writing programs in. Or take Logo, a language that six year olds can work with.

Or to pick a more complex example: years ago, my wife, who doesn't know how to program, nor even knows what a device driver is, wrote a custom device driver in AppleScript. What made that possible was not that AppleScript gave her a natural paradigm from which to model, but rather:

a) It gave her access to the right set or primitives. First class objects in AppleScript was the device's actions in very natural terms.
b) The GUI allowed her to view these first class objects and experiment easily.
c) The rest of the syntax of the language is simple.

On the other hand if she had to see something like the Cocoa Object Oriented view of what was going on and think in terms of initiating a message from an NSObject to another NSObject she wouldn't have been able to figure out how to get off the ground.

State happens

If anything, state is becoming more important as we move to distributed and parallel systems. There is a lot of work in isolating the state, sure, and FP has played a large role here (e.g. MapReduce, even if the kernels are written in Java). But then we get new technologies like touch, camera, and speech where huge amounts of real state are required to process even the simplest thing. As we remove unnecessary state from some parts of systems, new necessary state keeps popping up.

When I get my car I take my wallet, take my keys, take my coat. There is a notion of "taking" that crosses over objects.

"I" am taking "things," I never think about the verb in isolation of the action doer, which is essentially what FP is. Math is not intrinsic to human thinking, it took us 50,000+ years after we learned to speak and 5,000+ years after learned how to write to even come up with it. It is incredibly useful, but the curb is steep and we generally take it for granted (forgetting how hard it was to learn initially). Yes, math IS alien to humans, and we must make a conscious effort to learn it (it can't be absorbed while young like language), and any new math we encounter requires learning (category theory and monads).

Again I'll use Excel as an example, a very pure out of order execution functional language with almost no ability to model datatypes that probably on any given day, ten million novices are writing programs in. Or take Logo, a language that six year olds can work with.

Excel leverages what people learned in College about math and accounting. Naturalism is simply about leveraging skills that are already their vs. forcing people to develop new ones. Logo takes what six-year olds know about movement and spatial relationships, and packages them up with a very simple command language.

On the other hand if she had to see something like the Cocoa Object Oriented view of what was going on and think in terms of initiating a message from an NSObject to another NSObject she wouldn't have been able to figure out how to get off the ground.

I'm not sure how we got into end-user programming here. When I said "novice" I was talking about a CSE 101 student who was learning the tools but still didn't understand how to design programs very well. So after we get over the hurdle of know syntax, semantics, and libraries, if you sit a novice down with an OO language, they'll be able to fumble out a solution, while in an FP language, they would have a much harder time dealing with a unnatural verb-focused world. This is just my hunch, perhaps it is something we can test empirically someday.

Math and naturality

Math is not intrinsic to human thinking, it took us 50,000+ years after we learned to speak and 5,000+ years after learned how to write to even come up with it.

How do you know this? We have counting instruments from 30,000 BCE. We have evidence of division from 10,000 BCE. And you see math with children all the time.

More importantly I'm not sure when something is invented is terrible important into naturalness. Children all consider cars to be quite natural as a way of transportation. We were early Amazon customers, so my teenage daughter picked up immediately the way was as a family got stuff she wanted was Amazon and EBay. I grew up in a world of giant department stores which existed in the post WW1 era and died during the Reagan era. I found that an entirely natural system of how my family got stuff. Humans are social creates and naturally adopt to their social framework.

As for math vs. language. We seem to have a language mechanism when young. Certainly I agree that we don't learn math like we learn languages at age 6. I do think language at age 26 and math are comparable. Riding a bicycle, history, playing a musical instrument.... are all comparative to math. Language is unique here.

"I" am taking "things," I never think about the verb in isolation of the action doer, which is essentially what FP is.

I'm glad we agree on verbs vs. nouns analogy. "Packing up things is necessary before going on a trip". I can put the emphasis on "packing". Now English sentences don't allow me to completely isolate verbs from nouns, but larger structures do. Your own post contains the example of "learning language" vs. "learning math". The confusion about "doers" you mention in your last paragraph came because you phrased programming based on the verbs of programming not the nouns of the doer.

Essentially all human societies have math
Essentially all humans when they encounter math find it useful and all learn some
Even humans who are bad at math, ask question best answered by math

I'll agree though that math can be hard.

But then we get new technologies like touch, camera, and speech where huge amounts of real state are required to process even the simplest thing. As we remove unnecessary state from some parts of systems, new necessary state keeps popping up.

I'm losing you here. I'm not following the connection of these examples to statefulness.

____

The rest of this response is in the CS 101 post. Not sure why that one outdented.

How do you know this? We

How do you know this? We have counting instruments from 30,000 BCE. We have evidence of division from 10,000 BCE. And you see math with children all the time. More importantly I'm not sure when something is invented is terrible important into naturalness.

My point was that language evolved, its an intrinsic capability of the human brain that arose from millions of years of evolution. Math is not the same, its something we had to think about and invent. If you don't teach your kids math, the best they'll be able to do is count to 10 (b/c we only have 10 fingers). Its only through heroic effort of a few very smart individuals learning and building on each other's work that we've gotten to where we are today with math and logic!

ssentially all humans when they encounter math find it useful and all learn some. Even humans who are bad at math, ask question best answered by math.

I'm not disagreeing with you and I'm not anti-math. But developing an unnatural thought model (which is essentially what language is) based solely on math for communication seems neither useful nor viable (we can always apply math when we need to). Objects are then naturalistic entities that we can use to encapsulate math vs. using math to encapsulate math (as is somewhat preferred in FP).

I've seen people who can encapsulate math with math naturally, take SPJ for example, and they definitely think in ways that I can't really comprehend. I find that quite fascinating, but if you are already there you might not realize its anything very special or unusual. PERHAPS it is a next stage in our evolution, and I'm just resisting it. Or perhaps not. Anyways, that would occur over a million years, and by 2020 we'll just still mostly be using objects. (Perhaps we can speed it up by allowing only good Haskell programmers to reproduce, giving a little push to natural selection)

Does OOP really limit the model scope?

I agree that OOP doesn't allow all aspects of a person to be modeled in a single object so how would a RDMS model it? Some data of a person would be associated in an object collection with a unique (person number) and then that number would relate any other objects in any number of other object collections to that object. In this way, you can access traits as a group or individually and also relate them to any number of other traits in other object collections. (Think of a customer table related to invoices, orders, shipments, phone numbers, AR etc)

I intentionally used "object" and "object collection" instead of rows and tables but the idea is the same.

There is nothing in OOP that says that objects have to behave or exist on their own. RDMS seem to get along just fine by just duplicating a column between 2 or more tables.

I personally like multiple inheritance but how easy is it to find the code that needs fixed when it isn't obvious or easy to even list all the code that is being used to change the data?

What is most important about FP to you, is it the "no side effects", the math object libraries, or just the familiar Math terminology for standard CS concepts?

CS 101

When I said "novice" I was talking about a CSE 101 student who was learning the tools but still didn't understand how to design programs very well. So after we get over the hurdle of know syntax, semantics, and libraries, if you sit a novice down with an OO language, they'll be able to fumble out a solution, while in an FP language, they would have a much harder time dealing with a unnatural verb-focused world.

Let me start off to objecting to CS101. CS101 is a weird intermediate state. It is a very select group of people at a specific time of their life which cannot last. Either they learn the material and aren't CS101 people or they don't and have little need to program in a few months. The profile is entirely different if you pick CS201. Kids who are good at decomposing problems into precise steps and find they like it all find math rather natural. Almost without exception the people who liked CS101 enough to take the next year's course are reasonably naturally talented at programming and reasonably naturally talented at math. So when we start talking CS101, my first question would be why aim at that group?

When I work with someone who is good at even languages that don't appeal to people with lots of formal mathematical education, like Quark Express's Workflow Automator (professional desktop publishing automation ) almost without exception the graphical designers that like to abstract the process and think in terms of those abstractions like abstraction in most other areas of their lives. They are natural facile with mathematics, even when lacking formal education in it beyond the high school level.

CS101 is a small group in a short term state. Obviously we have a goal of having CS101 be as successful as possible in getting people to program. But talking about CS101 is very different than talking about the overall human condition when it comes to human machine interactions.

A far better argument of what you really mean would be if programming were mandatory and we were educating everyone in it would functional, structured or OO be the best paradigm? And there honestly I'd go structured. Most people are terrible at any kind of modeling but fairly facile at making lists of things to do in order. Toddlers understand the notion of a set of instruction and by about 5 they've mastered sequencing. 31-35 years ago when we had a world where every personal computer user knew a little programming, the language they knew was BASIC. I mentioned AppleScript as a similar language for Apple's products. HTML seems to be currently understandable to almost 1/2 the population that uses computers. Most macro languages are structured...

This is just my hunch, perhaps it is something we can test empirically someday.

I would argue we are testing this empirically. We have thousands of language domains where new languages are being created. Some of these languages are successful other are not. The successful ones spread to new domains where they compete with existing incumbents. That is a a lot of data about the three paradigms and how end users doing complex work handle them.

And the results IMHO is that it doesn't seem to matter much which paradigm you use. Which paradigm fits is highly dependent on the problem domain. To pick an example that is the exact opposite of what you are claiming would happen:

AutoLISP / Visual LISP as a way of modeling product manufacturing is so popular that most of AutoCAD's competitors include it as well. Pretty much anyone who is good at CAD learns AutoLISP. Engineers at least did find it easy to use mathematical style modeling. ObjectARX which is in the same space and does the same thing however is a purely professional product, people who do ObjectARX only do ObjectARX all day. So here the OO language migrated towards the complex professional only highly specialized while the Functional Language became the domain of the average enduser because the simplicity of mathematical modeling and the complexity, in this problem domain, of object oriented modeling. When you have a dozen parts interacting: AutoLISP, when you have a thousand parts interacting ObjectARX.

Which I think is what typically happens. For models in which the number of things interacting is huge OO shines. As the relative complexity of their interactions increases FP. If the interactions are simple and the number of objects manageable but the steps are complex, structured.

Lots of things doing simple stuff = object oriented
Few things doing hard stuff in a few steps = functional
Few things doing lots of simple stuff in complex sequencing = structured

But this comes down to the issue of naturalness.

Almost without exception the

Almost without exception the people who liked CS101 enough to take the next year's course are reasonably naturally talented at programming and reasonably naturally talented at math. So when we start talking CS101, my first question would be why aim at that group?

There are certain hurdles with today's programming environments that require some amount of significant investment. If we discount this investment, we can only talk about the few low-investment/end-user programming environments, and there aren't enough data points to make any very interesting conclusions.

When I work with someone who is good at even languages that don't appeal to people with lots of formal mathematical education, like Quark Express's Workflow Automator (professional desktop publishing automation ) almost without exception the graphical designers that like to abstract the process and think in terms of those abstractions like abstraction in most other areas of their lives.

I would argue that languages like Workflow automator are based on naturalistic flows, recipes, and highly encapsulated noun-like autonomous entities (dare I say objects). I'm not sure what sort of advanced math they need to know beyond what they picked up in elementary school.

A far better argument of what you really mean would be if programming were mandatory and we were educating everyone in it would functional, structured or OO be the best paradigm? And there honestly I'd go structured.

Structured is quite orthogonal to functional or object-oriented. It only emerges independently of the two paradigms because the alternative of unstructured programming was actually popular at some point. Of course, we can have structured without OO or FP, we would often call those languages modular or procedural (though even those concepts are arguably quite orthogonal to OO and FP).

AppleScript is very OO even if it doesn't have much support for programmer-defined abstraction. I think we are splitting at hairs on definitions. HTML is markup and also a bit OO and FP if you look closely (and of course, you need to couple it with JS to do anything complicated).

I would argue we are testing this empirically. We have thousands of language domains where new languages are being created. Some of these languages are successful other are not.

I think you are misunderstanding what I mean by testing empirically. It means setting up a controlled experiment to test exactly one factor of a language and eliminate influence from the other factors. In the real world, we don't really have such luxury; a language is never just one factor. We have feelings for why languages are successful or not, but to be honest we really don't have much clue to what is really going on. Check out some of Stefan Hanenberg's work for more of an idea on what experimental PL really means. Also, getting good empirical results becomes immensely more difficult as user investment increases, meaning its not easy being a usability researcher at AutoDesk.

Structured

I'd considered structured languages to be languages that aren't particularly object oriented nor functional. In reading your responses you tend to have a very open definition of object oriented. I would tend to define it like this

Joe takes the ball
a) If you need to develop a ball, a Joe, and a Joe.takes() method then OO
b) If you need to develop a takes method capable of handling Joe and ball then FP
c) If joe, takes and ball are 3 unrelated entities tied together by a line of code, as per the english sentence then Structured

Structured languages I'd include C, PHP, Perl and Python... Which is where I'm grouping workflow. If you are defining what I'm calling structured as Object Oriented then yeah I would agree that most people find structured / OO easier than functional.

_____

As far as testing empirically and then only testing one factor... I think those things may be useful. But it is hard to imagine FP and OO as just one factor. You and I both agree that they encourage different ways of thinking about reality. Very quickly that induces different responses to reality which creates different cultures.

I've never heard

I've never heard "structured" used like that before. I assumed you were taking the term from Dijkstra's famous "goto harmful" paper. So "structured" is simply the use of if, while, and subroutines, which covers just about any modern language that has some sort of function/procedure construct.

(c) is definitely object oriented. As long as you have "Joe takes ball", there is a subject, a verb, and a direct object. That is to say, natural language is very object-oriented, and that's the whole point of what OOP is trying to harness. Just as the point of functional programming is trying to harness the power of the lambda and mathematical thinking, or logic programming is trying to harness the power of automated reasoning.

structured

I'm using it the normal way, where C++ is considered object oriented and C is not. Where OO COBOL is object oriented but COBOL is not. In my book Object Oriented requires: polymorphism, encapsulation and inheritance. Object orientation wasn't an attempt to capture natural language, COBOL did attempt that decades before anyone had heard of Object Oriented.

I don't think I'm outside the norm in seeing Object Oriented as an attempt to control organizational complexity. When we moved from programs that were command line or batch oriented to GUIs the complexity of event programming vs. simple I/O required a paradigm shift in software construction and that was the problem that OO was meant to solve. OO wasn't harnessing natural language. C++ is arguably less natural than C. Objective-C / Cocoa and Java are morasses of language so unnatural that even experienced programmers in those languages have to constantly be looking things up to make sense of code.

I see. I will just disagree

I see. I will just disagree then on the meaning of structured, and any taxonomy scheme is going to have lots of overlap (e.g. Java is structured, imperative, object oriented, procedural, ....). COBOL was about natural language syntax, not thinking, and hence it comes off as very unnatural.

Objects should be supported with identity and encapsulation, but polymorphism isn't necessary at all. The only real requirement is that objects support object-thinking, which is just thinking about program organization in terms of nouns before verbs. The only reason I call this natural is because its how most of us think and speak (some people can think in terms of verbs and pure abstractions, but they are sort of rare). That's it, nothing you listed is a necessary or sufficient property of OOP.

A few comments and predictions

In 2020, 99% of programming will be conducted mostly by relatively young people with minimal, insufficient technical training and questionable ability to complete a more comprehensive training regimen. These developers will try to cobble together "solutions" as cheaply and as quickly as possible even though they have minimal or no knowledge, and even less interest, in the problem space. Just like today.

If we go back to the claims made in the late 80's and early 90's about OOP by the methodology gurus to the suits looking for silver bullets and to the young folks looking for the latest and greatest, OOP has been a "failure." The most common limited means of classification (single inheritance isA), the common lack of sufficient constraints on implementations of virtual methods, and on and on and on provide a poor set of tools for modelling most real world realities or abstractions. Even armed with superior tools for abstraction, classification, constraints, etc., carving up many or most real world problem domains into general purpose (broadly reusable) entities into "class libraries" is far too difficult or expensive a task for most individuals and organizations. Just try attaching a gadzillion important attributes to the so very common and important "nation state" or "country" abstraction and persisting this data for historical analysis and reporting - even though these abstract entities disappear and spring into being on a regular basis. A seemingly simple object/record representing a country, it's population and GDP, and direct foreign investment in that country over time is not really so simple or effective a model of reality :-/

That said, judicious use of OOP style abstraction (just for example, tight concrete data types - like a complex number class) is delightful. The "implementation inheritance" and "open recursion" model of virtual methods works well for GUIs and some other "fill-in-the-blanks" styled "OO frameworks." Sure, why not?

But basic functional language features and concepts will continue to gain currency, particularly as 1) low level language feature performance becomes less important in more domains (faster hardware, more memory, ever larger and faster runtime systems and vast libraries written in other languages geared for performance) or 2) more language compiler implementations feature deforestation style optimizations, better handling of unboxed native data types in parametric polymorphism implementation (ala .NET) and so on - perhaps made simpler or more effective by other language features in addition to fancier compiler trickery.

At a very basic level, writing safer code that creates new data structures from old ones (via map, comprehensions, etc.) instead of mutating old data structures (set-cdr!) will become much more commonplace.

These trends are already well underway. What I think will be much more commonplace in 2020 compared to today is that more programming languages will enjoy more features associated with structural typing, "duck typing" and so forth. This might be via C++ template style mechanisms or via type class (or Scala implicit) style dictionary passing or via some kind of structurally typed mixins or who knows what. But there will be more emphasis on ad hoc function/data abstractions, multiple classifications and general adaptations in a type safe manner.

Another much more commonplace "mainstream" language feature will be sytactic abstraction via "macros" ala Scheme or Nemerle or whatever. This will allow for things like more type safe SQL integration, formatting, safe and convenient representation of XML or JSON data, and, of course, new binding and control structures in carefully crafted libraries. Use of sytactic abstraction will require great skill and taste to use effectively, and yes, most programmers will abuse this feature, using it poorly and pointlessly.

Just my 2 cents.

Provenance in 2019

We won!

Provenance Today

Provenance is everywhere:

– It’s part of every storage system meta-data. – It’s part of every application.
– It’s secure.
– It’s queryable.
– It’s searchable -- globally.
– There are PA-search algorithms.



Units could improve programming.

Numeric Type systems could be extended to include units, including user-defined units. This would allow checking the usage of numbers derived from mathematically manipulating unit-denoted quantities against production rules predicting the units of the results of those operations.

Usually when we want a value of the _type_ number, we have a specific idea in mind of something which that number _means_. Often it's some quantity that can be expressed in terms of _units_. Having type systems that understand standard units could be a big help to discover errors via static analysis. For example, if you divide an acceleration by a distance and try to assign the result to something that's declared to be a duration, your program fails typecheck, because an acceleration divided by a distance gives one over the square of a duration, not a duration. Also, if you provide the arguments in the wrong order, static typecheck can fail immediately due to actual/formal unit mismatches instead of creating code that contains an undetected error, which it would if both quantities were simply of type "real".

Also, it could help with bug-free interoperability and code reuse, to the extent that incompatible units and erroneous conversions are a source of problems. If someone has defined a routine that takes a distance and a duration as arguments, then it shouldn't matter to the caller what units the routine was defined using. You get to provide arguments specifying *any* recognized distance and duration units you find convenient, and a correct autoconversion to the units used by the routine can be added by the compiler (or even statically avoided as much as possible, when a reciprocal autoconversion would apply to passing back the result).

Just a thought....

Ray Dillinger

d'accord

hence things like frink and units in f#. it is just that all the other languages are lame.

Frink, units, numbers

I'm coming a little late to the party, having just seen this discussion in my server logs, but my language Frink was indeed designed from the first day to have units of measure tracked with all calculations. Frink also has an extensive standard data file which means that you can just use many common units of measure and physical constants, and facts about the earth, planets, particles, etc., directly without going and looking them up. You can often cut-and-paste physical equations and quantities into Frink directly and they'll just work right. (It tries hard to use correct orthography and mathematical notation, including "invisible" multiplies (e.g. x^2 + 2x + 4)

I've found that it's pretty important to have units of measure as a built-in, first-class language feature. If units of measure are a bolted-on afterthought, it's quite likely that most existing libraries won't be able to use them at all without a lot of rewriting, if it's possible at all.

I noticed that Mathematica 9, released recently, was proud to finally have their units facility work throughout the language, as opposed to being an afterthought. (I dunno if it will still happily let you write "Meter + Second", though...) The Fortress language had planned to implement units of measure, too, but that project wound down recently.

What might not be immediately obvious in Frink is that it tries very hard to not just do the right thing with units of measure, but with numbers also. Its numerical type will automatically do the right thing with (arbitrary-size) integers, arbitrary-precision floating-point, exact rational numbers (of any size), complex numbers, and even interval arithmetic. This eliminates another common, silent source of error, and eliminates the programmer having to micro-manage the representation of every number at every stage through calculations, worry about silent overflow, etc.

I'm going to be very optimistic and say that in the year 2020, any language that doesn't trivially and by default do basic mathematics to the level that you learned in elementary school (e.g. rational numbers, arbitrary precision, no silent overflowing, and knowing that 1/2 does not equal 0) shall hopefully be looked on as an archaic mistake. It'll be much more important to Do The Right Thing with numbers than Do A Very Wrong Thing As Fast As Possible.

Surely not ALL calculations...

... my language Frink was indeed designed from the first day to have units of measure tracked with all calculations.

All calculations? Seriously, all? People do occasionally calculate things that aren't measured in a particular kind of unit, don't they? For example, percentage of problems complete, or test scores, or a count of the occurrences of the letter 'e' in a corpus of text or the ratio of e's to l's.

I'm with you about doing the Right Thing instead of doing a Wrong Thing Faster, and I'm convinced that units have value -- but I'm not at all convinced that units are beneficial or even appropriate for ALL calculations.

Ray