Lambda the Ultimate

I don't really see this happening; there is just no easy way to express design tweaks and layout in a client-independent way. To be honest, I don't see abstraction as getting much better in the future unless we are talking about something like machine learning/AI.

Actually, my vision was that the exact layout and styling could be parameterized and compiled per client and device, or even per session (if the medium allows). There would be default policies and templates for a variety of common use cases, of course. But developers could hand tweak rules as needed.

(Or a good DSL could even enable clients to tweak layouts and share them.)

Even if no code at all is shared between different platforms (unlikely) then it's still a big advantage to have a single language and standard library that targets multiple platforms. Currently if you want to develop for ios, android, desktop, web you have to learn 4 different sets of languages, tools and libraries. That is a huge time investment to make and it would be valuable to get the ability to write for the other platforms for free if you've mastered one.

Is the closest thing to this today Heroku-style PaaS?

Haxe, Curl, Ur/Web, and GWT are close on the languages side. (And there are probably more I don't know or don't recall.)

And I'm currently thinking about how to support it from Haskell - albeit, mostly in context of my Sirea project. I think I could get a lot more bang for each line of code if I focus on app servers rather than direct use of keyboard/video/mouse/sound - and that, with a little careful design and abstraction, I could still effectively support the 'local canvas' (wx, local audio, OpenGL).

I wouldn't be surprised to see such idioms in other apps - i.e. if you can 'serve' a web page, why not also 'serve' a wx form? A sort of parametric cross-compilation to different client models need not be very difficult if you have a good abstraction for API (and client parameters and extensions).

Today, the main limiting factor seems to be that most languages don't have effective support for partitioning or code distribution - i.e. to allow some parts of a program to run on the client, and other parts on the server. The secondary limiting factor is that our code often is too tightly bound to a particular GUI model (e.g. containing fragments of HTML rather than semantic descriptors, or deep integration with Win32 peculiarities). The third limiting factor is that HTML was historically bad at a lot of things GUI shells were good at, and vice versa - but that distinction is diminishing with the help of libraries, plugins, new HTML 2D and 3D canvases.

PaaS isn't at all what I was describing, though it also has value and may serve as a useful distribution mechanism. (I wouldn't be surprised to begin seeing 'meta-clouds' that run on multiple providers, switching based on time-variations in price and risk-management policies.)

I'm not sure about a DSL, but application servers is a decent call. We already have plenty of software "appliances," apps running on application servers, and virtualization uses for management & security. I think these can be further expanded to create well-managed app instances with small TCB's. The Mirage project is trying to do that in Ocaml and Barracuda inadvertantly does it for Lua app server for embedded use. (That they run their whole web site on their platform and a plug computer is pretty cool.)

Then, there's the MILS offerings and microkernel projects that let you choose isolated apps on microkernel, standards compliant runtimes (e.g. POSIX) or VM for legacy OS/apps. Great for TCB and gradual decomposition. I could imagine taking say a DNS server, putting it in a VM on one of these platforms, and moving its functionality into isolated protection domains piece by piece. Eventually, invisible to user, you have greatly improved modularity and assurance of the software without having to rewrite it all at once.

I will buy the DSL thing too. Furthermore, I predict that the DSL will be derived from a vendor-designed hack on top of JSON (or whatever the 2020 equivalent is; it could be anything but XML), and by 2020 we will still be hacking around implementation-specific quirks despite having a W3C or ISO standard for it.

(Incidentally: The implementation-specific quirks probably won't be around multitouch interaction. A model for multitouch interaction will be standardised by 2020, much like the model for mouse interaction is standardised today, and implemented in all the major UI toolkits, DOM and the application DSL. The same will be true of 3D displays, assuming they are deemed sufficiently useful for general-purpose user interfaces. There will, however, be a new fad in input or display technology which the standard can't handle, and this will result in multiple competing extensions.)

I also predict that by 2020, NoSQL databases will have converged on something close to a de facto standard (which will also have implementation-specific quirks that must be hacked around), but there there will be native support from all major vendors for this DSL, enabling tighter integration with application servers, at least in theory.

I think you're right about virtualization, too. By 2020, hardware virtualization will be ubiquitous to the point that it's done for each application; spawning a new virtual machine will be as easy as spawning a thread is today, and far easier than MPI. Moreover, there will be programming support for QoS in these virtual machines, so that (for example) the OS could slow down a core or relax the hardware exception model to conserve power if the program doesn't need it.

While increasing heterogeneous and concurrent architectures will certainly impact languages and frameworks (and there is much progress there already, e.g. to leverage clouds or GPUs), I'm more interested in how things like Project Glass, LEAP motion, Emotiv, and Touché might impact programming.

Project Glass could provide pervasive access to eye-cams, voice, and HUD feedback. Something like that would be a suitable basis for pen and paper programming, or card-based programming, or various alternatives that involve laying out objects in the real world and clarifying with voice (perhaps non-speech voice).

Well, GPUs should become more powerful and (carefully) more general purpose. But to keep their edge over CPUs, they'll need to maintain lower hardware abstraction (less caching, less virtualization). CPUs should remain similar to how they are today but less relevant as GPUs continue to provide an order of magnitude better performance in many data-parallel applications spaces, which will just continue to expand. Heterogeneous hardware will probably mean a mix of CPU vs. GPU nodes (this is beginning to happen now).

I disagree slightly on what multicore CPUs will look like. I think they'll be mostly homogeneous (some specialised functional units may be different, I agree), but the power use of each won't be static, it will be dynamic. Power can be altered by the OS by altering the clock speed, or dynamically controlling the cache size, or switching between out-of-order and in-order execution, or something else not yet thought of.

Of course, the inevitable result may be a virus which destroys your CPU by overheating it.

So, basically, C++11, then? ;)

Sure, except for the type system, tooling, and library bits. C++11 is a good step forward from C++03 to be sure, but as an “expert” in C++ I’m rather strongly biased against the language as a whole, cool new features be damned. If you already know it, then it’s a powerful tool; but the effort required to adequately learn it could, I think, be better spent elsewhere. And to be perfectly honest, I’m also biased toward the features I mention because they’re essentially the principles of my own language-in-progress.

I agree completely, and my tongue was firmly in cheek. C++ remains, unfortunately, extremely complicated, and all attempts at simplification-via-augmentation have so far had exactly the expected outcome.

I'm not actually sure I agree about tooling and libraries, though. I've had surprisingly good luck with Eclipse for the past couple of years, and I've gotten a lot of mileage out of just the standard library and Boost. But of course it's still a far cry from the situation with, say, Java.

The somewhat serious point behind my joke, though, is that the particular combination of features you named has yet to be gotten into a comprehensible form. In particular, a "heavily functional style" and "deterministic memory management" are pretty hard to reconcile, and even harder when you still want an "accessible type system." (Compare, for example, ATS, which is quite impressive but very far from accessible.) If you can achieve something in this area, I will be your biggest fan.

Yeah, the STL is great. The strength of C++ is really in its generic programming facilities—though iterators leave something to be desired in the way of composability. D-style ranges help, but have problems of their own. That aside, the real problem I have with C++ is plain usability. Even with good error messages and code navigation tools, it’s remarkable how often work gets interrupted by the need for careful thought about how to navigate some trap or another.

Thanks for the link to ATS. I hadn’t heard of it before, but at first glance I agree with you: it’s impressive—half because it seems genuinely useful, and half because it spares no complexity. As for my own language, there’s plenty yet to be done in the way of polish, so don’t hold your breath. ;)

I'm curious: you mention you expect lazy evaluation to be overrated. In what contexts is lazy evaluation frequently used today where strict evaluation may be a generally better choice?

Even the defenders of specific lazy functional programming languages seem to think that laziness is mostly a means to a end, and the end is purity. Me, I'd rather be able to write mostly-pure code in Scheme or Pure, introduce very careful doses of impurity, and have the intuitive advantages provided by strict reasoning.

Also, giving the power to the programmer to determine the evaluation strategy seems like a natural modularity need for larger and larger pure functional systems.

Thanks! Playing with the evaluation strategy for a system certainly sounds like great fun. I'll have to look into dependent types.

In lazy functional programming languages ;-) Here are a couple of reasons why I think lazy evaluation will not be victorious:

• Most programs do not make much use of laziness but laziness still makes reasoning about performance and memory hard.
• If a program does make essential use of laziness for efficiency or termination then it's clearer to have that marked explicitly in the code and types, which is exactly what strictness by default + explicit lazyness gives you. It's telling that Okasaki does this in his work on functional data structures. This is similar to explicit effects with monads or an effect system vs implicit effects.
• Laziness doesn't play well with parallelism.
• Laziness gives you a uniform slowdown on your program which even very smart compilers (GHC) can only partially undo.

And, as the people above have mentioned:

• Even Simon Peyton Jones has said that laziness mostly mattered for keeping him honest about purity. If he could wave a magic wand to make Haskell strict he'd probably do it.
• No laziness by default doesn't necessarily mean strict by default, e.g. total languages and separation of data/codata in dependently typed languages

Fair enough =X

Thanks for the detailed response. It looks like I have a lot of reading to do.

Laziness doesn't play well with parallelism.

I'm curious about this point. It would seem to me that programs written with no particular order of evaluation would be a huge advantage for parallelism. Would you be willing to expand on what you meant?

I nodded when Jules said, "Laziness doesn't play well with parallelism." So I'll take a stab at what it means, though Jules might mean something else. I read it as meaning, "Every kind of ambiguity can cause problems." When you combine subtle code A with subtle code B, you can get subtle squared, which can be slow when code only barely manages to figure out the wrong state is currently in hand. I can give you a concrete example, which even involves laziness as a factor.

Suppose you write a page cache, which I prefer as i/o over memory mapping (specifically so you can schedule i/o without being at the mercy of a VM flush daemon). When a page cache has dirty pages, you can write them eagerly or lazily. But an extreme of either can cause higher latency. Only so many iops (i/o operations per second) can be had, and when the pipe is full you don't get any more. Roughly speaking, any time you schedule a spike of i/o greater than iops available, you get latency. So the best time to do lazy i/o is when you're not busy, so work backlog is lower when you have a rush of activity. But scheduling that's a pain, because it doesn't correspond to specific events observed in code during normal operation. You actually have to schedule, ick, instead of responding passively to outside events.

Consider one extreme. Suppose you never write a page until no clean pages remain and a new page must be read. So you write a dirty page, then fill that newly clean page with the read you wanted. Result: two i/o operations (write then read) every time you just wanted to read, with less than ideal latency. That happens because recognizing need to write occurs at the last possible instant, when it's too late to take advantage of earlier free iops. Planning ahead can make resource scheduling more efficient.

In general, the sooner you know something will be necessary, the better job you can do satisfying need with available resources. If you can't find out in advance, because laziness defeats ability to predict, you can get caught on the raw side of costs. Thus anything that delays finding out the score can result in (possibly much) higher costs. Code can keep saying, "Wow, I wish I had known this before." That's what I guess was meant.

Good example, lets work it some more.

So assume we have a paging algorithm (P) I'm assuming the system will be something like:

1) Store pending reads from threads as soon as they come in.
2) If there is no memory available write prioritizing LRU (least recently used)
3) If there is memory available and pending reads, drop written LRU data and read.
4) If there is memory available no pending reads, then write prioritizing LRU. That way these pages can be dropped if needed.

(4) is different than your example but I think is more realistic. Now we'll let PL be a lazy version of P and PE be an eager version. I'm hard pressed to see how PE and PL will be any different. You obviously can't drop memory until is is written: so 3 must come after 2. Any system is going to want (1) to go first. And 4 is lowest priority in both.

______

OK so assuming you agree PL and PE are the same (both P) now assuming L is a lazy program using P and E is an eager program using P I'm still having trouble seeing the problem.

For L the situation is going to be something like

L1) Grab a computation
L2) If you need data from memory get it.
L3) If the memory isn't available (paged out) request the page from P (indirectly) suspend the thread and move to a new one.

Which is going to look exactly like the eager system's E1, E2, E3. As long as the lazy system keeps enough threads going it will be more efficient.

So I'm stuck seeing the problem. I may be being stupid here but I'm not getting it.

I don't mind paging algorithm analysis, but I only meant to illustrate a specific concrete example where maximum laziness can increase latency by not using ephemeral resources when available earlier. (Since I haven't said anything at all yet to predict how programming might change over the next seven years, I should try to squeeze in some. Note I think it will change only slightly, in directions where obvious pressure leads.) I think laziness can make it harder to schedule tasks on available parallel compute capacity.

In some ways paging isn't the best example of a problem under laziness, because write backlog is clearly visible pending work, and the best example would be something you were unsure needed to be done until demanded. I only picked that example because I saw both those extremes happen in a prototype, showing a time-based scheduler was needed to get optimal i/o in a caching-oriented app I won't describe much.

Here's an aside on memory mapping. A few years ago we shipped product using it, then QA folks found interesting latency blips under heavy load. At first we thought user space locks were at fault, because we could see—shudder—locks held up to several consecutive seconds. But further analysis revealed the OS flush daemon got behind, and the OS decided not to schedule a user space process for several seconds (exclamation points go here) causing all network traffic to back off because our TCP stack was in user space. The phrase "dead in the water" comes to mind. Online searches revealed that was a common fate for anyone (e.g. databases) doing heavy i/o with memory mapping. So I said: with a page cache we can do i/o when it suits us.

A first page cache prototype I wrote to get numbers justifying further pursuit had a problem of being either too eager or too lazy. Unless I made an effort to write dirty pages, they tended to accumulate and later cause the double i/o scenario. But simplistic efforts to write sooner caused eager writing — far earlier than necessary — with similar high i/o latency. But numbers were great when write backlog was not in the way, so folks loved the demo, even given effects of my dumb first-pass i/o scheduling. (Load balancing tasks gave another dev the storage daemon problem, so I don't have fresher full analysis than that.)

If you want better examples with parallel lazy tasks, it might be better to construct cases with realistic layering and need-to-know barriers from abstraction, impeding optimization when global visibility of facts is obscured. It's easy to create situations where the left hand in code doesn't know what the right hand is doing. A centralized page cache is not hard to optimize because you can see the score. A better hard problem example would be one where layers don't disclose what they're being lazy about.

Note it's important to try things and see what happens, to validate theory. (And theory is always needed to explain what's happening, since otherwise it's hard to construct and test a hypothesis.) A rapid feedback cycle is very helpful when getting empirical data about ideas. It would be nice if folks conducted more arguments by modifying code and seeing what happens, instead of counting angels on heads of pins. So I hope there's progress in live coding efforts that help devs try things more quickly.

I'd love to see folks write more prototypes in daemons you can run experiments on by twiddling a proc-file-system style api via browser. That would be a way of getting more technical benefit out of browsers. If I predicted that was going to happen (and I have no idea really), it would be a change in development process that only indirectly has an effect on programming. I guess I expect changes in programming to be driven by changes in context of development, such as hardware. For example, the last several years saw a lot of 32-bit apps and servers become 64-bit instead, with subtle changes in what one can do given a bigger address space (and maybe haphazard system support). We're not going to 128-bit in the next several years, but something else in hardware will change.

I agree with you on the testing issues.... "In theory there is no difference between theory and practice, in practice there is".

Getting the to the example though For layers to work each layer has to make sure the next layer has work to do. Lazy or eager not keeping sufficient work at all layers is bad design.

A better hard problem example would be one where layers don't disclose what they're being lazy about.

Which is what I mean. I don't see how laziness vs. eager makes this any worse. An eager function that doesn't say what it is going to need next is going to suffer from high latency and worse is going to have a harder time keeping the lower layers flooded.

Just out of curiosity, are you arguing eager and lazy algorithms always have the same effect, so there's no reason to distinguish? I suspect you don't mean that, but I have trouble pulling it out of your subtext. If I define eager as "try to do it sooner" and lazy as "try to do it later", then an eager algorithm submits work to be done early while a lazy approach only submits it later -- in some cases as late as feasible. If that's what makes load known and fills parallel pipes, then eager does a better job of that.

I strongly disagree that verb "test" is a synonym with verb "experiment", so your summary "testing issues" makes me wince. I don't want to fight about it though, unless you do. :-) I have noticed devs sometimes prefer to struggle along in ignorance rather than allow themselves to get testing cooties, out of fear of lower social status, which can lead to broken systems. Being effective means getting your hands dirty a lot.

I see where we are disagreeing now. I'd define more this way:

eager code = order of evaluation is intrinsic to the code. Unless specified otherwise all values are computed fully by first use.

lazy languages = most code just defines a chain of dependencies, how to resolve the chain is left up to the system.

So parallelism in lazy languages works like connection pooling in database. That is lifting lazy code into an evaluation monad which litters the code with "sparks" that threads can just grab sparks and resolve the vast bulk of the language easily.

In the case of layered situation or any other kind of constraint you simply change the evaluation strategy in the evaluation monad. I don't see how having code that is flexible about reordering can ever present complexities for parallelism over code that is inflexible about reordering. I can easily see how for example memory usage can become much much worse but I can't see how CPU can be assuming the programmers aren't just being stupid.

___

As for testing, yeah I meant testing as "experiment". Testing in the QA sense I don't see as particularly useful for making these sorts of judgements. Either the performance is hitting Amdahl's law targets, or the algorithm is wrong, or the parallelism model is. I don't see a need for QA trickery helping for the stuff we are talking about. Once you are hitting those targets and want to make sure nothing can knock you off them then QA trickery is to be desired.

I don't think that's how most people use lazy and eager. Lazy means 'as needed'. Eager means 'as soon as possible'. Order of evaluation is a separate issue. You could express something that has no particular order and the compiler might still implement it eagerly.

I agree the compiler in a lazy language is entitled to execute any code it wants eagerly. That's what happens in Haskell, which is the archtype for a lazy language. Similarly I think "as soon as possible" is something of a misnomer. In eager languages many computations are possible at any given time. Which ones get done when is the programmer's call in an eager language.

So I guess we do disagree I see the two as very related. But I do appear to be in the minority on the definition of lazy. What are lazy languages that guarantee all computation are only done when needed?

"Lazy" usually denotes the call-by-need evaluation strategy, whose implementation involve allocating thunks for delayed computations and, at access time, either checking if the thunk has already been forced and returning its result, or forcing it and mutating it (atomically) to put the evaluated result.

This is an implementation that is inherently based on side-effects, and if you want to avoid duplication of computation (as it is specified in the semantic), in a concurrent setting you would need to synchronize on thunk update. This make it a relatively bad evaluation model for parallelism.

Haskell, for example, emphasizes a *non-strict* evaluation strategy, meaning that they don't commit to call-by-need and reserve the right to duplicate computations when it helps on other aspects. In practice the implementations are call-by-need, with optimizations to call-by-value when they can prove it won't introduce non-termination. The future vagueness is fine in Haskell as its a pure language (you couldn't observe duplicated computations), but would be problematic as soon as you add implicit side effects.

Agree on thunk. This strategy implies that the author of the code can't determine when his functions are invoked or, excluding dependencies, in which order. And that's where the advantages lie.

Agree on Haskell. As far as side effect, let me just point to another example that is widely used. Excel is lazy, cells are recomputed when the Excel engine determines a cell needs to be and in whatever order the engine wants to recompute. There are no guarantees that duplication will be avoided. OLE embedded Excel is a side effect (it is essentially a monad on the spreadsheet). The policy on OLE is that duplication may occur, and is perfectly legitimate. The invoking application is entitled getting a fully resolved current state when they make the call. The invoking application is not entitled to any promises of state being maintained. So for example if Word documents A and B both make on OLE call against spreadsheet C, A can induce side effects which impact B's next read, and B can be unaware of this. And this setup is something that even fairly low skill end users seem to be comfortable with and understand.

I would agree thought that a lazy system that does a strict call by need with a promise of never duplicating side effects would create a whole other layer of issues about parallelism. If that was the point OK, but I'm hard pressed to even come up with an example of such a system.

While thunk synchronization is another concern, I actually meant something else.

When you try to optimize a program by making it parallel it is important that you subdivide it into parts that execute in parallel, and these parts can't be too small (then the overhead of executing in parallel it too great) and these parts can't be too large (not enough parallelism). Therefore it is important for the programmer to be able to predict and control how much time a certain piece takes.

It is generally accepted that mixing implicit effects (e.g. mutable state) with lazy evaluation is a bad idea because it becomes very hard to predict when effects happen. The time it takes to complete a computation is an implicit effect. In most cases that doesn't matter because we don't care when a computation happens, but for parallel programming we do care. Laziness makes predicting and controlling how much time the subdivided computation takes harder.

In parallel Haskell code you often have to explicitly control evaluation order, and there are even various tools for this. A common situation is that you tried to compute something in parallel, but what actually happened is that you only computed a thunk, and the real computation will happen serially when somebody accesses that thunk. In practice it seems that the easiest way to write parallel Haskell code is to use deepseq to essentially make the whole computation strict. When you make essential use of laziness you have to be more careful and write a custom evaluation strategy. This is one of the reasons why I think it's better to make strict the default with laziness when you need it, rather than the other way around.

I see. OK that is a more serious problem. I agree that using deepseq makes the code eager and if you have all over the place laziness is just an annoyance. Either the machine is picking the order of execution or the programmer is. If the programmer is then it makes sense to use eager. I agree with you completely if the code breaks naturally into parts, then deepseq works fine, laziness doesn't buy you much and may be harmful.

But I don't think that is often the case. Rather what I think is much more common is that the code has lots and lots of places to fork over meaningful work. Generally this works by no explicit breaks but pooling. So say I can break a computation into 10,000 unequal but reasonably large parts and spread them across 25 execution threads. This was the case I think is more common because it allows for all sorts of complex dependency resolutions at runtime. Especially those that are dependent on external factors I can't time, like getting data from external sources; or when I'm not entirely sure if I have 10, 20, or 100 execution threads. And here laziness helps a ton. And that's because the author didn't write the program assuming order of execution, I'm going to be able to find those many thousands of large parts all throughout the code and easily and safely mark those.

. When you make essential use of laziness you have to be more careful and write a custom evaluation strategy.

I agree with you here. I'm talking about casual use of laziness. Essential use of laziness in lots of parts of the code makes things very complex. But in general, as I mentioned above I don't know of too many examples of lazy code that makes essential use; other than things like moving around infinite data structures as streams.

And here laziness helps a ton. And that's because the author didn't write the program assuming order of execution, I'm going to be able to find those many thousands of large parts all throughout the code and easily and safely mark those.

I think you mean pure rather than lazy. In fact if your language guarantees a lazy evaluation order you can't just execute it in another order because it might cause nontermination. A strict evaluation order gives you strictly more flexibility, since any code that works under strict evaluation also works under lazy evaluation.

Yes, I'm talking about the advantages of pure. I get that in theory laziness has nothing to do with purity but in practice they go together. Evaluation order is really really import for side effects. So you can't have non-obvious ordering of side effects too much, which I think prevents there being much impurity mixed with lazy. Or to take the contrapositive: lazy implies mostly pure.

As I'm talking through this issue I'm realizing that there is a definite difference between
lazy algorithms which don't do anything as far as parallelism that's helpful
as opposed to lazy languages which create an entire ecosystems and that ecosystem tends to lead towards almost full β-reduction capable programs

...
As I'm talking through this issue I'm realizing that there is a definite difference between
lazy algorithms which don't do anything as far as parallelism that's helpful
as opposed to lazy languages
...

This is pretty much it. The confusion in this thread arises because (roughly speaking) your 'lazy algorithm' is another person's 'lazy language', and your 'lazy language' is 'does not force more or less ordering than is inherent in a problem'.

I agree that a language that does not force over-specification of things (such as ordering) will have more flexibility in implementation. I think that this is another form of accidental vs essential complexity.

Good nice to see we may be coming to consensus on the matters of fact and just a minor issue on the matter of language. And I can see the other side's position on language pretty clearly.

When you try to optimize a program by making it parallel it is important that you subdivide it into parts that execute in parallel, and these parts can't be too small (then the overhead of executing in parallel it too great) and these parts can't be too large (not enough parallelism). Therefore it is important for the programmer to be able to predict and control how much time a certain piece takes.

This seems like something a computer ought to be able to do for us to a certain extent. I know there's been some work on this in nested data parallelism, but I'm thinking of a more type-driven analysis based on sized types.

For instance, a function which operates on a recursive data structure would optimally be split into multiple tasks as long as the data structure is above a certain size. The size of the data structure is in the type with a sized type system. Where the size is static, the decision is made at compile-time. Where the size is dynamic, it's translated into runtime evidence that the language runtime would use to decide whether to fork a parallel computation. I haven't seen any papers on something like this though.

There is something called auto tuning. What you do is you write your algorithm like this:

parameter int n

let foo x = 
  if size x < n then ...serial base case...
  else let x1,x2 = split x
       let y1,y2 = par (foo x1) (foo x2)
       combine y1 y2

And then an external process searches for the best n. Note that often you don't want to follow the same splitting recursion for the serial base case, since there is often a more efficient way. And you might want to split differently at multiple different levels to optimize for cache line size, and L1, L2, L3 cache size. I doubt that this can be fully automated, but it would be interesting to be able to specify a spectrum of splitting patterns and the compiler searches for the best combination. There is already a lot of work on this for specialized domains like FFT and matrix multiply.

Cache-oblivious algorithms may have something to say here.

Autuning generally does better. The cost is deploy ability.

Yup, it's really hard to manually create an algorithm that does well for all cache sizes and architectures. Caches are really complicated, for example see this: Why is transposing a matrix of 512x512 much slower than transposing a matrix of 513x513?. The answer is basically that the cache is like a hash table with a really bad hash function. If you're unlucky that can lead to lots of hash collisions and reduce the effective cache size by a huge factor. And auto-tuning by its empirical nature of course optimizes all kinds of things, not just cache behavior. Better tools can probably solve the deploy-ability problem. JITs are quite pervasive and have pretty much the same deploy-ability problems.

I've noticed three big issues:

1. Specifying many variants. To handle many architectures, many algorithms and data reps are used. Coding hits an engineering barrier.

2. Tuning time. Now multiply by discrete settings. Machine learning algs are starting to be used, but the test space still takes awhile.

3. Runtime setting. Outside of HPC, we should assume co-resident kernels; not all mem, CPU, etc. is available. You don't want just one binary specialized for your arch.

Why is transposing a matrix of 512x512 much slower than transposing a matrix of 513x513?

That's one of the many cases where the simple recursive (cache-oblivious) algorithm with specialised leaf routines works very well. There's even some research on generating such leaf routines from naïve recursive formulations. It's actually not that much work to reorder a trace and linearise access patterns: it was my term project in a graduate compiler course. Disabling the loop-recognition pass (and converting the few register-register movsd to movaps) lead to output on par with GCC on loopy code (with small known iteration count) at O3, in 2008.

... but did you beat the perf of an autotuner?

Far from it. But the coding effort to get a specialised kernel was much more reasonable ;) It also seems interesting to generate autotuners from recursive formulations. In fact, it would probably be easier than trying to determine a very good synthetic model and then reorder things according to that model.

Stencil autotuning languages seem just like that, with the extra hint of the type of recurrence, right?

As far as I understand this request it already exists mostly except you have to have a parallel strategy (parMap). GCD by Apple also makes something like this very possible, and with just very small changes you get parallel strategies.

Non-strict semantics is good for parallelism, but laziness in principle forces everything to be evaluated at the last moment possible, which makes the program miss a lot of oportunities for parallel evaluation of expressions (e.g., you can't just start the evaluation of every argument for a function in parallel, because one of the arguments may turn out to be an infinite data structure that will start eating up all the memory, so you must wait until parts of the structure are needed to compute them, but that means you can't take advantage of parallelism). I guess a combination of non-strict semantics and eager evaluation (so-called "lenient evaluation"), as in Id and pH, can make most of parallelism.

Under the definitions I'm accustomed to, laziness is a memoizing strategy for non-strict evaluation order. Lenient evaluation is another reduction order entirely. It results in reasonably parallelizable code for some common cases, but I find it to be a pretty big hack. A better approach, IMO, is to separate evaluation order from semantics.

For parallel evaluation, lazy vs. lenient vs. whatever is a very tiny piece of the picture. But we've had that discussion several times now..

I don't think laziness should require compute last just compute in any order the system wants. The infinite data structure should start sucking up large amounts of memory, but not all of it because the evaluator finds other parts of the thread that can go first. Memory usage should be part of how the system chooses to pick which threads to focus on.

The problem I see with controlling evaluation strategies (at least lazy vs. eager) is when/how the programmer expresses this - i.e., is it at declaration time, at funcall time, at de-referencing time? The problem with declaring laziness early is that the referential transparency of treating lazy and eager values the same is both a boon (don't need to change change referring code) and a PITA (if you don't know that a value is lazy at some point in your code, you could get unexpected complexities). Explicit laziness (like Lazy(T) ) is nice because it's explicit and can be implemented in any language, but forces a getValue fn/monad to be used - but then code gets littered w/extra cruft. I think tooling and visualization of explicit and inferred evaluation strategies and some notion of meta-strategy based on usage domains would probably help, and even the JIT running various parts of the code w/different evaluation strategies and re-coding as necessary.

I would hope that we are already working on IDE and library design issues. I don't think FRP will catch on, but maybe something similar to it (like immediate mode UIs). I'm also not so sure about modules either, but my thoughts are much more controversial here (modules are bad, long live the one big code wiki!).

Your prediction about CPU/GPU merging is interesting. I'm going to counter-predict that we will better understand CPU/GPU workloads by 2020, and will have optimized each more for their loads, but we will have failed to unify them. Maybe by 2030 "data threads" will exist beside "control threads," hmm...

I agree that it won't be exactly FRP (which is why I added the caveat that it may not be in its current form). For example the continuous aspects of FRP will most likely be thrown out, and some impure updates added back in. You can already clearly see this trend in modern JS frameworks like Backbone, Angular, Knockout, Meteor, etc.

On modules, you can see dependency injection being quite popular in the Java world (and to a lesser extent in C#). It is basically a big workaround for the lack of proper modules. Scala has quite nice language support here. Perhaps something less structured like a big code wiki will win, but won't that make programming against a declared interface and linking components via their interfaces even more important? A literally global big ball of mud wiki where all internal details of everything are exposed doesn't seem very nice but perhaps you're thinking of solving that in some other way perhaps with tool support?

Yes, GPU/CPU will probably be popular by 2020 but the merging is likely in the long term. We already have technologies that can take advantage of both small and large scale parallelism, like FPGAs and stream processors, so we know that it's possible. I think that technology trends will make it profitable. As chip densities increase, CPUs will hit a wall because 2x transistors will only give you a couple of percent extra serial performance due to better designs (this has already happened). GPUs will hit a wall because if you just use those transistors to put more and more GPU cores on a chip, you'll run out of large scale parallelism in the problems you want to solve (this has already happened for some problems that do not have much parallelism in them, and will happen for more and more problems as GPUs get bigger).

Of course this could easily be hogwash, and this goes for all of those predictions. Although there is some reasoning behind each of them, it's so easy to overlook a crucial piece of information that will only become obvious as we actually go into the future.

I actually think the continuous bits of FRP will survive in the form of freaky evaluation models and/or databinding. Its the event composition parts that will fall away and become less relevant (like how Rx doesn't really solve UI programming problems).

I don't see dependency injection has related to modules, that is more AOP. My continuing point about modules is that search is just so much better; with search modules become as irrelevant as web hosts (they still exist, people just stop caring about them as concerns). Encapsulation is still a good idea, but that is encapsulated to the user of the module.

I don't think we've really scratched the surface of data parallelism yet to scratch the surface about where we are going with CPU/GPUs. The fact that we (the PL community) got this wrong and the systems and graphics communities got this right should serve as a warning that we might not know what we are talking about.

Agreed that the event composition part of FRP may go away and be replaced with imperative event handlers. With continuous bits I meant stuff like time integrals of real valued functions, not the data binding (which is very important).

Dependency injection AFAIK works like this. You have a component Foo that uses component Bar, and now instead of hard coding that reference to Bar in Foo, you let Foo take something with interface IBar as a parameter and pass in Bar at run time. This way you can also pass in a different IBar. Dependency injection frameworks attempt to keep the code somewhat reasonable in a language like Java and try to avoid passing everything around manually and instead you write a recipe for linking the components together. A module system or a powerful object system as in Scala is a much better solution for separating a program into interfaces and components, and linking those together.

I don't see how search is a replacement for interfaces and modules but this may just stem from different terminology since you say you still want encapsulation. Perhaps you mean search as a replacement for (nested) namespaces?

Hardware that can exploit both small and large scale parallelism seems inevitable at least in the long term, but yeah it could easily be wrong due to unknown unknowns.

About IDEs and libraries: perhaps I'm just not aware of it. Is there academic work on general purpose languages that explicitly considers the design of libraries and IDE together with the language?

Ah, yes, the pure continuous stuff in FRP is interesting but not very pragmatic. I was just referring to continuous timeless abstractions (like continuous binding). It took me a while to sort this out from the rest of the FRP work.

Module is such an overloaded term, its no wonder we aren't talking on the same page :). I see what you are saying now and I don't disagree. Modules to me represent units of linking, encapsulation, deployment, management, namespace management, etc... I just don't see a need for them to play active roles there, that somehow all the code should just live in a giant (inter) net where they are forced to be compatible.

Smalltalk: libraries and environment all came up together and it shows. Self also has Morphic and its (limited) dev environment (Maloney's UIST 95 paper is very good in this regards). I'm sure we could dig around from all the Smalltalk-related papers in the early/mid 90s and find something.

A more modern example would be Racket, though the term "language design" is sort of weird there (since libraries are often languages). Nothing extremely aggressive yet.

I actually think the continuous bits of FRP will survive in the form of freaky evaluation models and/or databinding.

What is a freaky evalution model, and will I be sorry I even asked...

Before we judge FRP, can we at least set a criteria FRP implementations must achieve before being worthy of being judged:

• Handling non-rectilinear layouts
• Constructive area geometry for shape compositing
• Vector-level clipping (a key point of FRP is to make reasoning about UI effects easier, but that becomes trickier if clipping takes place as part of rasterization)
• Vector-representation of text in a particular font.
• Handling dynamic collections (e.g. opening/closing windows, adding/removing elements from a DOM, etc.)

...and at the same time, not shaming research prototypes for failing to deliver on all these needs. For example, until mid 2000s, Cairo was the only major open source, cross-platform rasterizer available, and it basically only supported rasterization. Now, it supports some more things, but as its feature set has grown, it has become harder to think of pleasant ways for researchers to map their FRP interfaces to low-level interfaces. So you kind of need to think about what is needed to make FRP work-able on a large scale before criticizing it. And it is still really valuable in some scenarios. For example, for fun, I am writing a non-Euclidean Geometry drawing application, where you can draw lines and points on a sphere, etc. It's much easier without having to deal with state.

As per Cairo, the same is becoming true with FRP and GPUs. Conal's work on Vertigo, for example, makes sense only at the point of pixel and vertex shading, but everything becomes sketchy past that, and with GPGPU, imperative programming is the norm again.

Take a look at my latest draft on FRP.

I found that with the right, fairly unobtrusive, type system, it's possible to write FRP programs with a simple, predictable cost/evaluation model (call-by-value, with call-by-need only for dataflow cells). The really nice thing is that typing eliminates the need for explicit dependency tracking, which means that one of the most performance-hostile parts of dataflow programming (its bad interactions with GC) goes away.

What is academically interesting about "value types"?

In itself it is a minor thing not a major research area, but practically important for making other academic achievements count more in areas such as generic programming, dependent types, better GCs and parallelism. In combination with dependent types they allow you to express data structures that normally require unsafe C/C++. For example a variable length array is a dependent pair Array a = (n : Int, Vec n a) where Vec is the usual inductive type but with each pair represented as a flat pair instead of a pair of pointers. As far as I know, data representation is the major thing that keeps the efficiency of e.g. idiomatic OCaml or Haskell away from C and Fortran. Combined with staging on type class dictionaries this can make all those wonderful abstractions and generic techniques perform without overhead. You can take advantage of techniques that are not practical in other languages, e.g. generic radix sort beating C++'s built-in comparison sort, and problem specific matrix representations beating Fortran matrix representation for the common case where you have a semi sparse symmetric matrix. You could probably exploit that in Fortran for a >2x performance gain if you were willing to re-implement all matrix operations for that representation. Another example is automatic differentiation, which really sucks performance wise unless your values are unboxed. Less memory allocation and fewer pointers in the heap lets GC work shine. General efficiency lets the work on parallelism get absolute speedups instead of just relative to serial, and less memory traffic lets it scale better.

It could certainly be that all of this already exists somewhere; I didn't think of it when I wrote that but ATS perhaps? It hasn't gone "mainstream" in academia in any case.

You can take advantage of techniques that are not practical in other languages, e.g. generic radix sort beating C++'s built-in comparison sort,

Are you willing to explain better? Maybe there are ways to do that, even without ATS language.

To do radix sort on data other than character strings you need datatype generic programming. That means that you do something different depending on the data type.

See http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.154.2018 and Generalizing Generalized Tries.

The thing is that this is supposed to improve performance because it's asymptotically better, but when you try to sort a list of (Maybe Bool, Either Bool Char) and you represent the pair as a pair of pointers, the Maybe and Either with pointers, you already lost big time to naive C++ simply because following all those pointers is so much slower than using value types. When you write a radix sort for a particular data type in C++ it *is* faster than comparison sort (e.g. for integer arrays it is >2x faster).

"Value types" seems to be a particular mechanism or methodology for representation control.

Types are just a tool to me, but the underlying issue of programming over multiple representations is fascinating. For example, when designing 'clustered data parallel' algorithms, I found it profitable to switch the data representation for a tree (a webpage) based on the access pattern (what stage of processing it is in). Today, systems like NESL and Data Parallel Haskell assume you use one baked in representation, and that's wrong. There is a lot of space in designing constructs for easily, correctly, and effectively (in terms of performance) to do these conversions.

As for academic interest, I'd argue that the last two PLDI best paper awards were about this fundamental issue of language support for controlling data representation. Their domain of essentially implementing SQL is better trod in the DB literature, but designing constructs for controlling representation is the interesting part.

Some things reasonable people may disagree on. But I am certain emacs will still be the tool of choice for the discerning hacker.

Language and IDEs should be tightly related, like what we are seeing happening with LightTable. I believe this merging could be one of the significant advances that we will start to realize by 2020, if not very popular already.

Emacs is already such an environment for LISP, but support for other languages seems tacked on at best (code completion through ctags!). In a language where great code completion is possible, using Emacs feels like turning off your targeting computer and using the force.

Great list of programming language issues that look imminent. In theory, these should be the most clear of anything on an 8 year horizon. The things everyone will be doing in 8 years, are the things that look good in some toy language right now.

I mostly agree with your list but would name a few exceptions.

I really don't think automatic client/server program splitting will go anywhere; I'd move it to your list of things that will dwindle. Not all abstraction is a good thing, and data sent between client and server is a bad thing to abstract. For example, you really don't want the optimizer to send over the password map to the client for efficiency purposes.

On IDEs and library design, I think the existing practical systems are way better than most of what is being done by academics. IDEs require an army to make a good one. Library design requires a kind of taste that I don't see possible to get without actually working on a large program, and thus not being a theoretician any longer.

Something called value types are already present in C#.

I'm surprised about the assertion against text files. They seem good in every way except the number of bytes, and the number of bytes is usually not that important for interchange issues. In computer software, usually complexity is a bigger problem than machine resources.

On testing versus types, I suspect scripting languages are here forever for writing scripts. However, I suspect they will dwindle as a way to write anything large enough to be worth regression testing.

Its been awhile Lex, hope all is well.

It doesn't take an army to write a decent development environment, just people with skills that are not typically found in PL implementors. But its integrating post priori with a legacy ecosystem that really kills things. E.g. the visual studio core team is not really that large, but all the infrastructure and features around it require a lot of resources.

There are some essential principles to some library designs that we are just getting a handle on right now. Integrating those principles into our language design process can be very useful, but ya, we are still only going to be able to address domain N+1 in an ad hoc post priori manner.

The problem with text files is not efficiency, but addressibility and meta-data. In many ways, Smalltalk was just a bit early in its ideas, but I think we'll start moving to code databases soon enough, along with projected code editors that are able to present/edit richer code formats (e.g., including GUIDs next to names to track how names are resolved historically).

I really don't think automatic client/server program splitting will go anywhere;

I predicted it would go nowhere back when we had the discussion on LtU a few years back, and Joel even chimed in to give his perspective on GWT's code splitting, and I pushed back saying he was wrong. Back then, I was actively e-mailing researchers of various projects, letting them know my perspective and why their approach was doomed to fail, and what they should do about it.

That said, I think the approach I outlined back then is as good now as it was then, and better than automatic client/server programing splitting. The problem is academics didn't understand common fallacies in mobile, ad-hoc networks, and what kinds of APIs should be designed for them. So academics gave us really bad tools, and they are still not doing anything about it.

There are also some things I would like to be able to express more cleanly.

For example, data binding is clumsy today. Here is a simple example.

I have an (int) and want to specify a refinement type that says, Integer is in 1 .. 10. In most data binding APIs today, there is no clean way to utilize blame to guide control flow. I believe fixing this problem once and for all will lead to much cleaner APIs. We also have reasonable theories of refinement types and contracts, thanks to Contracts Made Manifest by Pierce et al. Using blame for control flow can only be truly possible if the blame can be an exception.

I actually don't even like the idea of directly tying blame to type casts or dynamic contract checking, but it's so much closer to what I really want in order to remove redundant logic. For example, inputting data into a text box is best done by first making everything a string. Then there are two different kinds of ways to alter control flow possibilities.

1. Block/filter out input, e.g. given a string, if non-integer characters are input, don't accept them.
2. Accept input, and ask the type system if the string can be cast to an int. This is actually kind of tricky from a usability perspective, because what do you do when the user types "-".

Accept input, and ask the type system if the string can be cast to an int. This is actually kind of tricky from a usability perspective, because what do you do when the user types "-".

That's an interesting problem, calling for interleaving parsing/validation with dataflow. Naively linking reactive data binding to a parser for integers would yield errors while typing in "-10". You could introduce a timeout after which the parse is attempted, but that seems crude. Seems the simplest answer would be to discriminate on whether a parse was partially successful, and if the parse ended because of the end of input, then don't display an error.

Basically, make parsing incremental where a partial parse "error" simply returns "need more input".

Yielding errors during parse seems quite acceptable in a reactive data-binding model. E.g. you could display near the textbox a red mark while invalid or a green mark while valid. The main constraint is that, in a reactive language, error models should be equally reactive - no exceptions or control flow that bypasses the model.

I think we're going to eventually settle on effects/effect-handlers and formalized effect systems. Not sure how first-class they'll be (via explicit instances, as in Scala if at all), but given that instances being second-class language citizens hasn't hurt Haskell, I think we'll find a good way to combine effect handlers and static effect checking.

The equivalences with Haskell-style monads already exist, but the syntax and intuition will be a lot easier for the common programmer. We already successfully teach people how to write exception handlers, after all.

I think Koka is a good language to look at in this regard.

Jeff,

There was a paper by Philip Stanley-Marbell about this 2 years ago. http://doppelganger.gemusehaken.org:8180/thepub/abstract.jsp?id=50

The title and abstract are very misleading, as Philip's real goal is to have a different kind of assembly language for compute clusters where the computation might be slower but cheapeer per watt to execute.

Sorry I missed this comment earlier. Yes, something like that. I don't have anything intelligent to say on whether that's the right approach or not, sorry for not being able to hold up my end.. But I'm glad to see some work on it, because some assembly that allows one to count watts needs to exist.

It's unclear how well the approach outlined in the paper I linked would hold up to various non-linear, convex optimization problems. I don't even know where to start to understand such difficult program reasoning challenges.

Previous Apple languages: NewtonScript, Dylan, SK8, Hypercard, Applescript, Quartz Composer, Fabrik (I'm sure I've forgotten something).

I'm not sure if Apple is going to get back into the language game again, it didn't seem to help them much out before.

I think they will have too if the windows platform is hugely succesfull in the mobile / tablet space and if the momentum can be partially credited to developer productivity. This increased productivity must of course be acknowledged by industry in order to put some pressure on apple leading to an increased focus on tooling and languages.

Microsoft's (my employer) and Google's language talent is really a huge differentiator compared to Apple. However, developer productivity rarely seems to move market needles; not many iOS developers loved or love Objective C! For language to become a factor, we really need to squeeze out a magnitude increase in productivity, which is a tough goal that no one has really achieved (at best, we can say our FP languages are 25-40% better, and even these claims are dubious).

Almost nobody claims their used language is a competitive advantage. One exception is Eiffel and their customers. Or actually the customers seem to be quiet.

Actually, Jane Street do, too.

Robert Dewar was always emphatic about that year after year sponsoring the ACMGNYR ICPC contests, at least when I was there a few years ago.

As primarily a Windows programmer switching from C++ to C# as my primary development language has given me easily a 3x-5x boost to my productivity.

I doubt it was that much, maybe 2x? The problem is that once C++ overhead is eliminated, there is still the fundamental task of programming and debugging that remains constant. We also tend to magnify observed productivity gains because they are so satisfying, and going back to the "slower" language is really hard (those C# programmers will never want to go back to C++).

I stand by 3x-5x. Not immediately of course, but once I became experienced in C# definitely.

I also need to make the caveat that its not just the language in isolation, but the tools and libraries that are available for the language. I believe the sophistication of tools and libraries are a direct result of the language design.

That's not true. Debugging C# programs is much easier (very little chance of undefined behavior, entire classes of errors are virtually impossible) and many programming patterns are much faster to implement (hint: anything with reflection).

I totally believe that you believe this, because I feel the same way. I just wish we could have some objective/formal experiment to prove this, and I suspect the benefit is probably not as much as we feel it is, but I totally accept that I could be wrong.

Reified generic types is definitely a big win for C#. I write all my code in C# these days.

I do most of my winders programming C#. Just the ability to edit code on a running program is worth 2X by itself. Which bring to mind there is a low of low hanging fruit left in just tool design and interface.

For instance, I remember back in the bad old days assembly programmers patching their programs while debugging, in C you can't do it. Not because it's impossible but because the compiler, linker, and debuggers weren't designed to support it. I wish I had that in C.

And yet, edit and continue for C# is not as robust as edit and continue for Java. I've barely been able to use edit and continue in C# because of my addiction to lambdas, where VS is just too conservative about that.

Also, I think real live programming will blow edit and continue away as the must have feature by 2020.

I'm willing to bet against this and predict that, perhaps with the exception of web interfaces, neither live editing nor mudball-style programming environments in general will become relevant for the vast majority of developers. Nor should they.

No idea what you mean by web interfaces, maybe you mean interactive programs? If I can demonstrate and prove this concept for say...compiler writing, would you be more willing to believe?

Programmers will not continue to carefully design, plan, and validate their programs before they write one line of code because...they don't even do that today. The style that most programmers follow will continue to be somewhat less than ideal and benefit from a "heads up display" to more quickly understand where they are actually going vs. where they thought they were going.

Hypercard was incredibly successful and popular. I'm hard pressed to see that as a failure.

Quartz composer's graphics language is used all over the place with Cocoa. I'd say it is the core reason that Apple was able to get the animations to work in 2007 on the hardware they had.

Apple mismanaged Hypercard royally.

Quartz Composer has only been successful in the art crowd, I haven't seen any products that actually contain QC code. As far as I can tell, its already deprecated.

My point was it wasn't a failure. I'm of two minds on whether Apple could have saved Hypercard. On the one hand I look at Powerpoint, the Access / Foxpro / Filemaker, Visual Basic and then the web as having eaten their market niche. On the other hand there is still I believe a huge niche still left untapped for a language easy enough for even non programmers to write good simple applications in that is widely used. Apple had it and who knows what could have been if they hadn't tried to monazite it.

BTW a modern Hypercard does still exist LiveCode. I've heard great things about it, and Apple has been supportive of LiveCode, but supportative is far short of buying the company and bundling it with every OSX system sold. A more amateur HyperCard which is free is PythonCard.

Granted. My point is that Apple hasn't been good historically in marketing and pushing their own PLs. I think at this point they might be a bit apprehensive about their experience, but who knows, they might wade back into the PL innovation market.

Apple view itself primarily as a hardware company and so sees its competition as Dell and Samsung while preferring to partner when possible with Microsoft and Adobe. Because their stock and trade has been GUI design their products have generally leaned away from text based scriptability. Moreover Steve Jobs has generally leaned towards a focus on professional development because he wants the software to be of high quality so Apple languages have had features but they've had features aimed squarely at the professional crowd.

So if you are saying they don't do it much, I'd agree. I don't agree they aren't successful at marketing though. When they have I don't see them being particularly bad at it. Hypercard was successfully marketed. People were unhappy with Apple's handling of Hypercard because it was immensely popular and they let it die, not because it wasn't popular and unsuccessfully marketed.

AppleScript is as successful as say Powershell and possibly more so. But again Apple doesn't see strategic advantage here so they are letting Applescript wither. I wish they didn't but I can't say they are wrong that AppleScript doesn't sell computers.

Objective-C is also an obvious counter example to the "Apple doesn't market their own languages". That language has become hugely successful due entirely to NeXT/Apple. MacApp before that. On the Apple 2 6502 Assembly and its ties to Apple Basic if you want to go back further were all successful. Dylan was a failure but Newton was a failure.

MacRuby, as I mentioned above is their big innovative language. They are trying to tie a classic IDE / development framework to a dynamic language with both OO and functional features. I'd consider that language innovation. If they can make MacRuby successful while at the same time Objective-C is still viable then they will have successfully migrated.

As an aside, I know you are a big smalltalk fan btw and F-Script which uses the MacRuby components is potentially a rather good Smalltalk with full access to Cocoa.

I think it is fair to say that in comparison to say: IBM, Microsoft, Adobe... Apple has been mostly disinterested in programming languages. I don't think it is fair to accuse them of marketing failures.

Apple has done a lot of great language work in the past, but they almost lost it in the late 90s, laid off all their researchers, and decided to razor focus on products and where they could add value/make money. So now we have an extremely focused, efficient company, why would they bother to get involved in language design when the pay off is so abstract? Its as much as they can do to make Objective C a bit better with reference counting and focus on making their tool set as good as they can.

The problem is, language design is not really that profitable for a company (Sun, Apple, Microsoft, Google, ...). The benefits are abstract and actually raise everyone together, it really is charity work for the industry vs. some special advantage. Apple is just too focused right now, they would have to accept being less efficient to do this kind of work.

I like F-script BTW, I totally enjoyed the OOPAL paper presented at OOPSLA a few years back. Its interesting to see Apple get involved with MacRuby, but it seems to be more of a port than something new.

I don't know of any other mainstream language designed to offer:

a) syntactical abstraction
b) native access to a full featured OS widget set
c) direct access to C calls
d) 2-5x slower than C (i.e. Java like performance)

If MacRuby works it will be a huge huge step forward for mainstream development. You could see by 2020 (and maybe by 2016) a language which might be consider an "acceptable LISP" being the primary development language for the most used computer platform (iOS) on the planet. That is a very very big deal. Just think about what happens when meta-programming is standard, something that every developer knows and uses on a regular basis.

And this also plays out in terms of dynamic languages. We really don't have dynamic languages with powerful IDE's and interface kits like what exist for static languages. With MacRuby, Ruby developers will have Interface Builder to tie graphical onscreen objects to code written in XCode. So this is also a very big step up for the dynamic language crowd.
____

I do agree that Apple has been very disinterested in language design and focused. No argument. Though I suspect a great deal of their behavior lately had had to do with the NeXT acquisition. When Steve Jobs came back he came back with an Object Oriented widget set (Cocoa) tied to Objective-C. The research was done at NeXT in the late 80s to the early 90s. They had a language solution, so they didn't need to research one.

OTOH Apple unquestionably did a lot of work with getting PowerPC compilation to be more efficient, that's not exciting from a LtU standpoint but it is real research (or at least real engineering), and research that Microsoft, IBM and Sony have benefited from. They are working on LLVM today.

They've already indicated the new language: Mac Ruby. The Mac Ruby api is already included on the iOS devices though right now they discourage 3rd party applications from using it. But they've been very open they see this as an import direction. There is a commercial version Ruby Motion which has lost of apps (sample Ruby Motion apps).

Apple knows that Objective-C is a problem for their 10 year plan. OTOH right now it is a competitive edge vs. Android and other systems.

"(Objective-C) is a competitive edge vs. Android and other systems"
Over Java/Android? Can you extrapolate? (In the NeXT days, obj-C was pretty intersting.)

Objectively smart phones are a little over 10 years behind laptops in terms of hardware performance and they were further behind in 2007 when the iPhone came out. If you think about the dominant GUIs on desktops it was Windows 3.1 which was not a smooth experience at all. Most other GUIs at the time the iPhone was released were terribly jerky. By creating a very narrow class of animations which were highly optimized Apple was able to achieve an illusion of hardware much more advanced than what they were really running on.

If you think about the core design ideas of the iPhone in 2007 were:

a) capacitive touchscreen as the primary or sole means of input
b) animation based interaction
c) high speed web rendering

(b) is all about the advantages of Objective-C. iOS developers are directly targeting one specific animation library (Core Animation) running on one specific CPU which is tweaked for these functions in a language which constantly exposes them performance related information. Animated interfaces that feel like they are running on a real time kernel are very attractive to humans. Just talk to regular people and they tell you how much faster their iPad is than their laptop, when the opposite is obviously the case.

When you contrast this with Android/Java which is:

a) Slower
b) Running on a wider range of hardware
c) Running on hardware that is not as tweaked for specific aspects of mobile GUI
d) Written in a higher level language
e) Often written with abstraction layers that write inefficient code.

Applications don't feel as good. The result is that people perceive these phones as worse.
As hardware has gotten better rapidly the advantages of C over Java are fading. The disadvantages of C are becoming more apparent. But in the end, games are still hugely important mobile applications, a huge chunk of sales and gaming is all about performance and consistent hardware.

____

A second major reasons is acculturation of new developers. Apple has a relationship with its developers which allows the entire platform to move quickly in line with Apple's strategic goals. By directly creating the toolsets that developers use they maintain frequent and regular communication with the iOS developer community which assists greatly in this. Google / Samsung have barely any relationship at all with their developer community. They can't meaningfully direct the platform in anyway. The result is that for iOS the application base is anticipatory of hardware / OS / strategic changes while for Android it is reactive.

_____

The third major reason is that Apple owns the most successful app market right now. Most mobile applications being written are being written in Objective-C / Cocoa. If that continues that creates huge lock-in for the platform similar to what Microsoft enjoys on the desktop. Conversely we already see on Sailfish and BB10 that other mobile operating systems can create an Android compatibility layer and run Android applications.

I predict that Apple will not be a major force in driving technology by 2020. It may still be driving device design.

Assuming that iOS is even still around in anything resembling its current form, the distinction between Android, WinRT (though it won't be called that in 2020), BlackBerry OS and iOS won't be relevant for most app developers, given the requirements of most apps (e.g. UI, network, filesystem). Third-party software tools and DSLs will make it all go away.

Well, except for that little bit. Oh, and that little bit. And that incompatible change introduced by one of the vendors in 2019.

But even these will be (mostly) virtualised by third party libraries. There may even be an open source de facto standard.

the distinction between Android, WinRT, BlackBerry OS and iOS won't be relevant for most app developers, given the requirements of most apps (e.g. UI, network, filesystem). Third-party software tools and DSLs will make it all go away.

No way does Apple let that happen. If they were able to crush abstraction layers in 2009 in iOS you don't think in 2010's when they are in a much stronger position they will crush it? Apple is going to need to assure that the people who are building the heavily used applications on their platform are acculturated to their platform. They will never tolerate a situation of being a generic commodity platform. That would destroy their margins and their entire business model. Apple will tolerate 3rd party tools that abstract in so far as those tools enhance user experience and bring value to the platform. The moment they begin taking it away they will ban associated application from their distribution model.

The only time that Apple isn't driving technology to create differentiators is when they have exited a market.

I thought that the adoption of Unix was a fairly "open" thing to do.

Apple more or less adopted NeXTStep which was based on a UNIX. In today's OSX, Darwin the BSD underlying OSX is open and an active participant in the open source Unix world. The layers above that though are entirely proprietary. The result is that Apple's customers get the advantages of an Open Source platform in terms of being able to use the wide range of hardware, while experiencing the advantages of proprietary platforms in terms of offering a unified vision.

For iOS though, Apple acts in the role of distributor, which makes allowing for GPL code complex. The result is that unless you are running the Developer SDK, which requires an OSX machine, iOS is closed to lots of the open source community. While there is still Darwin underneath iOS it exists as a service layer accessed via. Cocoa. If Apple tomorrow decided to replace Darwin with the NT kernel running Microsoft Services for UNIX I'd bet 80% of all iOS software wouldn't even notice.

_____

Getting back to your post above though, Apple has been crystal clear about what they think about cross platform development tools and libraries:

We know from painful experience that letting a third party layer of software come between the platform and the developer ultimately results in sub-standard apps and hinders the enhancement and progress of the platform. If developers grow dependent on third party development libraries and tools, they can only take advantage of platform enhancements if and when the third party chooses to adopt the new features. We cannot be at the mercy of a third party deciding if and when they will make our enhancements available to our developers.

This becomes even worse if the third party is supplying a cross platform development tool. The third party may not adopt enhancements from one platform unless they are available on all of their supported platforms. Hence developers only have access to the lowest common denominator set of features. Again, we cannot accept an outcome where developers are blocked from using our innovations and enhancements because they are not available on our competitor’s platforms. (Steve Jobs)

Since no one commented on this: AI is "abstract interpretation".

That's what I'd like to see, at a high level. A data model exhibits certain characteristics and cardinalities. A backwards-chaining language can fit interaction patterns to the data model, with easy specialization/refinement. You might even be able to infer the data model from the selected set of interaction patterns.

There are parallels between CSS and interaction pattern selection, as well. CSS applies multiple factors to arrive at a computed style, and an interaction pattern engine might apply multiple factors to arrive at a computed interaction.

Roughly...abstract interpretation, at the level of user intent.

I used to work in nanotech. I don't think that nanotech will be up to this point anywhere near 2020. Nor will quantum computers, for that matter.

What may happen by 2020 is a new type of FPGA (or a new technology like FPGAs) which makes reconfigurable hardware an almost-practical option for general-purpose programs. Need to do some crypto? Code up an AES2 coprocessor!

If this happens by 2020, it will be about as useful by then as GPU programming is today. In particular, there won't be anything closely resembling cross-vendor or reasonable OS support for it by then.

Okay, now what about 2020?

Oh that? Javascript, what else?

The idea that programming is like writing proofs cannot succeed on large scale

This reminds me of something I saw, many years ago, in a book on number theory. The author was expounding on the theme that the distinguishing feature of number theory is that it treats each number as an individual with its own unique personality. In doing so, they remarked that armies don't treat soldiers like numbers, rather arithmetic treats numbers like soldiers.

My point is that programming isn't difficult because we're mistakenly thinking it's like writing proofs. The problem is that we don't have good tools to make it easy to write proofs.

I've been frustrated for decades that we don't have good tools to help people do real mathematics (i.e., write proofs). AI isn't the answer; computers aren't good at thinking, just as people aren't good at computing. Tools should allow an ideal collaboration between computer and human, with each doing what they're best at. Type systems just shift the burden of proof around without getting at the essence of its difficulty. I've long had some preliminary notions of how real math tools might work, but doubt my ability to explain them to someone else except by demonstration, so really I'd have to build the tools myself. To program something that big myself I'd need a programming language with awesomely powerful abstraction. And if such a programming language did not exist I would then find it necessary to create one. And if I didn't know how, I'd have to make a systematic study of abstractive power. Of course, all this is purely hypothetical... oh, wait. Not so hypothetical, then.

We already have lay people programming at a high level by natural, organically developed (well, human) methods of communication.

Here's how they do it:
1) find a programmer
2) talk to them

On the whole, what you seem to be proposing (to me anyway) is a form of functional programming that is abstracted several levels above where a programmer is typically involved today. I understand you to mean that you are going to be seeing commodity "functions" as the primary means of developing an application rather than a general purpose, 3-4GL (such as C, or Java to name a couple).

I'll refer to this idea as "cellular" programming, because in effect that is a decent metaphor for what you are doing. You are generating "cells" (the commodity function) that when combined together form a large scale "organ" (application in my metaphor) which satisfies the functional requirements for its continued existence.

I think what you are saying in the section where you suggest such a new technology "must eliminate aspects of programming at which humans suck" [sic], is that what would constitute as programming would require a less "logical" mindset than what is currently required to write code. Instead, a user could get away with a more "functional" mindset. As example, you give the "auto parts" analogy. You can build a motor using auto-parts, with a decent knowledge about how a motor operates and what it needs in terms of component parts. You do not need the knowledge of how to derive the component parts from elementary materials (i.e. steel, plastic, lubricants, etc...). Or, more simply, you "know you need a camshaft" or some component similar in function, to regulate the valve timing. This can be several levels of abstraction above knowing how to derive the component part.

I really like the idea, and find it an attractive premise. However, I 'm not confident that (if what I describe is what you intended to communicate) we can ever reach a programming model that would approach that level of abstraction. Possibly because at that level of abstraction, the model wouldn't be useful to any other domain except for the domain in which you were working. The tools in that domain would be explicit to that domain.

I think we will likely see an increase in "pattern" based programming, to the point that maybe class templates (if you are using OOP and classes), for example, could be generated by selecting which pattern you are attempting to implement. It is along this train of thought that I could see improvements to IDEs in that they could suggest to you what logic structures to employ and alert you to the potential "anti-pattern" that you are constructing.

Of course, on the other hand, if what the future holds is a marked increased in personal, specialized computers, then your idea could bear fruit. Since the specialized computers, could be isolated to a specific domain.

Nothing changed, then.

Of course we tell the computer how to accomplish a task. Merely telling a computer what to do without "how" is using software not authoring it. The huge number of pre-written programs that exist today that 33 years ago required their own special code, or wouldn't have been possible at all is the evidence of progress. In particular the level of detail needed in the telling has gone down tremendously and the number of things the computer already knows how to do has gone way up.

33 years ago basic communication over a network socket was complex in practice. Nothing like Ruby on Rails could have existed. Networking 33 years ago was about trying to have single small binary files successfully transfer in over an error prone circuit while maintaining sessions. Today a web programmer casually uploads a binary graphics file and it loads on the client without the slightest bit of programming by the web programmer. Ruby on Rails is a step even above that and assumes that page to page to page transitions each involving multiple graphics with database reads and writes can be maintain a state. 33 years ago that wasn't true of files on a local floppy drive much less a remote application.

100 years from now it still going to be about telling computers how to do things. When it is merely telling If it is merely what, that's operations not programming.

Grasshopper, you have not snatched the pebble from my hand.

"writing PL/1, JCL and assembler in XEdit isn't much different than writing Java or Ruby in Eclipse"

It's VERY different. For one, you worry about fewer memory management issues and some bugs are impossible (less debugging). You're IDE's do plenty of work for you from giving you important info to automatically refactoring code to automated testing. The programming process in imperative languages might be conceptually similar, but I think "practical" difference is more important than "conceptual." Just compare even Genera LISP development to your PL/1 or assembler:

http://www.symbolics-dks.com/Genera-why-1.htm

I think a tremendous improvement would be to have these capabilities in modern IDEs. Slowly moving that direction. A Java guy told me Netbeans does plenty of this stuff or provides alternatives he thinks are better. Python is ridiculously easy to learn and use for all kinds of apps. Model-driven engineering like Sun's DASL and iMatix's technologies were quite a change. And we have little kids learning to piece programs together like lego's a la Scratch project. Correct by Construction approaches also improved from requiring entire teams of highly specialized individuals to being usable by undergrads with a specialist consulting (or w/ lightweight techniques just programmers + no specialist).

I'd say things have changed for software development quite a bit since the past. If productivity, usability, and reliability keep improving at the same pace until 2020 I can only imagine how much easier it will be to crank out software. It's not the ideal direction or pace, but it will be hard to say "nothing different."

Note: I totally agree with you regarding your how vs what, poor knowledge, bad quality, and bad estimates. This seems to stem from the economics and sociology of management of software projects. I've seen enough high quality software teams to know there's little technical reason for it: just bad management or uninformed developers.

I'm totally into the idea that programming language design can act as a enhancer for social programming. For example, type systems that make relevant abstractions easier to find and discover, allowing the crowdsourcing of a huge library fairly quickly. Creating a more forgiving usability gradient is also useful here (easy to start using, high ceiling to keep more experienced/skillful programmers interested).

Continuous and universal testing are also ideas that are taking hold already. What if I could change some code (in a code wiki perhaps) and then get feedback about whose code I broke with my change?

As for your application areas, I see the need for more PL people to get involved in machine learning, since all of these fields are heavily dependent on that. How do I code up a recurrent neural network quickly (if you check out Hinton's coursera, there is a lot of Matlab code involved), how do I debug that since the resulting network is opaque and I can only observe it's final behavior? How do we combine subsumption architectures (for robotics) with machine learned systems? We are kind of out of our league in many of these cases, but the areas could benefit from a good PL perspective, and, more importantly, vice versa!

For the machine learning bit, I'm see two things in the short term. First, more data conversion infrastructure for feeding these systems. Pads/ml and mashup style systems have been going under heavy reinvention in the startup and cloud space, and the need is growing. Second, a lot of the optimization work showing up in VLDB/SIGMOD/SOCC/OSDI for ML systems etc. are rehashes of classic compiler and query optimization ideas, and there's a lot more mileage to get out of it. (Heck, I've been tempted to show how to do a few things via incremental query REPLs!).

I agree with you about integrating ML with PL being important in the longer term. There are already projects like Bayesian model DSLs, but that is for the short term. I found the work of Pedro Domingos to be particularly exciting in terms of a framework chasing the big vision.

I can conceive of a VM which includes a profile for the real system on which it happens to be running—either handmade or more likely derived automatically from a suite of benchmarks. So the VM can know that, for example, using a vector now but moving the data to a linked list later will reduce overall runtime—under some set of assumptions. Choosing a representation statically could miss dynamic optimisations, and isn’t terribly interesting—it’s easy to infer that this or that interface is needed, and have a set of defaults in the case of ambiguity. That’s exactly what Haskell does with typeclasses for its numeric types.

The problem with dynamic representations is predictability: just as a GC may decide to run at an inopportune moment from your perspective, a representation manager might decide to do an expensive operation to save time later, causing a noticeable performance drop now. Of course, you could get clever and change representations lazily or incrementally. And at the cost of an extra indirection, you can hot-swap representations in constant time.

For example, one of the big reasons that people use C++ is for its deterministic resource management. I’d wager that “deterministic” is actually better than “fast on average” for realtime applications.

And I’m sure a “sufficiently smart VM” is as possible as a “sufficiently smart compiler”—the question is not whether it can be done, but whether such a costly thing will ever be.

Going to butcher your post a bit, but hopefully not too much so it's out of context:

...
will reduce overall runtime
...
The problem with dynamic representations is predictability: just as a GC may decide to run at an inopportune moment from your perspective, a representation manager might decide to do an expensive operation to save time later, causing a noticeable performance drop now.

The VM could be set to meet realtime requirements rather than reducing overall runtime.

In principle, such a VM could determine that removing itself from the running application (on a specific platform) was the best thing to do.

You're expecting the system to make incorrect choices. If it does so, then that's because it's making incorrect assumptions about the requirements it needs to fulfil. It follows that your source code has failed to express the correct requirements and is therefore erroneous or incomplete.

The problem with dynamic representations is predictability: just as a GC may decide to run at an inopportune moment from your perspective, a representation manager might decide to do an expensive operation to save time later, causing a noticeable performance drop now.

That happens only when you're trying to apply dynamic representations to a language that doesn't support them with a way of expressing the requirements to be met.

My point is that if you have a responsiveness requirement about how the program runs, then your program is *incorrect* unless your source code expresses the responsiveness requirement! The system needs to make choices to meet the requirements of your source code, rather than having you take over and mangle its abstractions and types by doing low-level bit-groveling.

A responsiveness requirement isn't an indication that you can't allow the runtime (or the compiler) to make choices about how and when to do an expensive operation. Instead, it's just one of the constraints that your program needs to express so that the tools can make choices appropriate to your requirements.

Ray Dillinger

a "pair" programming session between human and computer

Existing interactive programming environments such as Smalltalk and APL have this feel, which many developers seem to dislike. Finding out whether that’s a language issue or a deeper one of (perceived) control seems to be the first step to making such an environment popular. I think it’s a combination—not everybody loves REPLs, and most devs take no pleasure in “weird” languages.

Sanskrit is still used in parts of India, apparently. It's even apparently an official language in Uttarakhand, which is a apparently a place. If Sanskrit sentences are commutative, associative and idempotent, I wonder if this would be a good language for thinking about Reactive Demand Programming. David?

A language where sentences are commutative, associative, and idempotent could be quite declarative. But there are many formal languages with those properties (e.g. temporal logic programming, concurrent constraint), yet not many are good for speaking of data dependencies and observer influence (and thus for thinking about RDP).

If (as I interpreted) Pramatias spoke of words within a sentence, not full sentences, that would greatly undermine the expressiveness and scalability of Sanskrit as a declarative language. After all, a sentence is maybe 10-20 words. An essay or program is maybe 100-20000 'sentences'. If we're reducing complexity from permutations to combinations, it's the composition of sentences that matters.

We agree on what Pramatias meant but that's also what I meant when I wrote "sentences are commutative, associative, and idempotent." Those terms apply to operations, not operands, and I was viewing sentences as a concatenation of words.

Concatenation is the operator. A question is whether 'words' or 'sentences' are the operands; the latter enables much more declarative expression. I think your meaning is not obvious in what you wrote.

I find your topic line quite insulting. Please change it.

My meaning wasn't clear, which is why I was explaining it. My topic line was just silliness (along with pretty much everything else I've written in this thread). No offense intended. I've changed it. Sorry! I hope you and the rest of LtU have a great holiday!

Thanks.

Humans will continue to think in terms of analogy and metaphors, with a tendency to anthropomorphise entities in a program, and hence OO programming will remain popular and we should even see some significant advances on object systems. As for imprecision and fuzziness, we should expect our programming environments to support the way we think (and solve problems), not the other way around! Its not that our spoken language is imprecise, but we are.

But then there is a class of people that I barely understand, who think and communicate in radically alien ways from myself and the people I understand, they heavily use math and logic, they talk about elegant constructions where I see only complexity. Perhaps we will understand each other a bit better by 2020, where today we actually know what lambdas are now, and even use them (albeit lightly), but we still don't know what they mean by "monads."

Object oriented programming tends to freeze in a particular 'view' or model, whereas humans casually shift views to reify whatever model is most convenient. OO may remain popular, but that is not a good thing. Unless OO changes considerably to address the issue of heterogeneous views, it will continue to hamstring every human that uses it.

we should expect our programming environments to support the way we think (and solve problems), not the other way around!

Do you believe we should "support the way we think" even when the way we think is commonly short-sighted, close-minded, inconsistent, incoherent, inflexible, inefficient, and so on?

Effective thinking is a skill, not something that comes naturally to humans. I seek effective ways of thinking and problem solving that humans can easily intuit and adopt with exposure and a little education. Our programming environments can help educate and shape the way we think, such that we think better and find better solutions to problems. I would prefer shaping be subtle (via controlling the 'path of least resistance').

Its not that our spoken language is imprecise, but we are.

I think both are imprecise.

Object oriented programming tends to freeze in a particular 'view' or model

Good point. The future of OO programming should involve better support for model evolution and shifts.

Unless OO changes considerably to address the issue of heterogeneous views, it will continue to hamstring every human that uses it.

Its not like the FP crowd are any better at this, I don't see it as a disadvantage or advantage for OO.

Do you believe we should "support the way we think" even when the way we think is commonly short-sighted, close-minded, inconsistent, incoherent, inflexible, inefficient, and so on?

Are you quoting a TOS episode? Yes I do, Mr. Spock.

Effective thinking is a skill, not something that comes naturally to humans. I seek effective ways of thinking and problem solving that humans can easily intuit and adopt with exposure and a little education.

We'd all like the ability and luxury of reaching an enlightened state, but until then there is code to be written! What makes us so versatile is that we are willing to tackle problems that we barely understand or barely have the skills to solve. We don't need to understand everything before we dive in, we can learn by doing. Planning and knowledge are nice luxuries.

There is also a danger in becoming addicted to too much elegance and perfection. You freeze up and can't accomplish anything because you become afraid of problems you still don't understand. A little bit of ignorance can go a long way.

I think both are imprecise.

Language is imprecise because we are, not the other way around.

The FP crowd has begun to address heterogenous views via systematic uses of lenses, and is aided much by purity. Other paradigms - especially logic programming or functional-relational programming are relatively more effective than OOP for heterogenous views. I would not characterize the problem as FP vs. OO, but I think the FP crowd is "better at this".

I never advocated waiting on an "enlightened state". Effective thinking does not require big design up front. It just requires a better framework than humans have naturally - e.g. logic instead of lore, science instead of superstition. I do favor exploratory programming, but I think it's a mostly orthogonal issue.

To "support the way we think" - without shaping or directing how we think - suggests we support expression of ineffective, inefficient, inconsistent, incoherent, fragile mudballs of models, with all the intermediate bumbling and fumbling and fallacy and bikeshed coloring. Because that's often "the way we think", and it is not good or great.

I would not characterize the problem as FP vs. OO, but I think the FP crowd is "better at this".

Lenses and FRP are interesting but so far have not been shown as very pragmatic. Turns out the world is very impure.

It just requires a better framework than humans have naturally - e.g. logic instead of lore, science instead of superstition.

Pure logic and math implies more understanding than we often have; that's not to say it isn't useful but it often isn't applicable! Programmers are more like experimentalists/scientists, forming and testing hypotheses about the world, but then science and math are distinct activities. Imprecision and ignorance are expected.

To "support the way we think" - without shaping or directing how we think - suggests we support expression of ineffective, inefficient, inconsistent, incoherent, fragile mudballs of models, with all the intermediate bumbling and fumbling and fallacy and bikeshed coloring. Because that's often "the way we think", and it is not good or great.

But isn't that the point? For better or worse, we solve problems and accumulate knowledge in a very bumbling manner, probably related to the way our creative fuzzy neurons are firing around in our brain vs. the cool deterministic logic of the computer. I'm sure we can improve ourselves, but I don't see much point to becoming more like the computer that I can program and instruct anyways. Rather, let's just make the computer more like us?

Lenses have proven pragmatic in real apps and even a GUI architecture (lamdu's bottlelib), and are used to manipulate state through views. Functional relational programming (which I do not like to call FRP, which refers more often to functional reactive programming) has been used successefully in various forms (e.g. Ur/Web); it's also about managing state. The world is impure, and that is addressed.

Whether a logic requires more information than you have depends on the logic. There are many logics suitable for open or partial systems. That's the sort of property you should consider when seeking effective tools for thought.

I'm sure we can improve ourselves, but I don't see much point to becoming more like the computer that I can program and instruct anyways.

You seem to treat "improve ourselves" as wasted time. But I see it as an opportunity for time we spend anyway. People will gain skill with whatever tools they use, and exploration is a good way to gain experience. There is much opportunity to teach users to better comprehend computation and automation.

To "make the computer more like us" probably means to hide from users the differences between man and machine. How much computational overhead are you willing to expend toward deceiving your users? And are the results any better? (As far as I can tell, humans also find humans difficult to manage.)

I'm not really up on current research in lenses, but the last time I looked they didn't really address state transitions (induced by events). FRP has a similar problem, though I really appreciated the bi-directional nature of lenses (vs. FRP), which is something I'm designing to a more limited extent into the language I'm working on now (LPL).

The point about computers...we have found great benefits in making computers more like us via machine learning and neural networks. They begin to display all of our drawbacks (ignorance, tending to generalize, fuzziness) but then can do things like process spoken language, which is impossible to do logically (indeed, what is learned is mostly opaque to us). I see us doing more of these kinds of things, not less, and why should humans try to go in the opposite direction by changing us to be more logical when we are trying to get our computers to be less so?

Lenses address heterogeneous views, but it is easy to use the for eventful state transforms: on event read state through lens then write modified state through lens. Done. Boomerang language was built on that pattern.

Humans being illogical causes many problems even outside of computing. That will be true for illogical computers. I agree we'll see a lot more fuzzy aspects in programming, but I think that is no substitute for effective, provable reasoning. Meeting in the middle will better serve both man and machine.

Streams are isomorphic to state machines (possibly infinite-state). As a coinductive datatype, S(A) = να. A × α. However, coinductive types can be encoded in System F as να. F(α) = ∃α. α × (α → F(α)). So we have:

S(A) = να. A × α
     = ∃α. α × (α → A × α)

So a stream is essentially a way of saying "I have a hidden state, plus a state transition function to generate updated states and elements from this hidden state".

IMO, FRP has historically had trouble with events because streams are the wrong datatype for them. An event is something that may happen, so they are better modelled with the type E(A) = να. A + α. This is something that either finishes and produces an event, or tells you nothing has happened now, and you can wait and try again later.

In terms of temporal logic, a stream S(A) is "always A", and an event E(A) is "eventually A".

More operationally, event handling is like programming with promises or futures -- precisely the abstraction that modern libraries in JS and Scala provide! (Furthermore, since the eventually type constructor forms a monad, monadic notation gives very natural syntax for programming with them.)

My New Year's resolution for 2013 is to figure out events. I don't think your above understanding is very useful for the events I'm interested in dealing with for live programming. Consider:

• Events in a Paxos-style replicated distributed system (think withdraw or deposit bank actions). We need some notion of temporal ordering and history to ensure consistency between the replicas. For performance, we want to also process multiple (ordered) events at the same time.
• We want to debug a Kinect program that is basically a stateful transformer of a stream of video input frames (events), where the output is a stream of video output frames. Most computer vision algorithms are stateful; e.g. recognizing a multi frame gesture.
• Each time step in a physics engine can be seen as an event if we take events = state transition, which is quite convenient. Declaratively, certain forces are asserted that are then applied when they are true on the time step, causing other values to change and force applications to change. Discrete interactions like collisions can also be treated as events. Combine with the first point to replicate an RTS like Starcraft to multiple multi-player machines.

So I don't really see a promising future for promises and futures with respect to event handling :) But treating events as streams, or better yet strips (like a film strip used in AfterEffects), is quite useful in modelling event problems. You can even checkpoint your program along points in the strip to time travel (ala Bret Victor's video)

...to this kind of event. The primitives I think are more fundamental are a next-step operator and recursive types. Basically I'm suggesting using a variant of the modal mu-calculus as the type system for a temporal languages.

Your examples 2 and 3 both fall out nicely when you decompose temporal datatypes using next plus recursive types. The same goes for strips, too -- they are basically temporally lazy, possibly-unbounded lists. That is, Strip(A) = να. 1 + (A × next(α)). So they yield a sequence of frames, possibly finishing at some point. I find them useful for modelling things like animated transitions.

I am dubious that off-the-rack temporal logic applies as naturally to reasoning about Paxos, though, since distributed applications do not have a global notion of time -- nodes may run at arbitrarily-varying rates, and message delivery is similarly unpredictable. Next-step is not a natural logical operation in this setting.

You need to be super-careful with time travel, though. Even something as simple as permitting arbitrary values to be used a different time step than when they were created leads to horrendous space leaks. E.g., if you permit using a stream value on a later tick than when it was created, then you have to buffer the stream's values. Checkpoints don't help you avoid buffering, if the stream values happen to come from the network or user.

I partition types into time-varying (e.g., streams) and non-time-varying (e.g., booleans and integers) types, and only permit non-varying values to be used at different ticks. If a programmer wants to use it, they have to do the buffering by hand, so that the cost is evident in the source code.

I find UTC quite effective as a global notion of time in open or distributed systems. It's also better for modeling things like animated transitions, even in a closed system. And it is easy to understand buffering - e.g. exactly how long a value hangs around - if you make static latencies part of the type system.

So a stream is essentially a way of saying "I have a hidden state, plus a state transition function to generate updated states and elements from this hidden state".

That sounds a lot like an object, which is how I'm doing "reactive programming". Events are mutating method calls on objects. Reactive values are const methods, supported by a memoizing implementation. Similarly to what you're describing, any state that you want to accumulate over time must be captured explicitly in an object. There is no notion of 'time' built in to this model; only event ordering. If you have a nice object system (e.g. allows easily packaging subgraphs of objects), then this supports higher order FRP.

But I'm having trouble understanding your comment about event handling being similar to promises or futures. Can you elaborate? I'm guessing this is explained in your latest paper, which I haven't figured out yet. (Also, if you understand the preceding paragraph and can point out any pain points or limitations you see, that would also be very helpful to me.)

This sounds interesting. What's the secret sauce that gives you reactive programming rather than just an object system?

The incrementalization / memoizing implementation. For example, if you implemented a GUI in this style with a typical OOP language and compiler, you'd end up rebuilding the scene graph for the entire widget hierarchy every time you rendered a frame.

How does this work? Does this memoization have a semantic meaning, or is it purely for efficiency? Could you give a small example?

One problem with traditional FRP that I've mentioned before is that "GUI = pure function of time varying application state" doesn't work as smoothly as you'd think at first because widgets often have internal state that you don't want to expose in the application state. So logically a GUI is not a pure function of the state of the application. For example a textbox needs to keep track of where the cursor is, and a sortable table needs to keep track of which column it is sorted by, etc. How do you deal with this?

Memos are an (essential) optimization only. I'll try to give an example tonight. The fact that GUI widgets are stateful is one of the motivating examples for this approach. I think I prompted this question by misstating that the widget hierarchy would be recreated each frame. See my edit.

Edit: I looked back at my notes on this and I'm forgetting an important aspect in dealing with change notifications. I also think a decent example is going to need me to explain my object system a little bit more (which I call Processes, as I've mentioned a few times in the past). I don't have time to get into this tonight, but maybe by this weekend I can put something together.

Are you doing something like incrementalized immediate mode UI? Sounds like we have similar ideas.

This is very pretty but I'm having a bit of trouble imagining how this translates to GUI programming. How would you connect this to a GUI toolkit? This model seems entirely pull based, whereas for practical GUI programming you need a push based model?

First, my understanding is that Ur/Web doesn't use functional relational programming. Its type system supports static computation over record rows, and has a FRP layer whose API seems to focus on cells (state) more than normal FRP, so I don't know what you're referring to there.

Second, intrigued by your comment that lenses are now pragmatic, I couldn't find "lambdu's bottlelib" -- do you have a link? I'm struggling to understand what you mean by "proven pragmatic in real apps" for lenses. Is your statement grounded on experience reports somewhere or personal use, or are you treating the existence of some paper or library as proof of usability? The lenses work is interesting in theory, but I was not aware of non-theoretical connections to software engineering. (The roots in distributed file systems sounded interesting, as did trying to apply them to security, but neither projects seemed to have gone anywhere).

There isn't much to functional relational programming - just a design where state is kept in external relations and modified atomically by functionally specified transforms. On the server side, that's pretty much what Ur/Web does. It even has a type system designed for it. I say it's using functional relational programming because the description fits.

Lambdu is a live programming development for Haskell. (I'm on phone. Linking painful.) I've not used it, but read on reddit from a dev that he is extremely satisfied with the use of lenses. [Addendum: Oops! I mispelled it; this is 'lamdu'. I can't remember where I heard it uses lenses. Reddit, I think.]

IMO, a single app can serve for 'proven pragmatic' if it covers enough bases. Papers aren't practice. Wide adoption is a different issue. I've seen many apps based on lenses, for editing XML, for editable views of code, and for one web business app. IIRC, Lamdu uses lenses to integrate state (text boxes, etc.) with pure, declarative widgets.

I don't agree with the basic theme that functional is somehow machine like while object oriented is human. Structural, functional and object-orientation were all attempts by humans to bridge the human-computer divide. Computers have no need for organization, abstraction nor hierarchy. Computers think in terms of large complex enumerated lists which are permanently non-understandable to all humans.

That being said... I think the major failing of object orientation in terms of moving away from human thought the last 20 years has been abandoning multiple inheritance. When people think of things they think of them as having multiple overlapping and conflicting properties that are contextual. My sister besides being in my family and having her own is a banker and a fashion aficionado. It is impossible to model her behavior without considering all three of those hierarchies and probably several dozen more.

What is so fundamentally frustrating about object orientation as it exists today is that it doesn't deliver on its promise of being able to create models which correspond to human thought at all. When confronted with the difficulties of implementing multiple inheritance the object oriented community blinked and so OO programmers inevitably have to create hierarchies that don't remotely correspond to the way they think about real objects, but instead are an artifact of their organization of functions. I think the most interesting thing going on in object orientation is multi-paradigm: Scala, Perl, Ruby. They are moving us towards a casual object orientation where every developer anytime they have hierarchical data or a complex union of data almost automatically builds an object hierarchy / structure for it. While at the same time not losing the powerful tools for manipulating data on the functional side.

My aim isn't to argue with you. Both this subthread and your points are interesting and veer close to ideas I want to discuss; so this is more free association prompted by your lead. But first I'll back up to something good Sean said. (And, uh, sorry this is so long.)

Sean McDirmid: Humans will continue to think in terms of analogy and metaphors, with a tendency to anthropomorphize entities in a program, [then associates this with OO programming]

I'm inclined to agree with George Lakoff most thought is so heavily informed by metaphor, it's hard to tell whether thought in fact consists mainly of metaphor disguised as absolute reference via decoration by concrete episodic experience detail. (Metaphor may be low-level and close to the brain's "metal" rather than being high level and abstract.) But when a developer anthropomorphizes code, sometimes it's just shorthand for, "I wanted X to happen when I wrote this code, but I'm going to say code wants X to happen, because that's brief and close enough."

Object orientation affords a chance to reify a dev's intention to seek locality in control structure while imbuing components with local responsibility for invariants code must maintain to stay organized enough for a dev to reason about. (If code actually follows these rules, system behavior will look like this.) A dev can explicitly structure code as local agents whose roles encompass responsibilities that maintain the invariants making algorithms work. And when behavior has a interface other code can mimic, this affords an "interchangeable parts" model of engineering idealized by manufacturing physical objects in the industrial revolution. (In practice, it's hard to make fully interchangeable software parts.)

Folks vary a lot in what they think the point of OO programming is. I think it's locality, to simplify reasoning and standardize interaction points of code written by different devs. If you can keep a concept local to one small section of code, you can change your mind by swapping in a new piece. But if a concept is laced through all code in a global manner, you're stuck with it. And worse, if you wonder how a change affects runtime behavior, you might have to look everywhere, which really sucks. (I've been looking at too much C code written with philosophy, "Since I'm not using an object oriented language, I can let my dependencies creep randomly into every corner and no one can berate me.")

JeffB: I think the major failing of object orientation in terms of moving away from human thought the last 20 years has been abandoning multiple inheritance.

If all you want is to satisfy api from multiple base (or super) classes, then any dynamic language with duck-typing will let any object behave as though it inherits as many ways as needed. (Actual inheritance of state and methods is considered just a convenience, and might not be supported.) An OO language with static typing without multiple interface compliance can suffer from the limiting effect you note.

JeffB: What is so fundamentally frustrating about object orientation as it exists today is that it doesn't deliver on its promise of being able to create models which correspond to human thought at all.

While someone may have promised that, I don't think object orientation did. (But I know what you mean, as you can see from my earlier comment on anthropomorphizing shorthand.) Each community is a mixture of many folks saying different things. Some folks are more vocal, and snake oil pitches can sound deceptively simple: drink this simple elixir and you're golden. A conservative and nuanced pitch is easy to ignore because it doesn't promise much, and is hedged by unsatisfying caveats that sound like lack of confidence when merely accurate. I was inclined to tell folks OO was better than following no rules at all, but saying improved locality and reasoning "sucks less" is a weak pitch completely lacking glamor.

The folks saying you could get ideal code via domain modeling that matches human thought were completely wrong, and it was clear they were completely wrong from the beginning, but you can't tell them to "shut up" and stop spoiling the story with dreamy nonsense. It wasn't clear to everyone, and calling the new fad dreamy nonsense can and does get you branded as less bright than hipsters on the bandwagon, as long as you're in the minority. Nowadays the bandwagon occasionally exaggerates how much OO sucks. Hey, it's just an organizational tactic, and all systems suffer from creeping entropy via misalignment, over-specificity, and a host of other brittle narrowing effects.

Whatever replaces OO in coming fads, it will still be good to aim for interchangeable parts, fault isolation, locality for reasoning, and means to substitute components for testing and comparison based on measurement instead of reputation alone. I won't care what it's called.

Your last paragraph notes problems with hierarchies. Yeah, deep hierarchy sucks, while shallow and flat hierarchy can lend some loose organizational structure without becoming too severe of a strait jacket. Models have diminishing return as they get more involved. Some model is better than none, but far too much model leads to a world like the movie Brazil, where your can't get anything done without filling out form 27B-6 (unless Robert De Niro as technician Tuttle arrives on a zip-line to fix the problem without going through channels).

This is a great post, and much better than what I could articulate. One of the things missing from William Cook's definition of OO was the why of it all, how OOP is meant to solve problems, the intention so to speak; otherwise its all just observed technical classifications (objects look like this, functions look like this, end of story).

Whatever replaces OO in coming fads, it will still be good to aim for interchangeable parts, fault isolation, locality for reasoning, and means to substitute components for testing and comparison based on measurement instead of reputation alone. I won't care what it's called.

Objects, functions, logic, components, procedures, modules, relations, and so on, are really fundamental concepts now and not specific enough to be fads. There are many fads that re-spin OOP specifically in one way or the other; e.g., AOP, MDD, CES, but objects themselves are too vague and general to be fadish.

Objects are great (when you actually need them). Object-orientation, however, is a pox.

Hi MicCuster, happy new year! I think we mostly agree.

First off, excellent point about duck-typing. It will be interesting to see if duck-typing does in practice recreate complex multiple inheritance OO hierarchies. It is a very good point, duck-typing might very well be the solution for how one gets the predictability of single inheritance and the flexibility of multiple.

You don't see that much use of duck-typing though to build complex hierarchies. I think that has to do with the fact that dynamic languages are still not used much for huge projects. I was discussing MacRuby above. If we assume that by 2020 (going back to the theme of the post) MacRuby is the dominant OSX/iOS language and Apple is moving 200m+ handsets a year plus still say 5-25% of the computer market, we will have millions of Ruby developers who are building complex applications in Ruby which supports inheritance and duck-typing.

In terms of OO as one of the big strategies for controlling complexity I agree it works. The complexity of software written today is much higher than the software that was written before OO and yet the complexity of architecture is lower. OO deserves credit for complexity control.
Isolation of code has been success from the earliest days of introducing the notion of subroutines, to local variables... It is why the monadic approach of isolating state seems to be making a huge differences in terms of parallelism in practice.

In terms of interchangeable software parts I think libraries and system services have provided the models that are most used, what reuse really looks like in most practice. I don't see any evidence that OO programmers in general are able to make more use of libraries than structured or functional developers, and if anything structured programmers have an edge over either functional or OO in terms of system services. It would take us afield but in general I think the services approach, the UNIX approach, is the right one to achieve reusable code. I agree that Meyer wanted reuse to be a focus and had Eiffel rather than C++ become the first huge success story maybe it would have been but C++ did win and C++ just wasn't all that good at reuse.

There is one area or reuse that I think OO has been hugely success in where functional and structured mostly don't try and that is area of large complex language platform libraries being usable cross languages: COM, WIN32, .NET are huge success stories. Java becoming the assembler for the JVM and Java libraries being used by many dynamic languages running on the JVM is another interesting case, Qt is rather good example as it being used as a widget set for other languages though the vast majority of Qt code is still C++. While I'll give them credit for this area of success I suspect though this isn't so much because of properties of OO but rather GUI libraries are the most complex libraries people use right now and OO does a great job of modeling GUIs.

As for OO and fads, here we do disagree a bit. OO is the most used paradigm. The failures of mainstream software today are mostly the failures of OO. Anyone who is using anything other than OO needs to justify why and they do so in terms of the areas where OO has failed and why those failures are intrinsic. There is a huge difference between being the incumbent and being the challenger. When OO was being praised it was being praised relative to C. What OO promised was to fix C's shortcomings. The big shortcoming it did fix, what it was able to model effectively, was complex event handling. And so I think it is far more reasonable to say that OO does not model "the real world" any better than functional or structural. Rather all of them model some aspects of the real world.

Season's greetings. I like your points and hope folks read them. (I never mind name typos. Personally, I blame my father for having a goofy Irish surname. This is dry humor without a smiley in this case. Call me Rys, it's shorter.)

A political platform metaphor may improve clarity in what I say next. Things like OO programming are like political platforms, composed of lots of planks (varying in how closely related) where planks come and go over time, typically shedding weirder ones when they become a liability. But summaries of an old platform linger on long after like the smell of onions. You can make a similar metaphor about religious doctrine — politics, religion, and tech agendas have elements in common.

I can see how you and Sean filled in gaps to read me as saying OO was a fad, but I only meant to imply related ideas: 1) some planks of what folks claimed about object oriented programming were a fad, and 2) some planks of new upcoming tech are typically also fads.

Over the course of twenty or thirty years, things evolve enough that you can always find some old plank that now sounds quaintly ridiculous, even if other planks are now the accepted rational standard. I sometimes use a dubious rhetorical device of adopting the perspective of critics—possibly imagined—in phrasing my own views, since I'm not really wed to my own view, which I know is weird. Some folks seem to think OO was just a fad, and say a new broom will sweep it away, proved by (insert attack on one of the old weak planks here). I wanted to note benefits we get partially from OO style code we'll want to keep no matter what new tech regime becomes popular. Just like I mostly ignore politics, I don't care about changes in tech platform popularity.

I was tempted to write a dialog illustrating the idea of platforms and planks, featuring a conspiracy theorist named Rocko who claims a long list of facts are true, bundled together in one platform that must be accepted or rejected together as a whole. One of Rocko's planks is, "This apple is red," which is manifestly true, referring to an apple sitting in front of him. But some of Rocko's planks are dubious, like his theory about who shot JFK, and others are just false, like his theory that two plus two equals one. While this dialog could be funny, it's not worth the trouble. The moral of the dialog? A true plank does not make a platform true, nor does a false plank make a platform false. Discussion is more rational when each plank is treated like a separate line item. I'd prefer to see more line-item oriented discussion of planks in tech platforms, rather than voting them in or out as omnibus indivisible batches.

That said, I'm not going to contest anything you said, even though you mentioned COM, which is like waving a red flag in front of a bull I'm not channeling just this moment. I'll just grit my teeth and nod.

I don't agree with the basic theme that functional is somehow machine like while object oriented is human.

My point was that OO tries to model how we think, while functional tries to get us to think mathematically and logically. There are advantages to both, of course, but OO will continue to dominate the large grains of our designs even if we use functional within.

I totally agree about multiple inheritance. I'm a big fan of traits and even virtual classes. There is just so much to explore and take advantage of in the OO design space that I'm depressed about Java/C#, while Scala has great support for OO programming (via traits) that basically get ignored by its pro-FP community.

What is so fundamentally frustrating about object orientation as it exists today is that it doesn't deliver on its promise of being able to create models which correspond to human thought at all.

I would disagree in the absoluteness of your conclusion. OOP is still very accessible, given a blank slate and some object thinking, the average novice programmer can figure out how to get started in solving their problem. The problem with OOP is that these designs just hit walls way too early and to go further one basically has to avoid classes and go with something more ad hoc (but basically still OO).

Hi Sean. Happy New Year.

but OO will continue to dominate the large grains of our designs even if we use functional within.

I would agree with you that functional doesn't seem to be winning the large grain design war at all. It really isn't even interested in it. Functional is launching an attack from below not above: first class functions, lambda functions replacing for with map, replacing accumulators with folds, chaining methods together casually i.e. (1..10).to_s.reverse

The real issue between functional and object orientation is going to be on the issues of stateful vs. stateless. As web programming and thus distribution and parallelism become much more popular the cavalier attitude that used to exist towards state is disappearing. Once state management becomes a pain that developers always need to think about, selling state isolation won't be hard. If you have that and functional techniques: purity and higher order functions then object orientation just becomes like object orientation in a functional language, a way of organizing data.

My point was that OO tries to model how we think, while functional tries to get us to think mathematically and logically.

I understood your point, I was disagreeing. I think humans do think functionally. When I get my car I take my wallet, take my keys, take my coat. There is a notion of "taking" that crosses over objects. Similarly with mathematics. From a very young age children are able to understand that there is something about two trees, two block and two pants that has nothing to do with trees, blocks and pants. Mathematics and logic aren't alien to human thought they are a product of human thought. They are universal to so much of our thinking because they are embedded deeply into the very structures of human intelligence.

Now it also quite true that humans think in terms of things and they think in terms of classifications of things. Notions of "is a" and "has a" are intrinsic to the nature of human thought just as much as mathematics. But how humans conceive of things also involves context and complex hierarchies.

OOP is still very accessible, given a blank slate and some object thinking, the average novice programmer can figure out how to get started in solving their problem.

I don't know if I agree that OO helps here. I'd argue the big features that matter for novices are:

a) The right choice of first class objects
b) A really good GUI
c) Simplicity

Again I'll use Excel as an example, a very pure out of order execution functional language with almost no ability to model datatypes that probably on any given day, ten million novices are writing programs in. Or take Logo, a language that six year olds can work with.

Or to pick a more complex example: years ago, my wife, who doesn't know how to program, nor even knows what a device driver is, wrote a custom device driver in AppleScript. What made that possible was not that AppleScript gave her a natural paradigm from which to model, but rather:

a) It gave her access to the right set or primitives. First class objects in AppleScript was the device's actions in very natural terms.
b) The GUI allowed her to view these first class objects and experiment easily.
c) The rest of the syntax of the language is simple.

On the other hand if she had to see something like the Cocoa Object Oriented view of what was going on and think in terms of initiating a message from an NSObject to another NSObject she wouldn't have been able to figure out how to get off the ground.

If anything, state is becoming more important as we move to distributed and parallel systems. There is a lot of work in isolating the state, sure, and FP has played a large role here (e.g. MapReduce, even if the kernels are written in Java). But then we get new technologies like touch, camera, and speech where huge amounts of real state are required to process even the simplest thing. As we remove unnecessary state from some parts of systems, new necessary state keeps popping up.

When I get my car I take my wallet, take my keys, take my coat. There is a notion of "taking" that crosses over objects.

"I" am taking "things," I never think about the verb in isolation of the action doer, which is essentially what FP is. Math is not intrinsic to human thinking, it took us 50,000+ years after we learned to speak and 5,000+ years after learned how to write to even come up with it. It is incredibly useful, but the curb is steep and we generally take it for granted (forgetting how hard it was to learn initially). Yes, math IS alien to humans, and we must make a conscious effort to learn it (it can't be absorbed while young like language), and any new math we encounter requires learning (category theory and monads).

Again I'll use Excel as an example, a very pure out of order execution functional language with almost no ability to model datatypes that probably on any given day, ten million novices are writing programs in. Or take Logo, a language that six year olds can work with.

Excel leverages what people learned in College about math and accounting. Naturalism is simply about leveraging skills that are already their vs. forcing people to develop new ones. Logo takes what six-year olds know about movement and spatial relationships, and packages them up with a very simple command language.

On the other hand if she had to see something like the Cocoa Object Oriented view of what was going on and think in terms of initiating a message from an NSObject to another NSObject she wouldn't have been able to figure out how to get off the ground.

I'm not sure how we got into end-user programming here. When I said "novice" I was talking about a CSE 101 student who was learning the tools but still didn't understand how to design programs very well. So after we get over the hurdle of know syntax, semantics, and libraries, if you sit a novice down with an OO language, they'll be able to fumble out a solution, while in an FP language, they would have a much harder time dealing with a unnatural verb-focused world. This is just my hunch, perhaps it is something we can test empirically someday.

Math is not intrinsic to human thinking, it took us 50,000+ years after we learned to speak and 5,000+ years after learned how to write to even come up with it.

How do you know this? We have counting instruments from 30,000 BCE. We have evidence of division from 10,000 BCE. And you see math with children all the time.

More importantly I'm not sure when something is invented is terrible important into naturalness. Children all consider cars to be quite natural as a way of transportation. We were early Amazon customers, so my teenage daughter picked up immediately the way was as a family got stuff she wanted was Amazon and EBay. I grew up in a world of giant department stores which existed in the post WW1 era and died during the Reagan era. I found that an entirely natural system of how my family got stuff. Humans are social creates and naturally adopt to their social framework.

As for math vs. language. We seem to have a language mechanism when young. Certainly I agree that we don't learn math like we learn languages at age 6. I do think language at age 26 and math are comparable. Riding a bicycle, history, playing a musical instrument.... are all comparative to math. Language is unique here.

"I" am taking "things," I never think about the verb in isolation of the action doer, which is essentially what FP is.

I'm glad we agree on verbs vs. nouns analogy. "Packing up things is necessary before going on a trip". I can put the emphasis on "packing". Now English sentences don't allow me to completely isolate verbs from nouns, but larger structures do. Your own post contains the example of "learning language" vs. "learning math". The confusion about "doers" you mention in your last paragraph came because you phrased programming based on the verbs of programming not the nouns of the doer.

Essentially all human societies have math
Essentially all humans when they encounter math find it useful and all learn some
Even humans who are bad at math, ask question best answered by math

I'll agree though that math can be hard.

But then we get new technologies like touch, camera, and speech where huge amounts of real state are required to process even the simplest thing. As we remove unnecessary state from some parts of systems, new necessary state keeps popping up.

I'm losing you here. I'm not following the connection of these examples to statefulness.

____

The rest of this response is in the CS 101 post. Not sure why that one outdented.

How do you know this? We have counting instruments from 30,000 BCE. We have evidence of division from 10,000 BCE. And you see math with children all the time. More importantly I'm not sure when something is invented is terrible important into naturalness.

My point was that language evolved, its an intrinsic capability of the human brain that arose from millions of years of evolution. Math is not the same, its something we had to think about and invent. If you don't teach your kids math, the best they'll be able to do is count to 10 (b/c we only have 10 fingers). Its only through heroic effort of a few very smart individuals learning and building on each other's work that we've gotten to where we are today with math and logic!

ssentially all humans when they encounter math find it useful and all learn some. Even humans who are bad at math, ask question best answered by math.

I'm not disagreeing with you and I'm not anti-math. But developing an unnatural thought model (which is essentially what language is) based solely on math for communication seems neither useful nor viable (we can always apply math when we need to). Objects are then naturalistic entities that we can use to encapsulate math vs. using math to encapsulate math (as is somewhat preferred in FP).

I've seen people who can encapsulate math with math naturally, take SPJ for example, and they definitely think in ways that I can't really comprehend. I find that quite fascinating, but if you are already there you might not realize its anything very special or unusual. PERHAPS it is a next stage in our evolution, and I'm just resisting it. Or perhaps not. Anyways, that would occur over a million years, and by 2020 we'll just still mostly be using objects. (Perhaps we can speed it up by allowing only good Haskell programmers to reproduce, giving a little push to natural selection)

I agree that OOP doesn't allow all aspects of a person to be modeled in a single object so how would a RDMS model it? Some data of a person would be associated in an object collection with a unique (person number) and then that number would relate any other objects in any number of other object collections to that object. In this way, you can access traits as a group or individually and also relate them to any number of other traits in other object collections. (Think of a customer table related to invoices, orders, shipments, phone numbers, AR etc)

I intentionally used "object" and "object collection" instead of rows and tables but the idea is the same.

There is nothing in OOP that says that objects have to behave or exist on their own. RDMS seem to get along just fine by just duplicating a column between 2 or more tables.

I personally like multiple inheritance but how easy is it to find the code that needs fixed when it isn't obvious or easy to even list all the code that is being used to change the data?

What is most important about FP to you, is it the "no side effects", the math object libraries, or just the familiar Math terminology for standard CS concepts?

Almost without exception the people who liked CS101 enough to take the next year's course are reasonably naturally talented at programming and reasonably naturally talented at math. So when we start talking CS101, my first question would be why aim at that group?

There are certain hurdles with today's programming environments that require some amount of significant investment. If we discount this investment, we can only talk about the few low-investment/end-user programming environments, and there aren't enough data points to make any very interesting conclusions.

When I work with someone who is good at even languages that don't appeal to people with lots of formal mathematical education, like Quark Express's Workflow Automator (professional desktop publishing automation ) almost without exception the graphical designers that like to abstract the process and think in terms of those abstractions like abstraction in most other areas of their lives.

I would argue that languages like Workflow automator are based on naturalistic flows, recipes, and highly encapsulated noun-like autonomous entities (dare I say objects). I'm not sure what sort of advanced math they need to know beyond what they picked up in elementary school.

A far better argument of what you really mean would be if programming were mandatory and we were educating everyone in it would functional, structured or OO be the best paradigm? And there honestly I'd go structured.

Structured is quite orthogonal to functional or object-oriented. It only emerges independently of the two paradigms because the alternative of unstructured programming was actually popular at some point. Of course, we can have structured without OO or FP, we would often call those languages modular or procedural (though even those concepts are arguably quite orthogonal to OO and FP).

AppleScript is very OO even if it doesn't have much support for programmer-defined abstraction. I think we are splitting at hairs on definitions. HTML is markup and also a bit OO and FP if you look closely (and of course, you need to couple it with JS to do anything complicated).

I would argue we are testing this empirically. We have thousands of language domains where new languages are being created. Some of these languages are successful other are not.

I think you are misunderstanding what I mean by testing empirically. It means setting up a controlled experiment to test exactly one factor of a language and eliminate influence from the other factors. In the real world, we don't really have such luxury; a language is never just one factor. We have feelings for why languages are successful or not, but to be honest we really don't have much clue to what is really going on. Check out some of Stefan Hanenberg's work for more of an idea on what experimental PL really means. Also, getting good empirical results becomes immensely more difficult as user investment increases, meaning its not easy being a usability researcher at AutoDesk.

I'd considered structured languages to be languages that aren't particularly object oriented nor functional. In reading your responses you tend to have a very open definition of object oriented. I would tend to define it like this

Joe takes the ball
a) If you need to develop a ball, a Joe, and a Joe.takes() method then OO
b) If you need to develop a takes method capable of handling Joe and ball then FP
c) If joe, takes and ball are 3 unrelated entities tied together by a line of code, as per the english sentence then Structured

Structured languages I'd include C, PHP, Perl and Python... Which is where I'm grouping workflow. If you are defining what I'm calling structured as Object Oriented then yeah I would agree that most people find structured / OO easier than functional.

_____

As far as testing empirically and then only testing one factor... I think those things may be useful. But it is hard to imagine FP and OO as just one factor. You and I both agree that they encourage different ways of thinking about reality. Very quickly that induces different responses to reality which creates different cultures.

I've never heard "structured" used like that before. I assumed you were taking the term from Dijkstra's famous "goto harmful" paper. So "structured" is simply the use of if, while, and subroutines, which covers just about any modern language that has some sort of function/procedure construct.

(c) is definitely object oriented. As long as you have "Joe takes ball", there is a subject, a verb, and a direct object. That is to say, natural language is very object-oriented, and that's the whole point of what OOP is trying to harness. Just as the point of functional programming is trying to harness the power of the lambda and mathematical thinking, or logic programming is trying to harness the power of automated reasoning.

I'm using it the normal way, where C++ is considered object oriented and C is not. Where OO COBOL is object oriented but COBOL is not. In my book Object Oriented requires: polymorphism, encapsulation and inheritance. Object orientation wasn't an attempt to capture natural language, COBOL did attempt that decades before anyone had heard of Object Oriented.

I don't think I'm outside the norm in seeing Object Oriented as an attempt to control organizational complexity. When we moved from programs that were command line or batch oriented to GUIs the complexity of event programming vs. simple I/O required a paradigm shift in software construction and that was the problem that OO was meant to solve. OO wasn't harnessing natural language. C++ is arguably less natural than C. Objective-C / Cocoa and Java are morasses of language so unnatural that even experienced programmers in those languages have to constantly be looking things up to make sense of code.

I see. I will just disagree then on the meaning of structured, and any taxonomy scheme is going to have lots of overlap (e.g. Java is structured, imperative, object oriented, procedural, ....). COBOL was about natural language syntax, not thinking, and hence it comes off as very unnatural.

Objects should be supported with identity and encapsulation, but polymorphism isn't necessary at all. The only real requirement is that objects support object-thinking, which is just thinking about program organization in terms of nouns before verbs. The only reason I call this natural is because its how most of us think and speak (some people can think in terms of verbs and pure abstractions, but they are sort of rare). That's it, nothing you listed is a necessary or sufficient property of OOP.

hence things like frink and units in f#. it is just that all the other languages are lame.

I'm coming a little late to the party, having just seen this discussion in my server logs, but my language Frink was indeed designed from the first day to have units of measure tracked with all calculations. Frink also has an extensive standard data file which means that you can just use many common units of measure and physical constants, and facts about the earth, planets, particles, etc., directly without going and looking them up. You can often cut-and-paste physical equations and quantities into Frink directly and they'll just work right. (It tries hard to use correct orthography and mathematical notation, including "invisible" multiplies (e.g. x^2 + 2x + 4)

I've found that it's pretty important to have units of measure as a built-in, first-class language feature. If units of measure are a bolted-on afterthought, it's quite likely that most existing libraries won't be able to use them at all without a lot of rewriting, if it's possible at all.

I noticed that Mathematica 9, released recently, was proud to finally have their units facility work throughout the language, as opposed to being an afterthought. (I dunno if it will still happily let you write "Meter + Second", though...) The Fortress language had planned to implement units of measure, too, but that project wound down recently.

What might not be immediately obvious in Frink is that it tries very hard to not just do the right thing with units of measure, but with numbers also. Its numerical type will automatically do the right thing with (arbitrary-size) integers, arbitrary-precision floating-point, exact rational numbers (of any size), complex numbers, and even interval arithmetic. This eliminates another common, silent source of error, and eliminates the programmer having to micro-manage the representation of every number at every stage through calculations, worry about silent overflow, etc.

I'm going to be very optimistic and say that in the year 2020, any language that doesn't trivially and by default do basic mathematics to the level that you learned in elementary school (e.g. rational numbers, arbitrary precision, no silent overflowing, and knowing that 1/2 does not equal 0) shall hopefully be looked on as an archaic mistake. It'll be much more important to Do The Right Thing with numbers than Do A Very Wrong Thing As Fast As Possible.

... my language Frink was indeed designed from the first day to have units of measure tracked with all calculations.

All calculations? Seriously, all? People do occasionally calculate things that aren't measured in a particular kind of unit, don't they? For example, percentage of problems complete, or test scores, or a count of the occurrences of the letter 'e' in a corpus of text or the ratio of e's to l's.

I'm with you about doing the Right Thing instead of doing a Wrong Thing Faster, and I'm convinced that units have value -- but I'm not at all convinced that units are beneficial or even appropriate for ALL calculations.

Ray