Metaprogramming and Free Availability of Sources

Metaprogramming and Free Availability of Sources by François-René Rideau

This is an older paper about metaprogramming and it ties into open source development process and why metaprogramming requires free/open source software.

The beginning of the article outlines a list of types of metaprograms such as compilers/translators, phase splitters, code walkers, meta compilers,explainers, etc. Later on it says that metaprogramming is required as problem complexity increases and there are more actors/stakeholders involved.

In the traditional development process, that excludes metaprogramming, the near totality of the code constituting software is hand-typed by humans; the details of execution, the overall behavior, everything must follow directly from human thought; every stage of the development, every modification is the work of man. Now, the catch is that sooner or later, some new functionalities to add to a software project will require global architectural changes of some depth: all the global modifications of the program will have to be performed by hand, with all the inconsistency problems unwillingly introduced by these changes, that induce numerous bugs and cost lots of iterations of development (for a publicly documented such phenomenon, see the occasional API changes in the Linux kernel; for a spectacular example, see the explosive bug of the first Ariane V rocket). The alternative to these modifications is a complete reimplementation, whose cost is that of a rewrite from scratch. All in all, the incremental cost of traditional development is proportional to the size of code that differs between two iterations of the process.

Let us now make metaprogramming part of the development cycle. Since metaprogramming allows arbitrary processing of code by the computers themselves, it enables the programmer who has to face a structural change in his program to perform semi-automatically any task of instrumentation or transformation of his program, so as to make it conform the new architecture, to check his new code with respect to declared invariants, to enforce consistency of various parts of his program.

In statically typed languages or at least in better IDEs, it's possible to perform various refactorings across a codebase which can constitute metaprogramming in a crude respect:

tiny doses of metaprogramming used in traditional software projects are a determining factor for the continuation of these projects: choice, often static, of a development environment; filtering of programs with static analyzers, checkers of more or less sophisticated invariants, or other kinds of lint; use of editors that manage the indentation and structure of programs, that they can colorize and fontify; occasional use of search-and-replace tools on the source code; in the case of more daring hackers, “macros” and “scripts” written in elisp, perl or other languages, to edit and manipulate sources.

The question I pose to LtU is what are the ways in which metaprogramming can be introduced in the development of proprietary programs in mainstream languages such as JavaScript and Java.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

For C++ use Felix

Well, if you want to "meta-program" C++ you can use the C pre-processor (token macros), you can use templates (syntax macros) ...

Or you can use Felix, which has a real type system.

With the builtin ability to define and extend the grammar, you can invent Domain Specific Sublanguages (DSSLs) to codify housekeeping tasks.

You also use the hard coded tools built into the run time system and compiler to automate management of a wide variety of features known to interoperate combinatorially, such as product, sum, and exponential objects managed by a type system and a garbage collector.

Perhaps the most important of these builtin tools is the ability to do cooperative multi-tasking using fibres which interleave control using synchronous channels (a bit like CSP but without concurrency). Hand coding this is very hard. The model subsumes and, IMHO, is superior to, all of functional and procedural programming.

I really need a theorist to help develop a better type system for the fibration. However it solves some major problems, for example it eliminates some of the killers of functional programming: issues of evaluation strategy evaporate. The master/slave relationship between a function client and the function is also gone: functional programming can be recovered with an idiom which shows the machinery subsumes it.

JavaScript metaprogramming at run-time

Right now I'm working on a small metaprogramming library for form validation. I'm using a JavaScript object where the keys and values are transformed into the underlying configuration needed for existing libraries. The neat thing is that this makes the form validation configuration object a meta object that creates a few objects and hooks up the functionality.

What's even neater is that I can use this meta object describing a form and its validation as the basis for a code-generator.

The only disappointment is that I can't use macros in JavaScript and the only fear is that coworkers won't be able to hack on the meta-object and its implementation or won't understand it and will ignore it completely.