Home » forums » LtU Forum

Nimrod: A new statically typed, compiled programming language which supports metaprogramming

Nimrod is a statically typed, imperative programming language that tries to give the programmer ultimate power without compromises on runtime efficiency. This means it focuses on compile-time mechanisms in all their various forms.

Beneath a nice infix/indentation based syntax with a powerful (AST based, hygienic) macro system lies a semantic model that supports a soft realtime GC on thread local heaps. Asynchronous message passing is used between threads, so no "stop the world" mechanism is necessary. An unsafe shared memory heap is also provided for the increased efficiency that results from that model.

I would like to share this language here as it has recently had a new version released. Nimrod resembles Python in that it uses whitespace to delimit blocks and Pascal because of the way that types are defined (TType). It supports metaprogramming with macros and templates. Code example:

# compute average line length
var count = 0
var sum = 0

for line in stdin.lines:
  count += 1
  sum += line.len

echo "Average line length: ",
  if count > 0: sum / count else: 0

By dom96 at 2013-05-21 19:20 | LtU Forum | previous forum topic | next forum topic | other blogs | 15866 reads

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Looks like a nice, practical language

I like the pragmatic approach to the language design with type theory applied sensibly. It looks like a "user friendly", practical and performance-oriented language. I see it mainly as an alternative in the C++, D, Rust etc. language group. I prefer the syntax a lot over Rust for example, but in Rust and C++ you would have more control over memory allocation, reference borrowing etc. I guess it's a trade-off between performance and simplicity. How big is the performance sacrifice in this regard?

Another thing I'm concerned about is safety. How safe is the language compared to for example C++ or Rust when it comes to:

- dereferencing invalid pointers
- invalid array indexing
- arithmetic overflow/truncation/wrapping
- null
- exceptions
- side effects
- race conditions, deadlocks etc.

And are the safety checks performed statically or at runtime? I've gathered some information from the tutorial, but it would be nice with a more thorough description of the different safety aspects.

By phazer at Wed, 2013-05-22 10:57 | login or register to post comments

since you mentioned Rust

i have been holding my breath that Rust will turn out well, after BitC bit the dust. still, i've heard from people i respect that things like message passing might not be the best way to go, but of course everybody has their own take on what to optimize for.

(it really really really irks me that any time i use the term 'go' now i have to decide if it is a purposeful pun or not.)

By raould at Wed, 2013-05-22 17:55 | login or register to post comments

Rust

Message passing seems like a compromise to me: it's simple to understand, enables use of thread local heaps, is easy to implement and allows safe exchange of data between threads, but you're right it doesn't cover all aspects of concurrent programming.

I like Rust but I'm not convinced that putting the burden of specifying lifetime and borrowing semantics on the programmer is the way to go. Most of the time I don't care about that stuff (even when I write time critical code), I just want guarantees that the program is correct and that the performance and realtime characteristics are predictable and reasonable.

By phazer at Wed, 2013-05-22 20:01 | login or register to post comments

Burden the compiler instead

I'm not convinced that putting the burden of specifying lifetime and borrowing semantics on the programmer is the way to go

Writ large, I share your concern, and personally think the burden absolutely limits the wide application of Rust simpliciter. However, I consider its thoroughly explicit declarations to be a huge advantage in automated code generation. One could use Rust as the "compiled" language for a Lisp or Haskell, and its burden of low-level declarations become a rich set of primitives over which to mechanically optimize.

In the immediate "sniff test", Nimrod appears much closer to the level of programmatic thinking in which I tend to reside. But once we've accepted metaprogramming (just another source-to-source transformation to my mind), I start viewing a language in terms of what I can express through it, rather than in it.

That being said, Nimrod is exactly the kind of exercise in "fairly low but eloquent/expressive" that I like perusing for inspiration on those mid-level languages that emit down the language chain.

By Chad Wellington at Fri, 2013-05-24 23:44 | login or register to post comments

I think message passing is the only scalable model.

Having threads (or processes) that do not use a shared memory model and instead pass messages via explicit channels is probably the only way to implement anything so that it will scale smoothly to an installation that requires a server farm to run. And I am not speaking here of procedure calls when I say "message passing," as has been the fashion in the OO universe. I mean independently operating, asynchronous processes (or threads) that read and write a designated message queue in order to communicate with each other.

I'm pretty strongly of the opinion that shared-memory concurrency has given us both more, and more "interesting" bugs and safety problems than any other recently popular programming practice.

Incidentally, I am mystified by the choice of the name for Google's recent programming language to write massively-scaling apps at low level; they had the same basic epiphany about shared-memory concurrency being doomed for such scaling apps and found existing languages inadequate; I get that much. But then the world's biggest search company -- which is not keeping their new language secret -- picked a name for it which is such a common word that for some searches, their own flagship search engine ignores it. I'm thinking that move was either crazy, or in pursuit of some unknown strategy whose purpose I cannot fathom.

By Ray Dillinger at Thu, 2013-05-23 17:54 | login or register to post comments

all will be explained in the movie

you know. the google movie. (gag me with a spoon.)

By raould at Thu, 2013-05-23 21:57 | login or register to post comments

Category error

But then the world's biggest search company -- which is not keeping their new language secret -- picked a name for it

That's a category error. Google has thousands of employees and it's not like they hired Rob Pike to work on SEO. He certainly has a history of preferring brevity, so the name isn't out of character.

By munificent at Tue, 2013-05-28 17:04 | login or register to post comments

Note: I'm the author of Nimrod

- All variables including pointers start as initialized with a binary zero (nil, '\0' etc.) so dereferencing an invalid pointer is a runtime error. Not nullable pointers also exist but are not enforced yet.
- Array indexing, arithmetic overflow etc. is checked at compile time if possible but otherwise at runtime. Nimrod lacks dependent typing but hopefully version 2 will get it.

- Exceptions are checked not unlike Java, however the checking is optional; this is the best of both worlds IMO. Exception tracking is part of Nimrod's more general yet unfinished effect system: http://nimrod-code.org/manual.html#effect-system
- The concurrency model is still in development: For now if you use Nimrod's channels to communicate there can be no race conditions nor deadlocks, but there can be if you use the shared memory heap.

A better model that allows for safe memory sharing is in the works: The effect system will guarantee race freedom as well as deadlock freedom at compile time. I plan to have locks built into the language as non aliasable globals with an explicit locking order and 'shared' types that are annotated by which lock(s) they are supposed to be protected. Time will tell if that model restricts expressivity too much or not, but IMHO this should work way better than message passing for Nimrod's intended domains. ;-)

By Araq at Fri, 2013-05-24 22:13 | login or register to post comments

Useful info

Thanks for the response. Interesting that you plan to add other ways to do concurrent programming. A full dependent typing system is a big task to take on, it will be interesting to see how it turns out.

By phazer at Sat, 2013-05-25 20:30 | login or register to post comments

This is good work.

Nimrod looks very well-designed. I note that it's the first other language I'm aware of that has basically the same macro system (hygienic, AST based) as Scheme. And it looks learnable enough (syntax-rich and reasonably familiar in form to users of C'ish languages) to be learnt by many people.

There are serious scaling mistakes that it *doesn't* repeat, such as the debacle with header files in c++ programs. They are definitely aiming for "read a module once and once only," in compiling large projects.

By Ray Dillinger at Thu, 2013-05-23 18:53 | login or register to post comments

good for hunting rabbits?

I'm guessing the name Nimrod is related to Babel? And that's the tongue-in-check reference in a programming language context? It can't be the (very) slightly derogatory epithet aimed at a dimwit or numbskull.

I've heard Bugs Bunny use the word Nimrod more times than anyone else, so that's what comes to mind. I think it's what he called Elmer Fudd, naturally because he was a hunter.

The rest of this post is off-topic, but ironically it might increase discussion of your project if it entertains folks. This is a true story, but I'll have Wil say my lines. Your mission, if you choose to accept it, is to figure out what's going on in Bob's mind. The cartoon mentioned is one I thought of several months ago when reading this site — I forget the original context — but never found a reason to mention before.

     Wil, Bob, and another coworker joked about random topics while walking back to the office from lunch. The topic of cartoons was mentioned.
     "That reminds me of a cartoon I thought of recently," Wil said. "The characters are from Loonie Tunes."
     "Okay, what was it?" Bob encouraged.
     "The first panel shows Elmer Fudd at home in his living room," Wil explained, "reading the newspaper, feet in socks, wearing just undershirt and trousers. A 'ding-dong' in the air means the doorbell rings. Next panel: Elmer answers the door, paper in hand, to find Bugs Bunny and Daffy Duck standing there in hunting costumes just like Elmer usually wears, holding shotguns. With a solemn look, Bugs asks Elmer, 'Do you know why we're here?'"
     "Ah, ha, ha, ha!" Bob laughed while beaming. "I remember that one."
     "You remember that one?" Wil asked, totally puzzled.

After visualizing Wil's description, why does Bob think he remembers it?

By Rys McCusker at Fri, 2013-05-24 23:13 | login or register to post comments

memory

One test of whether you are remembering something or inventing it is how quickly and easily additional unspecified details pop up. Because all three characters are well defined and recognizable it is very easy to conjure up the details.

Another test of whether you remember something is if the characters and the situation seems familiar. This test is a little more prone to false positives but in the heat of conversation I often say I remember something as soon as it seems familiar without waiting for the details to pop up.

So what might have happened is he thought "I think i saw a comic with Bugs Bunny or Elmer Fudd in it a couple years ago" and "I can picture Daffy Duck standing just to the right of Bugs on the doorstep". And since he might remember it, the safer conversational tactic is to say that he knows it, because saying that you are unfamiliar with what the other person is talking about often ends the conversation. The fact that you'd invented the comic yourself was just an irrelevant detail.

By Anders Horn at Mon, 2013-05-27 06:02 | login or register to post comments

Deflating joke by explanation.

By way of context, when I design something like fiction, I intend parts to be familiar, as signposts telling an audience they're on track, as well as providing suggestive context I can leverage for free. Comic imagery is then best stereotypical: a guy reading a newspaper in his socks, sitting in an easy chair, ought to be familiar. The more stock such a starting image, the easier it is to implicitly ask folks to situate themselves in the POV of the character. The only novel element of such a cliché image is that it's Elmer Fudd, subsequently confirmed by the next panel. A doorbell ringing is another boring staple.

Simple turn-about seldom occurs in Loonie Tunes cartoons, maybe because it's too obvious; surprising story evolution is better. Bugs and Daffy armed with shotguns is therefore a kind of latent cliché, discarded by cartoonists as too simple. The question that Bugs asks Elmer, however, is the sort of thing a traffic cop asks after pulling you over: "Do you know why I stopped you?" And the way I phrased it is close to the way Samuel Jackson addresses young punks in Pulp Fiction when he and John Travolta pay them a visit in their hitmen capacity, and I'd expect Bob to recall that too. It's essentially a terrorizing tactic, to ask a victim to expound on their newest problem — not that I'm saying traffic cops aim to terrorize.

The part I like most about fiction, of any sort, is asking folks to infer something beyond what is given. Getting to infer things is a main source of entertainment, and generally seems latent in most humor. It's unlikely to occur unless you tickle the part of someone's memory that coughs up the association to drive any inference. [Edit: so if a reader infers your point before you get there, they'll like what you wrote more than otherwise, and most of this post exists mainly to support the idea so folks will do it more on purpose when writing.]

And since he might remember it, the safer conversational tactic is to say that he knows it...

Yes, maybe it was just a conversational gambit, but when offered on the tail of laughter in that sudden, spontaneous, carefree manner I associate with honest confession, I had trouble fitting that to what happened. Given no way to know, usually I just treat all the possible interpretations of someone's mindset as existing in superposition, with different probabilities attached. Horribly geeky, I know. My original question more or less asks what I should include in the probability curve.

By Rys McCusker at Tue, 2013-05-28 03:29 | login or register to post comments

Here is some belated feedback on Nimrod docs.

Before I say anything else off-topic, first I'll offer a short review of Nimrod docs. The good news: there are some docs. The bad news: they are quite terse. I can give examples of brevity below. Before I do, here's a characterization of why I find docs too brief, so far. In an explanation of each topic or sub-systems, I prefer to see both a general statement of a rule, plus a concrete example, rather than just one or the other. Parts of Nimrod docs now read like the budget constrains a writer to a single sentence per area of discussion, so we either get an example, or a rule, but not both. For example, the documentation page itemizes eleven other pages, each described by a single sentence, with the exception of internal documentation, described by these two sentences.

The internal documentation describes how the compiler is implemented. Read this if you want to hack the compiler.

The writing is reasonably clear, but somewhat short. With a couple code examples removed, the entirety of internal documentation is just a few hundred words long, about as much as I can copy by hand in about an hour, or skim in a couple minutes. Coding guidelines consist of six sentences and less than fifty words. In contrast, if we go to Nimrod's wiki and check the Coding-Guidelines, we find five sentences in about sixty words. On the topic of indentation, about which it's hard to get folks to stop talking, the entirety of text is this sentence:

Convention is to use 2 spaces, however 4 spaces are also fine.

I admire self-restraint about whitespace convention, but it's funny my sentence here about it is longer.

Edit: this next section may be irrelevant, per Araq's point that bug report quality has nothing to do with quality of docs. You can ignore the following quoted block.

I said to myself, "Maybe if I go read bug reports, I'll find long descriptions of problems with actual examples, so I can place more ideas in further context." The community page says bug reports are listed under issues, but when I sample a few links there, I usually do not see a whole paragraph per problem (anything longer than a single sentence). After picking a few at random, Gradha's Improve error message for redefinitions report looked like one of the longer ones, which actually spells out the issue in prose:

It would help if the redefinition error could also point to the file/line where the original definition is, since a naive case sensitive grep would not find the source of the problem.

That's longer and more complete than a few other bug reports I saw.

In general, compared to existing docs now, this review of them is fantastically verbose. You can flesh out docs without fear it drives off folks who want answers to questions. Sometimes a wall of text is better. You're entitled to do whatever you want, of course, but you get better docs quickly if each time you write something, you include 1) context, 2) generality, 3) examples, 4) comparison, and 5) warnings. (Feel free to add more to this though.) For example, suppose you wanted to write about how cross-walks prevent pedestrian injuries in city streets. Instead of just saying, "Use the cross-walk if you don't want to die," you can expand that to a whole paragraph like this:

Protecting pedestrians from homicide-by-automobile can be addressed by using a protocol in street organization and markings. Let's use crosswalks now, because tunnels seem too expensive. When little Billy wants to cross the street, he goes to the corner and presses the walk button, then waits for his turn. Note Billy should actually look first, to see whether a car will run the light when it's his turn.

For extra credit, if you want to frighten folks, include a YouTube clip of Brad Pitt getting splattered in Meet Joe Black. Obviously you won't find many movie clips about programming.

By Rys McCusker at Mon, 2013-05-27 18:16 | login or register to post comments

Re: Here is some belated feedback on Nimrod docs.

Thank you for your feedback. I know the docs are sparse but then your critisism is out of proportion: I'm not responsible for the length of a bug report nor do I have any problems understanding short bug reports. I have no idea how you got the idea that bug reports are some sort of documentation. It is as if you missed both the tutorials and the manual and only read the internal documentation!

By Araq at Wed, 2013-05-29 18:57 | login or register to post comments

Keep doing the right thing; looks good so far.

I'm sorry that read like criticism; it wasn't my intent to be critical. I meant to characterize shortness and offer a practical, constructive suggestion to improve docs quickly as you go in a first pass, by considering a nod toward each of the five items I put in a checklist (and more items if you think of them). I also hoped it would help other folks who write, so the last part of my post isn't aimed at you specifically.

What you have so far deserves praise. I see lots of signs of talent, taste, and good design, etc. So encouraging you is practically useful, because the world becomes a better place when folks like you do good things. I hope you keep at it. You seem so good at the coding side that just writing a bit more would promote what you're doing usefully. I seldom praise because I fear it can spoil internal motivation in other folks, if it helps create a craving to hear people say nice things (which I think is irrelevant :-).

When you write in the future, just add a bit more, but not so much it ruins narrative flow and distracts a reader. Just walking folks through what you find obvious is helpful. I wouldn't ask you to go back and make old docs longer; that would be work and not especially rewarding. However, in general don't follow suggestions from others without making it your own idea first, so you retain natural style and clarity.

I apologize for any implied criticism. And I hope my earlier goofing around with Elmer Fudd was not offensive. (Some folks might free associate Nimrod with him though.) Nimrod is a pretty good name in terms of uniqueness, brevity, ease-to-say, and lack of confusion with other words.

By Rys McCusker at Wed, 2013-05-29 22:46 | login or register to post comments