Lambda the Ultimate

Hi Ehud, I'm the site originator. Compiling on the server and supplying compiler warnings/errors (and maybe even running unit tests in a secure sandbox) is on my to-do list. Ideally, it wouldn't allow you to even submit content that isn't clean. I'm just getting started, but I appreciate your feedback!

I've been thinking about that, actually. I guess you should report errors, and let the page editor decide if he wants to go forward and post the faulty code.

Since I am thinking about using such a thing for teaching purposes, I want students to post ideas, even if they aren't sure how to make the work - so that their fellow students can fix errors, refactor the code, or fix the design.

I don't see why code that emits errors and warnings shouldn't be permitted to be posted. Indeed, I'd think that the server wouldn't care whether there are errors or not. Just capture whatever it is that the compiler emits and spit that out somewhere. The output could be anything from: "Compilation successful" to "Maximum number of Errors exceeded".

It'd be quite a different thing if we were talking about code that actually ran on the server. There yo would need the concept of clean and safe code - something that'd be a little hard to prove. The only danger in compiling code is feeding it code that causes the compiler to crash or go into endless loops (or just spend way too much time groking the code).

Just capture whatever it is that the compiler emits and spit that out somewhere.

In my imagination, compiler messages (let's not forget warnings) will appear below the line of code they relate to (similar to what compilers listings look like).

One issue with compiler messages is that the server can only operate in a single environment (a particular setup of Gentoo Linux), and I expect to take code submissions for Windows and other OS's I don't even own a copy of (ref List of environments). Some of this can be done on other machines over the LAN. This isn't such a big deal for portable code, but more so for stuff using the Win32 API or the .NET Framework like on The Code Project.

You are right, of course. You don't want to set up an infrastructure of the kind big companies QA departments use. I say, support compilation only for code that uses the language+standard libraries.

For the LP wiki I suspect that the best approach will be to have any compiler errors reported directly on the page in question. That would make it a lot easier for editors to find and fix any vandalism that might occur. I suppose that a similar rationale would apply to educational uses like Ehud is suggesting, except that the "vandals" in that case would be students who didn't fully understand what they were doing :-)

What about a user downloading and running what he thinks is the code for the "Hello World!" python example when in fact the code has been replaced with the equivalent of 'rm -rf /*' ? Compiles fine, no errors. However the output is not quite what the user is expecting.

Well, I'm not suggesting that compiler errors would be the only way to catch vandalism. It simply aids the process.

This is a good point, and I don't have a really good story for this right now other than "read before you run". Since it's targetted more at programmers than end customers I don't think this is too big of a deal, although a vandal could obscure malicious code. Maybe the MediaWiki validation feature, where a new version is not made available for download until someone verifies it's okay, could be helpful. I don't think I'm going to take any preemptive action just now though.

Actually I think the risk of code running on the server is less than you think as long as it runs in a jail. It's okay if the code or compiler crashes or runs forever. If it crashes, it only kills that process, and I can use gdb to collect a stack trace for display. If it runs too long, I just kill it. I can create a special restricted user account and use chroot and quotas to limit what it can do. Of course I can't run code that only works in other environments (especially if those environments can't be made secure like Windows 95).

When running code on the server one should also prevent network access and probably a handful of other things. A friend of mine running a public test-server for alogorithmic contests (where you supply a solution as source code) uses ptrace syscall to disallow file and network access. This is quite easy under Linux, where you can for example only allow a few syscalls, that are going to be needed.

Running untrusted code on the server is a problem tailor-made for Xen and VMWare. Both have free versions for something like this; Xen is OpenSource, VMWare has a "VM player" version. Setup a virtual machine with policies for network access, disk space, etc. and let the code run in it. When it's done (or enough time has elapsed), just tell the Hypervisor to kill the VM.

You could also setup the VM to have read-only access to an NFS share on the server, running code directly off it. This way you wouldn't need to set *any* limits on the running code or set any special policies. Anything the code did to the VM would dissapear when the VM was killed.

It's also OS-neutral; Linux, BSD, Solaris, Windows, etc..

The only downside is the heavier processing requirements; corporate servers typically run 15-20 VMs per server box (~4 VMs per CPU core).

--Bryan

That's a good point. It also fits with the general wiki spirit that it's okay to leave things undone. I'm also considering ideas for allowing articles to use code from other articles, or even extend/modify another article's code (sort of a "literate patch"). If you're interested in setting up your own copy for teaching I could post a patch of my changes, but it'd be even better if they could contribute directly to the main site. Thanks for your interest.

I think setting something local is the best approach, if I want students to feel safe. But I'll have to see how are tech people will react. I'll get in touch...

Thanks.

Perhaps the finished version of student pages (maybe reviewed by you first) could be moved onto the main LP site? That would allow the students to feel safe while they were learning and developing a particular piece of code, but also help to fill out the content on LP.

Can I download the changes you made to MediaWiki from somewhere?

Cheers!

This page explains how to get the lp source from svn.

My own thoughts on how LP might be used in an educational setting were more along the lines of a reference for well-written and (hopefully) very well-explained code. Kind of a dynamically generated, online textbook. I hadn't thought about getting students to collaborate on code. But I can see that being a useful teaching tool too.

The LP backend could ultimately become quite a useful collaborative development tool. And a kind of version control is already built in to the system via page histories. Something to think about for the future I suppose...

This is an interesting idea, but there's some risk with scaling up from one-article code samples to larger projects and it's not yet clear to me how to address this. Even if changes are restricted to a fixed list of trusted developers, large projects are inherently more fragile in that a change to one area can break many other areas. If the software had some kind of branching support, it might be possible to have a scheme where modules are independently edited on private branches without breaking other modules, then committed to a public view when they're believed to be safe.