LtU Forum

DSL for Streaming Network Traffic Analysis

This just announced at usenix 12:

In this paper, we introduce Chimera, a declarative query language for network traffic processing that bridges the gap between powerful intrusion detection systems and a simple, platform-independent SQL syntax. Chimera extends streaming SQL languages to better handle network traffic by adding structured data types, first-class functions, and dynamic window boundaries. We show how these constructs can be applied to real-world scenarios, such as side-jacking detection and DNS feature extraction. Finally, we describe the implementation and evaluation of a compiler that translates Chimera queries into low-level code for the Bro event language.

Unfortunately, paper is paywalled and not much details are there. I love various DSLs if they are well designed. The interesting property about any sort of network security monitoring is that it can't take very long to reach a decision on whether an event or packet is good or bad. Not all systems have to be instant, but where they have, the configuration language is quite simple - just a selector of properties and comparison against known values (I oversimply here...).

That said, there's ample room for research into slightly delayed responses - say 2 min to reach decision. Having a beautiful language would certainly help. Has anyone seen any related work?

DSL for Streaming Network Traffic Analysis

This just announced at usenix 12:

In this paper, we introduce Chimera, a declarative query language for network traffic processing that bridges the gap between powerful intrusion detection systems and a simple, platform-independent SQL syntax. Chimera extends streaming SQL languages to better handle network traffic by adding structured data types, first-class functions, and dynamic window boundaries. We show how these constructs can be applied to real-world scenarios, such as side-jacking detection and DNS feature extraction. Finally, we describe the implementation and evaluation of a compiler that translates Chimera queries into low-level code for the Bro event language.

Unfortunately, paper is paywalled and not much details are there. I love various DSLs if they are well designed. The interesting property about any sort of network security monitoring is that it can't take very long to reach a decision on whether an event or packet is good or bad. Not all systems have to be instant, but where they have, the configuration language is quite simple - just a selector of properties and comparison against known values (I oversimply here...).

That said, there's ample room for research into slightly delayed responses - say 2 min to reach decision. Having a beautiful language would certainly help. Has anyone seen any related work?

Announcement: Compiler/PLT discussion group in Berlin

[I hope it isn't considered out of line to post such a thing here.]

I'm launching a monthly meetup for PLT-related discussion in Berlin. We had a good get-together earlier this month, and are hoping to get some more people aboard the 10th of July. There will be talks, beverages, and at least half a dozen nerds.

See http://berlin.compilermeet.org for details.

Less is exponentially more: Rob Pike on Go and Why C++ Programmers Aren't Flocking to it.

I was asked a few weeks ago, "What was the biggest surprise you encountered rolling out Go?" I knew the answer instantly: Although we expected C++ programmers to see Go as an alternative, instead most Go programmers come from languages like Python and Ruby. Very few come from C++.

Starting from there, an interesting read on the design of Go, including influences and core philosophy.

Here's the link: Less is exponentially more.

Evolved Turing neural networks - Unorganized machines and the brain

(This is Alan Turing's 100th birthday. And I was delighted to finally stumble upon this that I had missed thus far and I find utterly interesting.)

Evolved Turing neural networks

by Craig Webster and William Fleming

In a report written in 1948 Alan Turing proposed the use of a genetic algorithm (GA) to “train” a particular type of neural network he called a B-type. Despite the apparent advantages of these networks Turing’s proposal has remained undeveloped until now [...]

Evolved design

Evolution is not a rational process and often yields solutions which are unexpected and quite different to those designed in a minimal, rational manner. As a result, evolved solutions are often very difficult to understand. For the same reason it is usually a mistake to second guess, or otherwise constrain, the “solution space” beforehand when tackling complex problems with GAs. [...]

Evolution selects only for the function of the overall network, not for the tidy compartmentalisation of function into individual components. Neural networks of all kinds and the products of GA design have the tendency to quickly become algorithmically opaque for this reason. The advantage of GAs, of course, is that evolution often works when no rational or minimal solution is known, or where finding an optimal solution by other methods would take an unreasonably long time (for example, the Travelling Salesman Problem which can take longer than the lifetime of the universe to solve).

Unorganized machines and the brain

In a fascinating and farsighted report written in 1948 Alan Turing suggested that the infant human cortex was what he called an unorganized machine[...]

Turing defined the class of unorganized machines as largely random in their initial construction, but capable of being trained to perform particular tasks. There is good reason to consider the cortex unorganized in this sense:
there is insufficient storage capacity in the DNA which controls the construction of the central nervous system to exactly specify the position and connectivity of every neurone and by not hard-wiring brain function before birth we are able to learn language and other socially important behaviors which carry great evolutionary advantage [...]

Link to the November 1946 letter of Alan Turing to William Ross Ashby, advising him to experiment with these ideas by using the ACE.

Also :

Turing's B-type neural networks and Turing's Last Programs (on AlanTuring.net), and

The Alan Turing Home Page (by Andrew Hodges)

Pythonect 0.3 released

Hi All,

I am pleased to announce the release of Pythonect 0.3.0, available from https://github.com/downloads/ikotler/pythonect/Pythonect-0.3.0.tar.gz

This version fixes several bugs and adds some important features.

Many thanks to everyone who contributed bug reports and feedback that went into this release!

What's New in Pythonect 0.3?
============================

Core and builtins
-----------------

- Feature #13: Improved print function. For example:

print "My favorite number is" + ": " + str(73)

- Feature #15: Implemented Stateful Interpreter. For example:


73
_ -> print "My favorite number is: " + str(_)


- Feature #17: Remote procedure URL can be an expression. For example:

fcn_73@"xmlrpc://localhost" + ":8000"

And

fcn_73@"xmlrpc://localhost:" + str(8000)

Are both equivalent to:

fcn_73@xmlrpc://localhost:8000

- Feature #18: Implemented Multiprocessing. For example:

"Hello, world" -> print &

Will print "Hello, world" in a new process

- Feature #20: Backslash can be used to join two or more physical lines into a logical line. For example:


print 7\
3


will print:

73

- Feature #22: Implemented None as pseudo remote protocol / URL. For example:

fcn_73@None

And

fcn_73@"None"

Are both equivalent to:

fcn_73

- Issue #14: Print does not act as a pass-through statement

- Issue #16: TypeError Exceptions are not been displayed

- Issue #19: Autloading is not working in a statement

- Issue #21: Preprocessor breaks on a List with a String that contains comma

Build
-----

- Issue #12: No newline at the end of _version.py (PEP8)

Regards,
Itzik Kotler

Null, bottom, etc.?

One of the annoying/frustrating things about Java's type system is that all reference types are inhabited by null so there's no way to specify, in the type system, that a reference actually refers to an instance and not null. It seems like Haskell's bottom is similar to this.

What are the differences? And where are the type systems that avoid having all types also inhabited by some nullary type?

Framing: The implementation side

One of the main obstacles for formally verified imperative software is the so called framing problem. A solution to the framing problem has to give a precise answer to the question: "What does a procedure modify and what does in leave unchanged?".

Some weeks ago I have presented the specification view of a new attempt to solve the framing problem (An approach to framing and mutability).

The following paper describes the implementation view of the frame contracts and how the implementation view and the specification view can be kept consistent.

Furthermore it demonstrates how loop invariants can be kept readable by using ghost functions.

'FeML: a skeleton of a femto-ML with nothing but polymorphic variants and functions

'FeML: a skeleton of a femto-ML with nothing but polymorphic variants and functions'
message at http://lists.canonical.org/pipermail/kragen-tol/2012-June/000958.html

As I said on the HN post, this sounds great but better options on the metal sound good - at the moment it's C or Forth. I would love to know what forum members think?

Why are exceptions not described as 'purely functional'?

Say we have the expression -

(/ 1 0)

In my purely functional programming language (with strict evaluation), I currently handle the evaluation by returning a 'violation' value that describes the problem like so -

(violation :v/contract/divByZero "Division by zero." ())

This ‘preserves’ referential transparency for all expressions by making all them replaceable by their value, if only by a violation. But this different than other purely functional languages I've seen.

Whenever I read a description of the properties of impure functional languages (such as ML), exceptions are described under the imperative section as an imperative feature. I find this a little odd as I'm not sure how a purely functional language would handle a division by zero error without exceptions (or without violations like I use).

Haskell has exceptions, but they can only be handled in the IO context. This makes sense to me because Haskell is lazy. My language is strict, so I don't see why I would be violating purity by having exceptions. After all, what makes a language purely functional is the pervasiveness of referential transparency (replacement of any expression with its resulting value). But why would we care about preserving referential transparency for the set of expressions that have no possible value in the first place? Would the replacement of violations in my language with exceptions technically render it 'impure'?

XML feed