Lambda the Ultimate

so probably the most relevant PLT-related systems are those that place complex (or not so complex) orders to offload as much of that as quickly as possible.

I'd call that rather direct involvement, even if it's not involvement in the causes of the crisis. So does this mean people responsible for these systems are going to be in high-demand now, I wonder?

From what I've seen, it's a very sticky situation for PLT fans in finance.

On the one hand, there's tremendous opportunity to apply the logical and linguistic tools of the PLT specialist to *tons* of problems in finance (order management/execution just being one such example). On the other hand, there's not widespread appreciation of the value of PLT ideas. If you interview somewhere on Wall Street, you'll probably be expected to understand Black-Scholes option pricing or at least the basics of bond pricing, but you won't be asked to explain how a fixed-point combinator works. Most people I've encountered view that sort of knowledge as a novelty, but otherwise not very valuable.

That's slowly changing, with examples like Jane Street Capital, some groups at CSFB, Goldman, and others putting Haskell and O'Caml to work (I decided to get into this business in the first place after reading the SPJ et al paper on composing contracts).

My guess is that in the near term, especially where order management and execution are concerned, most of the actual PLT work won't be in the financial institutions but rather in outside software shops (see products like KDB, Aleri, Coral8, Apama, etc).

I served as an outside reviewer on some of the software systems in use at larger banks the last time the mortgage industry collapsed. It wouldn't be appropriate to name names, but the banks were large players in the mortgage-backed securities industry. I managed to get one of them out before that implosion. With apologies to Kalani, the root of the problem isn't the need to dump toxic mortgage debt. It's too late for that, and automating the dump would only serve to further depress prices in the mortgage market.

The root of the problem is to avoid acquiring toxic mortgages in the first place. This is something that software systems can definitely help with (and historically have helped with), but what happened in the current case is that the policies for risk-taking shifted in truly absurd ways as exuberance overcame sense across the entire financial industry. Software is predicated on models of value. What happened here is that the models reflected irrational exuberance. Nobody who stepped back and looked at most of the sub-prime mortgages being issues could say with a straight face that those mortgages were sustainable. Not even in the jumbo-prime market, which is now also imploding.

Software can help us be straight with ourselves, but unfortunately it can also help us lie to ourselves. That's part of what happened here. As long as the software said the numbers were acceptable, local brokers were willing to sell the mortgages, secondary markets were willing to buy and re-package them, and investors were willing to provide debt-backing for them. Once the computers said "yes", people quickly stopped thinking about or examining fundamentals.

The root of the problem is to avoid acquiring toxic mortgages in the first place. This is something that software systems can definitely help with (and historically have helped with), but what happened in the current case is that the policies for risk-taking shifted in truly absurd ways as exuberance overcame sense across the entire financial industry. Software is predicated on models of value. What happened here is that the models reflected irrational exuberance.

That's a very good point; you're right that's where the whole trouble began. In that respect, my impression is that the failure of the ratings agencies undermined the (otherwise decent) valuation that software performed. I'm not sure what software could have done to mitigate that, but I have not been working at that level.

People talk about "irrational exuberance" like it is some kind of natural phenomenon externality beyond all control and yet when I read about software engineers working at high levels for the financial firms I keep getting the impression that it was mostly them who created the irrational exhuberance by collectively inventing a suite of competing derivative instruments whose main reliable behavior was to inflate in price. Trade in those derivatives was irresistable since the ROIs were unprecedented but the main productive activity was running data centers that competed with one other to find the best way to inflate the prices of various securities. Nothing rises forever and so that inflationary bubble in the securities whose prices were variously created and controlled by "hot new trading software" had to crash, bringing us to where we are.

It was "us". We software types did it in classic FUBAR style of overpromising and underdelivering (while still mostly getting out of the mess with a tidy paycheck).

The evidence is strewn on the ground of the financial markets: lots of instruments that are incredibly complex to understand, not clearly well defined, very highly priced (before the burst) and widely spread --- "we" (engineers) didn't understand the implications of what we were asked to build, we built it anyway, and this is the result.

-t

The New York Times article: How Wall Street Lied to Its Computers (relayed to me via Risks) is interesting in this regard.

Or, in easy to digest cartoon form.

Since this has gotten buried under posts, I thought I'd mark it.

That was an oddball one too; the spam account was dormant for two weeks before they decided to use it. Usually spam accounts are active within a day or two of their creation, if not minutes or hours.

I'm sure that gets by a bunch of automated measures that scrutinize new account posting.

How about deleting this story? It's a spammer's favourite, and there's precious little interesting content here.

I'll close of new comments.

Wall Street has unofficially reached sub T-0, meaning that the time required to trade a stock is above the margin of safety for doing so. For PLT, this means the company selling the most scalable complex event processing software wins. I believe the leading vendor is currently Progress Software.

As an aside, before he died, I used to love reading The Gary Halbert Newsletter, written by one of the best newsletter advertisers (copywriters) to ever live. I recall a wonderful explanation he gave, in layman terms, of this situation, without even really realizing the extent to which banks were acting on his comments: More Stock Secrets. All of these "secret trading systems" to manipulate the market are now becoming mainstream, but the cost to operate them is outside the grasp of the average stockholder. What's more, because they are automated, the trading systems can handle way more details combined in more sophisticated ways than your individual daytrader.