Lambda the Ultimate

They have a lot of links, including using speech for programming.
(look for Emacs)

I have the same problem, RSI shutdown my left wrist for six months, I had to learn the right-hand-only dvorak keyboard layout. Since I've gotten back to two-hand typing I've been thinking about a good design for a usable spoken programming language. My first thought was to encode lambda calculus in a spoken form, that's an easy design, just write code to recognize three sounds.

But that's not really general purpose, so I wonder whether a better design is to find a nice round binary number of sounds that I can easily recognize with simple code and use that for the spoken 'charset'.

Derek Elkins suggested a concatenative language for spoken programming, I think that's a great use for a language like Joy.

Does anyone here have further ideas or can point out possible pitfalls for such a language?

-- Shae Erisson - www.ScannedInAvian.com

Hi!

Sorry that i am not sure how to format this better:

http://harmonia.cs.berkeley.edu/harmonia/research/slssd/index.html

Summary

Software development environments have not changed very much in the past thirty years. While developers discuss software artifacts with one another in terms of high-level conceptual notions, their environments force them to use low-level text editors and program representations designed for compiler input. This shift in level is error-prone and inefficient; in addition, the environments create frustrating barriers for the growing numbers of software developers that suffer from repetitive strain injuries and other related disabilities that make typing difficult or impossible. Our research helps to lower those barriers by enabling developers to work at a more conceptual level and by reducing their dependence on typing and text.

The specific technical issues to be addressed in this research are driven by two approaches: multi-modal (notably speech) interaction, and semantic and structural search, navigation, and transformation. The technology we are creating is not limited to programming languages; it extends to other specification, design, and command languages that are used by developers and that can be formally defined. Our research will be embedded in the Harmonia framework, also being developed at UC Berkeley. The first prototype language for which the linguistically-based methods will be created is Java.

Our research will create the first version of a form of Java that is more naturally verbalized by human developers than the standard Java language. Methods will be created to translate this form to the same annotated abstract syntax representation used by conventional text-based tools. The major technical challenge is to resolve the ambiguities that the new form allows. That ambiguity resolution requires new algorithms for interacting lexical, syntactic, semantic, and program-specific analysis. New methods of accommodating lexical, syntactic, and semantic errors and inconsistencies will be created, in order to sustain language-based services when the artifacts are incomplete and incorrectly formed.

Interesting idea. I'm assuming that you are using a language dependant engine like Speechworks/Scansofts Naturally Speaking or IBM Via Voice, and not a speaker independent tech like MS SALT and underpinnings...

If you do the speaker independent then the best advice I can give you is to really think through the spoken grammar and test it and test it and test it and test it. For some ideas - check out how Australians bet on horses.

If you do the speaker dependent thing - I assume you'll be spending most of your time on error correcting, or you'll give up and just let Naturally Speaking/Via Voice be the generic interface.

Do remember speech is probabilistic. There's a 5 to 10% chance that any utterance will be wrong... so editing becomes a real issue...and you won't be able to bring error rate down to less than 2%, which means you have to do some serious testing.

Stay away from any speech words that get easily confused - especially words that start with s, or z (you'll see why if you look at the energy of s on a meter). And finally, KEEP YOUR COMMANDS TWO SYLLABLES, not one - one syllable words reco horribly.

Have I made my point about testing? Start testing now, and test often - you may find out what you're worried about is the wrong thing.

Good luck! Post back if you have success!

Ashok Khosla, CTO & CoFounder, TuVox Inc.

I think that speech recognition---although a tough and interesting problem, and cool in SciFi---is the wrong way to go. Instead, we should make it `easier' to edit programs. One way of thinking about this is that we should try to minimise the bandwidth required to program a computer.

For example, if Stephen Hawking had to type each letter in each word he wanted to use, he would have real problems communicating. Instead, he uses a text-to-speech synthesis program that has an optimised interface that requires few user inputs. The result is that he can communicate almost in real-time.

Of course, I am not advocating the replacement of the keyboard by some sort of weird device. Instead, I think we should make our interactions with our programming environments more efficient. If we can make sophisticated changes to programs quickly and easily, we should be able to be more productive and make fewer errors. E.g. I tend to slightly lose track of what I am doing when compiling code in a language such as Java, because I have to wait for the compile to finish and my mind tends to wander. I think a similar argument could be made when many keys need to be pressed to make a significant alteration. We should also be able to accomodate those who suffer from RSI or are otherwise physically less able, because less bandwidth is required from the programmer. This seems to be what IDEs should have been. Instead, they tend to be simply a combination of an editor, makefile generator, debugger and perhaps a WYSIWYG user interface editor.

I remember seeing a demo on the Microsoft Research website of a system where, if I remember correctly, the program's parse tree (or similar) and metadata (e.g. comments) we stored in a database. Users could customise their view onto the program (think CSS), could make quite sweeping changes to the code and they could even automatically translate from one programming language to another (though this was obviously quite limited). The system was similar to that proposed by McConnell in Code Complete.

If anyone has a link to this, could they post it, as I spent about 10 minutes looking, but I couldn't find it.

Standard Forth deliberately specified largely unambiguous american-english-like pronounceable names for all the Forth words that weren't obviously english words anyway (e.g. : is defined to be spoken as "colon" when used as a forth word, ! is defined to be spoken as "store" when used as a forth word) - see the bold entries in the glossary of the (usual-ANSI-workaround-last-draft) spec.

Unfortunately, the standardised spoken forms are a mix of descriptions of the characters and the functions they perform - as my chosen examples illustrate - :'s function is actually "define a word", whereas ! is spoken as the function it performs.

Nonetheelss, as forth also has pretty unambiguous and straightforward syntax, it thus would probably be relatively easy to make a spoken forth interpreter (you'd just have to wrap forth entries in an attention word and a terminator word - e.g "Computer! One One Plus dot ok" to which the computer could say "two ok") that maps to the existing standard language - though you'd end up shouting "colon!" at your computer an awful lot though if you used the current standard spoken forms, which might be embarrassing I guess.

Of course, forth is not a high-visibility language these days, but it's still potentially useful in some embedded systems - it's amazing how much you can do with so little compared to the bloat of a conventional OS.

Er (editing comment) - This obviously fits in with Derek Elkin's stuff about Joy too,seeing as forth is an ancestral mostly-concatenative language.