Home » forums » LtU Forum

Trying to get embedded python co-routines to work

Sorry if this has been talked about before, but I can't seem to find any resources on this site or with Google anywhere...

I'm trying to get embedded python co-routines to work, and thinking about what the best strategy would be, so I figured I'd ask if anyone had dealt with this before.

Basically what I'm looking to accomplish is a bunch of separate interpreter states (or threads I guess, if I can get that to work elegantly) that I schedule myself (i.e. when a certain process gets a tick, I tell the python thread to do a tick of its execution), with many sequential blocking functions for each process. So basically I want to have concurrently:

Process A trying to do actions P, Q, R
Process B trying to do actions X, Y, Z

where I schedule when each one gets a tick, and the responsibility for "am I done" checking is inside P, Q, etc, not in A or B

This would be a C application, with A and B being Python scripts and P, Q, etc being Python extensions written in C, possibly with Python wrappers...

Some ways I'm considering doing it:
- building off generators (no support of going to/from C or yielding the whole stack that I can find)
- throwing exceptions to escape to the top level and hacking continuations in
- using the debugging trace functionality accessing some shared data to allow me to step until I hit a blocking instruction (still requires nasty frame saving/restoring)
- somehow scheduling the python threads by myself (it seems like this should be trivial, but somehow I can't seem to find any interface for doing this... still searching through docs, I'm assuming it should be doable somehow...)

Does anyone have any experience with this or anything similar? I hope my descriptions are clear enough...

Thanks in advance,

Raoul

By raoulvp at 2006-06-02 16:43 | LtU Forum | previous forum topic | next forum topic | other blogs | 5410 reads

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Greenlets

I think Armin Rigo's greenlets do what you want. They should even work when you have C call frames from extention modules mixed into the call stack since greenlets work by juggling around stack segments.

By Per Vognsen at Fri, 2006-06-02 17:44 | login or register to post comments

Stackless

I think Christian Tismer initially wrote greenlets, not Armin, although I could be wrong.

Since it sounds like you're writing the whole application stackless' channels might work better. I think greenlets primary advantage is that they run in an unmodified version of python.

Check out the stackless stuff at:

http://www.stackless.org

In particular, check out the EVE Online presentation as that's the best proof-of-concept of real world scalibility of stackless. Unfortunately, there isn't a recording of the audio from the talk at PyCon.

Also, there isn't the best documenation for the current version of stackless (all of the tutorials are for a significantly different older version) but there are really only 3 or 4 key components you need to deal with.

By grant at Fri, 2006-06-02 18:05 | login or register to post comments

Try Lua

Seriously - try Lua instead of Python. It has excellent support for co-routines and its efficiency and small size make it a perfect match for embedded computing.
Section 2.11 of the Lua Reference Manual covers co-routines in more detail.

By ntupel at Sat, 2006-06-03 23:35 | login or register to post comments