[Ecls-list] libevent

[Brian Spilsbury]

On 5/26/06, Juan Jose Garcia Ripoll <lisp at arrakis.es> wrote:
> On Fri, 2006-05-26 at 17:11 +0900, Brian Spilsbury wrote:
> > What do people think about integrating libevent (conditionally) into
> > the ecl core?
>
> I have no problem with it. The source code of libevent should however be
> distributed separately.

Personally I'd rather bundle them for convenience, and being sure that
it should actually compile, but it isn't a significant issue.

> I have been reading the manpage of libevent and it seems to abstract
> over different mechanisms (select, poll, kqueue) provided by the
> operating systems to check existing input/output on a file descriptor.
> Besides, it includes events which are timers.
>
> What is not clear to me is what this library expects from the callback.
> Does it have to perform some amount of non-blocking read/write? Can
> these operations be interrupted by libevent?

libevent is just doing scheduling, so it tells you when an i/o
operation is possible.

What you do as a consequence is your problem. :)
(it might mean that a non-blocking connect has completed, that there
is a socket waiting to be accepted, etc, etc).

Generally you'd write all of your i/o operations to be non-blocking
and when they would have otherwise blocked you ask the scheduler to
tell you when it's worthwhile to try again.

> > Another reason is that I'd like to get coroutine support happening
> > sometime, and then have stream io integrated with scheduling, in which
> > case all blocking operations would need to be scheduler-aware.
>
> Why the interest in coroutines? Are native threads not a good option? If
> so, then you might consider revisiting the old thread code in ECL. It
> was user-based threads using alarm() IIRC.

I need to be able to handle at least 5,000 tasks concurrently. Ideally
it should scale up to 100,000 without problems.

> An issue of coroutines vs. native threads is that one has to check for
> the interruptibility of different functions. That is not so much a
> problem with native threads because they are rarely interrupted (except
> for some unix signals, but they typically denote something bad has
> happened).

It's not a problem with coroutines, since you generally explicitly yield.

For i/o, an operation which would have blocked ends up setting up the
scheduler to yield to it later, and then yielding to the next
coroutine which can now do i/o.

Other common patterns are producer coroutines which fill queues and
then yield to consumers, which then yield back to the producer, etc.

Regards,
Brian.