[Ecls-list] Latest changes ... on a branch

[Juan Jose Garcia-Ripoll]

Hi,

I have made progress along the new_signal GIT branch. The summary of
the situation seems to be as follows.

- Do not do _anything_ inside C signal handlers when they break into
functions you do not know about. Signals are deactivated whenever a
moderately complicated C function is called. By moderately complicated
we mean all but those without side effects on its input or which seem
more or less reentrant (i.e. strlen is safe, fread probably not)

- When signals are restored again, lisp can then process them. This is
performed via a clever hack based on mprotect'ing the lisp environment
so that when we write cl_env->disable_interrupts = 0, we enter the
signal handler. Seems harmless and is cheaper than keeping a machine
register for this task.

- We had no answer from the Boehm-Weiser folks, but it seems that the
garbage collector is not signal-reentrant. This is BAD, because it
forces us to deactivate signals and activate them again for every
allocation. AFAI can tell, the overall effect of interrupt control
seems to be about a 10% slowdown on allocation intensive code.

- I am auditing the whole code for making it signal reentrant. This
seems hard, as it forces us to check that whatever we do, the lisp
environment is kept on a sane state, and it affects most the control
of various lisp stacks.

- The lisp environment is now dynamically allocated. This is largely
due to the need of having a page that we can mprotect, but it is also
nice because we can make the threaded and non-threaded versions of ECL
use the same code and have similar performance.

- Since the lisp environment is now always a pointer, I am unifying
the code so that many functions take an explicit pointer instead of
querying where the lisp environment is. This leads to speed
improvements in the threaded code and can probably help the compiler
to optimize some code. I will hopefully post some guidelines in the
manual, in the chapter about embedding.

- I have reenabled the flag --with-__threads Please report on which
platforms this works and seems to give you a reasonable speedup
without enlarging the code too much.

- I am also using this chance to fix the names of many functions,
following the convention that ecl_ means C function, cl_ means
compiled Lisp function from CL package, si_ are Lisp compiled
functions not intended for the public, and ext_ is the prefix for Lisp
extensions. Hopefully these changes will also be documented.

- The other moral of this story is that stack overflows are even more
evil than signals. When a C function from the library is interrupted
by a signal, we can defer it and continue execution. Stack overflows
are not like that. There may be tricks such as mprotect'ing a page at
the end of the stack and then redelivering the signal so that the lisp
code can act, but this needs further research. In the mean time I have
instructed the compiler to insert an explicit stack check for every
lisp function with OPTIMIZE SAFETY >= 2.

- Finally there is a pending chapter that is probably going to be
written, and that is C streams. The first part of it is related to the
support of input formats with byte sizes that are not multiples of 8
bits. This is going to disappear because the code is not thread safe.
But the most important change is related to dropping the use of FILE
and C streams and moving on to POSIX file handlers. There was a clever
guy reading the mailing list who suggested doing this long ago, and to
that he wanted to add support for asynchronous operations and Unicode.
I guess he must have something better to do now, but I will try to
incorporate some ideas and do something he did not like: define
buffered streams as an abstraction that encapsulates on top of the
POSIX streams and provides support for UTF-8.

In the mean time I would like to ask you to try this branch and report
any problems you experience with it. It is by no means stable but I
need the feedback somehow.

Juanjo

-- 
Instituto de Física Fundamental, CSIC
c/ Serrano, 113b, Madrid 28009 (Spain)
http://juanjose.garciaripoll.googlepages.com