[cffi-devel] mid-level implementation-specific patches (was: uffi and CUSTOM:* FOREIGN-ENCODING* from clisp)

[James Bielman]

On Fri, 2005-12-09 at 19:22 +0100, Hoehle, Joerg-Cyril wrote:

> Pascal Bourguignon's proposed patch to src/strings.lisp adds
> #+clisp, #-clisp to the source. (cf. mailing archives from
> Mid-November).
> 
> I noticed the wide lack of such #+implementation-specific code in the
> CFFI sources. I like it, but it's limited. I would like to ask what
> are your thoughts on integration of such code?
> 
> I.e., there are a few places (e.g. in strings.lisp) where the CLISP
> version could clearly benefit from specific code. One of the typical
> improvements is to get rid of (loop ... do mem-ref #) to copy array
> contents elementwise and have the FFI built-ins handle them (using
> native loops). Remember, LOOP compiles to byte code, not tight
> memcpy()-like code that CMUCL can achieve.
> BTW, what happened to the proposed block memory interface?
> 
> The general idea is that while the low-level API is fine, as it is
> currently split across each implementation in cffi-xyz.lisp, there's
> also a need for high-level implementation-specific optimizations.
> Remember, as Peter Norvig said, a Lisp compiler is not going to help
> optimizing a Prolog compiler implemented in Lisp. Prolog needs its own
> optimizations. Same for FFI code.

I think the approach I would like to take to handle this situation is to
add a function to the backend for converting foreign strings to/from
Lisp that can be aware of encoding, as well as implement the block
memory interface.

Another idea I just had---if we run into more cases where optimizing
high-level functions for CLISP is a big performance win, but it doesn't
make sense to add new backend functions, how about adding
"cffi/src/optimizers-clisp.lisp" (and associated files for other
implementations) that use compiler macros to perform the optimization?  

I know these aren't guaranteed to be expanded but so far in my CFFI
experience they seem to be expanded when I want them.  I really would
like to avoid reader conditionals in the portable source as much as
possible.

My time for CFFI hacking is limited these days (alas) but I will try to
take a look at implementing this soon.

James