[cffi-devel] Re: CFFI defcallback closures?

[Hoehle, Joerg-Cyril]

Nikodemus,

Nikodemus Siivola wrote:
>> Unfortunately, last time I checked, not all Lisps support non
>> top-level DEFCALLBACKs. See:
><http://common-lisp.net/project/cffi/manual/html_node/defcallback.html>
>...and in even those that do (like SBCL), you should be aware that
>whereas a call like
>  (some-foreign-function callback) 
>is likely to be decently efficient, each call like
>  (let ((x 0)) (some-foreign-function (alien-lambda () (frob x))))
>is likely to be a _lot_ more expensive, as each call needs to generate
>a new callback.

I'm curious to know how precisely SBCL handles the details.

As far as CLISP is concerned, the second example works (if converted to CLISP FFI syntax), but will leak malloc()'ed memory if it's never deallocated.

That is, if your application does not provide for a means to receive the callback address from the alien world, so that the Lisp side can free it, then your application leaks memory.

An anonymous callback perfectly fits this bad scheme: a priori, you don't want to care about the callback address...
When is it released/destroyed?


Some Lisps have a concept where they track all Lisp-caused malloc()'s and automatically free them when the stack gets unwound, e.g. WITH-DYNAMIC-MALLOCS.  I suppose Lispworks and CormanLisp provide this.  CLISP does not, and such a concept would directly violate the (:allocation :malloc) of the CLISP FFI, which supports scenarios where the other side frees memory.  This is incompatible with WITH-DYNAMIC-MALLOCS, a duplicate free() is likely to cause a core dump.

Similarly, an automatic (EXT:FINALIZE (alien-lambda) #'FOREIGN-FREE) is likely to crash as you have no means to tell Lisp how long that callback must stay alive.  Does it have have stack semantics or is it killed at a later time?

This tells us how a lambda callback ought to be designed:
 + can create one,
 + be able to keep its handle on the Lisp side,
 + to be able to destroy it
   - via garbage collection
   - and/or via stack semantics
   - from the C side(?)
CLISP's FFI does not directly fir the bill.  You need a little wrapper code.
I haven't looked at how CFFI fits these requirements.


Second, you say "expensive".  What do you mean?
o The allocation of memory for the callback structure?
o The need to flush some instruction/data cache for stack-allocated trampolines?
o ...?

Regards,
	Jörg Höhle
PS, how's your business going?