[cffi-devel] offer of code contribution - with-array-as-foreign-pointer

[JTK]

> 
> You might be disappointed (or pleased, perhaps) to see that someone
> (Tamas K. Papp, in particular) has already done this:
> 
> http://tkpapp.blogspot.com/2007/12/announcement-ffa.html


I feel a bit chagrined that I didn't find  Papp's AAF before putting a lot of effort into 
my WAAF.

I've looked at the differences between FFA and my WAAF.    

From my admittedly quick reading of the docs, FFA passes Lisp arrays to
foreign land rather than copying for SBCL and maybe other Lisps.   This involves 
suspending garbage collection.    On the balance, I think  that in an era of multithreaded
64 bit Lisps and relatively abundant memory,  it is better not  to mess with Lisp's 
internals (like suspending GC), and it is safer to  allocate foreign memory and
copy.  eg, SBCL will pin arrays if you ask it, but I understand that this is done by 
globally suspending GC, which is not necessarily good when other threads are running.

In fact, I was forced to write this after a multithreaded SBCL application froze mysteriously
in foreign code with GC suspended.

FFA apparently uses its own knowledge about individual Lisp implementations to decide
whether to copy or pass lisp array addresses directly to foreign code.   WAAF, on
the other hand, knows only about CFFI, and infers all its knowledge from information exported 
by CFFI (like pointer sizes, if you want a pointer array).

Most of the effort in WAAF is in the set of fast copying function macrology that minimizes
the cost of copying to and from foreign memory.

At any rate, I think it would be very useful to have some form of this functionality
in CFFI, or in a visible user contrib section of it, because passing arrays to foreign 
code is a common but somewhat arduous and tricky task.


John











On Sep 11, 2010, at 2:45 PM, Mark Hoemmen wrote:

> On Sat, Sep 11, 2010 at 18:38, JTK <jetmonk at gmail.com> wrote:
>> I'm using cffi, and I need to perform the common task of passing Lisp arrays to foreign code.
>> 
>> This task comes up a lot in scientific code, matrix math, image processing, etc.
>> 
>> Normally, it involves a lot of boilerplate to allocate a pointer, copy data Lisp  to foreign
>> memory, copy the data back, and deallocate the memory.   Getting the copying to run
>> fast can be tricky.
>> 
>> Hence I wrote a package to automate it.
> 
> You might be disappointed (or pleased, perhaps) to see that someone
> (Tamas K. Papp, in particular) has already done this:
> 
> http://tkpapp.blogspot.com/2007/12/announcement-ffa.html
> 
> mfh