[cffi-devel] [PATCH] [RFC] Unloading libraries for delivery

[John Fremlin]

[Sorry; I accidentally sent the last email before finishing it (C-c C-c
instead of C-c C-e).]

The problem:

Using the excellent cffi-wrappers on platforms that are not Allegro does
not work well with delivery, because the path to the generated shared
object (presumably near the source code) is hardcoded into the image,
and the image will crash very early, before entering user code if this
shared object is not available.


The solution:

Close all shared libraries before delivery. After delivery reopen them,
searching *foreign-library-directories*.


Brief summary of the patch:

(1) The wrapper generation code is changed to use define-foreign-library
and not to explicitly wedge the path into the library name, but to put
the directory where it is generated into *foreign-library-directories*.

(2) cffi:close-all-foreign-libraries closes all open defined foreign
libraries. Call this before delivery.

(3) (cffi:do-reload-all-compile-time-open-libraries) is a macro that
will load all the required libraries again. Put this in your main
function.


What I want:

Feedback and comments; maybe this patch touches too much? Apparently
Stelian started doing something a little similar but half finished? If
it looks viable for the mainline, we can add test cases and
documentation.



Examples:

Here are a couple of example build procedures that add the application's
binary directory to *foreign-library-directories*, so that the .so files
can be put there. (Maybe it would be better to fully set
*foreign-library-directories*.)

I have tested it on AMD64 Linux Lispworks and SBCL.

For example, Lispworks:

(compile 
 (defun main ()
   (setf *debugger-hook* 'mtsnmp::mtsnmp-debugger-hook)
   (pushnew (make-pathname :name nil :type nil :version nil 
			   :defaults (truename (first sys:*line-arguments-list*)))
	    cffi:*foreign-library-directories* :test 'equalp)
   (handler-case
       (progn
	 (cffi:do-reload-all-compile-time-open-libraries)
	 (prog1 (apply 'my-application:main sys:*line-arguments-list*)
	   (finish-output)))
     (condition (c)
       (ignore-errors
	 (format *error-output* "Error: ~A~&" c)
	 (finish-output *error-output*))
       (lispworks:quit :status 1)))
   (lispworks:quit :status 0)))

(cffi:close-all-foreign-libraries)

(deliver 'main
 (multiple-value-bind 
       (second minute hour date month year day daylight-p zone)
     (get-decoded-time)
   (format nil "binaries/~4,'0D~2,'0D~2,'0D-~2,'0D~2,'0D-~A-~A-~A" 
	   year
	   month
	   date
	   hour
	   minute
	   (lisp-implementation-type) 
	   (lisp-implementation-version) (expt 2 (ceiling (log (log most-positive-fixnum 2) 2)))))
   1
   :multiprocessing t
   :keep-eval t
   :keep-conditions :all
   :keep-top-level nil
   :keep-debug-mode nil)




For example, SBCL

(in-package #:cl-user)

(defun main ()
  (unwind-protect
       (handler-case
	   (progn
	     (pushnew (make-pathname :name nil :type nil :version nil 
				     :defaults (truename (first *posix-argv*)))
		      cffi:*foreign-library-directories* :test 'equalp)
	     (cffi:do-reload-all-compile-time-open-libraries)
	     (apply 'my-application:main *posix-argv*)
	     (sb-ext:quit :unix-status 0))
	 (condition (c)
	   (ignore-errors
	     (format *error-output* "Error: ~A~&" c))
	   (sb-ext:quit :unix-status 1)))
    (sb-ext:quit :unix-status 100)))

(defun make-image ()
  (sb-ext:disable-debugger)

  (cffi:close-all-foreign-libraries)
  (sb-ext:save-lisp-and-die 
   (multiple-value-bind 
	 (second minute hour date month year)
       (get-decoded-time)
     (declare (ignore second))
     (format nil "binaries/~4,'0D~2,'0D~2,'0D-~2,'0D~2,'0D-~A-~A-~A" 
	     year
	     month
	     date
	     hour
	     minute
	     (lisp-implementation-type) 
	     (lisp-implementation-version) (expt 2 (ceiling (log (log most-positive-fixnum 2) 2)))))
   :executable t
   :toplevel 'main
   :purify t))

(make-image)


-------------- next part --------------
A non-text attachment was scrubbed...
Name: cffi-wrapper-safe-delivery.patch
Type: text/x-diff
Size: 10036 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/cffi-devel/attachments/20090717/f2bdfe76/attachment.patch>