[cells-gtk-devel] Re: Multithreading patch

Tue Dec 11 11:52:55 UTC 2007

On Mon
10 Dec 2007 14:45:37 +0100, Peter Denno <peter.denno at nist.gov> wrote:

> On Sunday 09 December 2007 09:58, Peter Hildebrandt wrote:
>> Here it is, finally.  Multithreading for cells-gtk on sbcl/linux.
>
> Hi Peter,
>
> I'll try this out and get back to you, hopefully today, while your
> mind is still in this stuff.

I think I finally got multiple calls to (init-gtk) straight  (so far  
multiple initializations have been an issue, since cells-gtk-init destroys  
state vital to open application windows).

Here's a patch for gtk-app.lisp that does three things

- handle mutiple calls to init-gtk gracefully [1]
- add a key parameter :close-all-windows to init-gtk (which closes all  
open windows cleanly by signalling gtk-main-quit [2])
- move the bookkeeping section beneath around to remove the warning about  
*system* being undefined

The patch is incremental to the previous one I mailed out.

Peter

------
[1] The solution was to separate loading the gtk librbaries from resetting  
cells-gtk, along these lines:

- load libs if necessary
- use lib func to check whether threading is initialized
   - if not, init threading, cells, gtk

[2] Yep, this works now.  The trick is, you can use (gtk-main-quit) to  
exit a gtk-main loop that is *running* in a different thread, and this  
will cleanly close all open windows.  However, it seems you cannot use it  
to cleanup a *stale* instance that was left due to a signal/condition

>
>> I built the patch against the current cvs (which is different from
>> the tar.gz linked on the website, which is different from the
>> asdf-install version).  Applications will work just as they used to
>> (a littler better, hopefully, see below).
>>
>> To run an application in the background, just use the new
>>
>> (start-win 'app-class initargs)
>>
>> which starts your application window in the background and returns
>> right to the repl.  The return value is the freshly created
>> instance of app-class.  That's it.
>>
>> To reset cells-gtk, there is the new (init-gtk), which closes all
>> open windows before calling cells-gtk-init, and also takes care of
>> threading.
>>
>> A couple details:
>>
>> - To simulate the idea of a "main window" (thanks Kenny for
>> pointing out that this is not something "natural") gtk-app now has
>> the slot terminate-on-exit -- if this is set to true, all open
>> windows will be closed when this is closed
>>
>> - Calls from the repl can be wrapped in (with-gdk-threads ...), in
>> theory this is the right thing to do.  In practice I have not seen
>> anything bad happening, when I didn't.
>>
>> - You can start the gtk thread manually by calling (start-gtk-main)
>>
>> - The gtk thread can be terminated by calling (stop-gtk-main).  In
>> order to use gtk again, you will have to restart lisp.
>>
>> Here's a silly example of interactive UI development:
>>
>> ;; make a package
>> (defpackage :cgtk-user (:use :cl :cgtk :cells))
>> (in-package :cgtk-user)
>>
>> ;; start a blank app
>> (start-win 'gtk-app :title (c-in "My App") :kids (c-in nil)) ; it
>> appears
>>
>> ;; bind it to a variable
>> (defparameter *app* *)
>>
>> ;; change its title
>> (setf (title *app*) "Application") ; the title changes
>>
>> ;; create a vbox to take the our widgets
>> (push (mk-vbox :kids (c-in nil)) (kids *app*))
>> (defparameter *vbox* (first *))
>>
>> ;; make a button
>> (push (mk-button :label "Click me" :on-clicked (callback (w e d)
>> (print 'you-clicked-me))) (kids *vbox*)) ; it shows up
>>
>> ;; make a counter
>> (push (mk-button :label (c-in "Me too") :on-clicked (let ((count
>> 0)) (callback (w e d) (print (incf count))))) (kids *vbox*))
>>
>> ;; relabel that button
>>
>> (setf (label (first *)) "Counter")
>>
>> ;; kill your window
>> (not-to-be *app*)
>>
>>
>> Some technical details:
>>
>> - The way gtk-app used to handle restarts resulted in effect in a
>> mutual recursion:  Closing a window resulted in signaling quit,
>> which led to transfer of control out of gtk-main into lisp, which
>> then exited, leaving the gtk-main task waiting for a return (that
>> would never come).  Each time you ran an application, that stack
>> would build up.
>>
>> The previous way to unwind that stack upon exit, like
>>
>>    (loop for i below (gtk-main-level)  do (gtk-main-quit))  ;; I
>> had this one somehow
>>
>> or
>>
>>    (loop while (> (gtk-main-level) 0) do (gtk-main-quit))   ;; from
>> cvs, results for me in endless loop, since gtk-main-level won't
>> reduce for me by calling gtk-main-quit like this
>>
>> does not do the trick, since gtk-main-quit only workd as expected
>> when called from within a gtk-main initiated callback, not after
>> control has been violently transfered out of gtk-main by a signal.
>> In effect, I found there seems to be no way to resume an open
>> instance of gtk-main once a lisp callback has used error or signal
>> to uncleanly jump out of it.  I believe that this could be the
>> reason why the termination code in gtk-app.lisp included several
>> fixes for different lisps.
>>
>> I rewrote gtk-app completely so that now in effect one instance of
>> gtk-main runs continously.  When an application with
>> :terminate-on-close is closed, gtk-main-quit is signalled through
>> gtk, gtk-main thus cleanly left, through gtk-quit callbacks open
>> windows are closed, and gtk-main is called again.  This way no call
>> stack piles up.  In the normal case there should be no callstack
>> building up this way.  You can check with (gtk-main-level).
>>
>> Only errors in callbacks still lead to recursive re-entries.
>> Therefore I left the current termination code in place, though my
>> changes should make this the exception rather than the normal case.
>>
>> If someone figures out how to reenter gtk-main after a callback
>> signaled an error, I'd be very excited to learn.
>>
>> - I added an additional error handler that catches lisp errors.
>> Thus, if a callback raises a lisp error, this is caught within the
>> thread running gtk-main and a message box is displayed.  The user
>> has the option to resignal the error (results in invoking the
>> debugger) or to "recklessly continue" (which works fine as long as
>> the callback has not caused inconsistent state).  This is helpful
>> if (like me) you deal with user input and you prototype does not
>> handle typos well ;)
>>
>> - cells-gtk has a major leak due to the nature it synchronized clos
>> with gtk objects:  Upon creation, each clos object in cells-gtk is
>> registered in *gtk-objects* along with the pointer to its gtk
>> equivalent.  However, this entry is not removed when the
>> gtk-instance is freed.  When subsequently new objects are created,
>> gtk may well reuse this space, causing cells-gtk to signal an error
>> like "new object has pointer x, but this is already known as
>> such-and-such".  Curiously, it is usually only *gtk-objects* which
>> holds a reference to an obsolete object.  For instance, running
>> test-gtk allocates some 330 objects in *gtk-objects*, about 100 of
>> which are not freed, when the app is closed.
>>
>> The right way to solve this would be to hook into the
>> on-delete-event signal of the widget object -- unfortunately it
>> seems this is not called every time gtk deletes a widget.  I
>> introduced some additional cleanup code in cells-gtk, thus covering
>> about 70% of leftover widgets.
>>
>> For the rest, I believe the right thing is to trust gtk when in
>> doubt -- thus I modified the gtk-objects registry to forget about
>> the old object if gtk has given the pointer to a new one. (It used
>> to throw an error and break gtk. I had it signal a warning in the
>> beginning, but I came to believe that this cannot be the right way
>> to deal with it; it would entail writing C-style memory management,
>> carefully keeping track of objects and manually freeing them.
>> Ewww.)
>>
>> This hasn't been a problem so far since it seems most cells-gtk
>> apps just call cells-gtk-init before launching their app, thus
>> wiping *gtk-objects* (and along with that all gtk state).  Not
>> exactly dynamic ;).
>>
>> For an example where the problem arises, consider the following
>> slot def:
>>
>> (caption-widgets :initform (c? (loop for caption in captions
>> collecting (mk-label :text caption))))
>>
>> Once you got to somehow free all the old labels everytime you
>> create new ones, things get -- ehh -- ugly.
>>
>> It looks like what we see here is the due penalty for using a
>> C-library for doing our window management.  Maybe one day CLIM will
>> be fast and sleek.
>>
>> - For every new window, upon creation a gtk-quit callback is
>> registered, closing that window once gtk-main-quit is called from
>> within a callback. You force this to happen by setting
>> terminate-on-close to t in a gtk-app window.
>>
>> - I moved some initialization from start-app to an own function,
>> init-gtk.
>>
>> - To prevent double initialization of glib's threading faacility
>> via g-thread-init, there is a global variable in gtk-app.lisp.
>> This works fine, until you recompile this file.  So I figured why
>> keep track of state when glib can do this for us?  It turns out
>> there is a macro in glib for this purpose.  Since it's a macro (not
>> a function), I added a respective function to gtk-adds.c.  A
>> dependency on glib-2.0 is introduced, since threading is a glib
>> feature, not gtk (this should be uncritical -- linux systems
>> without glib are -- errr -- rare).  I added the corresponding stuff
>> to Makefile.  Thus, when you compile cells-gtk with #+libcellsgtk,
>> it will use this function.  Otherwise there is a flag variable in a
>> closure around (init-gtk).
>>
>> - The threading code is currently SBCL specific.  However, I
>> singled out the needed primitives in four macro definitions,
>> therefore it should be easy to add the respective calls for other
>> implementations:
>>
>>
>> (defmacro thread-current ()
>>    #+sbcl `(identity sb-thread:*current-thread*)
>>    )
>>
>> (defmacro thread-make (thread-fn name)
>>    (declare (ignorable name))
>>    #+sbcl `(sb-thread:make-thread ,thread-fn :name ,name)
>>    )
>>
>> (defmacro thread-kill (thread)
>>    #+sbcl `(sb-thread:terminate-thread ,thread)
>>    )
>>
>> (defmacro thread-yield ()
>>    #+sbcl `(sb-thread:thread-yield)
>>    )
>>
>> - My changes will compile and run just fine without sbcl.
>> Everything should work normally, you just won't be able to use
>> (start-win).
>>
>> That's all I can think of right now.  Let me if clarifications are
>> needed.
>>
>> Regards,
>> Peter
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: cells-gtk-threading-2.patch
Type: application/octet-stream
Size: 2536 bytes
Desc: not available
URL: <https://mailman.common-lisp.net/pipermail/cells-gtk-devel/attachments/20071211/4ab1ddf4/attachment.obj>