From attila.lendvai at gmail.com  Mon Nov  5 21:11:08 2007
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Mon, 5 Nov 2007 22:11:08 +0100
Subject: [closer-devel] *root-context* and threads
Message-ID: <b54b34af0711051311k2462c436h28788fdd03f2a47a@mail.gmail.com>

hi!

we are having issues with our contextl-heavy code on sbcl with high
load. i suspect that the problem happens due to currupting sbcl's
internal state due to some missing locks in sbcl, but while i was
reviewing the code i had a question: seems like *active-context* is
holding some caches, but i can't see where *root-context* is protected
from concurrent access.

maybe the code should eq-check for *root-context* at some point and
enforce a separate *active-context* on first use? this will of curse
lower the cache efficiency, so maybe a cross-platform threading lib
like bordeaux-threads could be used to protect the global cache or
somehow ensure that *root-context* is rebound and copied in all
threads (i think this needs some platform specific code).

is this analysis correct at all?

-- 
 attila

fyi, the sbcl backtrace:

bad thing to be a type specifier: #<CONTEXTL:STANDARD-LAYER-CLASS NIL
{100CE950C1}>
*** Backtrace is:
000: ((LAMBDA (ERROR)) #<SIMPLE-ERROR {100A281D11}>)
001: ((LAMBDA (ERROR)) #<SIMPLE-ERROR {100A281D11}>)
002: ((LAMBDA (ERROR)) #<SIMPLE-ERROR {100A281D11}>)
003: (SIGNAL #<SIMPLE-ERROR {100A281D11}>)
004: (ERROR \"bad thing to be a type specifier: ~S\")
005: (SB-KERNEL:VALUES-SPECIFIER-TYPE #<CONTEXTL:STANDARD-LAYER-CLASS
NIL {100CE950C1}>)
006: (SB-KERNEL:SPECIFIER-TYPE #<CONTEXTL:STANDARD-LAYER-CLASS NIL
{100CE950C1}>)
007: (SUBTYPEP #<CONTEXTL:STANDARD-LAYER-CLASS NIL {100CE950C1}>
#<CONTEXTL:STANDARD-LAYER-CLASS EBR42-LAYER {1001760DB1}> #<unused
argument>)
008: ((SB-PCL::FAST-METHOD
LAYERED-FUNCTION-DEFINERS::|CONTEXTL::ADJOIN-LAYER-USING-CLASS| (T
CONTEXTL:STANDARD-LAYER-CLASS T)) #<error printing object>)
009: (CONTEXTL:ADJOIN-LAYER #<unavailable argument> #<unavailable argument>)
010: ((LAMBDA ()))


From pc at p-cos.net  Tue Nov  6 07:52:42 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Tue, 6 Nov 2007 08:52:42 +0100
Subject: [closer-devel] *root-context* and threads
In-Reply-To: <b54b34af0711051311k2462c436h28788fdd03f2a47a@mail.gmail.com>
References: <b54b34af0711051311k2462c436h28788fdd03f2a47a@mail.gmail.com>
Message-ID: <44911384-DA5A-4F34-9B01-AA0F8C24CC8F@p-cos.net>

Hi,

On 5 Nov 2007, at 22:11, Attila Lendvai wrote:

> hi!
>
> we are having issues with our contextl-heavy code on sbcl with high
> load. i suspect that the problem happens due to currupting sbcl's
> internal state due to some missing locks in sbcl, but while i was
> reviewing the code i had a question: seems like *active-context* is
> holding some caches, but i can't see where *root-context* is protected
> from concurrent access.

Indeed, there is no protection against concurrent access.

> maybe the code should eq-check for *root-context* at some point and
> enforce a separate *active-context* on first use? this will of curse
> lower the cache efficiency, so maybe a cross-platform threading lib
> like bordeaux-threads could be used to protect the global cache or
> somehow ensure that *root-context* is rebound and copied in all
> threads (i think this needs some platform specific code).
>
> is this analysis correct at all?

I don't think it's sufficient. Two different threads may see the same  
*active-context* at the same time, even if it's not the *root- 
context*, so accesses to the caches should probably be synchronized  
all the time. (Although it would probably be acceptable if caches are  
not updated every now and then.)

Currently, the ContextL caches are implemented using property lists.  
Does anyone know of a straightforward way to make property lists (or  
assocation lists or hashtables) threadsafe in an implementation- 
independent way, possibly using some kind of compatibility layer?


Pascal

> -- 
> attila
>
> fyi, the sbcl backtrace:
>
> bad thing to be a type specifier: #<CONTEXTL:STANDARD-LAYER-CLASS NIL
> {100CE950C1}>
> *** Backtrace is:
> 000: ((LAMBDA (ERROR)) #<SIMPLE-ERROR {100A281D11}>)
> 001: ((LAMBDA (ERROR)) #<SIMPLE-ERROR {100A281D11}>)
> 002: ((LAMBDA (ERROR)) #<SIMPLE-ERROR {100A281D11}>)
> 003: (SIGNAL #<SIMPLE-ERROR {100A281D11}>)
> 004: (ERROR \"bad thing to be a type specifier: ~S\")
> 005: (SB-KERNEL:VALUES-SPECIFIER-TYPE #<CONTEXTL:STANDARD-LAYER-CLASS
> NIL {100CE950C1}>)
> 006: (SB-KERNEL:SPECIFIER-TYPE #<CONTEXTL:STANDARD-LAYER-CLASS NIL
> {100CE950C1}>)
> 007: (SUBTYPEP #<CONTEXTL:STANDARD-LAYER-CLASS NIL {100CE950C1}>
> #<CONTEXTL:STANDARD-LAYER-CLASS EBR42-LAYER {1001760DB1}> #<unused
> argument>)
> 008: ((SB-PCL::FAST-METHOD
> LAYERED-FUNCTION-DEFINERS::|CONTEXTL::ADJOIN-LAYER-USING-CLASS| (T
> CONTEXTL:STANDARD-LAYER-CLASS T)) #<error printing object>)
> 009: (CONTEXTL:ADJOIN-LAYER #<unavailable argument> #<unavailable  
> argument>)
> 010: ((LAMBDA ()))
> _______________________________________________
> closer-devel mailing list
> closer-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/closer-devel

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From pc at p-cos.net  Tue Nov  6 10:28:53 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Tue, 6 Nov 2007 11:28:53 +0100
Subject: [closer-devel] *root-context* and threads
In-Reply-To: <b54b34af0711051311k2462c436h28788fdd03f2a47a@mail.gmail.com>
References: <b54b34af0711051311k2462c436h28788fdd03f2a47a@mail.gmail.com>
Message-ID: <AC9983CF-DB3A-42F0-B3FA-D582E6716AD5@p-cos.net>

Hi,

OK, I have looked into this - I will be using portable-threads from  
the GBBOpen project to make the code thread-safe. (Portable-threads  
gives me a better way than bordeaux-threads to write code that works  
both on multi-threaded and non-multi-threaded CL implementations.)

I hope I will have something by this evening.


Best,
Pascal

On 5 Nov 2007, at 22:11, Attila Lendvai wrote:

> hi!
>
> we are having issues with our contextl-heavy code on sbcl with high
> load. i suspect that the problem happens due to currupting sbcl's
> internal state due to some missing locks in sbcl, but while i was
> reviewing the code i had a question: seems like *active-context* is
> holding some caches, but i can't see where *root-context* is protected
> from concurrent access.
>
> maybe the code should eq-check for *root-context* at some point and
> enforce a separate *active-context* on first use? this will of curse
> lower the cache efficiency, so maybe a cross-platform threading lib
> like bordeaux-threads could be used to protect the global cache or
> somehow ensure that *root-context* is rebound and copied in all
> threads (i think this needs some platform specific code).
>
> is this analysis correct at all?
>
> -- 
> attila
>
> fyi, the sbcl backtrace:
>
> bad thing to be a type specifier: #<CONTEXTL:STANDARD-LAYER-CLASS NIL
> {100CE950C1}>
> *** Backtrace is:
> 000: ((LAMBDA (ERROR)) #<SIMPLE-ERROR {100A281D11}>)
> 001: ((LAMBDA (ERROR)) #<SIMPLE-ERROR {100A281D11}>)
> 002: ((LAMBDA (ERROR)) #<SIMPLE-ERROR {100A281D11}>)
> 003: (SIGNAL #<SIMPLE-ERROR {100A281D11}>)
> 004: (ERROR \"bad thing to be a type specifier: ~S\")
> 005: (SB-KERNEL:VALUES-SPECIFIER-TYPE #<CONTEXTL:STANDARD-LAYER-CLASS
> NIL {100CE950C1}>)
> 006: (SB-KERNEL:SPECIFIER-TYPE #<CONTEXTL:STANDARD-LAYER-CLASS NIL
> {100CE950C1}>)
> 007: (SUBTYPEP #<CONTEXTL:STANDARD-LAYER-CLASS NIL {100CE950C1}>
> #<CONTEXTL:STANDARD-LAYER-CLASS EBR42-LAYER {1001760DB1}> #<unused
> argument>)
> 008: ((SB-PCL::FAST-METHOD
> LAYERED-FUNCTION-DEFINERS::|CONTEXTL::ADJOIN-LAYER-USING-CLASS| (T
> CONTEXTL:STANDARD-LAYER-CLASS T)) #<error printing object>)
> 009: (CONTEXTL:ADJOIN-LAYER #<unavailable argument> #<unavailable  
> argument>)
> 010: ((LAMBDA ()))
> _______________________________________________
> closer-devel mailing list
> closer-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/closer-devel

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From pc at p-cos.net  Tue Nov  6 13:55:59 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Tue, 6 Nov 2007 14:55:59 +0100
Subject: [closer-devel] *root-context* and threads
In-Reply-To: <b54b34af0711051311k2462c436h28788fdd03f2a47a@mail.gmail.com>
References: <b54b34af0711051311k2462c436h28788fdd03f2a47a@mail.gmail.com>
Message-ID: <77138466-1C1E-4F18-B0CB-DDECB101D14E@p-cos.net>

Hi,

Due to a hint at comp.lang.lisp, I am not so sure anymore whether this  
is actually a ContextL issue.

Are you sure that my caches cause any problems?


Pascal

On 5 Nov 2007, at 22:11, Attila Lendvai wrote:

> hi!
>
> we are having issues with our contextl-heavy code on sbcl with high
> load. i suspect that the problem happens due to currupting sbcl's
> internal state due to some missing locks in sbcl, but while i was
> reviewing the code i had a question: seems like *active-context* is
> holding some caches, but i can't see where *root-context* is protected
> from concurrent access.
>
> maybe the code should eq-check for *root-context* at some point and
> enforce a separate *active-context* on first use? this will of curse
> lower the cache efficiency, so maybe a cross-platform threading lib
> like bordeaux-threads could be used to protect the global cache or
> somehow ensure that *root-context* is rebound and copied in all
> threads (i think this needs some platform specific code).
>
> is this analysis correct at all?
>
> -- 
> attila
>
> fyi, the sbcl backtrace:
>
> bad thing to be a type specifier: #<CONTEXTL:STANDARD-LAYER-CLASS NIL
> {100CE950C1}>
> *** Backtrace is:
> 000: ((LAMBDA (ERROR)) #<SIMPLE-ERROR {100A281D11}>)
> 001: ((LAMBDA (ERROR)) #<SIMPLE-ERROR {100A281D11}>)
> 002: ((LAMBDA (ERROR)) #<SIMPLE-ERROR {100A281D11}>)
> 003: (SIGNAL #<SIMPLE-ERROR {100A281D11}>)
> 004: (ERROR \"bad thing to be a type specifier: ~S\")
> 005: (SB-KERNEL:VALUES-SPECIFIER-TYPE #<CONTEXTL:STANDARD-LAYER-CLASS
> NIL {100CE950C1}>)
> 006: (SB-KERNEL:SPECIFIER-TYPE #<CONTEXTL:STANDARD-LAYER-CLASS NIL
> {100CE950C1}>)
> 007: (SUBTYPEP #<CONTEXTL:STANDARD-LAYER-CLASS NIL {100CE950C1}>
> #<CONTEXTL:STANDARD-LAYER-CLASS EBR42-LAYER {1001760DB1}> #<unused
> argument>)
> 008: ((SB-PCL::FAST-METHOD
> LAYERED-FUNCTION-DEFINERS::|CONTEXTL::ADJOIN-LAYER-USING-CLASS| (T
> CONTEXTL:STANDARD-LAYER-CLASS T)) #<error printing object>)
> 009: (CONTEXTL:ADJOIN-LAYER #<unavailable argument> #<unavailable  
> argument>)
> 010: ((LAMBDA ()))
> _______________________________________________
> closer-devel mailing list
> closer-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/closer-devel

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From attila.lendvai at gmail.com  Tue Nov  6 14:15:32 2007
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Tue, 6 Nov 2007 15:15:32 +0100
Subject: [closer-devel] *root-context* and threads
In-Reply-To: <77138466-1C1E-4F18-B0CB-DDECB101D14E@p-cos.net>
References: <b54b34af0711051311k2462c436h28788fdd03f2a47a@mail.gmail.com>
	<77138466-1C1E-4F18-B0CB-DDECB101D14E@p-cos.net>
Message-ID: <b54b34af0711060615h311f0beejcac023b1c9a6c37c@mail.gmail.com>

> Due to a hint at comp.lang.lisp, I am not so sure anymore whether this
> is actually a ContextL issue.
>
> Are you sure that my caches cause any problems?

this is most probably an sbcl issue, sorry for not being clear enough!
("i suspect that the problem happens due to currupting sbcl's internal
state due to some missing locks in sbcl")

i just got this question when looking for the source of the problem.

but nevertheless i'm not sure how (setf (getf ...)) behaves with
multiple threads, especially when a new entry is added. after some
thoughts i think the worst corruption that would happen is duplicate
entries and deleted entries, so the current situation may not be too
bad at all.

but having a protected global cache for the created classes/prototypes
may be a good idea nevertheless, because it would probably speed
things up in a threaded environment. but again, i'm talking without
actually checking the code, so take it with a piece of salt...

-- 
 attila


From pc at p-cos.net  Tue Nov  6 14:25:23 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Tue, 6 Nov 2007 15:25:23 +0100
Subject: [closer-devel] *root-context* and threads
In-Reply-To: <b54b34af0711060615h311f0beejcac023b1c9a6c37c@mail.gmail.com>
References: <b54b34af0711051311k2462c436h28788fdd03f2a47a@mail.gmail.com>
	<77138466-1C1E-4F18-B0CB-DDECB101D14E@p-cos.net>
	<b54b34af0711060615h311f0beejcac023b1c9a6c37c@mail.gmail.com>
Message-ID: <1807BDA0-2B6B-4FF3-BFD7-4DD257619EE3@p-cos.net>


On 6 Nov 2007, at 15:15, Attila Lendvai wrote:

>> Due to a hint at comp.lang.lisp, I am not so sure anymore whether  
>> this
>> is actually a ContextL issue.
>>
>> Are you sure that my caches cause any problems?
>
> this is most probably an sbcl issue, sorry for not being clear enough!
> ("i suspect that the problem happens due to currupting sbcl's internal
> state due to some missing locks in sbcl")

OK, no problem.

> i just got this question when looking for the source of the problem.
>
> but nevertheless i'm not sure how (setf (getf ...)) behaves with
> multiple threads, especially when a new entry is added. after some
> thoughts i think the worst corruption that would happen is duplicate
> entries and deleted entries, so the current situation may not be too
> bad at all.

I'm not sure either, but I have now asked the several vendors. Let's  
see what they say...

I am not changing the code until then - I hope that's ok.

What you could try is to use your own layer metaclasses and return nil  
as second values in adjoin-layer-using-class or remove-layer-using- 
class. This should disable caching.

I could also add a flag to globally disable caching, or even  
conditionally remove the caching code completely (using reader  
conditionals).

These options are easier and have less undesirable implications than  
changing the code to protect it against concurrency issues, and they  
may help to localize your problems.

Let me know what you think would work best for you at the moment...

> but having a protected global cache for the created classes/prototypes
> may be a good idea nevertheless, because it would probably speed
> things up in a threaded environment.

I doubt that. A cache is faster when it's small, and keeping the  
caches for layer children local to their parents keeps them small -  
and a small cache justifies the use of a property list, which should  
even make it faster. So I'd like to stick to that, if possible.

Unless you have performance issues, which I'd like to know about, of  
course. I haven't been able to test ContextL with a large-scale  
application yet...

> but again, i'm talking without
> actually checking the code, so take it with a piece of salt...

OK, I see. But it's good to be aware of these potential issues anyway...


Pascal

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From attila.lendvai at gmail.com  Tue Nov  6 14:53:21 2007
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Tue, 6 Nov 2007 15:53:21 +0100
Subject: [closer-devel] *root-context* and threads
In-Reply-To: <1807BDA0-2B6B-4FF3-BFD7-4DD257619EE3@p-cos.net>
References: <b54b34af0711051311k2462c436h28788fdd03f2a47a@mail.gmail.com>
	<77138466-1C1E-4F18-B0CB-DDECB101D14E@p-cos.net>
	<b54b34af0711060615h311f0beejcac023b1c9a6c37c@mail.gmail.com>
	<1807BDA0-2B6B-4FF3-BFD7-4DD257619EE3@p-cos.net>
Message-ID: <b54b34af0711060653h6bed296ag2a0b60575497fb8e@mail.gmail.com>

> These options are easier and have less undesirable implications than
> changing the code to protect it against concurrency issues, and they
> may help to localize your problems.
>
> Let me know what you think would work best for you at the moment...


just take your time to find the proper strategy in contextl. the
machines can mostly keep up with the load and this is not a mission
critical app, so if one or two restarts happen it's bearable...

and i think the issue is localized in sbcl (Nikodemus listed what the
things are that need to be protected from concurrent access).


> > but having a protected global cache for the created classes/prototypes
> > may be a good idea nevertheless, because it would probably speed
> > things up in a threaded environment.
>
> I doubt that. A cache is faster when it's small, and keeping the
> caches for layer children local to their parents keeps them small -
> and a small cache justifies the use of a property list, which should
> even make it faster. So I'd like to stick to that, if possible.


i have no numbers, but you should also consider that calling
make-instance on classes, then dispatching on their instances has many
"invisible" performance penalties. huge locks must be held while
defining the class, method caches must be flushed and refilled for new
instances, etc...


> Unless you have performance issues, which I'd like to know about, of
> course. I haven't been able to test ContextL with a large-scale
> application yet...


i'll report back if i have any numbers/thoughts, but for now
performance is not a critical issue.

thanks,

-- 
 attila


From pc at p-cos.net  Tue Nov  6 15:10:06 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Tue, 6 Nov 2007 16:10:06 +0100
Subject: [closer-devel] *root-context* and threads
In-Reply-To: <b54b34af0711060653h6bed296ag2a0b60575497fb8e@mail.gmail.com>
References: <b54b34af0711051311k2462c436h28788fdd03f2a47a@mail.gmail.com>
	<77138466-1C1E-4F18-B0CB-DDECB101D14E@p-cos.net>
	<b54b34af0711060615h311f0beejcac023b1c9a6c37c@mail.gmail.com>
	<1807BDA0-2B6B-4FF3-BFD7-4DD257619EE3@p-cos.net>
	<b54b34af0711060653h6bed296ag2a0b60575497fb8e@mail.gmail.com>
Message-ID: <CC670EFB-791B-4D0C-BD10-84D99D2ABCC1@p-cos.net>


On 6 Nov 2007, at 15:53, Attila Lendvai wrote:

>> These options are easier and have less undesirable implications than
>> changing the code to protect it against concurrency issues, and they
>> may help to localize your problems.
>>
>> Let me know what you think would work best for you at the moment...
>
> just take your time to find the proper strategy in contextl. the
> machines can mostly keep up with the load and this is not a mission
> critical app, so if one or two restarts happen it's bearable...

OK, that's good to know.

I am also already getting some good answers, so I think I will be able  
to solve the ContextL cache issues...

> and i think the issue is localized in sbcl (Nikodemus listed what the
> things are that need to be protected from concurrent access).
>
>>> but having a protected global cache for the created classes/ 
>>> prototypes
>>> may be a good idea nevertheless, because it would probably speed
>>> things up in a threaded environment.
>>
>> I doubt that. A cache is faster when it's small, and keeping the
>> caches for layer children local to their parents keeps them small -
>> and a small cache justifies the use of a property list, which should
>> even make it faster. So I'd like to stick to that, if possible.
>
> i have no numbers, but you should also consider that calling
> make-instance on classes, then dispatching on their instances has many
> "invisible" performance penalties. huge locks must be held while
> defining the class, method caches must be flushed and refilled for new
> instances, etc...

I strongly believe that I have taken taken all these issues into  
account, and that the current approach is optimal. A global cache  
would probably make things worse, as far as I can tell.

For example, I think I have reduced the creation of new instances to a  
minimum, and method caches should never be flushed in the current  
ContextL implementation. I have been hit seriously by method cache  
flushes for 'special' functions in AspectL, so I invested some time  
into redesigning what has now become layered functions in ContextL.

But I'd be happy to be proven wrong, so I can improve things, if  
possible...

>> Unless you have performance issues, which I'd like to know about, of
>> course. I haven't been able to test ContextL with a large-scale
>> application yet...
>
> i'll report back if i have any numbers/thoughts, but for now
> performance is not a critical issue.


OK, good.


Pascal

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From coffeemug at gmail.com  Sat Nov 10 22:54:10 2007
From: coffeemug at gmail.com (Vyacheslav Akhmechet)
Date: Sat, 10 Nov 2007 17:54:10 -0500
Subject: [closer-devel] OpenMCL and funcallable-standard-object
Message-ID: <b381ea40711101454x4a72dc8dh5e96746bf50d293d@mail.gmail.com>

On OpenMCL 1.0 the symbol FUNCALLABLE-STANDARD-OBJECT is not exported
from C2MOP. It seems to be exported on SBCL, CMUCL, ACL, CLISP and
LispWorks. On OpenMCL the symbol is interned in CCL package (and for
some reason isn't exported).

Is this expected?


From pc at p-cos.net  Sun Nov 11 14:12:50 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Sun, 11 Nov 2007 15:12:50 +0100
Subject: [closer-devel] OpenMCL and funcallable-standard-object
In-Reply-To: <b381ea40711101454x4a72dc8dh5e96746bf50d293d@mail.gmail.com>
References: <b381ea40711101454x4a72dc8dh5e96746bf50d293d@mail.gmail.com>
Message-ID: <44D950B9-75AD-4A04-AD62-E1129154AE3C@p-cos.net>


On 10 Nov 2007, at 23:54, Vyacheslav Akhmechet wrote:

> On OpenMCL 1.0 the symbol FUNCALLABLE-STANDARD-OBJECT is not exported
> from C2MOP. It seems to be exported on SBCL, CMUCL, ACL, CLISP and
> LispWorks. On OpenMCL the symbol is interned in CCL package (and for
> some reason isn't exported).
>
> Is this expected?

Yes, OpenMCL doesn't support the generic function invocation protocol  
as specified in AMOP, so there is not much use to export that symbol.

There was a suggestion how to make at least parts of the protocol  
work. As soon as I find the time to look more into this, it may happen  
that I add this in a later version of Closer to MOP. But don't hold  
your breath...

Pascal

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From coffeemug at gmail.com  Sun Nov 11 19:17:09 2007
From: coffeemug at gmail.com (Vyacheslav Akhmechet)
Date: Sun, 11 Nov 2007 14:17:09 -0500
Subject: [closer-devel] OpenMCL and funcallable-standard-object
In-Reply-To: <44D950B9-75AD-4A04-AD62-E1129154AE3C@p-cos.net>
References: <b381ea40711101454x4a72dc8dh5e96746bf50d293d@mail.gmail.com>
	<44D950B9-75AD-4A04-AD62-E1129154AE3C@p-cos.net>
Message-ID: <b381ea40711111117j6d72e154he4a2b49bdef3b38a@mail.gmail.com>

On 11/11/07, Pascal Costanza <pc at p-cos.net> wrote:
> Yes, OpenMCL doesn't support the generic function invocation protocol
> as specified in AMOP, so there is not much use to export that symbol.
Right, but I'm not talking about GF invocation. I have code that makes
instances of classes funcallable. Because ACL doesn't automatically
use FUNCALLABLE-STANDARD-OBJECT for classes with metaclass
FUNCALLABLE-STANDARD-CLASS I have code that looks like this:

(defclass foo (c2mop:funcallable-standard-object)
 (...)
 (:metaclass c2mop:funcallable-standard-class))

I can easily add #+allegro to the superclass, but OpenMCL does support
support FUNCALLABLE-STANDARD-OBJECT, it's just that it's not exported
by Closer.

> There was a suggestion how to make at least parts of the protocol
> work. As soon as I find the time to look more into this, it may happen
> that I add this in a later version of Closer to MOP. But don't hold
> your breath...
That was probably the code I posted which I wrote to make parts of
Weblocks work.


From pc at p-cos.net  Mon Nov 12 08:33:36 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Mon, 12 Nov 2007 09:33:36 +0100
Subject: [closer-devel] OpenMCL and funcallable-standard-object
In-Reply-To: <b381ea40711111117j6d72e154he4a2b49bdef3b38a@mail.gmail.com>
References: <b381ea40711101454x4a72dc8dh5e96746bf50d293d@mail.gmail.com>
	<44D950B9-75AD-4A04-AD62-E1129154AE3C@p-cos.net>
	<b381ea40711111117j6d72e154he4a2b49bdef3b38a@mail.gmail.com>
Message-ID: <0D518D42-16A1-4380-B927-664AAFA3B9A8@p-cos.net>


On 11 Nov 2007, at 20:17, Vyacheslav Akhmechet wrote:

> On 11/11/07, Pascal Costanza <pc at p-cos.net> wrote:
>> Yes, OpenMCL doesn't support the generic function invocation protocol
>> as specified in AMOP, so there is not much use to export that symbol.
> Right, but I'm not talking about GF invocation. I have code that makes
> instances of classes funcallable. Because ACL doesn't automatically
> use FUNCALLABLE-STANDARD-OBJECT for classes with metaclass
> FUNCALLABLE-STANDARD-CLASS I have code that looks like this:
>
> (defclass foo (c2mop:funcallable-standard-object)
> (...)
> (:metaclass c2mop:funcallable-standard-class))
>
> I can easily add #+allegro to the superclass, but OpenMCL does support
> support FUNCALLABLE-STANDARD-OBJECT, it's just that it's not exported
> by Closer.

OK, I'll make it available as soon as possible...

>> There was a suggestion how to make at least parts of the protocol
>> work. As soon as I find the time to look more into this, it may  
>> happen
>> that I add this in a later version of Closer to MOP. But don't hold
>> your breath...
> That was probably the code I posted which I wrote to make parts of
> Weblocks work.

Yes, very likely, now that you mention it... ;)


Pascal

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From pc at p-cos.net  Mon Nov 12 17:06:34 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Mon, 12 Nov 2007 18:06:34 +0100
Subject: [closer-devel] Thread-safety in ContextL, and related news
Message-ID: <27A5A68D-2341-491D-A9E9-ECD2C0AD6A83@p-cos.net>

Hi,

I have now added the necessary synchronization to (hopefully) ensure  
that layer context caches are thread-safe in ContextL. I also checked  
the rest of the ContextL, and found one other place where  
synchronization is necessary, and added the necessary code as well.  
(The changes are only in the repository, not in the official version  
of ContextL yet.)

The synchronization is done in a way that the caches won't hold  
duplicate entries, and creating a cache entry should always lead to a  
proper update of the cache - so cache entries cannot be missed by  
other threads. It should even be the case that CL implementations with  
'real' parallelism on multicore architectures should also be safe  
(whether this matters at the moment or not is a different question ;).

ContextL now depends on portable-threads of the GBBopen project.  
Please go to http://gbbopen.org/ and download the whole package from  
there - I have tested against the version in their SVN repository. You  
need to add gbbopen.asd to your asdf central registry, GBBopen figures  
out the rest for you. (There is also an asdf-install version of  
portable-threads, but that one seems to be outdated. GBBopen doesn't  
seem to be asdf-installable itself, but I am not sure - I actually  
don't use asdf-install myself.)

I have run my test suites successfully on Allegro 7.0, 8.0 and 8.1;  
clisp 2.42 (aka 2.41.1), CMUCL 19d, LispWorks 5.02, OpenMCL 1.0 and  
SBCL 1.0.11. I couldn't test MCL, because GBBopen doesn't compile for  
MCL - I will report this to the GBBopen guys. LispWorks 4.4.x doesn't  
run on Leopard anymore, so I cannot test it. (I will probably remove  
support for LispWorks < 5.0.x in the next release of ContextL.)

Please let me know if there are any problems or questions.


Best,
Pascal

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From pc at p-cos.net  Mon Nov 12 20:29:50 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Mon, 12 Nov 2007 21:29:50 +0100
Subject: [closer-devel] Thread-safety in ContextL, and related news
In-Reply-To: <27A5A68D-2341-491D-A9E9-ECD2C0AD6A83@p-cos.net>
References: <27A5A68D-2341-491D-A9E9-ECD2C0AD6A83@p-cos.net>
Message-ID: <08B54C0B-641A-4B6B-9368-F608DBA4EE70@p-cos.net>

Ah, I forgot to mention: The locks are only acquired on the slow  
paths, not on the fast paths. So as soon as the caches are filled,  
locks will never be acquired again. Therefore, the overall performance  
of ContextL shouldn't suffer from this change.

I just thought this may be interesting to some...

Pascal

On 12 Nov 2007, at 18:06, Pascal Costanza wrote:

> Hi,
>
> I have now added the necessary synchronization to (hopefully) ensure  
> that layer context caches are thread-safe in ContextL. I also  
> checked the rest of the ContextL, and found one other place where  
> synchronization is necessary, and added the necessary code as well.  
> (The changes are only in the repository, not in the official version  
> of ContextL yet.)
>
> The synchronization is done in a way that the caches won't hold  
> duplicate entries, and creating a cache entry should always lead to  
> a proper update of the cache - so cache entries cannot be missed by  
> other threads. It should even be the case that CL implementations  
> with 'real' parallelism on multicore architectures should also be  
> safe (whether this matters at the moment or not is a different  
> question ;).
>
> ContextL now depends on portable-threads of the GBBopen project.  
> Please go to http://gbbopen.org/ and download the whole package from  
> there - I have tested against the version in their SVN repository.  
> You need to add gbbopen.asd to your asdf central registry, GBBopen  
> figures out the rest for you. (There is also an asdf-install version  
> of portable-threads, but that one seems to be outdated. GBBopen  
> doesn't seem to be asdf-installable itself, but I am not sure - I  
> actually don't use asdf-install myself.)
>
> I have run my test suites successfully on Allegro 7.0, 8.0 and 8.1;  
> clisp 2.42 (aka 2.41.1), CMUCL 19d, LispWorks 5.02, OpenMCL 1.0 and  
> SBCL 1.0.11. I couldn't test MCL, because GBBopen doesn't compile  
> for MCL - I will report this to the GBBopen guys. LispWorks 4.4.x  
> doesn't run on Leopard anymore, so I cannot test it. (I will  
> probably remove support for LispWorks < 5.0.x in the next release of  
> ContextL.)
>
> Please let me know if there are any problems or questions.
>
>
> Best,
> Pascal
>
> -- 
> Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
> Vrije Universiteit Brussel, Programming Technology Lab
> Pleinlaan 2, B-1050 Brussel, Belgium
>
>
>
>
> _______________________________________________
> closer-devel mailing list
> closer-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/closer-devel

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From pc at p-cos.net  Sat Nov 17 19:23:26 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Sat, 17 Nov 2007 20:23:26 +0100
Subject: [closer-devel] Active layers...
Message-ID: <92ED49AC-DE20-43BF-86D7-071EF947BA11@p-cos.net>

Hi,

I have added a new introspective function to ContextL that I find  
extremely useful in my own attempts at actually using ContextL for  
some non-trivial examples.

Active-layers returns the set of currently active layers (without all  
the noise that the layer context objects produce). It accepts an  
optional parameter for passing a layer context, which defaults to the  
currently active layer context.

The functions ensure-active-layer and ensure-inactive-layer now call  
active-layers in the end to produce a useful response. If anyone of  
you counted on the return values previously returned by those two  
functions (the new active layer context), please let me know and I  
will immediately revert that change. (It's not critical to me one way  
or the other, I just find the call of active-layers a more useful  
response than the one before.)

On a somewhat unrelated note: I have removed the explicit loading of  
the gbbopen.asd file at the top of contextl.asd. The current version  
of GBBopen now adds an explicit portable-threads.asd, which is all  
that is needed for ContextL. (Thanks to Attila for informing me about  
this.)


Best,
Pascal

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From pc at p-cos.net  Sat Nov 17 23:54:44 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Sun, 18 Nov 2007 00:54:44 +0100
Subject: [closer-devel] OpenMCL and funcallable-standard-object
In-Reply-To: <0D518D42-16A1-4380-B927-664AAFA3B9A8@p-cos.net>
References: <b381ea40711101454x4a72dc8dh5e96746bf50d293d@mail.gmail.com>
	<44D950B9-75AD-4A04-AD62-E1129154AE3C@p-cos.net>
	<b381ea40711111117j6d72e154he4a2b49bdef3b38a@mail.gmail.com>
	<0D518D42-16A1-4380-B927-664AAFA3B9A8@p-cos.net>
Message-ID: <C5206F7E-BA2C-4B2D-BE23-B75972DD1E37@p-cos.net>


On 12 Nov 2007, at 09:33, Pascal Costanza wrote:

>
> On 11 Nov 2007, at 20:17, Vyacheslav Akhmechet wrote:
>
>> On 11/11/07, Pascal Costanza <pc at p-cos.net> wrote:
>>> Yes, OpenMCL doesn't support the generic function invocation  
>>> protocol
>>> as specified in AMOP, so there is not much use to export that  
>>> symbol.
>> Right, but I'm not talking about GF invocation. I have code that  
>> makes
>> instances of classes funcallable. Because ACL doesn't automatically
>> use FUNCALLABLE-STANDARD-OBJECT for classes with metaclass
>> FUNCALLABLE-STANDARD-CLASS I have code that looks like this:
>>
>> (defclass foo (c2mop:funcallable-standard-object)
>> (...)
>> (:metaclass c2mop:funcallable-standard-class))
>>
>> I can easily add #+allegro to the superclass, but OpenMCL does  
>> support
>> support FUNCALLABLE-STANDARD-OBJECT, it's just that it's not exported
>> by Closer.
>
> OK, I'll make it available as soon as possible...

It's now in the repository.


Pascal

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From pc at p-cos.net  Sun Nov 18 20:08:56 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Sun, 18 Nov 2007 21:08:56 +0100
Subject: [closer-devel] PATCH: Better print-object for layers: print the
	layer precedence list for unnamed layers
In-Reply-To: <b54b34af0707300940y1ca3b4f1rc54eac73d9aa1c16@mail.gmail.com>
References: <b54b34af0707200638j37619ab1s6e9d29913c8856be@mail.gmail.com>
	<6424F228-370F-4D11-88C2-4FBB948EC30C@p-cos.net>
	<b54b34af0707300940y1ca3b4f1rc54eac73d9aa1c16@mail.gmail.com>
Message-ID: <459C263B-F6C5-43A8-A095-11DA43EA3CFE@p-cos.net>

Hi Attila,

Do you still need this patch, or is the new active-layers function  
sufficient for your purposes?


Pascal

On 30 Jul 2007, at 18:40, Attila Lendvai wrote:

>> OK, this looks good, I will add this.
>
> thanks!
>
>> Two questions:
>>
>> - Does it matter if any of the layers in the precedence list are  
>> anonymous?
>
> i think all i wanted to know is a list of enabled layers...
>
>> - Shouldn't print-object additionally say that the layer itself is
>> anonymous?
>
> ...but any information over that can be potentially useful. when i
> need even more information, then i usually customize the slime
> inspector.
>
> i've also sent a patch to sbcl-devel that makes sbcl print the object
> identity for unnamed classes, which was also a must-have when i was
> debugging a problem i was looking for. but it's not in the official,
> yet.
>
> -- 
> attila

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From attila.lendvai at gmail.com  Sun Nov 18 20:33:56 2007
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Sun, 18 Nov 2007 21:33:56 +0100
Subject: [closer-devel] PATCH: Better print-object for layers: print the
	layer precedence list for unnamed layers
In-Reply-To: <459C263B-F6C5-43A8-A095-11DA43EA3CFE@p-cos.net>
References: <b54b34af0707200638j37619ab1s6e9d29913c8856be@mail.gmail.com>
	<6424F228-370F-4D11-88C2-4FBB948EC30C@p-cos.net>
	<b54b34af0707300940y1ca3b4f1rc54eac73d9aa1c16@mail.gmail.com>
	<459C263B-F6C5-43A8-A095-11DA43EA3CFE@p-cos.net>
Message-ID: <b54b34af0711181233g1c0e628dl17b9f902af1688b7@mail.gmail.com>

> Hi Attila,
>
> Do you still need this patch, or is the new active-layers function
> sufficient for your purposes?

hi!

looks good, i've unpulled my print-object patch.

thanks!

-- 
 attila


From rjekel at gmail.com  Tue Nov 20 04:18:46 2007
From: rjekel at gmail.com (Ross Jekel)
Date: Mon, 19 Nov 2007 20:18:46 -0800
Subject: [closer-devel] ContextL Layers and persistence
Message-ID: <7db65b60711192018u7bb9e1eayf79e7a61b14f45eb@mail.gmail.com>

Hi,

I'm new to the list, so I apologize if this has been discussed before and I
missed it.

I've been reading the papers by Pascal Costanza et al. and trying to get my
head around ContextL. The examples in the overview paper are pretty clear,
but I seldom can grasp the usefulness of employer/person/employee/address
models without considering persistence.

While this is not my strongest area, it would seem at first that the
decomposition of the data model into layers is similar to decomposition into
normal forms or tables in a relational model. If so, would it be possible to
create a persistent mixin layer class in the layer class hierarchy to
facilitate both automatic Object To Relational mappings of layered slot sets
and to allow a mixture of persistent and run-time layers (such as view
layers) on the same objects?

Also if one would use direct object persistence such as that in Franz's
AllegroCache, could one use the layer system and persistence simultaneously?
Allegro's solution, like layers, relies on a metaclass to implement its
feature. I've done a little MOP work, but haven't mixed-in two different
metaclasses yet. Would it be simply creating a class that inherits from both
layered-class and Allegro's persistence metaclass, then supplying the
(:metaclass that-class) form in the define-layered-class (I noticed you only
supply the (:metaclass 'layered-class) if a :metaclass form is not present).
>From the looks of the macros and what I understand of CLOS, it appears that
if persistence as a part of the metaclass on any layer, then it would need
to be used on all of them and all layers become persistent, right? Or maybe
don't understand how it would work to have different metaclasses specified
on different layers of the same object since with defclass as the underlying
mechanism one is redefining the object (is the metaclass for a given layer
only used to define the slots in that one layer?).

Thanks for any help you can provide in clearing up my understanding.

Ross
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/closer-devel/attachments/20071119/0745cb58/attachment.html>

From pc at p-cos.net  Tue Nov 20 11:16:44 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Tue, 20 Nov 2007 12:16:44 +0100
Subject: [closer-devel] ContextL Layers and persistence
In-Reply-To: <7db65b60711192018u7bb9e1eayf79e7a61b14f45eb@mail.gmail.com>
References: <7db65b60711192018u7bb9e1eayf79e7a61b14f45eb@mail.gmail.com>
Message-ID: <24EE23D5-952B-4B5C-8D5A-4FD3DA9A85B7@p-cos.net>

Hi Ross,

On 20 Nov 2007, at 05:18, Ross Jekel wrote:

> Hi,
>
> I'm new to the list, so I apologize if this has been discussed  
> before and I missed it.

Welcome to the list. :)

This mailing list is still relatively new, or at least hasn't been  
swamped with discussions, so you can still discuss whatever you want,  
and the chances are pretty low that whatever you ask has already been  
discussed before. ;)

> I've been reading the papers by Pascal Costanza et al. and trying to  
> get my head around ContextL. The examples in the overview paper are  
> pretty clear, but I seldom can grasp the usefulness of employer/ 
> person/employee/address models without considering persistence.

The example in the overview paper is not the best one, but the best we  
could come up with at that time. The typical use case that has been  
discovered independently by several people (not by us) is the  
generation of documents in different formats (pdf, html, json, etc.),  
and/or in different languages, from the same object graph. But there  
are also other use cases, and we are trying to come up with new  
examples for new papers...

> While this is not my strongest area, it would seem at first that the  
> decomposition of the data model into layers is similar to  
> decomposition into normal forms or tables in a relational model. If  
> so, would it be possible to create a persistent mixin layer class in  
> the layer class hierarchy to facilitate both automatic Object To  
> Relational mappings of layered slot sets and to allow a mixture of  
> persistent and run-time layers (such as view layers) on the same  
> objects?

This is an interesting idea, and should indeed be possible in  
principle (although it hasn't been tried yet, as far as I can tell).

In the ContextL implementation, each direct slot defined in a layered  
class has a record of which layer it is defined in. Each effective  
slot also records the set of layers of all the direct slots from which  
it is composed. So it is indeed possible to define a mapping from a  
layer to the slots that are defined in that layer. It's then a matter  
of finding a way to define which layers should be persisted and which  
shouldn't.

> Also if one would use direct object persistence such as that in  
> Franz's AllegroCache, could one use the layer system and persistence  
> simultaneously? Allegro's solution, like layers, relies on a  
> metaclass to implement its feature. I've done a little MOP work, but  
> haven't mixed-in two different metaclasses yet. Would it be simply  
> creating a class that inherits from both layered-class and Allegro's  
> persistence metaclass, then supplying the (:metaclass that-class)  
> form in the define-layered-class (I noticed you only supply the  
> (:metaclass 'layered-class) if a :metaclass form is not present).

No, it's not as simple as that. In general, metaclasses do not  
compose. That's the primary reason for the validate-superclass hook in  
the CLOS MOP, where you have to announce which metaclasses are  
compatible and which are not. (And this is not a specific limitation  
of the CLOS MOP, but a general limitation of all metaprogramming  
approaches.)

It could _accidentally_ just work to compose the two metaclasses, but  
normally, you have to review the different hooks of the CLOS MOP API,  
and find out what the interactions are. In this case, I'd expect that  
you at least need to define new subclasses for direct slots and  
effective slots, and provide new methods for direct-slot-definition- 
class, effective-slot-definition-class, and the slot-xyz-using-class  
functions.

Yes, the handling of the :metaclass option in define-layered-class is  
intentional, and is there to allow for such new metaclasses based on  
layered-class. But I can't guarantee that this hook in ContextL in  
sufficient - we simply don't have enough experience with that part of  
the ContextL implementation to claim that we know how all the details  
work.

> From the looks of the macros and what I understand of CLOS, it  
> appears that if persistence as a part of the metaclass on any layer,  
> then it would need to be used on all of them and all layers become  
> persistent, right? Or maybe  don't understand how it would work to  
> have different metaclasses specified on different layers of the same  
> object since with defclass as the underlying mechanism one is  
> redefining the object (is the metaclass for a given layer only used  
> to define the slots in that one layer?).

It doesn't make a lot of sense to make layers persistent because they  
are typically stateless. (Although you can define slots on them, but  
that's probably a different story.)

You probably rather mean to make layered classes persistent. (Right?)  
Yes, it is true that, once you have chosen a particular metaclass for  
a given class, you cannot change that decision anymore. The different  
classes in a class hierarchy can be instances of different  
metaclasses, but class inheriting from each must have compatible  
metaclasses.

The right place to make a distinction between persistent and non- 
persistent slots is in methods on direct-slot-definition-class and  
effective-slot-definition-class on the one hand, and the slot-xyz- 
using-class methods on the other hand. If you take a closer (ha!) look  
at the ContextL implementation, you can see that I use this kind of  
distinction in a few places. For example, a :special slot records the  
fact that it is special in its direct slot definition. The effective  
slot definition is then either a standard-effective-slot-definition or  
a special-effective-slot-definition. For special-effective-slot- 
definition, the new methods on slot-xyz-using-class will be used,  
which triggers the new semantics for special slots, whereas for  
standard-effective-slot-definition, these methods will not be called,  
but the default implementation for slot access in CLOS will be used.  
You could make similar distinctions for persistent and non-persistent  
slots. It all depends on whether you can determine whether a slot  
should be persistent as soon as effective-slot-definition-class is  
called on a slot.


Does that help?


Best,
Pascal

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From pc at p-cos.net  Sat Nov 24 18:26:25 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Sat, 24 Nov 2007 19:26:25 +0100
Subject: [closer-devel] ACL 8.0 generic-function-argument-precedence-order
In-Reply-To: <B02898F2-31C5-4CF1-98FA-E0A1065FF3E9@p-cos.net>
References: <b381ea40709132002kd9c7012se855a13a4194a8a7@mail.gmail.com>
	<B02898F2-31C5-4CF1-98FA-E0A1065FF3E9@p-cos.net>
Message-ID: <3FD4EFA6-698C-45C4-8343-3FCB7BBFBDD0@p-cos.net>


On 14 Sep 2007, at 07:32, Pascal Costanza wrote:

> On 14 Sep 2007, at 05:02, Vyacheslav Akhmechet wrote:
>
>> Hi,
>>
>>> From what I understand from MOP recommendation
>> generic-function-argument-precedence-order is meant to return only
>> required arguments. Quoting the recommendation, "This value is a list
>> of symbols, a permutation of the required parameters in the lambda
>> list of the generic function." So far I've verified that this is
>> honored in SBCL and LispWorks, but not ACL:
>>
>> (defmethod foo (a b c &rest d) nil)
>> (generic-function-lambda-list #'foo)
>> => (A B C &REST D)
>>
>> Am I misunderstanding the spec? Are there plans to fix this?
>
> I haven't been aware of this incompatibility yet. It should be  
> straightforward to provide a workaround in Closer to MOP, so I will  
> look into this.

There is now a fix for this problem in the Closer to MOP darcs  
repository.


Pascal

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From pc at p-cos.net  Mon Nov 26 00:31:34 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Mon, 26 Nov 2007 01:31:34 +0100
Subject: [closer-devel] ContextL: Garbage collector for layer activation
	caches added
Message-ID: <AC5A4D3F-586C-40BF-8B06-6CD76687ACF2@p-cos.net>

Hi,

The darcs repository for ContextL contains new changes which add a  
garbage collector for layer activation caches. The garbage collector  
works as follows:

- Whenever a layer is redefined or reinitialized (either via  
reinitialize-instance or via reevaluating a deflayer form for an  
already existing layer), all entries for that layer in the layer  
activation and deactivation caches are removed. Since layer-makunbound  
is expressed in terms of (setf find-class) and (setf class-name),  
which triggers a call of reinitialize-instance according to the CLOS  
MOP specification, this also removes the respective layer from the  
layer activation caches.

- Likewise, whenever methods are added to adjoin-layer-using-class and  
remove-layer-using-class, the layers which are instances of the layer  
metaclasses on which these methods are specialized are removed from  
the layer activation and deactivation caches. (More specifically, the  
activation caches for adjoin-layer-using-class and the deactivation  
caches for remove-layer-using-class.)

Both cases are primarily intended to better support development of new  
layer metaclasses, but also specific functionality when adjoin-layer- 
using-class / remove-layer-using-class depend on properties of the  
respective layers. Methods on these generic functions can now return a  
second value of t to indicate that the result can be cached, but are  
reassured that if their assumptions are invalidated, they will get  
called again no matter what.

ContextL now also exports a function clear-layer-caches which blindly  
clears all layer activation and deactivation caches, so that you can  
start building your caches from scratch.

There are now two asdf-files included in ContextL: contextl-with-layer- 
gc.asd includes the layer garbage collector, whereas contextl-without- 
layer-gc.asd doesn't include it. If you don't trust the garbage  
collector, this means that you can use the latter system definition.  
The default case (in contextl.asd) is to include the garbage collector.

The function clear-layer-caches doesn't exist when the garbage  
collector is not included.

The garbage collector adds a feature :contextl-layer-gc to *features*.  
However, don't count on this being there in the long run. The long- 
term goal is to keep the garbage collector as the default and to  
remove the version without the garbage collector. The feature is only  
used for internal purposes.

And yes, the garbage collector shouldn't affect the performance of  
layer activation and deactivation, and the meta-level API also works  
as before. ;)

Please let me know what you think, especially when you see problems in  
the approach I have taken.


Best,
Pascal

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From attila.lendvai at gmail.com  Mon Nov 26 17:30:41 2007
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Mon, 26 Nov 2007 18:30:41 +0100
Subject: [closer-devel] layer slots
Message-ID: <b54b34af0711260930x5de4247bm36cdd620e7c0a6e@mail.gmail.com>

hi,

while working on a new architecture for verrazano where backends are
layers, i wanted to add slots for layers.

after looking at the contextl code i ended up with two questions:
 - there's no public api to get the prototype from the layer-context
on which i could access the slots i've added to my layer
 - the prototype in the active layer-context is not instantiated with
make-instance but the class-prototype is used which means that layer
slots are effectively shared among threads (and maybe even after a
nested deactivation/activation of the layer?)

the test files suggest usage like this:

(assert (eq (slot0 (find-layer 'test-layer)) 'foo))

but find-layer also simply calls class-prototype after a find-class on
the layer name, which again is a shared instance. or am i badly
missing here something?

maybe it's ok to call class-prototype because there'll always be a
distinct class? if so, how does it interact with caching?

did i make some sense?

-- 
 attila


From attila.lendvai at gmail.com  Mon Nov 26 19:36:43 2007
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Mon, 26 Nov 2007 20:36:43 +0100
Subject: [closer-devel] Re: layer slots
In-Reply-To: <b54b34af0711260930x5de4247bm36cdd620e7c0a6e@mail.gmail.com>
References: <b54b34af0711260930x5de4247bm36cdd620e7c0a6e@mail.gmail.com>
Message-ID: <b54b34af0711261136g1afebcc9s5833dca38bf51581@mail.gmail.com>

> while working on a new architecture for verrazano where backends are
> layers, i wanted to add slots for layers.

to clarify what i'm trying to achive here: i would like to have slots
on layers that can hold data for "sessions" when the layer is active.

from a different point of view: i would like to represent verrazano
backends (that generate cffi/whatever bindings from gccxml output) as
layers and the layer called 'backend should have a slot called
'output-definition-queue that holds the definitions to be output for a
generation run. then i enable the actual backend layer and i should be
there initialized on the activated layer.

i've added this to my code:

(defun current-layer ()
  (contextl::layer-context-prototype (current-layer-context)))

and access the slot on (current-layer), but that is accessing a slot
of the class prototype with all its undesired consequences for my
use-case.

without contextl i would be doing something very similar: introduce a
first argument called backend, make-instance a backend instance and
dispatch on it on all the methods that are now layered methods.

am i misusing layers here?

-- 
 attila


From pc at p-cos.net  Mon Nov 26 22:21:27 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Mon, 26 Nov 2007 23:21:27 +0100
Subject: [closer-devel] layer slots
In-Reply-To: <b54b34af0711260930x5de4247bm36cdd620e7c0a6e@mail.gmail.com>
References: <b54b34af0711260930x5de4247bm36cdd620e7c0a6e@mail.gmail.com>
Message-ID: <FC4D03F3-EFF9-49E9-8267-5337686B0000@p-cos.net>

Hi,

On 26 Nov 2007, at 18:30, Attila Lendvai wrote:

> hi,
>
> while working on a new architecture for verrazano where backends are
> layers, i wanted to add slots for layers.
>
> after looking at the contextl code i ended up with two questions:
> - there's no public api to get the prototype from the layer-context
> on which i could access the slots i've added to my layer

No, but you can always get a prototype for a layer via find-layer. For  
example, you can say (find-layer 'foo) for getting the prototype for  
layer 'foo.

> - the prototype in the active layer-context is not instantiated with
> make-instance but the class-prototype is used which means that layer
> slots are effectively shared among threads (and maybe even after a
> nested deactivation/activation of the layer?)

Yes, layer slots are effectively shared, including across threads.

Layers are actually singletons, and that's intentional. It allows me  
to cache layer contexts and do a quick lookup of contexts on each  
layer activation/deactivation. If layers weren't singletons, I would  
have to create a layer instance for each activation/deactivation,  
because I wouldn't know under what other circumstances an already  
existing instance is already in use somewhere else. (I could do some  
form of layer instance pooling, but that would still be less efficient  
than the current scheme.)

When you define slots on layers, as in the following form, these slots  
are internally actually turned into class / shared slots.

(deflayer foo ()
   ((some-slots :accessor some-slot)))

You can access such a slot via the layer prototype: (some-slot (find- 
layer 'foo)).

There is always ever one such slot per layer.

> the test files suggest usage like this:
>
> (assert (eq (slot0 (find-layer 'test-layer)) 'foo))
>
> but find-layer also simply calls class-prototype after a find-class on
> the layer name, which again is a shared instance. or am i badly
> missing here something?

No, you found out right.

> maybe it's ok to call class-prototype because there'll always be a
> distinct class? if so, how does it interact with caching?

The fact that layers are singletons actually enables the caching  
scheme of ContextL.

If you really want to create the 'same' kind of layer several times,  
you can use subclassing on layers:

(deflayer foo1 (foo))
(deflayer foo2 (foo))
...

In this case, the slots from 'foo are still shared among all those  
layers, but you can redefine such slots in sublayers according to the  
semantics of overriding shared slots in CLOS.

However, that's probably overkill. See more on that in my next post.


Pascal

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From pc at p-cos.net  Mon Nov 26 22:32:47 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Mon, 26 Nov 2007 23:32:47 +0100
Subject: [closer-devel] Re: layer slots
In-Reply-To: <b54b34af0711261136g1afebcc9s5833dca38bf51581@mail.gmail.com>
References: <b54b34af0711260930x5de4247bm36cdd620e7c0a6e@mail.gmail.com>
	<b54b34af0711261136g1afebcc9s5833dca38bf51581@mail.gmail.com>
Message-ID: <5A4920B2-A03D-4035-98D3-06122122854A@p-cos.net>


On 26 Nov 2007, at 20:36, Attila Lendvai wrote:

>> while working on a new architecture for verrazano where backends are
>> layers, i wanted to add slots for layers.
>
> to clarify what i'm trying to achive here: i would like to have slots
> on layers that can hold data for "sessions" when the layer is active.
>
> from a different point of view: i would like to represent verrazano
> backends (that generate cffi/whatever bindings from gccxml output) as
> layers and the layer called 'backend should have a slot called
> 'output-definition-queue that holds the definitions to be output for a
> generation run. then i enable the actual backend layer and i should be
> there initialized on the activated layer.
>
> i've added this to my code:
>
> (defun current-layer ()
>  (contextl::layer-context-prototype (current-layer-context)))
>
> and access the slot on (current-layer), but that is accessing a slot
> of the class prototype with all its undesired consequences for my
> use-case.
>
> without contextl i would be doing something very similar: introduce a
> first argument called backend, make-instance a backend instance and
> dispatch on it on all the methods that are now layered methods.
>
> am i misusing layers here?

I am not sure, but I also have to admit that I don't fully understand  
the example yet.

However, what you could do is define a slot in a layer as a special  
slot:

(deflayer foo ()
   ((some-slot :initarg :some-slot :accessor some-slot :special t)))

Such a slot behaves like a special variable: It can have different  
bindings in different threads. There is a form in ContextL to activate  
a layer and at the same time bind such a slot:

(with-active-layers ((foo :some-slot 'some-value))
   ...)

This activates 'foo and binds some-slot to 'some-value with dynamic  
scope (other threads still see the old value of some-slot and the new  
binding will be automatically undone on return from that form).

This kind of layer activation is likely somewhat less efficient, since  
it involves looking up initialization keywords and calling  
reinitialize-instance on the layer prototypes.

A cheaper solution would actually involve to just use dletf:

(with-active-layers (foo)
   (dletf (((some-slot (find-layer 'foo)) 'some-value))
     ...))

Or maybe just a special variable:

(defvar *some-slot*)

(with-active-layers (foo)
   (let ((*some-slot* 'some-value))
     ...))

;)

I don't know which of those comes closer to solving your concrete  
problem.

If none of this does the job, please let me know and I will try to  
think harder about a better solution...


Thanks for the feedback,
Pascal

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium






From attila.lendvai at gmail.com  Thu Nov 29 08:35:54 2007
From: attila.lendvai at gmail.com (Attila Lendvai)
Date: Thu, 29 Nov 2007 09:35:54 +0100
Subject: [closer-devel] Re: layer slots
In-Reply-To: <5A4920B2-A03D-4035-98D3-06122122854A@p-cos.net>
References: <b54b34af0711260930x5de4247bm36cdd620e7c0a6e@mail.gmail.com>
	<b54b34af0711261136g1afebcc9s5833dca38bf51581@mail.gmail.com>
	<5A4920B2-A03D-4035-98D3-06122122854A@p-cos.net>
Message-ID: <b54b34af0711290035q1bb173eeue3257dae6310cf8b@mail.gmail.com>

> I don't know which of those comes closer to solving your concrete
> problem.
>
> If none of this does the job, please let me know and I will try to
> think harder about a better solution...

Pascal, thanks a lot for the exhaustive answer!

i'm sorry to let you know, but at the end i decided not to use
contextl in verrazano. the rationale is that it gives extra
dependencies and none of the alternative solutions were seamless
enough, so i just added the backend parameters by hand. this way the
resulting code will be more accessible to those who are not familiar
with contextl and i also felt some early overdesign in the air.

i wanted to have numerous layers like struct-emitting,
const-char-pointer-as-string, etc... and a backend layer would have
inherited a number of these. then later on the user could say
additionally an active and an inactive layer list on top of this that
is applied after the backend layer was enabled and before the
generation is started.

and all that while there's only a single cffi backend for verrazano...
so i decided to simply add an extra first backend parameter to the
relevant generics and if someone wants to customize the generation
they can just inherit from a backend and override as necessary.

it doesn't mean that contextl is not useful! but for now i think it
would be too much in verrazano.

-- 
 attila


From pc at p-cos.net  Thu Nov 29 19:43:44 2007
From: pc at p-cos.net (Pascal Costanza)
Date: Thu, 29 Nov 2007 20:43:44 +0100
Subject: [closer-devel] Re: layer slots
In-Reply-To: <b54b34af0711290035q1bb173eeue3257dae6310cf8b@mail.gmail.com>
References: <b54b34af0711260930x5de4247bm36cdd620e7c0a6e@mail.gmail.com>
	<b54b34af0711261136g1afebcc9s5833dca38bf51581@mail.gmail.com>
	<5A4920B2-A03D-4035-98D3-06122122854A@p-cos.net>
	<b54b34af0711290035q1bb173eeue3257dae6310cf8b@mail.gmail.com>
Message-ID: <D1F8B717-1051-44B0-828E-AA5CDB9DAEB2@p-cos.net>


On 29 Nov 2007, at 09:35, Attila Lendvai wrote:

>> I don't know which of those comes closer to solving your concrete
>> problem.
>>
>> If none of this does the job, please let me know and I will try to
>> think harder about a better solution...
>
> Pascal, thanks a lot for the exhaustive answer!
>
> i'm sorry to let you know, but at the end i decided not to use
> contextl in verrazano. the rationale is that it gives extra
> dependencies and none of the alternative solutions were seamless
> enough, so i just added the backend parameters by hand. this way the
> resulting code will be more accessible to those who are not familiar
> with contextl and i also felt some early overdesign in the air.
>
> i wanted to have numerous layers like struct-emitting,
> const-char-pointer-as-string, etc... and a backend layer would have
> inherited a number of these. then later on the user could say
> additionally an active and an inactive layer list on top of this that
> is applied after the backend layer was enabled and before the
> generation is started.
>
> and all that while there's only a single cffi backend for verrazano...
> so i decided to simply add an extra first backend parameter to the
> relevant generics and if someone wants to customize the generation
> they can just inherit from a backend and override as necessary.
>
> it doesn't mean that contextl is not useful! but for now i think it
> would be too much in verrazano.

No need to feel sorry. You should of course only use what actually  
contributes to a good solution.

I'd be interested to look at the code once you have it done. It may be  
useful to analyze and see how ContextL could have helped if it had the  
(yet to be found) right features...

Thanks a lot for consideration!


Pascal

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel, Programming Technology Lab
Pleinlaan 2, B-1050 Brussel, Belgium