From klaus at mu.dk Wed Oct 11 09:56:17 2006 From: klaus at mu.dk (Klaus Harbo) Date: Wed, 11 Oct 2006 11:56:17 +0200 Subject: [cl-muproc-devel] Re: Bordeaux-Threads usage In-Reply-To: <20df19c75ae21dc5841af379803b3619@mac.com> References: <20df19c75ae21dc5841af379803b3619@mac.com> Message-ID: <452CBFC1.4080308@mu.dk> Greg Pfeil wrote: > Hey, I was about to start using CL-MUPROC, and happily noticed that it's > using the Bordeaux-Threads library I wrote. However, it only seems to be > using it for SBCL and CMUCL. My aim is for it to be an extremely > portable threading layer, so I'm interested in knowing what problems you > may have had with it on on other platforms and any other functionality > you feel would be necessary in such a library (EG, I see you use > PROCESS-ALIVE-P, which Bordeaux-Threads doesn't currently supply). > > Also, if there are any general suggestions about the library, I'd also > be happy to hear those. > > In any case, I'm excited to start using CL-MUPROC. Thanks. > Hi Greg -- I sorry for taking so long to respond -- I'm afraid Thunderbird deemed your mail to be Spam so I didn't see until I was cleaning out folders this morning! cl-muproc started out on Allegro (while we were evaluating that product), then moved to Lispworks. My focus from the start was squarely on functionality, so I paid no attention to implementation independence at all. When cl-muproc was open sourced, I realized that I could help adoption by making porting easier, so I decided simply to isolate all LW-specific code in a separate layer. That way it would be clear for potential porters what was needed to make a port. I believe it was Vladimir Sekissov who introduced bordeaux-threads while working on the the SBCL/CMUCL port. Overall, I feel that cl-muproc portability model is fine as it is, that is that there is a clear separation between platform INdependent and dependent code. If you look at the code, you'll see that the layer isolating the two is really simple, which is good. I'm glad that bordeaux-threads exists -- it makes porting like this much easier -- and wouldn't mind that more ports relied on it. However, I also think it is a good idea for cl-muproc to have its own isolation layer, so that a port does not have to use bordeaux-threads. Wrt. process-alive-p it is used by the supervisor in cl-muproc, which absolutely needs to be able to detect if a process has terminated. Reading muproc-sbcl.lisp and muproc-cmucl.lisp it seems clear that the porting effort of this particular call was not too great, even if it was not part of bordeaux-threads... ;-) However, if your ambition for bordeaux-threads is that it be possible to use the same bordeaux-based MP code on many implementations, then you should perhaps consider adding calls like process-alive-p. cl-muproc's supervisor certainly needs it. Thanks for writing, good luck with cl-muproc! Any feed-back wrt. your experiences using muproc, good or bad, would be appreciated... best regards, -Klaus. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3368 bytes Desc: S/MIME Cryptographic Signature URL: From parnell.flynn at ronin-capital.com Mon Oct 30 21:30:38 2006 From: parnell.flynn at ronin-capital.com (Parnell Flynn) Date: Mon, 30 Oct 2006 15:30:38 -0600 Subject: [cl-muproc-devel] Multiprocessor systems Message-ID: Excuse me if I am being thick here but I have a question about the behavior of cl-muproc on multiprocessor systems in a Lispworks Lisp image. The lispworks documentation states that: "Each Lisp mp:process has a separate native thread, but Lisp code can only run in one thread at a time and a lock is used to enforce this." Which is as I think we can all agree something short of real multithreading. Now am I correct in thinking that with the cl-muproc library my processes will have a separate native thread AND not be subject to the "Lisp code lock" described above? Cheers, Parnell PS - I have just started to play with this library (only on a single proc box) and I really love the API so far! Parnell Flynn * 230 South Lasalle Street * Suite 400 * Chicago, IL 60604 * 312.244.5317 (o) * parnell.flynn at ronin-capital.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1991 bytes Desc: image001.gif URL: From parnell.flynn at ronin-capital.com Mon Oct 30 21:45:10 2006 From: parnell.flynn at ronin-capital.com (Parnell Flynn) Date: Mon, 30 Oct 2006 15:45:10 -0600 Subject: [cl-muproc-devel] Multiprocessor systems Message-ID: Excuse me if I am being thick here but I have a question about the behavior of cl-muproc on multiprocessor systems in a Lispworks Lisp image. The lispworks documentation states that: "Each Lisp mp:process has a separate native thread, but Lisp code can only run in one thread at a time and a lock is used to enforce this." Which is as I think we can all agree something short of real multithreading. Now am I correct in thinking that with the cl-muproc library my processes will have a separate native thread AND not be subject to the "Lisp code lock" described above? Cheers, Parnell PS - I have just started to play with this library (only on a single proc box) and I really love the API so far! -------------- next part -------------- An HTML attachment was scrubbed... URL: From klaus at mu.dk Tue Oct 31 07:58:16 2006 From: klaus at mu.dk (Klaus Harbo) Date: Tue, 31 Oct 2006 08:58:16 +0100 Subject: [cl-muproc-devel] Multiprocessor systems In-Reply-To: References: Message-ID: <45470218.7040806@mu.dk> Parnell Flynn wrote: > Excuse me if I am being thick here but I have a question about the > behavior of cl-muproc on multiprocessor systems in a Lispworks Lisp image. > > The lispworks documentation states that: ?Each Lisp mp:process has a > separate native thread, but Lisp code can only run in one thread at a > time and a lock is used to enforce this.? > > Which is as I think we can all agree something short of real > multithreading. > > Now am I correct in thinking that with the cl-muproc library my > processes will have a separate native thread AND not be subject to the > ?Lisp code lock? described above? > No, unfortunately muproc can't live up to your expectations in this regard: muproc uses the multiprocessing system of the underlying implementation, with whatever strengths and weaknesses it posseses. At Mu we use Lispworks, I and was disappointed to learn that LW5 did not have 'true' lisp-processes-can-actually-run-simultaneously multiprocessing. Perhaps someone on the listen could enlighten us wrt. 'true' multiprocessing in other implementations? > > Cheers, > > Parnell > > PS ? I have just started to play with this library (only on a single > proc box) and I really love the API so far! > Excellent! Do keep us posted wrt. your experiences. I know that generic servers and the supervisor are not documented very well (at all), but I'd encourage you to look at them; at least I've found them very useful indeed. -Klaus. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3368 bytes Desc: S/MIME Cryptographic Signature URL: From rudi at constantly.at Tue Oct 31 08:15:23 2006 From: rudi at constantly.at (Rudi Schlatte) Date: Tue, 31 Oct 2006 09:15:23 +0100 Subject: [cl-muproc-devel] Multiprocessor systems In-Reply-To: <45470218.7040806@mu.dk> References: <45470218.7040806@mu.dk> Message-ID: On Oct31, 2006, at 08:58, Klaus Harbo wrote: > No, unfortunately muproc can't live up to your expectations in this > regard: muproc uses the multiprocessing system of the underlying > implementation, with whatever strengths and weaknesses it > posseses. At Mu we use Lispworks, I and was disappointed to learn > that LW5 did not have 'true' lisp-processes-can-actually-run- > simultaneously multiprocessing. > > Perhaps someone on the listen could enlighten us wrt. 'true' > multiprocessing in other implementations? cmucl uses "green" threads (the OS sees cmucl as one process, and cmucl itself manages the scheduling between different lisp-side threads of execution). sbcl and openmcl use OS-native threads, both of them can run Lisp code simultaneously in multiple threads and, if run on a multi-core system, on multiple processors. Note that sbcl threads are not available on all platforms that sbcl runs on. (All this is to the best of my knowledge, corrections are welcome etc.) Cheers, Rudi From parnell.flynn at ronin-capital.com Tue Oct 31 13:01:12 2006 From: parnell.flynn at ronin-capital.com (Parnell Flynn) Date: Tue, 31 Oct 2006 07:01:12 -0600 Subject: [cl-muproc-devel] Multiprocessor systems In-Reply-To: Message-ID: Scieneer CL is another implementation with support for multiple CPUs/cores. For the moment I need a Lisp implementation that runs on windows so CMUCL and openmcl are out. Corman CL has multithreading but it has some non-ansi issues that make it almost impossible to use with other libraries (and it is only Wintel32). I am really interested in finding out if the SBCL will include full threading in the Win32 port. Thanks so much for the info, I put a question in to the LW support site asking them when they would add true multithreading, no response as of yet. For now I will enjoy a limited version of cl-muproc. Cheers, Parnell Parnell Flynn * 230 South Lasalle Street * Suite 400 * Chicago, IL 60604 * 312.244.5317 (o) * parnell.flynn at ronin-capital.com -----Original Message----- From: Rudi Schlatte [mailto:rudi at constantly.at] Sent: Tuesday, October 31, 2006 2:15 AM To: Klaus Harbo Cc: Parnell Flynn; cl-muproc-devel at common-lisp.net Subject: Re: [cl-muproc-devel] Multiprocessor systems On Oct31, 2006, at 08:58, Klaus Harbo wrote: > No, unfortunately muproc can't live up to your expectations in this > regard: muproc uses the multiprocessing system of the underlying > implementation, with whatever strengths and weaknesses it > posseses. At Mu we use Lispworks, I and was disappointed to learn > that LW5 did not have 'true' lisp-processes-can-actually-run- > simultaneously multiprocessing. > > Perhaps someone on the listen could enlighten us wrt. 'true' > multiprocessing in other implementations? cmucl uses "green" threads (the OS sees cmucl as one process, and cmucl itself manages the scheduling between different lisp-side threads of execution). sbcl and openmcl use OS-native threads, both of them can run Lisp code simultaneously in multiple threads and, if run on a multi-core system, on multiple processors. Note that sbcl threads are not available on all platforms that sbcl runs on. (All this is to the best of my knowledge, corrections are welcome etc.) Cheers, Rudi From rudi at constantly.at Tue Oct 31 13:07:35 2006 From: rudi at constantly.at (Rudi Schlatte) Date: Tue, 31 Oct 2006 14:07:35 +0100 Subject: [cl-muproc-devel] Multiprocessor systems In-Reply-To: References: Message-ID: <353AE261-3BC4-459B-AAD8-FA02FA3A35BE@constantly.at> On Oct31, 2006, at 14:01, Parnell Flynn wrote: > I am really interested in finding out if the SBCL will include full > threading in the Win32 port. I am not aware of anyone working on that feature at the moment, but Nikodemus Siivola recently did a fair amount of polishing on sbcl/ win32, and is available for consulting / implementation work (http:// sb-studio.net/). Cheers, Rudi