From astebakov at yahoo.com Thu Oct 6 02:11:17 2005 From: astebakov at yahoo.com (Andrew Stebakov) Date: Wed, 5 Oct 2005 19:11:17 -0700 (PDT) Subject: [cl-ppcre-devel] Loading of the package takes a long time... Message-ID: <20051006021118.48404.qmail@web52805.mail.yahoo.com> Hi, I just installed the package. The funcionality looks great but what really concerns me is the load time. It takes up to 15 sec to load it when I issue (asdf:oos 'asdf:load-op 'cl-ppcre) command. (I use Clisp on Win2K CPU PIII @ 600MHz, not much I know :). I am wondering if I create a stand-alone binary or use it as a shell script will it load that long every time the script will be executed? Is there a way to shorten the loading time (maybe select not all files? May be there is some "light" version of the package?) Thank you, Andrei __________________________________ Yahoo! Mail - PC Magazine Editors' Choice 2005 http://mail.yahoo.com From dpeschel at eskimo.com Thu Oct 6 06:50:37 2005 From: dpeschel at eskimo.com (Derek Peschel) Date: Wed, 5 Oct 2005 23:50:37 -0700 Subject: [cl-ppcre-devel] Loading of the package takes a long time... In-Reply-To: <20051006021118.48404.qmail@web52805.mail.yahoo.com>; from astebakov@yahoo.com on Wed, Oct 05, 2005 at 07:11:17PM -0700 References: <20051006021118.48404.qmail@web52805.mail.yahoo.com> Message-ID: <20051005235037.A29513@eskimo.com> On Wed, Oct 05, 2005 at 07:11:17PM -0700, Andrew Stebakov wrote: > I am wondering if I create a stand-alone binary or use > it as a shell script will it load that long every time > the script will be executed? Is there a way to shorten > the loading time (maybe select not all files? May be > there is some "light" version of the package?) Hi Andrei, If your LISP prints output during the loading process, analyzing that might tell you what exactly is going on. I'm running SBCL, and for me the loading process is a combination of: - SBCL trying to (require...) a file using the built-in method and failing and giving control to ASDF - ASDF reading the .asd file, resolving dependencies, reading more .asd files, and deciding what systems to load - loading the files that make up the systems - which may be compiled (.fasl format) - or may need compilation -- hopefully the results are saved in .fasl format for the next load -- if you determine that yours aren't being saved, you might want to fix the problem SBCL lets people create LISP images, which should speed up the loading process greatly, but I haven't tried it yet. You should definitely look into it on your LISP. ASDF can load changed source files, but I don't know how that works if you are starting with an image that already contains the systems you are trying to load. That's the only drawback I can think of right now. If you look in the climacs.asd file you will see the parts Climacs is divided into. It's certainly possible to create separate (defsystem...) forms for each part and put them in separate .asd files. But you still have to make sure the configuration runs. The cl-automaton package is the only component I know can be loaded separately. I'm not familiar enough with Climacs to know more. Getting LISP to give you more output is also possible, but I don't know how to do that in the case of ASDF. There's a (time) function but it would make the most sense timing small units of work, which I also don't know how to get ASDF to do. I've only timed the loading of the entire Climacs system. To help you more I would need to see what your system prints. Unfortunately I'm flying across the US on Friday to have eye surgery next week, which will probably cut me off from e-mail for weeks. So I can't spend any more time dealing with your question, but I hope I've helped a little. -- Derek From edi at agharta.de Thu Oct 6 09:09:07 2005 From: edi at agharta.de (Edi Weitz) Date: Thu, 06 Oct 2005 11:09:07 +0200 Subject: [cl-ppcre-devel] Loading of the package takes a long time... In-Reply-To: <20051006021118.48404.qmail@web52805.mail.yahoo.com> (Andrew Stebakov's message of "Wed, 5 Oct 2005 19:11:17 -0700 (PDT)") References: <20051006021118.48404.qmail@web52805.mail.yahoo.com> Message-ID: Hi! On Wed, 5 Oct 2005 19:11:17 -0700 (PDT), Andrew Stebakov wrote: > I just installed the package. The funcionality looks great but what > really concerns me is the load time. It takes up to 15 sec to load > it when I issue (asdf:oos 'asdf:load-op 'cl-ppcre) command. (I use > Clisp on Win2K CPU PIII @ 600MHz, not much I know :). Well, 15 seconds is really long but that might be partly due to your PC being a bit outdated. It is true, though, that loading CL-PPCRE takes some time because at load time some hash tables are created (in util.lisp). This takes longer on CLISP because CHAR-LIMIT is quite high there and because CLISP is sometimes a bit slower than other Lisps. For example, on my laptop CLISP needs about 2.4 seconds to load CL-PPCRE while LispWorks takes less than 0.3 seconds. > I am wondering if I create a stand-alone binary or use it as a shell > script will it load that long every time the script will be > executed? Is there a way to shorten the loading time (maybe select > not all files? May be there is some "light" version of the package?) You can save a custom image (this is implementation-specific) that already includes CL-PPCRE. If you do that you won't have any problems with the load time anymore. (Note that creating a "stand-alone" binary usually also involves creating a custom image.) Does that help? Cheers, Edi. From dpeschel at eskimo.com Thu Oct 6 10:15:21 2005 From: dpeschel at eskimo.com (Derek Peschel) Date: Thu, 6 Oct 2005 03:15:21 -0700 Subject: [cl-ppcre-devel] Loading of the package takes a long time... In-Reply-To: <20051005235037.A29513@eskimo.com>; from dpeschel@eskimo.com on Wed, Oct 05, 2005 at 11:50:37PM -0700 References: <20051006021118.48404.qmail@web52805.mail.yahoo.com> <20051005235037.A29513@eskimo.com> Message-ID: <20051006031521.A6073@eskimo.com> Sorry, I wasn't paying attention, and was thinking you were on the climacs-devel mailing list. So ignore the reference to climacs.asd and the cl-automaton package. And as Ed said, with a simple package, the loading time is not likely to be taken up by ASDF. -- Derek From penguin at ocean.vvo.ru Mon Oct 10 09:43:17 2005 From: penguin at ocean.vvo.ru (Igor Plekhov) Date: Mon, 10 Oct 2005 20:43:17 +1100 Subject: [cl-ppcre-devel] define-parse-tree-synonym Message-ID: <20051010094317.GA4411@ocean.vvo.ru> Small file (aa.lisp) with examples from the cl-ppcre documentation: (defpackage :aa (:use :common-lisp :cl-ppcre)) (in-package :aa) (define-parse-tree-synonym a-z (:char-class (:range #\a #\z) (:range #\a #\z))) (define-parse-tree-synonym a-z* (:greedy-repetition 0 nil a-z)) (defun ascii-char-tester (string) (scan '(:sequence :start-anchor a-z* :end-anchor) string)) If I just load this file than all is ok. If I compile it first and then load aa.x86f (I use CMUCL) than it stops with error: Unknown token A-Z* in parse-tree [Condition of type PPCRE-SYNTAX-ERROR] Restarts: 0: [CONTINUE] Return NIL from load of "aa.x86f". 1: [ABORT ] Return to Top-Level. Debug (type H for help) (CL-PPCRE::CONVERT-AUX #) Why does it not work? -- Registered Linux User #124759 From edi at agharta.de Mon Oct 10 14:16:45 2005 From: edi at agharta.de (Edi Weitz) Date: Mon, 10 Oct 2005 16:16:45 +0200 Subject: [cl-ppcre-devel] define-parse-tree-synonym In-Reply-To: <20051010094317.GA4411@ocean.vvo.ru> (Igor Plekhov's message of "Mon, 10 Oct 2005 20:43:17 +1100") References: <20051010094317.GA4411@ocean.vvo.ru> Message-ID: On Mon, 10 Oct 2005 20:43:17 +1100, Igor Plekhov wrote: > Small file (aa.lisp) with examples from the cl-ppcre documentation: > > (defpackage :aa > (:use :common-lisp :cl-ppcre)) > > (in-package :aa) > > (define-parse-tree-synonym a-z > (:char-class (:range #\a #\z) (:range #\a #\z))) > > (define-parse-tree-synonym a-z* > (:greedy-repetition 0 nil a-z)) > > (defun ascii-char-tester (string) > (scan '(:sequence :start-anchor a-z* :end-anchor) > string)) > > If I just load this file than all is ok. > If I compile it first and then load aa.x86f (I use CMUCL) than it > stops with error: > Unknown token A-Z* in parse-tree > [Condition of type PPCRE-SYNTAX-ERROR] > > Restarts: > 0: [CONTINUE] Return NIL from load of "aa.x86f". > 1: [ABORT ] Return to Top-Level. > > Debug (type H for help) > > (CL-PPCRE::CONVERT-AUX #) > > Why does it not work? I just checked and I see the same error message. However, your example works fine with LispWorks, AllegroCL, SBCL, and CLISP. I suspect this might be a bug in CMUCL - note the EVAL-WHEN that's part of the definition of DEFINE-PARSE-TREE-SYNONYM. It looks to me as if the code isn't executed at load time although it should. Maybe you'll want to ask on the CMUCL mailing list. Cheers, Edi. From penguin at ocean.vvo.ru Tue Oct 11 04:02:23 2005 From: penguin at ocean.vvo.ru (Igor Plekhov) Date: Tue, 11 Oct 2005 15:02:23 +1100 Subject: [cl-ppcre-devel] define-parse-tree-synonym In-Reply-To: References: <20051010094317.GA4411@ocean.vvo.ru> Message-ID: <20051011040223.GM11002@tcp> On Mon, 10 Oct, 2005 at 16:16:45 +0200, Edi Weitz wrote: > > > (define-parse-tree-synonym a-z > > (:char-class (:range #\a #\z) (:range #\a #\z))) > > > > (define-parse-tree-synonym a-z* > > (:greedy-repetition 0 nil a-z)) > > > > (defun ascii-char-tester (string) > > (scan '(:sequence :start-anchor a-z* :end-anchor) > > string)) > > I just checked and I see the same error message. However, your > example works fine with LispWorks, AllegroCL, SBCL, and CLISP. I > suspect this might be a bug in CMUCL - note the EVAL-WHEN that's part > of the definition of DEFINE-PARSE-TREE-SYNONYM. It looks to me as if > the code isn't executed at load time although it should. I have searched Google... It is certainly a bug in CMUCL :-( As a workaround it can be written as: (defparameter a-z '(:char-class (:range #\a #\z) (:range #\a #\z))) (defparameter a-z* `(:greedy-repetition 0 nil ,a-z)) (defun ascii-char-tester (string) (scan `(:sequence :start-anchor ,a-z* :end-anchor) string)) -- Registered Linux User #124759 From edi at agharta.de Tue Oct 11 06:55:52 2005 From: edi at agharta.de (Edi Weitz) Date: Tue, 11 Oct 2005 08:55:52 +0200 Subject: [cl-ppcre-devel] define-parse-tree-synonym In-Reply-To: <20051011040223.GM11002@tcp> (Igor Plekhov's message of "Tue, 11 Oct 2005 15:02:23 +1100") References: <20051010094317.GA4411@ocean.vvo.ru> <20051011040223.GM11002@tcp> Message-ID: On Tue, 11 Oct 2005 15:02:23 +1100, Igor Plekhov wrote: > I have searched Google... It is certainly a bug in CMUCL :-( Too bad. Do you have a URL? Will they fix it? > As a workaround it can be written as: > (defparameter a-z '(:char-class (:range #\a #\z) (:range #\a #\z))) > (defparameter a-z* `(:greedy-repetition 0 nil ,a-z)) > > (defun ascii-char-tester (string) > (scan `(:sequence :start-anchor ,a-z* :end-anchor) > string)) But that way you'll lose the constantness of the regular expression so CL-PPCRE won't be able to use the compiler macros to pre-compile the regular expression. (Just in case you're concerned about performance.) Cheers, Edi. From penguin at ocean.vvo.ru Tue Oct 11 22:54:55 2005 From: penguin at ocean.vvo.ru (Igor Plekhov) Date: Wed, 12 Oct 2005 09:54:55 +1100 Subject: [cl-ppcre-devel] define-parse-tree-synonym In-Reply-To: References: <20051010094317.GA4411@ocean.vvo.ru> <20051011040223.GM11002@tcp> Message-ID: <20051011225455.GN11002@tcp> On Tue, 11 Oct, 2005 at 08:55:52 +0200, Edi Weitz wrote: > > > I have searched Google... It is certainly a bug in CMUCL :-( > > Too bad. Do you have a URL? Will they fix it? Here are similar problems: http://groups.google.com/group/linux.debian.bugs.dist/browse_thread/thread/da096a2268700b57/87d0b29b4fe81ee%2387d0b29b4fe81ee?sa=X&oi=groupsr&start=1&num=3 > > As a workaround it can be written as: > > (defparameter a-z '(:char-class (:range #\a #\z) (:range #\a #\z))) > > (defparameter a-z* `(:greedy-repetition 0 nil ,a-z)) > > > > (defun ascii-char-tester (string) > > (scan `(:sequence :start-anchor ,a-z* :end-anchor) > > string)) > > But that way you'll lose the constantness of the regular expression so > CL-PPCRE won't be able to use the compiler macros to pre-compile the > regular expression. (Just in case you're concerned about > performance.) In that case will CL-PPCRE ever compile regular expression (at evaluation time for example)? If it will, at which time? Will it, if I change defparameter to defconstant? -- Registered Linux User #124759 From edi at agharta.de Wed Oct 12 07:43:54 2005 From: edi at agharta.de (Edi Weitz) Date: Wed, 12 Oct 2005 09:43:54 +0200 Subject: [cl-ppcre-devel] define-parse-tree-synonym In-Reply-To: <20051011225455.GN11002@tcp> (Igor Plekhov's message of "Wed, 12 Oct 2005 09:54:55 +1100") References: <20051010094317.GA4411@ocean.vvo.ru> <20051011040223.GM11002@tcp> <20051011225455.GN11002@tcp> Message-ID: On Wed, 12 Oct 2005 09:54:55 +1100, Igor Plekhov wrote: > Here are similar problems: > http://groups.google.com/group/linux.debian.bugs.dist/browse_thread/thread/da096a2268700b57/87d0b29b4fe81ee%2387d0b29b4fe81ee?sa=X&oi=groupsr&start=1&num=3 Thanks. > In that case will CL-PPCRE ever compile regular expression (at > evaluation time for example)? If it will, at which time? CL-PPCRE will always compile regular expressions. The question is whether it'll do it once or each time SCAN is invoked. It can pre-compile the regular expression if it can figure out that the regular expression in question is constant. See the compiler macros in api.lisp and the definition of DEFINE-COMPILER-MACRO in the CLHS. > Will it, if I change defparameter to defconstant? Maybe. I think that depends on the interplay between backquote and constants in each Lisp implementation (CMUCL, in your case). See CONSTANTP in the CLHS. Cheers, Edi.