From strandh at labri.fr Wed Mar 2 05:51:40 2005 From: strandh at labri.fr (Robert Strandh) Date: Wed, 2 Mar 2005 06:51:40 +0100 Subject: [climacs-devel] Climacs progress Message-ID: <16933.21612.490025.573209@serveur5.labri.fr> Dear mailing list member, The past week, Aleksandar Bakic has been working on making it possible for several buffer implementations to co-exist, so that an adequate buffer implementation can be selected at runtime based on the contents that it is supposed to hold. Aleksandar has been implementing the buffer protocol as a persistent data structure, which essentially means that all previous versions are kept around, making things like undo very easy. I have been working on the syntax framework. The HTML syntax module now uses the Earley parser, and it is plenty fast. This probably means that the CL syntax module can use an Earley parser as well. The main problem right now is figuring out what the protocol for interacting with an Earley parser is supposed to look like. Expect some interesting progress in the near future. Here is an itemized list of what happened since the last progress report: * Climacs now has better error handling. More conditions have been added that are now handled by the command loop or by individual commands, and the command loop no longer handles all error conditions, making it a bit easier to debug new code. * Optimized the code of clone-mark to use constant class names. Also adapted the code to the new buffer organization. * Updated persistent buffer and buffer tests to reflect changes in the buffer organization. (thanks to Aleksandar Bakic) * Improved on performance of incremental lexer for CL syntax by having a single mark and a length in each stack entry. Also, made line comments into single stack entries to cut down on number of stack entries. Eliminated whitespace as a stack entry. * Introduced a delegating buffer which is now what all Climacs buffers should be. The delegating buffer contains a real buffer which can have different possible implementations. Fixed up a lot of code that made assumptions that are no longer valid for a delegating buffer. (thanks to Aleksandar Bakic) * Updated documentation to reflect changes in buffer organization. * Documented buffer-line-number and buffer-column-number (thanks to Aleksandar Bakic) * Improved on HTML syntax. It is not yet usable, but serves as a good example of how to write grammars for syntax module. * Added a section in the documentation on incremental parsing. This section is still incomplete, as I am discovering how to interface with the Early parser. -- Robert Strandh --------------------------------------------------------------------- Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. --------------------------------------------------------------------- From csr21 at cam.ac.uk Mon Mar 7 16:42:24 2005 From: csr21 at cam.ac.uk (Christophe Rhodes) Date: Mon, 07 Mar 2005 16:42:24 +0000 Subject: [climacs-devel] update-syntax and delegating buffers Message-ID: Hi, The change to delegating buffers has had a side effect for me: my UPDATE-SYNTAX method used to be able to obtain the name of the buffer it was updating for, but it can no longer, because the delegating-buffer method for update-syntax calls update-syntax on the implementation buffer, which doesn't mix in the name-mixin class. Is there a reason for this surrounding method? Is it simply to save a generic function call per buffer operation? Should I not be attempting to find the name of a buffer from within UPDATE-SYNTAX? Thanks, Christophe From csr21 at cam.ac.uk Mon Mar 7 17:26:52 2005 From: csr21 at cam.ac.uk (Christophe Rhodes) Date: Mon, 07 Mar 2005 17:26:52 +0000 Subject: [climacs-devel] #\Space and #\Tab Message-ID: Hi, The act of not writing #\Space and #\Tab characters to climacs pane, but instead adjusting the cursor position, has the side-effect of breaking select-and-paste. Would it be reasonable instead to specialize stream-write-char and stream-write-string for climacs panes to produce the right graphical output (by adjusting the cursor position, as currently)? I think that if it were done this way, selection would "just work", as it's implemented over output records, and there are :around methods for output-recording-streams which call stream-add-character-output in the expected way. In summary, something like [untested] (defmethod stream-write-char ((pane climacs-pane) char) (with-slots (space-width tab-width) pane (case char ((#\Space) (stream-increment-cursor-position pane space-width 0)) ((#\Tab) (let ((x (stream-cursor-position pane))) (stream-increment-cursor-position pane (- tab-width (mod x tab-width)) 0))) (t (call-next-method))))) (defmethod stream-write-string ((pane climacs-pane) string) (let ((pos (position-if (lambda (c) (member c '(#\Space #\Tab))) string))) (cond (pos (stream-write-char pane (subseq string 0 pos)) (stream-write-char pane (char string pos)) (stream-write-string pane (subseq string (1+ pos)))) (t (call-next-method))))) and using write-char in climacs' DISPLAY-LINE and friends, should make text selections work as expected. Cheers, Christophe From a_bakic at yahoo.com Mon Mar 7 18:55:09 2005 From: a_bakic at yahoo.com (Aleksandar Bakic) Date: Mon, 7 Mar 2005 10:55:09 -0800 (PST) Subject: [climacs-devel] update-syntax and delegating buffers In-Reply-To: Message-ID: <20050307185509.85076.qmail@web40602.mail.yahoo.com> Hi, Is your update-syntax method in the cvs? I tried not to break things, sorry... I suppose we can move name-mixin into extended-standard-buffer's superclass list. I thought we should perhaps not have extended-*-buffer or climacs-buffer inherit just about anything, i.e., I'd like to hear arguments for and against cases like yours. The rationale for the delegating-buffer is to allow for specifying the buffer implementation at run time. When implementing it, I found that, in order to minimize changes in the existing code, the simplest solution was to delegate the two syntax-related methods, too. However, I had not planned for this, so please suggest how you would like it changed and I'll try to do it asap. Regards, Alex > Hi, > > The change to delegating buffers has had a side effect for me: my > UPDATE-SYNTAX method used to be able to obtain the name of the buffer > it was updating for, but it can no longer, because the > delegating-buffer method for update-syntax calls update-syntax on the > implementation buffer, which doesn't mix in the name-mixin class. > > Is there a reason for this surrounding method? Is it simply to save a > generic function call per buffer operation? Should I not be > attempting to find the name of a buffer from within UPDATE-SYNTAX? > > Thanks, > > Christophe __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From strandh at labri.fr Tue Mar 8 05:24:58 2005 From: strandh at labri.fr (Robert Strandh) Date: Tue, 8 Mar 2005 06:24:58 +0100 Subject: [climacs-devel] New to the list In-Reply-To: <8b7f19a4f4381d759993c6253f89ca72@hampshire.edu> References: <8b7f19a4f4381d759993c6253f89ca72@hampshire.edu> Message-ID: <16941.14122.929222.72650@serveur5.labri.fr> Hello, Sorry, your mail got lost in a large pile of unprocessed ones. David Christiansen writes: > Greetings all, > I've been following the Climacs project for about a month now through > the Web site and I'm interested in helping out. Great! > My Lisp skills are > still somewhat limited at this point, but I'd love to contribute > documentation and test code contributed by others on my Mac. Does > anyone have any pointers as to what I should get started with? Hard to say. The only documentation we have at the moment is the internals documentation, and it might be premature to start a user manual, especially since it resembles Emacs so much. One thing you could do, if you are interested in that kind of stuff, is to go through the chapters on the buffer protocols, and the syntax protocol and make sure all the appropriate Texinfo markup is in there, like @samp, @code, etc, and that there are cross references. > Which > sections of the code are relatively fixed, and which are still > sufficiently in flux that I should avoid them for now? In the internals documentation, what has to do with the buffer protocols and the syntax protocols is pretty much up to date, because these protocols are themselves quite stable. Anything that has to do with specific syntax modules, Earley parsing etc, is in flux. Take care, -- Robert Strandh --------------------------------------------------------------------- Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. --------------------------------------------------------------------- From strandh at labri.fr Tue Mar 8 05:35:10 2005 From: strandh at labri.fr (Robert Strandh) Date: Tue, 8 Mar 2005 06:35:10 +0100 Subject: [climacs-devel] update-syntax and delegating buffers In-Reply-To: References: Message-ID: <16941.14734.28108.552163@serveur5.labri.fr> Hello, Christophe Rhodes writes: > Hi, > > The change to delegating buffers has had a side effect for me: my > UPDATE-SYNTAX method used to be able to obtain the name of the buffer > it was updating for, but it can no longer, because the > delegating-buffer method for update-syntax calls update-syntax on the > implementation buffer, which doesn't mix in the name-mixin class. > > Is there a reason for this surrounding method? Is it simply to save a > generic function call per buffer operation? Should I not be > attempting to find the name of a buffer from within UPDATE-SYNTAX? This exact problem is discussed in the context of CLIM encapsulating streams, section C.1.2. Perhaps the solution to our problem is similar to that suggested there. -- Robert Strandh --------------------------------------------------------------------- Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. --------------------------------------------------------------------- From strandh at labri.fr Tue Mar 8 05:45:15 2005 From: strandh at labri.fr (Robert Strandh) Date: Tue, 8 Mar 2005 06:45:15 +0100 Subject: [climacs-devel] #\Space and #\Tab In-Reply-To: References: Message-ID: <16941.15339.153883.249744@serveur5.labri.fr> Christophe Rhodes writes: > Hi, > > The act of not writing #\Space and #\Tab characters to climacs pane, > but instead adjusting the cursor position, has the side-effect of > breaking select-and-paste. Would it be reasonable instead to > specialize stream-write-char and stream-write-string for climacs panes > to produce the right graphical output (by adjusting the cursor > position, as currently)? I think that if it were done this way, > selection would "just work", as it's implemented over output records, > and there are :around methods for output-recording-streams which call > stream-add-character-output in the expected way. > > In summary, something like [untested] > > (defmethod stream-write-char ((pane climacs-pane) char) > (with-slots (space-width tab-width) pane > (case char > ((#\Space) (stream-increment-cursor-position pane space-width 0)) > ((#\Tab) (let ((x (stream-cursor-position pane))) > (stream-increment-cursor-position > pane (- tab-width (mod x tab-width)) 0))) > (t (call-next-method))))) > (defmethod stream-write-string ((pane climacs-pane) string) > (let ((pos (position-if (lambda (c) (member c '(#\Space #\Tab))) string))) > (cond > (pos > (stream-write-char pane (subseq string 0 pos)) > (stream-write-char pane (char string pos)) > (stream-write-string pane (subseq string (1+ pos)))) > (t (call-next-method))))) > > and using write-char in climacs' DISPLAY-LINE and friends, should make > text selections work as expected. Sure, if such an easy fix makes text selection work, I am in favor. However, the idea of not using write-char on whitespace was to avoid having the output record there, which slows down incremental redisplay. If it is essential that the output record be present, we must probably urgently find someone to speed up McCLIM incremental redisplay. -- Robert Strandh --------------------------------------------------------------------- Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. --------------------------------------------------------------------- From csr21 at cam.ac.uk Tue Mar 8 12:35:44 2005 From: csr21 at cam.ac.uk (Christophe Rhodes) Date: Tue, 08 Mar 2005 12:35:44 +0000 Subject: [climacs-devel] update-syntax and delegating buffers In-Reply-To: <16941.14734.28108.552163@serveur5.labri.fr> (Robert Strandh's message of "Tue, 8 Mar 2005 06:35:10 +0100") References: <16941.14734.28108.552163@serveur5.labri.fr> Message-ID: Robert Strandh writes: > > Is there a reason for this surrounding method? Is it simply to save a > > generic function call per buffer operation? Should I not be > > attempting to find the name of a buffer from within UPDATE-SYNTAX? > > This exact problem is discussed in the context of CLIM encapsulating > streams, section C.1.2. Perhaps the solution to our problem is > similar to that suggested there. It could be, I think, but even then update-syntax is not part of the buffer protocol, so I don't think that it should provide a 'trampoline' method. Cheers, Christophe From csr21 at cam.ac.uk Tue Mar 8 12:38:47 2005 From: csr21 at cam.ac.uk (Christophe Rhodes) Date: Tue, 08 Mar 2005 12:38:47 +0000 Subject: [climacs-devel] update-syntax and delegating buffers In-Reply-To: <20050307185509.85076.qmail@web40602.mail.yahoo.com> (Aleksandar Bakic's message of "Mon, 7 Mar 2005 10:55:09 -0800 (PST)") References: <20050307185509.85076.qmail@web40602.mail.yahoo.com> Message-ID: Aleksandar Bakic writes: > Is your update-syntax method in the cvs? I tried not to break > things, sorry... It's not in CVS, and it's not your fault by any means. > The rationale for the delegating-buffer is to allow for specifying > the buffer implementation at run time. When implementing it, I found > that, in order to minimize changes in the existing code, the > simplest solution was to delegate the two syntax-related methods, > too. However, I had not planned for this, so please suggest how you > would like it changed and I'll try to do it asap. Well, what happens to ordinary climacs code if the two update-syntax methods don't delegate? In theory, they should be using the buffer protocol to interact with their buffer argument, so everything should carry on working transparently... but if it doesn't, it might be worth identifying what the problem is. If nothing too much breaks, then removing those trampolines would be ideal. Thanks, Christophe From a_bakic at yahoo.com Tue Mar 8 16:22:00 2005 From: a_bakic at yahoo.com (Aleksandar Bakic) Date: Tue, 8 Mar 2005 08:22:00 -0800 (PST) Subject: [climacs-devel] update-syntax and delegating buffers In-Reply-To: Message-ID: <20050308162200.60646.qmail@web40603.mail.yahoo.com> > Well, what happens to ordinary climacs code if the two update-syntax > methods don't delegate? In theory, they should be using the buffer > protocol to interact with their buffer argument, so everything should > carry on working transparently... I will see what exactly breaks (in your case, name-mixin is not in the buffer protocol) and will let you know soon. Alex __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From a_bakic at yahoo.com Tue Mar 8 17:45:12 2005 From: a_bakic at yahoo.com (Aleksandar Bakic) Date: Tue, 8 Mar 2005 09:45:12 -0800 (PST) Subject: [climacs-devel] update-syntax and delegating buffers In-Reply-To: <20050308162200.60646.qmail@web40603.mail.yahoo.com> Message-ID: <20050308174512.18601.qmail@web40605.mail.yahoo.com> One thing that breaks is this piece of update-syntax: (setf scan (make-instance 'standard-left-sticky-mark :buffer buffer It assumes that buffer is a standard-buffer instance and finds handy that make-instance for marks is part of the buffer protocol. One way to do the above with a delegating-buffer instance is to clone its low-mark, for example (whose buffer would be the implementation buffer). (If we go this route, we'll probably want to add another optional parameter to clone-mark, the offset.) Alex PS. I'll try to investigate further... --- Aleksandar Bakic wrote: > > Well, what happens to ordinary climacs code if the two update-syntax > > methods don't delegate? In theory, they should be using the buffer > > protocol to interact with their buffer argument, so everything should > > carry on working transparently... > > I will see what exactly breaks (in your case, name-mixin is not in the buffer > protocol) and will let you know soon. > > Alex > > > > > __________________________________ > Celebrate Yahoo!'s 10th Birthday! > Yahoo! Netrospective: 100 Moments of the Web > http://birthday.yahoo.com/netrospective/ > _______________________________________________ > climacs-devel mailing list > climacs-devel at common-lisp.net > http://common-lisp.net/cgi-bin/mailman/listinfo/climacs-devel > __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From a_bakic at yahoo.com Wed Mar 9 00:44:09 2005 From: a_bakic at yahoo.com (Aleksandar Bakic) Date: Tue, 8 Mar 2005 16:44:09 -0800 (PST) Subject: [climacs-devel] update-syntax and delegating buffers In-Reply-To: <20050308174512.18601.qmail@web40605.mail.yahoo.com> Message-ID: <20050309004409.5413.qmail@web40610.mail.yahoo.com> > PS. I'll try to investigate further... All I found were four calls to make-instance of a mark in cl-syntax.lisp, html-syntax.lisp and text-syntax.lisp. When they are replaced by equivalent calls to clone-mark, Climacs works without those two syntax-related methods specialized on delegating-buffer. Alex __________________________________ Celebrate Yahoo!'s 10th Birthday! Yahoo! Netrospective: 100 Moments of the Web http://birthday.yahoo.com/netrospective/ From strandh at labri.fr Thu Mar 10 05:13:43 2005 From: strandh at labri.fr (Robert Strandh) Date: Thu, 10 Mar 2005 06:13:43 +0100 Subject: [climacs-devel] #\Space and #\Tab In-Reply-To: <16941.15339.153883.249744@serveur5.labri.fr> References: <16941.15339.153883.249744@serveur5.labri.fr> Message-ID: <16943.55175.215300.904734@serveur5.labri.fr> Robert Strandh writes: > Christophe Rhodes writes: > > Hi, > > > > The act of not writing #\Space and #\Tab characters to climacs pane, > > but instead adjusting the cursor position, has the side-effect of > > breaking select-and-paste. Would it be reasonable instead to > > specialize stream-write-char and stream-write-string for climacs panes > > to produce the right graphical output (by adjusting the cursor > > position, as currently)? I think that if it were done this way, > > selection would "just work", as it's implemented over output records, > > and there are :around methods for output-recording-streams which call > > stream-add-character-output in the expected way. > > > > In summary, something like [untested] > > > > (defmethod stream-write-char ((pane climacs-pane) char) > > (with-slots (space-width tab-width) pane > > (case char > > ((#\Space) (stream-increment-cursor-position pane space-width 0)) > > ((#\Tab) (let ((x (stream-cursor-position pane))) > > (stream-increment-cursor-position > > pane (- tab-width (mod x tab-width)) 0))) > > (t (call-next-method))))) > > (defmethod stream-write-string ((pane climacs-pane) string) > > (let ((pos (position-if (lambda (c) (member c '(#\Space #\Tab))) string))) > > (cond > > (pos > > (stream-write-char pane (subseq string 0 pos)) > > (stream-write-char pane (char string pos)) > > (stream-write-string pane (subseq string (1+ pos)))) > > (t (call-next-method))))) > > > > and using write-char in climacs' DISPLAY-LINE and friends, should make > > text selections work as expected. > > Sure, if such an easy fix makes text selection work, I am in favor. Hold on a sec... Having select-and-paste implemented over output records seems backward. Only the application-specific code knows how to represent a selection, what objects are to be transmitted to another application, and how these objects should be structured. I am willing to admit that there could be some reasonable default based on output records, but that default may not be adapted to Climacs. -- Robert Strandh --------------------------------------------------------------------- Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. --------------------------------------------------------------------- From strandh at labri.fr Thu Mar 10 08:24:43 2005 From: strandh at labri.fr (Robert Strandh) Date: Thu, 10 Mar 2005 09:24:43 +0100 Subject: [climacs-devel] latest progress Message-ID: <16944.1099.111836.919246@serveur5.labri.fr> Dear mailing list member, The past week, I have concentrated my effort on the HTML syntax module, not because HTML is particularly interesting to the Climacs project, but because it is the syntax module that I currently use to determine what parsing-related functionality can be generalized, abstracted out, and used to implement other syntax modules. We are particularly interested in seeing how these generalizations can be used to implement the Common Lisp syntax module. Aleksandar Bakic continues to work on persistent buffer implementations. He has now started working on a line-oriented persistent buffer implementation that we hope will improve overall performance of Climacs buffer operations. Christophe Rhodes has been very helpful by pointing out potential problems with buffer delegation and with problems in the interaction between Climacs display functions and McCLIM cut-and-paste functionality. Here is an itemized list of what happened the past week: * Made html-syntax display from the sequence of lexemes as opposed to from the buffer itself * Split off html-syntax into a separate package. * Fixes of typos and minor problems in the documentation, and updates to the documentation related to delegating-buffer. (thanks to Aleksandar Bakic) * Performance improvements to the persistent buffer implementation (thanks to Aleksandar Bakic) * Introduced functions to traverse a parse stack encoded in the sequence of parser states that the Earley parser generates. * Modified the html-syntax to display from the parse states as long as a valid parse of the buffer exists, and from the sequence of lexemes when no valid parse exists. -- Robert Strandh --------------------------------------------------------------------- Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. --------------------------------------------------------------------- From a_bakic at yahoo.com Thu Mar 10 21:55:27 2005 From: a_bakic at yahoo.com (Aleksandar Bakic) Date: Thu, 10 Mar 2005 13:55:27 -0800 (PST) Subject: [climacs-devel] update-syntax and delegating buffers In-Reply-To: <20050309004409.5413.qmail@web40610.mail.yahoo.com> Message-ID: <20050310215527.34885.qmail@web40613.mail.yahoo.com> Any comments? I could check these changes in... Alex > All I found were four calls to make-instance of a mark in cl-syntax.lisp, > html-syntax.lisp and text-syntax.lisp. When they are replaced by equivalent > calls to clone-mark, Climacs works without those two syntax-related methods > specialized on delegating-buffer. __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From asf at boinkor.net Fri Mar 11 09:30:46 2005 From: asf at boinkor.net (Andreas Fuchs) Date: Fri, 11 Mar 2005 10:30:46 +0100 Subject: [climacs-devel] fixing the CLIM-CLX dependency in climacs Message-ID: <87oedqlebd.wl%asf@boinkor.net> Hi, at the moment, the Climacs defsystem depends on the CLIM-CLX system. Now that the beagle backend is in the distribution (and with an ASDF-INSTALLable McCLIM), I think it would make sense to make climacs depend on the backend-agnostic MCCLIM system instead. So, here's the patch: Index: climacs.asd =================================================================== RCS file: /project/climacs/cvsroot/climacs/climacs.asd,v retrieving revision 1.22 diff -u -r1.22 climacs.asd --- climacs.asd 7 Mar 2005 06:51:02 -0000 1.22 +++ climacs.asd 11 Mar 2005 09:22:34 -0000 @@ -42,7 +42,7 @@ :defaults *climacs-directory*)) collect `(:file ,(pathname-name p) :pathname ,p)))))) -(climacs-defsystem (:climacs :depends-on (:clim-clx :flexichain)) +(climacs-defsystem (:climacs :depends-on (:mcclim :flexichain)) "Persistent/binseq-package" "Persistent/binseq" "Persistent/obinseq" Cheers, -- Andreas Fuchs, , asf at jabber.at, antifuchs From strandh at labri.fr Thu Mar 17 05:42:01 2005 From: strandh at labri.fr (Robert Strandh) Date: Thu, 17 Mar 2005 06:42:01 +0100 Subject: [climacs-devel] latest progress Message-ID: <16953.6313.673294.425913@serveur5.labri.fr> Dear mailing list member, As usual, this message is meant for those of you who do not follow the climacs-cvs mailing list. This past week, work has been concentrated to two different areas: Aleksandar Bakic has been working on implementing a line-oriented persistent implementation of the buffer protocol. He also continues to clean up the rest of the Climacs code in order to remove dependencies on a particular buffer implementation. I have continued my work on syntax modules. In particular, I have been working on the HTML syntax module, again not in order to make the HTML syntax module directly usable, but in order to use it as a model for how other, more important, syntax modules might be written. I encourage anyone who is interested in a complete HTML syntax module (and who knows a bit more about HTML than I do) to check it out and to add new functionality. Here is an itemized list of what happened the past week: * The HTML module is able to do syntax highlighting of HTML tags (green), of parse errors (red), of the title text (boldface). * Climacs no longer depends on a particular McCLIM backend. (thanks to Andreas Fuchs) * Added the tag to HTML syntax * Line oriented persistent implementation of the buffer protocol using an adaptation of binary sequences by Robert Will. (thanks to Aleksandar Bakic) * Performance improvements and cleanups to the line oriented persistent implementation of the buffer protocol (thanks to Aleksandar Bakic) * Factored out incremental lexer from HTML syntax to the general syntax module, so that it can be useful from other syntax modules. * Documented the incremental lexer protocol. * Improvements to the buffer-test module. (thanks to Aleksandar Bakic) * Moved the code for updating positions of syntactic entities out of the specific syntax modules, so that it is now automatic for all classes that inherit from the parse-tree class. * Fixed a bug in update-syntax for HTML syntax. -- Robert Strandh --------------------------------------------------------------------- Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. --------------------------------------------------------------------- From amoroso at mclink.it Tue Mar 22 16:18:48 2005 From: amoroso at mclink.it (Paolo Amoroso) Date: Tue, 22 Mar 2005 17:18:48 +0100 Subject: [climacs-devel] Unbound CLASS-PRECEDENCE-LIST slot in EXTENDED-BINSEQ2-BUFFER when compiling Climacs Message-ID: <87br9bslg7.fsf@plato.moon.paoloamoroso.it> When I build Climacs from source with: (asdf:operate 'asdf:load-op :climacs) I get the error with backtrace included below. I use CMUCL Snapshot 2005-03 under Slackware Linux 10.0 with the latest Climacs, Flexichains and McCLIM CVS sources. Paolo ------------------------------------------------------------------------------ [...] ; Compiling Load Time Value of (PCL::ENSURE-ACCESSOR 'PCL::READER '(PCL:SLOT-ACCESSOR :GLOBAL CURSOR-X ...) ...): ; Compiling Load Time Value of (PCL::ENSURE-ACCESSOR 'PCL::READER '(PCL:SLOT-ACCESSOR :GLOBAL CURSOR-Y ...) ...): ; Converted DISPLAY-CACHE. ; Compiling DEFUN DISPLAY-CACHE: ; Compiling DEFMETHOD REDISPLAY-PANE-WITH-SYNTAX (CLIMACS-PANE BASIC-SYNTAX T): ; Compiling DEFMETHOD REDISPLAY-PANE (CLIMACS-PANE T): ; Byte Compiling Top-Level Form: ; Compiling DEFMETHOD FULL-REDISPLAY (CLIMACS-PANE): ; Byte Compiling Top-Level Form: ; /home/paolo/src/climacs/pane.x86f written. ; Compilation finished in 0:00:03. ; Loading #P"/home/paolo/src/climacs/pane.x86f". The slot PCL:CLASS-PRECEDENCE-LIST is unbound in the object #. [Condition of type UNBOUND-SLOT] Restarts: 0: [CONTINUE] Return NIL from load of #P"/home/paolo/src/climacs/pane.x86f". 1: [RETRY ] Retry performing # on #. 2: [ACCEPT ] Continue, treating # on # as having been successful. 3: [ABORT ] Return to Top-Level. Debug (type H for help) ("DEFMETHOD SLOT-UNBOUND (T T T)" # # # # ...) Source: Error finding source: Error in function DEBUG::GET-FILE-TOP-LEVEL-FORM: Source file no longer exists: target:pcl/slots.lisp. 0] backtrace 0: ("DEFMETHOD SLOT-UNBOUND (T T T)" # # # # ...) 1: ("DEFMETHOD SLOT-VALUE-USING-CLASS (STD-CLASS STANDARD-OBJECT STANDARD-EFFECTIVE-SLOT-DEFINITION)" # # # # ...) 2: (PCL::SAUT-CLASS-EQ (CLASS #) (PCL::CLASS-EQ #)) 3: (PCL::COMPUTE-APPLICABLE-METHODS-USING-TYPES # ((PCL::CLASS-EQ #))) 4: (PCL::SDFUN-FOR-CACHING # (#)) 5: ((FLET PCL::ADD-CLASS-LIST PCL::MAKE-EMF-CACHE) (#)) 6: (PCL::MAKE-EMF-CACHE # T # NIL ...) 7: (PCL::MAKE-FINAL-CACHING-DFUN # NIL NIL) 8: (PCL::MAKE-FINAL-DFUN-INTERNAL # NIL) 9: (PCL::MAKE-FINAL-DFUN # NIL) 10: (PCL::UPDATE-DFUN # NIL NIL NIL) 11: (PCL::REAL-ADD-METHOD # # NIL) 12: (PCL::REAL-ADD-NAMED-METHOD INITIALIZE-INSTANCE (:AFTER) (CLIMACS-BUFFER) (BUFFER &REST ARGS) ...) 13: (PCL::LOAD-DEFMETHOD-INTERNAL STANDARD-METHOD INITIALIZE-INSTANCE (:AFTER) (CLIMACS-BUFFER) ...) 14: (C::DO-CALL # 26 27 4 ...) 15: (LISP::FOP-FUNCALL-FOR-EFFECT) 16: (LISP::LOAD-GROUP #) 17: (LISP::FASLOAD #) 18: (LISP::INTERNAL-LOAD #P"/home/paolo/src/climacs/pane.x86f" #P"/home/paolo/src/climacs/pane.x86f" :ERROR :BINARY) 19: (LISP::INTERNAL-LOAD #P"/home/paolo/src/climacs/pane.x86f" #P"/home/paolo/src/climacs/pane.x86f" :ERROR NIL) 20: (LOAD #P"/home/paolo/src/climacs/pane.x86f" :VERBOSE NIL :PRINT ...) 21: ((METHOD ASDF:PERFORM NIL (ASDF:LOAD-OP ASDF:CL-SOURCE-FILE)) (#() . #(#)) # # #) 22: ("LAMBDA (G1578 G1579 G1580)" # # # #) 23: ((FLET #:G0 ASDF:OPERATE)) 24: (ASDF:OPERATE ASDF:LOAD-OP :CLIMACS) 25: (ASDF:OPERATE 2 ASDF:LOAD-OP :CLIMACS)[:EXTERNAL] 26: (EXTENSIONS:INTERACTIVE-EVAL (ASDF:OPERATE 'ASDF:LOAD-OP :CLIMACS)) 27: (LISP::%TOP-LEVEL) 28: ((LABELS LISP::RESTART-LISP EXTENSIONS:SAVE-LISP)) 0] ------------------------------------------------------------------------------ -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log From strandh at labri.fr Wed Mar 23 05:37:55 2005 From: strandh at labri.fr (Robert Strandh) Date: Wed, 23 Mar 2005 06:37:55 +0100 Subject: [climacs-devel] Unbound CLASS-PRECEDENCE-LIST slot in EXTENDED-BINSEQ2-BUFFER when compiling Climacs In-Reply-To: <87br9bslg7.fsf@plato.moon.paoloamoroso.it> References: <87br9bslg7.fsf@plato.moon.paoloamoroso.it> Message-ID: <16961.179.155947.926235@serveur5.labri.fr> Paolo Amoroso writes: > When I build Climacs from source with: > > (asdf:operate 'asdf:load-op :climacs) > > I get the error with backtrace included below. Hmm. I can't reproduce this problem with SBCL, which is usually pickier than CMUCL. The error suggests that when pane.x86f is loaded, the superclasses of extended-binseq2-buffer are not all known, but I can't see how that would be the case. -- Robert Strandh --------------------------------------------------------------------- Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. --------------------------------------------------------------------- From a_bakic at yahoo.com Wed Mar 23 07:30:02 2005 From: a_bakic at yahoo.com (Aleksandar Bakic) Date: Tue, 22 Mar 2005 23:30:02 -0800 (PST) Subject: [climacs-devel] Unbound CLASS-PRECEDENCE-LIST slot in EXTENDED-BINSEQ2-BUFFER when compiling Climacs In-Reply-To: <16961.179.155947.926235@serveur5.labri.fr> Message-ID: <20050323073002.35641.qmail@web40601.mail.yahoo.com> > The error suggests that when pane.x86f is loaded, the superclasses of > extended-binseq2-buffer are not all known, but I can't see how that > would be the case. Perhaps file Persistent/persistent-undo.lisp should be listed before pane.lisp in climacs.asd. I can't reproduce the error either with my a-bit-older cmucl, but I am going to upgrade soon. Alex __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From amoroso at mclink.it Wed Mar 23 10:37:19 2005 From: amoroso at mclink.it (Paolo Amoroso) Date: Wed, 23 Mar 2005 11:37:19 +0100 Subject: [climacs-devel] Unbound CLASS-PRECEDENCE-LIST slot in EXTENDED-BINSEQ2-BUFFER when compiling Climacs References: <20050323073002.35641.qmail@web40601.mail.yahoo.com> Message-ID: <874qf2bqcg.fsf@plato.moon.paoloamoroso.it> Aleksandar Bakic writes: > Perhaps file Persistent/persistent-undo.lisp should be listed before pane.lisp > in climacs.asd. I can't reproduce the error either with my a-bit-older cmucl, This fixes the build problem with my setup. Paolo -- Lisp Propulsion Laboratory log - http://www.paoloamoroso.it/log From eric at ping.ca Thu Mar 24 17:09:21 2005 From: eric at ping.ca (Eric Moncrieff) Date: Thu, 24 Mar 2005 12:09:21 -0500 Subject: [climacs-devel] M-q seems broken... Message-ID: <7k4qf1rmwu.fsf@ping.ca> Hi Climacs Developers, Today I decided I'd check out where Climacs was at. I downloaded the latest CVS sources for it and McCLIM, and loaded them up as per the document, under a recent CVS SBCL. I started typing, then decided to reflow a paragraph using M-q. And climacs crashed, bringing up the following backtrace in slime: There is no applicable method for the generic function # when called with arguments (# #). [Condition of type SIMPLE-ERROR] Restarts: 0: [ABORT] Abort handling SLIME request. 1: [DESTROY-THREAD] Destroy this thread (22501) Backtrace: 0: ((SB-PCL::FAST-METHOD NO-APPLICABLE-METHOD (T)) # # # (# #)) 1: ((SB-PCL::FAST-METHOD NO-APPLICABLE-METHOD (T)) # # # (# #)) 2: (CLIMACS-GUI::COM-FILL-PARAGRAPH) 3: ((SB-PCL::FAST-METHOD CLIM:EXECUTE-FRAME-COMMAND :AROUND (CLIMACS-GUI::CLIMACS T)) # #S(SB-PCL::FAST-METHOD-CALL :FUNCTION # :PV-CELL NIL :NEXT-METHOD-CALL NIL :ARG-INFO (2)) # (CLIMACS-GUI::COM-FILL-PARAGRAPH)) 4: ((FLET CLIMACS-GUI::DO-COMMAND) (CLIMACS-GUI::COM-FILL-PARAGRAPH)) 5: (CLIMACS-GUI::CLIMACS-TOP-LEVEL # :COMMAND-PARSER # :COMMAND-UNPARSER # :PARTIAL-COMMAND-PARSER # :PROMPT #) 6: (CLIMACS-GUI::CLIMACS-TOP-LEVEL #) 7: ((SB-PCL::FAST-METHOD CLIM:RUN-FRAME-TOP-LEVEL (CLIM:APPLICATION-FRAME)) # # # #) 8: ((SB-PCL::FAST-METHOD CLIM:RUN-FRAME-TOP-LEVEL :AROUND (CLIM:APPLICATION-FRAME)) # #S(SB-PCL::FAST-METHOD-CALL :FUNCTION # :PV-CELL NIL :NEXT-METHOD-CALL NIL :ARG-INFO (1 . T)) # NIL) 9: (SB-INT:EVAL-IN-LEXENV (CLIMACS-GUI::CLIMACS) #S(SB-KERNEL:LEXENV :FUNS NIL :VARS NIL :BLOCKS NIL :TAGS NIL :TYPE-RESTRICTIONS NIL :LAMBDA NIL :CLEANUP NIL :HANDLED-CONDITIONS NIL :DISABLED-PACKAGE-LOCKS NIL :POLICY ((COMPILATION-SPEED . 1) (DEBUG . 1) (INHIBIT-WARNINGS . 1) (SAFETY . 1) (SPACE . 1) (SPEED . 1) (SB-C::STACK-ALLOCATE-DYNAMIC-EXTENT . 3)))) 10: (SWANK::EVAL-REGION "(climacs-gui::climacs) " T) 11: ((LAMBDA NIL)) 12: ((SB-PCL::FAST-METHOD SWANK-BACKEND:CALL-WITH-SYNTAX-HOOKS (T)) # # #) 13: (SWANK:LISTENER-EVAL "(climacs-gui::climacs) ") 14: (SB-INT:EVAL-IN-LEXENV (SWANK:LISTENER-EVAL "(climacs-gui::climacs) ") #S(SB-KERNEL:LEXENV :FUNS NIL :VARS NIL :BLOCKS NIL :TAGS NIL :TYPE-RESTRICTIONS NIL :LAMBDA NIL :CLEANUP NIL :HANDLED-CONDITIONS NIL :DISABLED-PACKAGE-LOCKS NIL :POLICY ((COMPILATION-SPEED . 1) (DEBUG . 1) (INHIBIT-WARNINGS . 1) (SAFETY . 1) (SPACE . 1) (SPEED . 1) (SB-C::STACK-ALLOCATE-DYNAMIC-EXTENT . 3)))) 15: ((LAMBDA NIL)) 16: ((SB-PCL::FAST-METHOD SWANK-BACKEND:CALL-WITH-DEBUGGER-HOOK (T T)) # # # #) 17: ((LAMBDA NIL)) 18: (SWANK::CALL-WITH-REDIRECTED-IO # #) 19: (SWANK::HANDLE-REQUEST #) 20: ((LAMBDA NIL)) 21: ((LAMBDA NIL)) 22: (SWANK::CALL-WITH-REDIRECTED-IO # #) 23: (SWANK::REPL-LOOP #) 24: ((LAMBDA NIL)) 25: ("foreign function: call_into_lisp") 26: ("foreign function: funcall0") Perhaps this has something to do with the recent buffer implementation changes? Procedure to reproduce: (climacs-gui::climacs) ;; in new climacs window foo bar M-q Let me know if there's any more information you need. Thanks! Eric ** Eric Moncrieff eric at ping.ca 1. Creativity and innovation always builds on the past. 2. The past always tries to control the creativity that builds on it. 2. Free societies enable the future by limiting the power of the past. 4. Ours is less and less a free society. - Lawrence Lessig From a_bakic at yahoo.com Fri Mar 25 06:57:32 2005 From: a_bakic at yahoo.com (Aleksandar Bakic) Date: Thu, 24 Mar 2005 22:57:32 -0800 (PST) Subject: [climacs-devel] M-q seems broken... In-Reply-To: <7k4qf1rmwu.fsf@ping.ca> Message-ID: <20050325065733.77151.qmail@web40627.mail.yahoo.com> This method only exists for text-syntax. Try setting syntax to Text interactively first. (The bug should be fixed.) Alex > There is no applicable method for the generic function > # > when called with arguments > (# > #). > [Condition of type SIMPLE-ERROR] __________________________________ Do you Yahoo!? Yahoo! Small Business - Try our new resources site! http://smallbusiness.yahoo.com/resources/ From csr21 at cam.ac.uk Sun Mar 27 16:12:38 2005 From: csr21 at cam.ac.uk (Christophe Rhodes) Date: Sun, 27 Mar 2005 17:12:38 +0100 Subject: [climacs-devel] prolog syntax Message-ID: Hi, I've added the beginnings of a syntax for ISO Prolog. The commit message lists a number of problems I currently have, but I think I should probably air them here. The biggest, I think, is in dealing with comments, though in fact it's not clear that the problem is restricted to that. I believe that there is an assumption in the display code that lexemes contain no whitespace, and in particular contain no #\Newline characters. This is not really true in Prolog's case; the obvious case is the bracketed /* */ comment, but quoted tokens likewise can contain whitespace. (These cases are analogous, I think, to Common Lisp's #| |# comment and |Symbol With Spaces| -- one might argue over whether the comment is one lexeme or several -- or none -- but I think it's clear that |Symbol With Spaces| is exactly one token). The incremental lexer that I, erm, cut'n'pasted from html-syntax.lisp doesn't cut it for Prolog, for related reasons: the damaged region from deleting a quoting character is extensive. I think I can fix this -- it's similar to a problem I've already dealt with in my Tabcode parser. For a variety of reasons, I think, the incremental redisplay stuff doesn't work. I'm not convinced I understand why. :-/ I encourage interested parties to have a play with Set Syntax SPC Prolog, and see if they can characterize bugs, problems. (There are a number of incompletenesses in the Prolog parser itself that I'm aware of: floating point numbers, escaped quotes in quoted tokens, underscores in tokens, [A,B] list syntax (embarrassingly), and the like. It is relatively capable, though: see , and for eye candy...) Cheers, Christophe From strandh at labri.fr Mon Mar 28 05:15:26 2005 From: strandh at labri.fr (Robert Strandh) Date: Mon, 28 Mar 2005 07:15:26 +0200 Subject: [climacs-devel] prolog syntax In-Reply-To: References: Message-ID: <16967.37614.659052.767418@serveur5.labri.fr> Hello, I am going to keep this short since my computer is currently vulnerable to crackers (I am running Knoppix, because my RedHat was cracked the other day, and I have not gotten around to reinstalling RedHat yet). Christophe Rhodes writes: > > I've added the beginnings of a syntax for ISO Prolog. The commit > message lists a number of problems I currently have, but I think I > should probably air them here. > > The biggest, I think, is in dealing with comments, though in fact it's > not clear that the problem is restricted to that. I believe that > there is an assumption in the display code that lexemes contain no > whitespace, and in particular contain no #\Newline characters. I am not sure that is an assumption, but it is certainly the case for the existing syntaxes. However, one can definitely declare that to be the case. It is a matter of mapping lexemes to tokens in the programming language. > This > is not really true in Prolog's case; the obvious case is the bracketed > /* */ comment, but quoted tokens likewise can contain whitespace. > (These cases are analogous, I think, to Common Lisp's #| |# comment > and |Symbol With Spaces| -- one might argue over whether the comment > is one lexeme or several -- or none -- but I think it's clear that > |Symbol With Spaces| is exactly one token). The obvious solution to this problem is to have grammar rules that make language tokens (that may contain whitespace) out of lexemes (that may not). > The incremental lexer that I, erm, cut'n'pasted from html-syntax.lisp > doesn't cut it for Prolog, for related reasons: the damaged region > from deleting a quoting character is extensive. I think I can fix > this -- it's similar to a problem I've already dealt with in my > Tabcode parser. That is a problem only if one sees a literal character string as a token. A better view would be to see the quote character as a lexeme and the character string as a token produced by grammar rules. > For a variety of reasons, I think, the incremental redisplay stuff > doesn't work. I'm not convinced I understand why. :-/ I encourage > interested parties to have a play with Set Syntax SPC Prolog, and see > if they can characterize bugs, problems. Are you talking about McCLIM here? There are known problems in McCLIM incremental redisplay. For one thing, it displays each output record twice, once with a clipping region and once without. > (There are a number of incompletenesses in the Prolog parser itself > that I'm aware of: floating point numbers, escaped quotes in quoted > tokens, underscores in tokens, [A,B] list syntax (embarrassingly), and > the like. It is relatively capable, though: see > , > and > for eye > candy...) Nice! -- Robert Strandh --------------------------------------------------------------------- Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. --------------------------------------------------------------------- From csr21 at cam.ac.uk Mon Mar 28 16:18:45 2005 From: csr21 at cam.ac.uk (Christophe Rhodes) Date: Mon, 28 Mar 2005 17:18:45 +0100 Subject: [climacs-devel] prolog syntax In-Reply-To: <16967.37614.659052.767418@serveur5.labri.fr> (Robert Strandh's message of "Mon, 28 Mar 2005 07:15:26 +0200") References: <16967.37614.659052.767418@serveur5.labri.fr> Message-ID: Robert Strandh writes: > > This is not really true in Prolog's case; the obvious case is the > > bracketed /* */ comment, but quoted tokens likewise can contain > > whitespace. (These cases are analogous, I think, to Common > > Lisp's #| |# comment and |Symbol With Spaces| -- one might argue > > over whether the comment is one lexeme or several -- or none -- > > but I think it's clear that > > |Symbol With Spaces| is exactly one token). > > The obvious solution to this problem is to have grammar rules that > make language tokens (that may contain whitespace) out of lexemes > (that may not). It may be obvious to you, but I'm not sure it's obvious to me... how do you get the whitespace into the language tokens if they're not in the lexemes? You'd have to go back to the buffer contents? > > The incremental lexer that I, erm, cut'n'pasted from html-syntax.lisp > > doesn't cut it for Prolog, for related reasons: the damaged region > > from deleting a quoting character is extensive. I think I can fix > > this -- it's similar to a problem I've already dealt with in my > > Tabcode parser. > > That is a problem only if one sees a literal character string as a > token. A better view would be to see the quote character as a lexeme > and the character string as a token produced by grammar rules. Possibly. I'm slightly reluctant to go down this route for a number of reasons, most importantly that of maintainability: at the moment, my Prolog grammar and lexer are more-or-less direct transcriptions of the grammar and token syntax in the Prolog standard (modulo the fact that I've been lazy in the lexer with respect to some of the details). As you know, this Prolog work doesn't exist in a vacuum; I am hoping to hook this work into an adaptation of Peter Norvig's Prolog implementation from _Paradigms of Artificial Intelligence Programming_. This means that the lexer and parser can't be simply motivated by getting glyphs on the screen: and in particular I think that keeping whitespace characters of language tokens separate will cause more problems in terms of the multiple uses of this parser than it will solve; consider the difficulty of a line comment (Prolog's introduced by #\%, Common Lisp's by #\;), if whitespace is ignored by the lexer: how will the grammar be able to tell when a line-oriented comment ends? In any case, if it is the case that there is no absolutely hardwired assumption that whitespace cannot be part of tokens, I shall attempt to see what temporary problems can be fixed. > > For a variety of reasons, I think, the incremental redisplay stuff > > doesn't work. I'm not convinced I understand why. :-/ I encourage > > interested parties to have a play with Set Syntax SPC Prolog, and see > > if they can characterize bugs, problems. > > Are you talking about McCLIM here? There are known problems in McCLIM > incremental redisplay. For one thing, it displays each output record > twice, once with a clipping region and once without. Well, I was getting problems that seemed to be more climacs-related; errors relating to *CURSOR-POSITIONS* array access in HANDLE-WHITESPACE: something in the display code wanted to index off the end of that array. I couldn't reproduce this in HTML syntax, though, and nor could I reproduce in HTML syntax the observation that, with all the updating-output enabled, some of my tokens disappeared at various points. :-/ Also, I believe that one of the problems I saw was a result of one of my lexemes starting in the displayed window and ending off it, but I haven't investigated further. Cheers, Christophe From strandh at labri.fr Mon Mar 28 16:55:00 2005 From: strandh at labri.fr (Robert Strandh) Date: Mon, 28 Mar 2005 18:55:00 +0200 Subject: [climacs-devel] prolog syntax In-Reply-To: References: <16967.37614.659052.767418@serveur5.labri.fr> Message-ID: <16968.14052.414745.139503@serveur5.labri.fr> Christophe Rhodes writes: > > The obvious solution to this problem is to have grammar rules that > > make language tokens (that may contain whitespace) out of lexemes > > (that may not). > > It may be obvious to you, but I'm not sure it's obvious to me... how > do you get the whitespace into the language tokens if they're not in > the lexemes? You'd have to go back to the buffer contents? Yes, but generally speaking, you do not need the language tokens at all in the case of comments and such. You just display them, which can be done from the lexemes. > > That is a problem only if one sees a literal character string as a > > token. A better view would be to see the quote character as a lexeme > > and the character string as a token produced by grammar rules. > > Possibly. I'm slightly reluctant to go down this route for a number > of reasons, most importantly that of maintainability: at the moment, > my Prolog grammar and lexer are more-or-less direct transcriptions of > the grammar and token syntax in the Prolog standard (modulo the fact > that I've been lazy in the lexer with respect to some of the details). I can see that you would be reluctant. > As you know, this Prolog work doesn't exist in a vacuum; I am hoping > to hook this work into an adaptation of Peter Norvig's Prolog > implementation from _Paradigms of Artificial Intelligence > Programming_. This means that the lexer and parser can't be simply > motivated by getting glyphs on the screen: and in particular I think > that keeping whitespace characters of language tokens separate will > cause more problems in terms of the multiple uses of this parser than > it will solve; consider the difficulty of a line comment (Prolog's > introduced by #\%, Common Lisp's by #\;), if whitespace is ignored by > the lexer: how will the grammar be able to tell when a line-oriented > comment ends? Line oriented comments can be lexemes no problem, because the newline character will take care of that. Take care, -- Robert Strandh --------------------------------------------------------------------- Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. --------------------------------------------------------------------- From strandh at labri.fr Tue Mar 29 07:02:02 2005 From: strandh at labri.fr (Robert Strandh) Date: Tue, 29 Mar 2005 09:02:02 +0200 Subject: [climacs-devel] prolog syntax In-Reply-To: References: <16967.37614.659052.767418@serveur5.labri.fr> Message-ID: <16968.64874.363470.554751@serveur5.labri.fr> Christophe Rhodes writes: > > Possibly. I'm slightly reluctant to go down this route for a number > of reasons, most importantly that of maintainability: at the moment, > my Prolog grammar and lexer are more-or-less direct transcriptions of > the grammar and token syntax in the Prolog standard (modulo the fact > that I've been lazy in the lexer with respect to some of the details). As I said before, I can understand why you are reluctant to modifying the grammar with respect to that of the Prolog standard. However, it was never a goal for the syntax modules of Climacs to be able to import grammars from standards documents without modification, and I am not sure it will be possible in general. The best one can hope for is some minor tweaking of the grammar; in your case, it would be to write a grammar rule for a token which would normally be a lexeme. I think what it boils down to is whether you want to write an extremely complicated lexer that stops reading at the end of the window, or whether you want to add a rule to your grammar. In addition, the complicated lexer will likely be slow compared to the extra grammar rule. > As you know, this Prolog work doesn't exist in a vacuum; I am hoping > to hook this work into an adaptation of Peter Norvig's Prolog > implementation from _Paradigms of Artificial Intelligence > Programming_. This means that the lexer and parser can't be simply > motivated by getting glyphs on the screen: and in particular I think > that keeping whitespace characters of language tokens separate will > cause more problems in terms of the multiple uses of this parser than > it will solve; consider the difficulty of a line comment (Prolog's > introduced by #\%, Common Lisp's by #\;), if whitespace is ignored by > the lexer: how will the grammar be able to tell when a line-oriented > comment ends? Again, it was never an objective of the syntax modules to be able to serve as compiler front-ends. Some minor adaptations are to be expected for that to be possible. > In any case, if it is the case that there is no absolutely hardwired > assumption that whitespace cannot be part of tokens, I shall attempt > to see what temporary problems can be fixed. I think the only assumption that is hardwired at the moment is that a lexeme does not straddle the top and bot marks of the window. Take care, -- Robert Strandh --------------------------------------------------------------------- Greenspun's Tenth Rule of Programming: any sufficiently complicated C or Fortran program contains an ad hoc informally-specified bug-ridden slow implementation of half of Common Lisp. ---------------------------------------------------------------------