[climacs-devel] prolog syntax

[Robert Strandh]

Christophe Rhodes writes:
 > > The obvious solution to this problem is to have grammar rules that
 > > make language tokens (that may contain whitespace) out of lexemes
 > > (that may not). 
 > 
 > It may be obvious to you, but I'm not sure it's obvious to me... how
 > do you get the whitespace into the language tokens if they're not in
 > the lexemes?  You'd have to go back to the buffer contents?

Yes, but generally speaking, you do not need the language tokens at
all in the case of comments and such.  You just display them, which
can be done from the lexemes. 

 > > That is a problem only if one sees a literal character string as a
 > > token.  A better view would be to see the quote character as a lexeme
 > > and the character string as a token produced by grammar rules. 
 > 
 > Possibly.  I'm slightly reluctant to go down this route for a number
 > of reasons, most importantly that of maintainability: at the moment,
 > my Prolog grammar and lexer are more-or-less direct transcriptions of
 > the grammar and token syntax in the Prolog standard (modulo the fact
 > that I've been lazy in the lexer with respect to some of the details).

I can see that you would be reluctant.

 > As you know, this Prolog work doesn't exist in a vacuum; I am hoping
 > to hook this work into an adaptation of Peter Norvig's Prolog
 > implementation from _Paradigms of Artificial Intelligence
 > Programming_.  This means that the lexer and parser can't be simply
 > motivated by getting glyphs on the screen: and in particular I think
 > that keeping whitespace characters of language tokens separate will
 > cause more problems in terms of the multiple uses of this parser than
 > it will solve; consider the difficulty of a line comment (Prolog's
 > introduced by #\%, Common Lisp's by #\;), if whitespace is ignored by
 > the lexer: how will the grammar be able to tell when a line-oriented
 > comment ends?

Line oriented comments can be lexemes no problem, because the newline
character will take care of that. 

Take care, 
-- 
Robert Strandh

---------------------------------------------------------------------
Greenspun's Tenth Rule of Programming: any sufficiently complicated C
or Fortran program contains an ad hoc informally-specified bug-ridden
slow implementation of half of Common Lisp.
---------------------------------------------------------------------