[cl-ppcre-devel] Roles of scanner vs. parser vs. lexer?

[Edi Weitz]

[You forgot to Cc the mailing list.]

On Wed, 3 Aug 2005 15:49:58 -0700, Derek Peschel <dpeschel at eskimo.com> wrote:

> CL-PPCRE makes a start, with a grammar and a set of token types that
> the lexer produces.  The problem, as you know, is how to make sure
> the specification is complete while stating it in a way that's not
> tied to a particular implementation.  All the context-sensitive
> tricks (extended mode, backreference vs. octal constants, \Q and \E)
> make the spec harder to check too.

Definitely.  I'm afraid I won't be able to help very much, though.
CL-PPCRE's parser/lexer is very much an ad hoc implementation and it's
the first parser I ever wrote.

> Climacs's syntax highlighters work with parse trees, so CL-PPCRE's
> use of them seemed to make it a good match for Climacs.  I think
> Climacs needs to reconstruct the original text from the parse tree,
> and it parses the text incrementally, and a good syntax module will
> flag errors yet allow further changes to the text.  So I don't know
> if the Climacs parse tree could be ready to pass to CL-PPCRE.  But
> that would be ideal.

Without knowing anything about Climacs' internals: How about using
CL-PPCRE::PARSE-STRING and feeding the result to Climacs?

> Are dangling \Es legal in Perl too?

Yep.

  edi at vmware:~$ perl -le '$a = "\Q*\E\E"; print $a'
  \*

Cheers,
Edi.