[slime-devel] [investigation] Font lock, reader macros and SLIME repl

[Tobias C Rittweiler]

In article <87pqx0kmga.fsf at desktop-ng.home.sw4me.com>,
 Anton Kovalenko <anton at sw4me.com> wrote:

> Hello SLIME developers,
> 
> Both in the lisp-mode and in slime REPL, there is currently one major
> thing that Emacs isn't getting right: constructs using reader macros
> like #P"/home/" and #2A((1 0) (0 1)) are regarded as two sexps instead
> of one. E.g. command for wrapping a sexp may turn #P"/home/" into
> (#P)"/home/", etc. It gets even more awful with paredit-mode (that I
> prefer to use), but the problem is noticeable even without it. 
> 
> I have investigated one possible solution to it, and here are the
> results.
> 
> First, I decided to limit the scope of solution to some predefined
> "shape" of reader macro: for starters, when #\# is used as dispatch
> macro character, and its reader function calls READ to get the sexp
> following #[numarg]<char>. This is the case with all standard
> reader-macros and many implementation-specific and user-defined ones.
> Furthermore, I discarded the possibility to consult SWANK for the
> buffer's *readtable* (SWANK doesn't *know* it, anyway).
> 
> To get right behavior for #P, #2A and the like, the most appropriate
> syntax for all macro-characters seems to be `expression prefix': we want
> Emacs to regard the whole #P thing as it would regard a quote, backquote
> or comma.
> 
> Let's use the feature of font-lock mode in modern emacsen: syntactic
> keywords (in a nutshell, it's when a code used for text-coloring is
> [ab]used to add syntax-table properties to some text fragments).
> `parse-sexp-lookup-properties' set to true causes the sexp machinery
> to look into syntax-table properties of individual characters.
> 
> Let's add a syntactic keyword to lisp-mode:
> 
> (defun aak:add-lisp-reader-macros-syntactic-keyword ()
>   "Register # numarg macro-character as a font lock syntactic
> keyword, turning it into expression prefix."
>   (set (make-local-variable 'parse-sexp-lookup-properties) t)
>   (set (make-local-variable 'font-lock-syntactic-keywords)
>        '(("\\(\\W\\|$\\)\\#\\([0-9]*[A-Za-z]\\)" (2 "'")))))
> 
> (add-hook 'lisp-mode-hook 'aak:add-lisp-reader-macros-syntactic-keyword)

Yes, I've been using

(defun enable-cl-syntax ()
  (make-local-variable 'font-lock-syntactic-keywords)
  (set (make-local-variable 'parse-sexp-lookup-properties) t)
  (let ((regexp (regexp-opt '("#*" "#." "#=" "#A" "#C" "#P" "#S"))))
    (pushnew `(,regexp 0 "'") font-lock-syntactic-keywords :test 
#'equal)))


(add-hook 'lisp-mode-hook #'enable-cl-syntax)

for a long time now.

> With the hook above, any lisp-mode buffer with font-lock mode enabled
> gets good enough navigation/wrapping/unwrapping for previously
> misinterpreted items.
> 
> I decided to leave macro-characters handled by this regexp to [A-Za-z],
> because normal behavior of reading the following sexp can't really be
> expected from other characters: readtable additions along the lines
> of #"something" and #{something} are also popular, and we can't get
> _them_ right without knowing a concrete syntaxt they expect. Another
> decision that has its merits is to limit it further to _standard_
> macro-characters only.   

There could be a SWANK:DECLAIM-SYNTAX along SWANK:DECLAIM-INDENTATION
which people could put into their CL source files.

  -T.