[cl-interpol-devel] Support for Perl $1 and Makefile-like special variables

[Evgeniy Zhemchugov]

> I'm not sure about things like $1 and so on.  That'd require tight integration with CL-PPCRE, wouldn't it?

I was thinking about something like this:

(defvar *groups*)

(defmacro =~ (scanner string &body body)
  `(let (($_ ,string))
     (multiple-value-bind ($- $+ @- @+) (cl-ppcre:scan ,scanner $_)
       (let ((*groups* (map 'vector
                            (lambda (start end)
                              (subseq $_ start end))
                            @-
                            @+)))
         , at body))))

; Just as in cl-interpol
(defvar *readtable-copy* (copy-readtable))

; Readtable modification for $1, $2, etc
(set-macro-character #\$
  (lambda (stream char)
    (if (digit-char-p (peek-char nil stream nil #\newline t))
      ; Doesn't work well with symbols like $1:a. Also, interns unnecessary
      ; symbols. The alternative is to parse the stream until a terminating
      ; character occur (with some check for package designators), but
      ; how to determine whether a given character is a terminating one?
      (let ((symbol (read stream nil nil t)))
        (if (subtypep (type-of symbol) 'integer)
          `($ ,symbol)
          (car
            (multiple-value-list
              (intern (concatenate 'string "$" (symbol-name symbol)))))))
      (let ((*readtable* *readtable-copy*))
        (read (make-concatenated-stream
                (make-string-input-stream "$")
                stream)
              t
              nil
              t)))))

; Accessor for *groups*
(defun $ (index)
  (when (<= index (length *groups*))
    (aref *groups* (1- index))))

I haven't thought well the expansion of the #\$ macro character yet. Anyway,
it's an userspace code. As for cl-interpol, I want $1 to be expanded into ($ 1)
in interpolated strings as well. I see several possible approaches here. The one
with the most features seems to be an ad-hoc implementation of something similar
to Lisp readtable. The idea is to provide a function via *inner-delimiters*
variable that will do the reading and expansion for cl-interpol in the same way
as functions set via set-macro-character do that for the Lisp reader. If we
redefine the read-form as follows:

(defun read-form ()
  (let* ((start-delimiter (peek-char*))
         (end-delimiter (get-end-delimiter start-delimiter *inner-delimiters*)))
    (if (consp end-delimiter)
      (progn
        (read-char*)
        (funcall (cadr end-delimiter)
                 *stream*
                 start-delimiter
                 (car end-delimiter)))
      (cond ((null end-delimiter)
              (if *optional-delimiters-p*
                  (read-optional-delimited)
                  nil))
            (t
              `(progn
                ,@(progn
                   (read-char*)
                   (let ((*readtable* (copy-readtable*)))
                     ;; temporarily change the readtable
                     (set-syntax-from-char end-delimiter #\))
                     (read-delimited-list end-delimiter *stream* t)))))))))

then I can do what I want to do in this way:

(defun digit-reader (stream start end)
  (loop for d = start then (read-char stream nil nil t)
        while (and d (digit-char-p d))
        collect d into r
        finally (progn
                  (when d (unread-char d stream))
                  (return `($ ,(parse-integer (coerce r 'string)))))))

(setf *inner-delimiters*
      (list
        ; $(A B) should call the function A with argument B
        (list #\( #\) (lambda (stream start end)
                        (read-delimited-list #\) stream)))
        ; ${a} should expand to the value of variable a (it's for cases like
        ; "ab${cd}e")
        (list #\{ #\} (lambda (stream start end)
                        (let ((*readtable* (copy-readtable*)))
                          (set-syntax-from-char #\} #\))
                          (prog1
                            (read stream)
                            (when (char/= (read-char stream) #\})
                              (error "Interpolation error"))))))

(loop for d from 1 to 9
      do (push (list (digit-char d) nil #'digit-reader)
               *inner-delimiters*))

This solution seems to be backward compatible.

If you want to incorporate this or similar machinery in cl-ppcre then we can
create even better support for Perl variables by declaring dynamic variables
(or #\$ character macro expansions) $_, $`, $&, $', $+, $-, @+, @-, etc and
binding them on every call to cl-ppcre:scan if some flag is set. I'm not sure
that Perl approach to regular expressions will work well with Lisp conceptions
though. What do you think of it?