[slime-devel] Re: Bug in compiler note location resolving

[Helmut Eller]

Christophe Rhodes <csr21 at cam.ac.uk> writes:

> SBCL's behaviour, in sbcl-0.8.12.3, has been altered to return NIL on
> (get-macro-character #\Space).  I know that slime needs to keep
> workarounds for a while for those who are unwilling or unable to
> upgrade, but maybe it could be protected with a test against the
> compiling sbcl's behaviour?  e.g.
>
>   #+sbcl
>   (if (get-macro-character #\Space nil)
>       (defun cmucl-style-get-macro-character ...)
>       (defun cmucl-style-get-macro-character ...))
>
> or something similar?  It's not necessary, but it might be slightly
> more self-documenting.

Hmm, I can't remember the details, but I think there's also some
problem with #\\.  Back then, I added some comments and tests; I think
I was, um, not very excited about SBCL when I wrote it and didn't
commit them.  Do they make any sense to you?


--- swank-source-path-parser.lisp	2004-06-26 17:50:43.000000000 +0200
+++ local-swank-source-path-parser.lisp	2004-06-26 18:04:06.000000000 +0200
@@ -48,19 +48,64 @@
 ;; Just copy CMUCL's implementation, to get identical behavior.  The
 ;; SBCL implementation uses GET-RAW-CMT-ENTRY; we use
 ;; GET-COERCED-CMT-ENTRY, which seems to be what SET-MACRO-CHARACTER
-;; expects. -- Helmut Eller
+;; expects. -- Helmut Eller 
+;;
+;; Well, actually it doesn't matter. See below.  But GET-RAW-CMT-ENTRY
+;; seems to be bootstrap magic and is not available at runtime.
 (defun cmucl-style-get-macro-character (char table)
   (let ((rt (or table sb-impl::*standard-readtable*)))
     (cond ((sb-impl::constituentp char)
 	   (values (sb-impl::get-coerced-cmt-entry char rt) t))
 	  ((sb-impl::terminating-macrop char)
 	   (values (sb-impl::get-coerced-cmt-entry char rt) nil))
-	  (t nil))))
+	  (t 
+	   (values nil nil)))))
 
 #+cmu
 (defun cmucl-style-get-macro-character (char table)
   (get-macro-character char table))
 
+;; Unlike CMUCL, SBCL stores NIL values into the character-macro-table
+;; for constituent (in the CL sense) chars, and uses
+;; get-coerced-cmt-entry to convert those NILs to #'read-token.  In
+;; CMUCL all constituents are also macro-chars.
+;;
+;; CMUCL and SBCL use a somewhat strange encoding for CL's Character
+;; Syntax Types:
+;;
+;;  CL                    Implementation
+;;  ----------------	  --------------
+;;  Constituent           (constituentp x) i.e. (<= +char-attr-constituent+ x)
+;;  Macro Char            (constituentp x) or +char-attr-terminating-macro+ 
+;;  Single Escape         +char-attr-escape+
+;;  Invalid               (constituentp x) with undefined-macro-char as fun
+;;  Multiple Escape       +char-attr-multiple-escape+
+;;  Whitespace            +char-attr-whitespace+
+;;
+;; One effect of this encoding is that invalid chars are not detected
+;; inside tokens and it seems that there's no good way to distinguish
+;; constituents from macro-chars.
+
+(defun dump-readtable (rt)
+  (dotimes (code char-code-limit)
+    (let ((char (code-char code)))
+      (multiple-value-bind (fn terminatingp) (get-macro-character char rt)
+      (format t "~S[~D]: ~12,1T~A ~A~%" 
+	      char code fn terminatingp)))))
+			 
+;; (dump-readtable *readtable*)
+
+(let ((rt (copy-readtable nil)))
+  ;; If #\space is a macro-char, it shouldn't terminate tokens.
+  (assert (or (not (cmucl-style-get-macro-character #\space rt))
+	      (nth-value 1 (cmucl-style-get-macro-character #\space rt))))
+  ;; In SBCL (get-macro-character #\\) returns #'read-token, t.  And
+  ;; (set-macro-character #\\ #'read-token t) confuses #'read-string,
+  ;; because it uses the attributes in the readtable for parsing
+  ;; decisions.  Heck, the entire reader is written in a extremly
+  ;; confusing manner.  No wonder that it is full of bugs.
+  (assert (not (cmucl-style-get-macro-character #\\ rt))))
+
 (defun make-source-recording-readtable (readtable source-map) 
   "Return a source position recording copy of READTABLE.
 The source locations are stored in SOURCE-MAP."