From ch-lisp at bobobeach.com Thu Jan 27 21:41:30 2011 From: ch-lisp at bobobeach.com (Cyrus Harmon) Date: Thu, 27 Jan 2011 13:41:30 -0800 Subject: [editor-hints-devel] named-readtables/SLIME interaction problem Message-ID: <75FE1959-7DFC-4AE0-818B-BAC55DA6119B@bobobeach.com> I ran into a problem with SLIME where I was getting errors the second time I would compile a buffer with contents such as the following: (cl:eval-when (:compile-toplevel :load-toplevel :execute) (ql:quickload 'named-readtables)) (defpackage #:reader-macro-test (:use #:cl)) (in-package #:reader-macro-test) (eval-when (:compile-toplevel :load-toplevel :execute) (named-readtables:defreadtable readtable-test (:merge :standard))) (named-readtables:in-readtable readtable-test) It turns out that the problem has to do with the way SLIME deals with the style-warning that the defreadtable gives about overwriting the existing readtable. The defreadtable form clears out the bindings for the existing reader, then issues the warning. SLIME parses the buffer and picks up that we're in the #:reader-macro-test package and sets *readtable* to be the readtable that we've just clobbered and then in SLIMEs error/warning handling code it further frobs (a copy of) the (now empty-ish) readtable and then attempts to read using that readtable and fails. Tobias rightly points out that SLIME should have better support for named-readtables. It also seems to me that SLIME might want to make greater assurances about the state of the reader in its error handling code. But the simple, expedient fix is: --- named-readtables.lisp 2011-01-27 09:14:38.000000000 -0800 +++ /Users/sly/quicklisp/dists/quicklisp/software/named-readtables-20101006-darcs/named-readtables.lisp 2011-01-27 13:36:53.000000000 -0800 @@ -132,9 +132,9 @@ (cond ((not readtable) (setq readtable (make-readtable ',name))) (t - (setq readtable (%clear-readtable readtable)) (simple-style-warn "Overwriting already existing readtable ~S." - readtable))) + readtable) + (setq readtable (%clear-readtable readtable)))) ,@(loop for option in merge-clauses collect (process-option option 'readtable)) ,@(loop for option in case-clauses Explicit SLIME support for named-readtables would be great, but, in the meantime, it would be nice to see this in the named-readtables source. Thanks, Cyrus From toy.raymond at gmail.com Fri Jan 28 06:07:15 2011 From: toy.raymond at gmail.com (Raymond Toy) Date: Fri, 28 Jan 2011 01:07:15 -0500 Subject: [editor-hints-devel] Patch to support cmucl/unicode Message-ID: <4D425D13.7030803@gmail.com> The patch below adds support for cmucl/unicode. This basically adds an implementation of %make-readtable-iterator, updates %clear-readtable for cmucl (like sbcl), a fix for %get-dispatch-macro-character, a fix for %get-macro-character (like allegro). With these changes (and a small change in named-readtables.lisp that has already been mentioned by Stas Boukarev last month. With these changes cmucl passes all 36 of the tests. Oh, one thing. I noticed that in %clear-readtable, the last sexp is #+common-lisp. CMUCL has the :common-lisp feature, so the table is set twice. Perhaps this is oversight and the reader conditional should be #-(or allegro sbcl cmu)? Ray Change log: * Add implementation of %make-readtable-iterator for cmucl/unicode. * Implement %clear-readtable for cmucl. * Make %get-dispatch-macro-character return NIL where cmucl actually has #'lisp::dispatch-char-error. It seems named-readtables wants this. * Update %get-macro-character for cmucl, which behaves like allegro here. diff -u --recursive ./cruft.lisp /Volumes/share2/src/sourceforge/matlisp/matlisp-cvs/lib-src/named-readtables-0.9/cruft.lisp --- ./cruft.lisp 2009-10-10 04:19:32.000000000 -0400 +++ /Volumes/share2/src/sourceforge/matlisp/matlisp-cvs/lib-src/named-readtables-0.9/cruft.lisp 2011-01-28 00:49:30.000000000 -0500 @@ -235,8 +235,55 @@ subch-fn)))))))) #'grovel-macro-chars))) +;; This really only needed for CMUCL with unicode support. Without +;; unicode, the default implementation is probably fast enough. +#+(and cmu unicode) +(defun %make-readtable-iterator (readtable) + (let ((char-macro-ht (lisp::character-macro-hash-table readtable)) + (dispatch-tables (lisp::dispatch-tables readtable)) + (char-code 0)) + (with-hash-table-iterator (ht-iterator char-macro-ht) + (labels ((grovel-base-chars () + (if (>= char-code lisp::attribute-table-limit) + (grovel-unicode-chars) + (let* ((char (code-char (shiftf char-code (1+ char-code)))) + ;; Need %get-macro-character here, not + ;; get-macro-character because we want NIL + ;; to be returned instead of + ;; #'lisp::read-token. + (reader-fn (%get-macro-character char readtable))) + (if reader-fn + (yield char reader-fn) + (grovel-base-chars))))) + (grovel-unicode-chars () + (multiple-value-bind (more? char reader-fn) + (ht-iterator) + (if (not more?) + (values nil nil nil nil nil) + (yield char reader-fn)))) + (yield (char reader-fn) + (let ((disp-ht)) + (cond + ((setq disp-ht (cdr (assoc char dispatch-tables))) + (let ((disp-fn (get-macro-character char readtable)) + (sub-char-alist)) + (if (< (char-code char) lisp::attribute-table-limit) + (let ((disp (lisp::char-dispatch-table-table disp-ht))) + (dotimes (k lisp::attribute-table-limit) + (let ((f (aref disp k))) + (unless (eq f #'lisp::dispatch-char-error) + (push (cons (code-char k) f) + sub-char-alist))))) + (let ((disp-ht (lisp::char-dispatch-table-hash-table disp-ht))) + (maphash (lambda (k v) + (push (cons k v) sub-char-alist)) + disp-ht))) + (values t char disp-fn t sub-char-alist))) + (t + (values t char reader-fn nil nil)))))) + #'grovel-base-chars)))) -#-(or sbcl clozure allegro) +#-(or sbcl clozure allegro (and cmu unicode)) (eval-when (:compile-toplevel) (let ((*print-pretty* t)) (simple-style-warn @@ -246,7 +293,7 @@ On Unicode-aware implementations this may come with some costs.~@:>" (package-name '#.*package*) (lisp-implementation-type)))) -#-(or sbcl clozure allegro) +#-(or sbcl clozure allegro (and cmu unicode)) (defun %make-readtable-iterator (readtable) (check-type readtable readtable) (let ((char-code 0)) @@ -325,6 +372,11 @@ (do-readtable (char readtable) (set-syntax-from-char char #\A readtable)) (setf (sb-impl::dispatch-tables readtable) nil)) + #+:cmu + (prog1 readtable + (do-readtable (char readtable) + (set-syntax-from-char char #\A readtable)) + (setf (lisp::dispatch-tables readtable) nil)) #+ :allegro (prog1 readtable (do-readtable (char readtable) @@ -343,17 +395,28 @@ "Ensure ANSI behaviour for GET-DISPATCH-MACRO-CHARACTER." #+ :ccl (ignore-errors (get-dispatch-macro-character char subchar rt)) - #+ :common-lisp (get-dispatch-macro-character char subchar rt)) + #+cmu + (let ((f (get-dispatch-macro-character char subchar rt))) + ;; CMUCL returns #'lisp::dispatch-char-error, and named-readtables + ;; wants nil in those cases. + (unless (eq f #'lisp::dispatch-char-error) + f)) + #-(or :ccl :cmu) (get-dispatch-macro-character char subchar rt)) ;;; Allegro stores READ-TOKEN as reader macro function of each -;;; constituent character. +;;; constituent character. CMUCL does the same. (define-cruft %get-macro-character (char rt) "Ensure ANSI behaviour for GET-MACRO-CHARACTER." #+ :allegro (let ((fn (get-macro-character char rt))) (cond ((not fn) nil) ((function= fn #'excl::read-token) nil) (t fn))) - #+ :common-lisp (get-macro-character char rt)) + #+cmu + (let ((fn (get-macro-character char rt))) + (cond ((not fn) nil) + ((function= fn #'lisp::read-token) nil) + (t fn))) + #-(or :allegro cmu) (get-macro-character char rt)) From toy.raymond at gmail.com Fri Jan 28 06:30:17 2011 From: toy.raymond at gmail.com (Raymond Toy) Date: Fri, 28 Jan 2011 01:30:17 -0500 Subject: [editor-hints-devel] Patch to support cmucl/unicode In-Reply-To: <4D425D13.7030803@gmail.com> References: <4D425D13.7030803@gmail.com> Message-ID: <4D426279.8000306@gmail.com> On 1/28/11 1:07 AM, Raymond Toy wrote: > The patch below adds support for cmucl/unicode. This basically adds an > implementation of %make-readtable-iterator, updates %clear-readtable for > cmucl (like sbcl), a fix for %get-dispatch-macro-character, a fix for > %get-macro-character (like allegro). Oh, for cmucl non-unicode, a simple change to the default %make-readtable-iterator makes it work. Just change get-macro-character and get-dispatch-macro-character to %get-macro-character and %get-dispatch-macro-character. I'm not sure, but I think this is the right solution for all lisps to make sure these functions return what named-readtables wants. Ray