[parenscript-devel] ps-symbols (was Re: Conflict with % between Parenscript and CSS-Lite)

[Travis Cross]

Vladimir Sedach wrote:
> The gory details is that there was a patch pushed previously that 
> introduced some changes to the way Parenscript handled symbols, which 
> stripped out package information from macro names. I think this is 
> the third time someone has tried to introduce "Parenscript symbols."

That would have been me (on 2008-04-05).

> There really is nothing wrong (and a lot of things otherwise not 
> possible) with using Common Lisp symbols as-is.

I've given quite a bit of consideration to this issue, and I continue to 
believe that there is something wrong with this, conceptually, as a 
matter of practice, and with regard to our ability to make future design 
improvements to the system.  Perhaps you can convince me otherwise, but 
please consider the problems with this approach.  Here are my concerns:

Conceptually, I see the Parenscript system as a compiler.  I see PS code 
as data to the compiler, not as CL code.  As data, not code, I think 
there should be some basic barriers between PS and the lisp system. 
There are substantial semantic differences between PS/JS symbols and 
lisp symbols, and I think it suboptimal to have to import/use a ton of 
PS/JS symbols into every lisp package where I want to use PS.

As a user of the PS compiler, I would expect that simply calling the PS 
compiler interface with syntactically correct PS code should produce 
correct JS output.  But it doesn't unless I import/use all the possible 
PS specials.  Simple example:

cl-user> (require 'parenscript)
cl-user> (ps:ps (new foo))
"foo;" ;; wrong

That should at least raise an error to the user that things aren't going 
to happen as expected.

While I understand that using CL symbols does permit some interesting 
flexibility, at best it seems like a hack and causes leakage and 
inconsistency in the PS abstractions.  An longer example:

---
(in-package #:cl-user)

  ;; some library unrelated to PS/JS
(defpackage #:foolib (:use #:cl) (:export #:new))

(defpackage #:myjslib (:use #:cl #:ps) (:export #:bar))
(in-package #:myjslib)
(ps (defun bar () (do-some-js-magic)))

(in-package #:cl-user)
(defpackage #:myapp (:use #:cl #:foolib #:ps))
;; symbol conflict, resolve in favor of FOOLIB:NEW
(in-package #:myapp)

(ps (new foo))
  => "foo;" ;; I think this is wrong.  When deciding how to resolve the
            ;; NEW symbol conflict, I'm thinking about lisp, not PS.

(ps (ps:new foo))
  => "new foo;" ;; you might think this is acceptable, but then what
                ;; about...

(ps (myjslib:bar))
  => "bar();" ;; correct perhaps, but has completely different
              ;; semantics than above

(ps (bar))
  => "bar();" ;; following from the way we (didn't) handle the NEW
              ;; symbol, shouldn't this raise an error?  And if we ever
              ;; really wanted to get javascript 'packages' or
              ;; namespaces right, this would almost certainly get in
              ;; our way.
---

I could go on with examples -- there are other cases I feel that violate 
the principle of least surprise, but my basic point is that if we want a 
sane package system for PS we really need to treat it as distinct from 
the CL package system and implement one ourselves.

The reason for this is simple really.  The PS/JS code has a different 
'context' than the code in the lisp system.  The context for CL code is 
defined by the CL package system and lexical scoping rules -- the CL 
compiler.  The context for the JS output of PS, and hence for the PS 
code itself, is defined by how the output JS files are loaded into the 
browser.  If I compile:

(in-package :foo)
(output-ps-to-browser ;; made-up lisp function
   (ps (defvar a))

(in-package :bar)
(output-ps-to-browser
   (ps (let ((a 1))
         (incf a))))

Then the output will be subtly broken.  The variable 'a' should be 
treated by PS as special/global in BAR as well as FOO.  There should be 
a way to communicate to PS that both instances of JS output are going to 
be evaluated together -- that they should be treated in the same 
context.  CL packages are a lousy way to do this.  The CL package 
approach encourages the user (once they run head-on into these surprises 
a few times) to write all of the PS code for their application from a 
single CL package.  This discourages useful ways of emitting JS via PS 
from various places in one's application and libraries.

I believe this is why ps-symbols keep getting introduced into the 
library -- using the CL package system for PS causes subtle and 
surprising behavior.  There are definitely improvements that we can make 
to the ps-symbol approach, but I believe it is a better basis from which 
to work.

Cheers,

-- Travis