[Bese-devel] Advancing Parenscript - current and future improvements

[Red Daly]

Henrik Hjelte wrote:
>> It also has some 
>> cool (beta) features, like importing a CLOS class and sending instances 
>> of that class across the net as a JSON object.  
>>     
> FYI: If you use cl-json on the server, there are some updates in the
> darcs repo compared to the release version.
>
>   
Thanks, I'll update my cl-json when I have a chance.  I actually lied, 
though, when I said it generates strict JSON.  It actually generates a 
modified JSON that has support for typing and cross-references (though 
not circular references yet).
>> I would like to come up with a more elegant solution for namespacing 
>> Parenscript code.  Ideally there could be (defpackage) and (in-package) 
>> forms hide the dirty object declarations and such from a Parenscript 
>> user.  Ideas??
>>     
>
> Since the javascript is generated, there is no reason why this:
> myLibrary.myFunction();
>
> couldn't look like this in parenscript:
>
> (in-package :my-library)
> (my-function)
>   
I agree.  Most of the legwork should be done when Parenscript is 
translated to Javascript.
> Anything could be generated, myLibrary_myFunction or
> x345657as_myFunction if you want to avoid all name collisions. Some
> people care how the generated code looks, I don't very much.
>   
It would be ideal to generate code that is readable so that 
non-Parenscript users could use libraries written in Parenscript.  Using 
dot-delimited package qualifiers would be a decent convention.
> What I have thinked about and would very much like is that parenscript
> knew about dependencies and kept track of them. The idea is that you
> shouldn't need to include lots of large javascript files even though you
> only need one or two functions in a library. For the example above, when
> parenscript should generates code for (my-function), it would look in
> the package :my-library to see if there was a function definition for
> my-function there. If not it would look at the imports in my-library.
> Finally we could have "external" javascript definitions, for example the
> core javascript functions or the "window" global scoped functions (or
> objects). If there was no definition reachable for my-function
> parenscript would show a Warning. All typo errors would disappear! You
> could also warn about missing arguments and other things, and perhaps do
> optimizations.
> For those of us that care about performance and bandwidth,
> a great thing could be that you could compile a .js file that included
> exactly all functions you need and no more, in one file. 
>
> In common-lisp code:
> (with-js-environments (core-javascript window-object dojo-library)
>    (generate-all 
>      `((in-package :my-library)
>        (my-function))))
> would return something like:
> "
> function myLibrary_myFunction () {
>   alert("hello");
> };
> myLibrary_MyFunction();
> "
>   
This sounds like a good route to me, though figuring out dependencies 
might be tricky.  Deciding what to consider as part of the package could 
be accomplished through defined semantics.  defun, defconstant, 
defclass, ... and some other forms could define exportable parts of a 
package.  However, making sense of parenscript symbols seems more 
difficult.  For example,

(defpackage cow)
(in-package :cow)

(defun moo () (alert "global moooo")) ; defines function, moo, that is 
registered as a symbol of cow

(moo) ; should

(defun main ()
    (let ((moo (lambda () (alert "correct mooo")))) ; this is a local moo
       ; this should alert "correct mooo"
       ; it should not expand to cow.moo(), but just moo()
       (moo)))

Some level of program analysis seems necessary to determine the scope

I have not thought this through, though.  First I would like to figure out

> This would mean a lot of changes to parenscript. But it shouldn't be
> that difficult should it?
>
> One could also consider to start over again from the start with a new
> project. The parenscript code has it's quirks that I'm not particular
> found of, code such as dwim-join-strings.
>   
Overhauling Parenscript sounds fine.  If we decide to make drastic 
changes, it sounds reasonable to fork the project.  I am not intimately 
familiar with its innards right now, though I have noticed while working 
that the library isn't perfect.   It would be ideal in a fresh 
implementation to expose more of the compiler, since it seems that 
optimization and parenscript interpretation are important facets of this 
packaging system.
>
>   
>> Packaging issues
>>
>> Right now there is no consistent packaging strategy for Javascript code. 
>> In most cases, developers wishing to use the latest and greatest will 
>> download a project, add it to their own source tree, and then follow 
>> special instructions about how to actually include the scripts in their 
>> web sites. This works O.K. most of the time, but it is indeed not the 
>> smartest system.
>>
>> Parenscript should probably take advantage of Lisp's Another System 
>> Definition Facility (ASDF) to define packages. I have not worked out the 
>> details of how to define Parenscript systems yet. 
>>     
> Parenscript and client-side web-applications (the main target for
> javascript) are different from Lisp. I don't think you can just take
> ASDF and put it in parenscript. I think one need to ask what exactly is
> needed for a javascript environment. But something similar to asdf
> syntax-wise would be nice.
>   
What I meant is that parenscript project ASD files will contain a lisp 
source component and a parenscript source component.  The parenscript 
compiler then knows how to locate certain parenscript files within a 
parenscript project.  Say the "dom-util.paren" file in the 
"bobs-paren-utilities-system" package..

(defsystem bobs-paren
    (:components ((:module "lisp-src"
                                  (:file "bobs-paren-macros"))
                             (:module "paren-src" :depends-on ("lisp-src")
                                   (:paren-file "general-util"
                                   (:paren-file "dom-util" :depends-on 
("general-util"))))))
(defsystem reds-paren
    (:components ((:module "paren-src"
                                  (:paren-file "main-site" :depends-on 
("bobs-paren/dom-util"))))))

This isn't refined, but you can see how ASDF could be adequate for 
defining Parenscript systems.

>   
>> Packaging issues are not as important as namespace issues to me, since a 
>> large packaging system doesn't matter if there are name conflicts 
>> everywhere.
>>     
> I agree that packaging issues are of less importance (at least until
> there are any packages to get). But it is not name conflicts that I'm
> afraid of, it is loading and initializing all these files on the client.
> Including dojo is not nice! I need a mapcar function and what do I get,
> 20 files and three seconds load time. Whereas in my dream solution I
> would only get the little mapcar function from somewhere in my server
> side library at compile time, then saved to a little page-specific js
> file.
>   
With the compiler enhancements you talked about, an ASD file could 
delineate general project dependencies, and very specific package 
requirements could be determined.  This way, we can avoid 
over-inclusion.  A less full-blown solution is for ASD descriptions of a 
Parenscript file can include more than package-level dependency 
information (a la :depends-on ("bobs-paren/dom-util")).

> And thank you! clos in javascript is way cool.
>
> I'm interested in contributing to evolving parenscript. This mailing
> list isn't that crowded, so it is a good place to discuss on isn't it?
>
> Best wishes,
> Henrik Hjelte
>
>   
Sounds good. Thanks for your interest,
Red Daly