Adding CL:TAGBODY for SERIES support

Fri Feb 10 08:42:16 UTC 2023

Hello everyone,

How’s it going?

What is a good path forward for adding (tagbody …) to ParenScript?
Package “parenscript-tagbody-go” [1] currently provides a beta-version implementation using (defpsmacro …).
It seems like originally, even classes and objects were user-land macro packages outside of lisp implementations.
Nevertheless, integration into package “parenscript” seems to help with usability and discoverability.

The implementation seems to be mostly working.  Jason’s bug report seem to be fixed.
Two more bugs concerning duplicate labels and empty (tagbody) forms were fixed as well.

There is now also a preliminary package “parenscript-series” [2] that already offers
a custom js array scanner and two js array collectors.  One collector pushes the other unshifts.
The push collector was surprisingly difficult to implement, because SERIES macroexpands (push …) in (parenscript:chain arr (parenscript:push elem)) to whatever the current lisp implementation does. On SBCL 2.3.1, for example, it ends up with some SBCL internal form.
The solution is (parenscript:funcall (parenscript:getprop arr (quote push)) elem).

The biggest problems with collectors and scanners were unimplemented CL forms in ParenScript, e.g. (the …) and (make-list …).
It seems to be less work to reimplement the few SERIES operations that use too many of those CL features compared to porting all the missing CL features that the CL SERIES expanders rely on.

With this array scanner and these two array collectors, (series:map-fn …) gets translated to JavaScript correctly, without further intervention.
So the following works, now that (tagbody …) can be pipelined and compiled:

(ps*
  (series-expand:series-expand
    '(collect-js-array-push
       (map-fn T (lambda (x) (* x 2))
          (map-fn T #’1+
             (scan-js-array '(1 2 3))))))
;; => #<JavaScript code>
;; => [4, 6, 8] // when evaluated in JavaScript

Does it seem useful to integrate “parenscript-tagbody-go” into ParenScript itself or is a standalone package better maintenance-wise?
Something like "parenscript-series” seems like it may be better off as a separate package.
Nevertheless, SERIES is a pipelining compiler for pre-order traversal and with ParenScript being its own full compiler, integrating SERIES could be valuable.
Especially when implementing and composing operations from relational algebra having pipelining is a huge help.
Most of the usability mess with SERIES and CL seems to stem from the fact that it was *not* included in the standard and integrated into CL implementations properly [3 (page 55), 4 (page 6)].
SERIES got two honorable mentions in CLtL2 [5,6], but that seems to have been it from a usability standpoint.

What does experience suggest on these two matters?

Cheers,
Andrew

[1] https://dapperdrake.neocities.org/faster-loops-javascript
[2] https://dapperdrake.neocities.org/faster-loops-javascript <https://dapperdrake.neocities.org/faster-loops-javascript>
[3] https://dspace.mit.edu/handle/1721.1/6035
[4] https://dspace.mit.edu/handle/1721.1/6031
[5] http://cltl2.lisp.se/cltl/clm/node347.html#SECTION003400000000000000000
[6] http://cltl2.lisp.se/cltl/clm/node362.html#SECTION003500000000000000000

> On Jan 27, 2023, at 09:00, Andrew Easton <Andrew at easton24.com> wrote:
> 
> Hello everyone, 
> 
> How’s it going?
> 
> There is now a working version of (parenscript:defpsmacro tagbody (&body body) …)),
> See my blog post here: "https://dapperdrake.neocities.org/faster-loops-javascript <https://dapperdrake.neocities.org/faster-loops-javascript>” .
> 
> This adds nestable (tagbody … (tagbody …) …)) forms as a user-land library to parenscript.
> 
> Apologies for missing documentation.  It will be added gradually.  Also, hosting a tarball on
> Neocities.org <http://neocities.org/> is planned in future as well as somehow getting the library into quicklisp. A name
> like parenscript-tagbody-go seems useful.
> 
> This code is still missing the check of *DEFINED-OPERATORS*.  How is that supposed to look?
> 
> 
> 
> Cheers,
> Andrew 
> 
> 
> 
> 
> 
> 
>> On May 29, 2022, at 20:24, Andrew Easton <andrew at easton24.de <mailto:andrew at easton24.de>> wrote:
>> 
>> Hi Philipp,
>> 
>> That sounds like a good plan.
>> 
>> From my current vantage point, this seems like step
>> three.  I just got done with at step one.  Step two
>> is for me to get acquainted with the parenscript
>> codebase.
>> 
>> I will get a feel the existing code base and then we
>> can take the next steps from there.
>> 
>> I already cloned the git repository to my local
>> development machine.  The current commit for branch
>> master seems to be:
>> 
>> commit 1fd720bc4e2bc5ed92064391b730b9d4db35462a (HEAD -> master)
>> | Author: Vladimir Sedach <vas at oneofus.la <mailto:vas at oneofus.la>>
>> | Date:   Wed Jun 17 20:29:19 2020 -0700
>> 
>> 
>> Regarding tail-call optimization in JavaScript:
>> Jason Miller also recommended that.  Unfortunately, I
>> dug up information indicating that it is unsupported
>> in Google's V8 JavaScript implementation, see
>> [stackoverflow.com <http://stackoverflow.com/> (2017)].
>> 
>> Quoting part of my reply to Jason for the benefit of
>> future readers of this specific email:
>> 
>>> [...] This seems to necessitate a (loop (case ...))
>>> based approach, because SERIES may be used for loops
>>> with iteration counts greater than the stack size.
>>> Nevertheless, not all is lost.
>>> 
>>> PARENSCRIPT already compiles
>>> (block nil ((lambda () (return 3)))) as catch/throw
>>> correctly.  Note, the call in the body of the BLOCK.
>>> So at least some dynamic ((lambda () (go ...))) calls
>>> should be compilable; hopefully all of them.  Even if
>>> it only captures 70% of all use cases, that is way
>>> more than zero.
>> 
>> 
>> 
>> Cheers,
>> Andrew
>> 
>> 
>> 
>> [stackoverflow.com <http://stackoverflow.com/> (2017)], Answer by T.J. Crowder:
>> TITLE:
>> ES6 Tail Recursion Optimisation Stack Overflow,
>> URL:
>> https://stackoverflow.com/questions/42788139/es6-tail-recursion-optimisation-stack-overflow <https://stackoverflow.com/questions/42788139/es6-tail-recursion-optimisation-stack-overflow>
>> 
>> 
>> 
>> On Sat, May 21, 2022 at 10:58:57AM +0200, Philipp Marek wrote:
>>> Hi Andrew,
>>> 
>>> first of all -- how about registering on gitlab.common-lisp.net,
>>> so that you can become a developer for [1] and work with a branch
>>> using a Merge Request?
>>> It would be much easier to track your progress (and individual changes)
>>> that way.
>>> 
>>>> I have started to implement TAGBODY for PARENSCRIPT
>>>> [A,B,C].  The general idea is to imitate a jump table
>>>> by looping over a switch-case.  A GO (C-terminology:
>>>> jump) then sets the switch-variable to the next jump
>>>> destination.  The loop subsequently causes the switch
>>>> to branch to the jump target in the switch-variable.
>>>> Leaving the tagbody means leaving the loop.
>>> 
>>> Hmmm, okay.
>>> My first thought would've been to use a function for each
>>> part and just do tail recursion... but it seems that
>>> this isn't really supported in Javascript?!
>>> 
>>> 
>>> 
>>>> There are complications.  Common Lisp allows nested
>>>> tagbody-forms.  Common Lisp allows go-tags to be
>>>> referenced within the lexical scope *and* the dynamic
>>>> extent of a tagbody form.  This means that a LAMBDA
>>>> can close over a go-tag and jump there, see an
>>>> example in [B], of how inconvenient this can become
>>>> for compilation to JavaScript.
>>> 
>>> Yeah... that would be a good reason for simple function
>>> calls and tail recursion.
>>> 
>>> 
>>>> 1. I need a code review of the algorithm.
>>>>   The implementation in [B] seems to be
>>>>   satisfactory.  There are some test cases and
>>>>   examples.  Most there is the most hairy example I
>>>>   could find up to now.  I may have missed crucial
>>>>   details.
>>> 
>>> I'll take a look - but please let's try to get it into
>>> the git repo first, so that any discussions have some
>>> common state to refer to.
>>> 
>>> 
>>>> 2. My understanding of the CL:TAGBODY definition in
>>>>   the CLHS [4] may be wrong.  Which alternate
>>>>   interpretations does anybody here know of?
>>> 
>>> What are your questions, or points of confusion?
>>> 
>>> 
>>> 
>>> Ad 1: https://gitlab.common-lisp.net/parenscript/parenscript
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/parenscript-devel/attachments/20230210/53deb44f/attachment.html>