[iterate-devel] using keywords in clauses

[Zach]

Denis,

I got to thinking after I sent the last email and realized that my
example was a bit confusing, but you did an excellent job deciphering
it.  To sum up what it seems you are proposing:

Symbols in the iterate package like FOR, WHILE, COLLECT, APPENDING,
will be blessed in such a way that they can be accessed via symbols in
the keyword package.  Writing extensions to Iterate via
defmacro-clause and friends will remain the same, it will take a
symbol, not a keyword, and will not try to institute the same keyword
short cut as the blessed clauses.

Is that a fair description?  Is the root of your desire to do this the
fact that you want to use the symbols FOR, WHILE, COLLECT, etc in the
same package that you want to use Iterate?

As for your points, I will work backwards:

> Most of iterate keywords are in a keyword package already as they are
> used in common lisp loop macro. What kind of collision do you mean?

People don't usually worry about keyword collisions because keywords
act as syntactic markers and do not hold bindings.  I am reminded of a
recent post by Xach regarding binding certain functions to keywords in
your .rc file like...

(defun :go (&optional (thing *))
  "View THING with some appropriate viewer."
  ;; On my Mac laptop, I use "open" instead.
  (sb-ext:run-program "gnome-open" (list (princ-to-string thing))
                      :search t)
  thing)

These are functions that serve to make the Lisp environment a bit more
appealing, not to extend a program's capabilities.  This is a fine
thing to do right up until the point that someone decides to add these
definitions to some utility library.  Once that happens, loading that
library will clobber any definitions that were in place before.  This
is an example of how the keyword package is prone to collisions.  The
collision above is in the space of function definitions, but in
Iterate we have a similar space of clauses.  Currently those clauses
are represented as symbols which afford us all of the protections that
the package system gives us to avoid those kinds of collisions.

Probably the most confusing thing that my last message did was use the
code in the manual dedicated to extending FOR.  This made it seem like
I was proposing a means that we could extend FOR while I was talking
about working within the framework Iterate defines to build a
different kind of clause that happens to be named TEST:FOR.  Let's
say, as a better example, I wanted to define that in my package, that
happens to use some data structure that indexes for 1 instead of 0,
for loops in iterate are to always default to starting index 1 rather
than 0.  One way to do this is to define my own clause, FOR that is
local to the package and has the exact definition as ITERATE:FOR
except that that initial default 0 is turned into a 1.  Now I'm not
saying that I would ever do this, I'm not saying that this is a good
idea, I'm not even saying (now that I see what you are proposing) that
I cannot do this with your proposed change in place, but I am saying
that the way Iterate is currently set up, INDEX-FROM-ONE:FOR would be
on equal syntactic footing as ITERATE:FOR.  I do feel that this is an
extremely clean mechanism that holds a lot of similarity with standard
CL functionality.

And with that, I will walk back my objection from, "it might break
stuff" to "it isn't necessary and it doesn't look consistent".  So I
suppose that it is a matter of taste.  However, as you pointed out,
but I was unaware of, Iterate has defsynonym.  Why isn't your patch
nothing more than a single file that looks like:

(iterate:defsynonym :for iterate:for)
(iterate:defsynonym :collect iterate:collect)
...
(iterate:defsynonym :appending iterate:appending)

Why isn't this just a small separate library called iterate-keyword
that only exports ITERATE?

Note that including this in Iterate distribution would enable usage of
keywords as an alternate syntax for those clauses, drivers, and
collectors that happen to have been blessed by Iterate during its
creation, but it would also break any body's code that decided to
(defmacro-clause :for ...).  I'm not sure that this is worth worrying
about, but from my point of view this is wrong in the same way someone
defining a library that has (defun :go ...) in it is wrong.

One last thing, I would hope that you wouldn't actually attempt to
patch iterate to not export FOR, WHILE, etc as that really would break
code.

Zach


On Tue, Oct 16, 2012 at 1:06 AM, Denis Budyak <budden73 at gmail.com> wrote:
> Hi Zach, list!
>> If you encode FOR and COLLECT as special tokens that Iterate's
>> parser/compiler just understands you break the extensibility.
> Iterate-keywords makes minimal change to iterate. Roughly, it makes
> (defsynonym :any-clause-head any-clause-head), but only for
> standard iterate clauses, not for user defined. As synonyms do not
> work with special clauses like (next-iteration) (it can be considered a
> bug in synonym mechanism), patch to iterate itself was required.
>
> Consequently, it does not break your example.
> %(iterate-keywors:iter (test:for i in-whole-vector #(1 3)) (:collect i))
> (1 3)
>
> %(iterate-keywors:iter (:for i in-whole-vector #(1 3)) (:collect i))
> ...errs...
>
> For is intended to be extensible via second keyword, in-whole-vector
> in our case. If we define it as in manual,
> %(in-package :iterate-keywords)
> %(defmacro-clause (FOR var IN-WHOLE-VECTOR v) ...),
> It will work with iterate-keywords as intended:
> %(in-package:cl-user)
> %(iterate-keywords:iter (:for i in-whole-vector #(1 3)) (:collect i))
> (1 3)
> If we define any new clause, iterate-keywords won't touch it.
> We can define it in :test package and use the symbol
> from that package, that is,
> %(iter (test:our-clause))
> Additionally, we can make
> %(defsynonym :our-clause test:our-clause)
> and thus enable ourselves to use keyword:
> %(iter (:our-clause))
>
>> and even then, it would suffer from name collisions inherent in the keyword package
> Most of iterate keywords are in a keyword package already as they are
> used in common lisp loop macro. What kind of collision do you mean?