[iterate-devel] PATCH: support (finding (values ...))

[Hoehle, Joerg-Cyril]

Hi,
Max Mikhanosha suggests a multiple value extension
to FINDING... MAXIMIZING &optional INTO:
(finding (values x y z) maximizing (complex x y z))

I have good and bad news about that.
Bad news first.

I plan to reject this patch, based on the following critique of the
patch:
 - Generality: This syntax (VALUES ...) pattern based extension
   does not cover functions returning multiple values
   e.g. (finding (floor x y) maximizing (foo x y))
 - Arbitrary implementation limit to 9 *result-vars*
   (MULTIPLE-VALUE-LIMIT is the only one that would make sense,
    but it's not practical)
 - A lot more complexity added to the source for just one clause.
 - Unclear yet whether this syntax extension makes sense for other
clauses.
 - &optional INTO is not integrated. What would it look like?
   INTO (values a b c) INTO ((values a b c) max) INTO ((a b c) max)
 - Slight bug: clause as expression must define result and
   constantly yield either all, or only primary value, e.g.
   (for (values running-x running-y) = (finding ...))
   [if that were the only thing, I'd fix it myself before applying the
patch]

Now, the good news.

Iterate claims to be an extensible iteration facility. Thus I wonder
why many contributors in recent years have spent their time
understanding and
modifying the low-level implementation, instead of using Iterate's
documented
extension mechanisms?

Why not use the following definition?

(defmacro-clause (find-values expr maximizing max-expr &optional into
vars)
  "Please don't forget the documentation string here"
  ;; Not a good design: number of values differs whether or not INTO is
used
  (unless (or vars (eq 'values (if (consp expr) (first expr))))
    (error "Expression must start with VALUES: ~S" expr))
  (let ((temps (or vars (mapcar #'(lambda (s) (gensym "VALUE")) (rest
expr))))
	;;#L(gensym (if (symbolp !1) (symbol-name !1) "VALUE"))
	(max (gensym "MAX")) (temp (gensym "NEW")))
    `(progn
      (with (,max .,temps))
      (let ((,temp ,max-expr))
        (when (or (first-time-p) (> ,temp ,max))
	  (setq ,max ,temp)
	  ;;(multiple-value-setq ,temps ,expr)
	  (dsetq (values .,temps) ,expr)))
      ,.(if vars () (list `(finally (return (values .,temps))))) )))

This code is much shorter than the proposed patch.  If it's not of
general usage, it can be embedded in your application code.  Your
application does not depend on the Iterate maintainers to accept your
patch for you to make successful use of Iterate.
Doesn't that sound great?

Test cases
;(iter(find-values (values x y z) maximizing (complex x y z)))
;(iter(find-values (values x y z) maximizing (complex x y z) into (a b
c)))
;(iter (find-values (floor x y) maximizing (rem x y))) ; error
;(iter (find-values (floor x y) maximizing (rem x y) into (q r)))
;(iter (find-values (floor x y) maximizing (rem x y) into ((the integer
q) r)))

Consider possible extension: destructuring: (dsetq (values a (b . c) d)
#)
;(iter(find-values (values x y z) maximizing (complex x y z) into (a (b
. c) d))) ; breaks WITH
;(iter(find-values (values x y z) maximizing (complex x y z) into (a (b
c) d))) ; breaks WITH

What are the drawbacks of this macro w.r.t. a built-in clause?
 - No type inference (and type-dependent initial values)
 - No conflict when used together with other default accumulation
clauses

The first is a general problem of Iterate's API.
The second can be remedied as follows, using the technique described
in the Iterate manual.  With the update below, a conflict of gatherers
will result in a
"Duplicate variable: RESULT456" error.  Not pretty, but better than
silence.

(defmacro-clause (find-values expr maximizing max-expr &optional into
vars)
  "Please don't forget the documentation string here"
  ;; Not a good design: number of values differs whether or not INTO is
used
  (unless (or vars (eq 'values (if (consp expr) (first expr))))
    (error "Expression must start with VALUES: ~S" expr))
  (let ((temps (or vars
		   ;; Bind iterate::*result-var* so Iterate knows it's a
gatherer
		   ;; and will signal a conflict with other gathering
clauses.
		   (cons iterate::*result-var*
			 (mapcar #'(lambda (s) (gensym "VALUE")) (rest
(rest expr))))))
	;;#L(gensym (if (symbolp !1) (symbol-name !1) "VALUE"))
	(max (gensym "MAX")) (temp (gensym "NEW")))
    `(progn
      (with (,max .,temps))
      (let ((,temp ,max-expr))
	  (when (or (first-time-p) (> ,temp ,max))
	  (setq ,max ,temp)
	  ;;(multiple-value-setq ,temps ,expr)
	  (dsetq (values .,temps) ,expr)))
      ,.(unless vars
	  (list `(finally
		  (return-from ,iter::*block-name* (values
.,temps))))))))
;(iter(find-values (values x y z) maximizing (complex x y z))(collect
x)) ; error
; I think I'll add that to the testsuite as another example.

The latter also shows how to exit from a named loop via
iter::*block-name* -- this is not really great, and not documented,
but LEAVE does not work inside FINALLY (a bug in the example in the
original documentation, recently fixed), because expressions inside
finally are not walked (and I don't believe the solution is to walk
clauses in FINALLY, as that could lead to infinite loops at run-time
with malformed loops).

Note another current bug: Cannot have 2 clauses
maximizing &optional into
maximizing values &optional into
at the same time (one exact prefix of another).
Who jumps in to fix this?

Comments are welcome.
 - general mechanisms for multiple values?
 - better API for gatherer into the default value(s) vs.
iter::*result-var*?
 - finally (leave foo) & *block-name* issue?
 - exact prefix bug caretaker?

Regards,
	Jorg Hohle