[pro] Fwd: The Best Examples of "Code is Data"

[Vladimir Sedach]

Kazimir Majorinc:

> formulas. So far so good. However, some essential operations on formulas
> (substitution, unification, rules or inference ...) require analysis of
> the formula.
>
> Similar analysis of the function could be done only by transforming it
> back into formula with (caddr (function-lambda-expression Si)), and it
> is not simpler, even if it work, what is not guaranteed according to
> Hyperspec.

This example is confusion about phase separation
(http://axisofeval.blogspot.com/2010/07/whats-phase-separation-and-when-do-you.html),
but more relevantly it presents a false dichotomy for the solution
space and perfectly demonstrates the power of macros.

There is no need to choose between EVAL and lambdas - make the formula
operators be macros and the formulas will know how to analyze
(macro-expansion time) *and* evaluate (result of macro-expansion)
themselves - the formula interpreter falls out "for free" from doing a
COMPILE on a lambda that binds the formula free variables. As a bonus,
now the code is factored such that all the analysis code is grouped
with the definition of the relevant operators, instead of being
scattered around inside an interpreter.

A similar example came up in an ll1 mailing list discussion on macros
vs closures (http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg02060.html
- recommended reading as there is also discussion of good macros), but
using SQL query languages as the domain. CLSQL, for example, does it
with macros, which enable all kinds of compile[macro-expansion]-time
analysis and optimization. Smalltalkers do the same thing with
closures and using #doesNotUnderstand: (which is sort of like
re-binding APPLY):

http://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg02096.html

I wrote about how this technique might be used in Common Lisp:

http://carcaddar.blogspot.com/2009/04/closure-oriented-metaprogramming-via.html

I think the macro approach to queries is more succinct and powerful
(you are no longer limited to function calling syntax for your query
DSL). The query DSL is something that a lot of people seem to like -
LINQ in .NET attempts to do the same thing, but a much higher price
(an new syntax for expressing queries, a new protocol for query
consumers (database interfaces), and a horribly complicated
implementation).

The other really powerful example pointed out in the ll1 discussion
was SETF. I think SETF is one of the best examples of macros around.
It takes the idea of accessors, and turns them into a generic idea of
"places" that can be modified in the same way they are read.

This extends the language with an entirely new domain concept that
works transparently everywhere, with absolutely no changes needed to
any of its other parts. There are ways to fake "places" with closures
(ex: Oleg Kiselyov's "pointers as closures" trick, or the way dynamic
variables work in SRFI 39), but they work by completely changing the
accessor protocol, introducing a second protocol for updating places
(funcall the closure with new value), and because the "directions" to
the place are not reified they are not composable (ie - you can't do
(setf (gethash foo (aref bar)) baz) with closures), and cannot be
implemented efficiently.

Now think about what it would take to do this in Java - you'd need a
whole new DSL implemented on top of the Interpreter pattern, a Factory
to help you produce the grammar of that DSL, all with a Strategy
component so you can define new places. I bet it would be really
pleasant to use as well.

Vladimir