[pro] The Best Examples of "Code is Data"

[Pascal Costanza]

On 28 Sep 2010, at 07:17, Daniel Gackle wrote:

> A lot of the value in Lisp macros is how simple they make certain
> things. People sometimes ask for an example of macros that "can't be
> done without them" - but that's a contradiction. By definition,
> anything that can be done by a macro can be done by its expansion.  It
> can't, however, necessarily be done as easily. But this is a harder
> debate to have because "easy" gets so subjective so quickly.

I think the benefits of macro can be stated a bit more objectively.

One first has to agree on the idea that abstraction is a good idea. Abstraction is a good idea for at least two reasons:

+ It makes it easier to see what is actually going on in the code, without being distracted by unnecessary implementation details.
+ It makes it easier to change implementation details without affecting other parts in the code.

Consider a simple functional abstraction, a sort routine. If you can say (sort this-list), then it's pretty clear from reading this invocation what the programmer wants to do here. On top of that, a programmer can change his/her mind about the details of the sorting routine (whether it's a stable sort or not, etc.), without affecting the invocation sites. You can even change the implementation more dynamically, by way of using generic functions. [1]

Macros are good to provide syntactic abstractions, that is, abstractions that cannot be expressed easily using more conventional abstraction mechanisms, like functional or object-oriented ones. My favorite example for this, so far, is a while macro. Consider the following definition for a while _function_:

(defun while/f (predicate body)
  (when (funcall predicate)
    (funcall body)
    (while/f predicate body)))

Expressing a while construct like that is suggested in languages that provide functional abstractions, but no syntactic abstractions. It can be used as follows:

(while/f (lambda () (< x 10))
  (lambda ()
    (print x)
    (incf x)))

This a bit wordier than in other languages (such as ML, Haskell, Smalltalk, Ruby, etc.). But the issue here is not that 'lambda is a keyword that happens to be a few characters too long. The real issue here is that while/f is a leaky abstraction: It exposes an internal implementation detail, namely that it uses closures to do its job. As a user of while/f, you have to remember that while/f expects you to pass closures, and as a user, you also have a pretty good, rough idea how while/f is implemented internally by just looking at the interface.

A macro can help to abstract away that internal implementation detail, for example as a wrapper around the functional while/f:

(defmacro (predicate-expression &body body-expressions)
  `(while/f (lambda() ,predicate-expression)
     (lambda () , at body-expressions))

Now you can use it as follows:

(while (< x 10)
  (print x)
  (incf x))

This version is much improved with regard to the two criteria mentioned above: 1) The code is now clearer and easier to understand, because you are not distracted by the lambdas, which are totally unnecessary from a user perspective. On top of that, 2) it's now easier to change your mind about the internal implementation details. For example, you can decide to remove the use of closures completely (say, because you identified them as a performance bottleneck ;):

(defmacro while (predicate-expression &body body-expressions)
  (let ((begin (gensym)) (end (gensym)))
    `(tagbody
       ,begin (unless ,predicate-expression (go ,end))
       , at body-expressions
       (go ,begin)
       ,end)))

With this slightly optimized version, there is no need to change the invocation site at all. This is at least very hard to achieve with just functional abstractions, and depending on language probably impossible. [2]

So, to the best of my knowledge, that's the main reason why macros are beneficial, and that's the shortest example I can think of to illustrate that point. Just like any abstraction mechanism, macros pay off a lot more in large programs. (This is an important point here: Functional abstractions and object-oriented abstractions, to name just two popular ones, also only pay off in large programs!)


Pascal

[1] That's the true benefit of object-oriented programming. It has nothing to do with inheritance, encapsulation, modeling the "real" world in terms of objects, etc. It simply boils down to being able to choose different implementations for method/function signatures based on runtime criteria in a way that is slightly smarter than plain if/cond/case statements.
[2] One could imagine some facility for intercepting a compiler to perform custom low-level optimizations, but that is probably a lot messier than a macro mechanism.

-- 
Pascal Costanza, mailto:pc at p-cos.net, http://p-cos.net
Vrije Universiteit Brussel
Software Languages Lab
Pleinlaan 2, B-1050 Brussel, Belgium