[pro] The Best Examples of "Code is Data"

Thu Sep 9 15:26:54 UTC 2010

Mark,

Mark Haniford wrote:
> That's just Emacs.  Most modern tools in the Java or .NET world
> already do that type of stuff.    Apple's Dylan IDE probably would
> have that stuff 10-12 years ago, but NeXT came around and the rest is
> history.
>   

Unfortunately, there are serious problems introduced
by the use of macros.  There are "lines of code" that
cannot be seen in the source, because they are
produced by macros.

One place where this obviously causes problems
is code coverage tools.  You run all of your tests,
and the tool tells you what lines of code didn't
ever get tested.  (That's an oversimplified
nutshell description.)  The Clozure team
is working on a solutinon to this problem.
I don't know to what degree that work
could be applied to other areas of IDE
capabilites, such as you said.

One of the very first things you want when
you start writing macros is for auto-indentation
to do the right thing.  The IDE can't just
know this for a macro.

In case you want to know how we do this, read
the rest of this mail.  Fasten your seatbelts;
we're in for a bumpy ride!

To deal with this, in our system, we collect
information for each macro about how
it should be indented.  For conmmon
idiomatic cases, this is done automatically,
but you can do it manually or complicated
macros or where the automatic generation
isn't right.  There is a "little language"
for declaring how a form should be indented.
In some places in the code, the forms of
this "little language" are called "indentaiton
descriptors" (which I think is a good name).

We load all of the code of our system into
Lisp, just as anyone would.  During the
load, the loaders sees top level forms
that associate the names of the macros
with a little data structure in the "little
language". It puts these into a simple
hash table, keyed by the name of the function.

We also load swank (the Lisp side of
Slime).  This will automatically read
in the per-site swank modifications,
in site-init.lisp.  It sends this back to
Slime, with the keyword :indentation-update.
(It would be nice if this keyword contained
"indentation-descriptor" in its name, but
that's not important here.)

This causes Slime to call slime-handle-indentation-update.
This puts the new indentation desriptor
on both the slime-indent and the common-lisp-indent-function
properties of the symbol whose indentation we're been
talking about.  This gets examined by the code
in site-init.el (that's the Emacs side of the per-site
stuff).  In our case, a GNU Emacs function named
"cl-indent:function) gets called.  This is a very
big and complicated function, which interprets
the "little language" (the indentation descriptor).

Next, we have a top-level form in our

libs/slime/site-init.lisp

Next, we have a top-level form in our
site-init.  What it does is to make
a one-element list of the hash table,
and then set

swank::*application-hints-tables*

to that list.

Next, when we load swank, we also load
some of the nice contributed swank libraries,
and one of the ones we load is called

contrib/swank-indentation.lisp

also, back when we loaded slime,
we also loaded into GNU Emacs
the file

contrib/slime-indentation.el

The Lisp function

symbol-indentation

is re-defined to do indentation by first
looking for the symbol in our hash table,
and if it's not found, doing what the original
symbol-indentation function would have done.
(You can see that in swank.lisp.)

If it does find a result in the hash table, that
means we stated how the macro should indent,
using the "little language".  When Slime
needs to indent something, one of the things
it does is to send a message to Swank
that causes Swank to run symbol-indentation

> On Wed, Sep 8, 2010 at 10:39 PM, Laughing Water <lw at mt.net> wrote:
>   
>> Regarding the use of "it":
>>
>> What if your editor could reveal the referent of "it" with a simple mouse-over? Would that be an adequate solution?
>>
>> Come to think of it, programming could be easier in general if more information could appear in this "tool-tip" fashion rather than requiring the heavyweight mechanism of applying a tool that opens another window, or worse, looking up documentation in a separate web-browser.
>>
>> I may be the least professional Lisper in this discussion, and I haven't learned my tools yet. So take my ideas for whatever they're worth.
>>
>> LW
>>
>>
>> On Sep 8, 2010, at 8:34 AM, Hans Hübner wrote:
>>
>>     
>>> On Wed, Sep 8, 2010 at 09:59, Julian Squires <julian at cipht.net> wrote:
>>>       
>>>> I don't understand why anaphoric macros are so frowned upon.  Like any
>>>> powerful tool, it's easy to construct pathological situations where
>>>> they make code more difficult to understand.  However, I feel that
>>>> when tastefully used, they enhance the expressiveness of the language,
>>>> readibility of code, and they can help to reduce the verbosity of CL
>>>> of which many outsiders complain.  Of course, the prior two sentences
>>>> apply just as well to macros in general as they do to anaphora.
>>>>         
>>> The power of macros gives Lisp programmers the freedom to create the
>>> language that they need, but every single modification of the language
>>> makes it derive from what another programmer would expect.  Thus, in a
>>> setting where the written code serves not only as instructions to a
>>> computer, but also as prose communicating intent to another
>>> programmer, it is vital to establish borders to the amount of language
>>> modification permitted.
>>>
>>> There are certain idiomatic macro styles that are commonly understood
>>> amongs Lisp programmers, but the number of these styles is low.  One
>>> of them is the WITH-family used for implicit (and guaranteed) resource
>>> release, another is the DO-family for iteration.  These idiomatic
>>> styles are easy to recognize and well-established, so they are rather
>>> common.
>>>
>>> Another family of macros that is everywhere, idiomatic and
>>> well-established is top-level definers.  That family certainly is much
>>> broader, and often a simple top-level definition form expands into
>>> loads of code.  They are often part of a larger framework, and as
>>> top-level forms they are easy to recognize.
>>>
>>> Anaphoric macros usually are much harder to recognize visually, and
>>> they do "unexpected" things to source code on a micro-level.  This is
>>> what I object to.  Within the body of the anaphoric construct, a
>>> simple variable "it" suddenly becomes dependent on the local context.
>>> I find code with anaphoric macros harder to read than code that uses
>>> explicit variable names, because, in a linear reading flow, I am not
>>> told what a certain symbol is bound to.  Rather, I need to imagine the
>>> "it" myself when reading "awhen". [*]
>>>
>>> What I do support is the use of combined test-and-bind macros
>>> (WHEN-BIND, WHEN-LET or similar).  They serve the same purpose as
>>> anaphoric macros (reduce repetetive words in local contexts), but they
>>> are much more scalable and require giving things names.
>>>
>>> Anaphoric macros are a good example of things that one can do with
>>> Common Lisp and that one can also like about Common Lisp, but that are
>>> not useful in industrial software development practice.  When working
>>> together, making things as easy to read as possible is not only a
>>> question of style, but also of politeness and responsibility.  I am
>>> certainly not claiming that the macro types listed above are the only
>>> macros that are useful and should be allowed.  It is just that
>>> anaphoric macros, despite all their cuteness, tax code readers
>>> unnecessarily hard and thus have not been adopted as acceptable
>>> practice in multi-programmer environments.
>>>
>>> -Hans
>>>
>>> [*] I never really dug Forth for anything but  small programs because,
>>> like anaphoric macros in CL, Forth requires me to keep more state in
>>> my head when reading code.
>>>
>>> _______________________________________________
>>> pro mailing list
>>> pro at common-lisp.net
>>> http://common-lisp.net/cgi-bin/mailman/listinfo/pro
>>>       
>> _______________________________________________
>> pro mailing list
>> pro at common-lisp.net
>> http://common-lisp.net/cgi-bin/mailman/listinfo/pro
>>
>>     
>
> _______________________________________________
> pro mailing list
> pro at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/pro
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/pro/attachments/20100909/f9f86940/attachment.html>