From tcr at freebits.de Sat Dec 27 00:00:54 2008 From: tcr at freebits.de (Tobias C. Rittweiler) Date: Sat, 27 Dec 2008 01:00:54 +0100 Subject: [editor-hints-devel] My thoughts about markup and docstrings Message-ID: <87y6y2o5ah.fsf@freebits.de> Yesterday I talked with David Lichtblau on #lisp, and he told me he's currently working on structured docstrings. This subject falls under editor-hints' scope (cf. items Documentation Retrieval, and Pretty Docstrings in the [TODO] list.) I promised him to write up my thoughts about the topic and post it here. My stand on docstrings is that docstrings are totally inappropriate for documentation purposes. In my view, docstrings should be used to summarize what a function does in one, or two sentences for the reader of the code, not the user of the code. The rationale behind that thinking is that elaborate docstrings would take up too much screen space, and would seperate a) the body of a function from its lambda-list, and b) the function definition from other function definitions. What I want is a DEFINE-DOCUMENTATION macro to define elaborate documentation for a function, macro, &c. And I want Slime to bring me to a) the rendered documentation of a symbol-at-point, and b) the source of that documentation. (Cf. item Source Locations in [TODO].) I think you probably expected comments regarding the implementational model. I don't have any actually, as long as it's powerful enough to reflect MarkDown and Texinfo. There's also Rahul Jain's DefDoc which I wanted to evaluate before working on this. I know that Robert Strandh thought about this more concretly than I have done as part of his SICL project. -T. [TODO] - http://common-lisp.net/websvn/filedetails.php?repname=editor-hints&path=%2Ftrunk%2Feditor-hints%2FTODO&rev=0&sc=0 From rpgoldman at sift.info Sat Dec 27 01:18:00 2008 From: rpgoldman at sift.info (Robert Goldman) Date: Fri, 26 Dec 2008 19:18:00 -0600 Subject: [editor-hints-devel] My thoughts about markup and docstrings In-Reply-To: <87y6y2o5ah.fsf@freebits.de> References: <87y6y2o5ah.fsf@freebits.de> Message-ID: <49558248.6040103@sift.info> Tobias C. Rittweiler wrote: > Yesterday I talked with David Lichtblau on #lisp, and he told me he's > currently working on structured docstrings. This subject falls under > editor-hints' scope (cf. items Documentation Retrieval, and Pretty > Docstrings in the [TODO] list.) > > I promised him to write up my thoughts about the topic and post it here. > > > My stand on docstrings is that docstrings are totally inappropriate for > documentation purposes. In my view, docstrings should be used to > summarize what a function does in one, or two sentences for the reader > of the code, not the user of the code. I think I disagree with this pretty much totally. If the documentation strings were only for the reader of the code, rather than its user, then we wouldn't need documentation strings at all. Comments would suffice. > > The rationale behind that thinking is that elaborate docstrings would > take up too much screen space, and would seperate a) the body of a > function from its lambda-list, and b) the function definition > from other function definitions. I agree that the default placement of the docstrings can make them cause the problems that you name here. However, I don't see that that is a sufficient argument for making the docstrings into what I see as just a special case of a comment. Further, your argument above about the placement of the documentation string can be turned on its head. It's well known in software engineering by now that any time one moves the documentation any appreciable distance [1] away from the code, that documentation will inevitably stray from describing the actual state of the code, if it's ever furnished at all. Best, Robert [1] where "distance" isn't a simple metric --- there's a big distance if the documentation lives in a texinfo file, even more if it lives in a file that must be edited in a different tool from the programmer's editor. From tcr at freebits.de Sat Dec 27 03:17:48 2008 From: tcr at freebits.de (Tobias C. Rittweiler) Date: Sat, 27 Dec 2008 04:17:48 +0100 Subject: [editor-hints-devel] My thoughts about markup and docstrings References: <87y6y2o5ah.fsf@freebits.de> <49558248.6040103@sift.info> Message-ID: <87lju2nw6b.fsf@freebits.de> Robert Goldman writes: > Tobias C. Rittweiler wrote: > > > My stand on docstrings is that docstrings are totally inappropriate for > > documentation purposes. In my view, docstrings should be used to > > summarize what a function does in one, or two sentences for the reader > > of the code, not the user of the code. > > I think I disagree with this pretty much totally. If the documentation > strings were only for the reader of the code, rather than its user, then > we wouldn't need documentation strings at all. Comments would suffice. Perhaps. In my view, docstrings describe _what_ is done, comments _how_, or _why_ something is done. Therefore, docstrings and comments operate on two different views, the former on a conceptual, the latter on an implementational one. I like the fact that these two are fontified differently. And I also like that docstrings are available at runtime. This is still useful for the bulk of functions that do not need full-fledged documentation. > Further, your argument above about the placement of the documentation > string can be turned on its head. It's well known in software > engineering by now that any time one moves the documentation any > appreciable distance [1] away from the code, that documentation will > inevitably stray from describing the actual state of the code, if it's > ever furnished at all. In my ideal world, we wouldn't use raw text files as an interchange format for programm text anymore. We would use a format that would allow us to annotate, and interlink code sections (both, spatial as well as in terms of time.) You could then interlink documentation paragraphs to code sections. And your RCS should make sure that if a changeset touches such code sections, that the changeset also touches the interlinked sections (or explicitly specifies them as still being up-to-date.) Current programming text files represent the control flow of a program in kind of an adequate manner[*], but are not so good at representing the runtime data flow. -T. [*] Except for CLOS. Where flow of control can be obfuscated to death. From david at lichteblau.com Sat Dec 27 10:23:53 2008 From: david at lichteblau.com (David Lichteblau) Date: Sat, 27 Dec 2008 11:23:53 +0100 Subject: [editor-hints-devel] My thoughts about markup and docstrings In-Reply-To: <87y6y2o5ah.fsf@freebits.de> References: <87y6y2o5ah.fsf@freebits.de> Message-ID: <20081227102353.GA22136@radon> Hi, Quoting Tobias C. Rittweiler (tcr at freebits.de): > My stand on docstrings is that docstrings are totally inappropriate for > documentation purposes. In my view, docstrings should be used to > summarize what a function does in one, or two sentences for the reader > of the code, not the user of the code. > > The rationale behind that thinking is that elaborate docstrings would > take up too much screen space, and would seperate a) the body of a > function from its lambda-list, and b) the function definition > from other function definitions. > > What I want is a DEFINE-DOCUMENTATION macro to define elaborate > documentation for a function, macro, &c. And I want Slime to bring me to > a) the rendered documentation of a symbol-at-point, and b) the source of > that documentation. (Cf. item Source Locations in [TODO].) that happens to be relatively close to what I'm working on. While I would personally want to argue for an approach that would -allow- programmers to use docstrings (this is where we might disagree), my current project parse-docstrings already supports a macro that enables this documentation to be stored elsewhere instead, so it doesn't -force- programmers to write ordinary docstrings anymore. (This macro is currently called ANNOTATE-DOCUMENTATION, not DEFINE-DOCUMENTATION.) Let me explain why I arrived at this solution: My older documentation extractor called atdoc had special markup syntax for docstrings that I invented to allow special sections of a docstring for: - explicit descriptions of the arguments - explicit descriptions on the return values - special cross-references to explain slot accessors, conditions signalled, so-called constructor functions, etc. After releasing atdoc, many people told me that they liked my output, but wouldn't want to litter their docstrings with all this markup. Following that advice, I decided to pull all these special annotations out of the docstring, and into the macro. So instead of writing this: (defun foo (a b) ;example of the OLD atdoc syntax "@arg[a]{This is the important argument of type @class{bar}} @arg[b]{This is the @class{baz} being frobbed in some special way} @return{random junk of type @class{quuux}} @condition{big-mistake-error} @short{This function frobs A and B to create random junk.} Detailed description here. @see{the-other-function}" (frob a b)) You could now write: (defun foo (a b) "This functions frobs A and B to create random junk. Detailed description here." (frob a b)) (annotate-documentation (foo function) (:argument a "This is the important argument of type BAR") (:argument b "This is the BAZ being frobbed in some special way") (:return-value "random junk of type QUUUX") (:condition big-mistake-error) (:see-function the-other-function)) Or (and here our ideas might converge) you can also override the actual docstring completely like this: (defun foo (a b) "This isn't part of generated documentation, because we override it below." (frob a b)) (annotate-documentation (foo function) (:argument a "This is the important argument of type BAR") (:argument b "This is the BAZ being frobbed in some special way") (:return-value "random junk of type QUUUX") (:condition big-mistake-error) (:see-function the-other-function) "This functions frobs A and B to create random junk. Detailed description here.") When writing the macro, I was concerned that users would end up creating a dependency of their ASDF system to my project just in order to use the macro. So I decided to make the macro as small as possible, allowing users to copy&paste it into their package instead. This way, users would only have to depend on my project `parse-docstrings' when parsing and generating documentation, and not just to load their source code. Doing so requires some guarantees towards API compatibility, so the macro should better not change all the time. In the end, the macro is seven lines long, including a bad hack for SETF functions. It comes with its own documentation as an example. This is the file that users would copy&paste: http://repo.or.cz/w/parse-docstrings.git?a=blob;f=annotate-documentation.lisp And here is the code parsing it: http://repo.or.cz/w/parse-docstrings.git?a=blob;f=annotation-plist.lisp In my example above, you might have noticed that I didn't just switch from a docstring to the annotation macro, I also changed the markup syntax: The example using the macro is written using SBCL's docstring format rather than atdoc's syntax. That's why it's BAR rather than @class{bar}. Now, that's not actually a required change, because parse-docstrings leaves it completely up to users which syntax they want to use in the strings. parse-docstrings would ship with two syntax plugins out-of-the-box, one for SBCL-style docstrings and one for the @-syntax known from atdoc. But users could write their own parser plugins and parse texinfo, markdown, clixdoc syntax, or whatever they like. The parser to use is chosen with an approach somewhat similar to how hyperdoc finds URLs: We look for an internal symbol in the package being documented, and if it exists and is FBOUND, we call it to find the parser function. (If not, the default is currently SBCL-style syntax.) And personally I think that all this is the most important part: parse-docstrings offers -choice-. It doesn't force a particular solution on users, so it avoids falling into the trap of a niche approach that users dismiss as "not invented here". Instead, it is as configurable as possible, allowing at least parts of it to be reused. - docstring and/or annotation macro? the user gets to choose. (I now like the annotation macro much better than my old atdoc stuff, so I would recommend use of the annotation macro to users.) - markup syntax? the user gets to choose or implement his own. - output format? Not addressed in parse-docstrings at all. There will an SBCL-style texinfo-docstrings on top of it, an xml-docstrings project writing a big XML file for further processing, a replacement for atdoc providing XSL stylesheets, etc. (I hope that there would also be a SLIME-based solution.) In a way, thanks to the documented CLOS API for markup, we don't even enforce the whole idea where documentation is being "generated". For example, Michael Weber blogged about a "reverse docstrings" idea where docstrings would be pulled out of the texinfo documentation instead. Parse-docstrings' main API function is DOCUMENTATION*, which is like DOCUMENTATION, but - returns CLOS objects instead of a string - uses the annotation macro instead of (just) the docstring I could see another addition: - if a package declares that it wants to pull documentation from somewhere else entirely, it could provide a function to do so. That way, users like Michael could present CLOS objects that don't come from the source code at all. d. From tcr at freebits.de Sat Dec 27 14:04:28 2008 From: tcr at freebits.de (Tobias C. Rittweiler) Date: Sat, 27 Dec 2008 15:04:28 +0100 Subject: [editor-hints-devel] My thoughts about markup and docstrings References: <87y6y2o5ah.fsf@freebits.de> <20081227102353.GA22136@radon> Message-ID: <87k59ld89f.fsf@freebits.de> David Lichteblau writes: > (annotate-documentation (foo function) > (:argument a "This is the important argument of type BAR") > (:argument b "This is the BAZ being frobbed in some special way") > (:return-value "random junk of type QUUUX") > (:condition big-mistake-error) > (:see-function the-other-function)) This looks all good. I do have some suggestions: * In the documentation of ANNOTATE-DOCUMENTATION, do not talk about "forms", but about "clauses". * Add a (:RESTART) clause. * Instead of (:SEE-ALSO-FUNCTION) (:SEE-ALSO-TYPE) &c, make it (:SEE-ALSO :FUNCTION) and so on. Where the second clause argument is any symbol, and where this set of symbols can be extended. * For each symbol given in the clauses perform a consistency check, i.e. for (:CONDITION FOO) if FOO really names a condition, if not signal a style-warning. (It should probably be possible to turn this behaviour off by setting some special variable to NIL at compilation-time.) I experimented with a sexp-style documentation format, too. See [*]. > Parse-docstrings' main API function is DOCUMENTATION*, which is like > DOCUMENTATION, but > - returns CLOS objects instead of a string > - uses the annotation macro instead of (just) the docstring I'd call it STRUCTURED-DOCUMENTATION, ELABORATE-DOCUMENTATION, or similiar. [*] - http://common-lisp.net/websvn/filedetails.php?repname=editor-hints&path=%2Ftrunk%2Feditor-hints%2Fdoc%2Fdefreadtable-clhs-page.lisp&rev=0&sc=0 From rpgoldman at sift.info Sat Dec 27 19:48:30 2008 From: rpgoldman at sift.info (Robert Goldman) Date: Sat, 27 Dec 2008 13:48:30 -0600 Subject: [editor-hints-devel] My thoughts about markup and docstrings In-Reply-To: <87lju2nw6b.fsf@freebits.de> References: <87y6y2o5ah.fsf@freebits.de> <49558248.6040103@sift.info> <87lju2nw6b.fsf@freebits.de> Message-ID: <4956868E.7050801@sift.info> Tobias C. Rittweiler wrote: > Robert Goldman writes: > >> Tobias C. Rittweiler wrote: >> >>> My stand on docstrings is that docstrings are totally inappropriate for >>> documentation purposes. In my view, docstrings should be used to >>> summarize what a function does in one, or two sentences for the reader >>> of the code, not the user of the code. >> I think I disagree with this pretty much totally. If the documentation >> strings were only for the reader of the code, rather than its user, then >> we wouldn't need documentation strings at all. Comments would suffice. > > Perhaps. In my view, docstrings describe _what_ is done, comments _how_, > or _why_ something is done. Therefore, docstrings and comments operate > on two different views, the former on a conceptual, the latter on an > implementational one. > > I like the fact that these two are fontified differently. And I also > like that docstrings are available at runtime. This is still useful for > the bulk of functions that do not need full-fledged documentation. > Ah, then I think we are not in disagreement after all! My model of docstrings is that they should give enough information for a programmer who wishes to call the function (reference the variable, instantiate the class, etc.). So they should contain information about the type signature of the parameters and the return value(s); describe important side-effects; and name preconditions and postconditions if there are any. > >> Further, your argument above about the placement of the documentation >> string can be turned on its head. It's well known in software >> engineering by now that any time one moves the documentation any >> appreciable distance [1] away from the code, that documentation will >> inevitably stray from describing the actual state of the code, if it's >> ever furnished at all. > > In my ideal world, we wouldn't use raw text files as an interchange > format for programm text anymore. We would use a format that would allow > us to annotate, and interlink code sections (both, spatial as well as in > terms of time.) > > You could then interlink documentation paragraphs to code sections. And > your RCS should make sure that if a changeset touches such code > sections, that the changeset also touches the interlinked sections (or > explicitly specifies them as still being up-to-date.) > > Current programming text files represent the control flow of a > program in kind of an adequate manner[*], but are not so good at > representing the runtime data flow. I sympathize, but this kind of talk scares me. I see it leading inevitably to the kind of Java world, where one can't do anything unless one accepts the tyranny of an enormous IDE like eclipse. The Java folks have already entered this brave new world wherein one builds what seems like an endless succession of GUIs for programs that every day become more resistant to editing as text, and one must use an ever-growing bloat of infrastructure with un-readable XML, beans, tomcats and goodness only knows what all else.... Best, Robert From rpgoldman at sift.info Sun Dec 28 16:52:04 2008 From: rpgoldman at sift.info (Robert Goldman) Date: Sun, 28 Dec 2008 10:52:04 -0600 Subject: [editor-hints-devel] My thoughts about markup and docstrings In-Reply-To: <20081227102353.GA22136@radon> References: <87y6y2o5ah.fsf@freebits.de> <20081227102353.GA22136@radon> Message-ID: <4957AEB4.4090305@sift.info> I am very impressed by this design. Is there a simple version of these docstrings that is visible through the existing documentation interface? E.g., does parse-docstrings strip markup and then (SETF DOCUMENTATION)? FWIW, it also seems like use of this macro (and possibly shadowing some definer forms) would allow people to put their documentation at the BOTTOM of the defining form, if they like, thus avoiding Tobias' problem of separating header from function body, while avoiding the one I cited of having the documentation bitrot because it's distant from the code it should document... Best, r From david at lichteblau.com Sun Dec 28 17:08:57 2008 From: david at lichteblau.com (David Lichteblau) Date: Sun, 28 Dec 2008 18:08:57 +0100 Subject: [editor-hints-devel] My thoughts about markup and docstrings In-Reply-To: <4957AEB4.4090305@sift.info> References: <87y6y2o5ah.fsf@freebits.de> <20081227102353.GA22136@radon> <4957AEB4.4090305@sift.info> Message-ID: <20081228170857.GA23325@radon> Hi, Quoting Robert Goldman (rpgoldman at sift.info): > I am very impressed by this design. Is there a simple version of these > docstrings that is visible through the existing documentation interface? > E.g., does parse-docstrings strip markup and then (SETF DOCUMENTATION)? good question. So far, I'm inclined to say "no". For one thing, some people would probably prefer to keep traditional docstrings and this new kind of documentation entirely separate (like Tobias, if I understand correctly). Also, one would want DOCUMENTATION to be available right after loading a system. But to parse docstrings, you would first have to load the `parse-docstrings' system and its dependencies. Do we really want to force every author to depend on `parse-docstrings', in the ASDF :DEPENDS-ON meaning of dependency? Instead, I am hoping for a Slime contrib that would make the new documentation available easily through Emacs. This Slime contrib would have to use `parse-docstrings', but I'm hoping that this kind of dependency would be easier to accept for users. Would the SLIME contrib replace existing functionality of `slime-describe-symbol' to show the full documentation there, or would we offer an entirely new Emacs mode (and keybinding to get there)? I am not sure, but perhaps Tobias already has ideas in this direction. Note that in Nikodemus' original work, the function was actually called DOCUMENTATION, or more precisely: PARSE-DOCSTRINGS:DOCUMENTATION, which SHADOWed the normal CL:DOCUMENTATION. I already deviated from that by renaming it to DOCUMENTATION*, which makes the analogy to DOCUMENTATION very clear, but still avoids confusion between the two symbols, as well as issues for users wanting to IMPORT our package. (And now Tobias is suggesting a further rename, away from DOCUMENTATION*.) d. From rpgoldman at sift.info Sun Dec 28 17:43:53 2008 From: rpgoldman at sift.info (Robert Goldman) Date: Sun, 28 Dec 2008 11:43:53 -0600 Subject: [editor-hints-devel] My thoughts about markup and docstrings In-Reply-To: <20081228170857.GA23325@radon> References: <87y6y2o5ah.fsf@freebits.de> <20081227102353.GA22136@radon> <4957AEB4.4090305@sift.info> <20081228170857.GA23325@radon> Message-ID: <4957BAD9.5060603@sift.info> David Lichteblau wrote: > Hi, > > Quoting Robert Goldman (rpgoldman at sift.info): >> I am very impressed by this design. Is there a simple version of these >> docstrings that is visible through the existing documentation interface? >> E.g., does parse-docstrings strip markup and then (SETF DOCUMENTATION)? > > good question. > > So far, I'm inclined to say "no". > > For one thing, some people would probably prefer to keep traditional > docstrings and this new kind of documentation entirely separate (like > Tobias, if I understand correctly). > > Also, one would want DOCUMENTATION to be available right after loading a > system. But to parse docstrings, you would first have to load the > `parse-docstrings' system and its dependencies. Do we really want to > force every author to depend on `parse-docstrings', in the ASDF > :DEPENDS-ON meaning of dependency? > > > Instead, I am hoping for a Slime contrib that would make the new > documentation available easily through Emacs. This Slime contrib would > have to use `parse-docstrings', but I'm hoping that this kind of > dependency would be easier to accept for users. As someone who uses ELI more than SLIME, still, I would prefer to see a technique that extended the existing documentation facility, rather than supplanting it and dictating a particular development environment. I even work with people who use vim, and there are also Cusp, and Allegro and LispWorks' IDEs. In keeping with your notion of choice above all, I don't see any reason one would need to forbid my proposed approach above --- instead, couldn't one simply make that a configuration option? That said, where would such configuration options live? Would they be attached to an ASDF system (hard to see off hand how this would work), to a package that is documented (easier to implement, but does the policy of the documented symbol or that of *package* dominate?) or to an installation of parse-docstrings (probably not a good idea, since one might use multiple libraries with different configuration philosophies). I suspect all of these remarks are symptoms of an underlying discomfort with the concept. I'm uncomfortable with the idea of having documentation that is compiled separately, in some kind of batch mode, rather than being compiled incrementally together with the code. Of course, *manuals* can be compiled separately, but it seems like this approach sacrifices the interactive aspect of the lisp environment in order to avoid adding dependencies (per your earlier email). Best, R From david at lichteblau.com Sun Dec 28 18:21:44 2008 From: david at lichteblau.com (David Lichteblau) Date: Sun, 28 Dec 2008 19:21:44 +0100 Subject: [editor-hints-devel] My thoughts about markup and docstrings In-Reply-To: <4957BAD9.5060603@sift.info> References: <87y6y2o5ah.fsf@freebits.de> <20081227102353.GA22136@radon> <4957AEB4.4090305@sift.info> <20081228170857.GA23325@radon> <4957BAD9.5060603@sift.info> Message-ID: <20081228182144.GB23325@radon> Quoting Robert Goldman (rpgoldman at sift.info): > That said, where would such configuration options live? Would they be > attached to an ASDF system (hard to see off hand how this would work), > to a package that is documented (easier to implement, but does the > policy of the documented symbol or that of *package* dominate?) or to an > installation of parse-docstrings (probably not a good idea, since one > might use multiple libraries with different configuration philosophies). It could me much easier than all that: A macro like ANNOTATE-DOCUMENTATION would always be used, right? Either directly, or wrapped by some other macro. So let's have multiple versions of that macro. a) the tiny copy&pastable version that avoids a dependency and only "stuffs away" the documentation, to be parsed later b) a full-blown version PARSE-DOCSTRINGS:ANNOTATE-DOCUMENTATION that requires a dependency, and parses the arguments right away, for syntax validation, ability to document non-symbols, etc. c) a version called RPG-DOCSTRINGS:ANNOTATE-DOCUMENTATION (or actually, RPG-DOCSTRINGS:FOO, the name doesn't have to be identical) which also uses (SETF DOCUMENTATION) to update the main docstring. Whether or not feature (c) is actually hidden in an extra system and package RPG-DOCSTRINGS or is just a keyword argument to (b) is something I haven't decided yet -- I just wanted to point out that it *could* be an add-on system easily, should we choose not to include that feature directly in (b). d. From luismbo at gmail.com Sun Dec 28 18:23:53 2008 From: luismbo at gmail.com (Luis Oliveira) Date: Sun, 28 Dec 2008 18:23:53 +0000 Subject: [editor-hints-devel] My thoughts about markup and docstrings References: <87y6y2o5ah.fsf@freebits.de> <20081227102353.GA22136@radon> <4957AEB4.4090305@sift.info> <20081228170857.GA23325@radon> <4957BAD9.5060603@sift.info> Message-ID: <87tz8o6tvq.fsf@li14-157.members.linode.com> Robert Goldman writes: > As someone who uses ELI more than SLIME, still, I would prefer to see a > technique that extended the existing documentation facility, rather than > supplanting it and dictating a particular development environment. I > even work with people who use vim, and there are also Cusp, and Allegro > and LispWorks' IDEs. If I understand the overall design correctly, you have a couple of options: 1. implement a module for your favorite IDE. It seems to me that this approach has got plenty of potential since the IDE can employ pretty markup and make proper use of any metadata that the ANNOTATE-DOCUMENTATION macro might provide. 2. if it's not possible or convenient to implement such a module, then one could inspect the annotations, strip the markup and (setf documentation) as you suggested. > [...] > That said, where would such configuration options live? Would they be > attached to an ASDF system (hard to see off hand how this would work), It seems to me that this sort of thing is a global user preference and should not be dictated by the system/package. > I suspect all of these remarks are symptoms of an underlying discomfort > with the concept. I'm uncomfortable with the idea of having > documentation that is compiled separately, in some kind of batch mode, > rather than being compiled incrementally together with the code. This problem doesn't apply to option (1). To cater to option (2), perhaps the ANNOTATE-DOCUMENTATION macro could delegate its work when the PARSE-DOCSTRINGS system is pre-loaded? A beefier function in that system could then take care of checking user preferences and whatnot, and proceed with option (2) if appropriate. -- Lu?s Oliveira http://student.dei.uc.pt/~lmoliv/ From david at lichteblau.com Sun Dec 28 18:37:59 2008 From: david at lichteblau.com (David Lichteblau) Date: Sun, 28 Dec 2008 19:37:59 +0100 Subject: [editor-hints-devel] My thoughts about markup and docstrings In-Reply-To: <87tz8o6tvq.fsf@li14-157.members.linode.com> References: <87y6y2o5ah.fsf@freebits.de> <20081227102353.GA22136@radon> <4957AEB4.4090305@sift.info> <20081228170857.GA23325@radon> <4957BAD9.5060603@sift.info> <87tz8o6tvq.fsf@li14-157.members.linode.com> Message-ID: <20081228183759.GC23325@radon> Quoting Luis Oliveira (luismbo at gmail.com): > To cater to option (2), perhaps the ANNOTATE-DOCUMENTATION macro could > delegate its work when the PARSE-DOCSTRINGS system is pre-loaded? Sounds clever. The only danger I see is that the macro will behave differently in the user's environment than for the author of code that is being documented. Would the macro signal errors on invalid docstrings? Warnings? Style warnings? Signal errors in the parser but then catch and resignal them as style warnings? As a package author, I might not have seen these errors before making a release, because parse-docstrings wouldn't be loaded in my environment before I'm genering my documentation (and perhaps the symbol in question isn't being included in my documentation anyway, so I'm not getting the error at all). Some of my users might have parse-docstrings loaded, however, and they would get errors. In the case of warnings, users with SBCL would still complain (perhaps I'm actually using parse-docstrings, but ASDF on my Lisp ignores warnings). Catching and suppressing the errors might work. (Users with a weird *break-on-signals* would be still affected, but perhaps that falls under "then don't do that".) How about issues with API compatibility though? We would try to keep the macro's arguments compatible between versions, but what if we happen to change them after all? We wouldn't be able to catch that kind of error in the same way as parser errors. d. From david at lichteblau.com Sun Dec 28 19:16:10 2008 From: david at lichteblau.com (David Lichteblau) Date: Sun, 28 Dec 2008 20:16:10 +0100 Subject: [editor-hints-devel] My thoughts about markup and docstrings In-Reply-To: <4957BAD9.5060603@sift.info> References: <87y6y2o5ah.fsf@freebits.de> <20081227102353.GA22136@radon> <4957AEB4.4090305@sift.info> <20081228170857.GA23325@radon> <4957BAD9.5060603@sift.info> Message-ID: <20081228191610.GA23458@radon> Quoting Robert Goldman (rpgoldman at sift.info): > As someone who uses ELI more than SLIME, still, I would prefer to see a > technique that extended the existing documentation facility, rather than > supplanting it and dictating a particular development environment. Perhaps it would be helpful to understand how are you using ELI to view docstrings. Are you calling DOCUMENTATION in the listener manually? Using DESCRIBE manually? DESCRIBE through a keybinding? I'm asking because I never found the commands available to view docstrings particularly convenient before (neither in ELI nor in SLIME). So part of why I'm looking forward to something entirely new is that the "old way" didn't work very well for me anyway. If you're calling DOCUMENTATION or DESCRIBE manually, would alternative versions of those functions available in parse-docstrings be useful? Perhaps we should really rename the function that returns CLOS objects to STRUCTURED-DOCUMENTATION, and offer a function DOCUMENTATION* that returns a plaintext rendering of the markup. d. From rpgoldman at sift.info Sun Dec 28 20:00:50 2008 From: rpgoldman at sift.info (Robert P. Goldman) Date: Sun, 28 Dec 2008 14:00:50 -0600 Subject: [editor-hints-devel] My thoughts about markup and docstrings In-Reply-To: <20081228191610.GA23458@radon> References: <87y6y2o5ah.fsf@freebits.de> <20081227102353.GA22136@radon> <4957AEB4.4090305@sift.info> <20081228170857.GA23325@radon> <4957BAD9.5060603@sift.info> <20081228191610.GA23458@radon> Message-ID: <3936-SnapperMsg6FC6F8FAC57D8B87@[174.158.19.179]> It probably wouldn't be at all hard to cargo-cult together something that would work with ELI's inferior lisp. I use the existing eli interface to documentation now. It's never been totally satisfactory, of course, mostly because the CL documentation facility doesn't have an obvious extension path. I suggested the backend to DOCUMENTATION as a way to allow un-enhanced environments to get the benefits of your enhanced docs. Sneaking off from family events to think about CL, r ___ Robert P. Goldman Principal Scientist, SIFT, LLC www.sift.info ...... Original Message ....... On Sun, 28 Dec 2008 20:16:10 +0100 "David Lichteblau" wrote: >Quoting Robert Goldman (rpgoldman at sift.info): >> As someone who uses ELI more than SLIME, still, I would prefer to see a >> technique that extended the existing documentation facility, rather than >> supplanting it and dictating a particular development environment. > >Perhaps it would be helpful to understand how are you using ELI to view >docstrings. > >Are you calling DOCUMENTATION in the listener manually? > >Using DESCRIBE manually? DESCRIBE through a keybinding? > >I'm asking because I never found the commands available to view >docstrings particularly convenient before (neither in ELI nor in SLIME). > >So part of why I'm looking forward to something entirely new is that the >"old way" didn't work very well for me anyway. > >If you're calling DOCUMENTATION or DESCRIBE manually, would alternative >versions of those functions available in parse-docstrings be useful? > >Perhaps we should really rename the function that returns CLOS objects >to STRUCTURED-DOCUMENTATION, and offer a function DOCUMENTATION* that >returns a plaintext rendering of the markup. > > >d.