[Antik-devel] [lisp-stat] first draft of numerical lisp

Sun Oct 28 14:38:28 UTC 2012

On Sun, Oct 28, 2012 at 4:45 AM, David Hodge <davidbhodge at gmail.com> wrote:
>
> Hi Tamas and everyone,
>
> To introduce myself, I guess I am a software development guy, not an academic,  but with some interest in stats.
>
> And I have been part of a varying development teams, ranging from a couple of  guys in room, to multi country/multi continental/multi cultural projects that still make me shudder.
>
> So, my take on Mirko's effort is that it is not so much a formal "spec" but more of a way of generating some discussion and agreement about the effort that we are making to bring a numerical iso and common lispstat into the modern world. I certainly sympathise with your view that too much analysis  design will get you not very far, but in this case I think that while we don't really need a formal design effort, we do need a way of creating agreement about what our growing community will set out to achieve. And its a darn fine start, in my opinion.

Well, David is a voice of reason, while mine of a dream :-)

But seriously:  The reason I put up the project was several-fold:

- I had it in mind for along time, and Antik's redefinition of some of
CL's symbols (aref for example), gave me the impetus to put my vision
out in the open.  I think my vision is very much in line with Antik's,
adding a few more things here and there.

- While definitely not wanting to define and control the
one-ring-to-rule-them-all, I wanted to start a discussion that would
lead to a broad agreement what a facilities numerical analysis in
common lisp would provide. This would provide of map of what is
needed, what exists, and what needs to be provided.

- I would extend as much of CL as possible and sensible to encompass
what is needed.  For example, I proposed to augment make-array to
enable it to create foreign-arrays (and maybe a several less sensible
things - for that see the i/o and array sections of my proposal).

- I want to stress my personal opinion these facilities should be very
much lisp-like.  I do not want to re-create a Matlab/R/IDL clone in
lisp. For example, many of these interpreted languages support
vectorized operations for the simple reason that they are numerically
more efficient.  But that forces one to write different code for
scalar and vector arguments - if that code uses tests based on the
argument type.  Since CL is *compiled*, we use mappings to achieve
pretty much the same thing using a single well-tested routine.

>
>
> I do think its worthwhile for us to at least agree a starting position about things like visualisation ( gnuplot is a choice I agree with, though while I am familiar with grammar of graphics from ggplot, I wonder about the effort required) and the the use of Antik ( having just spent some time reviewing it and playing with it a bit, its also something I can emphatically agree with).  I hope we can get these infrastructure decisions out of the way asap and we can then move onto interesting things like reviving the lisp stat regression models and putting some AR models in and so on.

Here is the first one: numerical-lisp package that shadows and
redefines some of CL symbols.  If it is well written (to allow for
follow-up changes), we have a solid base to build applications on.

>
> I also don't think the proposal is advocating One Big Library to Rule Them All (™) - I am pretty sure we all agree that having a modular  approach is the best from both a maintainability and flexibility standpoint. (And Antik has just introduced me to asdf-system-connections, which is just groovy and answers an issue that I will respond to Tony in a different email). I think the idea of providing a variety of optional components and having the system manage the dependancies when you actually want to use them is the way to go.

Agreed.  I see only a few mid-level libraries that serve as glue
between underlying libraries (both foreign and CL) and top level user
and application applications/libraries.

>
> My experience with software development like this is that if we can agree the basics in terms of architecture and infrastructure quickly, then we at least have a basis for discussion when disagreements arise later. It will make the whole process go more smoothly and the cost is actually quite low, in comparison to long drawn out debates later.

So, first homework: read proposal for NL's here:
https://github.com/mirkov/numerical-lisp/tree/master/doc/developers
and augment/critique/modify.

>
> Hope that makes sense!

Thanks for your comments David
>
> On 27/10/2012, at 10:26 PM, Tamas Papp <tkpapp at gmail.com> wrote:
>
> > On Sat, Oct 27 2012, Mirko Vukovic <mirko.vukovic at gmail.com> wrote:
> >
> >> I put up my first draft of numerical lisp on
> >> https://github.com/mirkov/numerical-lisp
> >>
> >> You will have to clone it and look at the documentation in the
> >> doc/developers/index.html.
> >>
> >> Comments, feedback, participation welcome.
> >
> > Hi Mirko,
> >
> > I have never worked as part of a software development team and I have no
> > formal training in computer science, so I know very little about
> > designing libraries.
> >
> > That said, I am somewhat skeptical about the idea of first specifying
> > development guidelines, then a detailed user interface, and leaving
> > coding after that.  IMO coding in CL is not that costly and is the only
> > way to test the viability of a design.  I fear that writing up
> > specifications before code would be wasted effort.
> >
> > I don't see why you want to achieve everything ("numerical, statistical,
> > visualization, ...") in a single library.  That's great if you can pull
> > it off, but I find small, focused projects more manageable.  For
> > example, I would break up libraries like this:
> >
> > - linear algebra
> > - numerical methods
> > - random variates
> > - visualization
> >
> > (This pretty much follows the structure of the libraries I currently
> > have, so maybe I am biased).
> >
> > Statistical code beyond simple sample statistics is a very ambitious
> > goal which requires a lot of domain-specific knowledge.  I think it is
> > more realistic to just provide the building blocks and let people write
> > their own statistical code.
> >
> > Best,
> >
> > Tamas
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Common Lisp Statistics" group.
> > To post to this group, send email to lisp-stat at googlegroups.com.
> > To unsubscribe from this group, send email to lisp-stat+unsubscribe at googlegroups.com.
> > Visit this group at http://groups.google.com/group/lisp-stat?hl=en.
> >
> >
>
>
> _______________________________________________
> Antik-devel mailing list
> Antik-devel at common-lisp.net
> http://lists.common-lisp.net/cgi-bin/mailman/listinfo/antik-devel