[Gsll-devel] Introducing "Grid Structured Data"

[Liam Healy]

The timing is unfortunate; I started on GSD a while ago, once
I re-discovered Tamas's AFFI, and was wrapping it up when he
announced xarray.  I have looked over xarray to figure out what
we have in common, but not in depth.  I think any help from
interested/energetic/knowledgeable volunteers who could help
figure out a merge strategy would be most welcome in this area.

One of the things I did to integrate GSD with GSLL was to pull out the
foreign-array access as the system "c-array", with type conversion
between languages.  This was motivated by the work of a colleague, who
essentially tore off pieces of GSLL in order to make a port to a
robotics library in C++, and who had requested certain array
construction features in GSLL.  Now it should be easier to built an
interface to a new library, and I hope eventually to smoothly change
array objects to make them compatible with different libraries.
Unfortunately I haven't had the time I thought I would in January to
pursue this, but there is certainly much more work to be done in this
area.  I pushed to the GSLL master branch because that all worked as
before, and GSD are usable.

I think all of our goals are basically the same.
Per Tamas's points:
1 - Exactly the same, present in GSD "grid" now
2 - Not yet in GSD, but that is the goal as I say
3 - Not there; would be nice to integrate with GSL's views
    as we have discussed
4 - Present in GSD; see "compose.lisp".  Perhaps not all everyone
    is looking for, but with some thought and knowledge of AFFI,
    they're not too hard to write.

The general function to make most of the array composition functions
possible is map-grid and map-n-grids.  One thing I would like to have
but isn't there is reduction over an axis, as for instance with dot
product and matrix multiplication.  Also, I would like to be able to
treat a scalar as a grid but that isn't possible yet.

Liam



On Sun, Jan 10, 2010 at 2:39 AM, Tamas Papp <tkpapp at gmail.com> wrote:
> Hi Mirko,
>
> I only had a cursory look at Liam's library.  Slices seem to be based
> on affine indexing, which in theory is faster than xarray's views, but
> it is less general.
>
> At the moment, xarray does four things:
>
> 1. rank/dimension query and element access.  If it is an array-like
> object, you can just write methods and access it via a general
> interface.  Essentially redoing CL's array interface, but with generic
> functions.
>
> 2. object conversion framework.  You can use (take 'array object) to
> convert your object to a Lisp array.  TAKE is a generic function, you
> just specialize if you introduce new classes.  It takes options (like
> element-type for arrays, etc).  Think of R's as.* functions.
>
> 3. views.  This is the only part I blogged about, but it may be the
> least important.  It is just that the rest is still under development.
>
> 4. convenience functions for generating, mapping, manipulating
> array-like objects.  If your objects have a basic xarray interface
> (basically xdims and xref), xarray can take outer products, map into
> other objects, etc.  You can specialize methods for speed, too.
>
> There are a few details I am still ironing out, but basically
> everything is functional now.  I consider 1. and 2. above the most
> important features of xarray.
>
> My goal is to arrive at a state like R, where I don't have to think
> about whether my matrix is R's standard matrix, or implemented by the
> Matrix library, if I don't want to.  Things should "just work", and I
> should only have to think about details when I optimize my code.
>
> Caveat: xarray is still experimental.  1. above should be stable,
> 2. more or less, 3. might change, and 4. too.  I want to get it right.
> Discussion about people's needs/preferences would be welcome.
>
> Best,
>
> Tamas
>
> PS.: I am on the GSLL list, so no need to cc me.  My Princeton e-mail
> is deprecated now.
>
>
> On Sat, 09 Jan 2010, Mirko Vukovic wrote:
>
>> Any comments on how that compares to Tamas Papp's xarray?
>>
>> I am very excited, happy and grateful with the availability of numerical
>> libraries in CL.  But I am concerned that we may end up with several
>> non-conformant libraries for array accessing.
>>
>> In an ideal world, we would have a common library for array manipulation,
>> and also a common architecture for accessing libraries such as gsl (C),
>> netlib (fortran), and others (sundials for example).
>>
>> Respectfully,
>>
>> Mirko
>>
>> On Sun, Dec 27, 2009 at 10:31 PM, Liam Healy <lhealy at common-lisp.net> wrote:
>>
>> > Based on several expressed wishes for the ability to create, compose
>> > and extract array-like objects and pieces of such objects, I have
>> > introduced the "Grid Structured Data" collection at
>> > http://repo.or.cz/w/gsd.git, and rewritten GSLL to be built on top of
>> > that.  The GSLL interface is the same as before, but now it is
>> > possible to do subarrays, concatenation, slices, transpose, etc. on
>> > arrays (both CL arrays and GSLL marrays).  There is some documentation
>> > for gsd in gsd/documentation/grid/index.html which describes how it
>> > works.  There is more work to be done, but as it is now, it provides
>> > functions that people have asked for to create and manipulate marrays.
>> >  If you don't need that, you can go on as before and everything is the
>> > same.
>> >
>> > Liam
>> >
>> > _______________________________________________
>> > Gsll-devel mailing list
>> > Gsll-devel at common-lisp.net
>> > http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel
>> >
>