[Gsll-devel] Introducing "Grid Structured Data"

[A.J. Rossini]

My cursory look suggests a fair amount of similarity as well, and I
find attractive features in both -- but want just one API!

A reasonable proposal, from my highly biased perspective, would be a
3rd party merge of the better components of each.

(I've spent time with the xarray interface, which is the source of my
biases -- it mostly works for what I want to do, despite being a
simplification of the lisp-matrix access approach -- which isn't bad,
there are some lisp-matrix functionality which is strictly edge-case
relevant...


On Sun, Jan 10, 2010 at 3:28 PM, Liam Healy <lhealy at common-lisp.net> wrote:
> The timing is unfortunate; I started on GSD a while ago, once
> I re-discovered Tamas's AFFI, and was wrapping it up when he
> announced xarray.  I have looked over xarray to figure out what
> we have in common, but not in depth.  I think any help from
> interested/energetic/knowledgeable volunteers who could help
> figure out a merge strategy would be most welcome in this area.
>
> One of the things I did to integrate GSD with GSLL was to pull out the
> foreign-array access as the system "c-array", with type conversion
> between languages.  This was motivated by the work of a colleague, who
> essentially tore off pieces of GSLL in order to make a port to a
> robotics library in C++, and who had requested certain array
> construction features in GSLL.  Now it should be easier to built an
> interface to a new library, and I hope eventually to smoothly change
> array objects to make them compatible with different libraries.
> Unfortunately I haven't had the time I thought I would in January to
> pursue this, but there is certainly much more work to be done in this
> area.  I pushed to the GSLL master branch because that all worked as
> before, and GSD are usable.
>
> I think all of our goals are basically the same.
> Per Tamas's points:
> 1 - Exactly the same, present in GSD "grid" now
> 2 - Not yet in GSD, but that is the goal as I say
> 3 - Not there; would be nice to integrate with GSL's views
>    as we have discussed
> 4 - Present in GSD; see "compose.lisp".  Perhaps not all everyone
>    is looking for, but with some thought and knowledge of AFFI,
>    they're not too hard to write.
>
> The general function to make most of the array composition functions
> possible is map-grid and map-n-grids.  One thing I would like to have
> but isn't there is reduction over an axis, as for instance with dot
> product and matrix multiplication.  Also, I would like to be able to
> treat a scalar as a grid but that isn't possible yet.
>
> Liam
>
>
>
> On Sun, Jan 10, 2010 at 2:39 AM, Tamas Papp <tkpapp at gmail.com> wrote:
>> Hi Mirko,
>>
>> I only had a cursory look at Liam's library.  Slices seem to be based
>> on affine indexing, which in theory is faster than xarray's views, but
>> it is less general.
>>
>> At the moment, xarray does four things:
>>
>> 1. rank/dimension query and element access.  If it is an array-like
>> object, you can just write methods and access it via a general
>> interface.  Essentially redoing CL's array interface, but with generic
>> functions.
>>
>> 2. object conversion framework.  You can use (take 'array object) to
>> convert your object to a Lisp array.  TAKE is a generic function, you
>> just specialize if you introduce new classes.  It takes options (like
>> element-type for arrays, etc).  Think of R's as.* functions.
>>
>> 3. views.  This is the only part I blogged about, but it may be the
>> least important.  It is just that the rest is still under development.
>>
>> 4. convenience functions for generating, mapping, manipulating
>> array-like objects.  If your objects have a basic xarray interface
>> (basically xdims and xref), xarray can take outer products, map into
>> other objects, etc.  You can specialize methods for speed, too.
>>
>> There are a few details I am still ironing out, but basically
>> everything is functional now.  I consider 1. and 2. above the most
>> important features of xarray.
>>
>> My goal is to arrive at a state like R, where I don't have to think
>> about whether my matrix is R's standard matrix, or implemented by the
>> Matrix library, if I don't want to.  Things should "just work", and I
>> should only have to think about details when I optimize my code.
>>
>> Caveat: xarray is still experimental.  1. above should be stable,
>> 2. more or less, 3. might change, and 4. too.  I want to get it right.
>> Discussion about people's needs/preferences would be welcome.
>>
>> Best,
>>
>> Tamas
>>
>> PS.: I am on the GSLL list, so no need to cc me.  My Princeton e-mail
>> is deprecated now.
>>
>>
>> On Sat, 09 Jan 2010, Mirko Vukovic wrote:
>>
>>> Any comments on how that compares to Tamas Papp's xarray?
>>>
>>> I am very excited, happy and grateful with the availability of numerical
>>> libraries in CL.  But I am concerned that we may end up with several
>>> non-conformant libraries for array accessing.
>>>
>>> In an ideal world, we would have a common library for array manipulation,
>>> and also a common architecture for accessing libraries such as gsl (C),
>>> netlib (fortran), and others (sundials for example).
>>>
>>> Respectfully,
>>>
>>> Mirko
>>>
>>> On Sun, Dec 27, 2009 at 10:31 PM, Liam Healy <lhealy at common-lisp.net> wrote:
>>>
>>> > Based on several expressed wishes for the ability to create, compose
>>> > and extract array-like objects and pieces of such objects, I have
>>> > introduced the "Grid Structured Data" collection at
>>> > http://repo.or.cz/w/gsd.git, and rewritten GSLL to be built on top of
>>> > that.  The GSLL interface is the same as before, but now it is
>>> > possible to do subarrays, concatenation, slices, transpose, etc. on
>>> > arrays (both CL arrays and GSLL marrays).  There is some documentation
>>> > for gsd in gsd/documentation/grid/index.html which describes how it
>>> > works.  There is more work to be done, but as it is now, it provides
>>> > functions that people have asked for to create and manipulate marrays.
>>> >  If you don't need that, you can go on as before and everything is the
>>> > same.
>>> >
>>> > Liam
>>> >
>>> > _______________________________________________
>>> > Gsll-devel mailing list
>>> > Gsll-devel at common-lisp.net
>>> > http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel
>>> >
>>
>
> _______________________________________________
> Gsll-devel mailing list
> Gsll-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel
>



-- 
best,
-tony

blindglobe at gmail.com
Muttenz, Switzerland.
"Commit early,commit often, and commit in a repository from which we
can easily roll-back your mistakes" (AJR, 4Jan05).

Drink Coffee:  Do stupid things faster with more energy!