[Gsll-devel] practical differences between native CL array and the GSLL marray type?

[Liam Healy]

Yes, it is definitely my recommendation to use marrays
everywhere.  I am starting to get in the habit of doing so
myself.  One of my ideas is to extend CL array operations to
marrays so that you need not convert back and forth.  It is
also my intent that other math software libraries would also
use marrays (or an extension of them), so that they become a
standard way to interchange data between libraries.

There is more overhead for marrays than plain arrays, but on
SBCL where the native arrays are used in GSL directly, there
is very little, and there is no copying.  I recommend that
you build it with marray and then see if you have
performance problems.  However, if you are talking matrices
of 1000000x1000000, I think you will have problems in any
representation on almost any kind of hardware.  For single
floats, one such matrix is 4TB.  In that case, I recommend
you rethink the problem structure.  If these are sparse
matrices, you should use a sparse matrix representation.
GSL doesn't have such a representation, but others have been
interested in having it so it might be worth posting
something to the GSL mailing list -- perhaps someone has
written a GSL-compatible sparse array representation.

Liam



On Sat, Apr 11, 2009 at 8:12 AM, Malcolm Reynolds
<malcolm.reynolds at gmail.com> wrote:
> Hi all, hope this is the right place for this kind of request.
>
> I'm writing a program which will be doing Machine Learning over graph
> data structures. In it I'm going to be needing to represent a bunch of
> matrices, some sparse, some definitely not sparse, and do fairly basic
> operations on these matrices (add/ subtract/multiply, pseudoinverse,
> probably some others). I've started a basic prototype using the
> standard CL arrays but I always knew I would need GSLL to calculate a
> pseudo inverse.
>
> My question is, would it be a reasonably sensible decision to just use
> marrays for the whole program rather than worrying
> about converting to/ from CL-type arrays all the time? Are there any
> places where CL-arrays beat marrays either in terms of memory usage,
> access speed, or anything else? I might be needing to scale this
> system up be dealing with N*N square matrices where N is on the order
> of hundreds of thousands or even millions, if that makes any
> difference to the recommendation.
>
> Thanks in advance for any help!
>
> Malcolm Reynolds
>
> _______________________________________________
> Gsll-devel mailing list
> Gsll-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel
>