[GSLL-devel] [GSD] Time making large foreign-arrays vs large arrays

[Liam Healy]

Sumant,

I think this has something to do with the initial contents
(time (type-of (make-foreign-array 'double-float :dimensions '(1024 1024))))
Evaluation took:
  0.001 seconds of real time
  0.000000 seconds of total run time (0.000000 user, 0.000000 system)
  0.00% CPU
  341,884 processor cycles
  0 bytes consed

MATRIX-DOUBLE-FLOAT
Seems pretty snappy.  Generally, I'm not too happy with the idea of setting
values from a list, at least for large lists.  I tried to find an alternate
way to do this, but I also wanted to mimic the arguments to make-array, so
that's why it's in there.   I presume that your test is contrived and that
you wouldn't really make a list of 1 million elements and then load it into
a foreign array.  Can you try a problem where you're not using
:initial-contents?  If you don't like to make an array with indeterminate
contents, try this:

(time (type-of (map-grid :source '* :source-dims '(1024 1024)
:destination-specification '((foreign-array 1024 1024) double-float))))
Evaluation took:
  2.487 seconds of real time
  2.490000 seconds of total run time (2.490000 user, 0.000000 system)
  [ Run times consist of 0.220 seconds GC time, and 2.270 seconds non-GC
time. ]
  100.12% CPU
  5,292,435,364 processor cycles
  268,462,848 bytes consed

MATRIX-DOUBLE-FLOAT

(By the way, all make-foreign-array does is call make-grid,  i.e., it's less
"direct" than make-grid.)

Liam

On Wed, Apr 27, 2011 at 9:31 AM, Sumant Oemrawsingh <soemraws at xs4all.nl>wrote:

> Hi,
>
> I ran into a problem making foreign arrays. I have a two lists of
> lists of floats called *data-50* and *data-1024*, which are to be used
> as the initial contents of a matrix (first is 50x50, second
> 1024x1024). I do the following:
>
> CL-USER> (time (type-of (make-grid '((array) single-float)
> :initial-contents *data-50*)))
> Evaluation took:
>  0.000 seconds of real time
>  0.000000 seconds of total run time (0.000000 user, 0.000000 system)
>  100.00% CPU
>  228,096 processor cycles
>  27,744 bytes consed
>
> (SIMPLE-ARRAY SINGLE-FLOAT (50 50))
> CL-USER> (time (type-of (make-grid '((foreign-array) single-float)
> :initial-contents *data-50*)))
> Evaluation took:
>  0.036 seconds of real time
>  0.026995 seconds of total run time (0.026995 user, 0.000000 system)
>  75.00% CPU
>  79,104,003 processor cycles
>  294,688 bytes consed
>
> MATRIX-SINGLE-FLOAT
>
>
> With a 50x50 array, everything works out fine. However, when moving to
> the 1024x1024 data:
>
>
> CL-USER> (time (type-of (make-grid '((array) single-float)
> :initial-contents *data-1024*)))
> Evaluation took:
>  0.036 seconds of real time
>  0.035995 seconds of total run time (0.034995 user, 0.001000 system)
>  100.00% CPU
>  80,301,650 processor cycles
>  4,194,320 bytes consed
>
> (SIMPLE-ARRAY SINGLE-FLOAT (1024 1024))
> CL-USER> (time (type-of (make-grid '((foreign-array) single-float)
> :initial-contents *data-1024*)))
> Evaluation took:
>  686.167 seconds of real time
>  683.878034 seconds of total run time (683.608075 user, 0.269959 system)
>  [ Run times consist of 0.196 seconds GC time, and 683.683 seconds non-GC
> time. ]
>  99.67% CPU
>  3 forms interpreted
>  1,502,208,488,440 processor cycles
>  59,838,352 bytes consed
>
>  before it was aborted by a non-local transfer of control.
>
> ; Evaluation aborted on NIL.
>
>
> After over 10 minutes of intensive work, I gave up and aborted. When
> using make-foreign-array directly, instead of make-grid, I get the
> same problem. I would understand that foreign arrays take a bit more
> time to make; in the *data-50* example, about an order of magnitude
> more bytes were consed to make the foreign array (and needing more
> processor cycles).
>
> While this is not very thorough, I would expect there to be some
> linear relation between the array size and the time it
> takes. E.g. 50x50 foreign array takes less than a 0.1 second, so I
> would expect 1024x1024 to take less than 42.0 seconds. While that is
> still a long time for a normal-sized array, clearly, this is not the
> case. I can run some more tests later to find out where the problem
> lies. Or is this a known bug or limitation for foreign arrays and
> sbcl? Or am I just doing something wrong here?
>
> Thanks,
> Sumant
>
> --
> Sumant Oemrawsingh
>
> _______________________________________________
> GSLL-devel mailing list
> GSLL-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/gsll-devel/attachments/20110430/a554afdd/attachment.html>