[GSLL-devel] [GSD] Time making large foreign-arrays vs large arrays

Tue May 3 10:19:54 UTC 2011

Hi Liam,

Yes, this example is quite contrived, and I don't use
:initial-contents often. I thought it was worth mentioning because
making lisp and foreign arrays using that keyword differs by _so_ much
time, and I didn't quite understand why.

Thanks,
Sumant

On Sat, Apr 30, 2011 at 11:47:54PM -0400, Liam Healy wrote:
>    Sumant,
>    I think this has something to do with the initial contents
>    (time (type-of (make-foreign-array 'double-float :dimensions '(1024
>    1024))))
>    Evaluation took:
>      0.001 seconds of real time
>      0.000000 seconds of total run time (0.000000 user, 0.000000 system)
>      0.00% CPU
>      341,884 processor cycles
>      0 bytes consed
> 
>    MATRIX-DOUBLE-FLOAT
>    Seems pretty snappy.  Generally, I'm not too happy with the idea of
>    setting values from a list, at least for large lists.  I tried to find
>    an alternate way to do this, but I also wanted to mimic the arguments
>    to make-array, so that's why it's in there.   I presume that your test
>    is contrived and that you wouldn't really make a list of 1 million
>    elements and then load it into a foreign array.  Can you try a problem
>    where you're not using :initial-contents?  If you don't like to make an
>    array with indeterminate contents, try this:
>    (time (type-of (map-grid :source '* :source-dims '(1024 1024)
>    :destination-specification '((foreign-array 1024 1024) double-float))))
>    Evaluation took:
>      2.487 seconds of real time
>      2.490000 seconds of total run time (2.490000 user, 0.000000 system)
>      [ Run times consist of 0.220 seconds GC time, and 2.270 seconds
>    non-GC time. ]
>      100.12% CPU
>      5,292,435,364 processor cycles
>      268,462,848 bytes consed
> 
>    MATRIX-DOUBLE-FLOAT
>    (By the way, all make-foreign-array does is call make-grid,  i.e., it's
>    less "direct" than make-grid.)
>    Liam
> 
>    On Wed, Apr 27, 2011 at 9:31 AM, Sumant Oemrawsingh
>    <[1]soemraws at xs4all.nl> wrote:
> 
>      Hi,
>      I ran into a problem making foreign arrays. I have a two lists of
>      lists of floats called *data-50* and *data-1024*, which are to be
>      used
>      as the initial contents of a matrix (first is 50x50, second
>      1024x1024). I do the following:
>      CL-USER> (time (type-of (make-grid '((array) single-float)
>      :initial-contents *data-50*)))
>      Evaluation took:
>       0.000 seconds of real time
>       0.000000 seconds of total run time (0.000000 user, 0.000000 system)
>       100.00% CPU
>       228,096 processor cycles
>       27,744 bytes consed
>      (SIMPLE-ARRAY SINGLE-FLOAT (50 50))
>      CL-USER> (time (type-of (make-grid '((foreign-array) single-float)
>      :initial-contents *data-50*)))
>      Evaluation took:
>       0.036 seconds of real time
>       0.026995 seconds of total run time (0.026995 user, 0.000000 system)
>       75.00% CPU
>       79,104,003 processor cycles
>       294,688 bytes consed
>      MATRIX-SINGLE-FLOAT
>      With a 50x50 array, everything works out fine. However, when moving
>      to
>      the 1024x1024 data:
>      CL-USER> (time (type-of (make-grid '((array) single-float)
>      :initial-contents *data-1024*)))
>      Evaluation took:
>       0.036 seconds of real time
>       0.035995 seconds of total run time (0.034995 user, 0.001000 system)
>       100.00% CPU
>       80,301,650 processor cycles
>       4,194,320 bytes consed
>      (SIMPLE-ARRAY SINGLE-FLOAT (1024 1024))
>      CL-USER> (time (type-of (make-grid '((foreign-array) single-float)
>      :initial-contents *data-1024*)))
>      Evaluation took:
>       686.167 seconds of real time
>       683.878034 seconds of total run time (683.608075 user, 0.269959
>      system)
>       [ Run times consist of 0.196 seconds GC time, and 683.683 seconds
>      non-GC time. ]
>       99.67% CPU
>       3 forms interpreted
>       1,502,208,488,440 processor cycles
>       59,838,352 bytes consed
>       before it was aborted by a non-local transfer of control.
>      ; Evaluation aborted on NIL.
>      After over 10 minutes of intensive work, I gave up and aborted. When
>      using make-foreign-array directly, instead of make-grid, I get the
>      same problem. I would understand that foreign arrays take a bit more
>      time to make; in the *data-50* example, about an order of magnitude
>      more bytes were consed to make the foreign array (and needing more
>      processor cycles).
>      While this is not very thorough, I would expect there to be some
>      linear relation between the array size and the time it
>      takes. E.g. 50x50 foreign array takes less than a 0.1 second, so I
>      would expect 1024x1024 to take less than 42.0 seconds. While that is
>      still a long time for a normal-sized array, clearly, this is not the
>      case. I can run some more tests later to find out where the problem
>      lies. Or is this a known bug or limitation for foreign arrays and
>      sbcl? Or am I just doing something wrong here?
>      Thanks,
>      Sumant
>      --
>      Sumant Oemrawsingh
>      _______________________________________________
>      GSLL-devel mailing list
>      [2]GSLL-devel at common-lisp.net
>      [3]http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel
> 
> References
> 
>    1. mailto:soemraws at xs4all.nl
>    2. mailto:GSLL-devel at common-lisp.net
>    3. http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel

> _______________________________________________
> GSLL-devel mailing list
> GSLL-devel at common-lisp.net
> http://common-lisp.net/cgi-bin/mailman/listinfo/gsll-devel

-- 
Sumant Oemrawsingh