[GSLL-devel] [GSD] Time making large foreign-arrays vs large arrays

[Sumant Oemrawsingh]

Hi,

I ran into a problem making foreign arrays. I have a two lists of
lists of floats called *data-50* and *data-1024*, which are to be used
as the initial contents of a matrix (first is 50x50, second
1024x1024). I do the following:

CL-USER> (time (type-of (make-grid '((array) single-float) :initial-contents *data-50*)))
Evaluation took:
  0.000 seconds of real time
  0.000000 seconds of total run time (0.000000 user, 0.000000 system)
  100.00% CPU
  228,096 processor cycles
  27,744 bytes consed
  
(SIMPLE-ARRAY SINGLE-FLOAT (50 50))
CL-USER> (time (type-of (make-grid '((foreign-array) single-float) :initial-contents *data-50*)))
Evaluation took:
  0.036 seconds of real time
  0.026995 seconds of total run time (0.026995 user, 0.000000 system)
  75.00% CPU
  79,104,003 processor cycles
  294,688 bytes consed
  
MATRIX-SINGLE-FLOAT


With a 50x50 array, everything works out fine. However, when moving to
the 1024x1024 data:

  
CL-USER> (time (type-of (make-grid '((array) single-float) :initial-contents *data-1024*)))
Evaluation took:
  0.036 seconds of real time
  0.035995 seconds of total run time (0.034995 user, 0.001000 system)
  100.00% CPU
  80,301,650 processor cycles
  4,194,320 bytes consed
  
(SIMPLE-ARRAY SINGLE-FLOAT (1024 1024))
CL-USER> (time (type-of (make-grid '((foreign-array) single-float) :initial-contents *data-1024*)))
Evaluation took:
  686.167 seconds of real time
  683.878034 seconds of total run time (683.608075 user, 0.269959 system)
  [ Run times consist of 0.196 seconds GC time, and 683.683 seconds non-GC time. ]
  99.67% CPU
  3 forms interpreted
  1,502,208,488,440 processor cycles
  59,838,352 bytes consed
  
  before it was aborted by a non-local transfer of control.
  
; Evaluation aborted on NIL.


After over 10 minutes of intensive work, I gave up and aborted. When
using make-foreign-array directly, instead of make-grid, I get the
same problem. I would understand that foreign arrays take a bit more
time to make; in the *data-50* example, about an order of magnitude
more bytes were consed to make the foreign array (and needing more
processor cycles).

While this is not very thorough, I would expect there to be some
linear relation between the array size and the time it
takes. E.g. 50x50 foreign array takes less than a 0.1 second, so I
would expect 1024x1024 to take less than 42.0 seconds. While that is
still a long time for a normal-sized array, clearly, this is not the
case. I can run some more tests later to find out where the problem
lies. Or is this a known bug or limitation for foreign arrays and
sbcl? Or am I just doing something wrong here?

Thanks,
Sumant

-- 
Sumant Oemrawsingh