[Antik-devel] Setting values in an array using loop and setf, and :initial-contents

[Sumant S. R. Oemrawsingh]

Hi Liam,

I quickly did the following tests on my laptop, to verify where the
speed problems I've been having come from. See below for the forms and
results. The outcome is that grid:aref is slow (perhaps due to consing?)
and :initial-contents in make-foreign-array is sloooooooow.

A quick summary:

Setfing the values of an array inside a loop:

* With a cl-array, grid:aref is an order of magnitude slower than aref.
  Using grid:aref seems to cons about 4 times as much, and non-GC time
  is about a factor of 20 slower than just using cl aref.

* Grid:aref is more than two times as slow with a foreign array than
  with a cl-array. There is about 4 times as much consing as well.
   
Setting the values of an array using :initial-contents and a list:

* With make-array, this is blazingly fast (no time, no consing);
  probably because the list values are already stored properly in
  memory.

* With make-foreign array, this takes 50 minutes and a whole lot of
  consing (for a 1024x1024 array). Don't try this at home.

Basically, when using antik/grid, don't use :initial-contents. Still,
setfing values in a loop is an order of magnitude slower (when no
optimizations are done). I assume that if optimizations are applied,
where it is known at compile-time you have a cl-array, grid:aref should
be as fast as aref, and I will test this next. The thing is, though,
for a generic library of data processing functions, you can't optimize
for the specific array that you will use. A minor hit in performance
would still be acceptable, but an order of magnitude difference for a
cl-array when changing from aref to grid:aref is a bit crazy imo.

Sumant



;;;; loop-setf-test.lisp

(defun make-lists-list (rows columns)
  (loop for row below rows
     collect (loop for column below columns
		collect (* row column 1.0d0))))

(defun aref-loop-setf (rows columns)
  (let ((array (make-array (list rows columns) :element-type 'double-float)))
    (loop for row below rows
       do (loop for column below columns
	     do (setf (aref array row column) (* row column 1.0d0))))
    array))

(defun grid-aref-loop-setf (rows columns)
  (let ((array (make-array (list rows columns) :element-type 'double-float)))
    (loop for row below rows
       do (loop for column below columns
	     do (setf (grid:aref array row column) (* row column 1.0d0))))
    array))

(defun grid-aref-loop-setf-foreign (rows columns)
  (let ((array (grid:make-foreign-array 'double-float :dimensions (list rows columns))))
    (loop for row below rows
       do (loop for column below columns
	     do (setf (grid:aref array row column) (* row column 1.0d0))))
    array))

#|

Results on my laptop are as follows.

CL-USER> (time (progn (aref-loop-setf 1024 10240) (values)))
Evaluation took:
  0.234 seconds of real time
  0.233963 seconds of total run time (0.192970 user, 0.040993 system)
  [ Run times consist of 0.057 seconds GC time, and 0.177 seconds non-GC time. ]
  100.00% CPU
  654,633,933 processor cycles
  251,658,256 bytes consed
  
; No value
CL-USER> (time (progn (grid-aref-loop-setf 1024 10240) (values)))
Evaluation took:
  3.094 seconds of real time
  3.084532 seconds of total run time (3.054536 user, 0.029996 system)
  [ Run times consist of 0.103 seconds GC time, and 2.982 seconds non-GC time. ]
  99.71% CPU
  8,632,811,366 processor cycles
  922,738,912 bytes consed
  
; No value
CL-USER> (time (progn (grid-aref-loop-setf-foreign 1024 10240) (values)))
Evaluation took:
  7.608 seconds of real time
  7.591846 seconds of total run time (7.552852 user, 0.038994 system)
  [ Run times consist of 0.506 seconds GC time, and 7.086 seconds non-GC time. ]
  99.79% CPU
  21,235,314,846 processor cycles
  4,026,564,512 bytes consed
  
; No value
CL-USER> (progn (defvar *initial-contents* (make-lists-list 1024 1024)) (values))

; No value
CL-USER> (time (progn (make-array (list 1024 1024) :element-type 'double-float :initial-contents *initial-contents*) (values)))
Evaluation took:
  0.000 seconds of real time
  0.000000 seconds of total run time (0.000000 user, 0.000000 system)
  100.00% CPU
  1,456 processor cycles
  0 bytes consed
  
; No value
CL-USER> (progn (defvar *initial-contents* (make-lists-list 1024 1024)) (values))

; No value
CL-USER> (time (progn (make-array (list 1024 1024) :element-type 'double-float :initial-contents *initial-contents*) (values)))
Evaluation took:
  0.000 seconds of real time
  0.000000 seconds of total run time (0.000000 user, 0.000000 system)
  100.00% CPU
  1,002 processor cycles
  0 bytes consed

; No value
CL-USER> (time (progn (grid:make-foreign-array 'double-float :dimensions (list 1024 1024) :initial-contents *initial-contents*) (values)))
Evaluation took:
  2934.652 seconds of real time
  2919.775127 seconds of total run time (2919.301199 user, 0.473928 system)
  [ Run times consist of 0.199 seconds GC time, and 2919.577 seconds non-GC time. ]
  99.49% CPU
  8,190,419,850,119 processor cycles
  368,726,592 bytes consed
  
; No value
CL-USER> 

|#

-- 
Sumant Oemrawsingh