File I/O Metrics

[Alan Ruttenberg]

Here's an example of fast reading in ABCL. The implementation of certain
array types use java.nio buffers, which can be directly read into via the
java.nio functions.

(defun test-read (path)
  (let* ((f (new 'RandomAccessFile (namestring (truename path)) "r"))
         (channel (#"getChannel" f))
         (array (make-array (* 16 1024 1024) :element-type '(unsigned-byte
8)))
         (buffer (get-java-field array "elements" t)))
    (time (loop for count = (#"read" channel buffer)
                until (eql count -1)
                sum count
                do (#"position" buffer 0)
                ))))

On my machine, for a 5G file, the SBCL code in an earlier post takes 2.4
seconds. This code takes 1.4 sec. It's fastest if I use 2M buffers - 1.1
seconds. SBCL is also marginally faster with smaller buffer sizes.

In the ABCL source code the files "SimpleArray_*.java" are the
implementations of the nio buffer backed array types. See make_array.java
where the specific type of underlying array is chosen. There is a global
switch for array allocation choosing either direct allocation or
nio-buffers, with the default being nio buffers.

(get-java-field 'java$buffers "active" t)
-> #<org.armedbear.lisp.Java$Buffers$AllocationPolicy NIO {432E958E}>

I haven't looked into the :element-type 'character case.

Maybe someone who is familiar with the ABCL stream implementation is
interested in writing a fast path for read-sequence that uses the nio
calls? If so, shout. Otherwise I'll keep it on my
procrastinate-by-hacking-abcl list.

Alan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/armedbear-devel/attachments/20221031/e738d819/attachment.html>