[asdf-devel] Faster loading for deployed systems?

[Erik Pearson]

Initial results for require-system are improved over load system, for sure:

To successfully find a system:

asdf:require-system = 0.2 seconds
asdf:load-system  = 2.2 seconds
require = 0.001 seconds
member of *modules* = 0.00005 seconds

To fail to find a system ("notasystem")

asdf:require-system = 0.21 sec
asdf:load-system = 0.007 sec
require = 0.1 sec
(member .. *modules*) = 0.00006 sec

Since failing to find a system results in an exception for the first
three cases, time is probably less important a measurement in this
case.


? (time (asdf:require-system :messageq))
(ASDF:REQUIRE-SYSTEM :MESSAGEQ)
took 218,210 microseconds (0.218210 seconds) to run.
       4,523 microseconds (0.004523 seconds, 2.07%) of which was spent in GC.
During that period, and with 4 available CPU cores,
     179,988 microseconds (0.179988 seconds) were spent in user mode
      36,664 microseconds (0.036664 seconds) were spent in system mode
 2,175,856 bytes of memory allocated.
T
? (time (asdf:load-system :messageq))
(ASDF:LOAD-SYSTEM :MESSAGEQ)
took 2,198,531 microseconds (2.198531 seconds) to run.
        49,693 microseconds (0.049693 seconds, 2.26%) of which was spent in GC.
During that period, and with 4 available CPU cores,
     1,839,880 microseconds (1.839880 seconds) were spent in user mode
       353,310 microseconds (0.353310 seconds) were spent in system mode
 21,279,072 bytes of memory allocated.
T

But then again, for perspective ("LISTS" is the last module in the list.)

? (time (require "LISTS"))
(REQUIRE "LISTS")
took 138 microseconds (0.000138 seconds) to run.
During that period, and with 4 available CPU cores,
       0 microseconds (0.000000 seconds) were spent in user mode
       0 microseconds (0.000000 seconds) were spent in system mode
 1,296 bytes of memory allocated.
"LISTS"
NIL

? (time (member "LISTS" *modules* :test 'equal))
(MEMBER "LISTS" *MODULES* :TEST 'EQUAL)
took 51 microseconds (0.000051 seconds) to run.
During that period, and with 4 available CPU cores,
      0 microseconds (0.000000 seconds) were spent in user mode
      0 microseconds (0.000000 seconds) were spent in system mode
("LISTS")
?

And one more, time to handle missing module:

 (time (ignore-errors (require "list")))
(IGNORE-ERRORS (REQUIRE "list"))
took 100,039 microseconds (0.100039 seconds) to run.
       5,587 microseconds (0.005587 seconds, 5.58%) of which was spent in GC.
During that period, and with 4 available CPU cores,
      73,329 microseconds (0.073329 seconds) were spent in user mode
      26,665 microseconds (0.026665 seconds) were spent in system mode
 1,318,768 bytes of memory allocated.
 2 minor page faults, 0 major page faults, 0 swaps.

? (time (ignore-errors (asdf:load-system "notamodule")))
(IGNORE-ERRORS (ASDF:LOAD-SYSTEM "notamodule"))
took 6,925 microseconds (0.006925 seconds) to run.
During that period, and with 4 available CPU cores,
     6,666 microseconds (0.006666 seconds) were spent in user mode
         0 microseconds (0.000000 seconds) were spent in system mode
 52,880 bytes of memory allocated.
NIL
Component "notamodule" not found

? (time (ignore-errors (asdf:require-system "notamodule")))
(IGNORE-ERRORS (ASDF:REQUIRE-SYSTEM "notamodule"))
took 210,796 microseconds (0.210796 seconds) to run.
       4,817 microseconds (0.004817 seconds, 2.29%) of which was spent in GC.
During that period, and with 4 available CPU cores,
     179,988 microseconds (0.179988 seconds) were spent in user mode
      29,998 microseconds (0.029998 seconds) were spent in system mode
 2,210,896 bytes of memory allocated.
NIL
Component "notamodule" not found
?

? (time (member "notamodule" *modules* :test 'equal))
(MEMBER "notamodule" *MODULES* :TEST 'EQUAL)
took 61 microseconds (0.000061 seconds) to run.
During that period, and with 4 available CPU cores,
      0 microseconds (0.000000 seconds) were spent in user mode
      0 microseconds (0.000000 seconds) were spent in system mode
NIL

NIL
Erik.

On Tue, Apr 24, 2012 at 11:26 AM, Erik Pearson <erik at defun-web.com> wrote:
> Magically minimal. It looks great!
>
> On Tue, Apr 24, 2012 at 9:36 AM, Faré <fahree at gmail.com> wrote:
>>> Basically, yes. I would state, though, that I want the ASDF to work
>>> like the primary contract of REQUIRE. That is, this clause from CLHS:
>>>
>>> require tests for the presence of the module-name in the list held by
>>> *modules*. If it is present, require immediately returns.
>>>
>>> I think that ASDF's default mode of grokking the filesystem for
>>> changes is what I want in a development environment.
>>>
>> Is this what you want?
>> Should I commit that (exporting the symbols) as asdf 2.20.22?
>>
>> (in-package :asdf)
>> (defun component-loaded-p (c)
>>  (and (gethash 'load-op (component-operation-times (find-component c nil))) t))
>> (defun loaded-systems ()
>>  (remove-if-not 'component-loaded-p (registered-systems)))
>> (defun require-system (s)
>>  (load-system s :force-not (loaded-systems)))
>>
>>> I really appreciate the responses and the new code to play with. That
>>> really is exceptional. I'll send some feedback later, hopefully today,
>>> but I have a message queue processing system to build and a web site
>>> to get up :(
>>>
>> Have fun!
>>
>> —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org
>> You can only find happiness by striving towards something else.