Parsing big XML files with klacks and sbcl

[Attila Lendvai]

> This indeed keeps the memory usage in check.
> However a forced gc on every loop sounds less than ideal.


you can play with other options. a forced full gc is slow indeed, but
if you know your load characteristics it's not unreasonable to give
explicit notifications to the gc when to run and on which generations.


> I am a bit surprised that a streaming parser generates so much
> garbage, considering on of the main use cases is handling large files.
> Also I am wondering if the GC can be configured to run more
> aggressively without further explicit calls in the rest of the code.


IIRC maybe you can play with the configuration of strings? if you
don't need to deal with unicode content, then you can maybe spare some
memory by using CL:STRING instead of cxml's own unicode support? i
seem to remember something like this in cxml. they are called RUNEs
and RODs?

hth,

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
Cannot ever have anything resembling a free market when money is
interest-bearing debt forced into circulation at gunpoint.