Parsing big XML files with klacks and sbcl

[Mark Janssen]

On Tue, May 29, 2018 at 9:20 PM, Attila Lendvai <attila at lendvai.name> wrote:
>> (loop while t do
>>       (klacks:consume *src*))
>
> try to add (sb-ext:gc :full t) inside the loop. if that helps, then
> you're overwhelming SBCL's gc algorithm by allocating too much garbage
> between two gc's (or something along that line, maybe someone else
> with more knowledge of the details can elaborate).
>

This indeed keeps the memory usage in check.
However a forced gc on every loop sounds less than ideal.
I am a bit surprised that a streaming parser generates so much
garbage, considering on of the main use cases is handling large files.
Also I am wondering if the GC can be configured to run more
aggressively without further explicit calls in the rest of the code.