[hunchentoot-devel] hunchentoot file upload performance

[Mac Chan]

On 11/15/06, Edi Weitz <edi at agharta.de> wrote:
> Actually, the more I think about it the more I'm sure that
> FLEXI-STREAMS is the culprit.  I also have an idea how to make it
> faster, but I'm not sure if I'll find the time to do it in the next
> days.

No hurry, Edi. As long as we know there's a solution I'm at ease :-)

> > Again, my guess is that rfc2388 repeatedly call read-char instead of
> > grabbing a buffer with read-sequence and decode it as a chunk.
>
> Because of the way the streams are layered now, you probably wouldn't
> win much (if anything at all) if you used READ-SEQUENCE instead.

So I just did some profiling earlier using the tip from

http://thread.gmane.org/gmane.lisp.lispworks.general/5563/focus=5573

and found that most time are spent in tons of calls to read-char.

I then tried something like reading the whole buffer into a string and
use the (parse-mime string) method instead of (parse-mime stream)
(this is not a fix because if you upload 100mb file, it will allocate
a 100mb buffer).

The result is worse... will investigate later. Also I plan do similar
test with allegro-serve and see if it uses 100% cpu performing the
same task.