[hunchentoot-devel] hunchentoot file upload performance

[Edi Weitz]

On Tue, 14 Nov 2006 18:10:52 -0800, "Mac Chan" <emailmac at gmail.com> wrote:

> I've been using tbnl behind mod_lisp2 for quite a while.
>
> After Edi merged tbnl with hunchentoot I wanted to get rid of apache
> and run hunchentoot in standalone mode for easier deployment (not to
> mention the coolness factor of running everything in lisp).
>
> However, I've found that the file upload performance of hunchentoot
> is 4-10 times slower than tbnl (standalone or behind mod_lisp). I
> tested this by uploading a 30mb file in the test upload page.

Yes, I've seen the same.

> I'm guessing that Chunga might be the reason, since this is the only
> new component introduced.

That would be my guess as well - Chunga and/or FLEXI-STREAMS.
Including the fact that both Chunga and FLEXI-STREAMS make heavy use
of Gray streams.

> Again, my guess is that rfc2388 repeatedly call read-char instead of
> grabbing a buffer with read-sequence and decode it as a chunk.

That is certainly part of the problem.

The only way out is probably to write our own version of the RFC2388
library - which is one of my long-term plans.

> These two issues make migrating to hunchentoot particular painful
> because if one of the users uploads a huge file the whole site will
> become very unresponsive (in tbnl the cpu spike goes away a lot
> faster, but improving the i/o routine is still a big win).
>
> I don't want to blindly guess, but I don't know how to use
> lispworks' profiler to profile a multi-threaded server app.
>
> The lispworks profiler requires you to run (profile <forms>) and it
> will return the profiling data after running the <form>.
>
> Is there a way to profile hunchentoot without writing individual
> test case that simulates uploading a 30 mb file?

I'd like to know the answer too... :)

I think you better chances to get an answer to this question on the LW
mailing list.

Cheers,
Edi.