[hunchentoot-devel] Static file handling performance

[Andrei Stebakov]

I did some tests with httperf:

*First test with file-to-string function:*

Total: connections 100 requests 100 replies 89 test-duration 67.869 s

Connection rate: 1.5 conn/s (678.7 ms/conn, <=1 concurrent connections)
Connection time [ms]: min 492.0 avg 638.7 max 1151.0 median 572.5 stddev
150.4
Connection time [ms]: connect 1.3
Connection length [replies/conn]: 1.000

Request rate: 1.5 req/s (678.7 ms/req)
Request size [B]: 93.0

Reply rate [replies/s]: min 1.0 avg 1.3 max 1.8 stddev 0.2 (13 samples)
Reply time [ms]: response 511.0 transfer 126.4
Reply size [B]: header 368.0 content 21324.0 footer 0.0 (total 21692.0)
Reply status: 1xx=0 2xx=89 3xx=0 4xx=0 5xx=0

CPU time [s]: user 28.77 system 38.89 (user 42.4% system 57.3% total 99.7%)
Net I/O: 27.9 KB/s (0.2*10^6 bps)

Errors: total 11 client-timo 11 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0



*Second test with handle-static-file:*

Total: connections 100 requests 100 replies 16 test-duration 93.867 s

Connection rate: 1.1 conn/s (938.7 ms/conn, <=1 concurrent connections)
Connection time [ms]: min 594.0 avg 807.4 max 1393.0 median 690.5 stddev
240.2
Connection time [ms]: connect 0.6
Connection length [replies/conn]: 1.000

Request rate: 1.1 req/s (938.7 ms/req)
Request size [B]: 93.0

Reply rate [replies/s]: min 0.0 avg 0.2 max 1.0 stddev 0.3 (18 samples)
Reply time [ms]: response 637.1 transfer 166.0
Reply size [B]: header 348.0 content 21324.0 footer 0.0 (total 21672.0)
Reply status: 1xx=0 2xx=16 3xx=0 4xx=0 5xx=0

CPU time [s]: user 40.12 system 53.39 (user 42.7% system 56.9% total 99.6%)
Net I/O: 3.7 KB/s (0.0*10^6 bps)

Errors: total 84 client-timo 79 socket-timo 0 connrefused 0 connreset 5
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

As can be seen from the second test there were lots of time-out requests.
Also as I was watching the CPU usage (with top) during first test CPU load
was ~2%, during second test the CPU load was ~90% with memory constantly
growing and shrinking (probably GC was causing this load). Also during some
of handle-static-file tests I ran into heap allocation problems so I had to
unload the lisp image and restart it.
Well, it could be just my particular system's problem.

Andrew

On Fri, Jun 20, 2008 at 2:20 PM, Edi Weitz <edi at agharta.de> wrote:

> On Fri, 20 Jun 2008 12:56:33 -0400, "Andrei Stebakov" <lispercat at gmail.com>
> wrote:
>
> > I had to come up with some way to cache dynamic files that I have to
> > serve, so I ended up with a bunch of static files which I served by
> > a simple function:
> > (defun file-to-string (path)
> >   "Reads a file into a string"
> >   (if (probe-file path)
> >       (with-open-file (in path)
> >         (let ((str (make-string (file-length in))))
> >           (read-sequence str in)
> >           str))))
> >
> > The performance was very good, but then I thought that it's not the
> > proper way to serve static files as there is a hunchentoot function
> > handle-static-file.
> > When I started using the hunchentoot's function the response time
> > almost tripled and when I run "top" program to monitor CPU usage it
> > jumps up to 60% (on my PIII 600 MHz) CPU, whereas using
> > file-to-string CPU usage stays with 2% (maybe because the serving
> > time is much shorter top doesn't catch that CPU peak).
>
> How did you measure the response time?  Which version of FLEXI-STREAMS
> are you using?  Did you try with the development version?
>
> > My question is what could be the reason I see this behaviour? (I am
> > using SBCL 1.0.15 with latest dependences of hunchentoot-0.15.7)
>
> Look at the source code of handle-static-file.  It uses a fixed size
> buffer which is likely smaller than your file.
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/tbnl-devel/attachments/20080620/33aa8de3/attachment.html>