From xach at xach.com  Wed May  3 14:29:05 2006
From: xach at xach.com (Zach Beane)
Date: Wed, 3 May 2006 10:29:05 -0400
Subject: [tbnl-devel] large file uploads
Message-ID: <20060503142905.GX6686@xach.com>

I would like to prevent files over a certain size from being
uploaded. I don't want to check after the fact; I'd like to abort
request processing as soon as the content-length is known to be too
large.

Is it ok to close the content-stream and error out of
initialize-instance on the request? Is there a better way?

Zach


From edi at agharta.de  Wed May  3 15:32:26 2006
From: edi at agharta.de (Edi Weitz)
Date: Wed, 03 May 2006 17:32:26 +0200
Subject: [tbnl-devel] large file uploads
In-Reply-To: <20060503142905.GX6686@xach.com> (Zach Beane's message of "Wed,
	3 May 2006 10:29:05 -0400")
References: <20060503142905.GX6686@xach.com>
Message-ID: <umzdzp1it.fsf@agharta.de>

On Wed, 3 May 2006 10:29:05 -0400, Zach Beane <xach at xach.com> wrote:

> I would like to prevent files over a certain size from being
> uploaded. I don't want to check after the fact; I'd like to abort
> request processing as soon as the content-length is known to be too
> large.
>
> Is it ok to close the content-stream and error out of
> initialize-instance on the request? Is there a better way?

I would assume that it is "OK" in the sense that it doesn't wreak
havoc.  You'll be caught by the IGNORE-ERRORS in LISTEN-FOR-REQUEST
which in turn will close the stream.  If you're behind Apache, it will
then as a result of this sooner or later close the stream to the
client which is still trying to send stuff.  Of course, there's no way
to re-use these streams and/or to send a message to the client.

It's not pretty, though, but OTOH I think there /is/ no pretty way of
doing this.  Do you want to do this as some kind of anti-DoS measure?

Cheers,
Edi.


From xach at xach.com  Wed May  3 15:41:15 2006
From: xach at xach.com (Zach Beane)
Date: Wed, 3 May 2006 11:41:15 -0400
Subject: [tbnl-devel] large file uploads
In-Reply-To: <umzdzp1it.fsf@agharta.de>
References: <20060503142905.GX6686@xach.com> <umzdzp1it.fsf@agharta.de>
Message-ID: <20060503154115.GZ6686@xach.com>

On Wed, May 03, 2006 at 05:32:26PM +0200, Edi Weitz wrote:
> > Is it ok to close the content-stream and error out of
> > initialize-instance on the request? Is there a better way?
> 
> I would assume that it is "OK" in the sense that it doesn't wreak
> havoc.  You'll be caught by the IGNORE-ERRORS in LISTEN-FOR-REQUEST
> which in turn will close the stream.  If you're behind Apache, it will
> then as a result of this sooner or later close the stream to the
> client which is still trying to send stuff.  Of course, there's no way
> to re-use these streams and/or to send a message to the client.
> 
> It's not pretty, though, but OTOH I think there /is/ no pretty way of
> doing this.  Do you want to do this as some kind of anti-DoS measure?

Yes, though not an intentional DoS. My application accepts image files
and only processes files of a reasonable size. Allowing uploads of
very large files only to reject them later in the process consumes
CPU, bandwidth, and memory that could be used elsewhere.

I have a local patch for this (the large files were also crashing
SBCL, so I needed to do something quickly). If I figure out a nice
way to do it, I will send along a TBNL patch too.

Zach


From luke at balooga.com  Fri May 12 02:01:08 2006
From: luke at balooga.com (Luke J Crook)
Date: Thu, 11 May 2006 19:01:08 -0700
Subject: [tbnl-devel] Re: large file uploads
In-Reply-To: <20060503154115.GZ6686@xach.com>
References: <20060503142905.GX6686@xach.com> <umzdzp1it.fsf@agharta.de>
	<20060503154115.GZ6686@xach.com>
Message-ID: <4463EC64.2010707@balooga.com>

Zach Beane wrote:
> On Wed, May 03, 2006 at 05:32:26PM +0200, Edi Weitz wrote:

> Yes, though not an intentional DoS. My application accepts image files
> and only processes files of a reasonable size. Allowing uploads of
> very large files only to reject them later in the process consumes
> CPU, bandwidth, and memory that could be used elsewhere.

Could you use Javascript on the client to check the file size before 
allowing the upload?

Something like the following:

http://www.cs.tut.fi/~jkorpela/forms/file.html#restr


-Luke




From jwhbates at gmail.com  Fri May 12 05:21:21 2006
From: jwhbates at gmail.com (John Bates)
Date: Fri, 12 May 2006 01:21:21 -0400
Subject: [tbnl-devel] application/x-www-form-urlencoded
Message-ID: <3a1033990605112221h3cefa921v374823d4b786b6a0@mail.gmail.com>

I'm new to this framework and web development in general: I ran across
this problem today, and I think it's a bug (and I kinda fixed it), but
I'm not sure...

Anyway, using Laszlo as a front-end, I tried to post to a url served
by tbnl. Couldn't get at the POST data, couldn't get at it, couldn't
get at it. Finally, I noticed two things:

1) the header from laszlo for content-type was:
"application/x-www-form-urlencoded;charset=UTF-8"
2) in initialize-instance and recompute-request-parameters, the code was:

 (string-equal "application/x-www-form-urlencoded"
                              (string-assoc "content-type" headers-in))

So I tossed in:

 (string-equal "application/x-www-form-urlencoded"
                              (string-assoc "content-type" headers-in)
                              :end2 (length
"application/x-www-form-urlencoded))

which is not particularly efficient, but is at least correct.

Am I wrong?


From edi at agharta.de  Fri May 12 09:32:04 2006
From: edi at agharta.de (Edi Weitz)
Date: Fri, 12 May 2006 11:32:04 +0200
Subject: [tbnl-devel] New version 0.9.9 (Was:
	application/x-www-form-urlencoded)
In-Reply-To: <3a1033990605112221h3cefa921v374823d4b786b6a0@mail.gmail.com>
	(John Bates's message of "Fri, 12 May 2006 01:21:21 -0400")
References: <3a1033990605112221h3cefa921v374823d4b786b6a0@mail.gmail.com>
Message-ID: <ulkt7r3l7.fsf@agharta.de>

On Fri, 12 May 2006 01:21:21 -0400, "John Bates" <jwhbates at gmail.com> wrote:

> 1) the header from laszlo for content-type was:
> "application/x-www-form-urlencoded;charset=UTF-8"

Thanks for the report.  I've uploaded a new version (untested) which
should fix this.  It's kind of a workaround because actually for both
content types TBNL should do the right thing immediately if the
charset is explicitely specified.  But that would involve a major
change and including something like FLEXI-STREAMS and I don't have the
time for that now.

> Am I wrong?

No, your fix looked OK.  You didn't notice STARTS-WITH-P, though.

Cheers,
Edi.


From tobia.conforto at linux.it  Fri May 12 09:48:17 2006
From: tobia.conforto at linux.it (Toby)
Date: Fri, 12 May 2006 11:48:17 +0200
Subject: [tbnl-devel] application/x-www-form-urlencoded
In-Reply-To: <3a1033990605112221h3cefa921v374823d4b786b6a0@mail.gmail.com>
References: <3a1033990605112221h3cefa921v374823d4b786b6a0@mail.gmail.com>
Message-ID: <20060512094817.GL20880@localhost.localdomain>

John Bates wrote:
> the header from laszlo for content-type was:
> "application/x-www-form-urlencoded;charset=UTF-8"

FYI, according to RFC 2616 HTTP/1.1, what follows Content-Type is:

	type "/" subtype *( ";" attribute "=" value )

So if TBNL can't parse that, I's say it's a bug.
They also seem to say that compliant HTTP/1.1 software should treat a
missing charset parameter as ISO-8859-1, but I'm not sure.


Toby


From quasilists at gmail.com  Fri May 12 08:57:38 2006
From: quasilists at gmail.com (quasi)
Date: Fri, 12 May 2006 14:27:38 +0530
Subject: [tbnl-devel] multiple tbnl's
Message-ID: <87lkt7tybh.fsf@agni4.cltp.com>

hey folks,

Is it possible to run multiple tbnl instances behind the same
apache+mod_lisp ?  I mean, it is possible to have differnt lisp images
with tbnl running behind one apache+mod_lisp ?  This would mean that
the name of the tbnl in the apache.conf and the port, both have to be
different for the different images, right ?

thanks,

-- 
quasi

Utopia Unlimited!


From edi at agharta.de  Fri May 12 12:18:24 2006
From: edi at agharta.de (Edi Weitz)
Date: Fri, 12 May 2006 14:18:24 +0200
Subject: [tbnl-devel] multiple tbnl's
In-Reply-To: <87lkt7tybh.fsf@agni4.cltp.com> (quasi's message of "Fri, 12
	May 2006 14:27:38 +0530")
References: <87lkt7tybh.fsf@agni4.cltp.com>
Message-ID: <u8xp71lnz.fsf@agharta.de>

Hi!

On Fri, 12 May 2006 14:27:38 +0530, quasi <quasilists at gmail.com> wrote:

> Is it possible to run multiple tbnl instances behind the same
> apache+mod_lisp ?

Have you read the docs?  See the second paragraph of "General
overview."

> I mean, it is possible to have differnt lisp images with tbnl
> running behind one apache+mod_lisp ?  This would mean that the name
> of the tbnl in the apache.conf and the port, both have to be
> different for the different images, right ?

  <http://groups.google.com/group/comp.lang.lisp/browse_frm/thread/ff278732f19759aa/838eab482a543f43>

Cheers,
Edi.


From xach at xach.com  Fri May 12 12:49:55 2006
From: xach at xach.com (Zach Beane)
Date: Fri, 12 May 2006 08:49:55 -0400
Subject: [tbnl-devel] Re: large file uploads
In-Reply-To: <4463EC64.2010707@balooga.com>
References: <20060503142905.GX6686@xach.com> <umzdzp1it.fsf@agharta.de>
	<20060503154115.GZ6686@xach.com> <4463EC64.2010707@balooga.com>
Message-ID: <20060512124955.GL6686@xach.com>

On Thu, May 11, 2006 at 07:01:08PM -0700, Luke J Crook wrote:
> Zach Beane wrote:
> >On Wed, May 03, 2006 at 05:32:26PM +0200, Edi Weitz wrote:
> 
> >Yes, though not an intentional DoS. My application accepts image files
> >and only processes files of a reasonable size. Allowing uploads of
> >very large files only to reject them later in the process consumes
> >CPU, bandwidth, and memory that could be used elsewhere.
> 
> Could you use Javascript on the client to check the file size before 
> allowing the upload?
> 
> Something like the following:
> 
> http://www.cs.tut.fi/~jkorpela/forms/file.html#restr

Hmm, that page seems to suggest that there is no reliable way to do it
on the client side.

Zach


From ygingras at ygingras.net  Sun May 14 15:52:51 2006
From: ygingras at ygingras.net (Yannick Gingras)
Date: Sun, 14 May 2006 11:52:51 -0400
Subject: [tbnl-devel] send-headers and slow page generation
Message-ID: <m2u07ssiwc.fsf@ogre.ygingras.net>


Hi, 

I'd like to use TBNL for a web application that have a really slow
page generation rate.  At first I thought that send-headers was the
solution: write the page header and navigation before launching the
slow content generation and then write the content as it is generated.
I made this small example (using cl-who):

(defun test-page ()
  (with-html-output (*standard-output* (send-headers) :prologue t :indent t)
    (:html (:body (:p (sleep 3) (fmt "hi, i'm the ~dth loop" 0))
		  (:p (sleep 3) (fmt "hi, i'm the ~dth loop" 1))
		  (:p (sleep 3) (fmt "hi, i'm the ~dth loop" 2))
		  (finish-output *standard-output*)
		  (:p (sleep 3) (fmt "hi, i'm the ~dth loop" 3))
		  (:p (sleep 3) (fmt "hi, i'm the ~dth loop" 4))
		  (:p (sleep 3) (fmt "hi, i'm the ~dth loop" 5))))))

Running it (without the call to send-headers) in my REPL have the
desired result: each part is printer before the next call to sleep.
On the other hand, when I try it live with TBNL I have to wait for the
whole page to be generated.  Maybe is misinterpreted the
documentation.  Is it possible to write partial pages to the client
stream with TBNL?

I'm using apache2 with the latest mod_lisp2 and TBNL 0.9.9 on a
Debian Etch system.

-- 
Yannick Gingras


From edi at agharta.de  Sun May 14 19:02:10 2006
From: edi at agharta.de (Edi Weitz)
Date: Sun, 14 May 2006 21:02:10 +0200
Subject: [tbnl-devel] send-headers and slow page generation
In-Reply-To: <m2u07ssiwc.fsf@ogre.ygingras.net> (Yannick Gingras's message
	of "Sun, 14 May 2006 11:52:51 -0400")
References: <m2u07ssiwc.fsf@ogre.ygingras.net>
Message-ID: <umzdkjuq5.fsf@agharta.de>

On Sun, 14 May 2006 11:52:51 -0400, Yannick Gingras <ygingras at ygingras.net> wrote:

> I'd like to use TBNL for a web application that have a really slow
> page generation rate.  At first I thought that send-headers was the
> solution: write the page header and navigation before launching the
> slow content generation and then write the content as it is
> generated.  I made this small example (using cl-who):
>
> (defun test-page ()
>   (with-html-output (*standard-output* (send-headers) :prologue t :indent t)
>     (:html (:body (:p (sleep 3) (fmt "hi, i'm the ~dth loop" 0))
> 		  (:p (sleep 3) (fmt "hi, i'm the ~dth loop" 1))
> 		  (:p (sleep 3) (fmt "hi, i'm the ~dth loop" 2))
> 		  (finish-output *standard-output*)
> 		  (:p (sleep 3) (fmt "hi, i'm the ~dth loop" 3))
> 		  (:p (sleep 3) (fmt "hi, i'm the ~dth loop" 4))
> 		  (:p (sleep 3) (fmt "hi, i'm the ~dth loop" 5))))))
>
> Running it (without the call to send-headers) in my REPL have the
> desired result: each part is printer before the next call to sleep.
> On the other hand, when I try it live with TBNL I have to wait for
> the whole page to be generated.  Maybe is misinterpreted the
> documentation.  Is it possible to write partial pages to the client
> stream with TBNL?
>
> I'm using apache2 with the latest mod_lisp2 and TBNL 0.9.9 on a
> Debian Etch system.

Your information is sent in chunks to mod_lisp, but that's not what
your browser sees.  It sees what Apache will eventually send.

You can use SEND-HEADERS as described above to achieve what you want,
but only if you use TBNL in "stand-alone mode" or if you use a
front-end like Hunchentoot.  (I'm not sure about Araneida, maybe
that'll also work.)

Cheers,
Edi.


From edi at agharta.de  Sun May 14 19:43:07 2006
From: edi at agharta.de (Edi Weitz)
Date: Sun, 14 May 2006 21:43:07 +0200
Subject: [tbnl-devel] send-headers and slow page generation
In-Reply-To: <m2ody0s9ge.fsf@ogre.ygingras.net> (Yannick Gingras's message
	of "Sun, 14 May 2006 15:16:49 -0400")
References: <m2u07ssiwc.fsf@ogre.ygingras.net> <umzdkjuq5.fsf@agharta.de>
	<m2ody0s9ge.fsf@ogre.ygingras.net>
Message-ID: <uac9kjstw.fsf@agharta.de>

[Please use the mailing list.]

On Sun, 14 May 2006 15:16:49 -0400, Yannick Gingras <ygingras at ygingras.net> wrote:

> I know when I finished to generate the fast content, is there a way
> to pack and send a mod_lisp chunk?  Something like FINISH-OUTPUT
> would be great.

I don't think so.  You'll have to modify mod_lisp to get what you
want.  See the function FlushLispBuffSocket.

> Otherwise using Hunchentoot is doable.

Note that Hunchentoot only works with LispWorks.


From ygingras at ygingras.net  Sun May 14 19:16:49 2006
From: ygingras at ygingras.net (Yannick Gingras)
Date: Sun, 14 May 2006 15:16:49 -0400
Subject: [tbnl-devel] send-headers and slow page generation
In-Reply-To: <umzdkjuq5.fsf@agharta.de> (Edi Weitz's message of "Sun,
	14 May 2006 21:02:10 +0200")
References: <m2u07ssiwc.fsf@ogre.ygingras.net> <umzdkjuq5.fsf@agharta.de>
Message-ID: <m2lkt4s8v9.fsf@ogre.ygingras.net>

Edi Weitz <edi at agharta.de> writes:

> Your information is sent in chunks to mod_lisp, but that's not what
> your browser sees.  It sees what Apache will eventually send.
>
> You can use SEND-HEADERS as described above to achieve what you want,
> but only if you use TBNL in "stand-alone mode" or if you use a
> front-end like Hunchentoot.  (I'm not sure about Araneida, maybe
> that'll also work.)

I know when I finished to generate the fast content, is there a way to
pack and send a mod_lisp chunk?  Something like FINISH-OUTPUT would be
great.  Otherwise using Hunchentoot is doable.

-- 
Yannick Gingras


From ygingras at ygingras.net  Sun May 14 20:17:26 2006
From: ygingras at ygingras.net (Yannick Gingras)
Date: Sun, 14 May 2006 16:17:26 -0400
Subject: [tbnl-devel] send-headers and slow page generation
In-Reply-To: <uac9kjstw.fsf@agharta.de> (Edi Weitz's message of "Sun,
	14 May 2006 21:43:07 +0200")
References: <m2u07ssiwc.fsf@ogre.ygingras.net> <umzdkjuq5.fsf@agharta.de>
	<m2ody0s9ge.fsf@ogre.ygingras.net> <uac9kjstw.fsf@agharta.de>
Message-ID: <m2fyjcs6nd.fsf@ogre.ygingras.net>

Edi Weitz <edi at agharta.de> writes:

>> I know when I finished to generate the fast content, is there a way
>> to pack and send a mod_lisp chunk?  Something like FINISH-OUTPUT
>> would be great.
>
> I don't think so.  You'll have to modify mod_lisp to get what you
> want.  See the function FlushLispBuffSocket.

A quick look at mod_lisp2 (no FlushLispBuffSocket there) tells me that
the buffer size is 4096.  If I print at least that much between sleeps
I have the desired result: the browser receives a part of the page
then waits for the rest.  For now I think I will just lower the
mod_lips2 buffer size, maybe something like 512 or 1024.  That should
be enough since the interesting part of the pages will be at least
that big and its ok to wait until you have a reasonable chunk of page
to show.

Thanks for the pointers!

-- 
Yannick Gingras


From edi at agharta.de  Sun May 14 20:38:25 2006
From: edi at agharta.de (Edi Weitz)
Date: Sun, 14 May 2006 22:38:25 +0200
Subject: [tbnl-devel] send-headers and slow page generation
In-Reply-To: <m2fyjcs6nd.fsf@ogre.ygingras.net> (Yannick Gingras's message
	of "Sun, 14 May 2006 16:17:26 -0400")
References: <m2u07ssiwc.fsf@ogre.ygingras.net> <umzdkjuq5.fsf@agharta.de>
	<m2ody0s9ge.fsf@ogre.ygingras.net> <uac9kjstw.fsf@agharta.de>
	<m2fyjcs6nd.fsf@ogre.ygingras.net>
Message-ID: <u1wuwjq9q.fsf@agharta.de>

On Sun, 14 May 2006 16:17:26 -0400, Yannick Gingras <ygingras at ygingras.net> wrote:

> no FlushLispBuffSocket there

Right, sorry.  I was talking about mod_lisp1.

> Thanks for the pointers!

You're welcome... :)


From edi at agharta.de  Wed May 24 18:41:14 2006
From: edi at agharta.de (Edi Weitz)
Date: Wed, 24 May 2006 20:41:14 +0200
Subject: [tbnl-devel] New version 0.9.10
Message-ID: <uvervclkl.fsf@agharta.de>

ChangeLog:

  Version 0.9.10
  2006-05-24
  Prepare for LW 5.0 release

Download:

  http://weitz.de/files/tbnl.tar.gz


From edi at agharta.de  Thu May 25 10:10:10 2006
From: edi at agharta.de (Edi Weitz)
Date: Thu, 25 May 2006 12:10:10 +0200
Subject: [tbnl-devel] Re: TBNL newbie question
In-Reply-To: <ecb948b30605250254p1502dcc5se665632738b8efae@mail.gmail.com>
	(Yarek Kowalik's message of "Thu, 25 May 2006 05:54:19 -0400")
References: <ecb948b30605250254p1502dcc5se665632738b8efae@mail.gmail.com>
Message-ID: <ufyiye7p9.fsf@agharta.de>

Hi!

Please use the mailing list.

On Thu, 25 May 2006 05:54:19 -0400, "Yarek Kowalik" <yarek.kowalik at gmail.com> wrote:

> I'm really new to lisp, and definitely new to TBNL.  I'm having
> trouble getting TBLN respond to Apache, I wonder if you can point me
> to where I can get some help. I've been tryhing to figure out how to
> make this work for hours, and no luck, and it's getting really late
> (3:00 AM).  So, maybe you can help me?
>
>  Here is what is happening:
>
> 1.  I setup Apache 2 and it seems to be loading the mod_lisp2.so
> module fine, but then Apache it times out (I get 500 error) when
> trying to load the tbnl test page.  Not sure if TBNL is running OK
> (see below), or maybe Apache is misconfigured (unlikely - that seems
> easy).

Have you added the necessary parts for mod_lisp to the Apache
configuration?

Note that you can always work in TBNL's "stand-alone" mode to make
sure that the problem is not with Apache/mod_lisp.

> 2.  When I start TBNL, I get the following two errors - does that
> matter?
>
> [1]> (asdf:oos 'asdf:load-op :tbnl-test)
> ; loading system definition from /usr/share/common-lisp/systems/tbnl-
> test.asd into #<PACKAGE ASDF7345>
> ;; Loading file /usr/share/common-lisp/systems/tbnl-test.asd ...
>
> [...]
>
> ;; Loading file /var/cache/common-lisp-controller/yarek/clisp/kmrcl/os.fas
> ...
> ** - Continuable Error
> INTERN("PROGRAM-ID"): #<PACKAGE SYSTEM> is locked

See below.

> If you continue (by typing 'continue'): Ignore the lock and proceed
> The following restarts are also available:
> RETRY          :R1      Retry performing #<ASDF:LOAD-OP NIL #x000333CD05E0>
> on #<ASDF:CL-SOURCE-FILE "os" #x000333CCB428>.
> ACCEPT         :R2      Continue, treating #<ASDF:LOAD-OP NIL
> #x000333CD05E0> on #<ASDF:CL-SOURCE-FILE "os" #x000333CCB428> as having been
> successful.
> ABORT          :R3      ABORT
> Break 1 KL[2]> continue
>
> [....]
>
> ;; Loading file
> /var/cache/common-lisp-controller/yarek/clisp/tbnl/test/test.fas ...
> *** - OPEN: file
> #P"/var/cache/common-lisp-controller/yarek/clisp/tbnl/test/fz.jpg" does not
> exist

This seems to be a problem with the Debian distribution.

> The following restarts are available:
> RETRY          :R1      Retry performing #<ASDF:LOAD-OP NIL #x000333CBA7A0>
> on #<ASDF:CL-SOURCE-FILE "test" #x000333CBC380>.
> ACCEPT         :R2      Continue, treating #<ASDF:LOAD-OP NIL
> #x000333CBA7A0> on #<ASDF:CL-SOURCE-FILE "test" #x000333CBC380> as having
> been successful.
> ABORT          :R3      ABORT
> Break 1 TBNL-TEST[3]> accept
> 0 errors, 0 warnings
> NIL
> [4]> (tbnl:start-tbnl)
>
>  3. After typing "(tbnl:start-tbnl)" I get no clisp prompt - is this
> expected, or is it hung?

TBNL (or rather KMRCL) does not work with CLISP, see the docs.  Try
LispWorks, AllegroCL, SBCL, or CMUCL.

Cheers,
Edi.


From quasilists at gmail.com  Tue May 30 13:07:53 2006
From: quasilists at gmail.com (quasi)
Date: Tue, 30 May 2006 18:37:53 +0530
Subject: [tbnl-devel] tbnl/mod_lisp memory leak ?
Message-ID: <877j43psnq.fsf@agni4.cltp.com>

Hey,

I am using tbnl 0.9.10 and sbcl 0.9.11.  I have some functions which
generate HTML content using cl-who etc.  I have run then through heavy
testing and they work fine in the REPL (no memory leakage).

As soon as I am calling them via tbnl/mod_lisp the lisp memory usage
jumps tremendously.  I ran the ab2 benchmark (100 requests with
concurrency 1) and the memory usage went from 50Mb to 300Mb+.  It then
starts swapping.

Is this a known problem or am I doing something terribly wrong ?
-- 
quasi

Utopia Unlimited!


From edi at agharta.de  Tue May 30 17:56:12 2006
From: edi at agharta.de (Edi Weitz)
Date: Tue, 30 May 2006 18:56:12 +0100
Subject: [tbnl-devel] tbnl/mod_lisp memory leak ?
In-Reply-To: <877j43psnq.fsf@agni4.cltp.com> (quasi's message of "Tue, 30
	May 2006 18:37:53 +0530")
References: <877j43psnq.fsf@agni4.cltp.com>
Message-ID: <uejybz9ab.fsf@agharta.de>

On Tue, 30 May 2006 18:37:53 +0530, quasi <quasilists at gmail.com> wrote:

> I am using tbnl 0.9.10 and sbcl 0.9.11.  I have some functions which
> generate HTML content using cl-who etc.  I have run then through
> heavy testing and they work fine in the REPL (no memory leakage).
>
> As soon as I am calling them via tbnl/mod_lisp the lisp memory usage
> jumps tremendously.  I ran the ab2 benchmark (100 requests with
> concurrency 1) and the memory usage went from 50Mb to 300Mb+.  It
> then starts swapping.
>
> Is this a known problem or am I doing something terribly wrong ?

I'm not aware of any memory leaks in TBNL and it's hard to say if
you're doing something wrong without seeing your functions.  As a
simple test I'd suggest running the same example with another Lisp
that is known to work with TBNL, like CMUCL.  What you're seeing
/might/ be related to SBCL's threading implementation.  I don't know,
I usually don't use SBCL.

Good luck,
Edi.


From tobia.conforto at linux.it  Tue May 30 18:28:41 2006
From: tobia.conforto at linux.it (Toby)
Date: Tue, 30 May 2006 20:28:41 +0200
Subject: [tbnl-devel] tbnl/mod_lisp memory leak ?
In-Reply-To: <877j43psnq.fsf@agni4.cltp.com>
References: <877j43psnq.fsf@agni4.cltp.com>
Message-ID: <20060530182840.GE13563@localhost.localdomain>

quasi wrote:
> I have some functions which generate HTML content using cl-who etc.
> As soon as I am calling them via tbnl/mod_lisp the lisp memory usage
> jumps tremendously.

Can you try and benchmark a small function and see if the problem is
still there?  If so, then trim parts of it until you get either: 1. a
function that doesn't leak, in which case you can pinpoint where the
problem lies; or 2. a small function that still causes problems, which
you can post here.

In short: troubleshoot a bit more! :)


Toby

-- 
Signed/encrypted mail welcome.  GPG/PGP Key-Id: 0x15C5C2EA