[tbnl-devel] uploading a huge file / ajax progress bar

[Edi Weitz]

Hi!

On Thu, 9 Feb 2006 01:38:13 -0800, Mac Chan <emailmac at gmail.com> wrote:

> I notice that by uploading a 600mb file on win32 + lispworks + tbnl
> mod-lisp, the memory usage remains constant (ie. Lispworks is
> streaming the content directly to disk).
>
> However, with either freebsd/cmucl or linux/sbcl, top shows that all
> of the uploaded contents are first stored in memory and then get
> written to disk. This is a major scalability problem, I'm not sure
> if this is a design issue that can be fixed.

This should not happen.  Maybe you're using older versions of TBNL
here?  The code which is responsible for the streaming was
inadvertently left out from some released versions.

> I quickly went through the tbnl request post initialization routine
> and the rfc2388 module. It looks like what we need to do is to
> change rfc2388 to continuously write a report of the current upload
> progress to a file, and then add an entry point in tbnl to return a
> html page showing the current progress. This page can be launched by
> javascript when the upload form is submitted.
>
> Am I missing any important piece of information in this picture?  If
> anyone can give me more pointers I would try my hands on this as an
> exercise. I'm not sure if anyone would find this useful though.

What you describe seems to be the usual way to do it.  I also did
something similar once for a mod_perl application.

As far as changing RFC2388 is concerned, it would be nice if we could
convince Janis to add the changes to his distributions (and therefore
make them unobtrusive so they won't bother non-TBNL users).

Thanks,
Edi.