[tbnl-devel] fixed: tbnl / sbcl binary upload / download issues

[Travis Cross]

TBNL has played poorly with SBCL up to this point with regard to binary
data.  Uploading binary files never worked at all, and only 1/2 of the
binary download examples appeared to work out of the box.

As it turns out, these problems are not related to code in current
versions of TBNL itself, but in TBNL's dependencies: KMRCL and rfc2388.

Attached are the necessary patches.

*Changes*

-- kmrcl:
Prior to this patch, kmrcl opened streams in SBCL with :element-type
'base-char.  This effectively prevented streams from writing character
values above 127 or operating in a bivalent fashion.  SBCL will open
bivalent streams when :element-type :default is specified.

-- rfc2388
rfc2388 opens a file stream to save attachments.  If the
:external-format option is not given to OPEN, SBCL will default to
:ascii and only write characters up to 127.  The :external-format
:latin-1 option needs to be passed to OPEN.

*Testing*

The "Binary data, delivered from file", "Binary data, delivered from
RAM", and "File Upload" examples in the tbnl-test package all work
correctly now with SBCL-0.9.7, both when using TBNL directly (without a
front-end) and with mod_lisp2 / Apache2.

For binary upload testing, I uploaded a 165MB video.  I then downloaded
the video through the tbnl interface (with Firefox) and ensured that the
hashes of the two files matched.  I was a bit surprised, actually, that
TBNL / SBCL didn't choke on the size of the file ;)

To make sure these changes were production-ready, I ran ApacheBench2
against the binary image downloads:

/usr/sbin/ab2 -c 50 -n 1000 http://server/tbnl/test/image-ram.jpg
Complete requests:      1000
Failed requests:        0
Requests per second:    343.94 [#/sec] (mean)
Time per request:       145.373 [ms] (mean)

/usr/sbin/ab2 -c 50 -n 1000 http://server/tbnl/test/image.jpg
Complete requests:      1000
Failed requests:        0
Requests per second:    237.63 [#/sec] (mean)
Time per request:       210.412 [ms] (mean)

As far as I can tell, these changes make SBCL a fully supported platform
for TBNL.

Cheers,

-- Travis

PS - I should note a few things here for reference:
-- :tbnl-bivalent-streams needs to be enabled in *features*.
-- The unicode tests in the TBNL suite appear to work correctly.
-- Contrary to our prior expectations, the underlying issue was not
related to handling sb-unicode appropriately, and in fact, binary
handling does not seem to rely on sb-unicode at all.
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: kmrcl-1.84-sbcl-bivalent-streams-patch
URL: <https://mailman.common-lisp.net/pipermail/tbnl-devel/attachments/20051219/37a47ebe/attachment.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: rfc2388-sbcl-unicode-patch
URL: <https://mailman.common-lisp.net/pipermail/tbnl-devel/attachments/20051219/37a47ebe/attachment-0001.ksh>