[drakma-devel] Sending content without loading it all in memory?

[Zach Beane]

On Wed, Sep 10, 2008 at 05:03:33PM +0200, Edi Weitz wrote:
> On Wed, 10 Sep 2008 09:20:30 -0400, Zach Beane <xach at xach.com> wrote:
> 
> > Is there any way to send a non-chunked request without having the
> > entire content in memory at once?
> 
> You can set the content length header yourself.  In that case Drakma
> shouldn't use chunked encoding IIRC.
> 
>   http://weitz.de/drakma/#content-length

Unfortunately that's not the case, as the documentation says:

| A non-NIL content-length argument means that Drakma must build the
| request body in RAM and compute the content length even if it would
| have otherwise used chunked encoding - for example in the case of
| file uploads.

The code says it too:

   (when (and content-length
	      (not (or (arrayp content)
		       (listp content)
		       (eq content :continuation))))
     ;; CONTENT-LENGTH forces us to compute request body
     ;; in RAM
     ...)

> > I'm working on an Amazon S3 client based on Drakma, but sometimes I
> > want to upload (via PUT) very large files and I'd prefer to
> > determine the content-length via FILE-LENGTH and upload the body
> > buffer-by-buffer instead of all at once.
> 
> That should be no problem if you provide a function as the :content
> keyword argument or use the "continuation" facility.

I tried that too, but:

  (when (and (eq content :continuation) content-length)
    (error "CONTENT-LENGTH must be NIL if CONTENT is :CONTINUATION."))

I think I might be out of luck.

Zach