[drakma-devel] Closing streams with :want-stream t

[Edi Weitz]

On Fri, 02 Feb 2007 18:27:34 -0800, Chris Dean <ctdean at sokitomi.com> wrote:

> I have a file handle leak in my code (that is, I run out of file
> handles after running for a while) and I suspect that I'm not using
> drakma correctly.
>
> I wish to use the :want-stream t parameter and read the resulting
> stream directly, so I have this code:
>
>   (defun simple-get (url)
>     "Download the url using GET and return the body as a string."
>     (handler-case
>         (multiple-value-bind (stream code headers dummy-uri dummy-stream 
>                                      must-close?)
>             (drakma:http-request url :want-stream t :keep-alive nil 
>                                      :method :get)
>           (declare (ignore headers dummy-stream dummy-uri))
>           (unwind-protect 
>                (and stream
>                     code
>                     (= code 200)
>                     (with-output-to-string (out)
>                       (do ((ch (read-char stream nil :eof) 
>                                (read-char stream nil :eof)))
>                           ((not (characterp ch)))
>                         (princ ch out))))
>             (when (and stream must-close?)
>               (ignore-errors (close stream)))))
>       (error (condition) 
>         (format t "Error ~s: ~a~%" url condition)
>         nil)))
>
> Is there anything extra I need to do to make sure that all the
> streams opened by drakma are closed?
>
> My production code is much more complex, but the simple stub above
> will generate the out of file handles problem.  Besides the actual
> error I can use lsof on Linux and Mac OS and see many sockets stuck
> in CLOSED or CLOSE_WAIT states.  This is all under LispWorks 5.0.1
>
> Also, when using :want-stream nil I never encounter the problem.

The meaning of the sixth return value (MUST-CLOSE) is that you're not
allowed to re-use the stream, because according to the reply headers
the server will close the stream on its side.

However, if you do /not/ want to re-use the stream (which is obviously
the case in your example as your function doesn't return the stream),
you must of course always close it.  Drakma can't close it for you as
it doesn't know when you're done with it, and why would you want to
keep an open stream hanging around in your image that can't be
accessed by your code anyway?

In other words: It should be

              (when stream
                (ignore-errors (close stream)))))

above.

I'll re-word this in the documentation to make it more clear
(hopefully).

HTH,
Edi.