[hunchentoot-devel] Hunchentoot Errors (e.g. Couldn't write to #<SB-SYS:FD-STREAM for "socket ..." ...>)

Sun Oct 30 21:24:50 UTC 2011

Cyrus,

sorry that it took so long to respond to this.  The error handling
behavior in the v1.2.0 release has been much improved, so if you could
let me know if the problem still exists with the new version, it'd be
great.

Cheers,
Hans

On Tue, Apr 5, 2011 at 2:37 AM, Cyrus Harmon <ch-tbnl at bobobeach.com> wrote:
>
> Hello tbnl-folk,
>
> So, I've been seeing transient problems with hunchentoot for as long as I can remember, and I haven't had the chance to try to track down the problems until now. It seems there are a couple of classes of problems. In order of severity, the are:
>
> 1. things that crash hunchentoot/SBCL (or, in some cases, the host OS)
>
> 2. things that throw "unexpected" errors
>
> 3. things that throw "expected" errors (e.g. 404)
>
> As for class 1, these are tough to track down as the server is usually crashed by the time I get to it, but some causes I consider plausible are:
>
> * running out of some limited resource such as RAM, threads, or network sockets
> * possible SBCL bug in, say, signal handling or dealing with lots of threads
>
> I don't have any data to present on this class, at the moment, so let's move on to the next class.
>
> I often see errors such as this in the log:
>
> [2011-04-05 00:04:52 [ERROR]] Couldn't write to #<SB-SYS:FD-STREAM
>                    for "socket 74.115.254.12:80, peer: 217.72.218.248:2741"
>                    {B6FB859}>:
>  Broken pipe
> 0: (SB-DEBUG::MAP-BACKTRACE
>    #<CLOSURE (LAMBDA #) {B71EC4D}>
>    :START
>    0
>    :COUNT
>    536870911)
> 1: (BACKTRACE 536870911 #<SB-IMPL::STRING-OUTPUT-STREAM {B71EBE9}>)
> 2: (TRIVIAL-BACKTRACE:PRINT-BACKTRACE-TO-STREAM
>    #<SB-IMPL::STRING-OUTPUT-STREAM {B71EBE9}>)
> 3: (HUNCHENTOOT::GET-BACKTRACE)
> 4: ((FLET #:LAMBDA271)
>    #<SB-INT:SIMPLE-STREAM-ERROR "~@<~?: ~2I~_~A~:>" {B71EA51}>)
> 5: (SIGNAL #<SB-INT:SIMPLE-STREAM-ERROR "~@<~?: ~2I~_~A~:>" {B71EA51}>)
> 6: (ERROR
>    SB-INT:SIMPLE-STREAM-ERROR
>    :STREAM
>    #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741"
>      {B6FB859}>
>    :FORMAT-CONTROL
>    "~@<~?: ~2I~_~A~:>"
>    :FORMAT-ARGUMENTS
>    ("Couldn't write to ~S"
>     (#<SB-SYS:FD-STREAM
>        for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>)
>     "Broken pipe"))
> 7: (SB-IMPL::SIMPLE-STREAM-PERROR
>    "Couldn't write to ~S"
>    #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741"
>      {B6FB859}>
>    32)
> 8: (SB-IMPL::WRITE-OUTPUT-FROM-QUEUE
>    #<SB-SYS:FD-STREAM for "socket 74.115.254.12:80, peer: 217.72.218.248:2741"
>      {B6FB859}>)
> 9: (SB-IMPL::SUB-SUB-SERVE-EVENT NIL NIL)
> 10: (SB-IMPL::SUB-SERVE-EVENT NIL NIL NIL)
> 11: (SB-SYS:SERVE-ALL-EVENTS NIL)
> 12: (SB-IMPL::FINISH-FD-STREAM-OUTPUT
>     #<SB-SYS:FD-STREAM
>       for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>)
> 13: (FINISH-OUTPUT
>     #<SB-SYS:FD-STREAM
>       for "socket 74.115.254.12:80, peer: 217.72.218.248:2741" {B6FB859}>)
> 14: (HUNCHENTOOT:HANDLE-STATIC-FILE
>     #P"/home/sly/projects/www-cyrusharmon-org/static/images/fish-clusters.png"
>     NIL)
> 15: ((SB-PCL::FAST-METHOD HUNCHENTOOT:HANDLE-REQUEST
>      (HUNCHENTOOT:ACCEPTOR HUNCHENTOOT:REQUEST))
>     #<unavailable argument>
>     #<unavailable argument>
>     #<HUNCHENTOOT:ACCEPTOR (host *, port 80)>
>     #<HUNCHENTOOT:REQUEST {B701D09}>)
> 16: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-REQUEST (T))
>     #<unavailable argument>
>     #<unavailable argument>
>     #<HUNCHENTOOT:REQUEST {B701D09}>)
> 17: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-CONNECTION
>      (HUNCHENTOOT:ACCEPTOR T))
>     #<unavailable argument>
>     #<unavailable argument>
>     #<HUNCHENTOOT:ACCEPTOR (host *, port 80)>
>     #<USOCKET:STREAM-USOCKET {B6FB8F1}>)
> 18: ((SB-PCL::FAST-METHOD HUNCHENTOOT:PROCESS-CONNECTION :AROUND
>      (HUNCHENTOOT:ACCEPTOR T))
>     #<unavailable argument>
>     #S(SB-PCL::FAST-METHOD-CALL
>        :FUNCTION #<FUNCTION #>
>        :PV NIL
>        :NEXT-METHOD-CALL NIL
>        :ARG-INFO (2))
>     #<HUNCHENTOOT:ACCEPTOR (host *, port 80)>
>     #<USOCKET:STREAM-USOCKET {B6FB8F1}>)
> 19: ((LAMBDA ()))
> 20: ((FLET #:WITHOUT-INTERRUPTS-BODY-[BLOCK353]358))
> 21: ((FLET SB-THREAD::WITH-MUTEX-THUNK))
> 22: ((FLET #:WITHOUT-INTERRUPTS-BODY-[CALL-WITH-MUTEX]300))
> 23: (SB-THREAD::CALL-WITH-MUTEX
>     #<CLOSURE (FLET SB-THREAD::WITH-MUTEX-THUNK) {B0E14235}>
>     #S(SB-THREAD:MUTEX
>        :NAME "thread result lock"
>        :%OWNER #<SB-THREAD:THREAD
>                  "Hunchentoot worker (client: 217.72.218.248:2741)" RUNNING
>                  {B6FCE39}>
>        :STATE 1)
>     #<SB-THREAD:THREAD
>       "Hunchentoot worker (client: 217.72.218.248:2741)" RUNNING {B6FCE39}>
>     T)
> 24: (SB-THREAD::INITIAL-THREAD-FUNCTION)
> 25: ("foreign function: call_into_lisp")
> 26: ("foreign function: funcall0")
> 27: ("foreign function: new_thread_trampoline")
> 28: ("foreign function: #xB7FC5506")
>
> What does this whole "broken pipe" mean? Is this more of an "expected error" or is something going wrong here?
>
> On a slightly different note, I'd like to be able to run a relatively reliable hunchentoot server that can serve a modest traffic load without having to resort to caching proxy servers and the like. I recall some discussion of thread pools, epoll, etc... Has the state of the art for high-reliability hunchentoot instances changed in the last couple years? Any suggestions on improving it?
>
> thanks!
>
> Cyrus
>
>
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
>