Cant serve static files

Sat Feb 1 15:29:33 UTC 2014

Hi Hans,

The below hardly qualifies as a patch, just a general idea (I'll give
it a try for now and will see if I can make it better):

(defun tbnl:request-pathname (&optional (request tbnl:*request*)
(uri-prefix "."))
  (let* ((path (tbnl:url-decode (tbnl:script-name request)))
         (parts (parse-namestring path))
         (uri-parts (parse-namestring uri-prefix)))
    (iter
      (with pick := nil)
      (with uri-dirs := (cdr (pathname-directory uri-parts)))
      (for uri-part := (pop uri-dirs))
      (for part :in (cdr (pathname-directory parts)))
      (when (or pick (not (equal uri-part part)))
        (setf pick t)
        (collect part :into uri))
      (finally
       (return (make-pathname
                :host (pathname-host
                       (tbnl:acceptor-document-root
                        (tbnl::request-acceptor request)))
                :directory (cons :relative uri)
                :name (pathname-name parts)
                :type (pathname-type parts)))))))

------ test -----

(let ((tbnl:*acceptor* (make-instance
                        'rgol-acceptor :port 4242
                        :request-class 'rgol-request
                        :document-root #p"rgol:www;"
                        :message-log-destination #p"rgol:logs;messages.log"
                        :access-log-destination #p"rgol:logs;access.log")))
  (let ((r (make-instance 'tbnl:request
                          :acceptor tbnl:*acceptor*
                          :headers-in '(("Accept:*/*"))
                          :uri "/foo/bar/qux/baz.ext")))
    (let ((no-prefix (tbnl:request-pathname r))
          (prefix (tbnl:request-pathname r "/foo/")))
      (format t
              "~&no-prefix: ~s ~
               with-prefix: ~s ~%~
               resolved no-prefix: ~s ~%~
               resolved prefix: ~s"
              no-prefix prefix
              (merge-pathnames no-prefix #p"rgol:www;")
              (merge-pathnames prefix #p"rgol:www;")))))

;; no-prefix: #P"RGOL:;FOO;BAR;QUX;BAZ.EXT" with-prefix:
#P"RGOL:;BAR;QUX;BAZ.EXT"
;; resolved no-prefix: #P"RGOL:WWW;FOO;BAR;QUX;BAZ.EXT.NEWEST"
;; resolved prefix: #P"RGOL:WWW;BAR;QUX;BAZ.EXT.NEWEST"

----------------------------------------------

If I can suggest: would it be worth to make request-pathname a generic
function, rather then a plain function? I would imagine few have used
this mechanism to implement their own path resolution scheme, but
theoretically it could be useful. Its name and the way it's been used
seem to ask for it to be a method of request class.

Best,

Oleg

On Sat, Feb 1, 2014 at 2:11 PM, Hans Hübner <hans.huebner at gmail.com> wrote:
> Oleg,
>
> James Anderson privately shared the observation that if the document root
> logical pathname is translated (using translate-logical-pathname) before it
> is merged to the URI's path, the problem would be solved.  Can you give that
> a try?  If that works for you, then I'd like to see Hunchentoot be changed
> so that it call translate-logical-pathname for the configuration paths that
> the user supplies, in the constructor.  The behavior should be documented.
>
> I've also pondered whether it would be better to do the translation for
> every request, but James rightfully pointed out that one would want to see
> erroneous logical pathnames early on.  Also, the overhead might be
> considerable, so I'm tending towards doing it at server initialization time.
> This means that if a logical host definition is changed at run time, the
> server must be restarted to pick up the new definition.
>
> -Hans
>
>
> 2014-02-01 Left Right <olegsivokon at gmail.com>:
>>
>> Ha!
>> I see, I think I've given up on pathnames once, but now I've got more
>> time to try to figure them out, so maybe one day I'll have that patch
>> ;)
>>
>> Thanks for the info, though!
>>
>> Best,
>>
>> Oleg
>>
>> On Sat, Feb 1, 2014 at 12:32 PM, Hans Hübner <hans.huebner at gmail.com>
>> wrote:
>> > Oleg,
>> >
>> > what would be needed is complete support for logical pathnames (which is
>> > not
>> > present at the moment).  I believe that if the document root is a
>> > logical
>> > pathname, the incoming URL would need to be converted to a relative
>> > logical
>> > pathname before it is merged to the document root.
>> >
>> > At this point, I am not prepared to work on this myself.  As pathnames
>> > are
>> > hairy business, I would only want to merge a patch to support logical
>> > pathnames if it comes with thorough tests.
>> >
>> > Alternatively, I would accept a documentation patch that says that if
>> > the
>> > document root is a logical pathname, it must not contain a directory or
>> > file
>> > component. :)
>> >
>> > -Hans
>> >
>> >
>> > 2014-02-01 Left Right <olegsivokon at gmail.com>:
>> >
>> >> Thanks for the answer, Hans,
>> >>
>> >> I'm looking now at the misc.lisp,
>> >> create-folder-dispatcher-and-handler, am I right assuming this is the
>> >> function that handles paths merging?
>> >>
>> >> If so, I was thinking to override request-pathname on request to
>> >> create a URI with the required bits of the pathname. This would seem
>> >> to me like a better way to handle it. But I'm not certain what would
>> >> it mean in terms of other bits of the code, is it likely to break
>> >> other things?
>> >>
>> >> Example:
>> >>
>> >> (merge-pathnames (make-pathname :host "rgol" :name "game" :type
>> >> "html") #p"rgol:www;")
>> >> #P"RGOL:WWW;GAME.HTML.NEWEST"
>> >>
>> >> The reason I want to do it this way is because merge-pathnames will
>> >> only concatenate paths if the default-pathname is trivial: no
>> >> directories, wildcards etc. So it seems to me that the original
>> >> intention was to simply concatenate paths, but because for trivial
>> >> pathnames merge-pathnames worked as concatenation, it was used. Does
>> >> it make sense?
>> >>
>> >> Best,
>> >>
>> >> Oleg
>> >>
>> >> On Sat, Feb 1, 2014 at 10:59 AM, Hans Hübner <hans.huebner at gmail.com>
>> >> wrote:
>> >> > Oleg,
>> >> >
>> >> > the problem seems to be related how logical pathnames are merged.
>> >> > The
>> >> > partial pathname that is created by Hunchentoot and then merged with
>> >> > the
>> >> > document root pathname is considered to contain a directory component
>> >> > by
>> >> > CL.
>> >> > Therefore, the directory component that you are matching in your
>> >> > logical
>> >> > pathname host definition is not present when the two partial
>> >> > pathnames
>> >> > are
>> >> > merged.  To work around this behavior, I'd recommend that you define
>> >> > a
>> >> > separate logical pathname host for your document root.
>> >> >
>> >> > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST")
>> >> > '(("foo;*.*.*" "/tmp/*")))
>> >> > (("foo;*.*.*" "/tmp/*"))
>> >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt"
>> >> > #P"test:foo;"))
>> >> > NIL
>> >> > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST")
>> >> > '(("*.*.*" "/tmp/*")))
>> >> > (("*.*.*" "/tmp/*"))
>> >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt"
>> >> > #P"test:foo;"))
>> >> > (#P"/private/tmp/test.txt")
>> >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt"
>> >> > #P"test:"))
>> >> > (#P"/private/tmp/test.txt")
>> >> >
>> >> > -Hans
>> >> >
>> >> >
>> >> > 2014-01-31 Left Right <olegsivokon at gmail.com>:
>> >> >
>> >> >> Sorry, I've copied the wrong log entry, this is the correct one:
>> >> >>
>> >> >> 127.0.0.1 - [2014-01-31 17:35:44] "GET /game.html HTTP/1.1" 404 188
>> >> >> "-"
>> >> >> "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like
>> >> >> Gecko)
>> >> >> Chrome/28.0.1500.95 Safari/537.36"
>> >> >>
>> >> >>
>> >> >> On Fri, Jan 31, 2014 at 5:33 PM, Oleg Sivokon
>> >> >> <olegsivokon at gmail.com>
>> >> >> wrote:
>> >> >>>
>> >> >>> Hello list,
>> >> >>>
>> >> >>> There should be something very simple I've overlooked, yet I can't
>> >> >>> find
>> >> >>> it. Hunchentoot seems not to be able to locate the root directory,
>> >> >>> where
>> >> >>> I have my static content, and I can't get it to print any useful
>> >> >>> information about it. That's why I'm asking for your help.
>> >> >>>
>> >> >>> Below is my setup:
>> >> >>>
>> >> >>> (setf (logical-pathname-translations "rgol")
>> >> >>>        '(...
>> >> >>>          ("WWW;*.*.*" "/home/wvxvw/.../www/")
>> >> >>>          ("WWW;*;*.*.*" "/home/wvxvw/.../www/*")
>> >> >>>          ...))
>> >> >>>
>> >> >>> (make-instance 'hunchentoot:acceptor :port 4242
>> >> >>>     :document-root #p"rgol:www;"
>> >> >>>     :message-log-destination #p"rgol:logs;messages.log"
>> >> >>>     :access-log-destination #p"rgol:logs;access.log")
>> >> >>>
>> >> >>> I've defined another handler, which doesn't depend on static files,
>> >> >>> and
>> >> >>> it works fine, however, when I try to access static files, the log
>> >> >>> record looks like this:
>> >> >>>
>> >> >>> 127.0.0.1 - [2014-01-31 17:12:40]
>> >> >>> "GET /img/made-with-lisp-logo.jpg HTTP/1.1"
>> >> >>> 404 206 "http://localhost:4242/game.html"
>> >> >>> "Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101
>> >> >>> Firefox/23.0"
>> >> >>>
>> >> >>> But the file is definitely there, because if I try this in REPL:
>> >> >>>
>> >> >>> (directory #p"rgol:www;*.*")
>> >> >>> (#P"/home/wvxvw/.../www/game.html"
>> >> >>>  ... more files ...)
>> >> >>>
>> >> >>> My version of Hunchentoot is:
>> >> >>>
>> >> >>> hunchentoot:*hunchentoot-version*
>> >> >>> "1.2.17"
>> >> >>>
>> >> >>> $ sbcl --version
>> >> >>> SBCL 1.1.2-1.fc18
>> >> >>>
>> >> >>> Best,
>> >> >>>
>> >> >>> Oleg
>> >> >>
>> >> >>
>> >> >
>> >
>> >
>
>