From p2.edoc at gmail.com Fri Feb 14 18:14:47 2014 From: p2.edoc at gmail.com (peter) Date: Fri, 14 Feb 2014 18:14:47 +0000 Subject: Persistent sessions In-Reply-To: References: <5CDC4F92-D653-4F81-9268-50268E9D060E@flownet.com> Message-ID: At 8:59 AM -0800 14/2/14, Ron Garret wrote: >On Feb 14, 2014, at 2:31 AM, peter wrote: > >> I find that I have the same requirement and would like not to >> re-invent wheels so would much appreciate hearing any insights you >> gained rg. > >Turns out to be very easy. HT sessions can be accessed and set via >the generic function ht:session-db, which returns an alist. The >alist keys are fixnums and the values are opaque HT::SESSION >objects. To make a persistent session all you have to do is >serialize and restore this alist. All of the slots of an >HT::SESSION are atoms, except for HT::SESSION-DATA, which is an >alist, but it contains only values that you put there. So >serializing an HT::SESSION is not hard. > >The challenging part for me turned out to be restoring Hunchentoot's >internal session state on restart, because HT assumes that there is >an active request whenever a session is created: > >? (make-instance 'ht::session) >> Error: Unbound variable: *REQUEST* > >So you have to restore sessions using ALLOCATE-INSTANCE instead of >MAKE-INSTANCE. Other than that it is completely straightforward. >(I ended up writing a little general-purpose object serializer using >the MOP.) > >> For local hosting, I used a persistent global session-ID counter as a >> patch (ensuring session-IDs can't overlap, but that's not acceptable >> for cloud hosting. >> All I need really is persistence of the client's cookie validity, as >> all state data I keep outside HT. > >Not sure I understand this. HT natively provides unique (per >server) session id's. If you want to make session id's unique >across multiple servers that is a different problem than making >sessions persistent. But HT already has a hook for that: the >generic function NEXT-SESSION-ID. Is there a reason that doesn't >work for you? We're taking different approaches. Here I am obliged not to make anything persistent in the hunchentoot server file system. So the challenge is to make the cookie itself the only thing that is persistent (so stored in the browser). My vague plan is: - ignore the session-ID, use just the cookie-value (as in "0AD9630B5FEA2AEC6E3C24F493B1C0C1") - bypass session-too-old-p for now (later transport the session start time externally to the cookie-value) - so the sequence goes thus: - incoming url from new client (no cookie) HT makes new session and cookie, and sends cookie back to client - subsequence incoming requests carry a cookie, so handled as usual - re-boot HT (session DB data lost) - original client access (carried cookie) HT has to make a substitute session to serve this client hence ignore the session ID throughout, and search the session DB using the cookie value, as in: (defun get-stored-session (cookie-value) (cdr (assoc cookie-value (session-db *acceptor*) :test #'string= :key 'session-string))) Now I'll have to promulgate the consequences of this shift from managing session by ID to cookie-value. But I'm worried that I've overlooked something. Why wasn't this approach (indexing session by cookie-value instead of session-ID do in the first place). I.e. I fear there must be a good reason what session-IDs were used. Unless there was a need to prevent session persistence. I am cowering in anticipation of receiving a castigation for missing the most obvious thing. But persistent session without using any local file store is my requirement. peter From hans.huebner at gmail.com Sat Feb 1 12:11:17 2014 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Sat, 1 Feb 2014 07:11:17 -0500 Subject: Cant serve static files In-Reply-To: References: <87a9eckvy5.fsf@gmail.com> Message-ID: Oleg, James Anderson privately shared the observation that if the document root logical pathname is translated (using translate-logical-pathname) before it is merged to the URI's path, the problem would be solved. Can you give that a try? If that works for you, then I'd like to see Hunchentoot be changed so that it call translate-logical-pathname for the configuration paths that the user supplies, in the constructor. The behavior should be documented. I've also pondered whether it would be better to do the translation for every request, but James rightfully pointed out that one would want to see erroneous logical pathnames early on. Also, the overhead might be considerable, so I'm tending towards doing it at server initialization time. This means that if a logical host definition is changed at run time, the server must be restarted to pick up the new definition. -Hans 2014-02-01 Left Right : > Ha! > I see, I think I've given up on pathnames once, but now I've got more > time to try to figure them out, so maybe one day I'll have that patch > ;) > > Thanks for the info, though! > > Best, > > Oleg > > On Sat, Feb 1, 2014 at 12:32 PM, Hans H?bner > wrote: > > Oleg, > > > > what would be needed is complete support for logical pathnames (which is > not > > present at the moment). I believe that if the document root is a logical > > pathname, the incoming URL would need to be converted to a relative > logical > > pathname before it is merged to the document root. > > > > At this point, I am not prepared to work on this myself. As pathnames > are > > hairy business, I would only want to merge a patch to support logical > > pathnames if it comes with thorough tests. > > > > Alternatively, I would accept a documentation patch that says that if the > > document root is a logical pathname, it must not contain a directory or > file > > component. :) > > > > -Hans > > > > > > 2014-02-01 Left Right : > > > >> Thanks for the answer, Hans, > >> > >> I'm looking now at the misc.lisp, > >> create-folder-dispatcher-and-handler, am I right assuming this is the > >> function that handles paths merging? > >> > >> If so, I was thinking to override request-pathname on request to > >> create a URI with the required bits of the pathname. This would seem > >> to me like a better way to handle it. But I'm not certain what would > >> it mean in terms of other bits of the code, is it likely to break > >> other things? > >> > >> Example: > >> > >> (merge-pathnames (make-pathname :host "rgol" :name "game" :type > >> "html") #p"rgol:www;") > >> #P"RGOL:WWW;GAME.HTML.NEWEST" > >> > >> The reason I want to do it this way is because merge-pathnames will > >> only concatenate paths if the default-pathname is trivial: no > >> directories, wildcards etc. So it seems to me that the original > >> intention was to simply concatenate paths, but because for trivial > >> pathnames merge-pathnames worked as concatenation, it was used. Does > >> it make sense? > >> > >> Best, > >> > >> Oleg > >> > >> On Sat, Feb 1, 2014 at 10:59 AM, Hans H?bner > >> wrote: > >> > Oleg, > >> > > >> > the problem seems to be related how logical pathnames are merged. The > >> > partial pathname that is created by Hunchentoot and then merged with > the > >> > document root pathname is considered to contain a directory component > by > >> > CL. > >> > Therefore, the directory component that you are matching in your > logical > >> > pathname host definition is not present when the two partial pathnames > >> > are > >> > merged. To work around this behavior, I'd recommend that you define a > >> > separate logical pathname host for your document root. > >> > > >> > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST") > >> > '(("foo;*.*.*" "/tmp/*"))) > >> > (("foo;*.*.*" "/tmp/*")) > >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" > >> > #P"test:foo;")) > >> > NIL > >> > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST") > >> > '(("*.*.*" "/tmp/*"))) > >> > (("*.*.*" "/tmp/*")) > >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" > >> > #P"test:foo;")) > >> > (#P"/private/tmp/test.txt") > >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" > >> > #P"test:")) > >> > (#P"/private/tmp/test.txt") > >> > > >> > -Hans > >> > > >> > > >> > 2014-01-31 Left Right : > >> > > >> >> Sorry, I've copied the wrong log entry, this is the correct one: > >> >> > >> >> 127.0.0.1 - [2014-01-31 17:35:44] "GET /game.html HTTP/1.1" 404 188 > "-" > >> >> "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like > Gecko) > >> >> Chrome/28.0.1500.95 Safari/537.36" > >> >> > >> >> > >> >> On Fri, Jan 31, 2014 at 5:33 PM, Oleg Sivokon > > >> >> wrote: > >> >>> > >> >>> Hello list, > >> >>> > >> >>> There should be something very simple I've overlooked, yet I can't > >> >>> find > >> >>> it. Hunchentoot seems not to be able to locate the root directory, > >> >>> where > >> >>> I have my static content, and I can't get it to print any useful > >> >>> information about it. That's why I'm asking for your help. > >> >>> > >> >>> Below is my setup: > >> >>> > >> >>> (setf (logical-pathname-translations "rgol") > >> >>> '(... > >> >>> ("WWW;*.*.*" "/home/wvxvw/.../www/") > >> >>> ("WWW;*;*.*.*" "/home/wvxvw/.../www/*") > >> >>> ...)) > >> >>> > >> >>> (make-instance 'hunchentoot:acceptor :port 4242 > >> >>> :document-root #p"rgol:www;" > >> >>> :message-log-destination #p"rgol:logs;messages.log" > >> >>> :access-log-destination #p"rgol:logs;access.log") > >> >>> > >> >>> I've defined another handler, which doesn't depend on static files, > >> >>> and > >> >>> it works fine, however, when I try to access static files, the log > >> >>> record looks like this: > >> >>> > >> >>> 127.0.0.1 - [2014-01-31 17:12:40] > >> >>> "GET /img/made-with-lisp-logo.jpg HTTP/1.1" > >> >>> 404 206 "http://localhost:4242/game.html" > >> >>> "Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 > Firefox/23.0" > >> >>> > >> >>> But the file is definitely there, because if I try this in REPL: > >> >>> > >> >>> (directory #p"rgol:www;*.*") > >> >>> (#P"/home/wvxvw/.../www/game.html" > >> >>> ... more files ...) > >> >>> > >> >>> My version of Hunchentoot is: > >> >>> > >> >>> hunchentoot:*hunchentoot-version* > >> >>> "1.2.17" > >> >>> > >> >>> $ sbcl --version > >> >>> SBCL 1.1.2-1.fc18 > >> >>> > >> >>> Best, > >> >>> > >> >>> Oleg > >> >> > >> >> > >> > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p2.edoc at gmail.com Fri Feb 14 18:30:15 2014 From: p2.edoc at gmail.com (peter) Date: Fri, 14 Feb 2014 18:30:15 +0000 Subject: Persistent sessions In-Reply-To: References: Message-ID: stringify-session calls encode-session-string which uses (session-start session) which is bound at session initialization to (get-universal-time). session-verify tests validity by comparing the cookie session-string part to a new one made from a composite make from (encode-session-string id user-agent (real-remote-addr request) (session-start session)) but having just made a new session, that session-start component will not match. there's no way to get the original start time out of the cookie, so this part of the verification (together with the session expiry will have to be ignored. as the session start time is encoded inside the cookie, it seems to be that it does have something to do with the cookie. i suspect that you are using your model where you make the session-DB itself (the alist) persistent. i am not allowed to do this, hence why i am trying to i have to use the browser cookie, as that alone can be persistence (as in survive reboot or client or server). At 10:05 AM -0800 14/2/14, Ron Garret wrote: >Huh??? The last click time is just a slot in the session object. >It has nothing to do with the session ID or the cookie. > >On Feb 14, 2014, at 9:44 AM, peter wrote: > >> Given that the cookie is the product of md5-hexing the raw composite >> cookie data (in encode-session-string), there's no way to get the >> initial click (creation) time of the session (which is needed in >> order to ascertain whether the session has expired >> (session-too-old-p)). >> >> So presumably, like the session-ID, we'll need to carry the session >> start time externally (outside the md5-hexed part of the cookie). >> >> Are we thinking along the same rg? >> >> I'm puzzled that session persistence isn't a common requirement of >>hunchentoot. >> >> -peter > > >Content-Transfer-Encoding: 7bit >Content-Disposition: attachment; > filename=signature.asc >Content-Type: application/pgp-signature; > name=signature.asc >Content-Description: Message signed with OpenPGP using GPGMail > >Attachment converted: X:signature 1.asc ( / ) (0626022A) From avodonosov at yandex.ru Fri Feb 14 18:28:02 2014 From: avodonosov at yandex.ru (Anton Vodonosov) Date: Fri, 14 Feb 2014 22:28:02 +0400 Subject: Persistent sessions In-Reply-To: References: <5CDC4F92-D653-4F81-9268-50268E9D060E@flownet.com> Message-ID: <140971392402482@web27m.yandex.ru> 14.02.2014, 22:18, "peter" : > But I'm worried that I've overlooked something. Why wasn't this > approach (indexing session by cookie-value instead of session-ID do > in the first place). I.e. I fear there must be a good reason what > session-IDs were used. session-ID is a cookie value From olegsivokon at gmail.com Sat Feb 1 10:58:19 2014 From: olegsivokon at gmail.com (Left Right) Date: Sat, 1 Feb 2014 12:58:19 +0200 Subject: Cant serve static files In-Reply-To: References: <87a9eckvy5.fsf@gmail.com> Message-ID: Ha! I see, I think I've given up on pathnames once, but now I've got more time to try to figure them out, so maybe one day I'll have that patch ;) Thanks for the info, though! Best, Oleg On Sat, Feb 1, 2014 at 12:32 PM, Hans H?bner wrote: > Oleg, > > what would be needed is complete support for logical pathnames (which is not > present at the moment). I believe that if the document root is a logical > pathname, the incoming URL would need to be converted to a relative logical > pathname before it is merged to the document root. > > At this point, I am not prepared to work on this myself. As pathnames are > hairy business, I would only want to merge a patch to support logical > pathnames if it comes with thorough tests. > > Alternatively, I would accept a documentation patch that says that if the > document root is a logical pathname, it must not contain a directory or file > component. :) > > -Hans > > > 2014-02-01 Left Right : > >> Thanks for the answer, Hans, >> >> I'm looking now at the misc.lisp, >> create-folder-dispatcher-and-handler, am I right assuming this is the >> function that handles paths merging? >> >> If so, I was thinking to override request-pathname on request to >> create a URI with the required bits of the pathname. This would seem >> to me like a better way to handle it. But I'm not certain what would >> it mean in terms of other bits of the code, is it likely to break >> other things? >> >> Example: >> >> (merge-pathnames (make-pathname :host "rgol" :name "game" :type >> "html") #p"rgol:www;") >> #P"RGOL:WWW;GAME.HTML.NEWEST" >> >> The reason I want to do it this way is because merge-pathnames will >> only concatenate paths if the default-pathname is trivial: no >> directories, wildcards etc. So it seems to me that the original >> intention was to simply concatenate paths, but because for trivial >> pathnames merge-pathnames worked as concatenation, it was used. Does >> it make sense? >> >> Best, >> >> Oleg >> >> On Sat, Feb 1, 2014 at 10:59 AM, Hans H?bner >> wrote: >> > Oleg, >> > >> > the problem seems to be related how logical pathnames are merged. The >> > partial pathname that is created by Hunchentoot and then merged with the >> > document root pathname is considered to contain a directory component by >> > CL. >> > Therefore, the directory component that you are matching in your logical >> > pathname host definition is not present when the two partial pathnames >> > are >> > merged. To work around this behavior, I'd recommend that you define a >> > separate logical pathname host for your document root. >> > >> > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST") >> > '(("foo;*.*.*" "/tmp/*"))) >> > (("foo;*.*.*" "/tmp/*")) >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" >> > #P"test:foo;")) >> > NIL >> > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST") >> > '(("*.*.*" "/tmp/*"))) >> > (("*.*.*" "/tmp/*")) >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" >> > #P"test:foo;")) >> > (#P"/private/tmp/test.txt") >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" >> > #P"test:")) >> > (#P"/private/tmp/test.txt") >> > >> > -Hans >> > >> > >> > 2014-01-31 Left Right : >> > >> >> Sorry, I've copied the wrong log entry, this is the correct one: >> >> >> >> 127.0.0.1 - [2014-01-31 17:35:44] "GET /game.html HTTP/1.1" 404 188 "-" >> >> "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) >> >> Chrome/28.0.1500.95 Safari/537.36" >> >> >> >> >> >> On Fri, Jan 31, 2014 at 5:33 PM, Oleg Sivokon >> >> wrote: >> >>> >> >>> Hello list, >> >>> >> >>> There should be something very simple I've overlooked, yet I can't >> >>> find >> >>> it. Hunchentoot seems not to be able to locate the root directory, >> >>> where >> >>> I have my static content, and I can't get it to print any useful >> >>> information about it. That's why I'm asking for your help. >> >>> >> >>> Below is my setup: >> >>> >> >>> (setf (logical-pathname-translations "rgol") >> >>> '(... >> >>> ("WWW;*.*.*" "/home/wvxvw/.../www/") >> >>> ("WWW;*;*.*.*" "/home/wvxvw/.../www/*") >> >>> ...)) >> >>> >> >>> (make-instance 'hunchentoot:acceptor :port 4242 >> >>> :document-root #p"rgol:www;" >> >>> :message-log-destination #p"rgol:logs;messages.log" >> >>> :access-log-destination #p"rgol:logs;access.log") >> >>> >> >>> I've defined another handler, which doesn't depend on static files, >> >>> and >> >>> it works fine, however, when I try to access static files, the log >> >>> record looks like this: >> >>> >> >>> 127.0.0.1 - [2014-01-31 17:12:40] >> >>> "GET /img/made-with-lisp-logo.jpg HTTP/1.1" >> >>> 404 206 "http://localhost:4242/game.html" >> >>> "Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0" >> >>> >> >>> But the file is definitely there, because if I try this in REPL: >> >>> >> >>> (directory #p"rgol:www;*.*") >> >>> (#P"/home/wvxvw/.../www/game.html" >> >>> ... more files ...) >> >>> >> >>> My version of Hunchentoot is: >> >>> >> >>> hunchentoot:*hunchentoot-version* >> >>> "1.2.17" >> >>> >> >>> $ sbcl --version >> >>> SBCL 1.1.2-1.fc18 >> >>> >> >>> Best, >> >>> >> >>> Oleg >> >> >> >> >> > > > From farukscan at gmail.com Fri Feb 28 21:17:25 2014 From: farukscan at gmail.com (Faruk S. Can) Date: Fri, 28 Feb 2014 23:17:25 +0200 Subject: hunchentoot serving manifest files, ie. .appcache files. In-Reply-To: References: Message-ID: i am also scanning hunchentoot documentation. i have found about outgoing headers, the accessor header-out and setf of it creates a new one if not exists. maybe sent content type header can be setf ed by this at the beginnig of the lisp function that has been dispatched to the url. by url I mean script mentioning it as eventSource("url") in html5 document. same for cache-control header. Is there any inside thing in the implementation of server that shuld be changed? Am I naively thinking above not regarding implementation of the server? On Fri, Feb 28, 2014 at 11:09 PM, Faruk S. Can wrote: > I have read in w3schools about usage of server sent events in html5. > it says: > > - Set the "Content-Type" header to "text/event-stream" > - Specify that the page should not cache > - Output the data to send (*Always* start with "data: ") > > that is in echo line in php code as echo "data: The server time is: > {$time}\n\n"; > > - Flush the output data back to the web page > > http://www.w3schools.com/html/html5_serversentevents.asp > examples here are in php for server side. i am using hunchentoot on common > lisp. > > > > On Fri, Feb 28, 2014 at 10:27 PM, Faruk S. Can wrote: > >> I >> have another question about hunchentoot. Is it possible to use server >> sent events with hunchentoot, like other html5 improvements mentioned in >> w3schools-html5 such as app cache, local storage, session storage? as that >> one related to and depend on the server side support. >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From farukscan at gmail.com Fri Feb 28 21:09:48 2014 From: farukscan at gmail.com (Faruk S. Can) Date: Fri, 28 Feb 2014 23:09:48 +0200 Subject: hunchentoot serving manifest files, ie. .appcache files. In-Reply-To: References: Message-ID: I have read in w3schools about usage of server sent events in html5. it says: - Set the "Content-Type" header to "text/event-stream" - Specify that the page should not cache - Output the data to send (*Always* start with "data: ") that is in echo line in php code as echo "data: The server time is: {$time}\n\n"; - Flush the output data back to the web page http://www.w3schools.com/html/html5_serversentevents.asp examples here are in php for server side. i am using hunchentoot on common lisp. On Fri, Feb 28, 2014 at 10:27 PM, Faruk S. Can wrote: > I > have another question about hunchentoot. Is it possible to use server > sent events with hunchentoot, like other html5 improvements mentioned in > w3schools-html5 such as app cache, local storage, session storage? as that > one related to and depend on the server side support. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p2.edoc at gmail.com Fri Feb 14 18:34:48 2014 From: p2.edoc at gmail.com (peter) Date: Fri, 14 Feb 2014 18:34:48 +0000 Subject: Persistent sessions In-Reply-To: <140971392402482@web27m.yandex.ru> References: <5CDC4F92-D653-4F81-9268-50268E9D060E@flownet.com> <140971392402482@web27m.yandex.ru> Message-ID: At 10:28 PM +0400 14/2/14, Anton Vodonosov wrote: >14.02.2014, 22:18, "peter" : >> But I'm worried that I've overlooked something. Why wasn't this >> approach (indexing session by cookie-value instead of session-ID do >> in the first place). I.e. I fear there must be a good reason what >> session-IDs were used. > >session-ID is a cookie value i appreciate that (it is prepended to the md5-hex part of the cookie). and it is a convenient way to index sessions DB in HT. however beyond that, i see no purpose for it. and in the case of making sessions persistent without saving any state on the HT system, they seem irrelevant and need to be side-stepped. hence why i am trying to index the session DB using the md5-hex part of the cookie only. From farukscan at gmail.com Fri Feb 28 14:53:12 2014 From: farukscan at gmail.com (Faruk S. Can) Date: Fri, 28 Feb 2014 16:53:12 +0200 Subject: hunchentoot serving manifest files, ie. .appcache files. Message-ID: how to serve manifest files, ie. .appcache files wit hunchentoot? (mime-type #p"path/to/anifest/file") >nil nil is there a way to do? or is it jsut enough to proxy with apache or nginx front and configure those to serve manifest files, as it is noted in w3schools html5- appcache topic? before i try myself, just asked to bring it to attention for possible renewing improvements to the hunchentoot. -------------- next part -------------- An HTML attachment was scrubbed... URL: From p2.edoc at gmail.com Fri Feb 14 10:31:28 2014 From: p2.edoc at gmail.com (peter) Date: Fri, 14 Feb 2014 10:31:28 +0000 Subject: Persistent sessions In-Reply-To: <5CDC4F92-D653-4F81-9268-50268E9D060E@flownet.com> References: <5CDC4F92-D653-4F81-9268-50268E9D060E@flownet.com> Message-ID: I find that I have the same requirement and would like not to re-invent wheels so would much appreciate hearing any insights you gained rg. For local hosting, I used a persistent global session-ID counter as a patch (ensuring session-IDs can't overlap, but that's not acceptable for cloud hosting. All I need really is persistence of the client's cookie validity, as all state data I keep outside HT. So now I was thinking to modify start-session to be a fork for the make new session case. If the incoming request has no cookie, assume a fresh start so continue as of original start-session, otherwise take this as being an old session, now not found in the session DB. Hence make a replace session. The session-ID seems irrelevant if I can search of sessions using the session string only. Hence make get-stored-session use string= on the cookie-string. Ignoring overlapping session-IDs (they begin from 1 each HT boot), I'd need to eliminate reference to session-ID externally and embedded into the cookie string. But now I get into session-verify etc and expect to run into awkwardnesses. I don't fully understand the use of session-ID, why doesn't HT just use the vanilla cookie-string (without session-ID prefix) ... (despite it obviously being there for some purpose beyond optimized session-DB search). I can't yet get my head around how we can have multiple concurrent sessions to the same client, if that is the original intent. If not then why use the session-id, and why both externally as session-DB index, and internally in the cookie-string as in stringify-session. peter At 6:59 PM -0800 14/1/17, Ron Garret wrote: >I need to be able to save and restore HT sessions so that they will >survive a server restart. Before I dive into this I thought I would >ask if anyone has done this already so I don't reinvent the wheel. > >Thanks, >rg From farukscan at gmail.com Fri Feb 28 20:27:58 2014 From: farukscan at gmail.com (Faruk S. Can) Date: Fri, 28 Feb 2014 22:27:58 +0200 Subject: hunchentoot serving manifest files, ie. .appcache files. In-Reply-To: References: Message-ID: I have another question about hunchentoot. Is it possible to use server sent events with hunchentoot, like other html5 improvements mentioned in w3schools-html5 such as app cache, local storage, session storage? as that one related to and depend on the server side support. -------------- next part -------------- An HTML attachment was scrubbed... URL: From olegsivokon at gmail.com Fri Feb 28 16:03:37 2014 From: olegsivokon at gmail.com (Left Right) Date: Fri, 28 Feb 2014 18:03:37 +0200 Subject: hunchentoot serving manifest files, ie. .appcache files. In-Reply-To: References: Message-ID: You could (setf (gethash "manifest" tbnl::*mime-type-hash*) "text/cache-manifest") to address the immediate issue, however, since this symbol isn't exported, maybe there was a different plan for adding more mime-types. On Fri, Feb 28, 2014 at 4:53 PM, Faruk S. Can wrote: > how to serve manifest files, ie. .appcache files wit hunchentoot? > (mime-type #p"path/to/anifest/file") >>nil > nil > > is there a way to do? > > or is it jsut enough to proxy with apache or nginx front and configure those > to serve manifest files, as it is noted in w3schools html5- appcache topic? > > before i try myself, just asked to bring it to attention for possible > renewing improvements to the hunchentoot. From olegsivokon at gmail.com Sat Feb 1 15:29:33 2014 From: olegsivokon at gmail.com (Left Right) Date: Sat, 1 Feb 2014 17:29:33 +0200 Subject: Cant serve static files In-Reply-To: References: <87a9eckvy5.fsf@gmail.com> Message-ID: Hi Hans, The below hardly qualifies as a patch, just a general idea (I'll give it a try for now and will see if I can make it better): (defun tbnl:request-pathname (&optional (request tbnl:*request*) (uri-prefix ".")) (let* ((path (tbnl:url-decode (tbnl:script-name request))) (parts (parse-namestring path)) (uri-parts (parse-namestring uri-prefix))) (iter (with pick := nil) (with uri-dirs := (cdr (pathname-directory uri-parts))) (for uri-part := (pop uri-dirs)) (for part :in (cdr (pathname-directory parts))) (when (or pick (not (equal uri-part part))) (setf pick t) (collect part :into uri)) (finally (return (make-pathname :host (pathname-host (tbnl:acceptor-document-root (tbnl::request-acceptor request))) :directory (cons :relative uri) :name (pathname-name parts) :type (pathname-type parts))))))) ------ test ----- (let ((tbnl:*acceptor* (make-instance 'rgol-acceptor :port 4242 :request-class 'rgol-request :document-root #p"rgol:www;" :message-log-destination #p"rgol:logs;messages.log" :access-log-destination #p"rgol:logs;access.log"))) (let ((r (make-instance 'tbnl:request :acceptor tbnl:*acceptor* :headers-in '(("Accept:*/*")) :uri "/foo/bar/qux/baz.ext"))) (let ((no-prefix (tbnl:request-pathname r)) (prefix (tbnl:request-pathname r "/foo/"))) (format t "~&no-prefix: ~s ~ with-prefix: ~s ~%~ resolved no-prefix: ~s ~%~ resolved prefix: ~s" no-prefix prefix (merge-pathnames no-prefix #p"rgol:www;") (merge-pathnames prefix #p"rgol:www;"))))) ;; no-prefix: #P"RGOL:;FOO;BAR;QUX;BAZ.EXT" with-prefix: #P"RGOL:;BAR;QUX;BAZ.EXT" ;; resolved no-prefix: #P"RGOL:WWW;FOO;BAR;QUX;BAZ.EXT.NEWEST" ;; resolved prefix: #P"RGOL:WWW;BAR;QUX;BAZ.EXT.NEWEST" ---------------------------------------------- If I can suggest: would it be worth to make request-pathname a generic function, rather then a plain function? I would imagine few have used this mechanism to implement their own path resolution scheme, but theoretically it could be useful. Its name and the way it's been used seem to ask for it to be a method of request class. Best, Oleg On Sat, Feb 1, 2014 at 2:11 PM, Hans H?bner wrote: > Oleg, > > James Anderson privately shared the observation that if the document root > logical pathname is translated (using translate-logical-pathname) before it > is merged to the URI's path, the problem would be solved. Can you give that > a try? If that works for you, then I'd like to see Hunchentoot be changed > so that it call translate-logical-pathname for the configuration paths that > the user supplies, in the constructor. The behavior should be documented. > > I've also pondered whether it would be better to do the translation for > every request, but James rightfully pointed out that one would want to see > erroneous logical pathnames early on. Also, the overhead might be > considerable, so I'm tending towards doing it at server initialization time. > This means that if a logical host definition is changed at run time, the > server must be restarted to pick up the new definition. > > -Hans > > > 2014-02-01 Left Right : >> >> Ha! >> I see, I think I've given up on pathnames once, but now I've got more >> time to try to figure them out, so maybe one day I'll have that patch >> ;) >> >> Thanks for the info, though! >> >> Best, >> >> Oleg >> >> On Sat, Feb 1, 2014 at 12:32 PM, Hans H?bner >> wrote: >> > Oleg, >> > >> > what would be needed is complete support for logical pathnames (which is >> > not >> > present at the moment). I believe that if the document root is a >> > logical >> > pathname, the incoming URL would need to be converted to a relative >> > logical >> > pathname before it is merged to the document root. >> > >> > At this point, I am not prepared to work on this myself. As pathnames >> > are >> > hairy business, I would only want to merge a patch to support logical >> > pathnames if it comes with thorough tests. >> > >> > Alternatively, I would accept a documentation patch that says that if >> > the >> > document root is a logical pathname, it must not contain a directory or >> > file >> > component. :) >> > >> > -Hans >> > >> > >> > 2014-02-01 Left Right : >> > >> >> Thanks for the answer, Hans, >> >> >> >> I'm looking now at the misc.lisp, >> >> create-folder-dispatcher-and-handler, am I right assuming this is the >> >> function that handles paths merging? >> >> >> >> If so, I was thinking to override request-pathname on request to >> >> create a URI with the required bits of the pathname. This would seem >> >> to me like a better way to handle it. But I'm not certain what would >> >> it mean in terms of other bits of the code, is it likely to break >> >> other things? >> >> >> >> Example: >> >> >> >> (merge-pathnames (make-pathname :host "rgol" :name "game" :type >> >> "html") #p"rgol:www;") >> >> #P"RGOL:WWW;GAME.HTML.NEWEST" >> >> >> >> The reason I want to do it this way is because merge-pathnames will >> >> only concatenate paths if the default-pathname is trivial: no >> >> directories, wildcards etc. So it seems to me that the original >> >> intention was to simply concatenate paths, but because for trivial >> >> pathnames merge-pathnames worked as concatenation, it was used. Does >> >> it make sense? >> >> >> >> Best, >> >> >> >> Oleg >> >> >> >> On Sat, Feb 1, 2014 at 10:59 AM, Hans H?bner >> >> wrote: >> >> > Oleg, >> >> > >> >> > the problem seems to be related how logical pathnames are merged. >> >> > The >> >> > partial pathname that is created by Hunchentoot and then merged with >> >> > the >> >> > document root pathname is considered to contain a directory component >> >> > by >> >> > CL. >> >> > Therefore, the directory component that you are matching in your >> >> > logical >> >> > pathname host definition is not present when the two partial >> >> > pathnames >> >> > are >> >> > merged. To work around this behavior, I'd recommend that you define >> >> > a >> >> > separate logical pathname host for your document root. >> >> > >> >> > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST") >> >> > '(("foo;*.*.*" "/tmp/*"))) >> >> > (("foo;*.*.*" "/tmp/*")) >> >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" >> >> > #P"test:foo;")) >> >> > NIL >> >> > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST") >> >> > '(("*.*.*" "/tmp/*"))) >> >> > (("*.*.*" "/tmp/*")) >> >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" >> >> > #P"test:foo;")) >> >> > (#P"/private/tmp/test.txt") >> >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" >> >> > #P"test:")) >> >> > (#P"/private/tmp/test.txt") >> >> > >> >> > -Hans >> >> > >> >> > >> >> > 2014-01-31 Left Right : >> >> > >> >> >> Sorry, I've copied the wrong log entry, this is the correct one: >> >> >> >> >> >> 127.0.0.1 - [2014-01-31 17:35:44] "GET /game.html HTTP/1.1" 404 188 >> >> >> "-" >> >> >> "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like >> >> >> Gecko) >> >> >> Chrome/28.0.1500.95 Safari/537.36" >> >> >> >> >> >> >> >> >> On Fri, Jan 31, 2014 at 5:33 PM, Oleg Sivokon >> >> >> >> >> >> wrote: >> >> >>> >> >> >>> Hello list, >> >> >>> >> >> >>> There should be something very simple I've overlooked, yet I can't >> >> >>> find >> >> >>> it. Hunchentoot seems not to be able to locate the root directory, >> >> >>> where >> >> >>> I have my static content, and I can't get it to print any useful >> >> >>> information about it. That's why I'm asking for your help. >> >> >>> >> >> >>> Below is my setup: >> >> >>> >> >> >>> (setf (logical-pathname-translations "rgol") >> >> >>> '(... >> >> >>> ("WWW;*.*.*" "/home/wvxvw/.../www/") >> >> >>> ("WWW;*;*.*.*" "/home/wvxvw/.../www/*") >> >> >>> ...)) >> >> >>> >> >> >>> (make-instance 'hunchentoot:acceptor :port 4242 >> >> >>> :document-root #p"rgol:www;" >> >> >>> :message-log-destination #p"rgol:logs;messages.log" >> >> >>> :access-log-destination #p"rgol:logs;access.log") >> >> >>> >> >> >>> I've defined another handler, which doesn't depend on static files, >> >> >>> and >> >> >>> it works fine, however, when I try to access static files, the log >> >> >>> record looks like this: >> >> >>> >> >> >>> 127.0.0.1 - [2014-01-31 17:12:40] >> >> >>> "GET /img/made-with-lisp-logo.jpg HTTP/1.1" >> >> >>> 404 206 "http://localhost:4242/game.html" >> >> >>> "Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 >> >> >>> Firefox/23.0" >> >> >>> >> >> >>> But the file is definitely there, because if I try this in REPL: >> >> >>> >> >> >>> (directory #p"rgol:www;*.*") >> >> >>> (#P"/home/wvxvw/.../www/game.html" >> >> >>> ... more files ...) >> >> >>> >> >> >>> My version of Hunchentoot is: >> >> >>> >> >> >>> hunchentoot:*hunchentoot-version* >> >> >>> "1.2.17" >> >> >>> >> >> >>> $ sbcl --version >> >> >>> SBCL 1.1.2-1.fc18 >> >> >>> >> >> >>> Best, >> >> >>> >> >> >>> Oleg >> >> >> >> >> >> >> >> > >> > >> > > > From ron at flownet.com Fri Feb 14 18:48:21 2014 From: ron at flownet.com (Ron Garret) Date: Fri, 14 Feb 2014 10:48:21 -0800 Subject: Persistent sessions In-Reply-To: References: Message-ID: <53C851B9-6AAF-4FD0-8752-DCC7641C395E@flownet.com> On Feb 14, 2014, at 10:30 AM, peter wrote: > as the session start time is encoded inside the cookie No, it isn't. The session start time is used as a source of entropy to help make the session cookie secure. But the session cookie is an MD5 hash, and so you can?t get any information out of it. > i suspect that you are using your model where you make the session-DB > itself (the alist) persistent. Yes. Of course. > i am not allowed to do this Then you need to change the way that HT generates cookies so that they contain all the information you need in a way that allows you to extract it. Right now they don?t. (I?m a little surprised that you?re not allowed to do this. I have a hard time imagining how you could write a useful application without being allowed to write to non-volatile storage.) rg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: Message signed with OpenPGP using GPGMail URL: From ron at flownet.com Fri Feb 14 16:59:09 2014 From: ron at flownet.com (Ron Garret) Date: Fri, 14 Feb 2014 08:59:09 -0800 Subject: Persistent sessions In-Reply-To: References: <5CDC4F92-D653-4F81-9268-50268E9D060E@flownet.com> Message-ID: On Feb 14, 2014, at 2:31 AM, peter wrote: > I find that I have the same requirement and would like not to > re-invent wheels so would much appreciate hearing any insights you > gained rg. Turns out to be very easy. HT sessions can be accessed and set via the generic function ht:session-db, which returns an alist. The alist keys are fixnums and the values are opaque HT::SESSION objects. To make a persistent session all you have to do is serialize and restore this alist. All of the slots of an HT::SESSION are atoms, except for HT::SESSION-DATA, which is an alist, but it contains only values that you put there. So serializing an HT::SESSION is not hard. The challenging part for me turned out to be restoring Hunchentoot?s internal session state on restart, because HT assumes that there is an active request whenever a session is created: ? (make-instance 'ht::session) > Error: Unbound variable: *REQUEST* So you have to restore sessions using ALLOCATE-INSTANCE instead of MAKE-INSTANCE. Other than that it is completely straightforward. (I ended up writing a little general-purpose object serializer using the MOP.) > For local hosting, I used a persistent global session-ID counter as a > patch (ensuring session-IDs can't overlap, but that's not acceptable > for cloud hosting. > All I need really is persistence of the client's cookie validity, as > all state data I keep outside HT. Not sure I understand this. HT natively provides unique (per server) session id?s. If you want to make session id?s unique across multiple servers that is a different problem than making sessions persistent. But HT already has a hook for that: the generic function NEXT-SESSION-ID. Is there a reason that doesn?t work for you? rg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: Message signed with OpenPGP using GPGMail URL: From olegsivokon at gmail.com Fri Feb 28 21:40:48 2014 From: olegsivokon at gmail.com (Left Right) Date: Fri, 28 Feb 2014 23:40:48 +0200 Subject: hunchentoot serving manifest files, ie. .appcache files. In-Reply-To: References: Message-ID: Not exactly answering your question, but if you are looking for push functionality in browser, then WebSockets might be a better option. The reason server-sent events exist is by and large due to the handicapped nature of most popular web-development stack in use today, such as PHP interpreter that has to be started on every connection by an HTTP server, such as Apache httpd or ngnx. This means that traditional setup works in a stateless manner. This is unlike, say, most Java HTTP servers (Tomcat, JBoss etc) or Python (Twisted, Tornado), where the setup is normally stateful, meaning that you wouldn't need to restart the interpreter upon each request, instead, your server would persist the state until it is shut down. Hunchentoot belongs to the later category. Now, because it's not really possible to persist a PHP session across multiple requests, a technique first known as long polling appeared. The operating principle was that the server, in response to an HTTP request would not close the connection immediately, instead it would keep sending the information, whenever the application state at the server would demand it. The receiving side (the browser) would listen to updates on this connection and interpret them as "pushes". Event stream is a refinement of this scheme. WebSockets, on the other hand, are plain TCP sockets, which means that they don't require HTTP wrapping to function. I haven't had much experience with JavaScript WebSockets, but I used their analogy in Flash quite a bit. It was a relief not to depend on the bizarre rules of HTTP protocol and its encodings, headers and so on. So I would expect the same to hold for WebSockets. Best, Oleg On Fri, Feb 28, 2014 at 11:09 PM, Faruk S. Can wrote: > I have read in w3schools about usage of server sent events in html5. > it says: > > Set the "Content-Type" header to "text/event-stream" > Specify that the page should not cache > Output the data to send (Always start with "data: ") > > that is in echo line in php code as echo "data: The server time is: > {$time}\n\n"; > > Flush the output data back to the web page > > http://www.w3schools.com/html/html5_serversentevents.asp > examples here are in php for server side. i am using hunchentoot on common > lisp. > > > > On Fri, Feb 28, 2014 at 10:27 PM, Faruk S. Can wrote: >> >> I >> have another question about hunchentoot. Is it possible to use server >> sent events with hunchentoot, like other html5 improvements mentioned in >> w3schools-html5 such as app cache, local storage, session storage? as that >> one related to and depend on the server side support. >> > From avodonosov at yandex.ru Fri Feb 14 18:35:20 2014 From: avodonosov at yandex.ru (Anton Vodonosov) Date: Fri, 14 Feb 2014 22:35:20 +0400 Subject: Persistent sessions In-Reply-To: References: <5CDC4F92-D653-4F81-9268-50268E9D060E@flownet.com> Message-ID: <27991392402920@web19j.yandex.ru> But Peter is right that there is no need to presist attributes your application associate with user session via hunchentoot mechanism. You can just store the attributes your application intorduces in your own DB, using sessionID as a subkey. The Ron's solution ensures another aspect - when hunchentoot is restarted, the same sessions will remain to be active. If Peter wants this too, he needs to persist information needed to handle session timeouts. So, the approaches are not so different. From hans.huebner at gmail.com Sat Feb 1 08:59:27 2014 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Sat, 1 Feb 2014 03:59:27 -0500 Subject: Cant serve static files In-Reply-To: References: <87a9eckvy5.fsf@gmail.com> Message-ID: Oleg, the problem seems to be related how logical pathnames are merged. The partial pathname that is created by Hunchentoot and then merged with the document root pathname is considered to contain a directory component by CL. Therefore, the directory component that you are matching in your logical pathname host definition is not present when the two partial pathnames are merged. To work around this behavior, I'd recommend that you define a separate logical pathname host for your document root. TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST") '(("foo;*.*.*" "/tmp/*"))) (("foo;*.*.*" "/tmp/*")) TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" #P"test:foo;")) NIL TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST") '(("*.*.*" "/tmp/*"))) (("*.*.*" "/tmp/*")) TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" #P"test:foo;")) (#P"/private/tmp/test.txt") TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" #P"test:")) (#P"/private/tmp/test.txt") -Hans 2014-01-31 Left Right : > Sorry, I've copied the wrong log entry, this is the correct one: > > 127.0.0.1 - [2014-01-31 17:35:44] "GET /game.html HTTP/1.1" 404 188 "-" > "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) > Chrome/28.0.1500.95 Safari/537.36" > > > On Fri, Jan 31, 2014 at 5:33 PM, Oleg Sivokon wrote: > >> Hello list, >> >> There should be something very simple I've overlooked, yet I can't find >> it. Hunchentoot seems not to be able to locate the root directory, where >> I have my static content, and I can't get it to print any useful >> information about it. That's why I'm asking for your help. >> >> Below is my setup: >> >> (setf (logical-pathname-translations "rgol") >> '(... >> ("WWW;*.*.*" "/home/wvxvw/.../www/") >> ("WWW;*;*.*.*" "/home/wvxvw/.../www/*") >> ...)) >> >> (make-instance 'hunchentoot:acceptor :port 4242 >> :document-root #p"rgol:www;" >> :message-log-destination #p"rgol:logs;messages.log" >> :access-log-destination #p"rgol:logs;access.log") >> >> I've defined another handler, which doesn't depend on static files, and >> it works fine, however, when I try to access static files, the log >> record looks like this: >> >> 127.0.0.1 - [2014-01-31 17:12:40] >> "GET /img/made-with-lisp-logo.jpg HTTP/1.1" >> 404 206 "http://localhost:4242/game.html" >> "Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0" >> >> But the file is definitely there, because if I try this in REPL: >> >> (directory #p"rgol:www;*.*") >> (#P"/home/wvxvw/.../www/game.html" >> ... more files ...) >> >> My version of Hunchentoot is: >> >> hunchentoot:*hunchentoot-version* >> "1.2.17" >> >> $ sbcl --version >> SBCL 1.1.2-1.fc18 >> >> Best, >> >> Oleg >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p2.edoc at gmail.com Fri Feb 14 17:44:43 2014 From: p2.edoc at gmail.com (peter) Date: Fri, 14 Feb 2014 17:44:43 +0000 Subject: Persistent sessions Message-ID: Given that the cookie is the product of md5-hexing the raw composite cookie data (in encode-session-string), there's no way to get the initial click (creation) time of the session (which is needed in order to ascertain whether the session has expired (session-too-old-p)). So presumably, like the session-ID, we'll need to carry the session start time externally (outside the md5-hexed part of the cookie). Are we thinking along the same rg? I'm puzzled that session persistence isn't a common requirement of hunchentoot. -peter From hans.huebner at gmail.com Sat Feb 1 10:32:09 2014 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Sat, 1 Feb 2014 05:32:09 -0500 Subject: Cant serve static files In-Reply-To: References: <87a9eckvy5.fsf@gmail.com> Message-ID: Oleg, what would be needed is complete support for logical pathnames (which is not present at the moment). I believe that if the document root is a logical pathname, the incoming URL would need to be converted to a relative logical pathname before it is merged to the document root. At this point, I am not prepared to work on this myself. As pathnames are hairy business, I would only want to merge a patch to support logical pathnames if it comes with thorough tests. Alternatively, I would accept a documentation patch that says that if the document root is a logical pathname, it must not contain a directory or file component. :) -Hans 2014-02-01 Left Right : > Thanks for the answer, Hans, > > I'm looking now at the misc.lisp, > create-folder-dispatcher-and-handler, am I right assuming this is the > function that handles paths merging? > > If so, I was thinking to override request-pathname on request to > create a URI with the required bits of the pathname. This would seem > to me like a better way to handle it. But I'm not certain what would > it mean in terms of other bits of the code, is it likely to break > other things? > > Example: > > (merge-pathnames (make-pathname :host "rgol" :name "game" :type > "html") #p"rgol:www;") > #P"RGOL:WWW;GAME.HTML.NEWEST" > > The reason I want to do it this way is because merge-pathnames will > only concatenate paths if the default-pathname is trivial: no > directories, wildcards etc. So it seems to me that the original > intention was to simply concatenate paths, but because for trivial > pathnames merge-pathnames worked as concatenation, it was used. Does > it make sense? > > Best, > > Oleg > > On Sat, Feb 1, 2014 at 10:59 AM, Hans H?bner > wrote: > > Oleg, > > > > the problem seems to be related how logical pathnames are merged. The > > partial pathname that is created by Hunchentoot and then merged with the > > document root pathname is considered to contain a directory component by > CL. > > Therefore, the directory component that you are matching in your logical > > pathname host definition is not present when the two partial pathnames > are > > merged. To work around this behavior, I'd recommend that you define a > > separate logical pathname host for your document root. > > > > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST") > > '(("foo;*.*.*" "/tmp/*"))) > > (("foo;*.*.*" "/tmp/*")) > > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" > > #P"test:foo;")) > > NIL > > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST") > > '(("*.*.*" "/tmp/*"))) > > (("*.*.*" "/tmp/*")) > > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" > > #P"test:foo;")) > > (#P"/private/tmp/test.txt") > > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" > #P"test:")) > > (#P"/private/tmp/test.txt") > > > > -Hans > > > > > > 2014-01-31 Left Right : > > > >> Sorry, I've copied the wrong log entry, this is the correct one: > >> > >> 127.0.0.1 - [2014-01-31 17:35:44] "GET /game.html HTTP/1.1" 404 188 "-" > >> "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) > >> Chrome/28.0.1500.95 Safari/537.36" > >> > >> > >> On Fri, Jan 31, 2014 at 5:33 PM, Oleg Sivokon > >> wrote: > >>> > >>> Hello list, > >>> > >>> There should be something very simple I've overlooked, yet I can't find > >>> it. Hunchentoot seems not to be able to locate the root directory, > where > >>> I have my static content, and I can't get it to print any useful > >>> information about it. That's why I'm asking for your help. > >>> > >>> Below is my setup: > >>> > >>> (setf (logical-pathname-translations "rgol") > >>> '(... > >>> ("WWW;*.*.*" "/home/wvxvw/.../www/") > >>> ("WWW;*;*.*.*" "/home/wvxvw/.../www/*") > >>> ...)) > >>> > >>> (make-instance 'hunchentoot:acceptor :port 4242 > >>> :document-root #p"rgol:www;" > >>> :message-log-destination #p"rgol:logs;messages.log" > >>> :access-log-destination #p"rgol:logs;access.log") > >>> > >>> I've defined another handler, which doesn't depend on static files, and > >>> it works fine, however, when I try to access static files, the log > >>> record looks like this: > >>> > >>> 127.0.0.1 - [2014-01-31 17:12:40] > >>> "GET /img/made-with-lisp-logo.jpg HTTP/1.1" > >>> 404 206 "http://localhost:4242/game.html" > >>> "Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0" > >>> > >>> But the file is definitely there, because if I try this in REPL: > >>> > >>> (directory #p"rgol:www;*.*") > >>> (#P"/home/wvxvw/.../www/game.html" > >>> ... more files ...) > >>> > >>> My version of Hunchentoot is: > >>> > >>> hunchentoot:*hunchentoot-version* > >>> "1.2.17" > >>> > >>> $ sbcl --version > >>> SBCL 1.1.2-1.fc18 > >>> > >>> Best, > >>> > >>> Oleg > >> > >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hans.huebner at gmail.com Sat Feb 1 17:37:45 2014 From: hans.huebner at gmail.com (=?ISO-8859-1?Q?Hans_H=FCbner?=) Date: Sat, 1 Feb 2014 12:37:45 -0500 Subject: Cant serve static files In-Reply-To: References: <87a9eckvy5.fsf@gmail.com> Message-ID: Oleg, How is your idea an improvement over James' idea? It seems to me that a call to translate-logical-pathname would be far simpler and easier to review. I do not read iterate, so if we are moving forward with your approach, please use loop or other standard CL iteration facilities. Thanks, Hans Am 01.02.2014 10:29 schrieb "Left Right" : > Hi Hans, > > The below hardly qualifies as a patch, just a general idea (I'll give > it a try for now and will see if I can make it better): > > (defun tbnl:request-pathname (&optional (request tbnl:*request*) > (uri-prefix ".")) > (let* ((path (tbnl:url-decode (tbnl:script-name request))) > (parts (parse-namestring path)) > (uri-parts (parse-namestring uri-prefix))) > (iter > (with pick := nil) > (with uri-dirs := (cdr (pathname-directory uri-parts))) > (for uri-part := (pop uri-dirs)) > (for part :in (cdr (pathname-directory parts))) > (when (or pick (not (equal uri-part part))) > (setf pick t) > (collect part :into uri)) > (finally > (return (make-pathname > :host (pathname-host > (tbnl:acceptor-document-root > (tbnl::request-acceptor request))) > :directory (cons :relative uri) > :name (pathname-name parts) > :type (pathname-type parts))))))) > > ------ test ----- > > (let ((tbnl:*acceptor* (make-instance > 'rgol-acceptor :port 4242 > :request-class 'rgol-request > :document-root #p"rgol:www;" > :message-log-destination #p"rgol:logs;messages.log" > :access-log-destination #p"rgol:logs;access.log"))) > (let ((r (make-instance 'tbnl:request > :acceptor tbnl:*acceptor* > :headers-in '(("Accept:*/*")) > :uri "/foo/bar/qux/baz.ext"))) > (let ((no-prefix (tbnl:request-pathname r)) > (prefix (tbnl:request-pathname r "/foo/"))) > (format t > "~&no-prefix: ~s ~ > with-prefix: ~s ~%~ > resolved no-prefix: ~s ~%~ > resolved prefix: ~s" > no-prefix prefix > (merge-pathnames no-prefix #p"rgol:www;") > (merge-pathnames prefix #p"rgol:www;"))))) > > ;; no-prefix: #P"RGOL:;FOO;BAR;QUX;BAZ.EXT" with-prefix: > #P"RGOL:;BAR;QUX;BAZ.EXT" > ;; resolved no-prefix: #P"RGOL:WWW;FOO;BAR;QUX;BAZ.EXT.NEWEST" > ;; resolved prefix: #P"RGOL:WWW;BAR;QUX;BAZ.EXT.NEWEST" > > ---------------------------------------------- > > If I can suggest: would it be worth to make request-pathname a generic > function, rather then a plain function? I would imagine few have used > this mechanism to implement their own path resolution scheme, but > theoretically it could be useful. Its name and the way it's been used > seem to ask for it to be a method of request class. > > Best, > > Oleg > > On Sat, Feb 1, 2014 at 2:11 PM, Hans H?bner > wrote: > > Oleg, > > > > James Anderson privately shared the observation that if the document root > > logical pathname is translated (using translate-logical-pathname) before > it > > is merged to the URI's path, the problem would be solved. Can you give > that > > a try? If that works for you, then I'd like to see Hunchentoot be > changed > > so that it call translate-logical-pathname for the configuration paths > that > > the user supplies, in the constructor. The behavior should be > documented. > > > > I've also pondered whether it would be better to do the translation for > > every request, but James rightfully pointed out that one would want to > see > > erroneous logical pathnames early on. Also, the overhead might be > > considerable, so I'm tending towards doing it at server initialization > time. > > This means that if a logical host definition is changed at run time, the > > server must be restarted to pick up the new definition. > > > > -Hans > > > > > > 2014-02-01 Left Right : > >> > >> Ha! > >> I see, I think I've given up on pathnames once, but now I've got more > >> time to try to figure them out, so maybe one day I'll have that patch > >> ;) > >> > >> Thanks for the info, though! > >> > >> Best, > >> > >> Oleg > >> > >> On Sat, Feb 1, 2014 at 12:32 PM, Hans H?bner > >> wrote: > >> > Oleg, > >> > > >> > what would be needed is complete support for logical pathnames (which > is > >> > not > >> > present at the moment). I believe that if the document root is a > >> > logical > >> > pathname, the incoming URL would need to be converted to a relative > >> > logical > >> > pathname before it is merged to the document root. > >> > > >> > At this point, I am not prepared to work on this myself. As pathnames > >> > are > >> > hairy business, I would only want to merge a patch to support logical > >> > pathnames if it comes with thorough tests. > >> > > >> > Alternatively, I would accept a documentation patch that says that if > >> > the > >> > document root is a logical pathname, it must not contain a directory > or > >> > file > >> > component. :) > >> > > >> > -Hans > >> > > >> > > >> > 2014-02-01 Left Right : > >> > > >> >> Thanks for the answer, Hans, > >> >> > >> >> I'm looking now at the misc.lisp, > >> >> create-folder-dispatcher-and-handler, am I right assuming this is the > >> >> function that handles paths merging? > >> >> > >> >> If so, I was thinking to override request-pathname on request to > >> >> create a URI with the required bits of the pathname. This would seem > >> >> to me like a better way to handle it. But I'm not certain what would > >> >> it mean in terms of other bits of the code, is it likely to break > >> >> other things? > >> >> > >> >> Example: > >> >> > >> >> (merge-pathnames (make-pathname :host "rgol" :name "game" :type > >> >> "html") #p"rgol:www;") > >> >> #P"RGOL:WWW;GAME.HTML.NEWEST" > >> >> > >> >> The reason I want to do it this way is because merge-pathnames will > >> >> only concatenate paths if the default-pathname is trivial: no > >> >> directories, wildcards etc. So it seems to me that the original > >> >> intention was to simply concatenate paths, but because for trivial > >> >> pathnames merge-pathnames worked as concatenation, it was used. Does > >> >> it make sense? > >> >> > >> >> Best, > >> >> > >> >> Oleg > >> >> > >> >> On Sat, Feb 1, 2014 at 10:59 AM, Hans H?bner > > >> >> wrote: > >> >> > Oleg, > >> >> > > >> >> > the problem seems to be related how logical pathnames are merged. > >> >> > The > >> >> > partial pathname that is created by Hunchentoot and then merged > with > >> >> > the > >> >> > document root pathname is considered to contain a directory > component > >> >> > by > >> >> > CL. > >> >> > Therefore, the directory component that you are matching in your > >> >> > logical > >> >> > pathname host definition is not present when the two partial > >> >> > pathnames > >> >> > are > >> >> > merged. To work around this behavior, I'd recommend that you > define > >> >> > a > >> >> > separate logical pathname host for your document root. > >> >> > > >> >> > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations > "TEST") > >> >> > '(("foo;*.*.*" "/tmp/*"))) > >> >> > (("foo;*.*.*" "/tmp/*")) > >> >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" > >> >> > #P"test:foo;")) > >> >> > NIL > >> >> > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations > "TEST") > >> >> > '(("*.*.*" "/tmp/*"))) > >> >> > (("*.*.*" "/tmp/*")) > >> >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" > >> >> > #P"test:foo;")) > >> >> > (#P"/private/tmp/test.txt") > >> >> > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" > >> >> > #P"test:")) > >> >> > (#P"/private/tmp/test.txt") > >> >> > > >> >> > -Hans > >> >> > > >> >> > > >> >> > 2014-01-31 Left Right : > >> >> > > >> >> >> Sorry, I've copied the wrong log entry, this is the correct one: > >> >> >> > >> >> >> 127.0.0.1 - [2014-01-31 17:35:44] "GET /game.html HTTP/1.1" 404 > 188 > >> >> >> "-" > >> >> >> "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like > >> >> >> Gecko) > >> >> >> Chrome/28.0.1500.95 Safari/537.36" > >> >> >> > >> >> >> > >> >> >> On Fri, Jan 31, 2014 at 5:33 PM, Oleg Sivokon > >> >> >> > >> >> >> wrote: > >> >> >>> > >> >> >>> Hello list, > >> >> >>> > >> >> >>> There should be something very simple I've overlooked, yet I > can't > >> >> >>> find > >> >> >>> it. Hunchentoot seems not to be able to locate the root > directory, > >> >> >>> where > >> >> >>> I have my static content, and I can't get it to print any useful > >> >> >>> information about it. That's why I'm asking for your help. > >> >> >>> > >> >> >>> Below is my setup: > >> >> >>> > >> >> >>> (setf (logical-pathname-translations "rgol") > >> >> >>> '(... > >> >> >>> ("WWW;*.*.*" "/home/wvxvw/.../www/") > >> >> >>> ("WWW;*;*.*.*" "/home/wvxvw/.../www/*") > >> >> >>> ...)) > >> >> >>> > >> >> >>> (make-instance 'hunchentoot:acceptor :port 4242 > >> >> >>> :document-root #p"rgol:www;" > >> >> >>> :message-log-destination #p"rgol:logs;messages.log" > >> >> >>> :access-log-destination #p"rgol:logs;access.log") > >> >> >>> > >> >> >>> I've defined another handler, which doesn't depend on static > files, > >> >> >>> and > >> >> >>> it works fine, however, when I try to access static files, the > log > >> >> >>> record looks like this: > >> >> >>> > >> >> >>> 127.0.0.1 - [2014-01-31 17:12:40] > >> >> >>> "GET /img/made-with-lisp-logo.jpg HTTP/1.1" > >> >> >>> 404 206 "http://localhost:4242/game.html" > >> >> >>> "Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 > >> >> >>> Firefox/23.0" > >> >> >>> > >> >> >>> But the file is definitely there, because if I try this in REPL: > >> >> >>> > >> >> >>> (directory #p"rgol:www;*.*") > >> >> >>> (#P"/home/wvxvw/.../www/game.html" > >> >> >>> ... more files ...) > >> >> >>> > >> >> >>> My version of Hunchentoot is: > >> >> >>> > >> >> >>> hunchentoot:*hunchentoot-version* > >> >> >>> "1.2.17" > >> >> >>> > >> >> >>> $ sbcl --version > >> >> >>> SBCL 1.1.2-1.fc18 > >> >> >>> > >> >> >>> Best, > >> >> >>> > >> >> >>> Oleg > >> >> >> > >> >> >> > >> >> > > >> > > >> > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From edi at agharta.de Fri Feb 28 16:25:35 2014 From: edi at agharta.de (Edi Weitz) Date: Fri, 28 Feb 2014 17:25:35 +0100 Subject: hunchentoot serving manifest files, ie. .appcache files. In-Reply-To: References: Message-ID: I forgot where the original list came from, but it is obviously outdated, so if there are important mime type/suffix pairs which are missing, I'd suggest to send a patch for mime-types.lisp via GitHub. Cheers, Edi. On Fri, Feb 28, 2014 at 5:03 PM, Left Right wrote: > You could (setf (gethash "manifest" tbnl::*mime-type-hash*) > "text/cache-manifest") to address the immediate issue, however, since > this symbol isn't exported, maybe there was a different plan for > adding more mime-types. > > On Fri, Feb 28, 2014 at 4:53 PM, Faruk S. Can wrote: >> how to serve manifest files, ie. .appcache files wit hunchentoot? >> (mime-type #p"path/to/anifest/file") >>>nil >> nil >> >> is there a way to do? >> >> or is it jsut enough to proxy with apache or nginx front and configure those >> to serve manifest files, as it is noted in w3schools html5- appcache topic? >> >> before i try myself, just asked to bring it to attention for possible >> renewing improvements to the hunchentoot. > From ron at flownet.com Fri Feb 14 18:05:16 2014 From: ron at flownet.com (Ron Garret) Date: Fri, 14 Feb 2014 10:05:16 -0800 Subject: Persistent sessions In-Reply-To: References: Message-ID: Huh??? The last click time is just a slot in the session object. It has nothing to do with the session ID or the cookie. On Feb 14, 2014, at 9:44 AM, peter wrote: > Given that the cookie is the product of md5-hexing the raw composite > cookie data (in encode-session-string), there's no way to get the > initial click (creation) time of the session (which is needed in > order to ascertain whether the session has expired > (session-too-old-p)). > > So presumably, like the session-ID, we'll need to carry the session > start time externally (outside the md5-hexed part of the cookie). > > Are we thinking along the same rg? > > I'm puzzled that session persistence isn't a common requirement of hunchentoot. > > -peter -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: Message signed with OpenPGP using GPGMail URL: From olegsivokon at gmail.com Sat Feb 1 10:09:27 2014 From: olegsivokon at gmail.com (Left Right) Date: Sat, 1 Feb 2014 12:09:27 +0200 Subject: Cant serve static files In-Reply-To: References: <87a9eckvy5.fsf@gmail.com> Message-ID: Thanks for the answer, Hans, I'm looking now at the misc.lisp, create-folder-dispatcher-and-handler, am I right assuming this is the function that handles paths merging? If so, I was thinking to override request-pathname on request to create a URI with the required bits of the pathname. This would seem to me like a better way to handle it. But I'm not certain what would it mean in terms of other bits of the code, is it likely to break other things? Example: (merge-pathnames (make-pathname :host "rgol" :name "game" :type "html") #p"rgol:www;") #P"RGOL:WWW;GAME.HTML.NEWEST" The reason I want to do it this way is because merge-pathnames will only concatenate paths if the default-pathname is trivial: no directories, wildcards etc. So it seems to me that the original intention was to simply concatenate paths, but because for trivial pathnames merge-pathnames worked as concatenation, it was used. Does it make sense? Best, Oleg On Sat, Feb 1, 2014 at 10:59 AM, Hans H?bner wrote: > Oleg, > > the problem seems to be related how logical pathnames are merged. The > partial pathname that is created by Hunchentoot and then merged with the > document root pathname is considered to contain a directory component by CL. > Therefore, the directory component that you are matching in your logical > pathname host definition is not present when the two partial pathnames are > merged. To work around this behavior, I'd recommend that you define a > separate logical pathname host for your document root. > > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST") > '(("foo;*.*.*" "/tmp/*"))) > (("foo;*.*.*" "/tmp/*")) > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" > #P"test:foo;")) > NIL > TEST-LOGICAL-PATHNAMES> (setf (logical-pathname-translations "TEST") > '(("*.*.*" "/tmp/*"))) > (("*.*.*" "/tmp/*")) > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" > #P"test:foo;")) > (#P"/private/tmp/test.txt") > TEST-LOGICAL-PATHNAMES> (directory (merge-pathnames "test.txt" #P"test:")) > (#P"/private/tmp/test.txt") > > -Hans > > > 2014-01-31 Left Right : > >> Sorry, I've copied the wrong log entry, this is the correct one: >> >> 127.0.0.1 - [2014-01-31 17:35:44] "GET /game.html HTTP/1.1" 404 188 "-" >> "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) >> Chrome/28.0.1500.95 Safari/537.36" >> >> >> On Fri, Jan 31, 2014 at 5:33 PM, Oleg Sivokon >> wrote: >>> >>> Hello list, >>> >>> There should be something very simple I've overlooked, yet I can't find >>> it. Hunchentoot seems not to be able to locate the root directory, where >>> I have my static content, and I can't get it to print any useful >>> information about it. That's why I'm asking for your help. >>> >>> Below is my setup: >>> >>> (setf (logical-pathname-translations "rgol") >>> '(... >>> ("WWW;*.*.*" "/home/wvxvw/.../www/") >>> ("WWW;*;*.*.*" "/home/wvxvw/.../www/*") >>> ...)) >>> >>> (make-instance 'hunchentoot:acceptor :port 4242 >>> :document-root #p"rgol:www;" >>> :message-log-destination #p"rgol:logs;messages.log" >>> :access-log-destination #p"rgol:logs;access.log") >>> >>> I've defined another handler, which doesn't depend on static files, and >>> it works fine, however, when I try to access static files, the log >>> record looks like this: >>> >>> 127.0.0.1 - [2014-01-31 17:12:40] >>> "GET /img/made-with-lisp-logo.jpg HTTP/1.1" >>> 404 206 "http://localhost:4242/game.html" >>> "Mozilla/5.0 (X11; Linux x86_64; rv:23.0) Gecko/20100101 Firefox/23.0" >>> >>> But the file is definitely there, because if I try this in REPL: >>> >>> (directory #p"rgol:www;*.*") >>> (#P"/home/wvxvw/.../www/game.html" >>> ... more files ...) >>> >>> My version of Hunchentoot is: >>> >>> hunchentoot:*hunchentoot-version* >>> "1.2.17" >>> >>> $ sbcl --version >>> SBCL 1.1.2-1.fc18 >>> >>> Best, >>> >>> Oleg >> >> >