From mbaringer at common-lisp.net Thu Nov 1 05:30:29 2007 From: mbaringer at common-lisp.net (Marco Baringer) Date: Thu, 1 Nov 2007 00:30:29 -0500 (EST) Subject: [Bese-devel] New patches to ucw_ajax: 31-Oct-2007 Message-ID: <20071101053029.AC1281125@common-lisp.net> Wed Oct 31 18:16:09 EST 2007 attila.lendvai at gmail.com * more lock-held asserts M ./src/rerl/rerl-utils.lisp -11 +19 Wed Oct 31 18:16:00 EST 2007 attila.lendvai at gmail.com * added log-ignorable-error-backtrace M ./src/rerl/request-loop-error.lisp +14 Wed Oct 31 18:13:18 EST 2007 attila.lendvai at gmail.com * housekeeping M ./examples/src/shared-counter.lisp -2 +1 M ./examples/src/sum.lisp +1 M ./src/application-mixins/ajax-application.lisp +2 M ./src/application-mixins/cookie-session-application.lisp +1 M ./src/components/container.lisp +1 M ./src/control.lisp -1 +1 M ./src/helpers.lisp +3 M ./src/parenscript-utils.lisp +1 M ./src/per-application-parenscript.lisp +4 M ./src/rerl/protocol.lisp -2 +3 M ./test/core/action.lisp -3 +2 Wed Oct 31 18:12:33 EST 2007 attila.lendvai at gmail.com * rename httpd threads M ./src/backend/httpd.lisp -3 +3 Wed Oct 31 18:07:17 EST 2007 attila.lendvai at gmail.com * refactor app/session locking and session expiration. TODO left: sessions are not purged, user code must call remove-expired-sessions periodically for now M ./src/application-mixins/ajax-application.lisp -1 +2 M ./src/application-mixins/secure-application.lisp -2 +2 M ./src/packages.lisp -2 +3 M ./src/rerl/basic-action.lisp -8 +9 M ./src/rerl/basic-application.lisp -75 +87 M ./src/rerl/basic-classes.lisp -6 +1 M ./src/rerl/basic-session.lisp -6 +7 M ./src/rerl/conditions.lisp -1 +5 M ./src/rerl/protocol.lisp -10 +16 M ./src/rerl/rerl-utils.lisp +5 Wed Oct 31 11:14:08 EST 2007 attila.lendvai at gmail.com * invert the default value of call-as-response-handler M ./src/rerl/basic-application.lisp -2 +1 M ./src/rerl/standard-server.lisp -2 +1 M ./test/core/application.lisp -3 +3 M ./test/core/server.lisp -1 +2 Tue Oct 30 19:52:22 EST 2007 attila.lendvai at gmail.com UNDO: turn the assert-application-lock-held into a with-lock-held-on-application in delete-session M ./src/rerl/basic-application.lisp -4 +4 Tue Oct 30 11:34:23 EST 2007 attila.lendvai at gmail.com * housekeeping M ./src/components/container.lisp -1 +1 M ./src/components/dojo-widgets.lisp -1 +1 M ./src/rerl/basic-action.lisp -8 M ./src/rerl/basic-application.lisp +3 M ./src/rerl/request-loop-error.lisp -1 +2 M ./src/rerl/standard-server.lisp -1 +1 An updated tarball of ucw_ajax's source can be downloaded here: http://common-lisp.net/project/ucw/tarballs/ucw_ajax-20071031.tar.gz Darcsweb URL: http://uncommon-web.com/darcsweb/darcsweb.cgi?r=ucw_ajax;a=summary From mbaringer at common-lisp.net Fri Nov 2 05:30:22 2007 From: mbaringer at common-lisp.net (Marco Baringer) Date: Fri, 2 Nov 2007 00:30:22 -0500 (EST) Subject: [Bese-devel] New patches to ucw_ajax: 1-Nov-2007 Message-ID: <20071102053022.71A9444065@common-lisp.net> Thu Nov 1 10:16:14 EST 2007 attila.lendvai at gmail.com * added with-session-specific-temporary-file and open-session-specific-temporary-file M ./src/rerl/basic-action.lisp -3 +3 M ./src/rerl/rerl-utils.lisp +42 Thu Nov 1 07:31:36 EST 2007 attila.lendvai at gmail.com * added download-action-href-body M ./src/rerl/basic-action.lisp +11 Thu Nov 1 07:31:14 EST 2007 attila.lendvai at gmail.com * added bind dependency M ./src/packages.lisp +1 M ./ucw.asd -1 +1 Thu Nov 1 07:29:42 EST 2007 attila.lendvai at gmail.com * more cl-def usage M ./src/components/collapsible-pane.lisp -5 +2 M ./src/helpers.lisp -14 +8 M ./src/rerl/basic-action.lisp -19 +19 M ./src/rerl/basic-session.lisp -3 +1 M ./src/rerl/rerl-utils.lisp -2 +2 Wed Oct 31 22:43:15 EST 2007 attila.lendvai at gmail.com * added assert to find-session M ./src/rerl/basic-application.lisp -4 +5 An updated tarball of ucw_ajax's source can be downloaded here: http://common-lisp.net/project/ucw/tarballs/ucw_ajax-20071101.tar.gz Darcsweb URL: http://uncommon-web.com/darcsweb/darcsweb.cgi?r=ucw_ajax;a=summary From sasha-lisp at arcocene.org Fri Nov 2 19:52:31 2007 From: sasha-lisp at arcocene.org (Sasha Kovar) Date: Fri, 2 Nov 2007 11:52:31 -0800 Subject: [Bese-devel] ucw_ajax fix for mod_lisp cookies Message-ID: <20071102195231.GM81018@muted.org> (Take two) Hi. The attached patch brings the mod-lisp backend up to date for the following change: Thu Sep 13 16:17:00 PDT 2007 attila.lendvai at gmail.com * Clean up cookie handling. WARNING: api changed Thanks much, Sasha -------------- next part -------------- A non-text attachment was scrubbed... Name: mod-lisp-cookie-fix.patch Type: text/x-diff Size: 25635 bytes Desc: not available URL: From mbaringer at common-lisp.net Sat Nov 3 05:05:04 2007 From: mbaringer at common-lisp.net (Marco Baringer) Date: Sat, 3 Nov 2007 00:05:04 -0500 (EST) Subject: [Bese-devel] New patches to fiveam: 2-Nov-2007 Message-ID: <20071103050504.7DE06232B3@common-lisp.net> Thu Nov 1 03:15:14 EST 2007 Henrik Hjelte * added :before depends-on option to be used when you really want to safeguard that some other test has not run first. M ./src/run.lisp -1 +5 M ./t/tests.lisp +27 Wed Oct 31 03:50:10 EST 2007 Henrik Hjelte * depends-on with single symbol resolves dependencies ok M ./src/classes.lisp -2 +2 M ./src/run.lisp -1 +1 Wed Oct 31 03:45:16 EST 2007 Henrik Hjelte * bugfixed test of depends on symbol M ./t/tests.lisp -1 +2 Wed Oct 31 03:32:15 EST 2007 Henrik Hjelte * tests for dependencies with symbol M ./t/tests.lisp +31 Mon Aug 21 04:29:48 EDT 2006 Henrik Hjelte * Platform-independent pathnames in make-qbook M ./docs/make-qbook.lisp -2 +7 An updated tarball of fiveam's source can be downloaded here: http://common-lisp.net/project/bese/tarballs/fiveam-20071102.tar.gz Darcsweb URL: http://uncommon-web.com/darcsweb/darcsweb.cgi?r=fiveam;a=summary From mbaringer at common-lisp.net Sat Nov 3 05:30:25 2007 From: mbaringer at common-lisp.net (Marco Baringer) Date: Sat, 3 Nov 2007 00:30:25 -0500 (EST) Subject: [Bese-devel] New patches to ucw_ajax: 2-Nov-2007 Message-ID: <20071103053025.952F153112@common-lisp.net> Fri Nov 2 06:22:32 EST 2007 attila.lendvai at gmail.com * prefix the various urls of window with the application's url-prefix M ./src/components/window.lisp -31 +27 An updated tarball of ucw_ajax's source can be downloaded here: http://common-lisp.net/project/ucw/tarballs/ucw_ajax-20071102.tar.gz Darcsweb URL: http://uncommon-web.com/darcsweb/darcsweb.cgi?r=ucw_ajax;a=summary From henrik at evahjelte.com Tue Nov 6 11:12:06 2007 From: henrik at evahjelte.com (Henrik Hjelte) Date: Tue, 6 Nov 2007 12:12:06 +0100 Subject: [Bese-devel] single-threaded Message-ID: <50e8e4f60711060312s6aaa8773o965485f415cfbe62@mail.gmail.com> Tagging something as controversial is a good way to get immediate feedback :-) First, big congratulations to Attila and friends to the success with your company, it was great to see the posting on comp.lang.lisp ! About the list of recent patches, I see that you are dropping the single-threaded httpd backend. First reflection, don't ucw need a backend that works with all lisps, not only sbcl? Or are all lisps are threaded nowadays.. Second, we like to have a single-threaded version around for safety reasons. If you are suspicious of errors connected to threading, it is good to have a simple implementation around as a fallback option. Also, we have a setup that makes it possible to use a single threaded version in a production environment. It currently looks like this: One server only, at http://stix.to, dual core. We are poor right now but we hope it can grow. First, a nginx on port 80 that can proxy requests to either a test environment or production. Then another nginx process that serves static files, offloading ucw. But the most important thing is that it adds gzip compression on the fly to all requests (except graphics). That really improves the client performance a lot! You can do this with apache as well of course. This nginx process sends ucw requests further to a real proxy server: the haProxy. Nginx can't handle sticky sessions, but the haProxy can, we use cookie based sticky sessions. Then we have four instances of ucw running. They are currently running a multithreaded version of ucw, but we don't think it should be necessary. To further complicate the setup, also we have json-rpc servers that are stateless but running in ucw, so we have a stateless proxy in front of two instances of these as well. Using this setup, we think we can utilize the two cores on the server almost as good or good enough with a single threaded ucw. The queing of requests will be handled by the haProxy, so we don't need multiple threads for that purpose. And since there is a lock on most things in ucw, at most times there will be one thread running the application code anyway. Since we had some troubles, possibly related to threading, I just tried to revive the single threaded httpd backend. I did some simple performance tests using the ab program for some json-rpc requests, throught the proxy. And it seemed like the single threaded httpd backend had a higher throughput than the multithreaded httpd backend. I wouldn't want to say that this is a perfectly done performance test, but there are some theoretical reasons a single-threaded version may perform better: more threads means more context switches and CPU cache pollution. So, is it ok if I revive the single-threaded httpd, or make a single-threaded iolib based backend? Or do you have any plans that conflict with this? Is anyone interested in example files for configuring haProxy, nginx, apache and pound? I have been through them all by now. I can't say firmly which one is best, but I see it like this: apache. Well tested, lots of features. nginx. "They say" it is faster than apache. pound. Ok proxy with sticky sessions. Easy to set up. haProxy. Seems very trustworthy and stable from reading the docs. Configurable, easier to set up than expected. Better logging than pound. "They say" it has the best performance. Has sticky sessions through cookies. /Henrik Hjelte From attila.lendvai at gmail.com Tue Nov 6 11:32:20 2007 From: attila.lendvai at gmail.com (Attila Lendvai) Date: Tue, 6 Nov 2007 12:32:20 +0100 Subject: [Bese-devel] single-threaded In-Reply-To: <50e8e4f60711060312s6aaa8773o965485f415cfbe62@mail.gmail.com> References: <50e8e4f60711060312s6aaa8773o965485f415cfbe62@mail.gmail.com> Message-ID: > So, is it ok if I revive the single-threaded httpd, or make a > single-threaded iolib based backend? Or do you have any plans that > conflict with this? i'm planning another controversial change, namely completly depending on iolib and using usocket only for the httpd backend. the moves you see is mostly guided by making ucw a web server that can cope with high load as a public site (iolib, threads, etc). this does not mean that single threaded mode is not welcome of course, but for now i've merged the httpd backends for simplicity/cleaning up. i'm also planning to keep httpd in a fallback-for-mostly-debugging status and concentrate on a flawless DOS-proof iolib backend capable of handling thousands of connections. > Is anyone interested in example files for configuring haProxy, nginx, > apache and pound? I have been through them all by now. I can't say > firmly which one is best, but I see it like this: i'm also planning to add an iolib based load-balancer, because mod_proxy_balancer in apache is mixing up connections sometimes, returning entire responses to different clients (with a working session id!) > apache. Well tested, lots of features. well, tested... :) sorry for the short answer, expect more later -- attila From mbaringer at common-lisp.net Thu Nov 15 05:30:21 2007 From: mbaringer at common-lisp.net (Marco Baringer) Date: Thu, 15 Nov 2007 00:30:21 -0500 (EST) Subject: [Bese-devel] New patches to ucw_ajax: 14-Nov-2007 Message-ID: <20071115053021.1FD553203C@common-lisp.net> Sat Nov 10 09:55:56 EST 2007 attila.lendvai at gmail.com * change call-in-rendering-environment's arg order to follow the call-* conventions M ./src/components/ajax.lisp -3 +3 M ./src/components/form.lisp -1 +1 M ./src/contextl-integration.lisp -2 +2 M ./src/rerl/protocol.lisp -1 +1 M ./src/rerl/standard-component/standard-component.lisp -2 +2 Mon Nov 5 07:35:55 EST 2007 attila.lendvai at gmail.com * add with-thread-name to handle-action M ./src/rerl/basic-action.lisp -2 +5 Mon Nov 5 04:33:38 EST 2007 attila.lendvai at gmail.com * send a start message to the new acceptor M ./src/backend/httpd.lisp -1 +1 Mon Nov 5 04:33:32 EST 2007 attila.lendvai at gmail.com * KLUDGE: do not read the request if the server is overloaded M ./src/backend/httpd.lisp -2 +6 An updated tarball of ucw_ajax's source can be downloaded here: http://common-lisp.net/project/ucw/tarballs/ucw_ajax-20071114.tar.gz Darcsweb URL: http://uncommon-web.com/darcsweb/darcsweb.cgi?r=ucw_ajax;a=summary From mbaringer at common-lisp.net Tue Nov 27 05:15:10 2007 From: mbaringer at common-lisp.net (Marco Baringer) Date: Tue, 27 Nov 2007 00:15:10 -0500 (EST) Subject: [Bese-devel] New patches to yaclml: 26-Nov-2007 Message-ID: <20071127051510.812817914B@common-lisp.net> Mon Nov 26 07:01:30 EST 2007 attila.lendvai at gmail.com * added ID to the XML package M ./src/packages.lisp +1 An updated tarball of yaclml's source can be downloaded here: http://common-lisp.net/project/bese/tarballs/yaclml-20071126.tar.gz Darcsweb URL: http://uncommon-web.com/darcsweb/darcsweb.cgi?r=yaclml;a=summary