[Bese-devel] ucw and new core-server
Evrim Ulu
evrim at core.gen.tr
Sat Dec 22 13:48:43 UTC 2007
Sorry about the first one, mozilla did want to send a draft copy???
Hi Friends,
I'm authoring this email to inform you about our new progress.
The most important news is our http server is about to be ready. We've
finally take the advantage of our parser/renderer functions on monad
like streams named core-streams.
A more interesting news would be, the http server doesnt care if the
request is mod-lisp or http-request directly. Parser can differentiate
both requests and work universally. So no more :backend option neeeded
for the server to run.
It can be sad news that we've dropped the ucw and related libraries from
core-server since we've written most of the libs (about 15) and made
them small/compact/well-tested.
Here is the total dependency list: (#'s are commented libs ie rewritten)
http://www.core.gen.tr/projects/core-server/src/install/lib.conf
# Core Server 1.0 Dependencies
core-server darcs http://www.core.gen.tr/projects/core-server/
#ucw_dev darcs
http://common-lisp.net/project/ucw/repos/ucw_dev/
#ucw+ darcs http://www.core.gen.tr/projects/ucw+/
core-services darcs http://www.core.gen.tr/projects/core-services/
cl-prevalence darcs http://www.core.gen.tr/projects/cl-prevalence/
yaclml darcs http://www.core.gen.tr/projects/yaclml/
#trivial-garbage darcs
http://common-lisp.net/~loliveira/darcs/trivial-garbage/
#local-time darcs
http://common-lisp.net/project/local-time/darcs/local-time/
bordeaux-threads darcs
http://common-lisp.net/project/bordeaux-threads/darcs/bordeaux-threads/
arnesi darcs http://www.core.gen.tr/projects/arnesi_dev/
parenscript darcs
http://common-lisp.net/project/ucw/repos/parenscript/
#iterate darcs http://common-lisp.net/project/iterate/darcs/iterate
#rfc2388 darcs
http://common-lisp.net/project/ucw/repos/rfc2388-binary/
#rfc2109 darcs http://common-lisp.net/project/rfc2109/rfc2109/
#cl-l10n darcs
http://common-lisp.net/project/cl-l10n/repos/cl-l10n/
s-sysdeps darcs http://www.beta9.be/darcs/s-sysdeps
#dojo svn http://svn.dojotoolkit.org/dojo/trunk/
dojo tar http://www.core.gen.tr/projects/dojo-11132.tar.gz
fckeditor tar
http://dfn.dl.sourceforge.net/sourceforge/fckeditor/FCKeditor_2.5b.tar.gz
s-xml cvs common-lisp.net:/project/s-xml/cvsroot
slime cvs common-lisp.net:/project/slime/cvsroot
cl-ppcre tar http://weitz.de/files/cl-ppcre.tar.gz
cl-fad tar http://weitz.de/files/cl-fad.tar.gz
#split-sequence tar
http://common-lisp.net/project/ucw/ucw-boxset/split-sequence.tar.gz
#trivial-sockets darcs
http://common-lisp.net/project/bese/repos/trivial-sockets_until-i-can-merge-with-the-mainline
#puri tar http://files.b9.com/puri/puri-latest.tar.gz
#detachtty darcs
http://common-lisp.net/project/bese/repos/detachtty/
#net-telent-date tar
http://common-lisp.net/project/ucw/ucw-boxset/net-telent-date.tar.gz
#parse-number tar
http://common-lisp.net/project/asdf-packaging/parse-number-latest.tar.gz
asdf-binary-locations darcs
http://common-lisp.net/project/cl-containers/asdf-binary-locations/darcs/asdf-binary-locations
rt tar http://files.b9.com/rt/rt.tar.gz
The question that may arise is, what we've written to replace ucw and
friends?
This question was actually the pain that we've facing last 2 years.
We've used ucw in several projects and did evaluated several other
frameworks in other languages like plt,hops etc.
Since our aim is to make small/compact and reusable web applications,
we've pushed ucw to it's limits. The main problem we've faced is
definitely the mop based backtracking . I must admit it leaks as the
project grows.
The only control operator ucw has is the action/action-href macro which
saves the k to some slot which allows ucw to backtrack component tree.
== Web Framework ==
Our new web framework is like the PLT's. We did want small/compact and
easily testable web framework.
http://www.core.gen.tr/projects/core-server/src/applications/http.lisp
Here is the example application to demonstrate control operators of our
new web framework: http://www.core.gen.tr/projects/core-server/t/http.lisp
It has send/suspend, action/url, function/url and answer control operators.
* send/suspend is the same as the PLT's which escapes from call/cc at
the end of its' progn.
* action/url, function/url are similar to action-href in ucw which saves
the continuation.
* answer is the same as in ucw which returns from the last send/suspend
point.
Of course, we'r open to questions and advices that may make the control
operators solid. Our aim is now just to open a discussion about those
control operators and make them universal.
It was a problem for us while using ucw to make send/suspends only via
calling components. This disallows us to combine several components in
one send/suspend so, the component api was not enough. This operator now
allows us to combine without doing IoC everwhere.
== Template Application ==
Core-serveR actually has template applications to start a new project:
(defparameter *a-project* (make-darcs-application "www.newproject.com"
;; fqdn
"newproject" ;;
project name
"aycan at core.gen.tr"))
;; admin-email
(serialize *a-project*)
(evaluate *a-project*)
;;(share *a-project*)
Serialize generates source code from template application here:
http://www.core.gen.tr/projects/core-server/src/applications/darcs.lisp
This application can be overriden of course for other application
tepmlates, but that does the most of the work for us now. Once you've
serialized the application to projects/ folder inside the server,
evaluate can load the project to make it run. share method allows us to
do several darcs operations at once ie: init, record and put to a server
that you want.
Similar to darcs-application, we've build a git application for users.
== RFC Compliancy ==
We've implemented several
rfcs:http://www.core.gen.tr/projects/core-server/src/rfc/
2045, 2046, 2109, 2388, 2396, 2616, 2617, and some 822. Those rfc
implementations are very powerful and most of them are complete.
http://paste.lisp.org/display/42477 paste gives ideas about the
completeness of the http protocol.
== The Server Base aka Units ==
Servers are threads that manages others threads to serve response for
incoming requests. Units allows us to combine OOP and Threads seamlessly.
http://www.core.gen.tr/projects/core-server/src/units.lisp
The http server is actually only a socket server with custom http peer.
http://www.core.gen.tr/projects/core-server/src/servers/http.lisp
Peer is name that we give to the managed threads of the server. Simple
http peer is similar to:
http://www.core.gen.tr/projects/core-server/src/peers/http.lisp
== Installation and Support ==
The server itself can be installed on to several different linux distros
ie: Gentoo, Debian, Pardus and Ubuntu. It currently runs only on SBCL
but we've isolated the threads and sockets code to make it portable in
future.
Core-serveR installation is done very easily via installer script called
core-server-installer which can be found at
http://www.core.gen.tr/projects/core-server-installer-latest.tar.gz
Further distro specific documentation can be found at:
http://www.core.gen.tr/projects/core-server/doc/
If one runs installer as user, server run as user on port 8080. Once one
runs installer as root, core-server would gracefully configures apache,
mod-lisp and several other unix services like sudo. This allows us to
manage apache virtual hosts inside the server. Configuring other systems
is not much pain anymore. Also, we can have webdav enabled applications
in few seconds, see the template apache config:
http://www.core.gen.tr/projects/core-server/etc/vhost.conf
== Web Component Framework ==
We've done lots of projects on web to accomplish very simple tasks and
to get reusable, simple, fast and efficient results. Finally we propose
a new and simple javascript based web framework which employs
continuations and OOP.
http://paste.lisp.org/display/52919 URL shows the basics of the new
component framework. It employs automatic proxy generation for server
methods which employed k capturing. One may evaluate it here:
http://office1.core.gen.tr/test/test.core (warning: no html output
there, see firebug)
These methods allows developers to write methods of the component on the
client side to make the application fast. JSOn is used to
serialize/deserialize parameters and new dojo1.0 is used for
XMLHttpReqs. Simple requests can be done in <100ms since the protocol
overhead is very small. Actually we had lots of debates on dojo but, the
new api consists only 2 functions so switching to something (ie YUI)
other than dojo would be painless. Here are two functions:
baseUrl = '/test/test.core';
function funcall(url, parameters) {
var result;
console.debug('server.funcall ', url);
if (dojo.isObject(parameters)) {
for (var param in parameters) {
parameters[param] = serialize(parameters[param]);
};
};
dojo.xhrPost
({ url : baseUrl + url,
handleAs : 'text',
sync : true,
timeout : 10,
content : parameters,
load : function (json, args) {
result = eval(json);
},
error : function (err, args) {
throw 'Funcall error: ' + url + ', ' + err;
} });
return result;
};
function serialize(value) {
return dojo.toJson(value);
};
== Final Words ==
We did not commit the component framework yet but it'll be in the next
week. Meanwhile, we would be happy to hear from friends to make
core-server better. It's obvious that there are a lot more to go
especially documentations. We'll try to handle these issues in future,
so be ready!
Best Regards,
Evrim Ulu
evrim at core.gen.tr
http://www.core.gen.tr
More information about the bese-devel
mailing list