[Bese-devel] UCW: The Future.

[Henrik Hjelte]

On Tue, Apr 29, 2008 at 11:52 PM, Drew Crampsie <drewc at tech.coop> wrote:
> Hello UCW users/developers,
>
>  The UCW project has been without direction for quite a while now.
>  We've got one of the most advanced web application development
>  environments out there, but it's not used widely, unstable,
>  underdocumented and a bit of a mess. In order to alleviate this
>  situation, and to bring UCW into the mainstream lisp web world, I've
>  volunteered to step in as maintainer/visionary.

Great.

UCW is not perfect, but not as bad as it sounds either. I think it is more
used than people think, there are some real production sites using it.

>  I'd like to find out who's using UCW, how they are using it, what
>  versions they are using (-dev, -ajax, -something-else) and what they'd
>  like to see happen to the project.

The http://stix.to true beta uses ucw_ajax, with some internal additions.
We have tried almost all backends, but are currently using iolib.
As a frontend to add gzip compression and for static files we use the
nginx webserver (stable version)and the HAProxy as a proxy server to
distribute requests with sticky sessions to currently 4 sbcl processes on
one dual-core server. Two of the processes are dedicated to serve
json-rpc requests.

The persistence library is elephant with a postgresql backend.

Since the processes crashes or becomes unresponsive from time to
time we have a monitoring script to check and restart the processes.
No big deal, Ruby works the same way :-) And I think Java does too,
except it never crashes just takes forever to load. The crashes also
seems to have disappeared with the newest sbcl version.

So far, we have been quite happy with ucw, but the basic stix.to site is
quite basic so far, and we will need to develop or improve it further.
In what direction? I don't know yet.. we hope to need scalability and
great features, and a framework we like. It might be ucw or the dwim
branch or something new. But, I see no need to do everything
from scratch, so the basic webserver functionality we will take from
one of these projects. I like that there are choices of backends in ucw,
if you have problem with one you can easily choose another. But it
can go to far also. Two or three should be enough. One simple, one fast
and a third for fun..

>  I'm going to be very aggressive in
>  following a few ideals/guidlines for UCW that should make it more
>  widely usable.
>
>  1) No arbitrary breaking. The test suite will be run and its results
>  included in every commit message. the API will not change drastically
>  all the time. Libraries will not be added/removed without taking steps
>  to make sure things work. A buildbot will be setup. UCW will become a
>  lot more professional.

I agree completely. It is important to make it possible to contribute to the
project without being afraid of breaking things. I think this will
increase the quality and involvement of developers.
Also, everyone should be responsible of making sure that the tests
are of high quality as well. For some reason Lispers seldom like testing.

I would like a fanatic test setup:
* Unittests in the same file as the code, to serve as tests and documentation.
* Refusal to add new features if they don't come with testcases. Or put them
in a second class code repo until they have tests..
* Code without tests is code you are allowed to remove.

We might consider selenium tests for browser testing.

>  4) documentation will be written, and maintained. Now new features
>  will be added unless documented and tested.
But if something is well written and well tested i see no use for
documentation in the code. When the documentation gets rotten it only
adds to the confusion. I fully agree with Attila in his second response.
But high level descriptions and documentation serves a purpose.

>
>  5) modularization will be more explicit. I added a patch a while ago
>  to allow a more 'plugin' like architecture, and this will be
>  exploited. component libraries, form libraries etc will not be part of
>  UCW proper as there are many possible implementations. Rather, a good
>  flexible modular architecture will let the user choose to mix and
>  match higher level code, be it Lisp-on-Lines, ucw-ajax, or
>  what-have-you.

It is a good goal. The problem is that it is difficult to do this grand
design, perfect for everyone. UCW has already been very modular,
despite this I think it has failed on this account. The very OO style
design has not solved every problem. I think more plug-in functions
should be used. Not one function for standard-error-handler, since
everyone will want his own. Rather a funcall *standard-error-handler*.
or generic function on a application object.

>
>  6) Javascript will _not_be required, but will be well supported. A
>  'simple' set of ucw operators will be introduced and used for the
>  examples and documentation.

Ok, for some sites it might be necessary, but I think rather that more
javascript support should be a more important feature. Some ideas:
allow a component to add javascript to the head and to the end of
the body easily, not just add things to the current output stream.
Allow a component to say: I depend on xyz.js, and the code
generation framework could assemble all distinct include
files. Perhaps optimize file load by seeing that xyz.js is also in the
optimized file complete.js

>
>  I have great plans for UCW over the next couple months. I'd like to
>  ask everybody who reads this to chime in now and say your piece...
>  lets make UCW the poster-boy for lisp web frameworks!

If you have great plans you haven't mentioned, consider sharing them.
The last discussion on refactoring UCW did not occur on the mailing list,
and I think we should avoid the situation of going from one main
developer to another one to another one. If we could get a group working
it would be better, but then we need to share our plans and designs
better I think, if possible.

Thanks Marco and Attila for everything you have done, you have both
taught me a lot, and thanks Drew for stepping in and continuing this,
I think it will be great!

I hope and think we can help as well,

Henrik Hjelte