[elephant-devel] Re: Multi user client-server performance

[Alex Mizrahi]

 VV> I'm using sbcl, so it is good to know that elephant with Berkeley db
 VV> does not have such problems.
 VV> But is is surprising to know that such a well established and mature
 VV> lisp implementation as Allegro, cannot thread properly, while much
 VV> younger and not so mature sbcl works better than expensive commercial
 VV> solutions.

i think they just have different quality standards.

 VV> Anyway, I would like to hear the accounts of real life usage of
 VV> elephant in multi-user environment.

in elephant we have several different backends:
 * BDB one is primary one, APIs were modelled according to capabilities of 
BDB
 * CLSQL can work with any SQL database supported by CLSQL (that's good), 
but has some significant "sub-optimalities"
 * Postmodern can work with PostgreSQL only (uses "postmodern" library, 
hence the name) and has less sub-optimalities than CLSQL

postmodern one was specially developed to work with multiple threads, 
multiple processes and on multiple machines,
so it might be worth testing it if "multi-user" is essential for your 
application.
possible benefits :

 * PostgreSQL uses MVCC model rather than locking. so it might allow more 
concurrency
 * PostgreSQL handles deadlocks automatically (iirc you need separate 
deadlock-detection process with BDB)
 * you can have separate server with PostgreSQL and connect to it from 
different machines via network
 * it hash optional client-side cache that is automatically synchronized 
among different processes
 * this backend was designed and tested to work in concurrent environment 
with multiple processes, threads etc

but it has it's negative sides too:
 * inter-process communications have significant overhead, upto 10x, if 
we'll compare to in-process BDB
   (but it's offseted by cache to some point -- we assume that under 
constant load most data gets cached, so
    only index queries and writes need to communicate with PostgreSQL 
backend)
 * some features work differently from BDB backend -- or, in other words, 
they were not implemented correctly because there's no straightforward way 
to do this in SQL backend.
   most significant difference is sorting -- with db-postmodern you get 
correct ordering in index only if all keys are either integers or strings.
 * it's, um, less stable than BDB one, kinda still under development.

as for real-life usage, we are using elephant with postmodern backend as a 
database for a web site, which is assumed to be multi-user one, and it sort 
of works.
but OTOH our load, database size and complexity are relatively small, so i 
can't say it was tested in really harsh environment.