[elephant-devel] using one store with mutiple lisp instances

[Alex Mizrahi]

> I am curious why the postgres backend is so unpopular.

I would not recommend postgres backend as a default choice because it 
has its own "peculiarities". Particularly, it requires a discipline with 
use of types, as only strings and (64-bit) integers are properly sorted. 
Another major drawback is that there is no GC for blobs so it requires 
export/import to reclaim space from time to time.

But if using multiple machines is a requirement then it looks like 
postgres is the only option, as Oracle confirms that use of BDB DB on 
remote filesystem doesn't work for multiple clients:

http://download.oracle.com/docs/cd/E17076_02/html/programmer_reference/env_remote.html

> As far as I
> know there is no real data to support that it is slow in "real world"
> applications.

There is certain overhead when working with blobs, so it is slower than 
it might be in theory. But whether it is "slow" is another question. DB 
can bottleneck in multiple places (disk operations, concurrency locks, 
network...) so there is no simple answer.

To get a good performance with postgres backend one needs to understand 
that each DB-related operation means a DB query (which potentially goes 
over the network). So it is a good idea to structure code in a way that 
minimizes number of slot reads/writes, range queries etc.

> It also has caching mechanisms that
> work in multi-process applications.

Yeah, this gives some freedom with slot reads (although still has some 
overhead comparing to CLOS slot reads).

If we implement cache for range queries as well we can get to a 
situation where almost nothing is read from DB as long as there is 
enough memory for cache. This would have a huge impact of DB 
performance, I think.

Another possible improvement is to reimplement cache synchronization 
with notifications instead of DB tables, as this might drastically 
reduce overhead of cache synchronization. (Update notifications won't be 
written to disk.)

 > why should berkeley db have an edge in that case?

With improved caching probably postgres will have an edge for common 
applications.