[elephant-devel] using one store with mutiple lisp instances

[lukas.georgieff@hormail.com]

Hi,

since the last mails I am are working on an idea of a federated DB - so we
would use two separate DBs on two machines.
The application is designed to synchronize the stored data over Atom Feeds
(not because of the last problem, but as a main feature of the app).
Since it is not critical to be always 100% synchronized this concept is an
option.
We will gain some benefits in parallel reading-time and availability.
Additionally, we can use the our Backend without changes.


Sincerely,

Lukas


-----Original Message-----
From: Alex Mizrahi [mailto:alex.mizrahi at gmail.com] 
Sent: Samstag, 7. Mai 2011 18:20
To: elephant-devel at common-lisp.net
Subject: Re: [elephant-devel] using one store with mutiple lisp instances

> I am curious why the postgres backend is so unpopular.

I would not recommend postgres backend as a default choice because it 
has its own "peculiarities". Particularly, it requires a discipline with 
use of types, as only strings and (64-bit) integers are properly sorted. 
Another major drawback is that there is no GC for blobs so it requires 
export/import to reclaim space from time to time.

But if using multiple machines is a requirement then it looks like 
postgres is the only option, as Oracle confirms that use of BDB DB on 
remote filesystem doesn't work for multiple clients:

http://download.oracle.com/docs/cd/E17076_02/html/programmer_reference/env_r
emote.html

> As far as I
> know there is no real data to support that it is slow in "real world"
> applications.

There is certain overhead when working with blobs, so it is slower than 
it might be in theory. But whether it is "slow" is another question. DB 
can bottleneck in multiple places (disk operations, concurrency locks, 
network...) so there is no simple answer.

To get a good performance with postgres backend one needs to understand 
that each DB-related operation means a DB query (which potentially goes 
over the network). So it is a good idea to structure code in a way that 
minimizes number of slot reads/writes, range queries etc.

> It also has caching mechanisms that
> work in multi-process applications.

Yeah, this gives some freedom with slot reads (although still has some 
overhead comparing to CLOS slot reads).

If we implement cache for range queries as well we can get to a 
situation where almost nothing is read from DB as long as there is 
enough memory for cache. This would have a huge impact of DB 
performance, I think.

Another possible improvement is to reimplement cache synchronization 
with notifications instead of DB tables, as this might drastically 
reduce overhead of cache synchronization. (Update notifications won't be 
written to disk.)

 > why should berkeley db have an edge in that case?

With improved caching probably postgres will have an edge for common 
applications.


_______________________________________________
elephant-devel site list
elephant-devel at common-lisp.net
http://common-lisp.net/mailman/listinfo/elephant-devel