[elephant-devel] Understanding real-world use of Elephant

[Henrik Hjelte]

On Fri, 2007-05-25 at 16:32 -0400, lists at infoway.net wrote:
> 
> > I think that SQL databases are a safer bet than Berkeley DB
> > for having several processes on different machines talking to the same
> > store, so I will have one instance of postgresql running on a server
> > with scsi raid 10 and lots of ram.
> 
> Henrik, would you mind elaborating more on this? Why would SQL databases be safer than the BDB stores? I know they are handled by separate processes, potentially, on separate machines, so in essence, they are independent of your application. However, isn't BDB designed just to tackle that using an application library instead of a separate process?

I'm perhaps guided by feeling rather than theory. I have experience with
sql databases and trust them completely with my data, I don't know BDB
enough to do that. Also as Ian said they might be better when several
machines are involved. And with postgresql I can choose between
different replication methods so I can have a clustered solution if I
want to. Maybe I can with BDB also, I don't know.

Also, I have made some testing and evaluation. I intended to code tests
for the different Lisp ODB to be able to see their merits, and I made the 
project http://common-lisp.net/project/grand-prix for that purpose. 
However, since several ODB failed on the first test I made up my
mind early. The simple test is to see if several instances of Lisp can 
write to the same store at the same time. Rucksack can't and with elephant-BDB
I had also problems. Ian tried to help me solve them, but the fact that I had to
redesign the code a bit to make it work means I don't think BDB is as "safe" as the
sql backends. All sql based backends (perec, clsql,postmodern) worked. 
If you come up with some nice performance test you might consider adding it
to grand-prix, that was my idea with the project anyway.

I have also done some stress testing with my web-app. BDB is fastest, but the 
sql backends do good enough for me. Actually I found a performance problem with 
and ODB from a well known Lisp vendor that would take too long time doing commits. 
But they immediately started to fix it, so it is probably solved by now. But I
no longer have a license to check it out.

Henrik Hjelte