[elephant-devel] Re: BDB vs postmodern

[Alex Mizrahi]

 ??>> elephant reads/writes slots individually, so this means we cannot
 ??>> have more
 ??>> than 1000-10000 slots operations per second. that sounds quite
 ??>> prohibitive.

 JD> So with an SQL engine this translates to one SELECT/INSERT/UPDATE per
 JD> slot value read or write?

something like that

 JD>  Are you using transactions and prepared queries with PostgreSQL?

of course.

i've done some benchmarking on socket level -- simple [perl] client that 
connects to local echo service and repeatedly sends 10 bytes to socket and 
reads them back (simulating request/response).
on Pentium D 3Ghz (dual core) it was able to do 30000 requests per second. 
on AMD Athlon 2Ghz x64 dual core -- 15000 requests per seconds.

this effectively limits of what can be done via sockets, no matter how 
efficient protocol and code will be. of course in real world we need to 
spend some time processing data.
on my machine PostgreSQL was showing times like 0.1 ms for simple selects, 
so it's unlikely it can do more than 10000 requests.

i didn't benchmark BDB myself, but here's a document from Oracle: 
http://www.oracle.com/technology/products/berkeley-db/pdf/berkeley-db-perf.pdf
they mention performance like 100,000-1,000,000 operations per second on 
commodity hardware, that's orders of magnitude faster than what we can 
achieve via sockets.
reasons for such difference in performance are also mentioned in this 
document:
"Client/Server architecture increases complexity and slows down performance 
because applications must cross a process boundary and often a network link 
to access data."
that's what i was saying..