[elephant-cvs] CVS elephant/doc
rread
rread at common-lisp.net
Tue Jan 24 20:37:43 UTC 2006
Update of /project/elephant/cvsroot/elephant/doc
In directory common-lisp:/tmp/cvs-serv874
Modified Files:
elephant.texinfo tutorial.texinfo
Added Files:
installation.texinfo
Log Message:
Improving the documentation with some stuff about the tests, I hope.
--- /project/elephant/cvsroot/elephant/doc/elephant.texinfo 2005/11/23 17:51:34 1.2
+++ /project/elephant/cvsroot/elephant/doc/elephant.texinfo 2006/01/24 20:37:43 1.3
@@ -43,7 +43,7 @@
* Introduction:: Introducing Elephant!
* Tutorial:: A leisurely walk-through.
* Reference:: API documentation.
-* SQL back-end:: CL-SQL based implementation
+* Installation:: Installation and test-suite procedures and issues
* Design Notes:: Internals.
* Copying:: Your rights and freedoms.
* Concept Index::
@@ -57,7 +57,7 @@
@include tutorial.texinfo
@include reference.texinfo
@include notes.texinfo
- at include sql-backend.texinfo
+ at include installation.texinfo
@include copying.texinfo
@node Concept Index
--- /project/elephant/cvsroot/elephant/doc/tutorial.texinfo 2006/01/24 18:25:00 1.3
+++ /project/elephant/cvsroot/elephant/doc/tutorial.texinfo 2006/01/24 20:37:43 1.4
@@ -49,10 +49,12 @@
your store! We'll assume in this tutorial you created a folder
@code{testdb} in the current directory.
-Assuming you've managed to install Elephant properly,
+Assuming you've managed to install Elephant properly, and
+are using a BerkeleyDB installation.
@lisp
* (asdf:operate 'asdf:load-op :elephant)
+* (asdf:operate 'asdf:load-op :ele-bdb)
@end lisp
will load the relevant files.
--- /project/elephant/cvsroot/elephant/doc/installation.texinfo 2006/01/24 20:37:43 NONE
+++ /project/elephant/cvsroot/elephant/doc/installation.texinfo 2006/01/24 20:37:43 1.1
@c -*-texinfo-*-
@node Installation
@comment node-name, next, previous, up
@chapter Installation
@cindex Installation
@menu
* Installation Basics:: Basic installation
* Test-Suites:: Running the test suites
* SQL-Introduction:: The design and status of the SQL back-end extention.
* Extention Status:: The current status of the SQL back-end extention.
* Multi-repository Operation:: Specifying repositories
* Setting up PostGres:: An example
* Repository Migration:: How to move objects from one repository to another
@end menu
@node Installation Basics
@comment node-name, next, previous, up
@section Installation
Please see the file ``INSTALL'' in the source distribution for
more precise information; this is an overview.
Installation of Elephant itself is easy because of the asdf system.
Just execute:
@lisp
(asdf:operate 'asdf:load-op :elephant)
@end lisp
However, Elephant cannot function without a back-end repository.
Elephant presents exactly the same API no matter what you choose
as a repository. However, you have to use asdf to load the
code that interfaces to particular repository system.
The basic choices are to use the BerkeleyDB system or
a SQL based system. You must perform one of these:
@lisp
(asdf:operate 'asdf:load-op :ele-clsql)
(asdf:operate 'asdf:load-op :ele-bdb)
@end lisp
If you choose a SQL based system, you may have to
load a specific package for that system, such as:
@lisp
(asdf:operate 'asdf:load-op :ele-sqlite3)
@end lisp
or, for Postgres,
@lisp
(asdf:oos 'asdf:load-op :clsql-postgresql-socket)
@end lisp
You will have to have the CL-SQL package installed. Following the
documentation for CL-SQL under the section ``How CLSQL finds and loads foreign
libraries'' you may need to do something like:
@lisp
(clsql:push-library-path "/usr/lib/")
@end lisp
before doing
@lisp
(asdf:oos 'asdf:load-op :clsql-postgresql-socket)
@end lisp
in order for clsql to find the PostGres library libpq.so, for example.
Without modifcation, Elephant uses this as it's lib path:
@lisp
/usr/local/share/common-lisp/elephant-0.3/
@end lisp
So you could put a symbolic link to libpq.so there, where libmemutil.so and
libsleepycat.so will also reside.
Elephant is designed to allow multi-repository operation;
so you could concievably use two or more repositories at the
same time. More particularly, you can seamlessly migrate your
data from one repository to a different one at a later date.
In a long duration project, this might occur because of a licensing
or performance issue with a particular respository.
@node Test-Suites
@comment node-name, next, previous, up
@section Test-Suites
Elephant is moderately mature. Hopefully, it will work out-of-the-box
for you.
However, if you are using an LISP implementation different than the ones
on which it is developed and maintained (currently OpenMCL, SBCL, and ACL),
or as the repositories evolve, or just because of mistakes, you may need
to run the test suites. If you report a bug, we will ask you
to run these tests and report the output. Running them when you
first install things may give you a sense of confidence and understanding
that makes it worth the trouble.
There are three files that execute the tests. You should choose
one as a starting point based on what backend(s) you are using.
If using BerekleyDB, use
@lisp
BerkeleyDB-tests.lisp
@end lisp
If using both, use both of the above and also use:
@lisp
MigrationTests.lisp
@end lisp
The text of this file is included here to give the
casual reader an idea of how elepant test can be run in general:
@lisp
;; This file is an example of how to perform the
;; migration tests. You will have to modify it
;; slightly depending on the systems that want to test...
;; You can test migration even between two BDB respositories if you wish
(asdf:operate 'asdf:load-op :elephant)
(asdf:operate 'asdf:load-op :ele-clsql)
(asdf:operate 'asdf:load-op :clsql-postgresql-socket)
(asdf:operate 'asdf:load-op :ele-bdb)
(asdf:operate 'asdf:load-op :elephant-tests)
;; For sqlite-3..
;; (asdf:operate 'asdf:load-op :ele-sqlite3)
(in-package "ELEPHANT-TESTS")
;; The primary and secondary test-paths are
;; use for the migration tests.
;; This this configuration for testing between BDB and SQL....
(setq *test-path-primary* *testpg-path*)
;; (setq *test-path-primary* *testsqlite3-path*)
(setq *test-path-secondary* *testdb-path*)
;; This this configuration for testing from one BDB repository to another...
(setq *test-path-primary* *testdb-path*)
;; (setq *test-path-primary* *testsqlite3-path*)
(setq *test-path-secondary* *testdb-path2*)
(do-migrate-test-spec *test-path-primary*)
@end lisp
The appropriate test should execute for you with no errors.
If you get errors, you may wish to report it the
@code{ elephant-devel at common-lisp.net} email list.
@node SQL-Introduction
@comment node-name, next, previous, up
@section SQL-Introduction
Although originally designed as an interface to the BerkeleyDB system,
the original Elephant system has been experimenetally extended to
support the use of relational database management systems as the
implementation of the persistent store. This relies on Kevin Rosenberg's
CL-SQL interface to relational systems.
Although the BerkeleyDB system is an ideal object store for LISP objects,
one might prefer the licensing of a different system. For example, at
the time of this writing, it is my interpretation that one cannot use
the BerkeleyDB system behind a public website
http://www.sleepycat.com/download/licensinginfo.shtml#redistribute
unless one releases the entire web application as open source.
Neither the PostGres DBMS nor SQLite 3 has any such restriction. Elephant itself is released
under the GPL. It is somewhat debatable if the GPL allows one to construct
to construct a non-open-source web application but the preponderance of
opinion appears to be that it does. Thefore using Elephant and the other
GPLed software that it depends upon allows one to host a a non open-source
web application. This might be a reason to use Elephant on PostGres of SQLite rather
than Elephant on BerkeleyDB.
Other reasons to use a relational database system might include:
familiarity with those systems, the fact that some part of your application
needs to use the truly relational aspects of those systems, preference for
the tools associated with those systems, etc.
The SQL back-end extention of Elephant provides a function for migrating
data seamlessly between repositories. That is, one can quite easily move
data from a BerkeleyDB repository to a PostGres repository, and vice versa.
In fact, one of the most important aspects of the extention is that it
makes Elephant a multi-repository system, rather than a single repository
system, as addition to allowing different implementation strategies for
those repositories. This offers at least the possiblity than once
can develop using one backend, for example BerkeleyDB, and then later
move to MySQL.
At the time of this writing, the basic strategy for the SQL implementation
is quite simple. The same serializer used for the Sleepycat implementation
is employed, the byte-string is base64 encoded, and placed in a single
table which is managed by Elephant.
As of Elephant 0.3, Elephant has been tested to work with both Postgres, and
SQLite 3, thanks do Dan Knapp.
@node Extention Status
@comment node-name, next, previous, up
@section Extention Status
As far as is known at this writing, all functionality except nested transaction
support and cursor-put's that is supported by the BerkeleyDB backend is supported by the CL-SQL
based back-end. Concurrency and transaction atomicity has not been tested well
for the CL-SQL based system.
Additionally, it is NOT the case that the Elephant system currently provides
transaction support across multiple repositories; it provides the transaction
support provided by the underlying repository to the user in a per-repository
basis.
The PostGres backend is as currently employed is about 5 times slower than
the BerkeleyDB backend. This could probably change with continued development.
CL-SQL supports a lot of DBMS systems, but only PostGres has been tested.
The SQL back-end extention has only been tested under SBCL 0.8.18.
The SQL back-end is as easy to use as the BerkeleyDB back-end. However,
the multi-repository version somewhat complicates the underlying
persistent object management. At the time of this writing, the
community has not decided if this extention will be a part of
Elephant proper or a separate branch; if it is not made a part of
Elephant proper, a user might prefer the simpler (and better maintained?)
system if they only want to use the BerkeleyDB back-end.
@node Multi-repository Operation
@comment node-name, next, previous, up
@section Multi-repository Operation
Elephant now keeps a small hashtables that maps ``database specifications'' into
actual database connections.
If a database spec is a string, it is assumed to be a BerkeleyDB path.
If it is a list, it is a assumed to be a CL-SQL connection specification.
For example:
@lisp
ELE-TESTS> *testdb-path*
"/home/read/projects/elephant/elephant/tests/testdb/"
ELE-TESTS> *testpg-path*
(:postgresql "localhost.localdomain" "test" "postgres" "")
ELE-TESTS>
@end lisp
The tests now have a function @code{do-all-tests-spec} that take a spec and
based on its type attempt to open the correct kind of store controller and
perform the tests.
The routine @code{get-controller} takes this specifiation.
The basic strategy is that the ``database specification'' object is stored in
every persistent object and collection so that the repository can be found.
In this way, objects that reside in different repositories can coexist within
the LISP object space, allowing data migration.
@node Setting up PostGres
@comment node-name, next, previous, up
@section Setting up PostGres
To set up a PostGres based back end, you should:
@enumerate
@item Install postgres and make sure postmaster is running.
@item Create a database called ``test'' and set its permissions
to be reached by whatever connection specification you intend to use. The
tests use:
@lisp
(defvar *testpg-path*
'(:postgreql "localhost.localdomain" "test" "postgres" ""))
@end lisp
meaning that connections must be allowed to the database test, user ``postgres'',
no password, connected from the same machine ``localhost.localdomain''.
(This would be changed to something more secure in a real application.)
Typically you edit the file : pg_hba.conf to enable various kinds of connections
in postgres.
@item Be sure to enable socket connection to postgres when you invoke the postmaster.
@item Test that you can connect to the database with these credentials by running:
@code{ psql -h 127.0.0.1 -U postgres test}
Before you attempt to connect with Elephant.
@end enumerate
meaning that connections must be allowed to the database test, user ``postgres'',
no password, connected from the same machine ``localhost.localdomain''.
(This would be changed to something more secure in a real application.)
Furthermore, you must grant practically all creation/read/write privileges
to the user postgres on this schema, so that it can construct the tables
it needs.
Upon first opening a CL-SQL based store controller, the tables, indexes,
sequences, and so on needed by the Elephant system will be created in the
schema named ``test'' automatically.
To run the tests, execute:
@lisp
(asdf:operate 'asdf:load-op :elephant)
(asdf:operate 'asdf:load-op :ele-clsql)
(asdf:oos 'asdf:load-op :clsql-postgresql-socket)
(in-package "ELEPHANT-TESTS")
(do-all-tests-spec *testpg-path*)
@end lisp
This should produce a small number of errors (about 7) for those test having
to do with migration and the BerkeleyDB system specifically.
If you execute:
@lisp
(asdf:operate 'asdf:load-op :ele-bdb)
@end lisp
Then connection to the BerkeleyDB system will be enabled, and you should
be able to execute both
@lisp
(do-all-tests-spec *testpg-path*)
(do-all-tests-spec *testdb-path*)
@end lisp
with no errors in either case.
At present the system has only been tested under PostGres. Some code
parametrization would be required to work with other databases.
Setting up SQLite3 is even easier. Install SQLite3 (I had to use
the source rather than the binary install, in order to get the dynamic
libraries constructed.)
An example use of SQLLite3 would be:
@lisp
(asdf:operate 'asdf:load-op :elephant)
(asdf:operate 'asdf:load-op :ele-clsql)
(asdf:operate 'asdf:load-op :ele-sqlite3)
(in-package "ELEPHANT-TESTS")
(setq *test-path-primary* '(:sqlite3 "testdb"))
(do-all-tests-spec *test-path-primary*)
@end lisp
The file RUNTESTS.lisp, although possibly not exactly what you want,
contains useful example code.
You can of course migrate between the three currently supported repository
strategies in any combination: BDB, Postgresql, and SQLite3.
In all probability, other relational datbases would be very easy to
support but have not yet been tested. The basic pattern of
the ``path'' specifiers is (cons clsqal-database-type-symbol (normal-clsql-connection-specifier)).
@node Repository Migration
@comment node-name, next, previous, up
@section Repository Migration
This version of Elephant supports migration betwen store controllers,
whether of the same implementation strategy or not.
The tests @code{migrate1} - @code{migrate5} are demonstrations of this techinque.
The functions for performing these migrations are:
@code{migraten-pobj}
The name of this function is meant to imply that it is
destructive of the object in question, mutating it to
[23 lines skipped]
More information about the Elephant-cvs
mailing list