[bknr-devel] SBCL and OpenMCL ports done properly

[Kamen TOMOV]

On Wed, Feb 08 2006, Hans Hübner wrote:

> 2006/2/7, Kamen TOMOV <ktomov at web.de>:
>> On Tue, Feb 07 2006, Hans Hübner wrote:
>>
>> > ..
>> > I will propably work on database maintenance and operations so
>> > that BKNR based systems can be run for a long time without much
>> > nursery.
>>
>> What kind of nursery do you mean?
>>
>> > The snapshotting mechanism currently eats disk space without
>> > bounds,
>>
>> It seems to me it eats as much as necessary - no more no less. Or
>> am I missing something?
>
> If you regularily snapshot from a cron job, all snapshots will be
> kept on disk and need to be manually moved to offline media.
> Depending on the transaction rate, it may be neccessary to snapshot
> often, which will use a lot of disk space.  It would be good if the
> Datastore would provide an automatic mechanism that purges older
> snapshots.  This could be done on a simple (only keep N snapshots)
> or a more complex basis (keep the last three snapshots, one for each
> week, one for each month or some such).
>
> Also, a restore tool is needed that helps selecting a snapshot to
> restore.  I have been bitten many times while selecting an older
> snapshot to restore and selecting a too-early one.  Instead of
> manually having to select a certain snapshot, one should be able to
> specify a date and time (or manually created markpoint) to restore
> to and have the environment select the right snapshot file and
> rollforward location.
>
> It would also be good if a snapshot could be automatically created
> as soon as a certain restore time limit has been reached.  This
> would require adding up the execution time of restored and newly
> executed transactions and scheduling a snapshot only if it is
> required.  Together with the automatic selection of a snapshot to
> restore, this would make the database system much easier to use, and
> it would also help with the disk growth problem.
>

I see. These would probably be useful tools. 

I was thinking about the general purpose of the snapshotting
mechanism. It seems to me that it exists solely because the hardware
can not meet the demands of the applications only in terms of
store-objects restore times. For not so demanding applications I would
rather do without snapshotting. The prevalence model allows tracking
the history of the changes of the data.

As a framework user I would rather not rely on snapshotting to allow
changes of my store-object based objects. This is something that I
would fix in the framework.

In my opinion the main functionality must not depend on the
snapshotting mechanism, only the other way around.

>> > and the blob garbage collection is not automated.
>>
>> What exactly do you mean? If we delete blob wouldn't the memory be
>> cleaned?
>
> Yes.  Blob files cannot be deleted when the blob object is deleted
> because they may be needed again, should the user decide to restore
> an earlier snapshot.  An integrated snapsht archival function would
> be able to move deleted blobs offline together with the
> corresponding snapshot file and transaction log.  At the moment,
> there is the (bknr.datastore:delete-orphaned-blob-files) function
> which can be called to get rid of stray cats.

I meant the memory of the blob object, not the file. I suppose this
memory is garbage collected, right?

As to the disk memory I was surprised to find out (some time ago) that
the files are not deleted on deleting the blob object. Now I
understand your reasons. The integrated snapshot archival function
that you suggest sounds reasonable. Perhaps once we have it, we could
consider deleting the associated file on blob deletion.

Regards,

-- 
Kamen