[Ecls-list] ongoing testing

[Juan Jose Garcia Ripoll]

Hi Brian

I think I did not express my ideas properly in the former message. The
change I propose is indeed a radical one and it already contains much of
what you wrote.

However as a design principle (KIS keep it simple!) I think we should
not need to split between devices and streams. Doing so is an
unnecessary overhead. Let us take the example of a file which is read
octet-wise. Do we need to create a stream that just redirects to the
device? Should each device to be implemented handle its own buffering?
Can't we reuse these routines, i.e. have buffering both for files and
sockets?

I suggest some compromise between low-level and high-level access to
data. There should be a splitting as follows:

	device streams
		file-stream
		socket-stream
		string-stream
	wrapping stream
		buffering-stream
		character-stream
		byte-stream
	lisp streams
		broadcast-stream
		echo-stream
		...

All streams will provide the routines (implemented as pointers)

struct stream_dispatch {
        cl_fixnum *device_read(cl_object stream, void *buffer, cl_index
len, int block):
        cl_fixnum *device_write(cl_object stream, void *buffer, cl_index
len, int block);
	void *device_unread(cl_object stream, void *buffer, cl_index len);
        void *device_open(cl_object stream);
        void *device_close(cl_object stream);
        void *device_flush_output(cl_object stream);
        cl_object *device_position(cl_object stream);
        void *device_set_position(cl_object stream, cl_object position);
        int *device_elttype(cl_object stream);
};

The description of most routines is evident and you have already
sketched it. device_read() and device_write() will output data in a
blocking or a non-blocking fashion, a parameter that might be ignored by
the stream. With the semantics of the simple-streams, device_read() with
a length of 0 can be used to query the existence of data.

The important point is device_read() and device_write() need not work
octet-wise. The caller should query first the desired output format and
act accordingly. The allowed element types for a stream are the same
element types for an array. Thus we have binary streams, with aet_b8
(byte 8), aet_u8 (unsigned-byte 8)..., character streams with aet_bc or
aet_character, and higher level streams with aet_object ('T) which check
their input and act accordingly.

Take write-char, for instance. It will take the stream and query the
element type. If it is aet_bc, it will check that the character is a
base-character and then pass the data to device_write(). If it is aet_ch
or aet_t, there is no need to check and it will also pass the character
to the stream. Otherwise the character code will be obtained and it will
be written to the stream using write-byte.

Take a call to write-sequence with a vector argument. If the element
type of the sequence matches that of the device, there will be no need
to perform any further checks: a single call to device_write will
suffice.

Device streams such as files and sockets will use the octet type aet_b8.
A string-stream will instead be character oriented and use either aet_bc
(base-char) or aet_ch (character).

A buffering stream will inherit the element type of the stream it
buffers and create a lisp vector of the desired type to buffer the data.

A character stream will wrap around another stream and perform
conversions between formats. It will probably have a mini-buffer for
incomplete characters, or whatever is needed for parsing to/from UTF-8,
UTF-16, or other locales.

A byte-stream will be a wrapper on top of a device stream or a buffer
stream which allows for reading or writing data with byte sizes that are
not octets. The array element type will be either one of the supported
byte sizes or aet_object. In the latter case, the stream will support
writing bytes with strange sizes (unsigned-byte 7), (unsigned-byte 21),
a feature which is already present in ECL.

The advantage of this approach, in my point of view, is that you need
not introduce another abstraction (i.e. devices), because streams
already provide all functions you need. For instance, other lisp streams
such as broadcast streams can efficiently forward a call to
device_read/device_write to the streams they reference. And wrapping
streams (such as the buffering one) can be reused. For instance, a
character stream can be created on top of a string stream.

Regards,

Juanjo

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 191 bytes
Desc: This is a digitally signed message part
URL: <https://mailman.common-lisp.net/pipermail/ecl-devel/attachments/20060530/817755ab/attachment.sig>