[streams-standard-discuss] Welcome to streams-standard-discuss

[David Lichteblau]

Quoting Robert Strandh (strandh at labri.fr):
>  > CLOS
>  > dispatch in single charcter output is likely to be really rather slow
> Yes.  Could this be solved by simply adding stream-read-sequence and
> stream-write-sequence methods to the protocol?

Well, no.  I see the following kinds of streams:

  - unbuffered I/O

    This kind of stream is available in Unix as read() and write()
    (although there is no stream "object" in this layer other than the file
    descriptor).  But it is an important concept and usually the lower
    layer other concepts are building on.

    Common Lisp does not know unbuffered I/O.

    Simple Streams call this the "device level" and provide support for
    it by documenting the device level functions, allowing the user to
    implement them using CLOS and also to call them directly, bypassing
    upper layers (not very elegant but at least possible).

    Gray streams would allow for a stream to be unbuffered, but then
    there are all the single-character/single-byte functions in the
    API which do not make at lot of sense at this layer, because callers
    would only want to use the read-/write-sequence functions on such
    a stream for speed reasons.

  - buffered I/O (of characters or bytes)

    This is provided by C's stdio and by Common Lisp streams.  (But not
    in an extensible way using portable features in either language.)

    It is an abstraction needed for file I/O, sockets, etc.  A very
    common application that should be fast. 

    Gray streams mirror the Common Lisp functions for this layer, but
    have a speed and design problem: If buffering is used anyway, the
    right abstraction is to document the buffer instead of having many
    streams reimplement it poorly.

    Or at least that must be the idea behind simple streams.

    Answering your question: It does not help at all to provide fast
    read-/write-sequence at this layer if read-/write-byte are not fast,
    since this layer is all about the buffering.  If I wanted to prebuffer
    my output into long sequences and write those instead, I would not need
    this layer in the first place (it would buffer the same data a second time)
    and use unbuffered I/O instead.

  - more general stream-like objects which don't involve such a buffer

    stdio and Common Lisp do not know this.

    An example would be CLIM, which takes the Common Lisp API for
    streams with similar purposes but a completely different
    implementation.  A buffer of bytes would not help for such streams
    at all, yet re-using the existing stream API appears better than
    reinventing it.

    Gray streams try to provide for this using CLOS.

    CMUCL's ansi-streams solve it without CLOS by simulating their own
    object system.

Problems with the simple stream approach include streams which are
buffered in a sense, but in a different way than the original protocol
designer hoped they would be.  For example, string streams are special
case which users would not be able to implement if the simple streams
implementation would have provided for that.

Problems with CMUCL's ANSI-STREAM include that they are structures and
that other implementations might not willingly adopt some ad-hoc object
system in its stream layer just for compatibility with, say, SBCL.
OTOH simple streams, while simpler due to CLOS, seem to struggle with
that CLOS usage: As far as I understand it, there are funny attempts at
optimizing CLOS access away which are not exactly beautiful.

Extensibility for buffered I/O must be provided at the device level as
with simple streams.  Extensibility for arbitrary streams at or slightly
below the ANSI CL level.  So ANSI-STREAMs and simple streams are not
alternative proposals for the same problem, but for different problems.

My claim is that both abstractions are needed.  Buffered I/O because it
is the most common case.  More general streams because Lisp people are
used to them.  It would be nice if buffered I/O was implemented using
the general abstractions.  Right now I do not see how to implement, say,
simple streams as ANSI-STREAMs largely because the latter are
structures.

> Probably not if we
> need to keep track of column position as well, which is another reason
> for eliminating the GFs that talk about columns. 

The column position is optional already in that it may be NIL if the
stream does not know the concept.  So other than not being general
enough, it does not do any harm either.  Its importance would depend
upon who the major users are.  FRESH-LINE probably.  Does the
pretty-printer need it?


d.