[flexi-streams-devel] Issue with flexi-streams
rpgoldman at real-time.com
rpgoldman at real-time.com
Tue Jun 13 22:23:07 UTC 2006
I have been having some trouble using the publicly available
flexi-streams package (http://www.cliki.net/flexi-streams). The
flexi-streams package is described as follows:
FLEXI-STREAMS is a library which implements "virtual" bivalent streams
that can be layered atop real binary/bivalent streams. It can be used
to read and write character data in various single- or multi-octet
encodings which can be changed on the fly. It also supplies in-memory
binary streams which are similar to string streams.
Here's the problem: the flexi-stream library defines a flexi-stream
class which ISA stream and has an EXTERNAL-FORMAT slot (and an
:external-format initarg):
(defclass flexi-stream (trivial-gray-stream-mixin)
((stream :initarg :stream
:reader flexi-stream-stream
:documentation "The actual stream that's used for
input and/or output. It must be capable of reading/writing
octets with READ-SEQUENCE and/or WRITE-SEQUENCE.")
(external-format :initform (make-external-format :iso-8859-1)
:initarg :external-format
:accessor flexi-stream-external-format
:documentation "The encoding currently used
by this stream. Can be changed on the fly.")
(element-type :initform #+:lispworks 'lw:simple-char #-:lispworks 'character
:initarg :element-type
:accessor flexi-stream-element-type
:documentation "The element type of this stream.
Must be a subtype of CHARACTER."))
(:documentation "A FLEXI-STREAM object is a stream that's
`layered' atop an existing binary/bivalent stream in order to
allow for multi-octet external formats. FLEXI-STREAM itself is a
mixin and should not be instantiated."))
Flexi-streams allow this initarg to be a list of symbols.
Unfortunately, calling MAKE-INSTANCE on a flexi-stream blows up on ACL
because ACL has a :BEFORE method for (setf external-format) on STREAM,
which rejects the flexi-stream lists that are supplied as initargs. I
am not entirely sure why this is being triggered.
Here is an email that specifies what is going wrong, with a simple
test case that you can cut out and run under ACL, and a response from
the flexi-stream builder.
>>>>> "RPG" == rpgoldman <rpgoldman at real-time.com> writes:
>>>>> "EW" == Edi Weitz <edi at agharta.de> writes:
RPG> [...snip...]
EW> On Thu, 1 Jun 2006 14:35:26 -0500, "Robert P. Goldman"
EW> <rpgoldman at sift.info> wrote:
>>> I just pulled a copy of flexi-streams, using asdf-install, and
>>> had a couple of minor problems:
>>>
RPG> [...snip...]
>>> 2. Making a flexible stream with an external format blows up
>>> for me on ACL. E.g., cl-irc tries to make a flexi-io-stream
>>> with external format (:utf-8 :eof-type :crlf)
>>>
>>> This value is stuffed into the flexi-stream slot
>>> EXTERNAL-FORMAT. Unfortunately, IIUC, this also triggers an
>>> Allegro-specific method as follows:
>>>
>>> (METHOD (SETF STREAM-EXTERNAL-FORMAT) :BEFORE (T STREAM))
>>>
>>> This method checks to make sure that the value you are setting
>>> as a stream-external-format is an excl:external-format object.
>>> In this case, it clearly isn't, so the :before method hurls an
>>> error.
>>>
>>> I believe that this error is triggered because a
>>> flexi-io-stream ISA fundamental-binary-input-stream which ISA
>>> stream.
>>>
>>> I confess I'm not really sure what to do about this
>>> problem....
EW> Hmm, at first glance I'd say that this is either an AllegroCL
EW> bug or it is not related to FLEXI-STREAMS at all. The slot
EW> you're talking about above is called
EW> FLEXI-STREAMS::EXTERNAL-FORMAT, its name is not exported from
EW> the package. I can't see why setting the value of this slot
EW> should trigger an Allegro-specific method.
EW> (Note that on AllegroCL EXCL:EXTERNAL-FORMAT is accessible in
EW> various packages, including CL-USER. Maybe you're seeing a
EW> package conflict?)
RPG> The problem is not the FLEXI-STREAMS::EXTERNAL-FORMAT symbol,
RPG> but the use of :external-format as an initarg, IIUC. See the
RPG> (relatively) simple test case below for more details.
EW> It'd be nice if you could provide a simple test case which
EW> provokes this error message, so we can see what's really going
EW> on.
RPG> OK. Understand that I don't really know how flexi-streams
RPG> are supposed to work --- I just stumbled into this because I
RPG> use BEIRC, which uses CL-IRC, which (newly) uses
RPG> FLEXI-STREAMS. So I'm pretty far from understanding things,
RPG> and some of the things that I give may not be bugs; they may
RPG> be me using the package wrong.
RPG> My first shot at causing an error is the following:
RPG> (defpackage :flexi-test (:use :common-lisp :flex))
RPG> (in-package :flexi-test) (setf foo (open "/tmp/foo"
RPG> :direction :io)) (setf bar (make-flexi-stream foo)) Error:
RPG> :DEFAULT is not known to be a name for an external format.
RPG> Restart actions (select using :continue):
RPG> 0: Return to Top Level (an "abort" restart). 1: Abort
RPG> entirely from this (lisp) process.
RPG> [1] FLEXI-TEST(18): :bt Evaluation stack:
RPG> FLEX::NORMALIZE-EXTERNAL-FORMAT-NAME <-
RPG> FLEX::MAKE-EXTERNAL-FORMAT% <- MAKE-EXTERNAL-FORMAT <-
RPG> FLEX::MAYBE-CONVERT-EXTERNAL-FORMAT <- (METHOD
RPG> INITIALIZE-INSTANCE :AFTER ...) <- (:INTERNAL
RPG> (:EFFECTIVE-METHOD 1 T ...) 0) <- (METHOD MAKE-INSTANCE
RPG> (CLASS)) <- (METHOD MAKE-INSTANCE (SYMBOL)) <-
RPG> MAKE-FLEXI-STREAM <- [... EXCL::%EVAL ] <- LET* <-
RPG> [... EXCL::%EVAL ] <- EVAL <-
RPG> TPL:TOP-LEVEL-READ-EVAL-PRINT-LOOP <-
RPG> TPL:START-INTERACTIVE-TOP-LEVEL
RPG> [I'm honestly not sure where the :default value comes from,
RPG> and it may be that I am simply committing an error here and
RPG> that the :external-format keyword is mandatory, rather than
RPG> optional, as I have treated it.]
RPG> So I tried to do what cl-irc does:
RPG> (defun external-format-fixup (format)
RPG> (let ((new-format (copy-list format)))
RPG> (setf (getf (cdr new-format) :eol-style) :crlf)
RPG> new-format))
RPG> (defun mock-connect (&key (server "irc.freenode.net")
RPG> (port 6667)
RPG> (connection-type
RPG> 'connection)
RPG> (logging-stream t))
RPG> (let* ((stream (socket-connect server port))
RPG> (connection (make-connection
RPG> :connection-type connection-type
RPG> :network-stream stream
RPG> :client-stream
RPG> logging-stream
RPG> :server-name server)))
RPG> (values connection stream)))
RPG> (defun make-connection (&key (connection-type 'connection)
RPG> (user nil) (password nil)
RPG> (server-name "") (server-port
RPG> nil) (network-stream nil)
RPG> (outgoing-external-format
RPG> *default-outgoing-external-format*)
RPG> (client-stream t) (hooks nil))
RPG> (let ((output-stream (flexi-streams:make-flexi-stream
RPG> network-stream :element-type
RPG> 'character :external-format
RPG> (external-format-fixup
RPG> outgoing-external-format))))
RPG> output-stream))
RPG> ;; overlooked this dependency....
RPG> (asdf:oos 'asdf:load-op :trivial-sockets) (defun
RPG> socket-connect (server port)
RPG> "Create a socket connected to `server':`port' and return
RPG> stream for it." (trivial-sockets:open-stream server port
RPG> :element-type '(unsigned-byte 8)))
RPG> (setf *default-outgoing-external-format* '(:utf8)) That gets
RPG> me the error: FLEXI-TEST(62): (mock-connect) Error:
RPG> `:EOL-STYLE' does not name an external-format.
RPG> [condition type: NO-EXTERNAL-FORMAT-ERROR]
RPG> Restart actions (select using :continue):
RPG> 0: Return to Top Level (an "abort" restart). 1: Abort
RPG> entirely from this (lisp) process.
RPG> [1] FLEXI-TEST(63): :bt Evaluation stack:
RPG> (METHOD (SETF STREAM-EXTERNAL-FORMAT) :BEFORE ...) <-
RPG> (:INTERNAL (:EFFECTIVE-METHOD 2 NIL ...) 0) <- (:INTERNAL
RPG> (:EFFECTIVE-METHOD 2 T ...) 0) <- (METHOD
RPG> INITIALIZE-INSTANCE (STANDARD-OBJECT)) <- (:INTERNAL
RPG> (:EFFECTIVE-METHOD 1 T ...) 0) <- (METHOD MAKE-INSTANCE
RPG> (CLASS)) <- (METHOD MAKE-INSTANCE (SYMBOL)) <-
RPG> MAKE-FLEXI-STREAM <- LET <- MAKE-CONNECTION <- LET* <-
RPG> MOCK-CONNECT <- EVAL <- TPL:TOP-LEVEL-READ-EVAL-PRINT-LOOP
RPG> <- TPL:START-INTERACTIVE-TOP-LEVEL
RPG> The error happens when making the flexi-io-stream class. It
RPG> triggers this ACL :before method that checks the value of the
RPG> stream-external-format, and which seems to be seeing the
RPG> '(:utf8 :eol-style :crlf) value that cl-irc is stuffing into
RPG> the flexi-stream constructor.
RPG> ACL has a method that evidently checks the assignment to
RPG> stream-external-format for every stream to make sure that it
RPG> is a legitimate excl-external-format, IIUC. A flexi-stream
RPG> ISA stream, so somehow this is triggered. Your email seems
RPG> to suggest that you did NOT expect the flexi-stream-value to
RPG> become the stream-external-format, right? But this is the
RPG> make-instance call:
RPG> ((METHOD MAKE-INSTANCE (CLASS)) #<STANDARD-CLASS
RPG> FLEXI-IO-STREAM> :STREAM
RPG> #<MULTIVALENT stream socket connected from
RPG> #192.168.1.18/41332 to
RPG> kornbluth.freenode.net/6667 @ #x71dc7412>
RPG> :ELEMENT-TYPE CHARACTER :EXTERNAL-FORMAT (:UTF8 :EOL-STYLE
RPG> :CRLF))
RPG> So even though flexi-streams::external-format is not
RPG> exported, flexi-streams is using the :external-format keyword
RPG> in its make-instance call, and that is triggering the ACL
RPG> code. Would it be enough to fix things to change the
RPG> :initarg to ":flexi-stream-external-format"? That seems
RPG> cumbersome, though. I don't know another way to keep ACL
RPG> from grabbing up the :external-format keyword arg, though...
RPG> But again, I have not thought deeply about this.
>>>>> "EW" == Edi Weitz <edi at agharta.de> writes:
EW> I think you're basically right and what happens is this:
CL-USER> (defclass foo (excl::fundamental-character-stream)
EW> ((bar :initarg :external-format
EW> :initform (error "No external
EW> format."))))
EW> #<STANDARD-CLASS FOO>
CL-USER> (make-instance 'foo)
EW> #<FOO @ #x2122c9aa>
CL-USER> (slot-value * 'bar)
EW> :DEFAULT
EW> So, it looks like there's some :AROUND method within AllegroCL
EW> which modifies the initargs before the FOO instance is
EW> initialized. I couldn't find a place in their documentation
EW> where they explicitely talk about the :EXTERNAL-FORMAT
EW> initarg, and I'm hesitant to rename it in FLEXI-STREAMS
EW> because it is the right name, isn't it?
EW> At least, before I do that, it'd be nice if you could ask
EW> Franz about their take.
EW> Cheers, Edi.
--
Robert P. Goldman
Senior Scientist
Smart Information Flow Technologies (d/b/a SIFT, LLC)
211 N. First St., Suite 300
Minneapolis, MN 55401
Voice: (612) 384-3454
Email: rpgoldman at SIFT.info
More information about the Flexi-streams-devel
mailing list