[flexi-streams-devel] New version 0.5.3 (Was: in-memory streams have no way to set external-format?)

[Edi Weitz]

Hi!

On Wed, 08 Mar 2006 00:43:12 +0100, Andreas Fuchs <asf at boinkor.net> wrote:

> I would like to teach cl-irc about the goodness of external-formats,
> and because irc doesn't let users specify which external format to
> use, I need something as flexible as flexi-streams (-:
>
> So, once I've read an irc command (as a "line" of latin-1
> characters), I'd like to convert it to text that looks the least
> insane, which is why my code tries several external-formats in a row
> and returns as soon as it found one that doesn't throw an error:
>
> (defun try-decode-line (line external-formats)
>   (loop for external-format in external-formats
>         for decoded = nil
>         for error = nil
>         do (multiple-value-setq (decoded error)
>                (ignore-errors
>                  (flexi-streams:with-input-from-sequence (in line)
>                    (setf (flexi-streams:flexi-stream-external-format in)
>                          external-format)
>                    (read-line in))))
>         do (format t "~&tried ~s: ~S~% error: ~A~%" external-format decoded
>                    error)
>         if decoded
>           do (return decoded)))

If LINE is a string you want this:

  (defun try-decode-line (line external-formats)
    (loop for external-format in external-formats
          for decoded = nil
          for error = nil
          do (multiple-value-setq (decoded error)
                 (ignore-errors
                   (with-input-from-string (in line)
                     (let ((flexi (flexi-streams:make-flexi-stream in :external-format external-format)))
                       (read-line flexi)))))
          do (format t "~&tried ~s: ~S~% error: ~A~%" external-format decoded
                     error)
          if decoded
            do (return decoded)))

But actually I think you want LINE to be a sequence of octets, so this
is what you want:

  (defun try-decode-line (line external-formats)
    (loop for external-format in external-formats
          for decoded = nil
          for error = nil
          do (multiple-value-setq (decoded error)
                 (ignore-errors
                   (flexi-streams:with-input-from-sequence (in line)
                     (let ((flexi (flexi-streams:make-flexi-stream in :external-format external-format)))
                       (read-line flexi)))))
          do (format t "~&tried ~s: ~S~% error: ~A~%" external-format decoded
                     error)
          if decoded
            do (return decoded)))

Result:

  CL-USER 11 > (try-decode-line '(228 246 252)  '((:utf-8 :eol-style :lf)
                                                  (:latin-1 :eol-style :lf)))
  tried (:UTF-8 :EOL-STYLE :LF): NIL
   error: Unexpected value #xF6 in UTF-8 sequence.
  tried (:LATIN-1 :EOL-STYLE :LF): "äöü"
   error: T
  "äöü"

But you'll need version 0.5.3 to see that because there was a typo in
the code which generated the error messages.

> Hrmpf! Am I abusing flexi-streams too much or is that a bug? How
> should one read externally-formatted data from an in-memory stream,
> anyway?

You forgot that you have to create a flexi stream first - in-memory
streams happen to be provided by the same library but they're
something different.  Use MAKE-FLEXI-STREAM to turn them into
flexi streams.

> And are string-backed in-memory streams even allowed?

No.  CL already has WITH-INPUT-FROM-STRING... :)

> Thanks for your time and for developing flexi-streams. In return, I
> hope to be able to buy you a beverage of your choice in Hamburg (-:

Nice.  Looking forward to seeing you there!

Cheers,
Edi.